Databricks Engineer Job at Conflux Systems, Phoenix, AZ

  • Conflux Systems
  • Phoenix, AZ

Job Description

Databricks Engineer

We are seeking a highly skilled Databricks Engineer to lead the design, implementation, and optimization of scalable cloud-based data platforms. This role requires deep expertise in Lakehouse architecture, Databricks, Apache Spark, and Delta Lake, along with proven experience delivering end-to-end enterprise data engineering solutions.

Key Responsibilities

  • Design and deliver end-to-end data engineering pipelines, including batch and real-time streaming solutions.
  • Lead implementation of:
  • Cloud-based data lakehouse platforms integrating diverse data sources.
  • Real-time data processing pipelines for operational and analytical use cases.
  • Develop scalable ETL/ELT pipelines using PySpark, Scala, and SQL.
  • Implement advanced data modeling solutions including 3NF, dimensional modeling, and enterprise data warehousing strategies.
  • Design and build incremental data loading frameworks and metadata-driven ingestion pipelines.
  • Establish data quality frameworks and governance standards.
  • Implement and manage Unity Catalog, including fine-grained security and access controls.
  • Leverage Databricks components such as:
  • Delta Live Tables
  • Autoloader
  • Structured Streaming
  • Databricks Workflows
  • Integration with orchestration tools (e.g., Apache Airflow)
  • Drive CI/CD automation, deployment strategies, and DevOps best practices.
  • Optimize performance of pipelines, Spark jobs, and compute resources.
  • Provide architectural guidance and technical leadership across cross-functional teams.
  • Engage with stakeholders and clients to translate business requirements into scalable technical solutions.

Deep expertise in:

  • Databricks and cloud-native storage/compute platforms
  • Apache Spark (batch & streaming)
  • Delta Lake & Lakehouse architecture
  • Distributed data processing systems
  • Strong hands-on programming skills in Python, PySpark, Scala, and SQL.

Job Tags

Similar Jobs

YMCA of Honolulu, Kaimuki-Waialae

Parking Lot Attendant Job at YMCA of Honolulu, Kaimuki-Waialae

 ...aquatics, youth development, and senior programming, serving a diverse and active community. Role Description As a Parking Lot Attendant , you will play a key role in ensuring a safe, organized, and welcoming arrival experience for members and visitors throughout... 

Pena & Kahn, PLLC

Spanish Interpreter/Translator Job at Pena & Kahn, PLLC

 ...litigation law firm located in the Bronx, NY is looking to hire a Spanish interpreter. The interpreter coordinates and provides...  ...facilitate good communication. The interpreter will also provide sight translation of documents, including but not limited to, consent forms and... 

Ashley

Furniture Delivery Driver Job at Ashley

 ...that believes every member of our Distribution Center is an important part of providing an excellent pick up, delivery and service experience. As a Furniture Delivery Driver, you will be... Offering world class customer service Assembling, inspecting, loading... 

Intermountain Health

Charge Operating Room Registered Nurse Job at Intermountain Health

Intermountain Health seeks a Registered Nurse for the Operating Room to deliver safe, evidence-based perioperative care in a fast-paced surgical environment. The OR RN will serve as circulating and/or scrub nurse, performing thorough assessments, maintaining sterile technique... 

Alpine Springs

Part Time Program Technician - Weekends Day or Afternoon Shift Job at Alpine Springs

 ...Job Description Job Description Now Hiring Part-Time Program Technician for Weekends at our location in Linesville, PA - Alpine Springs Addiction Treatment...  ...Monitor client attendance and behavior at in-house and off grounds activities including: nature activities,...