Data Engineer (Databricks)
New
L
LivefrontData Engineering
Remote (USA)Full-TimeMiddle
Salary120,000 - 145,000 USD per year
Apply NowOpens the employer's application page
Job Details
- Experience
- 3-5 years of data engineering experience with at least 2 years in production Databricks environments
- Required Skills
- AWSPythonSQLMLFlowAzuredbtDatabricksPySpark
Requirements
- 3-5 years of total data engineering experience.
- At least 2 years of experience in production Databricks environments.
- Proven experience designing Lakehouse architectures (medallion patterns, Delta Lake, partitioning, Z-ordering).
- Deep expertise in Databricks features: Lakeflow, Autoloader, Structured Streaming, and Unity Catalog.
- Strong proficiency in SQL and Python with clean, performant coding standards.
- Solid working knowledge of AWS or Azure cloud services (storage, networking, IAM, compute).
- Hands-on experience with CI/CD for data, unit testing, and data quality frameworks.
- Excellent communication skills for explaining complex concepts to technical and non-technical stakeholders.
- Strong problem-solving skills and comfort with navigating ambiguous requirements.
Responsibilities
- Design and build production data pipelines using Lakeflow Declarative Pipelines, Autoloader, and Structured Streaming.
- Architect and implement Lakehouse solutions including medallion architecture, Delta Lake, and Unity Catalog.
- Maintain Databricks transformation layers including DLT pipelines, PySpark notebooks, and dbt.
- Design and maintain AI foundations including Feature Store, MLflow, and Model Serving for production ML and agent workflows.
- Collaborate with product and backend engineers to design data models, APIs, and application data contracts.
- Consult with clients to understand data challenges, develop strategies, and implement sustainable solutions.
- Champion data governance through Unity Catalog including access control, lineage, and data quality policies.
View Full Description & ApplyYou'll be redirected to the employer's site
