Databricks Solution Architect
United StatesFull-TimeLead
Salary102,000 - 133,000 USD per year
Apply NowOpens the employer's application page
Job Details
- Experience
- 8+ years of data engineering experience, with 4+ years building production workloads on Databricks.
- Required Skills
- PythonSQLSparkCI/CDTerraformDatabricksPySpark
Requirements
- 8+ years of data engineering experience, with 4+ years building production workloads on Databricks.
- Deep expertise in Apache Spark (PySpark and Spark SQL).
- Strong hands-on experience with Delta Lake, Unity Catalog, Databricks Workflows, and Delta Live Tables.
- Production experience on at least one major cloud (AWS, Azure, or GCP).
- Proficiency in Python and SQL; comfort with Scala is a plus.
- Experience designing medallion (bronze/silver/gold) architectures.
- Strong CI/CD and DevOps practice: Git, Terraform, Databricks Asset Bundles or dbx.
- Track record of leading technical projects end-to-end and mentoring engineers.
Responsibilities
- Architect and lead the implementation of an enterprise lakehouse on Databricks (Delta Lake, Unity Catalog, Photon, Workflows) across one or more major clouds (AWS, Azure, or GCP).
- Design scalable batch and streaming data pipelines using PySpark, Spark SQL, Structured Streaming, and Delta Live Tables.
- Define and enforce platform standards for data modeling (medallion architecture), CI/CD, code quality, testing, observability, and cost optimization.
- Lead the governance strategy using Unity Catalog in partnership with security and compliance.
- Optimize Spark workloads for performance and cost.
- Partner with ML engineers and data scientists to operationalize models using MLflow.
- Own the cloud infrastructure footprint for the platform.
- Mentor a team of data engineers; lead architecture reviews, code reviews, and technical design sessions.
- Engage with stakeholders to translate business needs into a roadmap for the data platform.
View Full Description & ApplyYou'll be redirected to the employer's site
