Solution Architect, Delivery (Databricks)
New
J
JobgetherData platforms
Fully remote working arrangement within the United States.Full-TimeSenior
Salary140,000 - 210,000 USD per year
Apply NowOpens the employer's application page
Job Details
- Experience
- At least 8 years of experience in data architecture, data engineering, or enterprise software development, including a minimum of 3 years designing and architecting enterprise solutions based on Apache Spark and the Databricks Data Intelligence Platform.
- Required Skills
- AWSPythonSQLGCPAzureSparkTerraformPySpark
Requirements
- At least 8 years of experience in data architecture, data engineering, or enterprise software development.
- At least 3 years designing and architecting enterprise solutions based on Apache Spark and the Databricks Data Intelligence Platform.
- Databricks Certified Data Engineer Professional or Databricks Certified Machine Learning Professional certification.
- Strong architectural proficiency in AWS, Azure, or GCP, including VPC/VNet networking, private endpoints, cloud identity and access management, Entra ID integration, and cloud object storage.
- Advanced knowledge of dimensional modeling, Data Vault, and Medallion Architecture.
- Hands-on experience implementing Unity Catalog across multiple workspaces in production environments.
- Advanced proficiency in Python, PySpark, and SQL.
- Practical experience with Infrastructure as Code tools such as Terraform and orchestration platforms such as Databricks Workflows or Airflow.
- Experience leading technical delivery teams and translating business and functional requirements into architectural designs, technical specifications, and user stories.
- Strong understanding of CI/CD pipelines, code quality standards, infrastructure automation, and modern software engineering practices.
- Experience designing solutions aligned with enterprise regulatory and security frameworks such as HIPAA, HITRUST, SOC 2, or FedRAMP.
- An active Public Trust or Secret clearance, or eligibility to obtain one, is considered an advantage.
Responsibilities
- Lead architecture and design of scalable Lakehouse solutions using the Databricks Data Intelligence Platform, Medallion Architecture, Delta Lake, and Databricks SQL.
- Design governance and security frameworks using Unity Catalog, access controls, data masking, ABAC, and Delta Sharing.
- Define and lead migrations from legacy data warehouses and Hadoop environments to Databricks architectures.
- Establish batch and real-time processing patterns using Structured Streaming, Delta Live Tables, Auto Loader, and Apache Spark optimization techniques.
- Guide engineering teams, establish reference architectures, conduct architectural reviews, review code, and implement standardized CI/CD practices.
- Optimize Spark and Delta Lake performance through query execution, partitioning, Z-Ordering, caching, and Photon configurations.
- Establish FinOps practices and compute governance standards to optimize Databricks consumption.
- Collaborate with machine learning and AI specialists to integrate MLflow, Feature Store, Model Serving, Vector Search, and RAG pipelines.
- Advise senior client stakeholders and lead technical workshops, architecture design sessions, and proof-of-concept evaluations.
- Develop reusable architectural blueprints, migration accelerators, reference architectures, and standardized configurations.
View Full Description & ApplyYou'll be redirected to the employer's site
