Data & AI Engineer (Databricks Specialist)
New
R
RevStarData and AI engineering
Remote (US-Based / Eastern or Central Time Zone Preferred), Eastern or Central Time Zone preferredFull-TimeMiddle
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- 3+ years in cloud-native data engineering with 2+ years of dedicated, hands-on Databricks, Apache Spark, Delta Lake, and MLflow experience.
- Required Skills
- PythonSQLMLFlowSparkCI/CDTerraformDatabricks
Requirements
- Have 3+ years of experience in cloud-native data engineering.
- Have 2+ years of dedicated, hands-on experience with Databricks, Apache Spark, Delta Lake, and MLflow.
- Demonstrate high proficiency in Python and SQL.
- Have proficiency with Spark frameworks and CI/CD.
- Have proficiency in infrastructure as code, including Terraform or Databricks CLI.
- Bring experience operating in fast-paced Agile/DevOps consulting environments.
- Have experience delivering cloud-agnostic architectures.
- Bring a proven track record of client communication.
- Databricks Certified Data Engineer (Associate or Professional) or Databricks ML Professional certification is strongly preferred.
Responsibilities
- Design, build, and optimize scalable ETL/ELT pipelines using Apache Spark and Delta Lake across multi-cloud environments.
- Implement Lakehouse architectures that process structured and unstructured data.
- Automate data ingestion, storage, and feature engineering workflows for analytics and real-time AI workloads.
- Tune Spark jobs for low latency, high throughput, and cloud cost-efficiency.
- Partner with ML engineers and data scientists to operationalize AI models in Databricks using MLflow.
- Implement CI/CD pipelines and infrastructure as code using Terraform and Databricks CLI.
- Enforce data security, access controls, and compliance standards within Unity Catalog and cloud ecosystems.
- Collaborate with client technical leaders, product managers, and engineering teams to translate data requirements into business outcomes.
- Write technical documentation and lead solution handoffs to client engineering teams.
View Full Description & ApplyYou'll be redirected to the employer's site
