Data & AI Engineer (Databricks Specialist)
New
R
RevStarData and AI engineering
Remote (US-Based / Eastern or Central Time Zone Preferred), Eastern or Central Time Zone preferredFull-TimeMiddle
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- 3+ years in cloud-native data engineering with 2+ years of dedicated, hands-on Databricks, Apache Spark, Delta Lake, and MLflow experience.
- Required Skills
- PythonSQLMLFlowSparkCI/CDTerraformDatabricks
Requirements
- Have 3+ years of experience in cloud-native data engineering.
- Have 2+ years of dedicated, hands-on experience with Databricks, Apache Spark, Delta Lake, and MLflow.
- Demonstrate high proficiency in Python, SQL, and Spark frameworks.
- Have experience with CI/CD and Infrastructure-as-Code, including Terraform or Databricks CLI.
- Bring a proven track record in fast-paced Agile/DevOps consulting environments.
- Have experience delivering cloud-agnostic architectures.
- Demonstrate client communication skills.
- An active Databricks Certified Data Engineer (Associate or Professional) or Databricks ML Professional certification is strongly preferred.
Responsibilities
- Design, build, and optimize scalable ETL/ELT pipelines using Apache Spark and Delta Lake across multi-cloud environments.
- Implement Lakehouse architectures that process structured and unstructured data at enterprise scale.
- Automate data ingestion, storage, and feature engineering workflows for analytics and real-time AI workloads.
- Tune Spark jobs for low latency, high throughput, and cloud cost-efficiency.
- Partner with ML engineers and data scientists to operationalize AI models in Databricks using MLflow.
- Implement CI/CD pipelines and Infrastructure-as-Code deployments using Terraform and Databricks CLI.
- Enforce data security, access controls, and compliance standards using Unity Catalog and cloud ecosystems.
- Collaborate with client tech leaders, product managers, and engineering teams to translate AI data requirements into business outcomes.
- Write technical documentation and lead solution handoffs to client engineering teams.
View Full Description & ApplyYou'll be redirected to the employer's site
