Data & AI Engineer
New
R
RevStarData and AI
Remote (AZ, DC, FL, GA, IA, IL, ME, NY, NJ, SC, TX, VA)Full-TimeMiddle
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- 3+ years in cloud-native data engineering with 2+ years of dedicated, hands-on Databricks, Apache Spark, Delta Lake, and MLflow experience.
- Required Skills
- PythonSQLMLFlowSparkCI/CDTerraformDatabricks
Requirements
- Bring 3+ years of experience in cloud-native data engineering.
- Have 2+ years of dedicated, hands-on experience with Databricks, Apache Spark, Delta Lake, and MLflow.
- Demonstrate high proficiency in Python and SQL.
- Have strong knowledge of Spark frameworks.
- Have experience with CI/CD and infrastructure-as-code using Terraform or Databricks CLI.
- Bring a track record in fast-paced Agile/DevOps consulting environments.
- Have experience delivering cloud-agnostic architectures.
- Demonstrate strong client communication skills.
- Databricks Certified Data Engineer (Associate or Professional) or Databricks ML Professional certification is strongly preferred.
Responsibilities
- Design, build, and optimize scalable ETL/ELT pipelines using Apache Spark and Delta Lake across multi-cloud environments.
- Implement Lakehouse architectures to process structured and unstructured data at enterprise scale.
- Automate data ingestion, storage, and feature engineering workflows for analytics and real-time AI workloads.
- Tune Spark jobs for low latency, high throughput, and cloud cost-efficiency.
- Partner with ML engineers and data scientists to operationalize AI models in Databricks using MLflow.
- Implement CI/CD pipelines and infrastructure-as-code with Terraform or Databricks CLI.
- Enforce data security, access controls, and compliance standards within Unity Catalog and cloud ecosystems.
- Collaborate with client technology leaders, product managers, and engineering teams to translate data requirements into business outcomes.
- Write technical documentation and lead solution handoffs to client engineering teams.
View Full Description & ApplyYou'll be redirected to the employer's site
