Senior Azure Databricks Engineer

Strong hands-on experience with Azure Databricks (DBX): clusters, jobs, notebooks, asset bundles Deep knowledge of Apache Spark: performance tuning, data partitioning, optimization strategies Experience designing and implementing MLOps frameworks in a production environment Proven experience with CI/CD automation (Jenkins and/or Azure DevOps pipelines) Familiarity with Azure cloud components: ADLS, Key Vault, AAD Solid Python skills for Databricks notebooks Experience with Git and branching strategies for data/ML projects Strong ownership mindset: ability to work independently, drive delivery, and communicate progress clearly Fluent English (spoken and written)

Optimize and productionize Databricks notebooks and jobs Tune Spark performance (cluster configuration, caching, data partitioning, shuffle optimization) Design and implement MLOps frameworks for model training, deployment, and monitoring Set up and maintain CI/CD pipelines for Databricks using Jenkins or Azure DevOps Work with Databricks Asset Bundles to manage code, configs, and deployments Configure and fine-tune clusters, jobs, and workflows for cost and performance efficiency Integrate with Azure services: ADLS, Key Vault, AAD, and others Own technical delivery, progress tracking, and quality