Lead/Sr AI Engineer
E
Enable Data IncorporatedAI Engineering
Bengaluru, Karnataka, IndiaFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Required Skills
- PythonSQLMLFlowAzure DevOpsMLOpsGenerative AIPySpark
Requirements
- Expertise in Azure Databricks including jobs, workflows, clusters, and Unity Catalog.
- Advanced proficiency in Python, specifically PySpark.
- Strong SQL skills including complex joins, window functions, and analytical queries.
- Deep understanding of Azure Cloud services: ADLS Gen2, ADF, Key Vault, and IAM.
- Experience with Azure DevOps and CI/CD pipeline orchestration.
- Solid understanding of ML lifecycle and MLOps best practices.
- Experience with model deployment using MLflow or similar frameworks.
- Knowledge of LLM fine-tuning (LoRA/QLoRA) and Generative AI integration.
- Proficiency in building RAG pipelines, vector search (FAISS/Azure Cognitive Search), and embeddings.
Responsibilities
- Design and build end-to-end data and AI pipelines using Azure Databricks.
- Develop robust ETL/ELT workflows using Python (PySpark) and SQL.
- Implement CI/CD pipelines for Databricks deployments including jobs and workflows.
- Integrate Databricks with Azure services like Data Lake, Key Vault, and Azure OpenAI.
- Develop and deploy ML and Generative AI models including LLMs and RAG pipelines.
- Collaborate with data scientists and stakeholders to move models into production.
- Optimize production jobs for performance, cost, and reliability.
- Implement logging, monitoring, and AI safety guardrails.
View Full Description & ApplyYou'll be redirected to the employer's site
