Senior GenAI Data Engineer (Databricks)
New
MexicoFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Required Skills
- SQLCI/CDDatabricksGitHubPrompt EngineeringLangChain
Requirements
- Strong hands-on experience with Databricks Lakehouse architecture.
- Advanced SQL skills and experience in data modeling at scale.
- Experience integrating LLM APIs (e.g., OpenAI, Anthropic) into engineering workflows.
- Practical experience building AI-driven features such as enrichment, anomaly detection, and classification.
- Experience with frameworks like LangChain, LlamaIndex, or similar orchestration tools.
- Knowledge of RAG systems and vector databases for semantic retrieval.
- Understanding of tool/function calling for secure LLM-to-database/API interactions.
- Exposure to prompt engineering and guardrailing techniques for reliable AI outputs.
- Familiarity with CI/CD pipelines, GitHub workflows, and YAML-based configurations.
- Strong problem-solving mindset with ability to operate autonomously in ambiguous environments.
- Excellent communication skills and ability to collaborate across technical and non-technical teams.
Responsibilities
- Architect and deploy AI agents within the Databricks ecosystem for data enrichment, anomaly detection, and automated reasoning.
- Build and maintain scalable data pipelines ensuring high data quality, reliability, and traceability.
- Develop advanced SQL models and semantic layers to support analytics and business intelligence use cases.
- Integrate LLM APIs such as OpenAI and Anthropic into production data workflows.
- Develop “chat with your data” solutions enabling natural language querying over enterprise datasets.
- Implement RAG pipelines using vector search and modern AI orchestration frameworks.
- Collaborate with cross-functional teams (Product, Engineering, Analytics, Business) to deliver scalable data solutions.
- Optimize performance, automation, and CI/CD processes for data and AI workflows.
- Ensure strong governance, documentation, and maintainability across all data assets.
View Full Description & ApplyYou'll be redirected to the employer's site