Sr. Data Engineer (Databricks)

View full description

Requirements:

• Bachelor's or Master's degree in computer science.
• 5+ years of industry experience in Data Engineering.
• Expertise in Spark and SQL.
• 2+ years of experience with the Databricks environment.
• Familiarity with real-time ML systems within Databricks is beneficial.

Responsibilities:

• Majority of the role will be software engineering – creating tables, views, Spark jobs, orchestrating within the Databricks environment.
• Develop, deploy, and manage scalable data pipelines on Databricks with robust integration for machine learning models.
• Implement real-time data processing and streaming using Databricks capabilities.
• Maintain high code quality with observability, metadata standards, and best practices.
• Work with data science, UI, and reporting teams to translate data requirements into models.
• Stay updated on emerging tech trends within the Databricks ecosystem.
• Share knowledge through events and empower internal teams with architectural insights.