Senior Data Engineer - Azure Databricks Platform
New
E
EnerMechEnergy services
This is a fully remote role in the UK, across time zones with colleagues in the UK, Americas, Middle East and Asia PacificFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- Typically 6+ years in data engineering, including 3+ years hands-on with Azure Databricks in production.
- Required Skills
- AzureTerraformAzure DevOps
Requirements
- Bachelor's degree in Computer Science, Engineering, Information Systems, or a related discipline, or equivalent practical experience.
- Typically 6+ years of data engineering experience.
- At least 3 years of hands-on Azure Databricks experience in production.
- Demonstrated ownership of medallion architecture design and data modelling.
- Production experience implementing data quality frameworks and data contracts.
- Experience implementing PII protection, such as masking, tokenisation, or access tiering, across environments.
- Platform engineering experience diagnosing infrastructure issues, not just consuming platform services.
- Track record of autonomous delivery, including direct stakeholder engagement.
- Databricks Certified Data Engineer Associate certification at minimum.
- Databricks Certified Data Engineer Professional certification is desirable.
- Microsoft AZ-400 certification, multi-environment Azure networking, CI/CD pipeline migration or setup, governance and identity patterns, and regulatory framework application are desirable.
Responsibilities
- Design, build, quality-check, and operate assigned Azure Databricks platform workstreams from scoping through production deployment.
- Maintain the bronze, silver, and gold medallion data model for assigned domains, including modelling, validation, documentation, and certification.
- Build and optimize Delta tables and materialized views for performance at scale.
- Design and implement data quality frameworks, quarantine and error handling, and validation against data contracts.
- Define job-specific compute configurations that balance workload performance and cost.
- Contribute to and troubleshoot Terraform-managed Azure infrastructure, including networking, storage, Key Vaults, Managed DevOps Pools, and Dev Centers.
- Own and maintain Azure DevOps build and release pipelines, variable groups and libraries, and repository security and policies.
- Support Azure environment configuration, including subscription access, RBAC, and Terraform state backend management across environments.
- Use AI-assisted development tooling and validate its outputs against authoritative sources before acting.
- Communicate with technical and non-technical stakeholders across the UK, Americas, Middle East, and Asia Pacific.
View Full Description & ApplyYou'll be redirected to the employer's site
