Senior/Lead Data Engineer
MexicoFull-TimeLead
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Required Skills
- PythonSQLAzureSparkCI/CDdbtDatabricksPySpark
Requirements
- Strong expertise in SQL and dimensional data modeling, including medallion architecture, SCD patterns, and dataset grain management.
- Extensive experience building production-grade data pipelines using Python, with strong testing practices (pytest), typing, and code quality tools.
- Deep hands-on experience with Spark/PySpark and performance troubleshooting using tools such as Spark UI.
- Strong experience with dbt, including model development, testing frameworks, and data documentation practices.
- Solid knowledge of Databricks, Delta Lake (MERGE, OPTIMIZE, Z-ORDER, time travel), and modern lakehouse architectures.
- Experience working with cloud data platforms and tools such as GitHub, CI/CD workflows, and secret management systems.
- Proven ability to operate in complex, distributed systems and make architectural trade-offs between cost, scalability, and performance.
- Strong communication skills with the ability to document technical decisions and translate engineering work into business impact.
- Leadership experience mentoring engineers, setting technical direction, and contributing to engineering best practices.
- Familiarity with Azure ecosystem tools (ADF, ADLS) and modern observability or AI-assisted engineering tools is a plus.
Responsibilities
- Design, build, and maintain scalable, idempotent end-to-end data pipelines using modern data stack principles to support analytics and AI workloads.
- Develop robust data models (star and snowflake schemas) and write high-quality, grain-aware SQL to build scalable and reliable data marts.
- Build production-grade Python systems with strong engineering discipline, including testing, type safety, and modular design.
- Develop and manage dbt models across layered architectures (staging, intermediate, marts), ensuring strong testing and documentation standards.
- Implement and enforce data quality frameworks, including validation checks, schema enforcement, and anomaly detection across pipelines.
- Develop and deploy Databricks-based workloads (including Asset Bundles) and operate within secure, governed cloud environments.
- Provide technical leadership by defining architecture standards, reviewing code, mentoring engineers, and guiding cross-team engineering decisions.
- Ensure reliability and observability of data systems, while proactively identifying and resolving performance or pipeline issues.
View Full Description & ApplyYou'll be redirected to the employer's site
