Lead Data Engineer
New
M
MonksData Engineering
Location: Colombia, Collaborate effectively across distributed teams and time zonesContractLead
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- 5+ years
- Required Skills
- PythonSQLETLGitSparkCI/CDDatabricks
Requirements
- 5+ years of production experience with dbt Core (dbt Certified Developer preferred).
- Hands-on experience leading data engineering teams, mentoring engineers, reviewing code, and owning technical delivery.
- Advanced knowledge of dbt, including Jinja macros, sources.yml, schema.yml tests, reusable model patterns, and incremental models.
- Hands-on experience with Databricks, Delta Lake, Unity Catalog, and Databricks SQL Warehouses.
- Advanced SQL skills, including CTEs, window functions, complex transformations, and Jinja templating.
- Experience migrating custom or legacy Python-based ETL pipelines into modular, source-agnostic dbt models.
- Experience validating output parity between modernized models and legacy production pipelines.
- Proficiency with Git-based pull request workflows and CI/CD gates, such as GitHub Actions.
- Strong technical leadership, stakeholder communication, problem-solving, and documentation skills.
Responsibilities
- Lead the end-to-end migration of legacy Python/Spark ETL configurations into modular, maintainable dbt Core models.
- Provide hands-on technical leadership, mentorship, code review, and delivery guidance to data engineers.
- Partner with internal client engineers to define migration priorities, technical standards, and implementation plans.
- Reverse-engineer existing Python ETL logic and translate it into scalable dbt SQL models, staging layers, and reusable Jinja macros.
- Design foundational shared macros, dimension tables, and medium-to-high-complexity domain models.
- Implement dbt sources, incremental model patterns, and schema.yml tests.
- Validate output parity between newly developed dbt models and legacy production pipelines before go-live.
- Establish and uphold engineering standards for model design, documentation, testing, and maintainability.
- Manage delivery through a Git-based, PR-reviewed workflow with automated dbt compile and dbt test CI/CD gates.
- Support scheduled dbt orchestration and production deployment within the Databricks ecosystem.
View Full Description & ApplyYou'll be redirected to the employer's site
