Senior/Lead Data Engineer
New
T
TruelogicAI-Native Software
Location: LatAmFull-TimeLead
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Required Skills
- PythonSQLGitSparkdbtDatabricksPySpark
Requirements
- Expertise in SQL and dimensional modeling (medallion architecture, SCDs).
- Experience designing idempotent pipelines utilizing incremental, checkpoint, and replaceWhere strategies.
- Extensive experience with production-grade Python engineering (type hints, pytest, ruff).
- Capability to diagnose and resolve Spark/PySpark jobs.
- Deep understanding of Delta Lake features (MERGE, OPTIMIZE, Z-ORDER).
- Hands-on expertise with dbt.
- Experience authoring and deploying jobs using Databricks Asset Bundles (DAB) and Unity Catalog.
- Strong adherence to disciplined Git workflows.
- Experience provisioning Service Principals and utilizing secret management tools (Azure Key Vault).
- Strong written technical communication skills.
Responsibilities
- Design and build robust, idempotent data pipelines from scratch utilizing a modern data stack.
- Design star and snowflake schemas, writing precise, grain-aware SQL to construct scalable data marts.
- Write production-grade, unit-tested Python code at the module level.
- Build and test dbt models across staging, intermediate, and mart layers.
- Author and deploy jobs using Databricks Asset Bundles (DAB).
- Implement rigorous data quality checks at source, intermediate, and destination layers.
- Maintain data governance through comprehensive dbt tests and documentation.
- Own data pipelines end-to-end, making technical design decisions and mentoring mid-level engineers.
- Define overarching technical direction including modeling standards and observability thresholds.
- Act as a technical leader and participate in hiring panels.
View Full Description & ApplyYou'll be redirected to the employer's site
