Senior/Lead Data Engineer
New
T
TruelogicAI Software
BrazilFull-TimeLead
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Required Skills
- PythonSQLGitSparkdbtDatabricksPySpark
Requirements
- Expertise in SQL and dimensional modeling methodologies (medallion architecture, SCDs, grain management).
- Proven ability to design idempotent pipelines utilizing incremental, checkpoint, and replaceWhere strategies.
- Extensive experience with production-grade Python engineering including type hints, pytest, and ruff.
- Deep understanding of Delta Lake features (MERGE, OPTIMIZE, Z-ORDER, time travel).
- Hands-on expertise with dbt (models, tests, exposures).
- Experience authoring and deploying jobs using Databricks Asset Bundles and Unity Catalog.
- Strong adherence to Git workflows and documentation practices.
- Experience with Service Principals and secret management tools like Azure Key Vault.
- Strong written technical communication skills.
Responsibilities
- Design and build robust, idempotent data pipelines from scratch utilizing a modern data stack.
- Design star and snowflake schemas, writing precise, grain-aware SQL to construct scalable data marts.
- Write production-grade, unit-tested Python code at the module level.
- Build and test dbt models across staging, intermediate, and mart layers.
- Author and deploy jobs using Databricks Asset Bundles (DAB).
- Implement rigorous data quality checks to prevent silent drops of nulls or duplicates.
- Maintain data governance through dbt tests and documentation-at-merge-time.
- Own data pipelines end-to-end, mentor mid-level engineers, and define technical direction.
View Full Description & ApplyYou'll be redirected to the employer's site
