Engenheiro de Dados AWS/Databricks
BrazilFull-Time
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Languages
- English proficiency for communication in international environments (fluency preferred)
- Required Skills
- SQLApache AirflowKafkadbtDatabricksPySpark
Requirements
- Solid experience as a Data Engineer in production environments
- Strong hands-on experience with Databricks (Delta Lake)
- Advanced knowledge of PySpark and SQL
- Experience building ETL/ELT pipelines in cloud-based environments
- Knowledge of Medallion architecture and modern data lakehouse concepts
- Experience with batch and streaming ingestion (Kafka, Auto Loader, Structured Streaming)
- Understanding of data modeling techniques (Dimensional Modeling, Data Vault)
- Experience with dbt and/or Delta Live Tables (DLT)
- Familiarity with data quality frameworks such as Great Expectations or DLT Expectations
- Experience with pipeline orchestration tools like Databricks Workflows or Apache Airflow
- Knowledge of APIs and multi-source data integration
- Experience with data observability, monitoring, and troubleshooting pipelines
- English proficiency for communication in international environments (fluency preferred)
Responsibilities
- Build and maintain scalable data pipelines for ingestion, transformation, and data curation
- Develop ELT/ETL workflows using PySpark and SQL within Medallion architecture (Bronze, Silver, Gold)
- Implement data quality rules, monitoring mechanisms, and remediation standards to ensure reliable datasets
- Perform data profiling and source system analysis to support governance and data cataloging initiatives
- Manage dataset cataloging and enforce data policies aligned with LGPD using Unity Catalog
- Integrate APIs, external data sources, and orchestration tools into end-to-end data workflows
- Support both batch and streaming ingestion processes using tools such as Auto Loader, Kafka, and Structured Streaming
- Collaborate on data modeling practices including Dimensional Modeling and Data Vault approaches
- Ensure observability and monitoring of pipelines to guarantee performance and reliability
View Full Description & ApplyYou'll be redirected to the employer's site
