Engenheiro de Dados Pleno
New
BrazilFull-TimeMiddle
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Required Skills
- PythonSQLGitAzureCI/CDdbtDatabricksPySpark
Requirements
- Strong experience with Python, SQL, PySpark, and especially dbt for data transformation workflows
- Hands-on experience with Databricks, Delta Lake, and Unity Catalog
- Solid knowledge of Azure cloud infrastructure and data engineering services
- Experience with Git and CI/CD processes, preferably using Azure DevOps
- Strong understanding of Lakehouse architecture and Medallion design pattern
- Ability to build and optimize scalable data pipelines in high-volume environments
- Strong analytical thinking and problem-solving skills
- Good communication skills and ability to work collaboratively in technical teams
Responsibilities
- Design, build, and maintain scalable data pipelines using Python, SQL, and PySpark, ensuring efficient processing of large-scale datasets
- Develop and manage data transformations using dbt, applying business rules and ensuring quality, documentation, and testability
- Structure and maintain data within a Lakehouse architecture using the Medallion model (Bronze, Silver, Gold layers)
- Implement and support data governance practices, including access control, lineage, and catalog management via Unity Catalog
- Ensure reliability and performance of data storage and processing using Delta Lake
- Contribute to DevOps practices, including Git versioning and CI/CD pipelines (Azure DevOps), ensuring secure and traceable deployments
- Collaborate with cross-functional teams to support data initiatives and promote engineering best practices
- Translate business requirements into robust, scalable, and maintainable data solutions
View Full Description & ApplyYou'll be redirected to the employer's site
