Data Engineer - Platform & Pipelines
New
BrazilTemporaryMiddle
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Languages
- English
- Experience
- 3+ years
- Required Skills
- PythonSQLApache AirflowETLData engineeringSparkDatabricks
Requirements
- Bachelor's degree in Computer Science, Engineering, or related.
- 3+ years of experience in Data Engineering.
- Strong proficiency in Apache Airflow and Databricks.
- Experience implementing Medallion/Delta Lake architectures.
- Strong SQL and Python skills.
- Advanced English communication skills.
Responsibilities
- Develop and maintain robust Airflow DAGs to orchestrate complex data transformations.
- Use Spark and Polars to clean, enrich, and aggregate data according to business logic.
- Enforce the Medallion Architecture patterns.
- Optimize processing workflows and SQL queries to reduce costs and execution time.
- Manage code deployment to on-prem and cloud infrastructure.
- Implement comprehensive data validation checks and quality gates.
- Maintain the metadata and catalog entries.
View Full Description & ApplyYou'll be redirected to the employer's site
