Engenheiro de Dados Sênior / Especialista - AWS / Snowflake / Iceberg
New
J
JobgetherData Engineering
BrazilFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Languages
- English
- Experience
- 5 years of proven experience working with AWS Cloud; 3 years of hands-on Snowflake experience; 2+ years with Apache Iceberg
- Required Skills
- AWSPythonSQLGitSnowflakeSparkCI/CDdbtPySpark
Requirements
- 5+ years of production experience with AWS Cloud.
- 3+ years of hands-on Snowflake experience including modeling and cost optimization.
- 2+ years of production experience with Apache Iceberg.
- Strong proficiency in Apache Spark and PySpark at scale.
- Solid knowledge of AWS S3, Glue, EMR, Athena, Lambda, and Step Functions.
- Advanced SQL skills for complex query optimization.
- Proficiency in Python for automation and engineering tasks.
- Experience with pipeline orchestration tools like Airflow, Step Functions, or dbt.
- Strong familiarity with Git, automated testing, and code review processes.
- Technical English proficiency for documentation reading.
- Mandatory: Ability to present a PowerPoint case demonstrating practical Snowflake experience.
Responsibilities
- Design and evolve data ingestion, transformation, and delivery pipelines across AWS, including Apache Iceberg tables on S3.
- Develop distributed data processing jobs using Apache Spark and PySpark, including incremental loads and historical reprocessing.
- Maintain interoperability between Snowflake and the Iceberg-based lakehouse, including metadata synchronization and external volumes.
- Optimize Iceberg table maintenance tasks such as compaction, snapshot expiration, and schema evolution.
- Monitor and optimize AWS and Snowflake performance and costs.
- Establish data engineering best practices for CI/CD, testing, documentation, and lineage.
- Serve as a technical reference through code reviews, architecture decisions, and mentorship.
View Full Description & ApplyYou'll be redirected to the employer's site
