Engenheiro de Dados SR
New
BrazilFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Languages
- Advanced English
- Required Skills
- DockerPythonApache AirflowMicrosoft AzureSparkCI/CDdbt
Requirements
- Advanced English proficiency for meetings, documentation, and stakeholder interaction.
- Solid experience working with cloud data environments, especially Microsoft Azure.
- Strong programming skills in Python.
- Proven experience with distributed data processing using Apache Spark.
- Experience building and managing workflows using Apache Airflow.
- Hands-on experience with CI/CD pipelines and modern DevOps practices.
- Knowledge of YAML/YML for configuration and automation workflows.
- Experience using Docker for containerized environments.
- Experience with dbt for data modeling and transformations.
- Familiarity with version control, testing, documentation, and software engineering best practices.
Responsibilities
- Develop, maintain, and optimize scalable end-to-end data pipelines supporting business and analytics needs.
- Build distributed data processing solutions using Apache Spark for high-volume workloads.
- Orchestrate workflows, scheduling, and automation using Apache Airflow.
- Design and implement data engineering solutions in Python following best engineering practices.
- Work within cloud environments, primarily on Microsoft Azure, ensuring scalable and secure data solutions.
- Implement CI/CD pipelines to support reliable and automated deployment processes.
- Use YAML/YML configurations to enable automation, infrastructure definitions, and pipeline standardization.
- Containerize applications and services using Docker to improve portability and scalability.
- Build and maintain transformation models using dbt.
- Collaborate with cross-functional teams to ensure data quality, reliability, performance, and alignment with business goals.
View Full Description & ApplyYou'll be redirected to the employer's site
