Engenheiro de Dados Senior
J
JobgetherData engineering
Based in BrazilFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Required Skills
- AWSPythonSQLGitSparkDatabricksPySpark
Requirements
- Have proven experience as a Senior Data Engineer, with expertise in data solution development, architecture, and optimization.
- Have practical experience with Cloudera and Databricks processing and transforming large data volumes.
- Know AWS and its core services for data engineering and cloud-based data processing.
- Have strong SQL skills and hands-on experience with Python and PySpark.
- Have experience with Apache Spark and distributed data processing technologies.
- Understand ETL/ELT processes and relevant tools.
- Have experience designing and working with Data Lake and Lakehouse architectures.
- Know data modeling and integration of information from multiple sources.
- Have experience with Git and code version control practices.
- Be able to monitor data pipelines, troubleshoot issues, and optimize processing performance.
- Additional advantages include AWS data services, Delta Lake, Hadoop ecosystem technologies, Kafka, and Airflow or similar orchestration tools.
- Additional advantages include CI/CD, infrastructure as code, automation, data governance, quality, security, observability, and data platform migration or modernization experience.
Responsibilities
- Design, develop, and maintain scalable, efficient, and reliable data pipelines.
- Implement batch and streaming data ingestion, transformation, and integration workflows.
- Build and optimize data engineering solutions using Cloudera, Databricks, and AWS.
- Develop data processing and transformation routines with Python, SQL, PySpark, and Apache Spark.
- Create, maintain, and improve ETL/ELT processes and Data Lake and Lakehouse architectures.
- Integrate data from multiple sources and maintain data quality, consistency, integrity, and reliability.
- Monitor pipelines, processing jobs, and queries; troubleshoot issues and improve performance.
- Apply data security, governance, cataloging, lineage, and observability practices.
- Contribute to data architecture evolution, platform modernization, and process improvement.
- Collaborate with architecture, analytics, data science, and business teams on technical and business requirements.
View Full Description & ApplyYou'll be redirected to the employer's site
