Data Engineer
New
Remote, Latin AmericaFull-TimeMiddle
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Required Skills
- PythonSQLETLGitData modelingAzure DevOpsPySpark
Requirements
- Bachelor’s degree in Computer Science, Information Technology, or equivalent experience.
- Proven experience in ETL data engineering with expertise in Python (PySpark).
- Proficiency in Azure Synapse Analytics (Notebooks, Pipelines, Linked Services, Azure Key Vault).
- Ability to write complex SQL queries and optimize performance using SparkSQL and MS SQL.
- Experience with version control systems, specifically Git and Azure DevOps.
- Strong problem-solving and analytical skills.
- Excellent communication skills for cross-functional collaboration.
- Familiarity with data integration best practices.
- Knowledge of Data Fabric concepts (data lakes, delta lakes, etc.) is a plus.
- Experience with Power BI, Tableau, or Agile methodologies is a plus.
Responsibilities
- Develop and maintain ETL data engineering processes using Python (PySpark) within Azure Synapse Analytics Notebooks and Pipelines.
- Design and build effective data storage structures in a Massively Parallel Processing (MPP) SQL Pool using star schemas, facts, and dimensions.
- Extract data from various sources including REST APIs, SQL database tables, and CSV files.
- Optimize ETL data notebooks/pipelines for scalability and performance.
- Implement data quality checks and validation processes to maintain accuracy and consistency.
- Collaborate with data architects to create data models and schemas.
- Monitor ETL jobs, diagnose issues, and ensure pipeline reliability.
- Maintain comprehensive documentation of processes and data flows.
- Ensure data security and compliance with data governance and privacy standards.
View Full Description & ApplyYou'll be redirected to the employer's site
