Senior Data Software Engineer
New
J
JobgetherData engineering
Based in BrazilFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Languages
- English proficiency at B2 level or higher
- Experience
- At least 3 years of relevant professional experience in data engineering or a related software engineering field.
- Required Skills
- GitPySpark
Requirements
- At least 3 years of relevant professional experience in data engineering or a related software engineering field.
- Hands-on experience with PySpark and Spark SQL for large-scale data workflows.
- Experience building and managing Lakeflow pipelines and Jobs for data orchestration.
- Proficiency with Auto Loader and JDBC/CDC ingestion methods.
- Experience working with source systems such as DB2 and SQL Server.
- Experience with Delta Lake and Unity Catalog for data storage, management, and governance.
- Background in data quality assurance and source reconciliation.
- Experience writing unit tests using pytest.
- Working knowledge of Git and Asset Bundles for version control and deployment management.
- English proficiency at B2 level or higher.
- Experience with DataStage migration projects is a nice-to-have.
- Background extracting data from Dynamics 365/Dataverse is a nice-to-have.
- Databricks certification is a nice-to-have.
Responsibilities
- Design, develop, and maintain scalable data pipelines for reliable data movement across multiple systems.
- Build and manage data ingestion processes from a variety of source systems.
- Write clean, efficient, maintainable, and well-tested code using established software engineering practices.
- Collaborate with cross-functional teams to translate data requirements into technical solutions.
- Monitor data pipeline performance and proactively identify and resolve issues.
- Participate in code reviews, uphold engineering quality standards, and share technical knowledge.
- Troubleshoot data quality issues and ensure dataset accuracy, consistency, and reliability.
- Contribute to improvements in data architecture, engineering practices, and development standards.
- Document technical processes, solutions, and workflows.
- Participate in agile development activities, including sprint planning and team ceremonies.
View Full Description & ApplyYou'll be redirected to the employer's site
