Senior Data Engineer - Databricks

CI&TData engineering

Listing location: Brazil Workplace type: Remote Structured job location: BrazilFull-TimeSenior

Salary not disclosed

Apply NowOpens the employer's application page

Job Details

Have proven experience building and managing large-scale data pipelines in Databricks.
Use PySpark, Delta Lake, and SQL.
Have strong Python and SQL programming skills for data processing and transformation.
Have a deep understanding of ETL/ELT frameworks, data warehousing, and distributed data processing.
Have hands-on experience with DataOps practices, including Git version control, CI/CD pipelines, automated testing, and infrastructure-as-code.
Be familiar with AWS, Azure, or GCP and related data services.
Be able to troubleshoot performance, scalability, and reliability issues.
Have proficiency in Git.
Have advanced English proficiency.
Experience with data contracts, schema evolution, and compatibility across services is a nice-to-have.
Experience with data quality frameworks such as Great Expectations, Soda, dbt tests, or custom-built solutions is a nice-to-have.
Experience integrating with Power BI is a nice-to-have.

Develop and maintain data pipelines in Databricks.
Integrate and transform data from APIs, relational databases, files, and other sources.
Collaborate with business and analytics teams to understand data requirements.
Ensure the quality, reliability, security, and governance of ingested data.
Follow DataOps practices, including code versioning, data tests, and CI/CD.
Document data engineering processes and best practices.
Provide technical guidance and mentor peers.
Collaborate with data scientists, analysts, and platform engineers.

View Full Description & ApplyYou'll be redirected to the employer's site