Data Engineer (Databricks)
New
BrazilFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Required Skills
- PythonSQLAzurePandasData modelingDatabricksPySpark
Requirements
- Solid experience with SQL, including advanced regex usage.
- Strong proficiency in Databricks, PySpark, and Python for data engineering and analytics.
- Knowledge of data manipulation techniques, text processing, and complex data parsing.
- Experience with SQL data modeling.
- Foundational understanding of distributed computing concepts.
- Hands-on experience with Data Lake architectures and large-scale data environments.
- Ability to operate in high-responsibility production environments.
- On-call availability after business hours and weekends when required.
- Strong problem-solving mindset and attention to detail.
- Experience with cloud data platforms, particularly Azure.
Responsibilities
- Collaborate with data engineering teams to translate business and technical requirements into scalable data solutions.
- Develop, maintain, and optimize data pipelines and platforms using Azure, Databricks (PySpark), Python, and SQL.
- Implement data processing and transformation logic using Python, Pandas, and ElementTree for XML integration.
- Ensure system availability, stability, and performance by monitoring, troubleshooting, and evolving production data solutions.
- Support data security, governance, and compliance through automation and proactive management.
- Implement data quality controls, testing frameworks, and monitoring strategies.
- Maintain comprehensive technical documentation for architectures and processes.
- Provide production support and change management follow-up.
View Full Description & ApplyYou'll be redirected to the employer's site
