Tech Lead Data - Databricks
C
CI&TData engineering
Workplace type: Remote; Listing location: BrazilFull-TimeLead
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Languages
- Advanced English
- Required Skills
- PythonSQLAzureDatabricksPySpark
Requirements
- Bring strong Data Engineering project experience and experience as a Tech Lead or Senior technical and operational team lead.
- Have strong hands-on Databricks experience with PySpark and Spark SQL.
- Have experience with batch pipelines and working knowledge of streaming when needed.
- Demonstrate mastery of Delta Lake and practical application of Bronze/Silver/Gold Medallion architecture.
- Have Azure data experience, including Azure Databricks, Azure Data Lake Storage, and integration with security and identity services such as Azure Active Directory.
- Have strong SQL and Python skills, plus fundamentals in dimensional and consumption modeling and schema or contract best practices.
- Have experience with DataOps, including Git, CI/CD for data pipelines, automated testing, and documentation.
- Be able to review design and code, resolve performance and scale issues, and guide teams.
- Have advanced English for stakeholder work and technical documentation.
Responsibilities
- Provide technical leadership to Data Engineering squads, supporting design, reviews, and decision-making.
- Co-define the technical vision, coding standards, modeling conventions, and Lakehouse best practices with the Data Architect.
- Design and oversee Databricks pipelines across Bronze, Silver, and Gold layers.
- Implement and reinforce version control, CI/CD, testing, and documentation practices.
- Optimize jobs and pipelines for partitioning, Spark performance, cost, and reliability.
- Ensure security, access control, and data policies in Azure and Databricks align with governance.
- Collaborate with business and analytics stakeholders to translate requirements into scalable solutions and delivery plans.
- Mentor engineers and create templates, playbooks, and other technical references.
View Full Description & ApplyYou'll be redirected to the employer's site