Lead Data Engineer - Databricks Platform
New
V
Vaillant GroupData engineering
Workplace type: remote; Locations: Katowice, al. Walentego Roździeńskiego 1B, Katowice, Country code: PLFull-TimeLead
Salary19,000 - 24,000 PLN per month
Apply NowOpens the employer's application page
Job Details
- Languages
- You communicate confidently in English.
- Experience
- You bring several years of experience in data engineering
- Required Skills
- PythonSQLSparkCI/CDDatabricks
Requirements
- Hold a degree in computer science, data engineering, information systems, or a comparable field.
- Have several years of experience in data engineering.
- Have taken ownership of technical leadership or senior engineering responsibilities.
- Have deep knowledge of Databricks and Apache Spark.
- Bring knowledge of Spark performance tuning, workload management, Delta Lake, and modern data processing architectures.
- Have strong programming skills in Python and SQL.
- Apply software engineering principles including testing, CI/CD, code quality, observability, and automation to data platform development.
- Communicate confidently in English.
- Experience with Spark Declarative Pipelines and Declarative Automation Bundles is a nice-to-have.
Responsibilities
- Define and drive the technical evolution of the enterprise data platform.
- Lead migration of existing workloads toward Spark Declarative Pipelines and modern engineering patterns.
- Design and implement core platform capabilities, reusable frameworks, and best practices.
- Review critical code contributions and establish engineering standards across the data ecosystem.
- Optimize Spark performance, cluster utilization, workload efficiency, and Databricks costs.
- Advise engineering teams and stakeholders on complex data challenges and robust technical designs.
- Support sprint planning, backlog refinement, effort estimation, and deployment practices.
- Evaluate technologies, promote automation and DevOps practices, and reduce technical debt.
View Full Description & ApplyYou'll be redirected to the employer's site