Senior Data Engineer (Databricks & PySpark)
New
O
OptiveumSustainability data
100% Remote (Warsaw, PL)ContractSenior
SalaryUp to 38 EUR / hour
Apply NowOpens the employer's application page
Job Details
- Languages
- Fluent English (both written and spoken).
- Experience
- 5–7 years of experience designing, developing, and operating data platforms and pipelines.
- Required Skills
- PythonGitCI/CDData modelingDatabricksPySpark
Requirements
- Have 5–7 years of experience designing, developing, and operating data platforms and pipelines.
- Bring extensive hands-on production experience with Databricks, including Azure Databricks and Databricks Workflows.
- Have expert-level proficiency in PySpark, Python, and Spark SQL.
- Have expertise in data modelling and data pipeline design.
- Have experience implementing CI/CD pipelines for data engineering workloads.
- Have experience with Git-based development and version control.
- Understand data lineage, metadata management, governance, auditing, and validation frameworks.
- Be fluent in written and spoken English.
- Sustainability reporting, ESG data, and regulatory reporting experience is a plus.
- Knowledge of procurement, supplier, finance, and ERP data domains is a plus.
- Experience developing Databricks applications, dashboards, or user-facing data tools is a plus.
Responsibilities
- Design and implement scalable data products using Databricks, Delta Lake, and PySpark.
- Build and maintain data pipelines processing datasets from ERP, procurement, sustainability, and external systems.
- Optimize workloads for performance, scalability, and cost efficiency, and build reusable engineering patterns.
- Implement automated data quality controls throughout the data lifecycle and resolve data issues before they affect reporting.
- Implement CI/CD pipelines, automated deployments, testing frameworks, and Infrastructure as Code.
- Support platform security controls and access management.
- Partner with sustainability experts, business analysts, and reporting teams on requirements gathering, solution design, and production releases.
View Full Description & ApplyYou'll be redirected to the employer's site
