Lead Data Engineer (Spark)
New
A
AddeptoData engineering
Workplace type: remote; Country code: PLFull-TimeLead
Salary21,000 - 31,920 PLN per month
Apply NowOpens the employer's application page
Job Details
- Languages
- Fluent English (C1 level) is a must.
- Experience
- 5+ years of proven commercial experience in implementing, developing, or maintaining Big Data systems.
- Required Skills
- AWSDockerPythonSQLApache AirflowJavaKubernetesSparkScala
Requirements
- Bring 5+ years of proven commercial experience implementing, developing, or maintaining Big Data systems.
- Have strong programming skills in Python or Java/Scala, including clean code and OOP design.
- Have experience designing and implementing data governance and data management processes.
- Be familiar with Big Data technologies such as Spark, Cloudera, Airflow, NiFi, Docker, Kubernetes, Iceberg, Trino, or Hudi.
- Have expertise implementing and deploying solutions in cloud environments, preferably AWS.
- Have a strong understanding of dimensional data and data modeling techniques.
- Have communication skills and consulting experience involving direct client interaction.
- Be able to work independently and take ownership of project deliverables.
- Hold a Master’s or Ph.D. in Computer Science, Data Science, Mathematics, Physics, or a related field.
- Be fluent in English at C1 level.
Responsibilities
- Design and develop scalable data management architectures, infrastructure, and platform solutions for streaming and batch processing using Big Data technologies.
- Design and implement data management and data governance processes and best practices.
- Contribute to the development of CI/CD and MLOps processes.
- Develop applications to aggregate, process, and analyze data from diverse sources.
- Collaborate with the Data Science team on data analysis and Machine Learning projects, including text/image analysis and predictive model building.
- Develop and organize data transformations using DBT and Apache Airflow.
- Translate business requirements into technical solutions and ensure optimal performance and quality.
View Full Description & ApplyYou'll be redirected to the employer's site
