Senior Data Engineer
New
A
AddeptoAI Consulting
Warszawa, Krucza 50, Kraków, Aleksandra Lubomirskiego 20, Wrocław, Świdnicka 40, Katowice, -, Gdańsk, -, Warszawa, Country code: PLFull-TimeSenior
Salary21,000 - 28,560 PLN per month
Apply NowOpens the employer's application page
Job Details
- Experience
- At least 5 years of commercial experience
- Required Skills
- PythonSQLApache AirflowMicrosoft Power BIApache KafkaAzureSparkdbtDatabricks
Requirements
- At least 5 years of commercial experience implementing, developing, or maintaining Big Data systems.
- Strong Python programming skills, including OOP design and clean code practices.
- Strong SQL skills with experience in performance tuning and data warehousing.
- Deep expertise in Big Data technologies including Apache Airflow, Dagster, and Databricks.
- Experience implementing and deploying solutions in cloud environments, preferably Azure.
- Experience designing and implementing data governance and management processes.
- Knowledge of dimensional data modeling techniques.
- Consulting experience guiding clients through architecture and technology selection.
- Familiarity with Spark, Azure Event Hub, or Kafka.
- Knowledge of building and deploying Power BI reports.
- Ability to work independently and take ownership of deliverables.
- Master’s or Ph.D. in Computer Science, Big Data, Mathematics, Physics, or related field.
Responsibilities
- Design and optimize scalable data processing pipelines for streaming and batch workloads using Databricks, Apache Airflow, and Dagster.
- Architect and implement end-to-end data platforms ensuring performance, reliability, and high availability.
- Lead the development of CI/CD and MLOps processes to automate deployments, monitoring, and model lifecycle management.
- Develop and maintain applications for aggregating and processing data from diverse sources.
- Collaborate with Data Science teams on Machine Learning projects, including feature engineering and model deployment.
- Design and manage complex data transformations using Databricks, DBT, and Apache Airflow.
- Translate business requirements into efficient technical solutions while maintaining data security and governance.
View Full Description & ApplyYou'll be redirected to the employer's site
