Senior Data Engineer (Databricks)
New
A
AddeptoBig Data
Workplace type: remote; Country code: PLFull-TimeSenior
Salary21,000 - 28,560 PLN per month
Apply NowOpens the employer's application page
Job Details
- Experience
- At least 5 years of commercial experience implementing, developing, or maintaining Big Data systems.
- Required Skills
- PythonSQLApache AirflowMicrosoft Power BIApache KafkaAzureSparkdbtDatabricks
Requirements
- At least 5 years of commercial experience implementing, developing, or maintaining Big Data systems.
- Strong Python programming skills, including clean code and object-oriented design.
- Strong SQL skills, including performance tuning and query optimization.
- Experience with data warehousing solutions.
- Experience designing and implementing data governance and data management processes.
- Deep expertise in Big Data technologies, including Apache Airflow, Dagster, and Databricks.
- Experience implementing and deploying solutions in cloud environments; Azure is preferred.
- Knowledge of building and deploying Power BI reports and dashboards.
- Strong understanding of dimensional data and data modeling techniques.
- Consulting experience and ability to guide clients on architecture, technology selection, and best practices.
- Ability to work independently and take ownership of project deliverables.
- Familiarity with Spark and Azure Event Hub or Kafka.
- Master’s or Ph.D. in Computer Science, Big Data, Mathematics, Physics, or a related field.
Responsibilities
- Design and optimize scalable data processing pipelines for streaming and batch workloads using Big Data technologies.
- Architect and implement end-to-end data platforms with high availability, performance, and reliability.
- Lead the development of CI/CD and MLOps processes for deployments, monitoring, and model lifecycle management.
- Develop and maintain applications that aggregate, process, and analyze data from diverse sources.
- Collaborate with Data Science teams on machine learning projects, including feature engineering and predictive model deployment.
- Design and manage data transformations using Databricks, DBT, and Apache Airflow.
- Translate business requirements into technical solutions while maintaining performance and data quality.
- Apply data security, compliance, and governance best practices across data pipelines.
View Full Description & ApplyYou'll be redirected to the employer's site
