Senior Big Data Engineer

New

TQLO SP. Z O.O.Big Data

Workplace type: remote; Locations: Warszawa, Skierniewicka, Warszawa, Country code: PL; 100% zdalnieContractSenior

Salary160 - 190 PLN per hour

Apply NowOpens the employer's application page

Job Details

Have senior-level experience designing and developing Big Data or Data Engineering solutions.
Demonstrate strong SQL and data modeling skills.
Have advanced Apache Spark knowledge, including DataFrame API, Spark SQL, and performance optimization.
Have practical experience using Python in Data Engineering.
Have experience designing, building, and optimizing ETL/ELT pipelines.
Understand processing of large structured and semi-structured datasets.
Have practical knowledge of Data Lake architecture, including Parquet, Delta Lake, and data partitioning.
Have experience with cloud platforms, particularly Microsoft Azure.
Know Git and CI/CD processes.
Have basic knowledge of statistics, data analysis, and data quality controls.
Be able to diagnose technical issues and optimize solution performance.
Be able to collaborate with technical and business stakeholders.
Be available immediately or with a maximum notice period of 14 days.

Design, build, and develop scalable ETL/ELT pipelines in Microsoft Azure.
Optimize Apache Spark processing using DataFrame API and Spark SQL.
Design and develop Data Lake solutions using Parquet, Delta Lake, and data partitioning.
Implement data quality controls and optimize data processing.
Maintain code using Git and automate deployments through CI/CD.
Prepare and process data for analytics solutions and Machine Learning projects.
Analyze and resolve technical issues involving performance and large-scale data processing.
Consult with technical and business stakeholders on project requirements.

View Full Description & ApplyYou'll be redirected to the employer's site