Senior Data Engineer / Data Architect
New
S
Startup HouseEnvironmental monitoring
Remote/Warsaw OfficeContractSenior
Salary160 - 200 PLN per hour
Apply NowOpens the employer's application page
Job Details
- Languages
- At least B2 in English
- Experience
- Senior experience in data engineering (around 5+ years)
- Required Skills
- PythonTerraform
Requirements
- Have taken at least 2–3 data pipeline projects from zero to production as an architect or lead data engineer.
- Have Azure Data Explorer in production experience on at least one project.
- Bring around 5+ years of senior data engineering experience, including IoT streams and time-series data.
- Have broad, hands-on experience with Azure data services.
- Know Azure Data Explorer (Kusto) and KQL, including table modelling, update policies, materialized views, retention policies, and query optimization for large volumes.
- Have experience with streaming ingestion on Azure, including IoT Hub, Event Hubs, parsing functions such as Azure Functions, and Blob Storage.
- Understand layered data architecture (raw/Bronze to Silver to aggregates), rollups, and time-window aggregations.
- Have Python and Terraform skills.
- Be able to own a production pipeline end to end independently.
- Have at least B2 English.
Responsibilities
- Take over the proof of concept and bring the streaming IoT measurement pipeline to production.
- Build the stream processor from Event Hubs to Azure Data Explorer.
- Design and schedule rollups and time-window aggregations.
- Implement data reprocessing after configuration changes.
- Model and optimize Azure Data Explorer tables, update policies, materialized views, retention policies, and queries for large data volumes.
- Work with the PM, ADX specialist, client's engineering team, and DevOps.
View Full Description & ApplyYou'll be redirected to the employer's site
