Senior Data Platform Engineer

New

MARGOData platforms

Workplace type: Remote; Listing location: WarsawFull-TimeSenior

Salary180 - 200 PLN per hour

Apply NowOpens the employer's application page

Job Details

Experience: 6+ years of experience in data engineering, with significant time on platform or infrastructure topics
Required Skills: PythonSQLApache AirflowAzureTerraformDatabricksAzure DevOpsPySpark

Have 6+ years of experience in data engineering, with significant experience in platform or infrastructure topics.
Bring strong Python expertise and advanced SQL skills, including PySpark and Delta Lake semantics.
Have hands-on production experience with Apache Airflow, including DAG design, deployment, operations, upgrades, and orchestration of Databricks and ADF workloads.
Have hands-on Azure resource management experience, including Key Vault, managed identities, service principals, and secret management.
Understand access management with Azure RBAC, Entra ID, and Unity Catalog permissions.
Have experience with Azure DevOps repositories, pipelines, service connections, environments, and CI/CD for data platforms.
Have experience with ADLS, networking and security basics, and Azure Data Factory.
Have Infrastructure as Code experience with Terraform and multi-environment promotion.
Have hands-on experience with Databricks and Unity Catalog, including compute, jobs, and governance.
Understand observability, monitoring, alerting, and incident management.
Understand Data Lakehouse, Medallion, and metadata-driven architectures, as well as cost management and performance.

Own the Apache Airflow platform, including environments, deployments, upgrades, configuration, connections, variables, pools, and DAG standards.
Migrate orchestration from Azure Data Factory to Airflow and evolve the metadata-driven orchestration framework.
Manage Azure resources with Terraform, including Databricks workspaces, Data Lake Storage, Data Factory, Key Vault, and networking.
Own secrets, identity, and access management using Key Vault, managed identities, service principals, Azure RBAC, Entra ID, and Unity Catalog.
Manage Azure DevOps repositories, CI/CD pipelines, branch policies, multi-environment promotion, deployment validation, and Terraform state.
Track Azure resource costs and governance through tagging, budgets, right-sizing, and cost attribution.
Operate Databricks, including Unity Catalog structure, cluster and job policies, Auto Loader patterns, and workload performance and cost optimization.
Improve platform observability, monitoring, alerting, incident response, runbooks, and operational processes.
Participate in architecture, code, and design reviews; define and promote engineering standards.
Support the Run/Operations team, resolve platform incidents, lead root cause analyses, and ensure operational handover and documentation.

View Full Description & ApplyYou'll be redirected to the employer's site