Senior Data Engineer (Databricks)
New
N
N-iXIndustrial software
Workplace type: remote; Locations: Kraków, Zabłocie 43A, Kraków, Country code: PLContractSenior
Salary7,000 - 7,900 USD per month
Apply NowOpens the employer's application page
Job Details
- Languages
- English C1+; Polish C1+ / Native
- Required Skills
- PythonSQLMicrosoft SQL ServerAzureSparkTerraformDatabricks
Requirements
- Extensive experience designing, building, and evolving scalable data platforms and production data pipelines using Databricks, Spark, and Spark Lakehouse on Azure.
- Experience with Azure SQL Database, Azure Blob Storage, MSSQL databases, and infrastructure provisioning with Terraform.
- Experience leading technical design, mentoring engineers, and establishing engineering standards across teams.
- Strong software engineering skills developing reusable data-processing frameworks, tooling, and automation with Python and SQL.
- Experience maintaining code quality with ruff and pytest.
- Strong understanding of data modeling, schema evolution, data contracts, and validation across multiple source systems, including experience evolving monolithic architectures.
- Practical experience with automated and integration testing, linting, quality gates, Terraform, and deployment pipelines using GitLab Actions.
- Experience improving observability and monitoring with Grafana and building reliable data services on Azure.
- Experience implementing data quality and lineage approaches with Databricks and Spark.
- Experience simplifying engineering workflows, including project and issue tracking in Linear, and creating reusable tools.
- Ability to identify and evaluate practical AI use cases using tools such as GitHub Copilot and ChatGPT, and define safeguards for quality and reliability.
- English proficiency at C1+ and Polish proficiency at C1+ / Native.
Responsibilities
- Shape the technical direction of the data platform, tooling, and engineering standards.
- Design and improve reusable frameworks, internal tools, and automation for data pipeline development.
- Define standards for testing, CI/CD, quality gates, and platform reliability.
- Develop reusable approaches to data modeling, schemas, data contracts, validation, and schema evolution.
- Improve observability, data quality, lineage, and developer experience across the platform.
- Turn recurring delivery challenges into scalable platform capabilities that reduce customer onboarding effort.
- Apply AI to engineering workflows, establish guardrails, and measure its impact on productivity and quality.
- Lead technical design discussions, mentor engineers, and guide architectural decisions and technical trade-offs.
- Document best practices and support adoption of standard platform patterns across teams.
View Full Description & ApplyYou'll be redirected to the employer's site