Senior Data Engineer - Platform

View full description

Requirements:

• Extensive experience in data engineering or platform engineering roles.
• Strong programming skills in Python and Java.
• Strong experience with modern data stacks (e.g., Spark, Kafka, DBT, Airflow, Lakehouse).
• Deep understanding of distributed systems, data architecture, and performance tuning.
• Experience with cloud platforms (AWS, GCP, or Azure) and Infrastructure-as-Code tools (Terraform, CloudFormation, etc.).
• Solid experience operating data services in Kubernetes, including Helm, resource tuning, and service discovery.
• Strong understanding of data modeling, data governance, and security best practices.
• Knowledge of CI/CD principles and DevOps practices in a data environment.

Responsibilities:

• Design, develop, and maintain scalable, secure, and high-performance data platforms.
• Build and manage data pipelines (ETL/ELT) using tools such as Apache Airflow, DBT, SQLMesh or similar.
• Architect and optimize lakehouse solutions (e.g., Iceberg).
• Lead the design and implementation of data infrastructure components (streaming, batch processing, orchestration, lineage, observability).
• Ensure data quality, governance, and compliance (GDPR, HIPAA, etc.) across all data processes.
• Automate infrastructure provisioning and CI/CD pipelines for data platform components using tools like Terraform, CircleCI, or similar.
• Collaborate cross-functionally with data scientists, analytics teams, and product engineers to understand data needs and deliver scalable solutions.
• Mentor experienced data engineers and set best practices for code quality, testing, and platform reliability.
• Monitor and troubleshoot performance issues in real-time data flows and long-running batch jobs.
• Stay ahead of trends in data engineering, proactively recommending new technologies and approaches to keep our stack modern and efficient.