Senior Data Engineer
New
CanadaFull-TimeSenior
SalaryCompetitive salary package and stock options.
Apply NowOpens the employer's application page
Job Details
- Experience
- 7+ years of experience in data engineering, including at least 2 years building scalable, low-latency data platforms processing more than 100 million events per day.
- Required Skills
- DockerPythonSQLApache AirflowGCPKafkaKubernetesTerraformdbt
Requirements
- 7+ years of experience in data engineering, including at least 2 years building scalable, low-latency data platforms processing more than 100 million events per day.
- Strong programming experience with at least one programming language, with advanced proficiency in Python and SQL.
- Hands-on experience with cloud-native technologies including Docker, Kubernetes, and Helm.
- Strong experience with relational databases and object storage technologies such as Apache Iceberg.
- Proven experience with Google Cloud Platform data services, including tools such as Composer, Dataproc, and Datastream.
- Experience designing scalable transformation layers using SQL-based modeling frameworks such as dbt.
- Experience with ETL orchestration tools and frameworks such as Apache Airflow and Airbyte.
- Production experience with streaming platforms such as Kafka.
- Knowledge of infrastructure, DevOps practices, and Infrastructure as Code tools such as Terraform.
- Deep understanding of distributed systems, storage architectures, transactions, and query processing.
- Experience working with open-source distributed query engines such as Trino (formerly PrestoSQL).
- Ability to work effectively in a fast-paced environment and adapt solutions based on evolving business needs.
Responsibilities
- Design and implement forward and reverse ETL patterns to deliver trusted data solutions to internal and external stakeholders.
- Develop scalable transformation frameworks that support analytics, reporting, and business intelligence needs across multiple teams.
- Expand and maintain a modern data lakehouse architecture, ensuring scalability, reliability, and performance as data volumes continue to grow.
- Build and optimize batch and streaming data ingestion pipelines across multiple systems and data sources.
- Collaborate with product, sales, marketing, and operations teams to understand data requirements and deliver effective solutions.
- Manage production systems, troubleshoot issues, and ensure timely resolution of data platform challenges.
- Improve data experimentation, cataloging, monitoring, and alerting capabilities.
- Establish reliable engineering practices around data quality, automation, and platform scalability.
- Contribute to architecture decisions involving distributed systems, storage solutions, and query optimization.
- Support the evolution of cloud-based data infrastructure using modern engineering approaches.
View Full Description & ApplyYou'll be redirected to the employer's site
