Senior Data Engineer

Demonstrated ability to navigate ambiguous data challenges and design scalable solutions Proficient in designing, building, and maintaining large-scale, reliable data pipeline systems Competence in designing and handling large-scale data pipeline systems Advanced SQL skills Proficiency in Python, with experience in Spark 3+ years of experience in data engineering, including data modeling and ETL pipelines Familiarity with cloud-based tools and infrastructure management using Terraform and Kubernetes is a plus

Explore, analyze, and onboard data sets Develop and maintain scalable data pipelines for collection, processing, and integration Optimize and monitor data pipeline performance Implement data quality control mechanisms Collaborate with stakeholders for seamless data flow Manage deployment and automation of pipelines and infrastructure using Terraform, Flyte, and Kubernetes Support strategic data analysis and operational tasks Lead end-to-end data pipeline development Integrate and manage data from 3+ distinct sources