Senior Data Engineer
New
J
JobgetherData Engineering
100% remote work within the AmericasFull-TimeSenior
SalaryCompetitive salary and stock options.
Apply NowOpens the employer's application page
Job Details
- Experience
- 5+ years of professional experience in Data Engineering, including at least 2 years building and operating scalable, low-latency data platforms processing more than 100 million events per day.
- Required Skills
- PythonSQLGCPKafkaKubernetesAirflowData engineeringTerraform
Requirements
- 5+ years of professional experience in Data Engineering, including at least 2 years building and operating scalable, low-latency data platforms processing more than 100 million events per day.
- Strong hands-on experience operating data infrastructure on Kubernetes, with cloud-native technologies such as Docker and Helm.
- Production experience with infrastructure as code and deployment automation using Terraform, Ansible, ArgoCD, or equivalent technologies.
- Deep understanding of distributed systems, including storage, transactions, and query processing, with hands-on experience operating open-source query engines such as Trino or Presto.
- Strong experience with object storage and open table formats, particularly Apache Iceberg.
- Proven experience with streaming and CDC technologies such as Kafka, Redpanda, and Debezium.
- Hands-on experience with orchestration and ELT tooling, particularly Airflow and Airbyte.
- Strong Python and SQL skills for building production-grade pipelines and platform tooling.
- Experience with Google Cloud Platform and data services such as GCS, Cloud Build, Cloud SQL, and Dataproc, or comparable experience with other major cloud platforms.
Responsibilities
- Design, build, and continuously evolve core data platform infrastructure, including distributed query engines, orchestration, warehousing, cataloging, and related platform capabilities.
- Own lakehouse infrastructure as code and manage deployments using Terraform and Ansible across Kubernetes-based environments.
- Build and maintain low-latency streaming and change-data-capture pipelines, as well as batch ingestion workflows landing data in Apache Iceberg.
- Develop scalable, reliable data ingestion and processing solutions capable of supporting hundreds of millions of events per day.
- Expand and optimize the business intelligence landscape so downstream teams and AI agents can access lakehouse data efficiently and independently.
- Establish and maintain platform reliability practices, including monitoring, alerting, on-call processes, incident response, maintenance windows, runbooks, and service-level objectives.
- Partner with DevOps, Analytics Engineering, and other stakeholders to identify infrastructure gaps and deliver solutions for evolving data requirements.
- Use Python and SQL to develop data pipelines, automation, platform tooling, and supporting services.
View Full Description & ApplyYou'll be redirected to the employer's site
