- Design, build, maintain and primarily operate scalable streaming and batch data pipelines, with a strong focus on maintenance, monitoring, troubleshooting and continuous improvement of existing pipelines
- Work with AWS services, including Redshift, EMR and ECS, to support data processing and analytics workloads
- Develop and maintain data workflows using Python and SQL
- Orchestrate and monitor pipelines using Apache Airflow
- Build and deploy containerized applications using Docker and Kubernetes
- Break down high-level system designs into well-defined, deliverable tasks with realistic estimates
- Collaborate with cross-functional teams in a fast-paced and distributed environment across the US and Europe
- Drive automation, observability and monitoring to improve reliability, performance and operational efficiency
- Support knowledge transfer and ownership handover as part of the planned transition to the consuming team
AWSDockerPython+5 more