- Build and extend the data lakehouse on AWS combining storage and warehouse technologies.
- Contribute to knowledge graph modeling and vector database integration for AI agents.
- Develop robust ETL/ELT pipelines for batch and real-time data ingestion and transformation.
- Prepare training datasets, feature stores, and workflows to support ML model and AI feature deployment.
- Collaborate with DevOps to deploy, containerize, and orchestrate data services using Docker/Kubernetes on AWS EKS.
- Implement CI/CD pipelines for automated testing and deployment of data workflows.
- Monitor performance, troubleshoot production issues, and recommend new technologies to improve pipeline reliability.
AWSDockerPython+7 more