- Design and implement reliable, scalable data ingestion and integration pipelines for structured, semi-structured, unstructured, and multi-modal data.
- Build and scale retrieval infrastructure, including vector storage, embedding pipelines, hybrid search, and graph-based knowledge representations.
- Develop and operate agent memory systems and pipelines for AI system signals to support observability and continuous improvement.
- Apply data quality, validation, monitoring, and testing frameworks in production pipelines.
- Monitor, troubleshoot, and optimize AI data pipelines and retrieval workflows for reliability, performance, and cost.
- Design and support evaluation workflows for AI systems, enabling offline testing and benchmarking.
- Lead pragmatic platform evolution by defining clear contracts between AI services and data systems.
AWSPythonTypeScript+4 more