- Work with model researchers to define what “good data” means for our models, including quality metrics, validation checks, and acceptance thresholds
- Explore open source datasets and create internal ones most suitable to build fundamental World Models
- Build algorithms for automated data quality assessment, data domain mixtures, and domain adaptation from synthetic to real data.
- Track datasets, metadata, provenance, and versions so experiments are reproducible and it’s clear what data went into which training and evaluation runs
- Own CI/CD and development tooling for the data stack (GitHub, Python, PyTorch), and automate repetitive workflows to reduce friction
- Track and optimize throughput, storage, and compute utilization across pipelines and related assets
PythonMachine LearningPyTorch+5 more