- Own, debug, and improve production pipelines for accuracy and efficiency.
- Design and iterate on LLM evaluations and LLM-as-a-judge systems, including datasets, metrics, and benchmarks.
- Run experiments on prompts, pipeline design, model choice, and inputs, and turn results into recommendations.
- Improve LLM efficiency by comparing models, restructuring pipeline steps, and replacing LLM calls with smaller task-specific models where appropriate.
- Expand the anonymization pipeline by evaluating and adapting existing models.
- Work with Python and quality engineers to take experiments into production.
- Document findings in experiment reports and design documents.
PythonMachine LearningData science+2 more