Sr Lead Machine Learning Engineer

Proven experience designing evaluation systems for agentic or LLM-based AI Deep expertise in statistical experimentation, benchmark creation, and human-AI interaction assessment Fluency in building data pipelines and tooling using Python, SQL, and distributed data processing frameworks Demonstrated ability to influence product and model roadmaps Adaptive-level proficiency in integrating AI tools into technical workflows

Design and implement comprehensive evaluation frameworks for agentic systems Build benchmarking pipelines for nuanced success indicators Lead development of observability tools and instrumentation for agent behavior Translate qualitative and quantitative signals into actionable insights Collaborate with researchers, engineers, and product teams Own benchmarking infrastructure for reproducible, scalable evaluation Champion rigorous experimental design and statistical analysis