Software Engineer, Benchmarking

Solid software engineering background with several years of professional experience. Ability to contribute high-quality, robust, and maintainable code. Comfortable diving deep into existing codebases and infrastructure. Ability to generate ideas for new benchmarks and experiments. Motivation by Epoch AI's mission. AI domain expertise is a strong plus but not required. Hands-on experience running LLM evaluations is a plus. Familiarity with evaluation frameworks like Inspect is a plus. Solid grasp of current AI trends is a plus.

Implement AI benchmarks within evaluation infrastructure. Develop existing benchmarks to quickly evaluate new model releases. Contribute to the development of brand new benchmarks. Collaborate with researchers, analysts, and engineers for accurate evaluation data.