Researcher, Benchmark Reviews
New
This role is fully remote; we are able to hire in many countries.Full-Time
Salary100,000 - 200,000 USD per year
Apply NowOpens the employer's application page
Job Details
- Required Skills
- Data AnalysisMachine Learning
Requirements
- Excellent written communication skills with the ability to produce clear, publication-quality drafts.
- Strong critical thinking skills regarding research methodologies.
- Familiarity with existing AI benchmarks, their strengths, weaknesses, and methodologies.
- Ability to analyze and break down dense data into component parts.
- Experience writing about AI/ML for a public or research audience (nice to have).
- Hands-on experience running or building AI evaluations (nice to have).
Responsibilities
- Assess a new benchmark at least every two weeks.
- Evaluate benchmark methodologies and determine what good performance implies about AI capabilities.
- Write and publish public-facing reports on new benchmarks.
- Periodically update reports as new versions are released or models make progress.
- Examine individual benchmark tasks in detail using coding agents while maintaining critical oversight.
View Full Description & ApplyYou'll be redirected to the employer's site
