Subject Matter Expert, ML Model Evaluation (PhD)
New
W
Weekday AIMachine learning evaluation
Applicants must be from any of these geographies: US, UK, Canada, the EU, Norway, Switzerland and Iceland.Contract
Salary150 USD per hour
Apply NowOpens the employer's application page
Job Details
- Required Skills
- Analytical SkillsAttention to detailCritical thinking
Requirements
- Hold a PhD or have equivalent advanced subject matter expertise in a relevant academic, technical, or professional discipline.
- Have a strong academic or professional background in a specialised domain.
- Be able to critically assess complex information and technical content.
- Evaluate research findings and domain-specific information for factual correctness and relevance.
- Bring strong analytical, research, and critical-thinking skills.
- Provide clear, structured written evaluation justifications.
- Demonstrate high attention to detail and identify subtle inaccuracies, inconsistencies, and gaps.
- Work independently and complete approximately five hours of asynchronous evaluation tasks.
- Have a finished or in-progress paper in NeurIPS, ICML, ICLR, COLM, or RLC.
Responsibilities
- Review machine learning model outputs generated in response to research-focused and domain-specific prompts.
- Evaluate AI-generated responses for factual accuracy, relevance, completeness, and overall quality.
- Assess the technical validity and reliability of model-generated information.
- Use a structured rating system to score responses consistently.
- Provide concise, clear, and well-reasoned written justifications for evaluation scores.
- Identify factual errors, gaps in reasoning, irrelevant information, and areas requiring improvement.
- Complete approximately five hours of independent evaluation work asynchronously.
- Maintain consistency and attention to detail throughout the evaluation process.
View Full Description & ApplyYou'll be redirected to the employer's site
