Agent Evals Specialist
New
Workable workplace: remote Workable remote: True Workable locations: Philippines Location: Philippines, Preferred overlap anytime during 10AM to 10 PM PSTFull-Time
Salary900 - 1,450 USD per month
Apply NowOpens the employer's application page
Job Details
- Required Skills
- QAEditingSlack
Requirements
- Strong written English. Can read dense technical content for hours without losing focus
- Consistent — your scoring on Monday matches your scoring on Friday
- Clear, specific feedback:"section 4 dropped the key requirement from page 17", not "this is confusing"
- Prior work as an AI trainer, tutor, or evaluator (Outlier, DataAnnotation, xAI, Surge, Mercor, Invisible, Toloka, etc.)
- Technical writing, editing, QA, translation, paralegal, or research-assistant background
- Markdown familiarity
- Slack and internal platform experience
Responsibilities
- Read the source and the AI agent's output side by side. Verify the content was captured accurately.
- Review what the AI agent did. What it created, changed, or left out.
- Score a short rubric covering accuracy, coverage, organization, and rule adherence. Full rubric provided at onboarding.
- Write detailed feedback about the mistake. This is the most important thing you produce since we use it to improve the agent.
- Submit. Move to the next task.
View Full Description & ApplyYou'll be redirected to the employer's site
