Quality Evaluator – English
New
A
Anyone AIAI training
Argentina - Fully Remote; Secondary Locations: Uruguay, Chile, Poland, Italy, Czech Republic, Estonia, Ecuador - Fully Remote, Portugal, Mexico - Fully Remote, Paraguay, Colombia - Fully Remote, France, Spain, Peru - Fully Remote, BrazilContract
Salary7 - 12 USD per hour
Apply NowOpens the employer's application page
Job Details
- Languages
- Fluency in English
- Required Skills
- Attention to detailWriting
Requirements
- Demonstrate fluency in English, with strong reading comprehension and writing skills.
- Have extensive hands-on experience using large language models (LLMs) and critically assessing their outputs.
- Bring strong analytical judgment and the ability to weigh multiple dimensions of response quality.
- Show excellent attention to detail and the ability to follow complex instructions carefully.
- Explain evaluation decisions clearly and convincingly.
- Be comfortable switching between AI models, tools, and multi-step workflows.
- Previous experience in AI response evaluation, data annotation, or quality assurance is a plus.
- Experience in editing, research, fact-checking, or analytical writing is a plus.
- Familiarity with evaluation rubrics and side-by-side comparison tasks is a plus.
Responsibilities
- Interact with different AI models and tools across a wide range of prompts and topics.
- Compare two or more AI-generated responses side by side.
- Assess accuracy, helpfulness, instruction-following, clarity, and overall response quality.
- Follow detailed rating guidelines and multi-step evaluation workflows.
- Write clear, well-reasoned explanations supporting evaluation decisions.
- Apply evaluation criteria consistently across tasks.
View Full Description & ApplyYou'll be redirected to the employer's site
