AI Response Labeler / Annotator
J
JobgetherArtificial Intelligence
Based in United StatesFull-Time
Salary38.46 - 40.87 USD per hour
Apply NowOpens the employer's application page
Job Details
- Languages
- Castilian Spanish, English
- Required Skills
- Analytical SkillsCritical thinking
Requirements
- Native-level or professional fluency in Castilian Spanish, with deep understanding of Spain-specific linguistic and cultural conventions.
- Strong English fluency and reading comprehension, including the ability to understand detailed guidelines and AI-generated content.
- Strong analytical and critical-thinking skills beyond language evaluation alone.
- Ability to assess content quality across different topics, formats, and user scenarios.
- Ability to evaluate factual accuracy, relevance, reasoning, clarity, instruction adherence, and overall usefulness.
- Excellent attention to detail and ability to maintain accuracy while completing repetitive evaluation tasks.
- Strong written communication skills with the ability to explain decisions clearly and concisely.
- Ability to independently apply structured criteria in ambiguous situations.
- Comfort working with detailed frameworks, rubrics, and quality standards.
- Ability to manage a high volume of evaluations while maintaining consistent judgment.
- Ability to receive feedback, adapt to evolving guidelines, and continuously improve evaluation quality.
Responsibilities
- Compare AI-generated responses side-by-side and determine which response better addresses user needs.
- Evaluate responses based on factual accuracy, relevance, completeness, reasoning, clarity, tone, safety, and instruction-following.
- Apply Castilian Spanish expertise to assess language quality, regional accuracy, cultural context, terminology, and natural usage in Spain.
- Review content in English, Spanish, or mixed-language scenarios across various AI tasks, including conversations, search results, file-based tasks, image-related responses, and content generation.
- Identify subtle quality differences, including unsupported claims, missing information, unclear reasoning, unnatural phrasing, and cultural inconsistencies.
- Follow detailed annotation guidelines and maintain consistent evaluation standards across a high volume of assignments.
- Document evaluation decisions with concise, evidence-based explanations when required.
- Participate in training sessions, calibration activities, qualification reviews, and ongoing quality improvement processes.
- Incorporate feedback to continuously improve evaluation accuracy and alignment with quality expectations.
View Full Description & ApplyYou'll be redirected to the employer's site
