AI Response Quality Evaluator
New
J
JobgetherArtificial Intelligence
Australia, At least 4 hours of overlap with Pacific Standard Time (PST)Contract
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Languages
- English
- Required Skills
- Analytical SkillsAttention to detailWritten communicationCritical thinking
Requirements
- Currently based in Australia.
- Willing and able to connect relevant Google applications to Gemini, subject to required consent and permissions.
- Active use of Google applications such as Gmail, Google Calendar, Google Photos, and Google Drive.
- Sufficient personal data or usage history within connected Google applications to support personalized retrieval evaluations.
- Strong analytical and critical-thinking skills, with excellent attention to detail.
- Excellent written English and the ability to communicate observations and evaluation results clearly.
- Ability to distinguish factual accuracy, relevance, personalization quality, and subtle contextual errors.
- Strong ability to follow detailed guidelines and apply evaluation criteria consistently.
- Comfortable working independently in a fully remote environment.
- Access to a desktop or laptop with a reliable internet connection.
- Bachelor’s degree or equivalent practical experience in any field.
- Ability to handle potentially sensitive personal information responsibly and maintain strict confidentiality.
- Availability to work with at least 4 hours of overlap with Pacific Standard Time (PST).
Responsibilities
- Evaluate AI-generated responses using personalized prompts and information retrieved from connected Google applications.
- Assess whether responses are relevant, accurate, helpful, contextually appropriate, and sufficiently personalized.
- Identify incorrect personalization, unsupported assumptions, irrelevant recommendations, inconsistencies, and other quality issues.
- Compare multiple AI-generated responses and assess differences in quality and overall user experience.
- Review nuanced AI outputs to identify both strengths and weaknesses, including subtle issues that may not be immediately apparent.
- Provide clear, detailed, structured, and well-reasoned feedback to support improvements to AI models and personalization systems.
- Apply project-specific evaluation guidelines consistently to ensure reliable and high-quality assessments.
- Maintain confidentiality and follow all applicable data privacy and security requirements throughout the project.
- Work independently and manage assigned evaluation tasks effectively within the project timeframe.
View Full Description & ApplyYou'll be redirected to the employer's site
