AI Response Quality Evaluator (Gemini)
New
G
Gramian ConsultancyIT Professional Services
Remote — Australia only, 4 hours of overlap with PSTContract
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Languages
- Excellent written English
- Required Skills
- Attention to detailWritten communicationGoogle WorkspaceAnalytical thinking
Requirements
- Currently based in Australia.
- Willing and able to connect relevant Google apps to Gemini, subject to required consent and permissions.
- Active use of Google applications, with sufficient personal data or history to support personalized retrieval evaluations.
- Strong analytical thinking and attention to detail.
- Excellent written English and ability to provide structured evaluation feedback.
- Ability to work independently in a remote environment.
- Access to a desktop or laptop with a reliable internet connection.
- Bachelor’s degree or equivalent practical experience in any field.
Responsibilities
- Evaluate AI-generated responses using personalized prompts and information retrieved from connected Google applications.
- Assess whether responses are relevant, accurate, helpful, and appropriately personalized.
- Identify incorrect personalization, unsupported assumptions, irrelevant recommendations, and inconsistencies.
- Compare multiple AI-generated responses and evaluate differences in overall user experience.
- Provide clear, detailed, and well-reasoned feedback to support AI model improvements.
- Follow project guidelines to ensure consistent and high-quality evaluations.
- Review nuanced AI responses and identify strengths, weaknesses, and subtle quality issues.
- Maintain data privacy and follow all confidentiality requirements throughout the project.
View Full Description & ApplyYou'll be redirected to the employer's site
