AI Evaluator: Assessing a Shopping Assistant
J
JobgetherAI E-commerce
Based in United StatesContract
Salary50 USD per hour
Apply NowOpens the employer's application page
Job Details
- Required Skills
- Analytical SkillsQuality AssurancePrompt Engineering
Requirements
- Experience in data evaluation, quality assurance, AI training, data annotation, software testing, or prompt engineering.
- Strong analytical and critical-thinking skills for identifying logical errors and inaccuracies.
- Familiarity with e-commerce search, online shopping, product discovery, and digital shopping experiences.
- Ability to analyze complex text interactions and distinguish between technically correct and useful responses.
- Experience creating structured evaluation criteria or annotation frameworks is valuable.
- Strong attention to detail and consistency in applying evaluation standards.
- Ability to work independently in a remote environment.
- Availability to commit to a sustained workload of at least 20 hours per week.
Responsibilities
- Review real user interaction traces within a dedicated evaluation platform.
- Identify logical failures, factual inaccuracies, and poor product recommendations.
- Analyze AI-generated responses from an e-commerce perspective.
- Create structured evaluation rubrics for judging response accuracy and helpfulness.
- Develop verifiers and mechanisms to consistently assess future model responses.
- Contribute insights to broader efforts to improve AI model behavior.
View Full Description & ApplyYou'll be redirected to the employer's site
