AI Safety Specialist - Evaluation Expert
New
M
MercorAI Research
EstoniaContractMiddle
Salary60 - 70 USD per hour
Apply NowOpens the employer's application page
Job Details
- Languages
- English
- Experience
- 5+ years
Requirements
- Bachelor's degree or higher in Journalism, Communications, Psychology, Sociology, Public Policy, Law, Biology, Chemistry, Computer Science, or a related discipline.
- 5+ years of professional experience in AI Safety, Trust & Safety, journalism, public policy, scientific research, security, or a related field.
- Excellent written English, critical thinking, and analytical reasoning skills.
- Ability to consistently evaluate nuanced and policy-sensitive scenarios.
- Experience with AI Safety, RLHF, SFT, Trust & Safety, or AI evaluation (preferred).
- Familiarity with safety policies, content moderation, or evaluation rubric development (preferred).
- Experience reviewing complex, high-risk, or ambiguous content (preferred).
Responsibilities
- Evaluate AI-generated responses for safety, factual accuracy, policy compliance, and overall quality.
- Review content involving misinformation, political persuasion, self-harm, violence, cyber, biosecurity, and other sensitive domains.
- Apply and refine evaluation rubrics for RLHF, SFT, and AI safety benchmarking.
- Identify unsafe outputs, hallucinations, reasoning failures, and policy violations.
- Provide structured feedback to improve model alignment and safety performance.
- Collaborate with AI researchers and safety teams on ongoing evaluation initiatives.
View Full Description & ApplyYou'll be redirected to the employer's site
