AI Operations Specialist

JobgetherAI operations

Fully remote opportunity for candidates located within the United States.ContractSenior

Salary65 - 75 USD per hour

Apply NowOpens the employer's application page

Job Details

Experience: Between 5 and 7 years of professional experience in operations, program management, analytics, consulting, applied AI, or a related field
Required Skills: PythonSQLRESTful APIsQuality Assurance

Have 5–7 years of professional experience in operations, program management, analytics, consulting, applied AI, or a related field.
Have hands-on experience regularly using AI tools and building an AI-powered workflow, automation, or analytical solution with tangible business impact.
Understand performance measurement, including accuracy, precision, recall, and sampling methods.
Demonstrate the ability to turn ambiguous or inefficient processes into structured workflows with clear criteria and measurable outcomes.
Have experience in quality assurance, auditing, review operations, guideline development, or comparable quality management.
Be able to work independently, manage competing priorities, and take ownership of projects.
Preferred: Practical experience with SQL, Python, APIs, or other tools used to analyze data and build automated workflows.
Preferred: Experience in trust and safety, content moderation, data annotation, labeling operations, or other human-in-the-loop processes at scale.
Preferred: Experience with model fine-tuning, AI evaluation frameworks, or calibrating LLM-based judges.
Preferred: Experience in a two-sided marketplace, travel industry, or another operations-intensive business environment.
Preferred: Experience establishing intake processes, prioritization frameworks, or operational procedures for a newly formed team.
Able to commit to a 12-month, full-time contract beginning November 23, 2026, at 40 hours per week.

Identify and prioritize AI automation opportunities based on business impact, efficiency, feasibility, and expected value.
Map existing workflows with process owners and determine which tasks require AI automation or human intervention.
Design and implement AI-powered workflows, including prompt development, pipeline configuration, quality controls, and performance monitoring.
Develop evaluation datasets and test cases using metrics such as accuracy, precision, recall, and sampling-based quality assessments.
Evaluate AI performance and establish quality standards and launch-readiness criteria with product teams.
Translate complex policies into evaluation criteria and develop labeling guidelines with subject matter experts.
Coordinate external vendor teams conducting data labeling and review, validating work through metrics, sampling, and quality checks.
Monitor deployed workflows for quality drift and performance issues, and implement corrective actions.
Document workflows, evaluation methods, lessons learned, and best practices.

View Full Description & ApplyYou'll be redirected to the employer's site