AI Operations Specialist - LLM & Conversation Systems
New
P
PavagoConversational AI
Source API remote eligibility restrictions: Philippines, U.S. Business HoursFull-Time
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Languages
- Excellent written English — required for editing customer-facing AI messages and documenting system changes.
- Required Skills
- RESTful APIsPrompt Engineering
Requirements
- Proven experience running LLM or AI systems in production.
- Hands-on experience monitoring and improving production AI outputs.
- Experience with prompt engineering and system prompt optimization.
- Strong understanding of LLM behavior, failure modes, and iterative testing.
- Experience analyzing AI-generated conversations or other customer-facing outputs.
- Ability to work with webhooks, APIs, knowledge bases, or similar integrations.
- Excellent written English for editing customer-facing AI messages and documenting system changes.
- Strong analytical and problem-solving skills.
- Strong attention to detail.
- Ability to work independently and take ownership of production systems.
Responsibilities
- Onboard clients and configure their AI conversation systems, campaigns, workflows, and knowledge bases.
- Set up webhook integrations and troubleshoot onboarding and configuration issues.
- Test client systems before they go live.
- Monitor live AI-generated conversations and review interactions for errors, inconsistencies, and missed opportunities.
- Diagnose AI failure modes, identify root causes, and escalate technical issues through resolution.
- Write and improve prompts, system instructions, and conversation logic based on performance.
- Test changes before and after deployment and document prompt and system updates.
- Analyze conversation transcripts and performance data to measure the effects of changes on conversion.
- Identify patterns across conversations, track outcomes, and communicate findings to leadership.
- Work with the founder on daily operations, prioritization, and production system improvements.
View Full Description & ApplyYou'll be redirected to the employer's site
