AI Operations Specialist - LLM & Conversation Systems
New
P
PavagoAI SMS Platform
Mexico. Colombia. Costa Rica. Argentina. Brazil. Peru, U.S. Business HoursFull-TimeMiddle
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Languages
- English
- Required Skills
- Data AnalysisRESTful APIsPrompt EngineeringLLM
Requirements
- Proven experience running LLM or AI systems in production environments.
- Hands-on experience monitoring, troubleshooting, and improving production AI outputs.
- Strong proficiency in prompt engineering and system prompt optimization.
- Deep understanding of LLM behavior, failure modes, and iterative testing.
- Experience analyzing AI-generated conversations or customer-facing outputs.
- Technical familiarity with webhooks, APIs, and knowledge base configurations.
- Excellent written English proficiency for messaging and documentation.
- Strong analytical, problem-solving, and detail-oriented mindset.
- Ability to work independently and manage production system ownership.
Responsibilities
- Onboard new clients by configuring platforms, webhook integrations, and knowledge bases.
- Monitor live AI-generated conversations to identify errors and inconsistencies.
- Diagnose root causes of AI failure modes and troubleshoot technical issues.
- Write, edit, and optimize prompts and system instructions to improve performance.
- Test configuration changes systematically before and after deployment.
- Analyze conversation transcripts and performance data to measure conversion impact.
- Collaborate directly with the founder to prioritize and implement system improvements.
View Full Description & ApplyYou'll be redirected to the employer's site
