QA Engineer, Voice AI Agents
New
A
ArbiterHealthcare AI
Location: Remote, USFull-TimeMiddle
Salary180,000 - 200,000 USD per year
Apply NowOpens the employer's application page
Job Details
- Experience
- 3 to 5 years in QA, support engineering, technical operations, or a similar role on a production software platform.
- Required Skills
- PythonSQLRESTful APIsJSONHIPAA
Requirements
- Have 3 to 5 years of experience in QA, support engineering, technical operations, or a similar role on a production software platform.
- Bring strong debugging instincts and investigate beyond the first plausible explanation.
- Be comfortable reading logs and querying data with SQL.
- Be comfortable working with APIs and JSON or YAML configuration.
- Have experience writing test plans and bug reports that engineers can act on.
- Use clear written communication and precise documentation.
- Apply sound judgment to severity and prioritization when multiple issues are live.
- Preferred: experience with voice AI, conversational AI, IVR, or contact center platforms.
- Preferred: scripting ability in Python or similar for automating checks and analysis.
- Preferred: exposure to LLM evaluation, prompt testing, or conversation quality scoring.
- Preferred: healthcare experience, especially payer or provider outreach, and familiarity with HIPAA and PHI handling.
- Preferred: experience with observability tools or call analytics tools.
Responsibilities
- Investigate reported and detected issues on live campaigns, including dropped calls, wrong agent responses, failed handoffs, incorrect scheduling outcomes, and data mismatches.
- Trace problems across call recordings, transcripts, system logs, campaign configuration, and integration data to identify root causes.
- Write reproducible bug reports with severity, member impact, and supporting evidence.
- Triage incoming issues, distinguish agent behavior from configuration errors, data problems, and platform bugs, and route them to the right owner.
- Run pre-launch QA on campaigns and workflow changes, testing scripts, edge cases, escalation paths, and handoffs to nurses or live staff.
- Perform ongoing conversation QA against defined quality standards and flag trends.
- Check campaigns for correct targeting, sequencing, disposition logic, and outcome capture.
- Build and maintain test cases, scoring rubrics, regression suites, and release checklists.
- Automate repetitive checks such as transcript analysis, configuration validation, and outcome reconciliation.
- Track quality metrics and report patterns to deployment, product, and engineering teams.
View Full Description & ApplyYou'll be redirected to the employer's site
