Director of Quality
New
L
LILTAI translation
Remote or hybrid, EMEA or US Eastern time zone, US Eastern time zoneFull-TimeDirector
SalaryLondon, UK: £130K – £160K • Offers Equity; Boston, MA / Washington, DC: $160K – $190K • Offers Equity
Apply NowOpens the employer's application page
Job Details
- Required Skills
- PythonSQLCompliance
Requirements
- Demonstrable hands-on experience designing, building and running LLM-based agents in production.
- Scripting or data skills, such as Python or SQL, and experience turning quality data into actionable metrics and preventive controls.
- Proven experience improving LLM output quality.
- Significant experience running quality systems, including leading internal and external audits and compliance programmes.
- Experience writing and maintaining processes and procedures.
- Experience leading structured continuous improvement, including root cause analysis, corrective and preventive actions, and KPI reviews.
- Experience managing and developing people.
- Ability to translate customer quality requirements into acceptance criteria and governance, and advise on quality, cost and speed trade-offs.
- Ability to work credibly with customers, executives and technical teams.
- Background in localisation, translation or language services is a plus.
- Experience in customer-facing quality roles for enterprise accounts is a plus.
- Experience collaborating with research scientists and product managers to turn quality insight into product requirements and shipped improvements is a plus.
Responsibilities
- Evaluate, design and build Quality Agents with well-defined tools and structured instructions.
- Define agent evaluation criteria, test sets and success metrics, and operate agents in production with monitoring and escalation paths.
- Partner with Research and Product on model improvements, AI Review, AI QA and emerging AI workflows.
- Own internal and external audits, manage evidence collection, and track corrective and preventive actions to closure.
- Maintain SOPs, workflows and quality documentation to meet customer, contractual and regulatory requirements.
- Advise customers on quality programmes, acceptance criteria, validation methods and risk-appropriate controls.
- Lead continuous improvement through root cause analysis, corrective and preventive actions, KPI reviews and retrospectives.
- Partner with Expert Supply and Ops to improve expert qualification, guidance and ongoing quality management.
- Lead and develop the Quality team and oversee customer-facing Quality Manager staffing.
View Full Description & ApplyYou'll be redirected to the employer's site
