Senior Engineering Manager, AI Quality & Governance
New
M
MitratechLegal Technology
Remote USFull-TimeManager
Salary$210K – $230K Annually USD
Apply NowOpens the employer's application page
Job Details
- Experience
- 8+ years
- Required Skills
- AWSBackend DevelopmentPythonTypeScriptLLM
Requirements
- 8+ years of professional software engineering experience.
- Proven experience leading engineers as a manager or technical lead.
- Demonstrated hands-on experience building, shipping, or operating production AI systems, specifically LLM-powered or agentic workflows.
- Strong experience designing and operating production backend systems and APIs.
- Proficiency in Python and TypeScript.
- Experience with AI observability, evaluation, or debugging systems (e.g., LangSmith, LangFuse).
- Practical experience designing or operating AI guardrails such as content filtering, redaction, and access controls.
- Familiarity with AI governance and risk frameworks such as NIST AI RMF or ISO/IEC 42001.
- Strong bias toward hands-on execution and platform thinking.
- Ability to engage thoughtfully with legal, compliance, and security stakeholders.
Responsibilities
- Own the AI evaluations and observability platform including tracing, logs, dashboards, and operational feedback loops.
- Design and operationalize automated and human-in-the-loop evaluation strategies for quality, safety, latency, and cost.
- Establish organizational standards for production-ready AI, including release gates and incident playbooks.
- Build and operate AI guardrails, content controls, and PII redaction capabilities.
- Translate governance and risk expectations into scalable engineering systems.
- Own platform-level SLOs and provide tier-two incident support for AI behavior.
- Act as an internal authority on AI quality and governance in stakeholder and customer conversations.
- Hire, mentor, and grow the AI Quality & Governance team.
View Full Description & ApplyYou'll be redirected to the employer's site
