Engineering Manager SDET - AI
Company:
Location:15+ countries
Languages:English
Seniority level:Manager, 8-12 years (minimum 1 year in testing AI products)
Experience:8-12 years (minimum 1 year in testing AI products)
Skills:
AWSLeadershipPythonSoftware DevelopmentArtificial IntelligenceCloud ComputingCypressGCPMachine LearningQA AutomationAPI testingSeleniumCI/CDMicroservicesMentoring
Requirements:
8-12 years of overall experience Minimum 1 year in testing AI products Good experience in Prompt Engineering Testing AI applications (conversations AI, voice AI, workflow AI) Architect and scale test automation frameworks for APIs, micro-services, ML pipelines, and agentic AI workflows Experience testing RAG based applications (TruLens, RAGAs) Leverage AI to improve testing coverage and efficiency Hands-on experience testing agentic AI frameworks (Agentic Studio, LangChain, AutoGen, crewAI) Strong programming skills in Python and Java or TypeScript/JavaScript Proven expertise in test automation frameworks (Selenium, Playwright, Cypress etc.) Experience with cloud platforms (AWS, GCP) and CI/CD pipelines
Responsibilities:
Review testing strategies for AI applications Establish evaluation pipelines for detecting hallucination, context retention, latency Generate test data to simulate customer usage scenarios Test for data, model, and code issues Define relevant KPIs for AI application testing Perform non-functional testing (Performance, Scalability, Reliability, Security) Implement A/B testing to identify customer impact Run defect triaging to identify root cause and mitigate gaps Ensure quality deliverables from AI teams Hire top talent and mentor SDET team on AI testing skills
