Lead Software Engineer in Test – Agentic AI
New
N
NextechHealthcare technology
Remote - USFull-TimeLead
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- 8+ years of experience in Software Development or Software Engineer in Test (SDET/SET) roles; 5+ years of hands-on experience developing automation frameworks using JavaScript/TypeScript or Java; 2+ years of experience testing AI-powered applications, LLM systems, Agentic AI platforms, orchestration frameworks, or autonomous workflows.
- Required Skills
- CypressJavaJavascriptTypeScriptCI/CDPostmanPlaywright
Requirements
- Bachelor’s degree in Computer Science, MIS, Engineering, or a related technical field.
- 8+ years of experience in Software Development or Software Engineer in Test (SDET/SET) roles.
- 5+ years of hands-on experience developing automation frameworks using JavaScript/TypeScript or Java.
- Experience with automation tools such as Playwright, Cypress, Postman, K6, or equivalent frameworks.
- 2+ years of experience testing AI-powered applications, LLM systems, Agentic AI platforms, orchestration frameworks, or autonomous workflows.
- Hands-on experience validating agent orchestration workflows, conversational AI applications, multi-step reasoning systems, and AI-driven business processes.
- Experience testing AI orchestration frameworks such as LangGraph, CrewAI, AutoGen, Semantic Kernel, or similar technologies.
- Expertise in API testing, backend integration testing, UI automation, and end-to-end workflow validation.
- Understanding of automation architecture, design patterns, anti-patterns, dependency injection, and inversion of control principles.
- Experience in cloud-native and containerized environments such as Azure, AWS, GCP, Docker, or Kubernetes, and integrating automated test suites into CI/CD pipelines.
- Experience validating asynchronous, event-driven, and distributed systems, with a focus on resiliency and fault tolerance.
- Preferred: experience with AI observability, telemetry, RAG pipeline validation, vector databases, or prompt evaluation frameworks.
Responsibilities
- Design, develop, and maintain scalable automation frameworks and regression test suites across UI, API, backend, and AI-driven applications.
- Lead quality engineering initiatives for Agentic AI systems, including LLM workflows, RAG pipelines, autonomous agents, and distributed orchestration platforms.
- Validate AI agent behaviors, including multi-agent coordination, tool calling, prompt chaining, memory retention, conversational workflows, and fallback and retry mechanisms.
- Develop and execute functional, integration, regression, end-to-end, and performance testing strategies across cloud-native and distributed environments.
- Collaborate with Software Engineers, Product Managers, and DevOps teams to integrate testing throughout the software development lifecycle.
- Build reusable automation solutions to validate APIs, orchestration layers, asynchronous workflows, enterprise integrations, and AI-driven business processes.
- Design and maintain test environments, CI/CD integrations, cloud execution platforms, and automation pipelines.
- Establish best practices for code quality, maintainability, scalability, dependency injection, observability, and testability.
- Lead validation of AI workflow resiliency, distributed system reliability, data consistency, and orchestration performance under failure and edge-case conditions.
- Mentor QA engineers and contribute to QA standards, automation strategies, reporting practices, and continuous improvement.
View Full Description & ApplyYou'll be redirected to the employer's site
