Software Development Engineer in Test - AI
New
J
JobgetherAI software
Based in IndiaFull-Time
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Required Skills
- GraphQLPythonSQLCI/CDRESTful APIsPrompt Engineering
Requirements
- Bachelor's or Master's degree in Computer Science, Software Engineering, Artificial Intelligence, Machine Learning, or a related field, or equivalent practical experience.
- Strong Python programming skills, including developing automation frameworks, reusable testing utilities, and validation workflows.
- Solid understanding of software testing principles, quality engineering methodologies, and modern test automation practices.
- Hands-on experience with AI/LLM testing, prompt and context engineering, AI evaluation frameworks, RAG architectures, and AI agents.
- Practical experience validating AI-powered workflows, including output evaluation, hallucination detection, response quality, and safety.
- Experience with API testing and automation, microservices testing, UI automation, and SQL data validation.
- Familiarity with AI agent technologies such as LangChain, LangGraph, LlamaIndex, CrewAI, Semantic Kernel, and MCP.
- Experience integrating automated testing into CI/CD pipelines.
- Experience with Docker, Kubernetes, and cloud platforms such as AWS, Azure, OCI, or GCP.
- Experience working in Agile or Scrum environments and collaborating with multidisciplinary engineering teams.
- Strong debugging, analytical, troubleshooting, and problem-solving abilities.
Responsibilities
- Design, implement, and optimize testing strategies for AI applications, chatbots, copilots, RAG pipelines, and autonomous agents.
- Develop and maintain evaluation frameworks for hallucinations, groundedness, toxicity, bias, response quality, prompt regressions, consistency, and safety.
- Build automated pipelines to validate LLM outputs using assertions, semantic scoring, embeddings, and evaluation metrics.
- Create synthetic and benchmark datasets for AI validation scenarios.
- Test agent workflows, planning, memory, context management, tool and function calling, structured outputs, and MCP integrations.
- Develop scalable UI and API automation frameworks, reusable testing libraries, and AI-assisted testing workflows.
- Automate testing of REST and GraphQL APIs, event-driven systems, streaming services, webhooks, microservices, and micro-frontends.
- Validate RAG components including document chunking, embeddings, retrieval, reranking, and vector search.
- Integrate automated tests and quality gates into CI/CD pipelines and validate deployments across containers, Kubernetes, and cloud environments.
- Monitor application behavior and evaluate latency, token consumption, throughput, cost, scalability, and reliability.
View Full Description & ApplyYou'll be redirected to the employer's site
