AI Software Development Engineer in Test
New
J
JobgetherAI software
Based in IndiaFull-Time
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Required Skills
- PythonSQLKubernetesAPI testingCI/CDPrompt Engineering
Requirements
- Strong programming skills in Python and ability to develop and maintain automation frameworks and testing solutions.
- Solid understanding of software testing principles, quality engineering methodologies, and modern software development practices.
- Hands-on experience with AI and LLM testing, prompt engineering, context engineering, AI evaluation frameworks, RAG systems, and AI agents.
- Experience building and maintaining CI/CD pipelines and working with Docker, Kubernetes, and cloud platforms such as AWS, Azure, OCI, or GCP.
- Strong expertise in API testing and automation, microservices testing, and UI automation.
- Experience with SQL, database testing, and data validation.
- Familiarity with Agile and Scrum methodologies and cross-functional engineering collaboration.
- A bachelor's or master's degree in computer science, engineering, artificial intelligence, machine learning, or a related discipline is preferred; equivalent practical experience and demonstrable projects are also valued.
- Robot Framework and Playwright experience are preferred.
- Knowledge of AI development frameworks and ecosystems, vector databases, embeddings, and semantic search is preferred.
Responsibilities
- Design and implement testing strategies for AI applications, copilots, chatbots, RAG pipelines, and autonomous agents.
- Develop evaluation frameworks to assess hallucinations, groundedness, toxicity, bias, response quality, relevance, consistency, and safety.
- Build automated validation pipelines for LLM outputs using assertions, semantic scoring, embeddings, and evaluation metrics.
- Create synthetic and benchmark datasets for AI testing scenarios.
- Test multi-agent workflows, MCP integrations, and AI system performance, including latency, token consumption, cost, throughput, and scalability.
- Develop and maintain Python-based UI and API automation frameworks and reusable testing libraries.
- Automate testing across APIs, event-driven systems, streaming applications, webhooks, microservices, and micro frontend architectures.
- Integrate automated tests into CI/CD pipelines and implement quality gates.
- Validate RAG architectures, vector databases, AI agents, cloud deployments, and distributed systems.
- Monitor AI system behaviour using observability, logging, and distributed tracing tools.
View Full Description & ApplyYou'll be redirected to the employer's site
