QA Engineer - GTM Applications
IndiaFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- 6+ years
- Required Skills
- PythonSalesforceAPI testingCI/CDLLM
Requirements
- Bachelor’s degree in Computer Science, Engineering, Information Systems, or a related field.
- 6+ years of experience in QA, SDET, or test automation roles within complex enterprise environments.
- Strong hands-on experience in Python for building scalable test automation frameworks.
- Proven experience testing AI/ML or LLM-based systems with understanding of non-deterministic behavior and evaluation challenges.
- Expertise with test automation tools such as pytest, Selenium, Playwright, Postman, or equivalent frameworks.
- Strong understanding of LLM behavior, including prompt sensitivity, temperature effects, token limits, and failure modes.
- Hands-on experience with Salesforce testing, including APIs, Lightning Web Components, and event-driven architectures.
- Strong API testing expertise including contract validation, schema verification, and error handling.
- Experience integrating automated testing into CI/CD pipelines to enforce continuous quality gates.
- Familiarity with RAG evaluation metrics and tools such as RAGAS or LangSmith for AI system assessment.
- Experience with performance testing tools such as k6, Locust, or JMeter for high-scale systems.
- Working knowledge of security testing principles including OWASP guidelines and prompt injection mitigation strategies.
Responsibilities
- Define and own the end-to-end QA strategy for agentic AI and GTM workflows, establishing quality standards for non-deterministic and LLM-powered systems.
- Design AI-specific evaluation frameworks covering hallucination detection, prompt scoring, response faithfulness, and agent task completion metrics.
- Develop and maintain scalable automated test suites in Python using frameworks such as pytest and Robot Framework across unit, integration, and system levels.
- Build and execute RAG pipeline testing strategies, including retrieval accuracy, semantic relevance, context grounding, and end-to-end response validation.
- Design QA automation for integrated systems including Salesforce, Slack applications, APIs, and cloud-based GTM tools.
- Integrate automated testing into CI/CD pipelines (GitHub Actions, Jenkins, Copado) to enforce quality gates on every deployment.
- Conduct performance, load, and scalability testing for LLM-powered APIs, analyzing latency, throughput, and degradation under stress.
- Perform security and adversarial testing, including prompt injection resistance, data leakage prevention, and validation with DevSecOps teams.
- Establish regression strategies for probabilistic systems using statistical thresholds, variance analysis, and snapshot-based validation techniques.
- Collaborate with engineering and product teams from requirements definition through release to ensure testability and quality alignment.
View Full Description & ApplyYou'll be redirected to the employer's site
