QA Engineer - GTM Applications
New
Remote, INDFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- 6+ years
- Required Skills
- PythonSalesforceSeleniumCI/CDRESTful APIsPostman
Requirements
- Bachelor’s degree in Computer Science, Engineering, Information Systems, or a related field.
- 6+ years of QA or SDET experience building and maintaining automated test frameworks in production.
- Hands-on experience testing AI/ML systems and understanding of non-deterministic output evaluation.
- Strong proficiency in Python for test automation (fixtures, mocks, data factories).
- Experience with test frameworks like pytest, Selenium, Playwright, REST-assured, or Postman.
- Practical understanding of LLM behaviors, including temperature effects, token limits, and prompt sensitivity.
- Hands-on Salesforce QA testing experience including Lightning Web Components (LWC) and API integrations.
- API testing proficiency for REST endpoints, including contract validation and schema conformance.
- Experience integrating automated tests into CI/CD pipelines.
- Familiarity with RAG evaluation metrics and tooling such as RAGAS or LangSmith.
- Experience with performance/load testing tools like Locust, k6, or JMeter.
- Basic knowledge of security testing (OWASP Top 10, prompt injection, output sanitization).
Responsibilities
- Define and own the end-to-end test strategy for agentic AI workstreams, establishing quality standards for probabilistic, non-deterministic systems.
- Design and implement AI-specific evaluation frameworks covering hallucination detection, prompt quality, agent task completion, and output faithfulness.
- Build and maintain automated test suites in Python (e.g., pytest, robot framework) for unit, integration, and system-level scenarios.
- Develop RAG pipeline test coverage for retrieval precision, semantic relevance, and end-to-end accuracy using tools like RAGAS.
- Integrate automated tests into CI/CD pipelines (e.g., GitHub Actions, Jenkins) to enforce quality gates on pull requests.
- Execute performance and load testing for LLM-powered APIs to measure latency and throughput.
- Conduct security and adversarial testing including prompt injection attempts and output validation.
- Own API contract testing for integrations including Salesforce, Marketo, Snowflake, Gong, Clari, and G-Suite.
View Full Description & ApplyYou'll be redirected to the employer's site
