QA Automation Engineer
New
J
JobgetherAI software
Fully remote position for candidates currently residing in the United States.Full-TimeMiddle
SalaryTargeted annual compensation range of $95,553–$143,329
Apply NowOpens the employer's application page
Job Details
- Experience
- At least 4 years of QA automation engineering experience
- Required Skills
- PythonGitJavascriptQA AutomationTypeScriptAPI testingCI/CDPlaywright
Requirements
- At least 4 years of QA automation engineering experience, with ownership of automated testing programs or significant automation initiatives.
- Hands-on experience building and maintaining Playwright automation, including fixtures, resilient locators, assertions, network handling, and trace-based debugging.
- Strong programming skills in TypeScript or JavaScript and experience writing maintainable test code.
- Experience reviewing changes using Git-based workflows.
- Experience with API testing, CI/CD integration, test isolation, and troubleshooting across browser, application, and backend layers.
- Experience testing authentication, authorization, permissions, customer-data separation, and other security-sensitive application behavior.
- Familiarity with defense-in-depth concepts and validating that multiple layers of security controls work together.
- Ability to interpret complex requirements, identify edge cases, and balance test depth, customer risk, and delivery priorities.
- Strong written and verbal communication skills for explaining defects, uncertainty, tradeoffs, coverage gaps, and release risks.
- Strong analytical and troubleshooting skills, with the ability to work independently and collaborate across an engineering team.
- Ability to obtain a Department of Defense Secret security clearance.
- LLM application, retrieval-augmented generation, or AI agent testing experience is highly desirable; Python experience is preferred.
Responsibilities
- Translate product requirements, customer use cases, and feature interactions into testable outcomes and acceptance criteria with Product and Engineering.
- Design, implement, maintain, and expand Playwright automation for user journeys, feature functionality, regressions, API integrations, and end-to-end workflows.
- Develop a risk-based testing strategy that accounts for customer impact, feature dependencies, roadmap priorities, and delivery requirements.
- Test conversational AI workflows, including retrieval, citations, model selection, tool execution, interruptions, timeouts, and partial failures.
- Evaluate AI response quality using representative datasets and scoring criteria for accuracy, grounding, instruction following, and safe responses.
- Integrate automated tests into CI/CD pipelines, investigate flaky tests, maintain test data and fixtures, and provide actionable failure diagnostics.
- Document defects and release readiness with reproducible evidence, customer impact, severity, coverage gaps, and residual risks.
- Extend automation across APIs, authentication, billing, model routing, file parsing, MCP and tool execution, and provider API compatibility.
- Validate security-sensitive behavior, including user isolation, permission boundaries, input validation, rate limits, and safe error handling.
- Maintain automation infrastructure, fixtures, test data, documentation, and decision history as the product and engineering organization grow.
View Full Description & ApplyYou'll be redirected to the employer's site
