Senior Software Development Test Engineer (SDET), AI & Automation
New
K
Keeper SecurityCybersecurity software
Location: Remote, USFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- 5+ years of experience in SDET, software development, test automation or another code-intensive quality engineering role
- Required Skills
- PythonJavaTestRailRustCI/CDRESTful APIs
Requirements
- Have 5+ years of experience in SDET, software development, test automation or another code-intensive quality engineering role.
- Have hands-on experience testing AI/ML, LLM-powered or model-integrated software systems.
- Understand the differences between deterministic software testing and non-deterministic AI/ML validation.
- Have experience defining evaluation criteria, expected behaviors, tolerances or scoring approaches for AI-driven functionality.
- Have strong programming skills in Python, Java, JavaScript/TypeScript, Rust or another general-purpose programming language.
- Have significant experience designing, developing and maintaining automated test frameworks.
- Have experience building automated functional, integration, API and end-to-end test coverage.
- Have experience testing REST APIs and distributed application workflows using Postman, REST Assured or similar platforms.
- Have experience integrating automated tests into CI/CD pipelines using GitHub Actions, Jenkins, GitLab CI or similar technologies.
- Have experience working in an Agile software development team with QA, Engineering and Product.
- Understand shift-left testing, continuous quality and testability principles; experience with TestRail or a similar test management platform is required.
- Have debugging and troubleshooting experience using logs, APIs, application telemetry and development tools, plus experience with Git-based workflows and code review.
- Have a bachelor’s degree in Computer Science, Engineering or a related technical field, or equivalent practical experience.
Responsibilities
- Partner with AI engineers, software engineers and product managers during design and development to define test strategy, acceptance criteria and automation requirements.
- Design and maintain automated testing frameworks for AI-powered applications, backend services, APIs and supporting software systems.
- Develop testing strategies for deterministic software behavior and non-deterministic AI/ML outputs.
- Define measurable evaluation criteria, tolerances and quality thresholds for AI model and LLM-driven functionality.
- Build automated regression, integration, API and end-to-end test coverage alongside feature development.
- Develop test harnesses and evaluation workflows for prompts, models, agents, tool-calling workflows and AI-integrated applications.
- Integrate automated validation and quality gates into CI/CD pipelines.
- Build and maintain test datasets, mocks, fixtures and synthetic data for repeatable validation.
- Analyze model and application failures, logs and telemetry to identify root causes and produce reproducible defect reports.
- Improve test stability, maintainability, testability and diagnostic capabilities, and contribute to quality engineering standards.
View Full Description & ApplyYou'll be redirected to the employer's site
