Senior Automation Tester in Python (Agentic Gateway Testing)
New
E
EPAMAI agent platforms
Opportunity to work remotely within PolandFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Languages
- English B2
- Experience
- 3+ years of QA engineering experience for AI/ML or agent systems
- Required Skills
- PythonCI/CD
Requirements
- Have 3+ years of QA engineering experience for AI/ML or agent systems.
- Bring hands-on experience testing AI agent systems in a production agentic AI project.
- Have experience testing LLM agent workflows, including tool calling and multi-step reasoning.
- Demonstrate expertise in security testing for authorization and authentication.
- Have skills in performance benchmarking for cloud services.
- Use Python and pytest for test automation.
- Familiarity with MCP protocol testing is listed as a nice-to-have.
- Background in agent framework testing, such as LangChain, LangGraph, or Crew, is a nice-to-have.
- Knowledge of AWS Bedrock Agents testing is a nice-to-have.
- Understanding of chaos engineering for agentic systems is a nice-to-have.
- Familiarity with audit log validation using CloudTrail or OpenTelemetry traces is a nice-to-have.
Responsibilities
- Design and execute test suites for agentic workflows, including tool discovery, tool invocation, and multi-step agent workflows.
- Validate MCP protocol behavior, including tool list aggregation, tool call routing, and streaming response handling.
- Verify agent authorization mechanisms, including unauthorized tool access attempts and policy enforcement.
- Conduct agent performance testing covering tool discovery latency, concurrent agent sessions, and agent-to-agent latency.
- Build Python-based test automation using pytest with async support, mock MCP servers, and agent workflow simulation.
- Validate agentic observability, including trace completeness, agent step logging, and audit trail verification.
- Collaborate with engineering teams to define quality standards for agent framework design and runtime architecture.
- Integrate test suites into deployment pipelines to support CI/CD automation.
View Full Description & ApplyYou'll be redirected to the employer's site
