Applied AI Engineer
New
D
DscoutUX Research Tech
Remote - IndiaFull-TimeMiddle
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- 2-5 years
- Required Skills
- Backend DevelopmentMachine LearningData modelingA/B testingPrompt EngineeringLangChain
Requirements
- 2-5 years of software engineering experience
- Hands-on experience building or operating LLM-powered features or agents in production
- Fluency with prompting and context engineering
- Experience building or maintaining evaluation harnesses for AI systems (offline eval sets, LLM-as-judge or human-in-the-loop scoring)
- Comfort reasoning about non-deterministic agent behavior across production traffic
- Experience running experiments (A/B, staged rollouts, production replay) to validate outcomes
- Proven track record of shipping features used by real users
- High-agency mindset and comfort investigating ambiguous underperformance problems
- Comfort using AI coding tools (e.g., Cursor, Claude Code, Copilot)
Responsibilities
- Own the production improvement loop across agent behavior, customer and operator feedback, evaluation, experimentation, and verified business outcomes
- Instrument agent workflows so model interactions, tool use, decisions, failures, human edits, and downstream outcomes can be understood in context
- Define meaningful quality standards, representative evaluation datasets, regression coverage, and production monitoring
- Investigate why agents underperform across context, knowledge, instructions, tools, routing, guardrails, or workflow design
- Design and ship targeted behavior improvements, including changes to prompting, context construction, decision logic, tool use, and human-review paths
- Build backend services, APIs, data models, and feedback pipelines that make agent behavior observable, steerable, and reproducible
- Run controlled experiments, production replays, or staged rollouts to measure whether changes improve quality and downstream business results
- Partner with Product, Data Science, and Sales to prioritize high-value problems and define customer and business success
- Ship with appropriate safeguards for privacy, security, reliability, human oversight, and safe operational rollout
View Full Description & ApplyYou'll be redirected to the employer's site
