Senior AI Engineer
New
K
KotaFintech, HR Tech
We work remotely and our team is spread across the world.Full-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- 5+ years of software engineering experience, with at least 2 years working directly on LLM or AI agent systems in production
- Required Skills
- AWSAzure.NETRegression testingPrompt EngineeringLLMGenerative AI
Requirements
- 5+ years of software engineering experience.
- 2+ years of experience working directly on LLM or AI agent systems in production.
- Strong grasp of agent architecture, context management, tool use, retrieval strategies, and prompt design.
- Experience building evaluation frameworks including golden datasets and regression testing.
- Ability to distinguish and analyze retrieval vs. generation failure modes.
- Comfort working across the technical stack.
- Strong written communication skills for both technical and non-technical stakeholders.
- Background in fintech, payroll, HR tech, or benefits (preferred).
- Experience with AWS, .NET, or Azure (preferred).
- Experience generating synthetic evaluation data (preferred).
- Experience in early-stage or scale-up environments (preferred).
Responsibilities
- Design and ship AI agent systems end to end, from early experiments through to production.
- Own the data access layer, retrieval strategy, and integration with internal APIs and Metabase.
- Build and own the evaluation framework from scratch, including golden datasets and regression harnesses.
- Perform retrieval vs. generation failure analysis and mitigate risks like prompt injection and data boundary violations.
- Set the technical direction for AI at Kota and lead agent architecture design.
- Collaborate with the AI Product Lead to identify and surface new AI product opportunities.
View Full Description & ApplyYou'll be redirected to the employer's site
