Senior AI Engineer
New
M
Monte CarloData Observability
Remote, AmericasFull-TimeSenior
Salary$180K - $240K; $180K – $240K • Offers Equity
Apply NowOpens the employer's application page
Job Details
- Required Skills
- PythonMachine LearningData scienceLLM
Requirements
- Experience building agents in production with real autonomy and internal loops.
- Deep understanding of models and their behavior, going beyond simple API integration.
- Expertise in designing and owning eval frameworks (golden datasets, regression suites).
- Proven experience monitoring agents in production post-launch.
- Strong proficiency in Python.
- Solid ML or data science background.
- Ability to work from ambiguous problem statements to research and prototype solutions.
- Comfort working with LLMs and using AI tooling daily.
- Ability to prioritize shipping functional production code over perfect implementations.
Responsibilities
- Take open problems end-to-end from research and prototyping through production.
- Design and ship agent-powered features such as root-cause analysis, incident triage, and monitor generation.
- Build evaluation infrastructure including golden datasets, regression suites, and offline/online scoring.
- Own retrieval and context pipelines over customer metadata, lineage, and query history.
- Instrument agent behavior in production including traces, failure taxonomies, and cost/latency budgets.
- Partner with data science on detection quality and experiment design.
- Set the technical bar for LLM integration patterns and guardrails.
View Full Description & ApplyYou'll be redirected to the employer's site
