Senior Applied AI/ML Scientist
New
S
Sprout SocialAI/ML software
Location: Remote Canada. Candidates for this remote work opportunity must be based in either Alberta, British Columbia or Ontario.Full-TimeSenior
Salary140,700 - 177,900 CAD per year
Apply NowOpens the employer's application page
Job Details
- Experience
- 5+ years of applied AI/ML experience
- Required Skills
- PythonSQLA/B testing
Requirements
- Have 5+ years of applied AI/ML experience and a track record evaluating, measuring, and improving the quality of AI/ML or LLM systems that shipped and drove measurable impact.
- Have designed evaluation frameworks and built and calibrated reliable judges or automated evaluators for ML, LLM, or agentic systems.
- Demonstrate sound judgment in model selection and validation.
- Have rigorous experimentation and statistical skills, including A/B testing, power analysis, and measurement design.
- Have foundational ML and modeling knowledge, including Python, common ML frameworks, transformers, and embeddings.
- Have SQL skills and comfort working with large datasets.
- Be able to work with product, design, and engineering teams and translate technical quality questions into actionable terms for stakeholders.
- Preferred: Experience evaluating production LLM-driven or agentic products, including prompt quality, judge design, hallucination measurement, or tool-calling reliability.
- Preferred: Experience defining quality standards or evaluation methodologies adopted by other teams.
- Preferred: Familiarity with evaluation tooling and observability such as Datadog or Phoenix, MLOps practices, and responsible-AI practices.
Responsibilities
- Own end-to-end development of agents and agentic features, defining requirements, success criteria, evaluations, guardrails, and tools with engineering, design, and product teams.
- Lead model selection, development, and validation by assessing whether models or configurations perform better.
- Design and run A/B tests and other experiments to measure AI feature impact on customers.
- Set and evolve quality standards across agents in partnership with engineers building agents and evaluation infrastructure.
- Improve agent platform capabilities through prompt iteration, reusable judges, context-window management, tool use, and adoption of new models.
- Communicate AI capabilities, limitations, and quality to product, design, engineering, and leadership.
View Full Description & ApplyYou'll be redirected to the employer's site
