Senior AI Development Lead

New
A
AirbnbAI and machine learning
US - Remote Eligible. You must live in a state where Airbnb, Inc. has a registered entity.Full-TimeLead
Salary$157,000 — $171,000 USD
Apply NowOpens the employer's application page

Job Details

Experience
8+ years in product or program management, analytics, consulting or applied AI
Required Skills
PythonSQLData AnalysisRESTful APIsPrompt Engineering

Requirements

  • Have 8+ years of experience in product or program management, analytics, consulting, or applied AI.
  • Have a proven record of taking technical programs from idea to production and owning outcomes such as backlog, cycle time, or launch speed.
  • Be strong in SQL and quality statistics, including accuracy, precision, recall, sampling, and reviewer agreement.
  • Have shipped an AI-based workflow, agent, or automation that others rely on, and be able to show before-and-after results and explain what changed when it failed.
  • Be technically fluent enough to assess engineering estimates and understand how prompts, pipelines, and evaluations fit together.
  • Have experience making messy processes measurable and judging success by business outcomes.
  • Be able to turn qualitative feedback, including reviewer comments and disagreements, into actionable quality improvements.
  • Write clearly and work effectively with sales, operations, product managers, and engineers.
  • Be comfortable taking initiative on new problems and trying solutions hands-on.
  • Hands-on Python and API experience is a plus.
  • Experience with LLM evaluation, experimentation and measurement, or human-in-the-loop work at scale is a plus.

Responsibilities

  • Drive supported AI workflows and features from design to launch, defining quality standards and measurement plans with engineering, product, and business partners.
  • Prioritize projects by business impact and make trade-offs with managers and business and product leads.
  • Map how teams perform work today and determine how AI can handle it and when people should step in.
  • Design and build AI workflows, including prompts, pipelines, and tooling, and monitor quality and drift after launch.
  • Build evaluation test sets, define accuracy and agreement metrics, and calibrate LLM judges against human reviewers.
  • Partner with engineers on architecture and tooling decisions, and assess estimates and scope.
  • Turn unclear guidelines into consistent reviewer criteria and work with subject matter experts on labeling guidelines and human-verified labels.
  • Run labeling with vendor teams and assess their work using agreement metrics and spot checks.
  • Agree on launch-quality criteria with product teams and provide evidence and recommendations for go/no-go decisions.
  • Build reusable evaluation and workflow components, document findings, and help other teams adopt them.
View Full Description & ApplyYou'll be redirected to the employer's site
$157,000 — $171,000 USD
Apply Now