Lead Engineer, AI Platform

New

JobgetherAI software

Based in SpainFull-TimeLead

SalaryAnnual cash compensation of $170,000 USD, benchmarked to U.S. compensation levels regardless of location.

Apply NowOpens the employer's application page

Job Details

Languages: Excellent English proficiency in spoken, written, and reading communication, equivalent to CEFR C2 / ILR 5.
Experience: 7+ years of experience building and shipping production software
Required Skills: PythonRuby on RailsCI/CD

Have 7+ years of experience building and shipping production software.
Have experience with LLM-powered agents capable of taking real actions within products, ideally.
Have experience with complex, tool-using AI systems involving multiple tools, planning, orchestration, or sub-agents.
Demonstrate shipped software and explain how its effectiveness and reliability were measured.
Have experience with Ruby on Rails and/or Python, and be able to become productive quickly in unfamiliar technologies.
Have experience building evaluation or observability infrastructure for ML/AI systems, including evaluation pipelines, scorers, dashboards, or CI/CD systems.
Be familiar with evaluation frameworks such as Braintrust, LangSmith, or similar tools.
Have experience designing datasets, annotation workflows, or labeling pipelines for machine learning or AI evaluation.
Use experimentation and empirical results to guide technical decisions.
Be comfortable working in a fast-paced environment with ambiguity and changing technical requirements.
Have technical leadership and people-management capabilities, balancing team leadership with hands-on engineering.
Have excellent spoken, written, and reading English proficiency equivalent to CEFR C2 / ILR 5.

Design, build, and own evaluation infrastructure, including CI/CD pipelines, scorers, datasets, and systems for assessing AI agents.
Develop observability and diagnostic capabilities to locate quality issues across planning, execution, tool selection, and agent trajectories.
Investigate failures across AI workflows and turn findings into prototypes, improvements, or priorities for AI engineering teams.
Build datasets using human annotation, AI-generated examples, and simulated conversations.
Develop experimentation frameworks for prompts, models, and agent harnesses, including evaluating new and open-source models against production baselines.
Identify opportunities to improve AI system cost and latency through model selection, caching, routing, and other strategies.
Set technical direction and manage day-to-day priorities for the AI Quality engineering team while contributing hands-on.
Partner with AI Core engineering teams to measure product changes and their effect on quality.
Establish engineering practices and evaluation approaches for production AI systems.

View Full Description & ApplyYou'll be redirected to the employer's site