Lead AI Engineer, LLM Systems

New
H
hermeneutic researchProprietary trading
Workable workplace: remote; Workable locations: Singapore. Hong Kong, Hong Kong, Hong Kong. United StatesFull-TimeLead
Salary not disclosed
Apply NowOpens the employer's application page

Job Details

Experience
7+ years of software or ML engineering, including 3+ years shipping LLM-backed systems to real users in production, and 2+ years leading other engineers while still shipping.
Required Skills
Python

Requirements

  • Have 7+ years of software or ML engineering experience.
  • Have 3+ years shipping LLM-backed systems to real users in production, with ownership of latency, cost, and quality.
  • Have 2+ years leading other engineers while continuing to ship.
  • Demonstrate a track record of building a function from nothing.
  • Bring command of inference economics, including tokens, caching, routing, structured outputs, tool use, and quantisation, with cost-per-request figures for systems you ran.
  • Use measurement discipline: instrument before making changes, evaluate against outcomes rather than model judges, and demonstrate an evaluation that found issues missed by a vibe check and a component retired based on data.
  • Have strong Python skills, including async concurrency.
  • Have experience with self-hosted serving such as vLLM or similar, adapters and GPU capacity planning, or an end-to-end provider migration.
  • Bring depth in vector and graph retrieval, with experience building and measuring retrieval and judgement about when it is not the right fix.
  • Use sound judgement when working with AI coding agents on a largely AI-written codebase, and retain ownership of shipped work.

Responsibilities

  • Maintain LLM-layer reliability through latency budgets, concurrency and queueing, degradation modes, structured-output validation, and provider failover.
  • Build evaluation tied to research outcomes, including offline sets from real queries, online experiments, regression gates, and calibrated model-generated confidence.
  • Control costs through per-workload attribution, difficulty-based model routing, caching, batching, and self-hosted serving and GPU-capacity economics.
  • Manage model strategy and lifecycle with pinned versions, replayable test batteries, rollback, provider diversification, and fine-tuning or adapters where useful.
  • Shape agent architecture using evidence, retiring components that do not earn their keep and improving tool use, memory, and retrieval where they improve outcomes.
  • Ground and trace outputs with citations, hallucination controls, audit trails, and defined boundaries for model assertions and suggestions.
  • Review and mentor engineers, then hire and lead an AI engineering group as the function grows.
View Full Description & ApplyYou'll be redirected to the employer's site
View details
Apply Now