Senior/Principal Local LLM & Generative AI Platform Engineer

P
Parallel WirelessTelecommunications AI
US; RemoteFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page

Job Details

Experience
Typically 7+ years of hands-on experience in production software, ML platform, search, data, or infrastructure engineering, including meaningful recent experience shipping LLM-powered systems; exceptional candidates with equivalent depth are welcome.
Required Skills
DockerPythonKubernetesCI/CDLLM

Requirements

  • BSc or MSc in Computer Science, Computer Engineering, Electrical Engineering, Data Science, or a related field, or equivalent practical experience.
  • Typically 7+ years of hands-on experience in production software, ML platform, search, data, or infrastructure engineering, including meaningful recent experience shipping LLM-powered systems.
  • Strong Python engineering skills and experience designing maintainable APIs, services, libraries, and data pipelines.
  • Strong understanding of transformer-based language models and production inference, including tokenization, context management, batching, KV caching, parallelism, quantization, structured output, and tool calling.
  • Demonstrated experience building production RAG or enterprise-search systems using embeddings, vector and/or lexical search, metadata filtering, reranking, source attribution, and retrieval evaluation.
  • Experience defining task-specific LLM evaluations using representative datasets, baselines, domain-expert review, automated metrics, human feedback, error analysis, and regression thresholds.
  • Experience deploying and operating containerized services on Linux using Docker and Kubernetes or an equivalent orchestration environment.
  • Practical experience with GPU-backed model serving, performance profiling, capacity planning, monitoring, and reliability engineering.
  • Strong knowledge of distributed-system fundamentals, authentication and authorization, API security, secrets handling, encryption, auditability, and data lifecycle controls.
  • Experience with Git, automated testing, CI/CD, infrastructure as code, observability, and production incident response.
  • Ability to lead an ambiguous cross-functional initiative, explain complex AI behavior clearly, and help teams ship safely on a shared platform.

Responsibilities

  • Own the architecture and technical roadmap for a secure local LLM platform deployed in company-controlled infrastructure.
  • Partner with engineering, product, support, IT, information security, legal, and domain experts to prioritize use cases and define measurable requirements.
  • Build a modular inference and model-gateway layer with stable APIs, model routing, streaming, concurrency controls, and quotas.
  • Evaluate open-weight language, code, embedding, reranking, and multimodal models against company-specific tasks.
  • Optimize model serving across CPU, GPU, and accelerator resources while protecting output quality.
  • Design and operate RAG and enterprise-search pipelines, including ingestion, retrieval, freshness, citations, and deletion.
  • Enforce source-system permissions and integrate identity, SSO, role-based access control, secrets management, and audit logging.
  • Establish evaluation datasets, automated evaluations, release gates, regression tests, canary releases, and rollback processes.
  • Implement observability and safe tool-calling workflows with least-privilege access, bounded execution, and traceability.
  • Build production operations and security foundations, integrate employee tools, and support adoption through documentation, training, and collaboration.
View Full Description & ApplyYou'll be redirected to the employer's site
View details
Apply Now