Senior/Principal Local LLM & Generative AI Platform Engineer
P
Parallel WirelessTelecommunications AI
US; RemoteFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- Typically 7+ years of hands-on experience in production software, ML platform, search, data, or infrastructure engineering, including meaningful recent experience shipping LLM-powered systems; exceptional candidates with equivalent depth are welcome.
- Required Skills
- DockerPythonKubernetesCI/CDLLM
Requirements
- BSc or MSc in Computer Science, Computer Engineering, Electrical Engineering, Data Science, or a related field, or equivalent practical experience.
- Typically 7+ years of hands-on experience in production software, ML platform, search, data, or infrastructure engineering, including meaningful recent experience shipping LLM-powered systems.
- Strong Python engineering skills and experience designing maintainable APIs, services, libraries, and data pipelines.
- Strong understanding of transformer-based language models and production inference, including tokenization, context management, batching, KV caching, parallelism, quantization, structured output, and tool calling.
- Demonstrated experience building production RAG or enterprise-search systems using embeddings, vector and/or lexical search, metadata filtering, reranking, source attribution, and retrieval evaluation.
- Experience defining task-specific LLM evaluations using representative datasets, baselines, domain-expert review, automated metrics, human feedback, error analysis, and regression thresholds.
- Experience deploying and operating containerized services on Linux using Docker and Kubernetes or an equivalent orchestration environment.
- Practical experience with GPU-backed model serving, performance profiling, capacity planning, monitoring, and reliability engineering.
- Strong knowledge of distributed-system fundamentals, authentication and authorization, API security, secrets handling, encryption, auditability, and data lifecycle controls.
- Experience with Git, automated testing, CI/CD, infrastructure as code, observability, and production incident response.
- Ability to lead an ambiguous cross-functional initiative, explain complex AI behavior clearly, and help teams ship safely on a shared platform.
Responsibilities
- Own the architecture and technical roadmap for a secure local LLM platform deployed in company-controlled infrastructure.
- Partner with engineering, product, support, IT, information security, legal, and domain experts to prioritize use cases and define measurable requirements.
- Build a modular inference and model-gateway layer with stable APIs, model routing, streaming, concurrency controls, and quotas.
- Evaluate open-weight language, code, embedding, reranking, and multimodal models against company-specific tasks.
- Optimize model serving across CPU, GPU, and accelerator resources while protecting output quality.
- Design and operate RAG and enterprise-search pipelines, including ingestion, retrieval, freshness, citations, and deletion.
- Enforce source-system permissions and integrate identity, SSO, role-based access control, secrets management, and audit logging.
- Establish evaluation datasets, automated evaluations, release gates, regression tests, canary releases, and rollback processes.
- Implement observability and safe tool-calling workflows with least-privilege access, bounded execution, and traceability.
- Build production operations and security foundations, integrate employee tools, and support adoption through documentation, training, and collaboration.
View Full Description & ApplyYou'll be redirected to the employer's site