GenAI Platform Engineer – LLM Gateway
New
J
JobgetherGenAI infrastructure
Fully remote working arrangement from IndiaFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Languages
- Excellent written and verbal communication skills in English
- Experience
- At least 9 years of professional software engineering experience, including a minimum of 3 years developing and delivering production-grade GenAI or LLM-based applications and platforms.
- Required Skills
- AWSDockerPythonFastAPIRESTful APIsGitHub Actions
Requirements
- Bachelor's or Master's degree in Computer Science, Computer Applications, Engineering, or a related discipline, or equivalent practical experience.
- At least 9 years of professional software engineering experience, including a minimum of 3 years developing and delivering production-grade GenAI or LLM-based applications and platforms.
- Strong proficiency in Python backend development, including asynchronous programming, FastAPI or similar frameworks, REST APIs, and streaming responses.
- Hands-on experience integrating LLM provider APIs such as OpenAI, Azure OpenAI, and AWS Bedrock.
- Experience implementing tool calling, function calling, and integrations across multiple LLM providers.
- Experience deploying and operating services on AWS using containerization technologies and CI/CD pipelines.
- Practical knowledge of Docker, GitHub Actions, and automated testing in software delivery workflows.
- Strong understanding of backend architecture, scalable services, API integrations, and production software engineering practices.
- Ability to troubleshoot complex production issues, conduct root-cause analysis, and implement long-term solutions.
- Excellent written and verbal communication skills in English and ability to collaborate across globally distributed teams.
- Experience with AI/LLM gateways or API gateways, model routing, API key management, access controls, or cost tracking is preferred.
- Knowledge of agentic AI architectures and Model Context Protocol (MCP), and experience with LangGraph, LangChain, or similar frameworks, are preferred.
Responsibilities
- Design, develop, and operate enterprise LLM Gateway features primarily using Python.
- Build and maintain multi-provider model integrations, intelligent routing, failover, API key management, and access controls.
- Develop usage metering, budget management, cost transparency, and chargeback reporting features.
- Integrate agentic AI capabilities and develop MCP servers, cross-provider tool calling, and agent-to-agent connectivity.
- Maintain open-source platform foundations, assess upstream releases, manage custom extensions, and deliver upgrades without regressions.
- Build automated quality gates in CI/CD, including unit, integration, contract, and performance testing.
- Implement observability using metrics, distributed tracing, and logging tools such as OpenTelemetry and Langfuse.
- Monitor platform performance and resolve latency, reliability, error-rate, and infrastructure-cost issues.
- Participate in production support and on-call rotations, investigate root causes, resolve incidents, and conduct blameless postmortems.
- Support internal engineering teams with technical guides, runbooks, code samples, reusable AI agent skills, and onboarding.
View Full Description & ApplyYou'll be redirected to the employer's site