Sarvam AI logo
Sarvam AI

SDE2 - API Platform

BengaluruPosted 23 days agofulltime
Apply on Sarvam AI's career page

CV for this role

Rewrite your CV for what Sarvam AI asks for

This job is free to read. A CV tailored to it comes with Weekly and Monthly.

Tailor my CV

Job description preview

From Sarvam AI’s original posting

ABOUT THE ROLE Sarvam's model APIs (LLM, ASR, TTS, Vision, Translate) are how developers and enterprises ship products on our foundation models. Our OpenAI-compatible LLM APIs route every request across multiple model deployments, with streaming, failover, metering, billing, and rate-limiting all part of the same stack. Today it's FastAPI and Python; we're in the middle of rewriting it in Go. We're hiring a Software Engineer (SDE 2) to own core parts of this platform. The features. The reliability. The performance. This is an IC role. WHAT YOU'LL DO - Building and evolving the LLM API platform, from the moment a request hits the edge to the response that goes back out - The Python→Go rewrite: building services in Go alongside the existing Python stack, without compromising reliability - Streaming infrastructure: SSE for Chat Completions and Responses, long-lived connections, clean handling of cancellations and client disconnects - Routing and failover: weighted provider chains, retries with backoff, and latency-driven admission control across model deployments - Protocol compatibility: keeping the API faithful to the OpenAI spec across inference servers (vLLM, SGLang, Dynamo) that…

Open full posting on Sarvam AI’s site

Similar roles on Seoulstart

Browse all Global jobs