Building a Multilingual Voice AI Pipeline with Bhashini for Public Welfare Surveys

Bhashini is run by the Digital India Bhashini Division under MeitY as part of the National Language Translation Mission, and access starts with registration on ULCA, the Universal Language Contribution API platform that fronts it. An integrator registers, verifies by email, then generates credentials from the My Profile section: a userID and a ulcaApiKey. Bhashini currently caps an integrator at five keys, and registered app names must be lower-case and may contain underscores.

Inference is not a single call. Every request is a pair. The first is the pipeline config call, a POST to https://meity-auth.ulcacontrib.org/ulca/apis/v0/model/getModelsPipeline carrying userID and ulcaApiKey as headers, along with the task sequence you want and a pipeline ID. It answers with the serviceId and modelId that will serve each task for the language pair you asked for.

That same response carries the endpoint and credential for the second call, under pipelineInferenceAPIEndPoint: a callbackUrl, which for the Dhruva pipeline is https://dhruva-api.bhashini.gov.in/services/inference/pipeline, and an inferenceApiKey whose name is Authorization and whose value is the token to send. The compute call goes there.

The config response is a capability negotiation rather than just an auth handshake. It tells you which models can actually serve the language pair you asked for, and it is stable for a given pipeline and pair, which makes it a good candidate for a cache rather than something to run on the hot path of every request.

The compute call accepts a pipelineTasks array and Bhashini runs the tasks in sequence, feeding each stage's output into the next. A full speech-to-speech round trip is three entries: asr, translation, tts.

Each task carries its own config block. The asr task takes language.sourceLanguage, a serviceId, an audioFormat, and a samplingRate. The translation task takes language.sourceLanguage and language.targetLanguage plus its own serviceId. The tts task takes language.sourceLanguage, a serviceId, and a gender for the voice. Input audio goes in base64 under inputData.audio[].audioContent, and text-only pipelines use inputData.input[].source instead.

The language codes have to line up across the chain or the request fails: the ASR source and the translation source must match, and the translation target must match the TTS source. It reads as an obvious constraint and it is still a common source of confusing errors, because a mismatch surfaces at the stage that received bad input rather than at the stage that was misconfigured.

We tracked latency per stage rather than only end to end. With three models behind one HTTP call, an end-to-end number tells you that you missed two seconds but not which stage to fix, and the three stages do not degrade at the same rate under load.

Survey questions are authored once, in one language, and the pipeline delivers them in whichever language the respondent speaks. Bhashini's neural machine translation service handles the conversion in both directions: question out, answer back.

Model choice matters here, because Bhashini exposes several translation services with different coverage. bhashini/iiith/nmt-all from IIIT Hyderabad handles 34 languages bidirectionally, while AI4Bharat's IndicTrans v2, exposed as ai4bharat/indictrans-v2-all-gpu--t4, covers 18 including English, Hindi, Telugu, Tamil, and Sanskrit. Narrower models from IIT Bombay, CDAC, and AUKBC serve specific pairs.

Removing the interpreter removed the bottleneck, and it also removed a source of variance. A machine translation of a survey question is identical on every call, which is what makes responses comparable across states and languages. An interpreter paraphrasing under time pressure produces a slightly different question every time.

Most of what a survey says is fixed. The greeting, the question text, the instructions, and the closing do not change between respondents. Only the personalized fragments do, and there are far fewer of those than there is fixed content.

So we split the work. Fixed content is synthesized ahead of time through BullMQ queues backed by Redis, processed with FFmpeg, and stored in Azure Blob Storage, keyed by language. By the time a call connects, that audio already exists and is served straight from storage with no synthesis step at all. Dynamic content uses variable placeholders, is synthesized in real time, and is stitched into the fixed audio.

The effect is that the expensive part of TTS is paid once per language during pre-generation rather than once per call. For a survey that runs hundreds of thousands of times, that is the difference between a synthesis bill and a storage bill.

It also takes the least predictable component off the critical path. TTS latency varies with text length and with load. A blob fetch does not.

Bhashini does not expose one model per task. It exposes a catalog, and the services are grouped by language family as much as by task. On ASR, bhashini/ai4bharat/conformer-multilingual-asr covers all 22 scheduled languages, while other services are scoped to Dravidian languages, to Indo-Aryan languages, or to specific sets from IIT Madras. TTS is organized the same way: Bhashini/IITM/TTS spans 23 languages, AI4Bharat's Coqui-based services are split by family, and IISc's covers 11.

That means the serviceId is not a constant you set once. Sending every language to a single broad multilingual model is the simple option and it is not always the accurate one, because a family-specific model can outperform a general one on the languages it was built for. Routing per language means maintaining a table and revisiting it when Bhashini publishes new services.

This is the part of a Bhashini integration that is easiest to underestimate. The API surface is small and the payloads are simple. The work is in knowing which of the available models to send each language to, and confirming that with your own audio rather than trusting a catalog description.

A three-stage pipeline on an external service has three places to fail and one HTTP response to explain it. When TTS fails after ASR and translation have already succeeded, the useful output is not a generic error. It is the transcription and the translation you already have, plus a clear signal about which stage dropped.

So each stage is independently observable, with its own latency and its own failure mode, rather than the pipeline being treated as one opaque call. That is also what makes per-stage latency tracking worth the instrumentation: the same data that tells you where the two seconds went tells you what broke.

The dependency is worth stating plainly, because it is the main risk of building on Bhashini. It is an external, publicly funded service, and your pipeline is only as available as it is. Designing for partial failure is not defensive over-engineering here. It is the difference between a survey call that degrades and one that ends.