Tech • AI • Robotics
How to build with Gemini 3.5 Transcribe
7/10TL;DR
Google has released a live translation demo built on the Gemini API, LiveKit and Google Cloud Run that can broadcast one speaker’s audio in multiple languages with shared sessions per language.
KEY POINTS
How the system works
The application creates an event ID tied to a LiveKit room, where a presenter streams audio from a microphone or browser tab. Listeners join a web page, choose a target language and receive a translated audio feed plus captions. The setup is designed for live events where attendees can scan a QR code, connect on a phone and listen in their preferred language.
One translation session per language
The core optimization is to maintain at most one active translation session for each requested target language. If a first listener requests Chinese, the system opens a new Gemini Live Translate session for Chinese. If additional listeners request French after one French session already exists, they are attached to the existing stream instead of triggering another session, reducing duplicated processing and websocket overhead.
Automatic session cleanup
Translation sessions are closed when the last listener for a language disconnects. If one French listener leaves while another remains, the French websocket stays open; once the final listener exits, the session is shut down. This lifecycle management allows the service to conserve compute and keep only active language channels running.
Technology stack and deployment
The demo is open source and built with Next.js. It is deployed to Google Cloud Run, which supports the long-running websocket connections needed for live translation sessions. For local development, LiveKit can run in Docker, while production use can rely on LiveKit Cloud, which offers a hosted option with a free starting tier.
Credentials and media transport
The app requires a LiveKit API key, API secret and websocket URL to negotiate WebRTC connections. Audio is broadcast over WebRTC, while caption data is sent through WebRTC data channels. On Cloud Run, secrets such as API credentials are stored in Google Secret Manager rather than hard-coded in the application.
How Gemini handles translation
For each newly requested language, the server opens a websocket connection to the Gemini Live API. Incoming source audio is streamed as raw PCM data, and the model returns translated audio plus transcription output. The app publishes both the translated speech and text segments to participants subscribed to that language.
Interim and final captions
Captioning is handled in two stages. Interim transcription text can be shown immediately for low-latency feedback, while finalized text is later consolidated into complete paragraphs. Each transcription event includes the target language and segment metadata before being distributed to listeners in that language channel.
Current scaling limits
The present demo keeps event and translation state in memory, which means it is limited to one Cloud Run instance. Scaling beyond that would require moving session state and bridge management into an external database or shared backend. The design is therefore optimized as a practical demo rather than a horizontally scaled production architecture.
Capacity for smaller events
The current setup is presented as suitable for roughly 15 to 20 simultaneous languages and around 200 to 300 listeners, especially for small or medium-size events. CPU use rises because each language opens a long-running websocket session. For larger deployments, a more scalable approach would split languages into separate rooms and externalize state, allowing the service to grow far beyond the single-instance model.
CONCLUSION
The demo shows how Gemini Live Translate can be integrated into real-time event infrastructure with efficient per-language session sharing. Its current architecture is best suited to smaller multilingual events, but it outlines a clear path toward larger-scale live translation systems.
Explain this