Tech • AI • Robotics
Build a Live Translation Broadcast App with the Gemini Live API and LiveKit
7/10TL;DR
Google has released the Gemini 3.5 live translation model through the Gemini API, alongside an open-source demo that streams spoken audio into multiple languages in real time using LiveKit, Next.js, and Google Cloud Run.
KEY POINTS
Real-time multilingual broadcast
The demo is designed for live events where one speaker can be heard by listeners in different languages at the same time. Audience members join through a page, choose a target language such as Chinese or French, and receive translated audio and captions on their phones, making it practical for presentations and small multilingual gatherings.
One translation session per language
The system creates a single live translation session for each requested target language, then attaches additional listeners to that same stream. If two people select French, both are subscribed to one existing French session rather than triggering duplicate processing, which reduces compute load and avoids unnecessary API usage.
Automatic session cleanup
Sessions remain active only while at least one listener is connected. When a participant leaves, the service checks whether others are still using that language stream; if not, the related WebSocket is closed, allowing the app to conserve resources during low traffic or after an event ends.
Architecture built on WebRTC and WebSockets
The app uses a LiveKit room as the transport layer for the event, with audio sent through WebRTC and captions delivered over WebRTC data channels. A translation bridge opens a WebSocket connection to the Gemini API, sends raw PCM audio, requests translated speech plus transcript output, and republishes the translated audio and text back to participants in the room.
Transcript handling and live captions
The service publishes interim transcription segments immediately so listeners can follow along with low latency. When the model returns a final segment, the text is consolidated into a complete paragraph, with metadata such as the target language and segment ID passed to the connected clients.
Open-source implementation
The application is available as an open-source project and is built with Next.js. For local development, LiveKit can run in Docker, while a hosted LiveKit Cloud option is positioned as the simpler starting point, requiring an API key, secret, and WebSocket URL.
Cloud Run deployment model
Deployment to Google Cloud Run is container-based through a Dockerfile that installs dependencies and serves the app. Secrets including API credentials are stored in Google Secret Manager, while the service can be configured for public access and scale down to zero when idle, avoiding charges when there is no traffic.
Current scaling limits
The demo keeps translation state in memory for a single event, which means it currently runs with a maximum of one Cloud Run instance. To scale beyond that, state management would need to move into a database or other shared store so multiple instances could coordinate translation bridges safely.
Practical capacity
The main bottleneck is the number of simultaneous target languages, since each language requires its own long-lived WebSocket session. The setup is described as suitable for roughly 10 to 15 active languages, and around 200 to 300 listeners for smaller events, with broader scaling possible if languages are split into separate rooms or services.
CONCLUSION
The release turns Gemini 3.5 live translation into a practical building block for custom multilingual event apps, but the current reference design is aimed at modest deployments rather than large-scale broadcast infrastructure.
Explain this