Tech • IA • Crypto
Gemma Playground: Robot Duck
TL;DR
Two small robotic ducks powered by Gemma 4 demonstrate on-device multimodal AI, combining vision, audio, and speech to interact with their environment in real time.
KEY POINTS
Compact Robots Running Advanced AI
Two miniature bipedal robots, known as Open Duck Mini v2, showcase how advanced AI models can operate locally on small devices. Each duck integrates Gemma 4 E2B, enabling real-time reasoning and interaction without relying on cloud processing.
Different Hardware Platforms
The robots run on distinct embedded systems: one uses a Raspberry Pi 5, while the other is powered by a NVIDIA Jetson Orin Nano. This comparison highlights how varying hardware capabilities can support on-device AI inference at different performance levels.
Multimodal Perception
Equipped with microphones, cameras, and speakers, the ducks process both visual and auditory inputs. This allows them to interpret their surroundings and respond contextually, demonstrating practical multimodal AI in a compact form factor.
Integrated Speech Pipeline
The system combines multiple components: Parakeet handles speech-to-text, Gemma 4 performs reasoning and language generation, and Cocoro converts responses back into speech. This creates a fully local conversational loop with minimal latency.
Expressive Interaction Design
The robots feature LED indicators and antenna-like elements to signal states such as listening or responding. An “attention mode” visually communicates when the system is actively processing input, enhancing user interaction.
Conversational Capabilities
In testing, the robot successfully explained the concept of a large language model and introduced itself, demonstrating coherent, responsive dialogue. The system identifies itself as “Autumn,” reflecting its embedded AI identity.
Open-Source Foundation
The project builds on publicly available designs developed by Antoine Piron, emphasizing accessibility and experimentation in robotics and AI integration.
Future Autonomous Behavior
Planned developments include enabling the ducks to walk, communicate with each other, and explore environments autonomously, pushing toward collaborative, mobile AI agents.
CONCLUSION
The project illustrates how powerful multimodal AI like Gemma 4 can be embedded into small, low-cost robots, signaling a shift toward fully autonomous, locally intelligent devices.
Full transcript
I am Autumn, a small bipedal robot duck. >> Today, I'm very happy to bring you those two tiny [music] ducks to showcase what Gemma 4 can do on device. They are open mini ducks version two by Antoine Piron on GitHub. And the idea was to bring the power of Gemma 4 to do on device inference and to make those ducks smarter. In terms of the technical stack, this duck runs Gemma 4 E2B on light RT on a Raspberry Pi 5. And this duck runs Gemma 4 E2B on a Jetson Orin Nano. They both have microphone, cameras, and speakers. And through multimodal inputs of Gemma 4, they're able to process and understand their environment like never before. Let's see what uh duck Gemma with a Raspberry Pi 5 can do. So, when we wait for the system to boot, just going to describe some of the features that the this project online built. They built antennas for expressiveness. There are some LEDs just to show some expressions. And a speaker next to it. And then it enters attention mode, which is this [music] loop that just shows some expressiveness and uh the fact that Gemma is now listening. And we're going to try that on the other duck. And it will send you some love also on the booting phase if everything goes right. Hey Gemma, explain me what a large language model is. We're using Parakeet to do speech to text. Gemma 4 will do the inference and we'll use Cocoro to do text to speech. >> A large language model is a complex AI designed to understand and generate human-like text. I am here to assist you with tasks and interactions. >> Introduce yourself. Tell me more about you. >> I am Autumn, a small duck robot's brain. >> Very snappy. [music] Uh Autumn is ODM, so Open Duck Mini. Uh so, that's why it calls itself ODM, which I think is pretty cool. So, this is a great way to bring intelligence onto those ducks. The next step is for them to start walking, to see each other, talk to each other, and maybe go autonomously explore the world on their own.