Chinnu: I Built an AI Robot Dog That Speaks Telugu

I built a robot dog. Not a demo that does one trick for the camera — a real interactive companion that sees the room, talks back in natural spoken Telugu, obeys voice commands, and gets happy when you pat its head. Its name is Chinnu, and when you ask who it is, it says “నా పేరు Chinnu బాబు. ”

Underneath, Chinnu is a SunFounder PiDog — twelve servos, a camera, touch sensors, an IMU, and a speaker — running on a Raspberry Pi 5. A GPT vision brain gives it conversation and a personality; a tiny model I trained myself gives it something rarer: the ability to understand Telugu voice commands completely offline, with no cloud and no API bill. This is the story of building it — and of the parts where the hardware fought back.

What Chinnu can do

The goal was a companion that feels alive, not a gadget you operate. So it leans on voice, vision, and touch instead of an app.

• Talks in natural Telugu. A camera-plus-GPT brain lets Chinnu read the scene, hold a conversation, remember what was said, and answer with a personality — in spoken Telugu, not robotic phrases.
• Wakes to its name. Say “చిన్నూ” and it perks up — “ఆ, చెప్పు! ” — and starts chatting. Say “ఆపు” or “stop” to drop back to command mode.
• Does tricks on voice. కూర్చో (sit), నిలబడు (stand), డాన్స్ (dance), షేక్ హ్యాండ్ (shake), మొరుగు (bark), గెంతు (jump) — and it still obeys a command thrown in mid-conversation.
• Has a pet mode. Pat its head and it reacts like a real dog — wags its tail, glows, gives a happy bark, and sometimes a warm Telugu line like “ఇంకా నిమురు బాబు! ” No microphone needed, so it always works.
• Sees and follows. Face tracking, body-follow, photo capture, and an optional camera web stream for when you want to see what it sees.

The hard part: understanding Telugu, offline

Talking to the cloud is easy. The interesting problem was the opposite: letting Chinnu understand its wake word and core commands with no internet at all — instantly, privately, and free — in my language and my voice. That meant training a small neural network of my own.

The technique is keyword spotting (KWS). I recorded each command — “చిన్నూ”, కూర్చో, నిలబడు, ఇలా రా, డాన్స్, ఆపు — about forty times in my own voice, plus “unknown” (other words and noise) and “silence” classes: 320 clips in all. Each one-second clip becomes a 40-band log-mel spectrogram — treating the audio like an image — and feeds a small DS-CNN, the depthwise-separable architecture from Google’s “Hello Edge” work, written in PyTorch.

How well it works

After training, the model is exported to an INT8 ONNX file — about 40 KB — and runs on the Raspberry Pi 5’s CPU with ONNX Runtime. The numbers held up better than I expected:

# Chinnu's offline ear: a 1-second clip -> 40-band log-mel -> INT8 ONNX model.
import onnxruntime as ort
import numpy as np

session = ort.InferenceSession("chinnu_kws_int8.onnx")  # ~40 KB, runs on the Pi 5 CPU

def listen(clip):
    feats = peak_normalize(log_mel(clip))    # same normalization as training
    logits = session.run(None, {"input": feats})[0]
    return LABELS[int(np.argmax(logits))]    # "chinnu" / "sit" / "dance" / ... in ~11 ms

Validation accuracy landed at 99.4%, held-out clips at 98% through ONNX, and inference takes about 11 milliseconds on the Pi — fast enough to feel instant, and entirely offline. The whole model and pipeline are open source at github. com/imdhileep/chinnu-kws.

You learn more about AI from one afternoon spent making a $60 board recognize your own voice than from a month of reading about it. Constraints teach; abundance distracts.

When the hardware fought back

• The mic was almost too quiet. The onboard microphone captured near noise levels even at full gain. The fix was per-clip peak-normalization, applied identically in training and inference — with silence left un-normalized so the background never amplifies into a false command.
• Training and inference didn’t match. Training recorded with arecord, but the first live loop used a different audio path; the feature distribution shifted and the model collapsed to always predicting one command. The fix was simply using the same capture path everywhere.
• An always-on loop broke the mic. Hammering the recorder every second put the ALSA audio device into a bad state and recording went dead. The fix was a triggered design — pat to record once, then classify — instead of listening non-stop.
• Python 3.13 had no TensorFlow. There were no wheels for it on Mac or the Pi, so the whole stack moved to PyTorch and ONNX Runtime — which turned out to be the more industry-standard, portfolio-friendly choice anyway.
• The camera had a purple tint. It turned out to be a NoIR camera, with no IR-cut filter, so software white balance couldn’t fully fix the cast. For vision and ML the tint doesn’t matter, so it stayed — a real fix is an IR-cut filter or a standard camera module.

What’s next

A few upgrades are queued: a proper Camera Module 3 (autofocus and correct daylight color) or an IR-cut filter for the current one; a USB microphone for more reliable recognition, since the onboard mic is the real bottleneck; and eventually distilling the cloud brain into a local model so even the conversation runs fully offline.

Chinnu isn’t a product, and that’s the point. It’s the most I’ve learned from a single build — wiring a model to the physical world, watching it fail in ways no tutorial warns you about, and fixing it. A $60 board, a free weekend, and a model trained on my own Telugu — and now there’s a little robot dog on my desk that looks up when I say చిన్నూ.