Teach the Ear
Computers can't hear like you. They turn sound into a picture, then learn what each kind of sound's picture looks like.
Press a sound button (or turn on your microphone) and watch the sound turn into a picture! Low sounds at the bottom, high sounds at the top.
Sound picture (spectrogram) quiet
Robo's quiz score as it hears more examples
Checkpoints: what Robo's sound memories looked like
Saved as Robo collects examples.
How it works
Sound becomes a picture
The computer splits sound into low, middle and high parts many times a second. Loud parts glow brightly.
Every sound has a shape
A whistle is a thin line up high. A hum is a stripe down low. A clap is a short splash everywhere.
Collect examples
Each recording is saved as a little picture in its box. The big picture is their average.
Find the closest
For a new sound, Robo finds the saved pictures that look most alike, and they vote.
What just happened?
- Sound turned into a picture called a spectrogram: up = high notes, bright = loud.
- Each example you recorded became a small picture saved in a box.
- With only 1 example Robo often guessed wrong. With more, its quiz score went up. You saw it on the chart.
- To guess, Robo looked for the most similar saved pictures and let them vote.
- Voice assistants do the same first step: turn sound into a picture, then a model learns from lots of examples.
For grown-ups: what's really going on
- Spectrogram: the Web Audio
AnalyserNoderuns an FFT (2048 samples) about 60× per second. The magnitudes are pooled into 32 log-spaced bands (80 Hz–8 kHz), roughly like a mel spectrogram. - Segmentation: a simple energy-based onset detector (adaptive noise floor) captures ~0.75 s after each onset, so every example is aligned to its start.
- Features: the clip is pooled to a 32 × 12 time-frequency image, normalised to its own peak (so volume matters less), plus its average spectrum.
- Classifier: k-nearest neighbours (k = 3, cosine distance, distance-weighted votes). The quiz score is leave-one-out accuracy: each example is classified by all the others.
- Real systems (keyword spotting, speech recognition like Whisper) feed log-mel spectrograms into CNNs or transformers trained on thousands of hours of audio. Teachable Machine's audio mode uses a pretrained network's embeddings instead of raw pixels, which generalises better than this.