Neural Networks

Meet one neuron

This is one tiny brain cell. It has one job: decide should we go play outside? Flip the switches on the right and watch it think!

Green wire = pushes toward YES Red wire = pushes toward NO Thicker wire = cares more

Teach it with examples

Here are 8 days and what a real kid decided. Can Robo's neuron learn to agree with all of them?

Tap a day to load it into the neuron. Press Learn! to let it fix its own knobs.

Mistakes
0/8
Lessons
0

Snapshots while learning

Press Learn! — snapshots appear after lesson 1, 5, 10 and when it's done. Tap one to travel back in time.

How it works

Signals come in

Each switch is a signal: on (1) or off (0). Sunny? Raining? Friends coming?

Each one has a knob

The knob says how much the neuron cares. Rain gets a big red knob because rain ruins everything!

Add it all up

The neuron adds up every signal × its knob, plus its starting mood.

Decide

Total bigger than zero? YES! Smaller? NO. That's it — that's one neuron.

Learn from oopsies

When it's wrong, it turns the knobs of the switches that were on a tiny bit — toward the right answer.

What just happened?

For grown-ups: what's really going on

This is a perceptron, the 1958 model by Frank Rosenblatt and the building block of every neural network.

  • Weighted sum: z = w₁x₁ + w₂x₂ + w₃x₃ + w₄x₄ + b. The knobs are the weights w; the starting mood is the bias b.
  • Activation: "Yes / No" is the step function y = z > 0 ? 1 : 0. "How sure?" is the sigmoid σ(z) = 1 / (1 + e^(−z)), which gives a smooth probability. Modern networks use smooth activations (sigmoid, tanh, ReLU) because they can be trained with gradients.
  • Learning rule: for each example, if the prediction ŷ is wrong, update wᵢ ← wᵢ + η(y − ŷ)xᵢ and b ← b + η(y − ŷ) with learning rate η = 0.5. Only weights for active inputs change (because xᵢ = 0 otherwise).
  • Guarantee: the perceptron convergence theorem says this rule finds a perfect answer in a finite number of steps whenever the data is linearly separable — i.e. one straight cut (a hyperplane) can split YES days from NO days. These 8 days are. Problems like XOR are not, which is why we stack many neurons into layers (see Brain Painter).
  • Further reading: Polo Club interactive explainers · TensorFlow Playground.