Spam Filter
Junk messages use sneaky words. Sort a few messages by hand and watch a robot work out which words are the suspicious ones.
Here is a pile of messages. Tap Spam or Fine on a few of them. I will tell you which words you were looking at, and start learning too!
The message pile
The bar under each message is the robot's guess, how likely it is spam. Tap a message to peek inside the robot's head.
Challenges
What just happened?
- Every time you marked a message, the robot counted which words showed up in spam and which in fine messages.
- Words like FREE and CLICK got a high spam score. Words like HOMEWORK and LUNCH got a high fine score.
- To judge a message it adds up the scores of its words. Big total means probably spam.
- The test messages were hidden, so the accuracy line is honest, just like a real exam.
- Sneaky spellings and new topics fooled it, because it only knows the words it has seen. More labelled examples fix that.
For grown-ups: what's really going on
This is a multinomial Naive Bayes classifier on a bag of words: each message becomes a list of lowercase word counts and word order is ignored. "Naive" because it assumes words are independent given the class: P(spam | words) ∝ P(spam) · ∏ P(word | spam).
- It is trained incrementally: each label just updates counts, so it is exact and instant. Un-marking a message subtracts its counts.
- Laplace (add-one) smoothing:
P(w | c) = (count(w, c) + 1) / (total words in c + V), so a word never seen in one class does not zero the whole product.Vis the vocabulary size so far. - We work in log-odds:
log P(spam)/P(fine) + Σ log P(w|spam)/P(w|fine). Sums are stable and each word's term is its colour-coded score. Probability =1 / (1 + e−total). Words never seen are skipped. - Accuracy is measured on held-out messages. Misspellings and distribution shift (a new topic) break the word statistics; retraining on fresh labels repairs them.
- Accuracy hides the kinds of error. Precision = of messages flagged spam, how many really were (false alarms hurt this). Recall = of real spam, how many were caught (misses hurt this). The next lesson digs into both.
Further reading: Jurafsky and Martin, Speech and Language Processing, ch. 4 (Naive Bayes) · Paul Graham, A Plan for Spam (2002).