I’m preparing an all-day hands-on workshop. My main topics are all about neural networks, but I have a few classical techniques too, including naive Bayes classification.

I start my explanation of naive Bayes by explaining the general idea (“the intuition”). In the image below, there are 40 data items. Each item has three predictor variables: job-type, eye-color, country. The variable to predict has three possible values (0, 1, 2). You can imagine the variable to predict is personality extraversion.

In the example, the goal is to predict the class of a person who is (barista, hazel, italy). Notice that the 30-item dataset doesn’t have anyone that matches the input. Also notice the predictor variables are all categorical. Naive Bayes works on counts/frequencies of items so all data must be categorical.

I ask attendees, “Suppose you know only that the person is a barista. What do you predict?” Attendees usually see that of the 5 barista items, 4 are class 0, so the best guess is class 0. Mathematically, P(0 | barista) = 4/5 = 0.80.

Then I ask, “Suppose you know only that the person has hazel eye color.” Of the 9 hazel items, 5 are class 0, so again the best guess is class 0. Mathematically, P(0 | hazel) = 5/9 = 0.55.

Finally, if you know only that the person is from Italy, 5 of 7 are class 1, so class 1 is the best guess. Mathematically, P(1 | italy) = 5/7 = 0.71.

At this point, the best overall guess is class 0 because two out of three guesses were class 0. The technique is called “naive” because it looks at each predictor column independently, and “Bayes” because it’s based on observed probabilities.

This is the intuition behind naive Bayes classification. The next step is to combine the observed probabilities in a logical way, but that’s another blog post.

A naive person is someone who is overly optimistic and believes in the good intentions of others. This doesn’t always work well with aliens.

Right: In “Independence Day” (1996) a large group of people went to the top of an office building below an alien spaceship, carrying “Welcome!” signs. The aliens were not impressed.