I wrote an article titled “Naive Bayes Classification Using C#” in the May 2022 edition of Microsoft Visual Studio Magazine. See https://visualstudiomagazine.com/articles/2022/05/02/naive-bayes-classification-csharp.aspx.

I present a complete demo program. The demo uses a set of 40 data items where each item consists of a person’s occupation (actor, baker, clerk or diver), eye color (green or hazel), country (Italy, Japan or Korea), and their personality optimism score (0, 1 or 2). You want to predict a person’s optimism score from their occupation, eye color and country. (This is an example of multiclass classification because the variable to predict, optimism, has three or more possible values.)

The first few data items look like:

actor  green  korea  1
baker  green  italy  0
diver  hazel  japan  0
diver  green  japan  1
clerk  hazel  japan  2
. . . 

The demo sets up an item to predict as (“baker”, “hazel”, “italy”). Next, the demo scans through the data and computes and displays smoothed (“add 1”) joint counts. For example, the 5 in the screenshot means that there are 4 bakers who have optimism class = 0.

The demo computes the raw, unsmoothed class counts as (19, 14, 7). This means there are 19 people with optimism class = 0, 14 people with class = 1, and 7 people with class = 2. Notice that 19 + 14 + 7 = 40, the number of data items.

The smoothed joint counts and the raw class counts are combined mathematically to produce evidence terms of (0.0027, 0.0013, 0.0021). These correspond to the likelihoods of class (0, 1, 2). Because the largest evidence value is at index [0], the prediction for the (“baker”, “hazel”, “italy”) person is class 0.

Evidence terms are somewhat difficult to interpret so the demo converts the three evidence terms to pseudo-probabilities: (0.4418, 0.2116, 0.3466). The values are not true mathematical probabilities but because they sum to 1.0 they can loosely be interpreted as probabilities. The largest probability is at index [0].

Naive Bayes classification is called “naive” because it analyzes each predictor column independently. This doesn’t take into account interactions between predictor values. For example, in the demo data, maybe clerks who have green eyes might have some special characteristics. The technique is “Bayesian” because the math is based on observed counts of data rather than some underlying theory.

The technique presented in the article works only with categorical data. There are other forms of naive Bayes classification that can handle numeric data. However, you must make assumptions about the math properties of the data, for example that the data has a normal (Gaussian) distribution with a certain mean and standard deviation.

Naive Bayes classification isn’t used as much as it used to be because techniques based on neural networks are much more powerful. However, neural techniques usually require lots of data. Naive Bayes classification often works well with small datasets.

You can find the complete C# demo code in the VSM article at the URL/link above.

In many of the comedy movies that I like, there is a naive character whose lack of sophistication leads to funny situations. Left: In “Dumb and Dumber To” (2014), buddies Lloyd (actor Jim Carrey) and Harry (Jeff Daniels) are orders of magnitude beyond naive but somehow always manage to emerge with success. Center: In “Stuck On You” (2003), conjoined twins Bob (Matt Damon) and Walt (Greg Kinnear) go to Hollywood so Walt can become an actor. The brothers are nice to everyone including their neighbor April (Eva Mendes) who is blissfully unaware of her surroundings. Right: In “Game Night” (2018) wife Annie (Rachel McAdams) is oblivious to danger when she and husband Max (Jason Bateman) are in a sketchy bar filled with not-very-nice criminals.