Sometimes it’s useful to have terminology labels to help communicate topics in machine learning.

1. Supervised learning – All the training and test data items have class labels. Example: The Iris Dataset problem where all items are labeled as “setosa” (0), “versicolor” (1), or “virginica” (2). Example: The MNIST problem where each 28 x 28 image of a handwritten is labeled as ‘0’ or ‘1’ or . . .’9′. Example: The IMDB movie review problem where each review is labeled as negative = 0 (“It was a bad movie”) or positive = 1 (“Great movie”).

2. Unsupervised learning – None of the data items have class labels, or you ignore all labels. Example: You have a dataset of employee information (age, salary, etc.) and you perform k-means cluster analysis on the data and then look for patterns in the clustered data. Example: You create an autoencoder on security log data and look for data items that have large reconstruction error (anomalies).

3. Semi-Supervised learning – The data has only a few labeled items and many unlabeled data items. Example: A set of medical data where 100 patients have a disease and 100,000 patients have unknown status (maybe have the disease, maybe not). You need to create a prediction system that somehow uses information in the unlabeled data.

4. Self-Supervised learning – The data doesn’t have explicit labels but is modified to create artificial labels. Example: You have the entire several billion words of text of Wikipedia and remove one word from each sentence and create a model that predicts the missing word in each sentence. Example: You take MNIST image data and remove a 3 x 3 block from each and create a model that predicts the missing pixels. The idea is to create an abstract representation that can be fine-tuned to perform some “downstream task”.

There are many problems that don’t fit nicely into one of these four labels. An example is “one-shot learning” where you have labeled data but only one or two of each class. Example: You run a bank that has 10,000 customers. You have one signature on file for each customer. You want to create a system that examines a new signature and predicts if the signature is valid or not.

It’s important to remember that in most cases terminology is just slapping labels on ideas — the labels don’t have any inherent values by themselves. It’s the ideas that are important.

My first undergraduate college degree was in cognitive psychology from the University of California at Irvine. One of the things that mildly irritated me was that many areas of psychology are, to a large extent, just slapping labels on conditions, such as “introvert”, “extrovert”, “schizophrenic”. Here are three fictional introverts. Left: Oliver Twist from the book of the same name by Charles Dickens. There at least 14 different film versions of “Oliver Twist”. This image is from the 1968 musical version. Center: Waldo from the “Where’s Waldo?” series of books. Right: Bianca from the movie “Lars and the Real Girl” (2007).