One of the major problems facing people who are new to machine learning is that there are about one hundred fundamental ML techniques, and each of these fundamental techniques has many variations. That said, here’s my attempt to list the 12 most common techniques for binary classification.

A binary classification problem is one where the goal is to predict a variable that can be one of just two possible values. An example is predicting the sex of a person (male = 0, female = 1) based on predictors/features such as age, annual income, height, political leaning (conservative, moderate, liberal), and so on.

Some techniques can use only numeric predictors (such as age), some techniques can use only categorical predictors (such as political leaning), and some can handle mixed numeric and categorical predictors.

Logistic regression is somewhat unique in that there are many training/optimization algorithms that can be used.

Some of these 12 techniques naturally extend to handle multi-class classification, such as predicting political leaning from age, income, and so on. Some techniques can only handle multi-class classification using a hack called one-versus-all (OVA), which is generally a poor approach.

Some of the common techniques for training logistic regression models include stochastic gradient decent (SGD), iterated Newton-Raphson, L-BFGS optimization, simplex optimization, evolutionary optimization, and stochastic dual coordinate ascent (SDCA).

Three more or less random images from an Internet search for “binary classification art”. I classify all three as “good”.