Author Archives: jamesdmccaffrey

The form of a linear regression prediction equation is y’ = (w0 * x0) + (w1 * x1) + . . + (wn * xn) + b where y’ is the predicted value, the xi are predictor values, the wi … Continue reading →

One afternoon, I set out to come up with an algorithm for decision tree regression that is as simple as possible. I knew the result would predict poorly, but my ultimate goal is to use many of these super-simple decision … Continue reading →

As usual with complex problems, explaining what the problem is, is more difficult than explaining the solution. So bear with me. The goal of a machine learning regression problem is to predict a single numeric value. For example, a bank … Continue reading →

RBF networks for regression had a brief period of popularity in the 1990s. They are now rarely used because other techniques, notably, neural networks, kernel ridge regression, and gradient boost regression, are much easier to train and typically have better … Continue reading →

One Saturday morning, after I walked my two dogs (Riley and Kevin) and a visiting dog (Llama), I figured I would take a look at the XBGoost (“extreme gradient boosting”) regression code library. The goal of a machine learning regression … Continue reading →

Arguably the most basic form of machine learning regression is linear regression. The liner regression prediction model is y’ = (x0 * w0) + (x1 * w1) + . . (xn * wn) + b. The y’ is the predicted … Continue reading →