Author Archives: jamesdmccaffrey

Bottom line: Encoding categorical predictor variables for use in tree-based regression systems is a surprisingly complex topic. There are two main encoding schemes: 1. Always use simple ordinal encoding. 2. Use ordinal, or one-hot, or smoothed target encoding: a. binary … Continue reading →

As usual with complex problems, explaining what the problem is, is more difficult than explaining the solution. So bear with me. But the bottom line is that I tried an experiment that worked, but gave only a small improvement to … Continue reading →

I wrote an article titled “Neural Network Bistratum Regression Using C#” in the October 2026 edition of Microsoft Visual Studio Magazine. See https://visualstudiomagazine.com/articles/2026/10/01/neural-network-bistratum-regression-using-csharp.aspx. Neural network bistratum regression is a machine learning technique that uses a neural network with exactly two … Continue reading →

The form of a linear regression prediction equation is y’ = (w0 * x0) + (w1 * x1) + . . + (wn * xn) + b where y’ is the predicted value, the xi are predictor values, the wi … Continue reading →

One afternoon, I set out to come up with an algorithm for decision tree regression that is as simple as possible. I knew the result would predict poorly, but my ultimate goal is to use many of these super-simple decision … Continue reading →

As usual with complex problems, explaining what the problem is, is more difficult than explaining the solution. So bear with me. The goal of a machine learning regression problem is to predict a single numeric value. For example, a bank … Continue reading →

RBF networks for regression had a brief period of popularity in the 1990s. They are now rarely used because other techniques, notably, neural networks, kernel ridge regression, and gradient boost regression, are much easier to train and typically have better … Continue reading →