Category Archives: Machine Learning

Bottom line: I decided to implement random forest regression using Extra (“Extremely Randomized”) Tree learners. For my demo dataset, the technique worked slightly better, but not significantly better, than the standard random forest architecture that uses regular decision trees as … Continue reading →

Bottom line: I compared my from-scratch C# implementation of gradient boosting regression on training data with categorical predictor variables with the scikit-learn GradientBoostingRegression library module and got essentially the same results. The goal of a machine learning regression problem is … Continue reading →

Data smoothing is a classical statistics technique that removes noise from a dataset so that overall patterns and trends are visible. There are many algorithms for smoothing. I put together a demo of data smoothing using the Nadaraya-Watson (NW) kernel … Continue reading →

Suppose you have a tiny dataset of nine items like this: Sex Hand Age Height City Score (y) ——————————————- M right 24 tall Anaheim 0.60 F left 49 short Anaheim 0.70 M right 33 medium Boulder 0.20 M ambi 38 … Continue reading →

Bottom line: Encoding categorical predictor variables for use in tree-based regression systems is a surprisingly complex topic. There are two main encoding schemes: 1. Always use simple ordinal encoding. 2. Use ordinal, or one-hot, or smoothed target encoding: a. binary … Continue reading →

As usual with complex problems, explaining what the problem is, is more difficult than explaining the solution. So bear with me. But the bottom line is that I tried an experiment that worked, but gave only a small improvement to … Continue reading →

I wrote an article titled “Neural Network Bistratum Regression Using C#” in the October 2026 edition of Microsoft Visual Studio Magazine. See https://visualstudiomagazine.com/articles/2026/10/01/neural-network-bistratum-regression-using-csharp.aspx. Neural network bistratum regression is a machine learning technique that uses a neural network with exactly two … Continue reading →

The form of a linear regression prediction equation is y’ = (w0 * x0) + (w1 * x1) + . . + (wn * xn) + b where y’ is the predicted value, the xi are predictor values, the wi … Continue reading →

One afternoon, I set out to come up with an algorithm for decision tree regression that is as simple as possible. I knew the result would predict poorly, but my ultimate goal is to use many of these super-simple decision … Continue reading →

As usual with complex problems, explaining what the problem is, is more difficult than explaining the solution. So bear with me. The goal of a machine learning regression problem is to predict a single numeric value. For example, a bank … Continue reading →