Author Archives: jamesdmccaffrey

Support Vector Regression Using From-Scratch-Python SVR with SGD Applied to the Diabetes Dataset

I write code almost every day. Like most skills, writing code is something that must be practiced. And besides that, I enjoy writing code. I had recently implemented a version of kernel support vector regression (SVR), from-scratch, using Python. So, … Continue reading

Posted in Machine Learning, Scikit | Leave a comment

Using Trimmed Kernel Ridge Regression to Approximate Support Vector Regression With C#

Bottom line: I refactored a C# implementation of trimmed kernel ridge regression to approximate support vector regression. The old version had ugly repeated code in two different SGD training phases, so I created a helper method to remove the code … Continue reading

Posted in Machine Learning | Leave a comment

Support Vector Regression with SGD Training From Scratch Using C#

The goal of a machine learning regression problem is to predict a single numeric value. For example, a bank might want to predict the maximum safe load amount for a customer, based on age, account balance, annual income, and so … Continue reading

Posted in Machine Learning | Leave a comment

Example of Anomaly Detection Using scikit IsolationForest

I came across a relatively obscure module in the scikit-learn library called IsolationForest. The module is used for anomaly detection. In a nutshell, if you apply a decision tree to a dataset, because of the way the branches are created, … Continue reading

Posted in Machine Learning, Scikit | Leave a comment

“The AI Coding Revolution: Productivity Boom and Employment Crisis” on the Pure AI Web Site

I contributed some technical content and opinions to an article titled “The AI Coding Revolution: Productivity Boom and Employment Crisis” on the Pure AI web site. See https://pureai.com/articles/2026/06/01/the-ai-coding-revolution-productivity-boom-and-employment-crisis.aspx. AI-assisted coding is easily the biggest technology change that I’ve seen in … Continue reading

Posted in Uncategorized | Leave a comment

Why I Never Use a Voting Regressor Model for Machine Learning Regression

Bottom line: I never use a voting regressor model because the number of meta parameters (which regression models to use) and model parameters (such as a model learning rate) is astronomically large and therefore unmanageable in practice. A voting regressor … Continue reading

Posted in Machine Learning, Scikit | Leave a comment

Implementing Ridge Regression from Scratch Python in the Scikit Style Using SAG Training

Ridge regression, aka linear ridge regression, is just linear regression with L2 regularization integrated. The idea is to discourage large model weights, which is a cause of model overfitting. Note: linear ridge regression is sometimes confused with kernel ridge regression, … Continue reading

Posted in Machine Learning, Scikit | Leave a comment

Refactoring My Trimmed Kernel Ridge Regression to Approximate Support Vector Regression Using C#

Bottom line: I refactored my C# implementation of trimmed kernel ridge regression to approximate support vector regression. The refactoring works fine, but the code is so long, there are multiple ways to organize it, and so I might have to … Continue reading

Posted in Machine Learning | Leave a comment

Some Thoughts About C# Program Design and the “dynamic” Keyword

One night, I just couldn’t sleep. So I thought about program design and the C# “dynamic” keyword. (My life would be considered sad by most people, but I’m happy with it). My standard machine learning regression system implemented in C# … Continue reading

Posted in Machine Learning | Leave a comment

“AdaBoost.R2 Regression Using C#” in Visual Studio Magazine

I wrote an article titled “AdaBoost.R2 Regression Using C#” in the June 2026 edition of Microsoft Visual Studio Magazine. See https://visualstudiomagazine.com/articles/2026/06/01/adaboost-r2-regression-using-csharp.aspx. The goal of a machine learning regression problem is to predict a single numeric value. For example, a bank … Continue reading

Posted in Machine Learning | Leave a comment