Author Archives: jamesdmccaffrey

I like to write code every day. Writing code is a skill that must be practiced, and besides, I enjoy writing code. Over the course of my career, I saw countless colleagues move from an engineering role to an engineering-management … Continue reading →

One morning before work, I figured I’d implement kernel support vector regression (SVR), from scratch, using C#. It took me a bit longer than expected but I got a demo up and running. My implementation uses hard-wired RBF (radial basis … Continue reading →

A simple decision tree for regression will almost always overfit the training data where the model accuracy on the training data is near 100% but accuracy on new, previously unseen data is very poor. The idea of AdaBoost.R2 (adaptive boosting … Continue reading →

One day, I decided to implement kernel support vector regression (SVR), from scratch, using Python. One of the biggest downsides to SVR training is that almost all implementations use an algorithm called SMO (sequential minimal optimization) for training. SMO is … Continue reading →

Bottom line: I used AI to analyze my existing implementation of gradient boosting regression using C#, and it came up with some significant performance improvements. The goal of a machine learning regression problem is to predict a single numeric value. … Continue reading →