Author Archives: jamesdmccaffrey

Bottom line: I decided to implement AdaBoost.R2 regression using Extra (“Extremely Randomized”) Trees learners. Bottom line: For my demo dataset, the technique worked better than the standard architecture that uses regular decision trees as the learners. As is often the … Continue reading →

One Sunday evening, I was sitting in my living room. I decided to implement a regression system (to predict a single numeric value), using a neural network with exactly two hidden layers, from scratch, using Python with NumPy. The effort … Continue reading →

I like to write code every day. Writing code is a skill that must be practiced, and I enjoy writing code. Over the course of my career, I saw countless colleagues move from an engineering role to an engineering-management role, … Continue reading →

The goal of a machine learning regression problem is to predict a single numeric value. Quadratic regression is an enhanced form of basic linear regression. One of several ways to train a quadratic regression model is to use relaxed Moore-Penrose … Continue reading →

AdaBoost.R2 (“adaptive boosting regression, version 2”) is a technique to predict a single numeric value. AdaBoost works by creating a collection of simple decision trees. The final prediction is the average of all the tree predictions. Each tree is trained … Continue reading →

In general, when training a machine learning regression model using stochastic gradient descent, I do not use early-exit from the training loop. Any type of early-exit requires some sort of parameter that must be determined by trial and error, which … Continue reading →

Machine learning quadratic regression is a technique to predict a single numeric value. There are many ways to train a quadratic regression model. One training approach that is fast, reliable, and scales to large datasets — but is very complicated … Continue reading →