The goal of a machine learning regression problem is to predict a single numeric value. For example, you might want to predict the price of a house in a particular area based on square footage, year built, number of bedrooms, and so on.

There are about a dozen metrics to evaluate a trained regression model, but most are rarely used. In my regression project scenarios, I use prediction mean squared error (MSE), prediction accuracy (to within a specified percentage of the true target value), and coefficient of determination (R2).

A model MSE value is a bit difficult to interpret because 1.) it heavily penalizes outlier predictions, and 2.) it depends on how the target y values are scaled. But MSE is useful because many regression techniques minimize MSE.

Accuracy is the most interpretable metric, but it requires an arbitrary closeness-to-actual-target parameter (like 5% or 10%). Plus accuracy is not very granular, and therefore can be misleading (such as when a model just barely predicts many y values within the specified percentage tolerance).

R2 is sort of a normalized accuracy (larger values are better). R2 doesn’t depend on how the target y values are scaled but R2 isn’t too easy to interpret. R2 is mostly useful to compare two entirely different regression models. R2 is the default “score” metric for scikit-learn regression models and classification models.

A metric called mean absolute scaled error (MASE) is common in the time series regression community, but MASE is rarely used in standard regression problem scenarios. MASE doesn’t heavily penalize outlier predictions like MSE does (because it uses absolute value of error instead of error-squared), and MASE doesn’t depend on how the target y data is scaled.

Note: The MASE implementation shown in this blog post works only for standard regression, not time series regression. The MASE version for TSR predicts a baseline value as the previous target y instead of the average of the target y values.

One Sunday morning, just to entertain myself, I decided to implement a MASE function using the C# language:

  public static double MASE(dynamic model,
    double[][] dataX, double[] dataY)
  {
    // Mean Absolute Scaled Error
    // error relative to baseline predict-mean target y
    // standard tabular data scenarios, not TSR scenarios

    // 1. compute model mean absolute error (MAE)
    int n = dataX.Length;
    double sum = 0.0;
    for (int i = 0; i "lt" n; ++i) // "lt" is less-than
    {
      double predY = model.Predict(dataX[i]);
      sum += Math.Abs(predY - dataY[i]);
    }
    double modelMAE = sum / n;

    // 2. compute mean of target y values
    double sumY = 0.0;
    for (int i = 0; i "lt" n; ++i)
      sumY += dataY[i];
    double meanY = sumY / n;

    // 3. compute baseline MAE (always predict mean y)
    double baseSum = 0.0;
    for (int i = 0; i "lt" n; ++i)
      baseSum += Math.Abs(meanY - dataY[i]);
    double baseMAE = baseSum / n;

    if (baseMAE "lt" 1.0e-12)
      return modelMAE;

    // 4. compute MASE as ratio of model to baseline
    double result = modelMAE / baseMAE;
    return result;
  }

The C# keyword “dynamic” allows the MASE() function to be used with any C# implementation of a regression model (LinearRegression, NearestNeighborsRegressor, QuadraticRegressor, etc.), as long as the model has a Predict() method. For example:

. . .
SomeRegressor model = new SomeRegressor(lrnRate, maxEpochs);
model.Train(trainX, trainY);
maseTrain = MASE(model, trainX, trainY);
Console.WriteLine("MASE for train data = " + maseTrain);
. . .

I put together a demo of MASE with one of my C# linear regression implementations. Output of a demo run:

Begin C# linear regression using MP pinv (QR-Householder)
 training with L2 regularization

Loading synthetic train (200) and test (40) data
Done

First three train X:
 -0.1660  0.4406 -0.9998 -0.3953 -0.7065
  0.0776 -0.1616  0.3704 -0.5911  0.7562
 -0.9452  0.3409 -0.1654  0.1174 -0.7192

First three train y:
  0.4840
  0.1568
  0.8054

Creating and training Linear Regression model
 using QR p-inverse with regularization
Setting L2 lamda = 1.0000
Done

Coefficients/weights:
-0.2618  0.0331  -0.0453  0.0353  -0.1132
Bias/constant: 0.3618

Evaluating model

Accuracy train (within 0.10) = 0.4700
Accuracy test (within 0.10) = 0.6500

MSE train = 0.0026
MSE test = 0.0019

MASE train = 0.2619
MASE test = 0.2402

End demo

Because MASE is an error metric, for MASE, lower values are better. The MASE metric can be as low as 0.0 (perfect predictions, no error — impossible in practice) up to 1.0 (the model predicts exactly like just predicting the average of the target y values for any input), or MASE can be greater than 1.0 which means the model predicts even worse than just predicting the average y for any input.

MASE has some nice properties. And MASE is very common in the time series regression community, but MASE is almost never used in the tabular data regression community. The main reason why R2 is nearly universal for tabular data regression is mostly because R2 is used by scikit-learn, and scikit-learn has a virtual monopoly on machine learning library code and usage patterns.

See https://jamesmccaffreyblog.com/2026/04/20/refactoring-my-demo-of-linear-regression-with-closed-form-training-using-csharp/ for one version of linear regression code, and the data.

I am a big fan of old science fiction movies from the 1950s and 1960s. The limited non-digital technology of the time meant that all movies were relatively expensive and difficult to create, and so there aren’t many sci-fi movies from those two decades. But starting in the 1990s, digital technology advanced to the point where a reasonably professional movie could be made for roughly $2.0 million dollars. This led to an explosion of sci-fi movies. Most of these movies are bad, but every now and then a low-budget sci-fi movie surprises me in a good way. Unlike a regression model where evaluation is objective, it’s difficult to compute a subjective evaluation metric for movies.

In “Ice Spiders” (2007), a group of young skiers on an isolated Utah mountain run into giant genetically altered spiders that are the product of a secret government lab. This movie has a nice combination of interesting plot, pretty good acting, humor-vs-scary, and excellent special effects. The movie doesn’t try to be anything more than it is. My grade = solid B.