I wrote an article titled “Neural Regression Using PyTorch: Model Accuracy” in the March 2021 edition of the online Microsoft Visual studio Magazine. See https://visualstudiomagazine.com/articles/2021/03/12/pytorch-model-accuracy.aspx.

The goal of a regression problem is to predict a single numeric value. The example in the article is predicting the price of a house based on its area in square feet, air conditioning (yes or no), style (“art_deco,” “bungalow,” “colonial”) and local school (“johnson,” “kennedy,” “lincoln”). There are several classical statistics techniques for regression problems. Neural regression solves a regression problem using a neural network. The article is the fourth in a series of four articles that present a complete end-to-end production-quality example of neural regression using PyTorch.

In a classification problem, such as predicitng the style of a house (“art_deco,” “bungalow,” “colonial”), there is a built-in notion of accuracy. If you create a classification model and then run 100 data items through the model and you predict 80 styles correctly and 20 styles incorrectly, your model accuracy is 80%. But regression problems are different. If you predict the price of a house to be $350,000 and the true price is $360,000, is the prediction correct or not?

The most common approach for computing model accuracy for a regression problem is to calculate the number of predicted values that are within a specified percentage of the correct target values. For example, if a target price in the training data is $400,000 and the specified accuracy percentage is 10 percent, then a predicted price value between $360,000 and $440,000 would be logged as correct, but any other predicted price value would be logged as wrong.

There are two ways to compute model accuracy. One technique is to iterate through a Dataset object one item at a time so that you can examine each item. A less flexible but more efficient design is to compute accuracy on the entire Dataset using a set operation approach. My article shows both techniques.

def accuracy_quick(model, dataset, pct):
  # set approach
  # assumes model.eval()
  n = len(dataset)
  X = dataset[0:n][0]  # all predictor values
  Y = dataset[0:n][1]  # all target prices
  with T.no_grad():
    oupt = model(X)      # all computed prices

  max_deltas = T.abs(pct * Y)    # max allowable deltas
  abs_deltas = T.abs(oupt - Y)   # actual differences
  
  results = abs_deltas "less-than" max_deltas
  acc = T.sum(results, dim=0).item() / n  # dim not needed
  return acc

Creating and using neural networks using low-level code libraries such as PyTorch and TensorFlow gives you tremendous flexibility but can be challenging. The difficulty of using TensorFlow led to the creation of the Keras library, which is essentially a high-level wrapper to make TensorFlow easier to use. I have seen several similar efforts to create a high-level wrapper library for PyTorch, but none of these wrapper libraries is being widely used at this time.

The classic way to explain the difference between accuracy and precision is to use an image of an archery target with arrows in it. Here are three snapshots from Internet videos of archery examples where the arrow did not quite make it to the target (nobody was seriously injured). Left: The technique of pushing the arrow out from the bow does not work as well as standard techniques. Center: This girl’s attempt at a medieval fair archery booth had a geometry problem and did not fare well. Right: This Japanese girl probably should have used the eyes-open technique rather than the eyes-closed technique.