I Use AI to Improve My Bagging Tree Regression System Using C#

Posted on August 11, 2026 by jamesdmccaffrey

A simple decision tree for regression will almost always overfit the training data where the model accuracy on the training data is near 100% but accuracy on new, previously unseen data is very poor.

The simple idea of bagging tree regression (“bootstrap aggregation”) is to create a collection/ensemble of many simple decision trees, where each tree is trained on a different subset of the rows of the training data.

For example, if the source set of training data has 200 rows, you could create 50 simple trees, where each tree is trained on 180 randomly selected rows. The selection is done “with replacement” so for a random subset of training data, some rows will be used more than once and some rows won’t be used at all.

After all the trees are trained, a prediction for an input vector x is just the average of the predictions of the trees in the collection. A very simple and crude idea that sometimes works well, and sometimes doesn’t work very well.

Bagging tree regression is a specific kind of random forest regression. In bagging tree regression, the subsets of the training data use all columns. In random forest regression, a random subset of columns is used.

Even though bagging tree regression is a type of random forest regression, the two techniques are usually considered distinct.

Bagging tree regression was first described in 1994 paper by a fellow named Leo Breiman, even though the idea had been used for at least a decade before. Then in 1981, Breiman slightly modified bagging tree regression (randomly select columns) and called the new technique random forest regression, even though the idea had been used for years before.

So, even though bagging tree regression is just a special case of random forest regression, the term bagging tree regression had been in use for seven years so there was no getting rid of that term.

Bagging tree regression evolved into random forest regression, which evolved into AdaBoost.R and AdaBoost.R2 regression, which evolved into gradient boosting regression — a fascinating (but very long) story. Maybe I’ll post that story some day.

I used AI to optimize my bagging tree regression system. AI was remarkably accurate. I made a demo. The demo data is synthetic. It was generated by a 5-10-1 neural network with random weight and bias values. The data looks like:

-0.1660,  0.4406, -0.9998, -0.3953, -0.7065,  0.4840
 0.0776, -0.1616,  0.3704, -0.5911,  0.7562,  0.1568
-0.9452,  0.3409, -0.1654,  0.1174, -0.7192,  0.8054
 0.9365, -0.3732,  0.3846,  0.7528,  0.7892,  0.1345
. . .

The first five values on each line are predictors. The last value on each line is the target y value to predict. There are 200 training items and 40 test items. Here’s the output of a demo:

Begin C# Bagging Tree regression demo

Loading synthetic train (200) and test (40) data
Done

First three train X:
 -0.1660  0.4406 -0.9998 -0.3953 -0.7065
  0.0776 -0.1616  0.3704 -0.5911  0.7562
 -0.9452  0.3409 -0.1654  0.1174 -0.7192

First three train y:
  0.4840
  0.1568
  0.8054

Setting nTrees = 100
Setting maxDepth = 6
Setting minSamples = 2
Setting minLeaf = 1
Setting nRows = 150

Creating and training BaggingTreeRegression model
Done

Accuracy train (within 0.10) = 0.8050
Accuracy test (within 0.10) = 0.5500

MSE train = 0.0006
MSE test = 0.0017

Predicting for x =
  -0.1660   0.4406  -0.9998  -0.3953  -0.7065
Predicted y = 0.4802

End demo

Notice that the model still overfits quite a bit. This is because the synthetic data is more or less spread evenly. Because of this weakness, the more powerful gradient boosting regression model is often used instead of bagging tree regression or random forest regression. That said however, for some problem scenarios, bagging tree and random forest regression work quite well.


Identifying ugly code and ugly regression models is a subjective task, but after 50+ years of writing code, I think I’m pretty decent at doing so. On the other hand, there are many domains where I am utterly unable to distinguish good from ugly.

I entered “ugly fashion model” into Google image search and got these three photos (among hundreds).

Left: This fashion model doesn’t look ugly to my eye, but apparently there’s something seriously wrong with her looks, or possibly the clothes she’s wearing.

Center: Another one that looks fine to me. Maybe she is judged too short for a fashion model, or maybe her face looks too ordinarily-pretty instead of model-pretty.

Right: OK, I don’t think there’s much disagreement here. It looks like she carries her dinner plate around her neck in order to take advantage of any feeding opportunities. This woman should avoid the Macy’s Thanksgiving Day parade — they might think she got loose and try to tie her down.

Demo program. Replace “lt” (less than), “gt”, “lte”, “gte” with Boolean operator symbols (my blog editor often chokes on symbols).

using System;
using System.IO;
using System.Collections.Generic;

namespace BaggingTreeRegression
{
  internal class BaggingTreeRegressionProgram
  {
    static void Main(string[] args)
    {
      Console.WriteLine("\nBegin C# Bagging Tree" +
        " regression demo ");

      // 1. load data
      Console.WriteLine("\nLoading synthetic train (200)" +
        " and test (40) data");
      string trainFile =
        "..\\..\\..\\Data\\synthetic_train_200.txt";
      int[] colsX = new int[] { 0, 1, 2, 3, 4 };
      double[][] trainX =
        MatLoad(trainFile, colsX, ',', "#");
      double[] trainY =
        MatToVec(MatLoad(trainFile,
        new int[] { 5 }, ',', "#"));

      string testFile =
        "..\\..\\..\\Data\\synthetic_test_40.txt";
      double[][] testX =
        MatLoad(testFile, colsX, ',', "#");
      double[] testY =
        MatToVec(MatLoad(testFile,
        new int[] { 5 }, ',', "#"));
      Console.WriteLine("Done ");

      Console.WriteLine("\nFirst three train X: ");
      for (int i = 0; i "lt" 3; ++i)
        VecShow(trainX[i], 4, 8);

      Console.WriteLine("\nFirst three train y: ");
      for (int i = 0; i "lt" 3; ++i)
        Console.WriteLine(trainY[i].ToString("F4").
          PadLeft(8));

      // 2. create and train model
      int nTrees = 100;
      int maxDepth = 6;
      int minSamples = 2;
      int minLeaf = 1;
      int numSplitCols = -1;  // use all
      int nRows = 150;  // train data for each tree

      Console.WriteLine("\nSetting nTrees = " + nTrees);
      Console.WriteLine("Setting maxDepth = " + maxDepth);
      Console.WriteLine("Setting minSamples = " + minSamples);
      Console.WriteLine("Setting minLeaf = " + minLeaf);
      Console.WriteLine("Setting nRows = " + nRows);
      // bagging tree regression always uses all columns
      // Console.WriteLine("(Using all columns) ");

      Console.WriteLine("\nCreating and training" +
        " BaggingTreeRegression model ");
      BaggingTreeRegressor btr =
        new BaggingTreeRegressor(nTrees, nRows, maxDepth,
        minSamples, minLeaf, numSplitCols, seed: 0);
      btr.Train(trainX, trainY);
      Console.WriteLine("Done ");

      // 3. evaluate model
      double accTrain = btr.Accuracy(trainX, trainY, 0.10);
      Console.WriteLine("\nAccuracy train (within 0.10) = " +
        accTrain.ToString("F4"));
      double accTest = btr.Accuracy(testX, testY, 0.10);
      Console.WriteLine("Accuracy test (within 0.10) = " +
        accTest.ToString("F4"));

      double mseTrain = btr.MSE(trainX, trainY);
      Console.WriteLine("\nMSE train = " +
        mseTrain.ToString("F4"));
      double mseTest = btr.MSE(testX, testY);
      Console.WriteLine("MSE test = " +
        mseTest.ToString("F4"));

      // 4. use model
      double[] x = trainX[0];
      Console.WriteLine("\nPredicting for x = ");
      VecShow(x, 4, 9);
      double predY = btr.Predict(x);
      Console.WriteLine("Predicted y = " +
        predY.ToString("F4"));

      Console.WriteLine("\nEnd demo ");
      Console.ReadLine();
    } // Main()

    // ------------------------------------------------------
    // helpers for Main()
    // ------------------------------------------------------

    static double[][] MatLoad(string fn, int[] usecols,
      char sep, string comment)
    {
      List"lt"double[]"gt" result = 
        new List"lt"double[]"gt"();
      string line = "";
      FileStream ifs = new FileStream(fn, FileMode.Open);
      StreamReader sr = new StreamReader(ifs);
      while ((line = sr.ReadLine()) != null)
      {
        if (line.StartsWith(comment) == true)
          continue;
        string[] tokens = line.Split(sep);
        List"lt"double"gt" lst = new List"lt"double"gt"();
        for (int j = 0; j "lt" usecols.Length; ++j)
          lst.Add(double.Parse(tokens[usecols[j]]));
        double[] row = lst.ToArray();
        result.Add(row);
      }
      sr.Close(); ifs.Close();
      return result.ToArray();
    }

    static double[] MatToVec(double[][] mat)
    {
      int nRows = mat.Length;
      int nCols = mat[0].Length;
      double[] result = new double[nRows * nCols];
      int k = 0;
      for (int i = 0; i "lt" nRows; ++i)
        for (int j = 0; j "lt" nCols; ++j)
          result[k++] = mat[i][j];
      return result;
    }

    static void VecShow(double[] vec, int dec, int wid)
    {
      for (int i = 0; i "lt" vec.Length; ++i)
        Console.Write(vec[i].ToString("F" + dec).
          PadLeft(wid));
      Console.WriteLine("");
    }

  } // class Program

  // ========================================================

  public class BaggingTreeRegressor
  {
    public int nTrees;
    public int maxDepth;
    public int minSamples;
    public int minLeaf;
    public int numSplitCols;
    public int nRows;
    public List"lt"DecisionTreeRegressor"gt" trees;
    public Random rnd;

    public BaggingTreeRegressor(int nTrees, int nRows,
      int maxDepth, int minSamples, int minLeaf,
      int numSplitCols, int seed = 0)
    {
      this.nTrees = nTrees;
      this.nRows = nRows;  // num train rows, each tree
      this.maxDepth = maxDepth;
      this.minSamples = minSamples;
      this.minLeaf = minLeaf;
      this.numSplitCols = numSplitCols;
      this.trees = new List"lt"DecisionTreeRegressor"gt"();
      this.rnd = new Random(seed);
    }

    public void Train(double[][] trainX, double[] trainY)
    {
      int totalRows = trainX.Length;
      int nCols = trainX[0].Length;

      // reusable buffers for all trees
      double[][] subsetX = new double[this.nRows][];
      for (int i = 0; i "lt" this.nRows; ++i)
        subsetX[i] = new double[nCols];
      
      double[] subsetY = new double[this.nRows];

      // train each tree using buffers
      for (int t = 0; t "lt" this.nTrees; ++t)
      {
        // populate the buffers
        for (int r = 0; r "lt" this.nRows; ++r)
        {
          int rndIndex = this.rnd.Next(0, totalRows);

          // Array.Copy() uses low-level native memory moves
          // much faster than a nested C# for-loop
          Array.Copy(trainX[rndIndex], subsetX[r], nCols);
          subsetY[r] = trainY[rndIndex];
        }

         DecisionTreeRegressor dtr = new DecisionTreeRegressor(
          this.maxDepth,
          this.minSamples,
          this.minLeaf,
          this.numSplitCols,
          false,
          this.rnd.Next(0, 1_000_000)
        );

        dtr.Train(subsetX, subsetY); // safe to reuse
        this.trees.Add(dtr);
      }
    }

    // ------------------------------------------------------

    public double Predict(double[] x)
    {
      double sum = 0.0;
      for (int t = 0; t "lt" this.nTrees; ++t)
        sum += this.trees[t].Predict(x);
      return sum / this.nTrees;
    }

    // ------------------------------------------------------

    public double Accuracy(double[][] dataX, double[] dataY,
      double pctClose)
    {
      int numCorrect = 0; int numWrong = 0;
      for (int i = 0; i "lt" dataX.Length; ++i)
      {
        double actualY = dataY[i];
        double predY = Predict(dataX[i]);
        if (Math.Abs(predY - actualY) "lt"
          (pctClose * Math.Abs(actualY)))
          ++numCorrect;
        else
          ++numWrong;
      }
      return (numCorrect * 1.0) / (numWrong + numCorrect);
    }

    // ------------------------------------------------------

    public double MSE(double[][] dataX, double[] dataY)
    {
      int n = dataX.Length;
      double sum = 0.0;
      for (int i = 0; i "lt" n; ++i)
      {
        double actualY = dataY[i];
        double predY = this.Predict(dataX[i]);
        sum += (actualY - predY) * (actualY - predY);
      }
      return sum / n;
    }

  } // class BaggingTreeRegressor

  // ========================================================

  public class DecisionTreeRegressor
  {
    public int maxDepth;
    public int minSamples;
    public int minLeaf;
    public int numSplitCols;
    public List"lt"Node"gt" tree;
    public Random rnd;
    public bool saveRows;
    public double[][] trainX;
    public double[] trainY;

    // ............................................

    public class Node
    {
      public int id;
      public int colIdx;
      public double thresh;
      public int left;
      public int right;
      public double value;
      public bool isLeaf;
      public List"lt"int"gt" rows;

      public Node()
      {
        this.id = -1;
        this.colIdx = -1;
        this.thresh = 0.0;
        this.left = -1;
        this.right = -1;
        this.value = 0.0;
        this.isLeaf = false;
        this.rows = null;
      }
    }

    // ............................................

    public DecisionTreeRegressor(int maxDepth = 3,
      int minSamples = 2, int minLeaf = 1,
      int numSplitCols = -1, bool saveRows = false,
      int seed = 0)
    {
      this.maxDepth = maxDepth;
      this.minSamples = minSamples;
      this.minLeaf = minLeaf;
      this.numSplitCols = numSplitCols;
      this.saveRows = saveRows;
      this.tree = new List"lt"Node"gt"();

      int numNodes = (int)Math.Pow(2, (maxDepth + 1)) - 1;
      for (int i = 0; i "lt" numNodes; ++i)
        this.tree.Add(null);
      this.rnd = new Random(seed);
    }

    // ------------------------------------------------------
    // public: ctor(), Train(), Predict()
    // private: BestSplit(), TreeTargetMean()
    // ------------------------------------------------------

    public void Train(double[][] trainX, double[] trainY)
    {
      this.trainX = trainX;
      this.trainY = trainY;

      // boundary IDs based on max allowed depth
      int maxID = (int)Math.Pow(2, (this.maxDepth + 1)) - 2;
      int maxStartID = (int)Math.Pow(2, this.maxDepth) - 1;

      // initialize root data rows
      List"lt"int"gt" allRows = new List"lt"int"gt"(this.trainX.Length);
      for (int i = 0; i "lt" this.trainX.Length; ++i)
        allRows.Add(i);
      double grandMean = this.TreeTargetMean(allRows);

      Node root = new Node();
      root.id = 0;
      root.value = grandMean;
      root.isLeaf = false;
      root.rows = allRows;
      this.tree[0] = root;

      // build tree breadth-first
      for (int i = 0; i "lt" this.tree.Count; ++i)
      {
        Node currNode = this.tree[i];
        if (currNode == null) continue;

        // safety checks
        if (currNode.id "gte" maxStartID ||
          currNode.rows.Count "lt" this.minSamples)
        {
          currNode.isLeaf = true;
          currNode.left = -1;   // isolate node boundaries
          currNode.right = -1;
          currNode.colIdx = -1;
          continue;
        }

        double[] splitInfo = this.BestSplit(currNode.rows);
        int colIdx = (int)splitInfo[0];
        double splitVal = splitInfo[1];

        // check for split failure
        if (colIdx == -1)
        {
          currNode.isLeaf = true;
          currNode.left = -1;
          currNode.right = -1;
          currNode.colIdx = -1;
          continue;
        }

        // got a valid split point
        currNode.colIdx = colIdx;
        currNode.thresh = splitVal;

        // avoid continuous resizing allocations
        List"lt"int"gt" leftIdxs =
          new List"lt"int"gt"(currNode.rows.Count);
        List"lt"int"gt" rightIdxs =
          new List"lt"int"gt"(currNode.rows.Count);

        for (int k = 0; k "lt" currNode.rows.Count; ++k)
        {
          int r = currNode.rows[k];
          if (this.trainX[r][colIdx] "lte" splitVal)
            leftIdxs.Add(r);
          else
            rightIdxs.Add(r);
        }

        int leftID = currNode.id * 2 + 1;
        int rightID = currNode.id * 2 + 2;

        // check both branches
        bool leftValid = (leftID "lte" maxID &&
          leftIdxs.Count "gte" this.minLeaf);
        bool rightValid = (rightID "lte" maxID &&
          rightIdxs.Count "gte" this.minLeaf);

        if (leftValid == true && rightValid == true)
        {
          // create child nodes
          currNode.left = leftID;
          Node leftNode = new Node();
          leftNode.id = leftID;
          leftNode.rows = leftIdxs;
          leftNode.value =
            this.TreeTargetMean(leftNode.rows);
          this.tree[leftID] = leftNode;

          currNode.right = rightID;
          Node rightNode = new Node();
          rightNode.id = rightID;
          rightNode.rows = rightIdxs;
          rightNode.value =
            this.TreeTargetMean(rightNode.rows);
          this.tree[rightID] = rightNode;
        }
        else
        {
          // structural asymmetry/failure edge case
          // make parent into a leaf node
          currNode.isLeaf = true;
          currNode.left = -1;
          currNode.right = -1;
          currNode.colIdx = -1;
        }
      }

      // delete row info (for ensembles)
      if (this.saveRows == false)
      {
        for (int i = 0; i "lt" this.tree.Count; ++i)
          if (this.tree[i] != null)
            this.tree[i].rows = null;
      }
    }

    // ------------------------------------------------------

    public double Predict(double[] x)
    {
      int p = 0;
      double lastValidValue = 0.0;
      while (p != -1 && p "lt" this.tree.Count)
      {
        Node currNode = this.tree[p];

        if (currNode == null) break;

        lastValidValue = currNode.value;

        if (currNode.isLeaf == true ||
          currNode.colIdx == -1 ||
          currNode.left "gte" this.tree.Count ||
          currNode.right "gte" this.tree.Count)
          break;

        if (x[currNode.colIdx] "lte" currNode.thresh)
          p = currNode.left;
        else
          p = currNode.right;
      }
      return lastValidValue;
    }

    // ------------------------------------------------------

    private double[] BestSplit(List"lt"int"gt" rows)
    {
      // optimized for performance at expense of clarity
      int bestColIdx = -1;
      double bestThresh = 0.0;
      double bestVar = double.MaxValue;
      int nRows = rows.Count;
      int nCols = this.trainX[0].Length;

      if (nRows == 0)
        throw new Exception("empty data in BestSplit()");

      int[] colIndices = new int[nCols];
      for (int k = 0; k "lt" nCols; ++k)
        colIndices[k] = k;

      for (int i = 0; i "lt" nCols; ++i) // Fisher-Yates
      {
        int ri = rnd.Next(i, nCols);
        int tmp = colIndices[i];
        colIndices[i] = colIndices[ri];
        colIndices[ri] = tmp;
      }

      int numColsToUse;
      if (this.numSplitCols == -1)
        numColsToUse = nCols;
      else
        numColsToUse = Math.Min(this.numSplitCols, nCols);

      int[] activeCols = new int[numColsToUse];
      for (int k = 0; k "lt" numColsToUse; ++k)
        activeCols[k] = colIndices[k];

      int[] sortedRows = new int[nRows];
      double[] featureKeys = new double[nRows];

      double totalSum = 0.0;
      double totalSumSq = 0.0;
      for (int i = 0; i "lt" nRows; ++i)
      {
        double yCurr = this.trainY[rows[i]];
        totalSum += yCurr;
        totalSumSq += yCurr * yCurr;
      }

      for (int j = 0; j "lt" activeCols.Length; ++j)
      {
        int colIdx = activeCols[j];
        for (int i = 0; i "lt" nRows; ++i)
        {
          int r = rows[i];
          sortedRows[i] = r;
          featureKeys[i] = this.trainX[r][colIdx];
        }

        // sort both arrays based on keys
        Array.Sort(featureKeys, sortedRows);
        double leftSum = 0.0;
        double leftSumSq = 0.0;

        for (int i = 0; i "lt" nRows - 1; ++i)
        {
          int currRowIdx = sortedRows[i];
          double yCurr = this.trainY[currRowIdx];
          leftSum += yCurr;
          leftSumSq += yCurr * yCurr;

          int leftCount = i + 1;
          int rightCount = nRows - leftCount;

          double currFeatureVal = featureKeys[i];
          double nextFeatureVal = featureKeys[i + 1];

          // because x-features are sorted, this
          // code skips over previously seen values
          if (currFeatureVal == nextFeatureVal)
            continue;

          if (leftCount "lt" this.minLeaf ||
            rightCount "lt" this.minLeaf)
            continue;

          // tricky math to compute variances
          double rightSum = totalSum - leftSum;
          double rightSumSq = totalSumSq - leftSumSq;

          double tmp1 =
            (leftSum / leftCount) * (leftSum / leftCount);
          double leftVar = (leftSumSq / leftCount) - tmp1;

          double tmp2 =
            (rightSum / rightCount) * (rightSum / rightCount);
          double rightVar = (rightSumSq / rightCount) - tmp2;

          if (leftVar "lt" 0.0) leftVar = 0.0;
          if (rightVar "lt" 0.0) rightVar = 0.0;

          double weightedVar =
            ((leftCount * leftVar) + (rightCount * rightVar))
            / nRows;

          if (weightedVar "lt" bestVar)
          {
            bestColIdx = colIdx;
            // bestThresh =
            //   (currFeatureVal + nextFeatureVal) / 2.0;
            bestThresh = currFeatureVal; // simpler approach
            bestVar = weightedVar;
          }
        }
      }
      // using a tuple would create version dependency
      double[] result = new double[2];
      result[0] = 1.0 * bestColIdx;
      result[1] = bestThresh;
      return result;
    }

    // ------------------------------------------------------

    private double TreeTargetMean(List"lt"int"gt" rows)
    {
      if (rows == null || rows.Count == 0) return 0.0;
      double sum = 0.0;
      for (int i = 0; i "lt" rows.Count; ++i)
      {
        sum += this.trainY[rows[i]];
      }
      return sum / rows.Count;
    }

  } // class DecisionTreeRegressor

} // ns

Training data:

# synthetic_train_200.txt
#
-0.1660,  0.4406, -0.9998, -0.3953, -0.7065,  0.4840
 0.0776, -0.1616,  0.3704, -0.5911,  0.7562,  0.1568
-0.9452,  0.3409, -0.1654,  0.1174, -0.7192,  0.8054
 0.9365, -0.3732,  0.3846,  0.7528,  0.7892,  0.1345
-0.8299, -0.9219, -0.6603,  0.7563, -0.8033,  0.7955
 0.0663,  0.3838, -0.3690,  0.3730,  0.6693,  0.3206
-0.9634,  0.5003,  0.9777,  0.4963, -0.4391,  0.7377
-0.1042,  0.8172, -0.4128, -0.4244, -0.7399,  0.4801
-0.9613,  0.3577, -0.5767, -0.4689, -0.0169,  0.6861
-0.7065,  0.1786,  0.3995, -0.7953, -0.1719,  0.5569
 0.3888, -0.1716, -0.9001,  0.0718,  0.3276,  0.2500
 0.1731,  0.8068, -0.7251, -0.7214,  0.6148,  0.3297
-0.2046, -0.6693,  0.8550, -0.3045,  0.5016,  0.2129
 0.2473,  0.5019, -0.3022, -0.4601,  0.7918,  0.2613
-0.1438,  0.9297,  0.3269,  0.2434, -0.7705,  0.5171
 0.1568, -0.1837, -0.5259,  0.8068,  0.1474,  0.3307
-0.9943,  0.2343, -0.3467,  0.0541,  0.7719,  0.5581
 0.2467, -0.9684,  0.8589,  0.3818,  0.9946,  0.1092
-0.6553, -0.7257,  0.8652,  0.3936, -0.8680,  0.7018
 0.8460,  0.4230, -0.7515, -0.9602, -0.9476,  0.1996
-0.9434, -0.5076,  0.7201,  0.0777,  0.1056,  0.5664
 0.9392,  0.1221, -0.9627,  0.6013, -0.5341,  0.1533
 0.6142, -0.2243,  0.7271,  0.4942,  0.1125,  0.1661
 0.4260,  0.1194, -0.9749, -0.8561,  0.9346,  0.2230
 0.1362, -0.5934, -0.4953,  0.4877, -0.6091,  0.3810
 0.6937, -0.5203, -0.0125,  0.2399,  0.6580,  0.1460
-0.6864, -0.9628, -0.8600, -0.0273,  0.2127,  0.5387
 0.9772,  0.1595, -0.2397,  0.1019,  0.4907,  0.1611
 0.3385, -0.4702, -0.8673, -0.2598,  0.2594,  0.2270
-0.8669, -0.4794,  0.6095, -0.6131,  0.2789,  0.4700
 0.0493,  0.8496, -0.4734, -0.8681,  0.4701,  0.3516
 0.8639, -0.9721, -0.5313,  0.2336,  0.8980,  0.1412
 0.9004,  0.1133,  0.8312,  0.2831, -0.2200,  0.1782
 0.0991,  0.8524,  0.8375, -0.2102,  0.9265,  0.2150
-0.6521, -0.7473, -0.7298,  0.0113, -0.9570,  0.7422
 0.6190, -0.3105,  0.8802,  0.1640,  0.7577,  0.1056
 0.6895,  0.8108, -0.0802,  0.0927,  0.5972,  0.2214
 0.1982, -0.9689,  0.1870, -0.1326,  0.6147,  0.1310
-0.3695,  0.7858,  0.1557, -0.6320,  0.5759,  0.3773
-0.1596,  0.3581,  0.8372, -0.9992,  0.9535,  0.2071
-0.2468,  0.9476,  0.2094,  0.6577,  0.1494,  0.4132
 0.1737,  0.5000,  0.7166,  0.5102,  0.3961,  0.2611
 0.7290, -0.3546,  0.3416, -0.0983, -0.2358,  0.1332
-0.3652,  0.2438, -0.1395,  0.9476,  0.3556,  0.4170
-0.6029, -0.1466, -0.3133,  0.5953,  0.7600,  0.4334
-0.4596, -0.4953,  0.7098,  0.0554,  0.6043,  0.2775
 0.1450,  0.4663,  0.0380,  0.5418,  0.1377,  0.2931
-0.8636, -0.2442, -0.8407,  0.9656, -0.6368,  0.7429
 0.6237,  0.7499,  0.3768,  0.1390, -0.6781,  0.2185
-0.5499,  0.1850, -0.3755,  0.8326,  0.8193,  0.4399
-0.4858, -0.7782, -0.6141, -0.0008,  0.4572,  0.4197
 0.7033, -0.1683,  0.2334, -0.5327, -0.7961,  0.1776
 0.0317, -0.0457, -0.6947,  0.2436,  0.0880,  0.3345
 0.5031, -0.5559,  0.0387,  0.5706, -0.9553,  0.3107
-0.3513,  0.7458,  0.6894,  0.0769,  0.7332,  0.3170
 0.2205,  0.5992, -0.9309,  0.5405,  0.4635,  0.3532
-0.4806, -0.4859,  0.2646, -0.3094,  0.5932,  0.3202
 0.9809, -0.3995, -0.7140,  0.8026,  0.0831,  0.1600
 0.9495,  0.2732,  0.9878,  0.0921,  0.0529,  0.1289
-0.9476, -0.6792,  0.4913, -0.9392, -0.2669,  0.5966
 0.7247,  0.3854,  0.3819, -0.6227, -0.1162,  0.1550
-0.5922, -0.5045, -0.4757,  0.5003, -0.0860,  0.5863
-0.8861,  0.0170, -0.5761,  0.5972, -0.4053,  0.7301
 0.6877, -0.2380,  0.4997,  0.0223,  0.0819,  0.1404
 0.9189,  0.6079, -0.9354,  0.4188, -0.0700,  0.1907
-0.1428, -0.7820,  0.2676,  0.6059,  0.3936,  0.2790
 0.5324, -0.3151,  0.6917, -0.1425,  0.6480,  0.1071
-0.8432, -0.9633, -0.8666, -0.0828, -0.7733,  0.7784
-0.9444,  0.5097, -0.2103,  0.4939, -0.0952,  0.6787
-0.0520,  0.6063, -0.1952,  0.8094, -0.9259,  0.4836
 0.5477, -0.7487,  0.2370, -0.9793,  0.0773,  0.1241
 0.2450,  0.8116,  0.9799,  0.4222,  0.4636,  0.2355
 0.8186, -0.1983, -0.5003, -0.6531, -0.7611,  0.1511
-0.4714,  0.6382, -0.3788,  0.9648, -0.4667,  0.5950
 0.0673, -0.3711,  0.8215, -0.2669, -0.1328,  0.2677
-0.9381,  0.4338,  0.7820, -0.9454,  0.0441,  0.5518
-0.3480,  0.7190,  0.1170,  0.3805, -0.0943,  0.4724
-0.9813,  0.1535, -0.3771,  0.0345,  0.8328,  0.5438
-0.1471, -0.5052, -0.2574,  0.8637,  0.8737,  0.3042
-0.5454, -0.3712, -0.6505,  0.2142, -0.1728,  0.5783
 0.6327, -0.6297,  0.4038, -0.5193,  0.1484,  0.1153
-0.5424,  0.3282, -0.0055,  0.0380, -0.6506,  0.6613
 0.1414,  0.9935,  0.6337,  0.1887,  0.9520,  0.2540
-0.9351, -0.8128, -0.8693, -0.0965, -0.2491,  0.7353
 0.9507, -0.6640,  0.9456,  0.5349,  0.6485,  0.1059
-0.0462, -0.9737, -0.2940, -0.0159,  0.4602,  0.2606
-0.0627, -0.0852, -0.7247, -0.9782,  0.5166,  0.2977
 0.0478,  0.5098, -0.0723, -0.7504, -0.3750,  0.3335
 0.0090,  0.3477,  0.5403, -0.7393, -0.9542,  0.4415
-0.9748,  0.3449,  0.3736, -0.1015,  0.8296,  0.4358
 0.2887, -0.9895, -0.0311,  0.7186,  0.6608,  0.2057
 0.1570, -0.4518,  0.1211,  0.3435, -0.2951,  0.3244
 0.7117, -0.6099,  0.4946, -0.4208,  0.5476,  0.1096
-0.2929, -0.5726,  0.5346, -0.3827,  0.4665,  0.2465
 0.4889, -0.5572, -0.5718, -0.6021, -0.7150,  0.2163
-0.7782,  0.3491,  0.5996, -0.8389, -0.5366,  0.6516
-0.5847,  0.8347,  0.4226,  0.1078, -0.3910,  0.6134
 0.8469,  0.4121, -0.0439, -0.7476,  0.9521,  0.1571
-0.6803, -0.5948, -0.1376, -0.1916, -0.7065,  0.7156
 0.2878,  0.5086, -0.5785,  0.2019,  0.4979,  0.2980
 0.2764,  0.1943, -0.4090,  0.4632,  0.8906,  0.2960
-0.8877,  0.6705, -0.6155, -0.2098, -0.3998,  0.7107
-0.8398,  0.8093, -0.2597,  0.0614, -0.0118,  0.6502
-0.8476,  0.0158, -0.4769, -0.2859, -0.7839,  0.7715
 0.5751, -0.7868,  0.9714, -0.6457,  0.1448,  0.1175
 0.4802, -0.7001,  0.1022, -0.5668,  0.5184,  0.1090
 0.4458, -0.6469,  0.7239, -0.9604,  0.7205,  0.0779
 0.5175,  0.4339,  0.9747, -0.4438, -0.9924,  0.2879
 0.8678,  0.7158,  0.4577,  0.0334,  0.4139,  0.1678
 0.5406,  0.5012,  0.2264, -0.1963,  0.3946,  0.2088
-0.9938,  0.5498,  0.7928, -0.5214, -0.7585,  0.7687
 0.7661,  0.0863, -0.4266, -0.7233, -0.4197,  0.1466
 0.2277, -0.3517, -0.0853, -0.1118,  0.6563,  0.1767
 0.3499, -0.5570, -0.0655, -0.3705,  0.2537,  0.1632
 0.7547, -0.1046,  0.5689, -0.0861,  0.3125,  0.1257
 0.8186,  0.2110,  0.5335,  0.0094, -0.0039,  0.1391
 0.6858, -0.8644,  0.1465,  0.8855,  0.0357,  0.1845
-0.4967,  0.4015,  0.0805,  0.8977,  0.2487,  0.4663
 0.6760, -0.9841,  0.9787, -0.8446, -0.3557,  0.1509
-0.1203, -0.4885,  0.6054, -0.0443, -0.7313,  0.4854
 0.8557,  0.7919, -0.0169,  0.7134, -0.1628,  0.2002
 0.0115, -0.6209,  0.9300, -0.4116, -0.7931,  0.4052
-0.7114, -0.9718,  0.4319,  0.1290,  0.5892,  0.3661
 0.3915,  0.5557, -0.1870,  0.2955, -0.6404,  0.2954
-0.3564, -0.6548, -0.1827, -0.5172, -0.1862,  0.4622
 0.2392, -0.4959,  0.5857, -0.1341, -0.2850,  0.2470
-0.3394,  0.3947, -0.4627,  0.6166, -0.4094,  0.5325
 0.7107,  0.7768, -0.6312,  0.1707,  0.7964,  0.2757
-0.1078,  0.8437, -0.4420,  0.2177,  0.3649,  0.4028
-0.3139,  0.5595, -0.6505, -0.3161, -0.7108,  0.5546
 0.4335,  0.3986,  0.3770, -0.4932,  0.3847,  0.1810
-0.2562, -0.2894, -0.8847,  0.2633,  0.4146,  0.4036
 0.2272,  0.2966, -0.6601, -0.7011,  0.0284,  0.2778
-0.0743, -0.1421, -0.0054, -0.6770, -0.3151,  0.3597
-0.4762,  0.6891,  0.6007, -0.1467,  0.2140,  0.4266
-0.4061,  0.7193,  0.3432,  0.2669, -0.7505,  0.6147
-0.0588,  0.9731,  0.8966,  0.2902, -0.6966,  0.4955
-0.0627, -0.1439,  0.1985,  0.6999,  0.5022,  0.3077
 0.1587,  0.8494, -0.8705,  0.9827, -0.8940,  0.4263
-0.7850,  0.2473, -0.9040, -0.4308, -0.8779,  0.7199
 0.4070,  0.3369, -0.2428, -0.6236,  0.4940,  0.2215
-0.0242,  0.0513, -0.9430,  0.2885, -0.2987,  0.3947
-0.5416, -0.1322, -0.2351, -0.0604,  0.9590,  0.3683
 0.1055,  0.7783, -0.2901, -0.5090,  0.8220,  0.2984
-0.9129,  0.9015,  0.1128, -0.2473,  0.9901,  0.4776
-0.9378,  0.1424, -0.6391,  0.2619,  0.9618,  0.5368
 0.7498, -0.0963,  0.4169,  0.5549, -0.0103,  0.1614
-0.2612, -0.7156,  0.4538, -0.0460, -0.1022,  0.3717
 0.7720,  0.0552, -0.1818, -0.4622, -0.8560,  0.1685
-0.4177,  0.0070,  0.9319, -0.7812,  0.3461,  0.3052
-0.0001,  0.5542, -0.7128, -0.8336, -0.2016,  0.3803
 0.5356, -0.4194, -0.5662, -0.9666, -0.2027,  0.1776
-0.2378,  0.3187, -0.8582, -0.6948, -0.9668,  0.5474
-0.1947, -0.3579,  0.1158,  0.9869,  0.6690,  0.2992
 0.3992,  0.8365, -0.9205, -0.8593, -0.0520,  0.3154
-0.0209,  0.0793,  0.7905, -0.1067,  0.7541,  0.1864
-0.4928, -0.4524, -0.3433,  0.0951, -0.5597,  0.6261
-0.8118,  0.7404, -0.5263, -0.2280,  0.1431,  0.6349
 0.0516, -0.8480,  0.7483,  0.9023,  0.6250,  0.1959
-0.3212,  0.1093,  0.9488, -0.3766,  0.3376,  0.2735
-0.3481,  0.5490, -0.3484,  0.7797,  0.5034,  0.4379
-0.5785, -0.9170, -0.3563, -0.9258,  0.3877,  0.4121
 0.3407, -0.1391,  0.5356,  0.0720, -0.9203,  0.3458
-0.3287, -0.8954,  0.2102,  0.0241,  0.2349,  0.3247
-0.1353,  0.6954, -0.0919, -0.9692,  0.7461,  0.3338
 0.9036, -0.8982, -0.5299, -0.8733, -0.1567,  0.1187
 0.7277, -0.8368, -0.0538, -0.7489,  0.5458,  0.0830
 0.9049,  0.8878,  0.2279,  0.9470, -0.3103,  0.2194
 0.7957, -0.1308, -0.5284,  0.8817,  0.3684,  0.2172
 0.4647, -0.4931,  0.2010,  0.6292, -0.8918,  0.3371
-0.7390,  0.6849,  0.2367,  0.0626, -0.5034,  0.7039
-0.1567, -0.8711,  0.7940, -0.5932,  0.6525,  0.1710
 0.7635, -0.0265,  0.1969,  0.0545,  0.2496,  0.1445
 0.7675,  0.1354, -0.7698, -0.5460,  0.1920,  0.1728
-0.5211, -0.7372, -0.6763,  0.6897,  0.2044,  0.5217
 0.1913,  0.1980,  0.2314, -0.8816,  0.5006,  0.1998
 0.8964,  0.0694, -0.6149,  0.5059, -0.9854,  0.1825
 0.1767,  0.7104,  0.2093,  0.6452,  0.7590,  0.2832
-0.3580, -0.7541,  0.4426, -0.1193, -0.7465,  0.5657
-0.5996,  0.5766, -0.9758, -0.3933, -0.9572,  0.6800
 0.9950,  0.1641, -0.4132,  0.8579,  0.0142,  0.2003
-0.4717, -0.3894, -0.2567, -0.5111,  0.1691,  0.4266
 0.3917, -0.8561,  0.9422,  0.5061,  0.6123,  0.1212
-0.0366, -0.1087,  0.3449, -0.1025,  0.4086,  0.2475
 0.3633,  0.3943,  0.2372, -0.6980,  0.5216,  0.1925
-0.5325, -0.6466, -0.2178, -0.3589,  0.6310,  0.3568
 0.2271,  0.5200, -0.1447, -0.8011, -0.7699,  0.3128
 0.6415,  0.1993,  0.3777, -0.0178, -0.8237,  0.2181
-0.5298, -0.0768, -0.6028, -0.9490,  0.4588,  0.4356
 0.6870, -0.1431,  0.7294,  0.3141,  0.1621,  0.1632
-0.5985,  0.0591,  0.7889, -0.3900,  0.7419,  0.2945
 0.3661,  0.7984, -0.8486,  0.7572, -0.6183,  0.3449
 0.6995,  0.3342, -0.3113, -0.6972,  0.2707,  0.1712
 0.2565,  0.9126,  0.1798, -0.6043, -0.1413,  0.2893
-0.3265,  0.9839, -0.2395,  0.9854,  0.0376,  0.4770
 0.2690, -0.1722,  0.9818,  0.8599, -0.7015,  0.3954
-0.2102, -0.0768,  0.1219,  0.5607, -0.0256,  0.3949
 0.8216, -0.9555,  0.6422, -0.6231,  0.3715,  0.0801
-0.2896,  0.9484, -0.7545, -0.6249,  0.7789,  0.4370
-0.9985, -0.5448, -0.7092, -0.5931,  0.7926,  0.5402

Test data:

# synthetic_test_40.txt
#
 0.7462,  0.4006, -0.0590,  0.6543, -0.0083,  0.1935
 0.8495, -0.2260, -0.0142, -0.4911,  0.7699,  0.1078
-0.2335, -0.4049,  0.4352, -0.6183, -0.7636,  0.5088
 0.1810, -0.5142,  0.2465,  0.2767, -0.3449,  0.3136
-0.8650,  0.7611, -0.0801,  0.5277, -0.4922,  0.7140
-0.2358, -0.7466, -0.5115, -0.8413, -0.3943,  0.4533
 0.4834,  0.2300,  0.3448, -0.9832,  0.3568,  0.1360
-0.6502, -0.6300,  0.6885,  0.9652,  0.8275,  0.3046
-0.3053,  0.5604,  0.0929,  0.6329, -0.0325,  0.4756
-0.7995,  0.0740, -0.2680,  0.2086,  0.9176,  0.4565
-0.2144, -0.2141,  0.5813,  0.2902, -0.2122,  0.4119
-0.7278, -0.0987, -0.3312, -0.5641,  0.8515,  0.4438
 0.3793,  0.1976,  0.4933,  0.0839,  0.4011,  0.1905
-0.8568,  0.9573, -0.5272,  0.3212, -0.8207,  0.7415
-0.5785,  0.0056, -0.7901, -0.2223,  0.0760,  0.5551
 0.0735, -0.2188,  0.3925,  0.3570,  0.3746,  0.2191
 0.1230, -0.2838,  0.2262,  0.8715,  0.1938,  0.2878
 0.4792, -0.9248,  0.5295,  0.0366, -0.9894,  0.3149
-0.4456,  0.0697,  0.5359, -0.8938,  0.0981,  0.3879
 0.8629, -0.8505, -0.4464,  0.8385,  0.5300,  0.1769
 0.1995,  0.6659,  0.7921,  0.9454,  0.9970,  0.2330
-0.0249, -0.3066, -0.2927, -0.4923,  0.8220,  0.2437
 0.4513, -0.9481, -0.0770, -0.4374, -0.9421,  0.2879
-0.3405,  0.5931, -0.3507, -0.3842,  0.8562,  0.3987
 0.9538,  0.0471,  0.9039,  0.7760,  0.0361,  0.1706
-0.0887,  0.2104,  0.9808,  0.5478, -0.3314,  0.4128
-0.8220, -0.6302,  0.0537, -0.1658,  0.6013,  0.4306
-0.4123, -0.2880,  0.9074, -0.0461, -0.4435,  0.5144
 0.0060,  0.2867, -0.7775,  0.5161,  0.7039,  0.3599
-0.7968, -0.5484,  0.9426, -0.4308,  0.8148,  0.2979
 0.7811,  0.8450, -0.6877,  0.7594,  0.2640,  0.2362
-0.6802, -0.1113, -0.8325, -0.6694, -0.6056,  0.6544
 0.3821,  0.1476,  0.7466, -0.5107,  0.2592,  0.1648
 0.7265,  0.9683, -0.9803, -0.4943, -0.5523,  0.2454
-0.9049, -0.9797, -0.0196, -0.9090, -0.4433,  0.6447
-0.4607,  0.1811, -0.2389,  0.4050, -0.0078,  0.5229
 0.2664, -0.2932, -0.4259, -0.7336,  0.8742,  0.1834
-0.4507,  0.1029, -0.6294, -0.1158, -0.6294,  0.6081
 0.8948, -0.0124,  0.9278,  0.2899, -0.0314,  0.1534
-0.1323, -0.8813, -0.0146, -0.0697,  0.6135,  0.2386
Posted in Machine Learning | Leave a comment

The Boston Area House Price Problem With From-Scratch Support Vector Regression Using Python

Posted on August 10, 2026 by jamesdmccaffrey

I recently refactored my basic kernel support vector regression system, implemented from scratch, using Python. I tested my refactored code using synthetic data, but I figured I should run the system on some real world data.

One of the standard datasets for machine learning regression problems is the Boston Area House dataset. The data has 506 items. Each item represents a town or village near Boston. The goal is to predict the median house price in a town. Each row has 14 values, where the first 13 in columns [0] to [12] are predictors, and the last value in column [14] is the target median house price in the town. The 13 predictor values are things like crime rate in town, tax rate in town, density of Black people in town, percent houses built before 1940 in town, and so on. The data is from a 1978 research paper so the median house prices are very low (often around $20,000).

I fetched the raw data from lib.stat.cmu.edu/datasets/boston and normalized it by dividing each of the 12 numeric predictor columns, and the house price column, by either 1, 10, 100, or 1000 so that all numeric values are between 0.0 and 1.0. One of the predictors is Boolean (town is adjacent to Charles River = 1, or not adjacent = 0). I randomly split the 506 data items into a 400-item training set and a 106-item test set.

The output of my from-scratch SVR system is:

Begin scratch Python SVR using SMO training

Loading normed train (400) and test (106) data
Done

First three train X:
[0.0003 0.0000 0.2357 0.0000 0.4690 . . . 0.2285]
[0.0003 0.0000 0.2357 0.0000 0.4690 . . . 0.1008]
[0.0003 0.0000 0.0727 0.0000 0.4580 . . . 0.0735]

First three train y:
0.4320
0.6940
0.6680

Creating SVR-SMO model
Setting gamma = 0.5000
Setting C = 1.00
Setting epsilon = 0.0100
Setting max_iter = 100
Setting tol = 0.000010

Creating and training SVR model using SMO
Done

Model dual coefs:
[-1.0000  1.0000  1.0000  1.0000  . . .  0.3089
 -0.2297  1.0000 -1.0000 -1.0000  . . . -1.0000
 -1.0000  1.0000 -1.0000 -1.0000  . . .  1.0000
 . . .
 -0.3150  0.8940 -1.0000  1.0000  . . .  0.1407
 -0.0657]

Model bias = 0.7074

Number support vectors = 400

Train accuracy (0.16) = 0.7925
Test accuracy (0.15) = 0.7642

Train MSE = 0.0056
Test MSE = 0.0039

Train R2 = 0.8380
Test R2 = 0.8677

Predicting for train_X[0]
Predicted y = 0.4576

End demo

The results of my from-scratch SVR system were quite close to the results I got using the scikit-learn SVR module:

Number model support vectors: [341] == 341
Model bias: 0.6911

Train accuracy (0.15) = 0.7950
Test accuracy (0.15) = 0.7642

Train MSE = 0.0057
Test MSE = 0.0039

Train R2 = 0.8373
Test R2 = 0.8682

The main difference is that the scikit-learn SVR is a wrapper around a system called libsvm, which is highly optimized to reduce the number of support vectors. Anyway, the demo validated my from-scratch SVR system.

Support vector regression was briefly popular in the late 1990s and early 2000s, but faded from widespread use quickly when people discovered that the closely related kernel ridge regression (KRR) is better in just about every way than SVR (KRR is easier to implement, much easier to tune, and usually give better prediction accuracy).


One of the significant predictor variables in the Boston Area House Dataset is the density of Black people in a town. Briefly, more Blacks, lower house prices. Some things haven’t changed much since 1978. When I first started working on this SVR problem, it was graduation season. Once again, I was not disappointed with graduation-related news stories. See also “Carnival Cruise brawl”, “Airport Passenger Brawl”, “Wedding Reception Brawl”, “High School Basketball Game Brawl”, and so on.

Demo program. Replace “lt” (less than), “gt”, “lte”, “gte” with Boolean operator symbols (my lame blog editor chokes on symbols).

# boston_svr_smo.py

# kernel support vector regression from scratch.
# uses a hard-wired RBF kernel function.

import numpy as np

# -----------------------------------------------------------

np.set_printoptions(precision=4, suppress=True,
  floatmode='fixed', linewidth=160)

# -----------------------------------------------------------
# external eval functions: accuracy(), mse(), r2_score()
# -----------------------------------------------------------

def accuracy(model, data_X, data_y, pct_close):
  n = len(data_X)
  n_correct = 0; n_wrong = 0
  for i in range(n):
    x = data_X[i].reshape(1,-1)
    y = data_y[i]
    pred_y = model.predict(x)[0]
    if np.abs(y - pred_y) "lt" np.abs(y * pct_close):
      n_correct += 1
    else: 
      n_wrong += 1
  return n_correct / (n_correct + n_wrong)

# -----------------------------------------------------------

def mse(model, data_X, data_y):
  n = len(data_X)
  sum = 0.0
  for i in range(n):
    x = data_X[i].reshape(1,-1)
    y = data_y[i]
    pred_y = model.predict(x)[0]
    diff = pred_y - y
    sum += diff * diff
  return sum /n

# -----------------------------------------------------------

def r2_score(model, data_X, data_y):
  # coefficient of determination == scikit score()
  ss_res = 0.0
  ss_tot = 0.0
  n = len(data_X)
  mean_y = np.mean(data_y)
  for i in range(n):
    x = data_X[i].reshape(1,-1)
    y = data_y[i]
    pred_y = model.predict(x)[0]
    ss_res += (y - pred_y) * (y - pred_y)
    ss_tot += (y - mean_y) * (y - mean_y)
  result = 1.0 - (ss_res / ss_tot)
  return result

# ===========================================================

class KernelSVR:
  def __init__(self, gamma=0.1, epsilon=0.1, C=1.0,
    max_iter=100, tol=1.0e-3, seed=1):
    self.gamma = gamma
    self.epsilon = epsilon
    self.C = C
    self.max_iter = max_iter
    self.tol = tol  # for KKT conditions
    self.rnd = np.random.RandomState(seed)
    
    self.alpha = None
    self.alpha_star = None
    self.b = 0.0
    self.supp_X = None
    self.supp_y = None
    self.dual_weights = None
  # ---------------------------------------------------------

  def kernel_matrix(self, X1, X2):
    sq_dist = np.sum(X1**2, axis=1).reshape(-1, 1) + \
      np.sum(X2**2, axis=1) - 2 * np.dot(X1, X2.T)
    return np.exp(-self.gamma * sq_dist)

  # ---------------------------------------------------------

  def fit(self, X, y):
    n = X.shape[0]
    self.alpha = np.zeros(n)
    self.alpha_star = np.zeros(n)
    self.b = np.mean(y)
    
    K = self.kernel_matrix(X, X)
    n_passes = 0
    while n_passes "lt" self.max_iter:
      num_changed_alphas = 0
      
      for i in range(n):
        # prediction and error for i
        pred_i = np.dot(self.alpha_star - \
          self.alpha, K[:, i]) + self.b
        err_i = pred_i - y[i]
        
        # check KKT conditions for SVR within tolerance
        if ((err_i "gt" self.epsilon + self.tol and \
             self.alpha[i] "lt" self.C) or \
            (err_i "gt" self.epsilon + self.tol and \
             self.alpha_star[i] "gt" 0) or
            (err_i "lt" -self.epsilon - self.tol and \
             self.alpha[i] "gt" 0) or \
            (err_i "lt" -self.epsilon - self.tol and \
             self.alpha_star[i] "lt" self.C)):
            
          # pick a random second index j != i
          j = i
          while j == i:
            j = self.rnd.randint(0, n)
              
          pred_j = np.dot(self.alpha_star - \
            self.alpha, K[:, j]) + self.b
          err_j = pred_j - y[j]
          
          # save old values
          a_i_old, as_i_old = self.alpha[i], \
            self.alpha_star[i]
          a_j_old, as_j_old = self.alpha[j], \
            self.alpha_star[j]
          
          # kernel second derivative step denominator
          eta = K[i, i] + K[j, j] - 2 * K[i, j]
          if eta "lte" 0:
            continue
              
          # SVR linear constraint constant
          constraint = (as_i_old - a_i_old) + (as_j_old - a_j_old)
          
          # joint proxy variable s_j = (alpha_j* - alpha_j)
          s_j_old = as_j_old - a_j_old
          s_j_new = s_j_old + (err_i - err_j) / eta
          
          # bounds L and H for the proxy s_j
          L = max(-self.C, constraint - self.C)
          H = min(self.C, constraint + self.C)
          s_j_new = np.clip(s_j_new, L, H)
          
          if abs(s_j_new - s_j_old) "lt" 1.0e-5:
            continue
              
          # reconstruct alpha_j and alpha_j* 
          if s_j_new "gte" 0.0:
            self.alpha_star[j] = s_j_new
            self.alpha[j] = 0.0
          else:
            self.alpha_star[j] = 0.0
            self.alpha[j] = -s_j_new
              
          # update alpha_i and alpha_i*
          s_i_new = constraint - s_j_new
          if s_i_new "gte" 0.0:
            self.alpha_star[i] = s_i_new
            self.alpha[i] = 0.0
          else:
            self.alpha_star[i] = 0.0
            self.alpha[i] = -s_i_new
              
          # update bias
          b1 = self.b - err_i - ((self.alpha_star[i] - \
               self.alpha[i]) - (as_i_old - a_i_old)) * \
               K[i, i] - ((self.alpha_star[j] - \
               self.alpha[j]) - (as_j_old - a_j_old)) * \
               K[i, j]
          b2 = self.b - err_j - ((self.alpha_star[i] - \
               self.alpha[i]) - (as_i_old - a_i_old)) * \
               K[i, j] - ((self.alpha_star[j] - \
               self.alpha[j]) - (as_j_old - a_j_old)) * \
               K[j, j]
          
          if 0.0 "lt" self.alpha[i] "lt" self.C or \
            0.0 "lt" self.alpha_star[i] "lt" self.C:
            self.b = b1
          elif 0 "lt" self.alpha[j] "lt" self.C or \
            0.0 "lt" self.alpha_star[j] "lt" self.C:
            self.b = b2
          else:
            self.b = (b1 + b2) / 2.0
              
          num_changed_alphas += 1
          
      if num_changed_alphas == 0:
        n_passes += 1
      else:
        n_passes = 0
        
    # prune: store only explicit support vectors
    sv_mask = (self.alpha "gt" 1.0e-5) | \
      (self.alpha_star "gt" 1.0e-5)
    self.supp_X = X[sv_mask]
    self.supp_y = y[sv_mask]
    
    # dual weights combine alpha* and alpha
    self.dual_weights = \
      (self.alpha_star - self.alpha)[sv_mask]
    return self
  
  # ---------------------------------------------------------  

  def predict(self, X):
    K = self.kernel_matrix(X, self.supp_X)
    return np.dot(K, self.dual_weights) + self.b

  # ---------------------------------------------------------

  def get_supp_idxs(self):
    result = []
    for i in range(len(self.alpha)):
      # a non-zero wt is associated with a supp vector
      if np.abs(self.alpha[i]) "gt" 1.0e-5 or \
        np.abs(self.alpha_star[i]) "gt" 1.0e-5:
        result.append(i)
    return result

# ===========================================================

def main():
  print("\nBegin scratch Python SVR using SMO training ")

  print("\nLoading normed train (400) and test (106) data")
  train_Xy = np.loadtxt(".\\Data\\boston_train_400.txt",
   usecols=[0,1,2,3,4,5,6,7,8,9,10,11,12,13], delimiter=",")
  train_X = train_Xy[:,[0,1,2,3,4,5,6,7,8,9,10,11,12]]
  train_y = train_Xy[:,13]

  test_Xy = np.loadtxt(".\\Data\\boston_test_106.txt",
    usecols=[0,1,2,3,4,5,6,7,8,9,10,11,12,13], delimiter=",")
  test_X = test_Xy[:,[0,1,2,3,4,5,6,7,8,9,10,11,12]]
  test_y = test_Xy[:,13]
  print("Done ")

  print("\nFirst three train X: ")
  for i in range(3):
    print(train_X[i])
  print("\nFirst three train y: ")
  for i in range(3):
    print("%0.4f " % train_y[i])

  # ** SCIKIT results **
  # Setting gamma = 0.5000
  # Setting C = 1.000000
  # Setting epsilon = 0.0100
  # Number model support vectors: [341] == 341
  # Model bias: 0.6911
  # Train accuracy (0.15) = 0.7950
  # Test accuracy (0.15) = 0.7642
  # Train MSE = 0.0057
  # Test MSE = 0.0039
  # Train R2 = 0.8373
  # Test R2 = 0.8682

  # create and train model
  print("\nCreating SVR-SMO model ")
  gamma = 0.50
  epsilon = 0.01
  C = 1.0
  max_iter = 100
  tol = 1.0e-5

  print("Setting gamma = %0.4f " % gamma)
  print("Setting C = %0.2f " % C)
  print("Setting epsilon = %0.6f " % epsilon)
  print("Setting max_iter = " + str(max_iter))
  print("Setting tol = %0.6f " % tol)

  print("\nCreating and training SVR model using SMO ")

  model = KernelSVR(gamma=gamma, epsilon=epsilon, C=C, 
    max_iter=max_iter, tol=tol)

  model.fit(train_X, train_y)
  print("Done ")

  # print("\nModel alpha: ")
  # print(model.alpha)
  # print("\nModel alpha*: ") 
  # print(model.alpha_star)

  print("\nModel dual coefs: ")
  print(model.dual_weights)

  print("\nModel bias = %0.4f " % model.b)

  supp_vec_idxs = model.get_supp_idxs()
  print("\nNumber support vectors = " + \
    str(len(supp_vec_idxs)))
  # or len(model.dual_weights)
  # or len(model.supp_X)

  acc_train = accuracy(model, train_X, train_y, 0.15)
  print("\nTrain accuracy (0.16) = %0.4f" % acc_train)
  acc_test = accuracy(model, test_X, test_y, 0.15)
  print("Test accuracy (0.15) = %0.4f" % acc_test)

  mse_train = mse(model, train_X, train_y)
  print("\nTrain MSE = %0.4f" % mse_train)
  mse_test = mse(model, test_X, test_y)
  print("Test MSE = %0.4f" % mse_test)

  r2_train = r2_score(model, train_X, train_y)
  print("\nTrain R2 = %0.4f" % r2_train)
  r2_test = r2_score(model, test_X, test_y)
  print("Test R2 = %0.4f" % r2_test)

  print("\nPredicting for train_X[0] ")
  x = train_X[0].reshape(1,-1)
  pred_y = model.predict(x)[0]
  print("Predicted y = %0.4f " % pred_y)

  print("\nEnd demo ")

# -----------------------------------------------------------

if __name__ == "__main__":
  main()

Training data:


# boston_train_400.txt
# norm constants: 100, 100, 30, 1, 1, 10, 100, 20, 30, 800, 30, 400, 40, 50
#
# crime     zoning      indus       river       nox         rooms       oldness     dist        access      tax         pup_tch     black       low_stat    med_val
0.00027310, 0.00000000, 0.23566667, 0.00000000, 0.46900000, 0.64210000, 0.78900000, 0.24835500, 0.06666667, 0.30250000, 0.59333333, 0.99225000, 0.22850000, 0.43200000
0.00027290, 0.00000000, 0.23566667, 0.00000000, 0.46900000, 0.71850000, 0.61100000, 0.24835500, 0.06666667, 0.30250000, 0.59333333, 0.98207500, 0.10075000, 0.69400000
0.00032370, 0.00000000, 0.07266667, 0.00000000, 0.45800000, 0.69980000, 0.45800000, 0.30311000, 0.10000000, 0.27750000, 0.62333333, 0.98657500, 0.07350000, 0.66800000
0.00069050, 0.00000000, 0.07266667, 0.00000000, 0.45800000, 0.71470000, 0.54200000, 0.30311000, 0.10000000, 0.27750000, 0.62333333, 0.99225000, 0.13325000, 0.72400000
0.00088290, 0.12500000, 0.26233333, 0.00000000, 0.52400000, 0.60120000, 0.66600000, 0.27802500, 0.16666667, 0.38875000, 0.50666667, 0.98900000, 0.31075000, 0.45800000
0.00144550, 0.12500000, 0.26233333, 0.00000000, 0.52400000, 0.61720000, 0.96100000, 0.29752500, 0.16666667, 0.38875000, 0.50666667, 0.99225000, 0.47875000, 0.54200000
0.00211240, 0.12500000, 0.26233333, 0.00000000, 0.52400000, 0.56310000, 1.00000000, 0.30410500, 0.16666667, 0.38875000, 0.50666667, 0.96657500, 0.74825000, 0.33000000
0.00170040, 0.12500000, 0.26233333, 0.00000000, 0.52400000, 0.60040000, 0.85900000, 0.32960500, 0.16666667, 0.38875000, 0.50666667, 0.96677500, 0.42750000, 0.37800000
0.00117470, 0.12500000, 0.26233333, 0.00000000, 0.52400000, 0.60090000, 0.82900000, 0.31133500, 0.16666667, 0.38875000, 0.50666667, 0.99225000, 0.33175000, 0.37800000
0.00093780, 0.12500000, 0.26233333, 0.00000000, 0.52400000, 0.58890000, 0.39000000, 0.27254500, 0.16666667, 0.38875000, 0.50666667, 0.97625000, 0.39275000, 0.43400000
0.00629760, 0.00000000, 0.27133333, 0.00000000, 0.53800000, 0.59490000, 0.61800000, 0.23537500, 0.13333333, 0.38375000, 0.70000000, 0.99225000, 0.20650000, 0.40800000
0.00637960, 0.00000000, 0.27133333, 0.00000000, 0.53800000, 0.60960000, 0.84500000, 0.22309500, 0.13333333, 0.38375000, 0.70000000, 0.95005000, 0.25650000, 0.36400000
0.01053930, 0.00000000, 0.27133333, 0.00000000, 0.53800000, 0.59350000, 0.29300000, 0.22493000, 0.13333333, 0.38375000, 0.70000000, 0.96712500, 0.16450000, 0.46200000
0.00784200, 0.00000000, 0.27133333, 0.00000000, 0.53800000, 0.59900000, 0.81700000, 0.21289500, 0.13333333, 0.38375000, 0.70000000, 0.96687500, 0.36675000, 0.35000000
0.00802710, 0.00000000, 0.27133333, 0.00000000, 0.53800000, 0.54560000, 0.36600000, 0.18982500, 0.13333333, 0.38375000, 0.70000000, 0.72247500, 0.29225000, 0.40400000
0.00725800, 0.00000000, 0.27133333, 0.00000000, 0.53800000, 0.57270000, 0.69500000, 0.18982500, 0.13333333, 0.38375000, 0.70000000, 0.97737500, 0.28200000, 0.36400000
0.00852040, 0.00000000, 0.27133333, 0.00000000, 0.53800000, 0.59650000, 0.89200000, 0.20061500, 0.13333333, 0.38375000, 0.70000000, 0.98132500, 0.34575000, 0.39200000
0.01232470, 0.00000000, 0.27133333, 0.00000000, 0.53800000, 0.61420000, 0.91700000, 0.19884500, 0.13333333, 0.38375000, 0.70000000, 0.99225000, 0.46800000, 0.30400000
0.00988430, 0.00000000, 0.27133333, 0.00000000, 0.53800000, 0.58130000, 1.00000000, 0.20476000, 0.13333333, 0.38375000, 0.70000000, 0.98635000, 0.49700000, 0.29000000
0.00750260, 0.00000000, 0.27133333, 0.00000000, 0.53800000, 0.59240000, 0.94100000, 0.21998000, 0.13333333, 0.38375000, 0.70000000, 0.98582500, 0.40750000, 0.31200000
0.00671910, 0.00000000, 0.27133333, 0.00000000, 0.53800000, 0.58130000, 0.90300000, 0.23410000, 0.13333333, 0.38375000, 0.70000000, 0.94220000, 0.37025000, 0.33200000
0.00955770, 0.00000000, 0.27133333, 0.00000000, 0.53800000, 0.60470000, 0.88800000, 0.22267000, 0.13333333, 0.38375000, 0.70000000, 0.76595000, 0.43200000, 0.29600000
0.00772990, 0.00000000, 0.27133333, 0.00000000, 0.53800000, 0.64950000, 0.94400000, 0.22273500, 0.13333333, 0.38375000, 0.70000000, 0.96985000, 0.32000000, 0.36800000
0.01002450, 0.00000000, 0.27133333, 0.00000000, 0.53800000, 0.66740000, 0.87300000, 0.21195000, 0.13333333, 0.38375000, 0.70000000, 0.95057500, 0.29950000, 0.42000000
0.01354720, 0.00000000, 0.27133333, 0.00000000, 0.53800000, 0.60720000, 1.00000000, 0.20875000, 0.13333333, 0.38375000, 0.70000000, 0.94182500, 0.32600000, 0.29000000
0.01387990, 0.00000000, 0.27133333, 0.00000000, 0.53800000, 0.59500000, 0.82000000, 0.19950000, 0.13333333, 0.38375000, 0.70000000, 0.58150000, 0.69275000, 0.26400000
0.01151720, 0.00000000, 0.27133333, 0.00000000, 0.53800000, 0.57010000, 0.95000000, 0.18936000, 0.13333333, 0.38375000, 0.70000000, 0.89692500, 0.45875000, 0.26200000
0.01612820, 0.00000000, 0.27133333, 0.00000000, 0.53800000, 0.60960000, 0.96900000, 0.18799000, 0.13333333, 0.38375000, 0.70000000, 0.62077500, 0.50850000, 0.27000000
0.00097440, 0.00000000, 0.19866667, 0.00000000, 0.49900000, 0.58410000, 0.61400000, 0.16889500, 0.16666667, 0.34875000, 0.64000000, 0.94390000, 0.28525000, 0.40000000
0.00080140, 0.00000000, 0.19866667, 0.00000000, 0.49900000, 0.58500000, 0.41500000, 0.19671000, 0.16666667, 0.34875000, 0.64000000, 0.99225000, 0.21925000, 0.42000000
0.00175050, 0.00000000, 0.19866667, 0.00000000, 0.49900000, 0.59660000, 0.30200000, 0.19236500, 0.16666667, 0.34875000, 0.64000000, 0.98357500, 0.25325000, 0.49400000
0.00027630, 0.75000000, 0.09833333, 0.00000000, 0.42800000, 0.65950000, 0.21800000, 0.27005500, 0.10000000, 0.31500000, 0.61000000, 0.98907500, 0.10800000, 0.61600000
0.00127440, 0.00000000, 0.23033333, 0.00000000, 0.44800000, 0.67700000, 0.02900000, 0.28604500, 0.10000000, 0.29125000, 0.59666667, 0.96352500, 0.12100000, 0.53200000
0.00141500, 0.00000000, 0.23033333, 0.00000000, 0.44800000, 0.61690000, 0.06600000, 0.28604500, 0.10000000, 0.29125000, 0.59666667, 0.95842500, 0.14525000, 0.50600000
0.00159360, 0.00000000, 0.23033333, 0.00000000, 0.44800000, 0.62110000, 0.06500000, 0.28604500, 0.10000000, 0.29125000, 0.59666667, 0.98615000, 0.18600000, 0.49400000
0.00122690, 0.00000000, 0.23033333, 0.00000000, 0.44800000, 0.60690000, 0.40000000, 0.28604500, 0.10000000, 0.29125000, 0.59666667, 0.97347500, 0.23875000, 0.42400000
0.00188360, 0.00000000, 0.23033333, 0.00000000, 0.44800000, 0.57860000, 0.33300000, 0.25502000, 0.10000000, 0.29125000, 0.59666667, 0.99225000, 0.35375000, 0.40000000
0.00229270, 0.00000000, 0.23033333, 0.00000000, 0.44800000, 0.60300000, 0.85500000, 0.28447000, 0.10000000, 0.29125000, 0.59666667, 0.98185000, 0.47000000, 0.33200000
0.00253870, 0.00000000, 0.23033333, 0.00000000, 0.44800000, 0.53990000, 0.95300000, 0.29350000, 0.10000000, 0.29125000, 0.59666667, 0.99225000, 0.77025000, 0.28800000
0.00219770, 0.00000000, 0.23033333, 0.00000000, 0.44800000, 0.56020000, 0.62000000, 0.30438500, 0.10000000, 0.29125000, 0.59666667, 0.99225000, 0.40500000, 0.38800000
0.00043370, 0.21000000, 0.18800000, 0.00000000, 0.43900000, 0.61150000, 0.63000000, 0.34073500, 0.13333333, 0.30375000, 0.56000000, 0.98492500, 0.23575000, 0.41000000
0.00053600, 0.21000000, 0.18800000, 0.00000000, 0.43900000, 0.65110000, 0.21100000, 0.34073500, 0.13333333, 0.30375000, 0.56000000, 0.99225000, 0.13200000, 0.50000000
0.00049810, 0.21000000, 0.18800000, 0.00000000, 0.43900000, 0.59980000, 0.21400000, 0.34073500, 0.13333333, 0.30375000, 0.56000000, 0.99225000, 0.21075000, 0.46800000
0.00013600, 0.75000000, 0.13333333, 0.00000000, 0.41000000, 0.58880000, 0.47600000, 0.36598500, 0.10000000, 0.58625000, 0.70333333, 0.99225000, 0.37000000, 0.37800000
0.00020550, 0.85000000, 0.02466667, 0.00000000, 0.41000000, 0.63830000, 0.35700000, 0.45938000, 0.06666667, 0.39125000, 0.57666667, 0.99225000, 0.14425000, 0.49400000
0.00014320, 1.00000000, 0.04400000, 0.00000000, 0.41100000, 0.68160000, 0.40500000, 0.41624000, 0.16666667, 0.32000000, 0.50333333, 0.98225000, 0.09875000, 0.63200000
0.00154450, 0.25000000, 0.17100000, 0.00000000, 0.45300000, 0.61450000, 0.29200000, 0.39074000, 0.26666667, 0.35500000, 0.65666667, 0.97670000, 0.17150000, 0.46600000
0.00103280, 0.25000000, 0.17100000, 0.00000000, 0.45300000, 0.59270000, 0.47200000, 0.34660000, 0.26666667, 0.35500000, 0.65666667, 0.99225000, 0.23050000, 0.39200000
0.00171710, 0.25000000, 0.17100000, 0.00000000, 0.45300000, 0.59660000, 0.93400000, 0.34092500, 0.26666667, 0.35500000, 0.65666667, 0.94520000, 0.36100000, 0.32000000
0.00110270, 0.25000000, 0.17100000, 0.00000000, 0.45300000, 0.64560000, 0.67800000, 0.36127500, 0.26666667, 0.35500000, 0.65666667, 0.99225000, 0.16825000, 0.44400000
0.00126500, 0.25000000, 0.17100000, 0.00000000, 0.45300000, 0.67620000, 0.43400000, 0.39904500, 0.26666667, 0.35500000, 0.65666667, 0.98895000, 0.23750000, 0.50000000
0.00019510, 0.17500000, 0.04600000, 0.00000000, 0.41610000, 0.71040000, 0.59500000, 0.46114500, 0.10000000, 0.27000000, 0.62000000, 0.98310000, 0.20125000, 0.66000000
0.00043790, 0.80000000, 0.11233333, 0.00000000, 0.39800000, 0.57870000, 0.31100000, 0.33057500, 0.13333333, 0.42125000, 0.53666667, 0.99225000, 0.25600000, 0.38800000
0.00057890, 0.12500000, 0.20233333, 0.00000000, 0.40900000, 0.58780000, 0.21400000, 0.32490000, 0.13333333, 0.43125000, 0.63000000, 0.99052500, 0.20250000, 0.44000000
0.00135540, 0.12500000, 0.20233333, 0.00000000, 0.40900000, 0.55940000, 0.36800000, 0.32490000, 0.13333333, 0.43125000, 0.63000000, 0.99225000, 0.32725000, 0.34800000
0.00128160, 0.12500000, 0.20233333, 0.00000000, 0.40900000, 0.58850000, 0.33000000, 0.32490000, 0.13333333, 0.43125000, 0.63000000, 0.99225000, 0.21975000, 0.41800000
0.00158760, 0.00000000, 0.36033333, 0.00000000, 0.41300000, 0.59610000, 0.17500000, 0.26436500, 0.13333333, 0.38125000, 0.64000000, 0.94235000, 0.24700000, 0.43400000
0.00091640, 0.00000000, 0.36033333, 0.00000000, 0.41300000, 0.60650000, 0.07800000, 0.26436500, 0.13333333, 0.38125000, 0.64000000, 0.97727500, 0.13800000, 0.45600000
0.00195390, 0.00000000, 0.36033333, 0.00000000, 0.41300000, 0.62450000, 0.06200000, 0.26436500, 0.13333333, 0.38125000, 0.64000000, 0.94292500, 0.18850000, 0.46800000
0.00078960, 0.00000000, 0.42766667, 0.00000000, 0.43700000, 0.62730000, 0.06000000, 0.21257500, 0.16666667, 0.49750000, 0.62333333, 0.98730000, 0.16950000, 0.48200000
0.00101530, 0.00000000, 0.42766667, 0.00000000, 0.43700000, 0.62790000, 0.74500000, 0.20261000, 0.16666667, 0.49750000, 0.62333333, 0.93415000, 0.29925000, 0.40000000
0.00087070, 0.00000000, 0.42766667, 0.00000000, 0.43700000, 0.61400000, 0.45800000, 0.20452500, 0.16666667, 0.49750000, 0.62333333, 0.96740000, 0.25675000, 0.41600000
0.00056460, 0.00000000, 0.42766667, 0.00000000, 0.43700000, 0.62320000, 0.53700000, 0.25070500, 0.16666667, 0.49750000, 0.62333333, 0.96600000, 0.30850000, 0.42400000
0.00083870, 0.00000000, 0.42766667, 0.00000000, 0.43700000, 0.58740000, 0.36600000, 0.22513000, 0.16666667, 0.49750000, 0.62333333, 0.99015000, 0.22750000, 0.40600000
0.00044620, 0.25000000, 0.16200000, 0.00000000, 0.42600000, 0.66190000, 0.70400000, 0.27003500, 0.13333333, 0.35125000, 0.63333333, 0.98907500, 0.18050000, 0.47800000
0.00036590, 0.25000000, 0.16200000, 0.00000000, 0.42600000, 0.63020000, 0.32200000, 0.27003500, 0.13333333, 0.35125000, 0.63333333, 0.99225000, 0.16800000, 0.49600000
0.00035510, 0.25000000, 0.16200000, 0.00000000, 0.42600000, 0.61670000, 0.46700000, 0.27003500, 0.13333333, 0.35125000, 0.63333333, 0.97660000, 0.18775000, 0.45800000
0.00050590, 0.00000000, 0.14966667, 0.00000000, 0.44900000, 0.63890000, 0.48000000, 0.23897000, 0.10000000, 0.30875000, 0.61666667, 0.99225000, 0.24050000, 0.47800000
0.00051880, 0.00000000, 0.14966667, 0.00000000, 0.44900000, 0.60150000, 0.45100000, 0.22136000, 0.10000000, 0.30875000, 0.61666667, 0.98997500, 0.32150000, 0.45000000
0.00071510, 0.00000000, 0.14966667, 0.00000000, 0.44900000, 0.61210000, 0.56800000, 0.18738000, 0.10000000, 0.30875000, 0.61666667, 0.98787500, 0.21100000, 0.44400000
0.00056600, 0.00000000, 0.11366667, 0.00000000, 0.48900000, 0.70070000, 0.86300000, 0.17108500, 0.06666667, 0.33750000, 0.59333333, 0.99225000, 0.13750000, 0.47200000
0.00053020, 0.00000000, 0.11366667, 0.00000000, 0.48900000, 0.70790000, 0.63100000, 0.17072500, 0.06666667, 0.33750000, 0.59333333, 0.99015000, 0.14250000, 0.57400000
0.00039320, 0.00000000, 0.11366667, 0.00000000, 0.48900000, 0.64050000, 0.73900000, 0.15460500, 0.06666667, 0.33750000, 0.59333333, 0.98387500, 0.20500000, 0.44000000
0.00042030, 0.28000000, 0.50133333, 0.00000000, 0.46400000, 0.64420000, 0.53600000, 0.18329500, 0.13333333, 0.33750000, 0.60666667, 0.98752500, 0.20400000, 0.45800000
0.00028750, 0.28000000, 0.50133333, 0.00000000, 0.46400000, 0.62110000, 0.28900000, 0.18329500, 0.13333333, 0.33750000, 0.60666667, 0.99082500, 0.15525000, 0.50000000
0.00042940, 0.28000000, 0.50133333, 0.00000000, 0.46400000, 0.62490000, 0.77300000, 0.18075000, 0.13333333, 0.33750000, 0.60666667, 0.99225000, 0.26475000, 0.41200000
0.00115040, 0.00000000, 0.09633333, 0.00000000, 0.44500000, 0.61630000, 0.69600000, 0.17476000, 0.06666667, 0.34500000, 0.60000000, 0.97957500, 0.28350000, 0.42800000
0.00120830, 0.00000000, 0.09633333, 0.00000000, 0.44500000, 0.80690000, 0.76000000, 0.17476000, 0.06666667, 0.34500000, 0.60000000, 0.99225000, 0.10525000, 0.77400000
0.00081870, 0.00000000, 0.09633333, 0.00000000, 0.44500000, 0.78200000, 0.36900000, 0.17476000, 0.06666667, 0.34500000, 0.60000000, 0.98382500, 0.08925000, 0.87600000
0.00068600, 0.00000000, 0.09633333, 0.00000000, 0.44500000, 0.74160000, 0.62500000, 0.17476000, 0.06666667, 0.34500000, 0.60000000, 0.99225000, 0.15475000, 0.66400000
0.00114320, 0.00000000, 0.28533333, 0.00000000, 0.52000000, 0.67810000, 0.71300000, 0.14280500, 0.16666667, 0.48000000, 0.69666667, 0.98895000, 0.19175000, 0.53000000
0.00228760, 0.00000000, 0.28533333, 0.00000000, 0.52000000, 0.64050000, 0.85400000, 0.13573500, 0.16666667, 0.48000000, 0.69666667, 0.17700000, 0.26575000, 0.37200000
0.00211610, 0.00000000, 0.28533333, 0.00000000, 0.52000000, 0.61370000, 0.87400000, 0.13573500, 0.16666667, 0.48000000, 0.69666667, 0.98617500, 0.33600000, 0.38600000
0.00139600, 0.00000000, 0.28533333, 0.00000000, 0.52000000, 0.61670000, 0.90000000, 0.12105000, 0.16666667, 0.48000000, 0.69666667, 0.98172500, 0.30825000, 0.40200000
0.00171200, 0.00000000, 0.28533333, 0.00000000, 0.52000000, 0.58360000, 0.91900000, 0.11055000, 0.16666667, 0.48000000, 0.69666667, 0.98917500, 0.46650000, 0.39000000
0.00131170, 0.00000000, 0.28533333, 0.00000000, 0.52000000, 0.61270000, 0.85200000, 0.10612000, 0.16666667, 0.48000000, 0.69666667, 0.96922500, 0.35225000, 0.40800000
0.00128020, 0.00000000, 0.28533333, 0.00000000, 0.52000000, 0.64740000, 0.97100000, 0.12164500, 0.16666667, 0.48000000, 0.69666667, 0.98810000, 0.30675000, 0.39600000
0.00263630, 0.00000000, 0.28533333, 0.00000000, 0.52000000, 0.62290000, 0.91200000, 0.12725500, 0.16666667, 0.48000000, 0.69666667, 0.97807500, 0.38875000, 0.38800000
0.00100840, 0.00000000, 0.33366667, 0.00000000, 0.54700000, 0.67150000, 0.81600000, 0.13387500, 0.20000000, 0.54000000, 0.59333333, 0.98897500, 0.25400000, 0.45600000
0.00123290, 0.00000000, 0.33366667, 0.00000000, 0.54700000, 0.59130000, 0.92900000, 0.11767000, 0.20000000, 0.54000000, 0.59333333, 0.98737500, 0.40525000, 0.37600000
0.00222120, 0.00000000, 0.33366667, 0.00000000, 0.54700000, 0.60920000, 0.95400000, 0.12740000, 0.20000000, 0.54000000, 0.59333333, 0.99225000, 0.42725000, 0.37400000
0.00142310, 0.00000000, 0.33366667, 0.00000000, 0.54700000, 0.62540000, 0.84200000, 0.11282500, 0.20000000, 0.54000000, 0.59333333, 0.97185000, 0.26125000, 0.37000000
0.00131580, 0.00000000, 0.33366667, 0.00000000, 0.54700000, 0.61760000, 0.72500000, 0.13650500, 0.20000000, 0.54000000, 0.59333333, 0.98325000, 0.30100000, 0.42400000
0.00150980, 0.00000000, 0.33366667, 0.00000000, 0.54700000, 0.60210000, 0.82600000, 0.13737000, 0.20000000, 0.54000000, 0.59333333, 0.98627500, 0.25750000, 0.38400000
0.00130580, 0.00000000, 0.33366667, 0.00000000, 0.54700000, 0.58720000, 0.73100000, 0.12387500, 0.20000000, 0.54000000, 0.59333333, 0.84657500, 0.38425000, 0.40800000
0.00144760, 0.00000000, 0.33366667, 0.00000000, 0.54700000, 0.57310000, 0.65200000, 0.13796000, 0.20000000, 0.54000000, 0.59333333, 0.97875000, 0.34025000, 0.38600000
0.00071650, 0.00000000, 0.85500000, 0.00000000, 0.58100000, 0.60040000, 0.84100000, 0.10987000, 0.06666667, 0.23500000, 0.63666667, 0.94417500, 0.35675000, 0.40600000
0.00092990, 0.00000000, 0.85500000, 0.00000000, 0.58100000, 0.59610000, 0.92900000, 0.10434500, 0.06666667, 0.23500000, 0.63666667, 0.94522500, 0.44825000, 0.41000000
0.00150380, 0.00000000, 0.85500000, 0.00000000, 0.58100000, 0.58560000, 0.97000000, 0.09722000, 0.06666667, 0.23500000, 0.63666667, 0.92577500, 0.63525000, 0.34600000
0.00098490, 0.00000000, 0.85500000, 0.00000000, 0.58100000, 0.58790000, 0.95800000, 0.10031500, 0.06666667, 0.23500000, 0.63666667, 0.94845000, 0.43950000, 0.37600000
0.00387350, 0.00000000, 0.85500000, 0.00000000, 0.58100000, 0.56130000, 0.95600000, 0.08786000, 0.06666667, 0.23500000, 0.63666667, 0.89822500, 0.68150000, 0.31400000
0.00259150, 0.00000000, 0.72966667, 0.00000000, 0.62400000, 0.56930000, 0.96000000, 0.08941500, 0.13333333, 0.54625000, 0.70666667, 0.98027500, 0.42975000, 0.32400000
0.00325430, 0.00000000, 0.72966667, 0.00000000, 0.62400000, 0.64310000, 0.98800000, 0.09062500, 0.13333333, 0.54625000, 0.70666667, 0.99225000, 0.38475000, 0.36000000
0.00881250, 0.00000000, 0.72966667, 0.00000000, 0.62400000, 0.56370000, 0.94700000, 0.09899500, 0.13333333, 0.54625000, 0.70666667, 0.99225000, 0.45850000, 0.28600000
0.01192940, 0.00000000, 0.72966667, 0.00000000, 0.62400000, 0.63260000, 0.97700000, 0.11355000, 0.13333333, 0.54625000, 0.70666667, 0.99225000, 0.30650000, 0.39200000
0.00590050, 0.00000000, 0.72966667, 0.00000000, 0.62400000, 0.63720000, 0.97900000, 0.11637000, 0.13333333, 0.54625000, 0.70666667, 0.96440000, 0.27800000, 0.46000000
0.00329820, 0.00000000, 0.72966667, 0.00000000, 0.62400000, 0.58220000, 0.95400000, 0.12349500, 0.13333333, 0.54625000, 0.70666667, 0.97172500, 0.37575000, 0.36800000
0.00976170, 0.00000000, 0.72966667, 0.00000000, 0.62400000, 0.57570000, 0.98400000, 0.11730000, 0.13333333, 0.54625000, 0.70666667, 0.65690000, 0.43275000, 0.31200000
0.00322640, 0.00000000, 0.72966667, 0.00000000, 0.62400000, 0.59420000, 0.93500000, 0.09834500, 0.13333333, 0.54625000, 0.70666667, 0.94562500, 0.42250000, 0.34800000
0.00352330, 0.00000000, 0.72966667, 0.00000000, 0.62400000, 0.64540000, 0.98400000, 0.09249000, 0.13333333, 0.54625000, 0.70666667, 0.98520000, 0.36475000, 0.34200000
0.00249800, 0.00000000, 0.72966667, 0.00000000, 0.62400000, 0.58570000, 0.98200000, 0.08343000, 0.13333333, 0.54625000, 0.70666667, 0.98010000, 0.53300000, 0.26600000
0.00544520, 0.00000000, 0.72966667, 0.00000000, 0.62400000, 0.61510000, 0.97900000, 0.08343500, 0.13333333, 0.54625000, 0.70666667, 0.99225000, 0.46150000, 0.35600000
0.01628640, 0.00000000, 0.72966667, 0.00000000, 0.62400000, 0.50190000, 1.00000000, 0.07197000, 0.13333333, 0.54625000, 0.70666667, 0.99225000, 0.86025000, 0.28800000
0.03321050, 0.00000000, 0.65266667, 1.00000000, 0.87100000, 0.54030000, 1.00000000, 0.06608000, 0.16666667, 0.50375000, 0.49000000, 0.99225000, 0.67050000, 0.26800000
0.04097400, 0.00000000, 0.65266667, 0.00000000, 0.87100000, 0.54680000, 1.00000000, 0.07059000, 0.16666667, 0.50375000, 0.49000000, 0.99225000, 0.66050000, 0.31200000
0.02779740, 0.00000000, 0.65266667, 0.00000000, 0.87100000, 0.49030000, 0.97800000, 0.06729500, 0.16666667, 0.50375000, 0.49000000, 0.99225000, 0.73225000, 0.23600000
0.02155050, 0.00000000, 0.65266667, 0.00000000, 0.87100000, 0.56280000, 1.00000000, 0.07583000, 0.16666667, 0.50375000, 0.49000000, 0.42317500, 0.41625000, 0.31200000
0.02368620, 0.00000000, 0.65266667, 0.00000000, 0.87100000, 0.49260000, 0.95700000, 0.07304000, 0.16666667, 0.50375000, 0.49000000, 0.97927500, 0.73825000, 0.29200000
0.02330990, 0.00000000, 0.65266667, 0.00000000, 0.87100000, 0.51860000, 0.93800000, 0.07648000, 0.16666667, 0.50375000, 0.49000000, 0.89247500, 0.70800000, 0.35600000
0.02733970, 0.00000000, 0.65266667, 0.00000000, 0.87100000, 0.55970000, 0.94900000, 0.07628500, 0.16666667, 0.50375000, 0.49000000, 0.87962500, 0.53625000, 0.30800000
0.01496320, 0.00000000, 0.65266667, 0.00000000, 0.87100000, 0.54040000, 1.00000000, 0.07958000, 0.16666667, 0.50375000, 0.49000000, 0.85400000, 0.33200000, 0.39200000
0.01126580, 0.00000000, 0.65266667, 1.00000000, 0.87100000, 0.50120000, 0.88000000, 0.08051000, 0.16666667, 0.50375000, 0.49000000, 0.85820000, 0.30300000, 0.30600000
0.02149180, 0.00000000, 0.65266667, 0.00000000, 0.87100000, 0.57090000, 0.98500000, 0.08116000, 0.16666667, 0.50375000, 0.49000000, 0.65487500, 0.39475000, 0.38800000
0.01413850, 0.00000000, 0.65266667, 1.00000000, 0.87100000, 0.61290000, 0.96000000, 0.08747000, 0.16666667, 0.50375000, 0.49000000, 0.80255000, 0.37800000, 0.34000000
0.02446680, 0.00000000, 0.65266667, 0.00000000, 0.87100000, 0.52720000, 0.94000000, 0.08682000, 0.16666667, 0.50375000, 0.49000000, 0.22157500, 0.40350000, 0.26200000
0.01223580, 0.00000000, 0.65266667, 0.00000000, 0.60500000, 0.69430000, 0.97400000, 0.09386500, 0.16666667, 0.50375000, 0.49000000, 0.90857500, 0.11475000, 0.82600000
0.01342840, 0.00000000, 0.65266667, 0.00000000, 0.60500000, 0.60660000, 1.00000000, 0.08786500, 0.16666667, 0.50375000, 0.49000000, 0.88472500, 0.16075000, 0.48600000
0.01425020, 0.00000000, 0.65266667, 0.00000000, 0.87100000, 0.65100000, 1.00000000, 0.08829500, 0.16666667, 0.50375000, 0.49000000, 0.91077500, 0.18475000, 0.46600000
0.01463360, 0.00000000, 0.65266667, 0.00000000, 0.60500000, 0.74890000, 0.90800000, 0.09854500, 0.16666667, 0.50375000, 0.49000000, 0.93607500, 0.04325000, 1.00000000
0.01833770, 0.00000000, 0.65266667, 1.00000000, 0.60500000, 0.78020000, 0.98200000, 0.10203500, 0.16666667, 0.50375000, 0.49000000, 0.97402500, 0.04800000, 1.00000000
0.01519020, 0.00000000, 0.65266667, 1.00000000, 0.60500000, 0.83750000, 0.93900000, 0.10810000, 0.16666667, 0.50375000, 0.49000000, 0.97112500, 0.08300000, 1.00000000
0.02242360, 0.00000000, 0.65266667, 0.00000000, 0.60500000, 0.58540000, 0.91800000, 0.12110000, 0.16666667, 0.50375000, 0.49000000, 0.98777500, 0.29100000, 0.45400000
0.02010190, 0.00000000, 0.65266667, 0.00000000, 0.60500000, 0.79290000, 0.96200000, 0.10229500, 0.16666667, 0.50375000, 0.49000000, 0.92325000, 0.09250000, 1.00000000
0.01800280, 0.00000000, 0.65266667, 0.00000000, 0.60500000, 0.58770000, 0.79200000, 0.12129500, 0.16666667, 0.50375000, 0.49000000, 0.56902500, 0.30350000, 0.47600000
0.02300400, 0.00000000, 0.65266667, 0.00000000, 0.60500000, 0.63190000, 0.96100000, 0.10500000, 0.16666667, 0.50375000, 0.49000000, 0.74272500, 0.27750000, 0.47600000
0.02449530, 0.00000000, 0.65266667, 0.00000000, 0.60500000, 0.64020000, 0.95200000, 0.11312500, 0.16666667, 0.50375000, 0.49000000, 0.82510000, 0.28300000, 0.44600000
0.02313900, 0.00000000, 0.65266667, 0.00000000, 0.60500000, 0.58800000, 0.97300000, 0.11943500, 0.16666667, 0.50375000, 0.49000000, 0.87032500, 0.30075000, 0.38200000
0.00139140, 0.00000000, 0.13500000, 0.00000000, 0.51000000, 0.55720000, 0.88500000, 0.12980500, 0.16666667, 0.37000000, 0.55333333, 0.99225000, 0.36725000, 0.46200000
0.00091780, 0.00000000, 0.13500000, 0.00000000, 0.51000000, 0.64160000, 0.84100000, 0.13231500, 0.16666667, 0.37000000, 0.55333333, 0.98875000, 0.22600000, 0.47200000
0.00084470, 0.00000000, 0.13500000, 0.00000000, 0.51000000, 0.58590000, 0.68700000, 0.13509500, 0.16666667, 0.37000000, 0.55333333, 0.98307500, 0.24100000, 0.45200000
0.00070220, 0.00000000, 0.13500000, 0.00000000, 0.51000000, 0.60200000, 0.47200000, 0.17774500, 0.16666667, 0.37000000, 0.55333333, 0.98307500, 0.25275000, 0.46400000
0.00054250, 0.00000000, 0.13500000, 0.00000000, 0.51000000, 0.63150000, 0.73400000, 0.16587500, 0.16666667, 0.37000000, 0.55333333, 0.98900000, 0.15725000, 0.49200000
0.00066420, 0.00000000, 0.13500000, 0.00000000, 0.51000000, 0.68600000, 0.74400000, 0.14576500, 0.16666667, 0.37000000, 0.55333333, 0.97817500, 0.17300000, 0.59800000
0.00057800, 0.00000000, 0.08200000, 0.00000000, 0.48800000, 0.69800000, 0.58400000, 0.14145000, 0.10000000, 0.24125000, 0.59333333, 0.99225000, 0.12600000, 0.74400000
0.00068880, 0.00000000, 0.08200000, 0.00000000, 0.48800000, 0.61440000, 0.62200000, 0.12989500, 0.10000000, 0.24125000, 0.59333333, 0.99225000, 0.23625000, 0.72400000
0.00091030, 0.00000000, 0.08200000, 0.00000000, 0.48800000, 0.71550000, 0.92200000, 0.13503000, 0.10000000, 0.24125000, 0.59333333, 0.98530000, 0.12050000, 0.75800000
0.00100080, 0.00000000, 0.08200000, 0.00000000, 0.48800000, 0.65630000, 0.95600000, 0.14235000, 0.10000000, 0.24125000, 0.59333333, 0.99225000, 0.14200000, 0.65000000
0.00083080, 0.00000000, 0.08200000, 0.00000000, 0.48800000, 0.56040000, 0.89800000, 0.14939500, 0.10000000, 0.24125000, 0.59333333, 0.97750000, 0.34950000, 0.52800000
0.00056020, 0.00000000, 0.08200000, 0.00000000, 0.48800000, 0.78310000, 0.53600000, 0.15996000, 0.10000000, 0.24125000, 0.59333333, 0.98157500, 0.11125000, 1.00000000
0.00078750, 0.45000000, 0.11466667, 0.00000000, 0.43700000, 0.67820000, 0.41100000, 0.18943000, 0.16666667, 0.49750000, 0.50666667, 0.98467500, 0.16700000, 0.64000000
0.00125790, 0.45000000, 0.11466667, 0.00000000, 0.43700000, 0.65560000, 0.29100000, 0.22833500, 0.16666667, 0.49750000, 0.50666667, 0.95710000, 0.11400000, 0.59600000
0.00083700, 0.45000000, 0.11466667, 0.00000000, 0.43700000, 0.71850000, 0.38900000, 0.22833500, 0.16666667, 0.49750000, 0.50666667, 0.99225000, 0.13475000, 0.69800000
0.00069110, 0.45000000, 0.11466667, 0.00000000, 0.43700000, 0.67390000, 0.30800000, 0.32399000, 0.16666667, 0.49750000, 0.50666667, 0.97427500, 0.11725000, 0.61000000
0.00086640, 0.45000000, 0.11466667, 0.00000000, 0.43700000, 0.71780000, 0.26300000, 0.32399000, 0.16666667, 0.49750000, 0.50666667, 0.97622500, 0.07175000, 0.72800000
0.00021870, 0.60000000, 0.09766667, 0.00000000, 0.40100000, 0.68000000, 0.09900000, 0.31098000, 0.03333333, 0.33125000, 0.52000000, 0.98342500, 0.12575000, 0.62200000
0.00014390, 0.60000000, 0.09766667, 0.00000000, 0.40100000, 0.66040000, 0.18800000, 0.31098000, 0.03333333, 0.33125000, 0.52000000, 0.94175000, 0.10950000, 0.58200000
0.00040110, 0.80000000, 0.05066667, 0.00000000, 0.40400000, 0.72870000, 0.34100000, 0.36545000, 0.06666667, 0.41125000, 0.42000000, 0.99225000, 0.10200000, 0.66600000
0.00046660, 0.80000000, 0.05066667, 0.00000000, 0.40400000, 0.71070000, 0.36600000, 0.36545000, 0.06666667, 0.41125000, 0.42000000, 0.88577500, 0.21525000, 0.60600000
0.00037680, 0.80000000, 0.05066667, 0.00000000, 0.40400000, 0.72740000, 0.38300000, 0.36545000, 0.06666667, 0.41125000, 0.42000000, 0.98050000, 0.16550000, 0.69200000
0.00031500, 0.95000000, 0.04900000, 0.00000000, 0.40300000, 0.69750000, 0.15300000, 0.38267000, 0.10000000, 0.50250000, 0.56666667, 0.99225000, 0.11400000, 0.69800000
0.00034450, 0.82500000, 0.06766667, 0.00000000, 0.41500000, 0.61620000, 0.38400000, 0.31350000, 0.06666667, 0.43500000, 0.49000000, 0.98442500, 0.18575000, 0.48200000
0.00021770, 0.82500000, 0.06766667, 0.00000000, 0.41500000, 0.76100000, 0.15700000, 0.31350000, 0.06666667, 0.43500000, 0.49000000, 0.98845000, 0.07775000, 0.84600000
0.00035100, 0.95000000, 0.08933333, 0.00000000, 0.41610000, 0.78530000, 0.33200000, 0.25590000, 0.13333333, 0.28000000, 0.49000000, 0.98195000, 0.09525000, 0.97000000
0.00020090, 0.95000000, 0.08933333, 0.00000000, 0.41610000, 0.80340000, 0.31900000, 0.25590000, 0.13333333, 0.28000000, 0.49000000, 0.97637500, 0.07200000, 1.00000000
0.00229690, 0.00000000, 0.35300000, 0.00000000, 0.48900000, 0.63260000, 0.52500000, 0.21774500, 0.13333333, 0.34625000, 0.62000000, 0.98717500, 0.27425000, 0.48800000
0.00251990, 0.00000000, 0.35300000, 0.00000000, 0.48900000, 0.57830000, 0.72700000, 0.21774500, 0.13333333, 0.34625000, 0.62000000, 0.97357500, 0.45150000, 0.45000000
0.00135870, 0.00000000, 0.35300000, 1.00000000, 0.48900000, 0.60640000, 0.59100000, 0.21196000, 0.13333333, 0.34625000, 0.62000000, 0.95330000, 0.36650000, 0.48800000
0.00435710, 0.00000000, 0.35300000, 1.00000000, 0.48900000, 0.53440000, 1.00000000, 0.19375000, 0.13333333, 0.34625000, 0.62000000, 0.99225000, 0.57725000, 0.40000000
0.00375780, 0.00000000, 0.35300000, 1.00000000, 0.48900000, 0.54040000, 0.88600000, 0.18325000, 0.13333333, 0.34625000, 0.62000000, 0.98810000, 0.59950000, 0.38600000
0.00217190, 0.00000000, 0.35300000, 1.00000000, 0.48900000, 0.58070000, 0.53800000, 0.18263000, 0.13333333, 0.34625000, 0.62000000, 0.97735000, 0.40075000, 0.44800000
0.00140520, 0.00000000, 0.35300000, 0.00000000, 0.48900000, 0.63750000, 0.32300000, 0.19727000, 0.13333333, 0.34625000, 0.62000000, 0.96452500, 0.23450000, 0.56200000
0.00289550, 0.00000000, 0.35300000, 0.00000000, 0.48900000, 0.54120000, 0.09800000, 0.17937500, 0.13333333, 0.34625000, 0.62000000, 0.87232500, 0.73875000, 0.47400000
0.00045600, 0.00000000, 0.46300000, 1.00000000, 0.55000000, 0.58880000, 0.56000000, 0.15560500, 0.16666667, 0.34500000, 0.54666667, 0.98200000, 0.33775000, 0.46600000
0.00070130, 0.00000000, 0.46300000, 0.00000000, 0.55000000, 0.66420000, 0.85100000, 0.17105500, 0.16666667, 0.34500000, 0.54666667, 0.98195000, 0.24225000, 0.57400000
0.00110690, 0.00000000, 0.46300000, 1.00000000, 0.55000000, 0.59510000, 0.93800000, 0.14446500, 0.16666667, 0.34500000, 0.54666667, 0.99225000, 0.44800000, 0.43000000
0.00114250, 0.00000000, 0.46300000, 1.00000000, 0.55000000, 0.63730000, 0.92400000, 0.16816500, 0.16666667, 0.34500000, 0.54666667, 0.98435000, 0.26250000, 0.46000000
0.00407710, 0.00000000, 0.20666667, 1.00000000, 0.50700000, 0.61640000, 0.91300000, 0.15240000, 0.26666667, 0.38375000, 0.58000000, 0.98810000, 0.53650000, 0.43400000
0.00623560, 0.00000000, 0.20666667, 1.00000000, 0.50700000, 0.68790000, 0.77700000, 0.16360500, 0.26666667, 0.38375000, 0.58000000, 0.97597500, 0.24825000, 0.55000000
0.00614700, 0.00000000, 0.20666667, 0.00000000, 0.50700000, 0.66180000, 0.80800000, 0.16360500, 0.26666667, 0.38375000, 0.58000000, 0.99225000, 0.19000000, 0.60200000
0.00315330, 0.00000000, 0.20666667, 0.00000000, 0.50400000, 0.82660000, 0.78300000, 0.14472000, 0.26666667, 0.38375000, 0.58000000, 0.96262500, 0.10350000, 0.89600000
0.00382140, 0.00000000, 0.20666667, 0.00000000, 0.50400000, 0.80400000, 0.86500000, 0.16078500, 0.26666667, 0.38375000, 0.58000000, 0.96845000, 0.07825000, 0.75200000
0.00412380, 0.00000000, 0.20666667, 0.00000000, 0.50400000, 0.71630000, 0.79900000, 0.16078500, 0.26666667, 0.38375000, 0.58000000, 0.93020000, 0.15900000, 0.63200000
0.00298190, 0.00000000, 0.20666667, 0.00000000, 0.50400000, 0.76860000, 0.17000000, 0.16875500, 0.26666667, 0.38375000, 0.58000000, 0.94377500, 0.09800000, 0.93400000
0.00441780, 0.00000000, 0.20666667, 0.00000000, 0.50400000, 0.65520000, 0.21400000, 0.16875500, 0.26666667, 0.38375000, 0.58000000, 0.95085000, 0.09400000, 0.63000000
0.00462960, 0.00000000, 0.20666667, 0.00000000, 0.50400000, 0.74120000, 0.76900000, 0.18357500, 0.26666667, 0.38375000, 0.58000000, 0.94035000, 0.13125000, 0.63400000
0.00575290, 0.00000000, 0.20666667, 0.00000000, 0.50700000, 0.83370000, 0.73300000, 0.19192000, 0.26666667, 0.38375000, 0.58000000, 0.96477500, 0.06175000, 0.83400000
0.00331470, 0.00000000, 0.20666667, 0.00000000, 0.50700000, 0.82470000, 0.70400000, 0.18259500, 0.26666667, 0.38375000, 0.58000000, 0.94737500, 0.09875000, 0.96600000
0.00447910, 0.00000000, 0.20666667, 1.00000000, 0.50700000, 0.67260000, 0.66500000, 0.18259500, 0.26666667, 0.38375000, 0.58000000, 0.90050000, 0.20125000, 0.58000000
0.00520580, 0.00000000, 0.20666667, 1.00000000, 0.50700000, 0.66310000, 0.76500000, 0.20740000, 0.26666667, 0.38375000, 0.58000000, 0.97112500, 0.23850000, 0.50200000
0.00511830, 0.00000000, 0.20666667, 0.00000000, 0.50700000, 0.73580000, 0.71600000, 0.20740000, 0.26666667, 0.38375000, 0.58000000, 0.97517500, 0.11825000, 0.63000000
0.00082440, 0.30000000, 0.16433333, 0.00000000, 0.42800000, 0.64810000, 0.18500000, 0.30949500, 0.20000000, 0.37500000, 0.55333333, 0.94852500, 0.15900000, 0.47400000
0.00092520, 0.30000000, 0.16433333, 0.00000000, 0.42800000, 0.66060000, 0.42200000, 0.30949500, 0.20000000, 0.37500000, 0.55333333, 0.95945000, 0.18425000, 0.46600000
0.00106120, 0.30000000, 0.16433333, 0.00000000, 0.42800000, 0.60950000, 0.65100000, 0.31680500, 0.20000000, 0.37500000, 0.55333333, 0.98655000, 0.31000000, 0.40200000
0.00102900, 0.30000000, 0.16433333, 0.00000000, 0.42800000, 0.63580000, 0.52900000, 0.35177500, 0.20000000, 0.37500000, 0.55333333, 0.93187500, 0.28050000, 0.44400000
0.00127570, 0.30000000, 0.16433333, 0.00000000, 0.42800000, 0.63930000, 0.07800000, 0.35177500, 0.20000000, 0.37500000, 0.55333333, 0.93677500, 0.12975000, 0.47400000
0.00206080, 0.22000000, 0.19533333, 0.00000000, 0.43100000, 0.55930000, 0.76500000, 0.39774500, 0.23333333, 0.41250000, 0.63666667, 0.93122500, 0.31250000, 0.35200000
0.00339830, 0.22000000, 0.19533333, 0.00000000, 0.43100000, 0.61080000, 0.34900000, 0.40277500, 0.23333333, 0.41250000, 0.63666667, 0.97545000, 0.22900000, 0.48600000
0.00196570, 0.22000000, 0.19533333, 0.00000000, 0.43100000, 0.62260000, 0.79200000, 0.40277500, 0.23333333, 0.41250000, 0.63666667, 0.94035000, 0.25375000, 0.41000000
0.00164390, 0.22000000, 0.19533333, 0.00000000, 0.43100000, 0.64330000, 0.49100000, 0.39132500, 0.23333333, 0.41250000, 0.63666667, 0.93677500, 0.23800000, 0.49000000
0.00190730, 0.22000000, 0.19533333, 0.00000000, 0.43100000, 0.67180000, 0.17500000, 0.39132500, 0.23333333, 0.41250000, 0.63666667, 0.98435000, 0.16400000, 0.52400000
0.00214090, 0.22000000, 0.19533333, 0.00000000, 0.43100000, 0.64380000, 0.08900000, 0.36983500, 0.23333333, 0.41250000, 0.63666667, 0.94267500, 0.08975000, 0.49600000
0.00082210, 0.22000000, 0.19533333, 0.00000000, 0.43100000, 0.69570000, 0.06800000, 0.44533500, 0.23333333, 0.41250000, 0.63666667, 0.96522500, 0.08825000, 0.59200000
0.00368940, 0.22000000, 0.19533333, 0.00000000, 0.43100000, 0.82590000, 0.08400000, 0.44533500, 0.23333333, 0.41250000, 0.63666667, 0.99225000, 0.08850000, 0.85600000
0.00048190, 0.80000000, 0.12133333, 0.00000000, 0.39200000, 0.61080000, 0.32000000, 0.46101500, 0.03333333, 0.39375000, 0.54666667, 0.98222500, 0.16425000, 0.43800000
0.00015380, 0.90000000, 0.12500000, 0.00000000, 0.39400000, 0.74540000, 0.34200000, 0.31680500, 0.10000000, 0.30500000, 0.53000000, 0.96585000, 0.07775000, 0.88000000
0.00611540, 0.20000000, 0.13233333, 0.00000000, 0.64700000, 0.87040000, 0.86900000, 0.09005000, 0.16666667, 0.33000000, 0.43333333, 0.97425000, 0.12800000, 1.00000000
0.00663510, 0.20000000, 0.13233333, 0.00000000, 0.64700000, 0.73330000, 1.00000000, 0.09473000, 0.16666667, 0.33000000, 0.43333333, 0.95822500, 0.19475000, 0.72000000
0.00656650, 0.20000000, 0.13233333, 0.00000000, 0.64700000, 0.68420000, 1.00000000, 0.10053500, 0.16666667, 0.33000000, 0.43333333, 0.97982500, 0.17250000, 0.60200000
0.00534120, 0.20000000, 0.13233333, 0.00000000, 0.64700000, 0.75200000, 0.89400000, 0.10699000, 0.16666667, 0.33000000, 0.43333333, 0.97092500, 0.18150000, 0.86200000
0.00520140, 0.20000000, 0.13233333, 0.00000000, 0.64700000, 0.83980000, 0.91500000, 0.11442500, 0.16666667, 0.33000000, 0.43333333, 0.96715000, 0.14775000, 0.97600000
0.00825260, 0.20000000, 0.13233333, 0.00000000, 0.64700000, 0.73270000, 0.94500000, 0.10394000, 0.16666667, 0.33000000, 0.43333333, 0.98355000, 0.28125000, 0.62000000
0.00550070, 0.20000000, 0.13233333, 0.00000000, 0.64700000, 0.72060000, 0.91600000, 0.09650500, 0.16666667, 0.33000000, 0.43333333, 0.96972500, 0.20250000, 0.73000000
0.00785700, 0.20000000, 0.13233333, 0.00000000, 0.64700000, 0.70140000, 0.84600000, 0.10664500, 0.16666667, 0.33000000, 0.43333333, 0.96017500, 0.36975000, 0.61400000
0.00578340, 0.20000000, 0.13233333, 0.00000000, 0.57500000, 0.82970000, 0.67000000, 0.12108000, 0.16666667, 0.33000000, 0.43333333, 0.96135000, 0.18600000, 1.00000000
0.00540500, 0.20000000, 0.13233333, 0.00000000, 0.57500000, 0.74700000, 0.52600000, 0.14360000, 0.16666667, 0.33000000, 0.43333333, 0.97575000, 0.07900000, 0.87000000
0.00090650, 0.20000000, 0.23200000, 1.00000000, 0.46400000, 0.59200000, 0.61500000, 0.19587500, 0.10000000, 0.27875000, 0.62000000, 0.97835000, 0.34125000, 0.41400000
0.00162110, 0.20000000, 0.23200000, 0.00000000, 0.46400000, 0.62400000, 0.16300000, 0.22145000, 0.10000000, 0.27875000, 0.62000000, 0.99225000, 0.16475000, 0.50400000
0.00114600, 0.20000000, 0.23200000, 0.00000000, 0.46400000, 0.65380000, 0.58700000, 0.19587500, 0.10000000, 0.27875000, 0.62000000, 0.98740000, 0.19325000, 0.48800000
0.00221880, 0.20000000, 0.23200000, 1.00000000, 0.46400000, 0.76910000, 0.51800000, 0.21832500, 0.10000000, 0.27875000, 0.62000000, 0.97692500, 0.16450000, 0.70400000
0.00056440, 0.40000000, 0.21366667, 1.00000000, 0.44700000, 0.67580000, 0.32900000, 0.20388000, 0.13333333, 0.31750000, 0.58666667, 0.99225000, 0.08825000, 0.64800000
0.00104690, 0.40000000, 0.21366667, 1.00000000, 0.44700000, 0.72670000, 0.49000000, 0.23936000, 0.13333333, 0.31750000, 0.58666667, 0.97312500, 0.15125000, 0.66400000
0.00061270, 0.40000000, 0.21366667, 1.00000000, 0.44700000, 0.68260000, 0.27600000, 0.24314000, 0.13333333, 0.31750000, 0.58666667, 0.98362500, 0.10400000, 0.66200000
0.00079780, 0.40000000, 0.21366667, 0.00000000, 0.44700000, 0.64820000, 0.32100000, 0.20701500, 0.13333333, 0.31750000, 0.58666667, 0.99225000, 0.17975000, 0.58200000
0.00210380, 0.20000000, 0.11100000, 0.00000000, 0.44290000, 0.68120000, 0.32200000, 0.20503500, 0.16666667, 0.27000000, 0.49666667, 0.99225000, 0.12125000, 0.70200000
0.00037050, 0.20000000, 0.11100000, 0.00000000, 0.44290000, 0.69680000, 0.37200000, 0.26223500, 0.16666667, 0.27000000, 0.49666667, 0.98057500, 0.11475000, 0.70800000
0.00061290, 0.20000000, 0.11100000, 1.00000000, 0.44290000, 0.76450000, 0.49700000, 0.26059500, 0.16666667, 0.27000000, 0.49666667, 0.94267500, 0.07525000, 0.92000000
0.00015010, 0.90000000, 0.04033333, 1.00000000, 0.40100000, 0.79230000, 0.24800000, 0.29425000, 0.03333333, 0.24750000, 0.45333333, 0.98880000, 0.07900000, 1.00000000
0.00009060, 0.90000000, 0.09900000, 0.00000000, 0.40000000, 0.70880000, 0.20800000, 0.36536500, 0.03333333, 0.35625000, 0.51000000, 0.98680000, 0.19625000, 0.64400000
0.00019650, 0.80000000, 0.05866667, 0.00000000, 0.38500000, 0.62300000, 0.31500000, 0.45446000, 0.03333333, 0.30125000, 0.60666667, 0.85400000, 0.32325000, 0.40200000
0.00038710, 0.52500000, 0.17733333, 0.00000000, 0.40500000, 0.62090000, 0.31300000, 0.36586000, 0.20000000, 0.36625000, 0.55333333, 0.99225000, 0.17850000, 0.46400000
0.00045900, 0.52500000, 0.17733333, 0.00000000, 0.40500000, 0.63150000, 0.45600000, 0.36586000, 0.20000000, 0.36625000, 0.55333333, 0.99225000, 0.19000000, 0.44600000
0.00042970, 0.52500000, 0.17733333, 0.00000000, 0.40500000, 0.65650000, 0.22900000, 0.36586000, 0.20000000, 0.36625000, 0.55333333, 0.92930000, 0.23775000, 0.49600000
0.00078860, 0.80000000, 0.16500000, 0.00000000, 0.41100000, 0.71480000, 0.27700000, 0.25583500, 0.13333333, 0.30625000, 0.64000000, 0.99225000, 0.08900000, 0.74600000
0.00036150, 0.80000000, 0.16500000, 0.00000000, 0.41100000, 0.66300000, 0.23400000, 0.25583500, 0.13333333, 0.30625000, 0.64000000, 0.99225000, 0.11750000, 0.55800000
0.00082650, 0.00000000, 0.46400000, 0.00000000, 0.43700000, 0.61270000, 0.18400000, 0.27513500, 0.13333333, 0.36125000, 0.53333333, 0.99225000, 0.21450000, 0.47800000
0.00081990, 0.00000000, 0.46400000, 0.00000000, 0.43700000, 0.60090000, 0.42300000, 0.27513500, 0.13333333, 0.36125000, 0.53333333, 0.99225000, 0.26000000, 0.43400000
0.00053720, 0.00000000, 0.46400000, 0.00000000, 0.43700000, 0.65490000, 0.51000000, 0.29802000, 0.13333333, 0.36125000, 0.53333333, 0.98212500, 0.18475000, 0.54200000
0.00141030, 0.00000000, 0.46400000, 0.00000000, 0.43700000, 0.57900000, 0.58000000, 0.31600000, 0.13333333, 0.36125000, 0.53333333, 0.99225000, 0.39600000, 0.40600000
0.00064660, 0.70000000, 0.07466667, 0.00000000, 0.40000000, 0.63450000, 0.20100000, 0.39139000, 0.16666667, 0.44750000, 0.49333333, 0.92060000, 0.12425000, 0.45000000
0.00055610, 0.70000000, 0.07466667, 0.00000000, 0.40000000, 0.70410000, 0.10000000, 0.39139000, 0.16666667, 0.44750000, 0.49333333, 0.92895000, 0.11850000, 0.58000000
0.00035370, 0.34000000, 0.20300000, 0.00000000, 0.43300000, 0.65900000, 0.40400000, 0.27458500, 0.23333333, 0.41125000, 0.53666667, 0.98937500, 0.23750000, 0.44000000
0.00092660, 0.34000000, 0.20300000, 0.00000000, 0.43300000, 0.64950000, 0.18400000, 0.27458500, 0.23333333, 0.41125000, 0.53666667, 0.95902500, 0.21675000, 0.52800000
0.00100000, 0.34000000, 0.20300000, 0.00000000, 0.43300000, 0.69820000, 0.17700000, 0.27458500, 0.23333333, 0.41125000, 0.53666667, 0.97607500, 0.12150000, 0.66200000
0.00055150, 0.33000000, 0.07266667, 0.00000000, 0.47200000, 0.72360000, 0.41100000, 0.20110000, 0.23333333, 0.27750000, 0.61333333, 0.98420000, 0.17325000, 0.72200000
0.00075030, 0.33000000, 0.07266667, 0.00000000, 0.47200000, 0.74200000, 0.71900000, 0.15496000, 0.23333333, 0.27750000, 0.61333333, 0.99225000, 0.16175000, 0.66800000
0.00049320, 0.33000000, 0.07266667, 0.00000000, 0.47200000, 0.68490000, 0.70300000, 0.15913500, 0.23333333, 0.27750000, 0.61333333, 0.99225000, 0.18825000, 0.56400000
0.00492980, 0.00000000, 0.33000000, 0.00000000, 0.54400000, 0.66350000, 0.82500000, 0.16587500, 0.13333333, 0.38000000, 0.61333333, 0.99225000, 0.11350000, 0.45600000
0.00349400, 0.00000000, 0.33000000, 0.00000000, 0.54400000, 0.59720000, 0.76700000, 0.15512500, 0.13333333, 0.38000000, 0.61333333, 0.99060000, 0.24925000, 0.40600000
0.00790410, 0.00000000, 0.33000000, 0.00000000, 0.54400000, 0.61220000, 0.52800000, 0.13201500, 0.13333333, 0.38000000, 0.61333333, 0.99225000, 0.14950000, 0.44200000
0.00261690, 0.00000000, 0.33000000, 0.00000000, 0.54400000, 0.60230000, 0.90400000, 0.14170000, 0.13333333, 0.38000000, 0.61333333, 0.99075000, 0.29300000, 0.38800000
0.00269380, 0.00000000, 0.33000000, 0.00000000, 0.54400000, 0.62660000, 0.82800000, 0.16314000, 0.13333333, 0.38000000, 0.61333333, 0.98347500, 0.19750000, 0.43200000
0.00369200, 0.00000000, 0.33000000, 0.00000000, 0.54400000, 0.65670000, 0.87300000, 0.18011500, 0.13333333, 0.38000000, 0.61333333, 0.98922500, 0.23200000, 0.47600000
0.00318270, 0.00000000, 0.33000000, 0.00000000, 0.54400000, 0.59140000, 0.83200000, 0.19993000, 0.13333333, 0.38000000, 0.61333333, 0.97675000, 0.45825000, 0.35600000
0.00245220, 0.00000000, 0.33000000, 0.00000000, 0.54400000, 0.57820000, 0.71700000, 0.20158500, 0.13333333, 0.38000000, 0.61333333, 0.99225000, 0.39850000, 0.39600000
0.00402020, 0.00000000, 0.33000000, 0.00000000, 0.54400000, 0.63820000, 0.67200000, 0.17662500, 0.13333333, 0.38000000, 0.61333333, 0.98802500, 0.25900000, 0.46200000
0.00475470, 0.00000000, 0.33000000, 0.00000000, 0.54400000, 0.61130000, 0.58800000, 0.20009500, 0.13333333, 0.38000000, 0.61333333, 0.99057500, 0.31825000, 0.42000000
0.00181590, 0.00000000, 0.24600000, 0.00000000, 0.49300000, 0.63760000, 0.54300000, 0.22702000, 0.16666667, 0.35875000, 0.65333333, 0.99225000, 0.17175000, 0.46200000
0.00351140, 0.00000000, 0.24600000, 0.00000000, 0.49300000, 0.60410000, 0.49900000, 0.23605500, 0.16666667, 0.35875000, 0.65333333, 0.99225000, 0.19250000, 0.40800000
0.00283920, 0.00000000, 0.24600000, 0.00000000, 0.49300000, 0.57080000, 0.74300000, 0.23605500, 0.16666667, 0.35875000, 0.65333333, 0.97782500, 0.29350000, 0.37000000
0.00341090, 0.00000000, 0.24600000, 0.00000000, 0.49300000, 0.64150000, 0.40100000, 0.23605500, 0.16666667, 0.35875000, 0.65333333, 0.99225000, 0.15300000, 0.50000000
0.00303470, 0.00000000, 0.24600000, 0.00000000, 0.49300000, 0.63120000, 0.28900000, 0.27079500, 0.16666667, 0.35875000, 0.65333333, 0.99225000, 0.15375000, 0.46000000
0.00241030, 0.00000000, 0.24600000, 0.00000000, 0.49300000, 0.60830000, 0.43700000, 0.27079500, 0.16666667, 0.35875000, 0.65333333, 0.99225000, 0.31975000, 0.44400000
0.00066170, 0.00000000, 0.10800000, 0.00000000, 0.46000000, 0.58680000, 0.25800000, 0.26073000, 0.13333333, 0.53750000, 0.56333333, 0.95610000, 0.24925000, 0.38600000
0.00067240, 0.00000000, 0.10800000, 0.00000000, 0.46000000, 0.63330000, 0.17200000, 0.26073000, 0.13333333, 0.53750000, 0.56333333, 0.93802500, 0.18350000, 0.45200000
0.00050230, 0.35000000, 0.20200000, 0.00000000, 0.43790000, 0.57060000, 0.28400000, 0.33203500, 0.03333333, 0.38000000, 0.56333333, 0.98505000, 0.31075000, 0.34200000
0.00034660, 0.35000000, 0.20200000, 0.00000000, 0.43790000, 0.60310000, 0.23300000, 0.33203500, 0.03333333, 0.38000000, 0.56333333, 0.90562500, 0.19575000, 0.38800000
0.00050830, 0.00000000, 0.17300000, 0.00000000, 0.51500000, 0.63160000, 0.38100000, 0.32292000, 0.16666667, 0.28000000, 0.67333333, 0.97427500, 0.14200000, 0.44400000
0.00037380, 0.00000000, 0.17300000, 0.00000000, 0.51500000, 0.63100000, 0.38500000, 0.32292000, 0.16666667, 0.28000000, 0.67333333, 0.97350000, 0.16875000, 0.41400000
0.00034270, 0.00000000, 0.17300000, 0.00000000, 0.51500000, 0.58690000, 0.46300000, 0.26155500, 0.16666667, 0.28000000, 0.67333333, 0.99225000, 0.24500000, 0.39000000
0.00030410, 0.00000000, 0.17300000, 0.00000000, 0.51500000, 0.58950000, 0.59600000, 0.28075000, 0.16666667, 0.28000000, 0.67333333, 0.98702500, 0.26400000, 0.37000000
0.00033060, 0.00000000, 0.17300000, 0.00000000, 0.51500000, 0.60590000, 0.37300000, 0.24061000, 0.16666667, 0.28000000, 0.67333333, 0.99035000, 0.21275000, 0.41200000
0.00054970, 0.00000000, 0.17300000, 0.00000000, 0.51500000, 0.59850000, 0.45400000, 0.24061000, 0.16666667, 0.28000000, 0.67333333, 0.99225000, 0.24350000, 0.38000000
0.00013010, 0.35000000, 0.05066667, 0.00000000, 0.44200000, 0.72410000, 0.49300000, 0.35189500, 0.03333333, 0.35500000, 0.51666667, 0.98685000, 0.13725000, 0.65400000
0.00024980, 0.00000000, 0.06300000, 0.00000000, 0.51800000, 0.65400000, 0.59700000, 0.31334500, 0.03333333, 0.52750000, 0.53000000, 0.97490000, 0.21625000, 0.33000000
0.00025430, 0.55000000, 0.12600000, 0.00000000, 0.48400000, 0.66960000, 0.56400000, 0.28660500, 0.16666667, 0.46250000, 0.58666667, 0.99225000, 0.17950000, 0.47800000
0.00030490, 0.55000000, 0.12600000, 0.00000000, 0.48400000, 0.68740000, 0.28100000, 0.32327000, 0.16666667, 0.46250000, 0.58666667, 0.96992500, 0.11525000, 0.62400000
0.00061620, 0.00000000, 0.14633333, 0.00000000, 0.44200000, 0.58980000, 0.52300000, 0.40068000, 0.10000000, 0.44000000, 0.62666667, 0.91152500, 0.31675000, 0.34400000
0.00018700, 0.85000000, 0.13833333, 0.00000000, 0.42900000, 0.65160000, 0.27700000, 0.42676500, 0.13333333, 0.43875000, 0.59666667, 0.98107500, 0.15900000, 0.46200000
0.00015010, 0.80000000, 0.06700000, 0.00000000, 0.43500000, 0.66350000, 0.29700000, 0.41720000, 0.13333333, 0.35000000, 0.56666667, 0.97735000, 0.14975000, 0.49000000
0.00028990, 0.40000000, 0.04166667, 0.00000000, 0.42900000, 0.69390000, 0.34500000, 0.43960500, 0.03333333, 0.41875000, 0.65666667, 0.97462500, 0.14725000, 0.53200000
0.00079500, 0.60000000, 0.05633333, 0.00000000, 0.41100000, 0.65790000, 0.35900000, 0.53551500, 0.13333333, 0.51375000, 0.61000000, 0.92695000, 0.13725000, 0.48200000
0.00072440, 0.60000000, 0.05633333, 0.00000000, 0.41100000, 0.58840000, 0.18500000, 0.53551500, 0.13333333, 0.51375000, 0.61000000, 0.98082500, 0.19475000, 0.37200000
0.00017090, 0.90000000, 0.06733333, 0.00000000, 0.41000000, 0.67280000, 0.36100000, 0.60632500, 0.16666667, 0.23375000, 0.56666667, 0.96115000, 0.11250000, 0.60200000
0.00043010, 0.80000000, 0.06366667, 0.00000000, 0.41300000, 0.56630000, 0.21900000, 0.52928500, 0.13333333, 0.41750000, 0.73333333, 0.95700000, 0.20125000, 0.36400000
0.08982960, 0.00000000, 0.60333333, 1.00000000, 0.77000000, 0.62120000, 0.97400000, 0.10611000, 0.80000000, 0.83250000, 0.67333333, 0.94432500, 0.44000000, 0.35600000
0.03849700, 0.00000000, 0.60333333, 1.00000000, 0.77000000, 0.63950000, 0.91000000, 0.12526000, 0.80000000, 0.83250000, 0.67333333, 0.97835000, 0.33175000, 0.43400000
0.05201770, 0.00000000, 0.60333333, 1.00000000, 0.77000000, 0.61270000, 0.83400000, 0.13613500, 0.80000000, 0.83250000, 0.67333333, 0.98857500, 0.28700000, 0.45400000
0.04261310, 0.00000000, 0.60333333, 0.00000000, 0.77000000, 0.61120000, 0.81300000, 0.12545500, 0.80000000, 0.83250000, 0.67333333, 0.97685000, 0.31675000, 0.45200000
0.03836840, 0.00000000, 0.60333333, 0.00000000, 0.77000000, 0.62510000, 0.91100000, 0.11477500, 0.80000000, 0.83250000, 0.67333333, 0.87662500, 0.35475000, 0.39800000
0.03678220, 0.00000000, 0.60333333, 0.00000000, 0.77000000, 0.53620000, 0.96200000, 0.10518000, 0.80000000, 0.83250000, 0.67333333, 0.95197500, 0.25475000, 0.41600000
0.04222390, 0.00000000, 0.60333333, 1.00000000, 0.77000000, 0.58030000, 0.89000000, 0.09523500, 0.80000000, 0.83250000, 0.67333333, 0.88260000, 0.36600000, 0.33600000
0.03474280, 0.00000000, 0.60333333, 1.00000000, 0.71800000, 0.87800000, 0.82900000, 0.09523500, 0.80000000, 0.83250000, 0.67333333, 0.88637500, 0.13225000, 0.43800000
0.03696950, 0.00000000, 0.60333333, 0.00000000, 0.71800000, 0.49630000, 0.91400000, 0.08761500, 0.80000000, 0.83250000, 0.67333333, 0.79007500, 0.35000000, 0.43800000
0.13522200, 0.00000000, 0.60333333, 0.00000000, 0.63100000, 0.38630000, 1.00000000, 0.07553000, 0.80000000, 0.83250000, 0.67333333, 0.32855000, 0.33325000, 0.46200000
0.04898220, 0.00000000, 0.60333333, 0.00000000, 0.63100000, 0.49700000, 1.00000000, 0.06662500, 0.80000000, 0.83250000, 0.67333333, 0.93880000, 0.08150000, 1.00000000
0.05669980, 0.00000000, 0.60333333, 1.00000000, 0.63100000, 0.66830000, 0.96800000, 0.06783500, 0.80000000, 0.83250000, 0.67333333, 0.93832500, 0.09325000, 1.00000000
0.09232300, 0.00000000, 0.60333333, 0.00000000, 0.63100000, 0.62160000, 1.00000000, 0.05845500, 0.80000000, 0.83250000, 0.67333333, 0.91537500, 0.23825000, 1.00000000
0.08267250, 0.00000000, 0.60333333, 1.00000000, 0.66800000, 0.58750000, 0.89600000, 0.05648000, 0.80000000, 0.83250000, 0.67333333, 0.86970000, 0.22200000, 1.00000000
0.11108100, 0.00000000, 0.60333333, 0.00000000, 0.66800000, 0.49060000, 1.00000000, 0.05871000, 0.80000000, 0.83250000, 0.67333333, 0.99225000, 0.86925000, 0.27600000
0.18498200, 0.00000000, 0.60333333, 0.00000000, 0.66800000, 0.41380000, 1.00000000, 0.05685000, 0.80000000, 0.83250000, 0.67333333, 0.99225000, 0.94925000, 0.27600000
0.15288000, 0.00000000, 0.60333333, 0.00000000, 0.67100000, 0.66490000, 0.93300000, 0.06724500, 0.80000000, 0.83250000, 0.67333333, 0.90755000, 0.58100000, 0.27800000
0.09823490, 0.00000000, 0.60333333, 0.00000000, 0.67100000, 0.67940000, 0.98800000, 0.06790000, 0.80000000, 0.83250000, 0.67333333, 0.99225000, 0.53100000, 0.26600000
0.23648200, 0.00000000, 0.60333333, 0.00000000, 0.67100000, 0.63800000, 0.96200000, 0.06930500, 0.80000000, 0.83250000, 0.67333333, 0.99225000, 0.59225000, 0.26200000
0.17866700, 0.00000000, 0.60333333, 0.00000000, 0.67100000, 0.62230000, 1.00000000, 0.06930500, 0.80000000, 0.83250000, 0.67333333, 0.98435000, 0.54450000, 0.20400000
0.15874400, 0.00000000, 0.60333333, 0.00000000, 0.67100000, 0.65450000, 0.99100000, 0.07596000, 0.80000000, 0.83250000, 0.67333333, 0.99225000, 0.52700000, 0.21800000
0.09187020, 0.00000000, 0.60333333, 0.00000000, 0.70000000, 0.55360000, 1.00000000, 0.07902000, 0.80000000, 0.83250000, 0.67333333, 0.99225000, 0.59000000, 0.22600000
0.07992480, 0.00000000, 0.60333333, 0.00000000, 0.70000000, 0.55200000, 1.00000000, 0.07665500, 0.80000000, 0.83250000, 0.67333333, 0.99225000, 0.61400000, 0.24600000
0.20084900, 0.00000000, 0.60333333, 0.00000000, 0.70000000, 0.43680000, 0.91200000, 0.07197500, 0.80000000, 0.83250000, 0.67333333, 0.71457500, 0.76575000, 0.17600000
0.24393800, 0.00000000, 0.60333333, 0.00000000, 0.70000000, 0.46520000, 1.00000000, 0.07336000, 0.80000000, 0.83250000, 0.67333333, 0.99225000, 0.70700000, 0.21000000
0.22597100, 0.00000000, 0.60333333, 0.00000000, 0.70000000, 0.50000000, 0.89500000, 0.07592000, 0.80000000, 0.83250000, 0.67333333, 0.99225000, 0.79975000, 0.14800000
0.14333700, 0.00000000, 0.60333333, 0.00000000, 0.70000000, 0.48800000, 1.00000000, 0.07947500, 0.80000000, 0.83250000, 0.67333333, 0.93230000, 0.76550000, 0.20400000
0.08151740, 0.00000000, 0.60333333, 0.00000000, 0.70000000, 0.53900000, 0.98900000, 0.08640500, 0.80000000, 0.83250000, 0.67333333, 0.99225000, 0.52125000, 0.23000000
0.05293050, 0.00000000, 0.60333333, 0.00000000, 0.70000000, 0.60510000, 0.82500000, 0.10839000, 0.80000000, 0.83250000, 0.67333333, 0.94595000, 0.46900000, 0.46400000
0.11577900, 0.00000000, 0.60333333, 0.00000000, 0.70000000, 0.50360000, 0.97000000, 0.08850000, 0.80000000, 0.83250000, 0.67333333, 0.99225000, 0.64200000, 0.19400000
0.08644760, 0.00000000, 0.60333333, 0.00000000, 0.69300000, 0.61930000, 0.92600000, 0.08956000, 0.80000000, 0.83250000, 0.67333333, 0.99225000, 0.37925000, 0.27600000
0.13359800, 0.00000000, 0.60333333, 0.00000000, 0.69300000, 0.58870000, 0.94700000, 0.08910500, 0.80000000, 0.83250000, 0.67333333, 0.99225000, 0.40875000, 0.25400000
0.05872050, 0.00000000, 0.60333333, 0.00000000, 0.69300000, 0.64050000, 0.96000000, 0.08384000, 0.80000000, 0.83250000, 0.67333333, 0.99225000, 0.48425000, 0.25000000
0.07672020, 0.00000000, 0.60333333, 0.00000000, 0.69300000, 0.57470000, 0.98900000, 0.08167000, 0.80000000, 0.83250000, 0.67333333, 0.98275000, 0.49800000, 0.17000000
0.38351800, 0.00000000, 0.60333333, 0.00000000, 0.69300000, 0.54530000, 1.00000000, 0.07448000, 0.80000000, 0.83250000, 0.67333333, 0.99225000, 0.76475000, 0.10000000
0.09916550, 0.00000000, 0.60333333, 0.00000000, 0.69300000, 0.58520000, 0.77800000, 0.07502000, 0.80000000, 0.83250000, 0.67333333, 0.84540000, 0.74925000, 0.12600000
0.14236200, 0.00000000, 0.60333333, 0.00000000, 0.69300000, 0.63430000, 1.00000000, 0.07870500, 0.80000000, 0.83250000, 0.67333333, 0.99225000, 0.50800000, 0.14400000
0.09595710, 0.00000000, 0.60333333, 0.00000000, 0.69300000, 0.64040000, 1.00000000, 0.08195000, 0.80000000, 0.83250000, 0.67333333, 0.94027500, 0.50775000, 0.24200000
0.24801700, 0.00000000, 0.60333333, 0.00000000, 0.69300000, 0.53490000, 0.96000000, 0.08514000, 0.80000000, 0.83250000, 0.67333333, 0.99225000, 0.49425000, 0.16600000
0.41529200, 0.00000000, 0.60333333, 0.00000000, 0.69300000, 0.55310000, 0.85400000, 0.08037000, 0.80000000, 0.83250000, 0.67333333, 0.82365000, 0.68450000, 0.17000000
0.20716200, 0.00000000, 0.60333333, 0.00000000, 0.65900000, 0.41380000, 1.00000000, 0.05890500, 0.80000000, 0.83250000, 0.67333333, 0.92555000, 0.58350000, 0.23800000
0.11951100, 0.00000000, 0.60333333, 0.00000000, 0.65900000, 0.56080000, 1.00000000, 0.06426000, 0.80000000, 0.83250000, 0.67333333, 0.83022500, 0.30325000, 0.55800000
0.07403890, 0.00000000, 0.60333333, 0.00000000, 0.59700000, 0.56170000, 0.97900000, 0.07273500, 0.80000000, 0.83250000, 0.67333333, 0.78660000, 0.66000000, 0.34400000
0.14438300, 0.00000000, 0.60333333, 0.00000000, 0.59700000, 0.68520000, 1.00000000, 0.07327500, 0.80000000, 0.83250000, 0.67333333, 0.44840000, 0.49450000, 0.55000000
0.14050700, 0.00000000, 0.60333333, 0.00000000, 0.59700000, 0.66570000, 1.00000000, 0.07637500, 0.80000000, 0.83250000, 0.67333333, 0.08762500, 0.53050000, 0.34400000
0.18811000, 0.00000000, 0.60333333, 0.00000000, 0.59700000, 0.46280000, 1.00000000, 0.07769500, 0.80000000, 0.83250000, 0.67333333, 0.07197500, 0.85925000, 0.35800000
0.28655800, 0.00000000, 0.60333333, 0.00000000, 0.59700000, 0.51550000, 1.00000000, 0.07947000, 0.80000000, 0.83250000, 0.67333333, 0.52742500, 0.50200000, 0.32600000
0.45746100, 0.00000000, 0.60333333, 0.00000000, 0.69300000, 0.45190000, 1.00000000, 0.08291000, 0.80000000, 0.83250000, 0.67333333, 0.22067500, 0.92450000, 0.14000000
0.10834200, 0.00000000, 0.60333333, 0.00000000, 0.67900000, 0.67820000, 0.90800000, 0.09097500, 0.80000000, 0.83250000, 0.67333333, 0.05392500, 0.64475000, 0.15000000
0.25940600, 0.00000000, 0.60333333, 0.00000000, 0.67900000, 0.53040000, 0.89100000, 0.08237500, 0.80000000, 0.83250000, 0.67333333, 0.31840000, 0.66600000, 0.20800000
0.73534100, 0.00000000, 0.60333333, 0.00000000, 0.67900000, 0.59570000, 1.00000000, 0.09013000, 0.80000000, 0.83250000, 0.67333333, 0.04112500, 0.51550000, 0.17600000
0.11812300, 0.00000000, 0.60333333, 0.00000000, 0.71800000, 0.68240000, 0.76500000, 0.08970000, 0.80000000, 0.83250000, 0.67333333, 0.12112500, 0.56850000, 0.16800000
0.07022590, 0.00000000, 0.60333333, 0.00000000, 0.71800000, 0.60060000, 0.95300000, 0.09373000, 0.80000000, 0.83250000, 0.67333333, 0.79995000, 0.39250000, 0.28400000
0.12048200, 0.00000000, 0.60333333, 0.00000000, 0.61400000, 0.56480000, 0.87600000, 0.09756000, 0.80000000, 0.83250000, 0.67333333, 0.72887500, 0.35250000, 0.41600000
0.07050420, 0.00000000, 0.60333333, 0.00000000, 0.61400000, 0.61030000, 0.85100000, 0.10109000, 0.80000000, 0.83250000, 0.67333333, 0.00630000, 0.58225000, 0.26800000
0.08792120, 0.00000000, 0.60333333, 0.00000000, 0.58400000, 0.55650000, 0.70600000, 0.10317500, 0.80000000, 0.83250000, 0.67333333, 0.00912500, 0.42900000, 0.23400000
0.12247200, 0.00000000, 0.60333333, 0.00000000, 0.58400000, 0.58370000, 0.59700000, 0.09988000, 0.80000000, 0.83250000, 0.67333333, 0.06162500, 0.39225000, 0.20400000
0.37661900, 0.00000000, 0.60333333, 0.00000000, 0.67900000, 0.62020000, 0.78700000, 0.09314500, 0.80000000, 0.83250000, 0.67333333, 0.04705000, 0.36300000, 0.21800000
0.07367110, 0.00000000, 0.60333333, 0.00000000, 0.67900000, 0.61930000, 0.78100000, 0.09678000, 0.80000000, 0.83250000, 0.67333333, 0.24182500, 0.53800000, 0.22000000
0.09338890, 0.00000000, 0.60333333, 0.00000000, 0.67900000, 0.63800000, 0.95600000, 0.09841000, 0.80000000, 0.83250000, 0.67333333, 0.15180000, 0.60200000, 0.19000000
0.10062300, 0.00000000, 0.60333333, 0.00000000, 0.58400000, 0.68330000, 0.94300000, 0.10441000, 0.80000000, 0.83250000, 0.67333333, 0.20332500, 0.49225000, 0.28200000
0.06444050, 0.00000000, 0.60333333, 0.00000000, 0.58400000, 0.64250000, 0.74800000, 0.11002000, 0.80000000, 0.83250000, 0.67333333, 0.24487500, 0.30075000, 0.32200000
0.05581070, 0.00000000, 0.60333333, 0.00000000, 0.71300000, 0.64360000, 0.87900000, 0.11579000, 0.80000000, 0.83250000, 0.67333333, 0.25047500, 0.40550000, 0.28600000
0.13913400, 0.00000000, 0.60333333, 0.00000000, 0.71300000, 0.62080000, 0.95000000, 0.11111000, 0.80000000, 0.83250000, 0.67333333, 0.25157500, 0.37925000, 0.23400000
0.14420800, 0.00000000, 0.60333333, 0.00000000, 0.74000000, 0.64610000, 0.93300000, 0.10013000, 0.80000000, 0.83250000, 0.67333333, 0.06872500, 0.45125000, 0.19200000
0.15177200, 0.00000000, 0.60333333, 0.00000000, 0.74000000, 0.61520000, 1.00000000, 0.09571000, 0.80000000, 0.83250000, 0.67333333, 0.02330000, 0.66125000, 0.17400000
0.13678100, 0.00000000, 0.60333333, 0.00000000, 0.74000000, 0.59350000, 0.87900000, 0.09103000, 0.80000000, 0.83250000, 0.67333333, 0.17237500, 0.85050000, 0.16800000
0.09390630, 0.00000000, 0.60333333, 0.00000000, 0.74000000, 0.56270000, 0.93900000, 0.09086000, 0.80000000, 0.83250000, 0.67333333, 0.99225000, 0.57200000, 0.25600000
0.09724180, 0.00000000, 0.60333333, 0.00000000, 0.74000000, 0.64060000, 0.97200000, 0.10325500, 0.80000000, 0.83250000, 0.67333333, 0.96490000, 0.48800000, 0.34200000
0.05666370, 0.00000000, 0.60333333, 0.00000000, 0.74000000, 0.62190000, 1.00000000, 0.10024000, 0.80000000, 0.83250000, 0.67333333, 0.98922500, 0.41475000, 0.36800000
0.09966540, 0.00000000, 0.60333333, 0.00000000, 0.74000000, 0.64850000, 1.00000000, 0.09892000, 0.80000000, 0.83250000, 0.67333333, 0.96682500, 0.47125000, 0.30800000
0.12802300, 0.00000000, 0.60333333, 0.00000000, 0.74000000, 0.58540000, 0.96600000, 0.09478000, 0.80000000, 0.83250000, 0.67333333, 0.60130000, 0.59475000, 0.21600000
0.06288070, 0.00000000, 0.60333333, 0.00000000, 0.74000000, 0.63410000, 0.96400000, 0.10360000, 0.80000000, 0.83250000, 0.67333333, 0.79502500, 0.44475000, 0.29800000
0.09924850, 0.00000000, 0.60333333, 0.00000000, 0.74000000, 0.62510000, 0.96600000, 0.10990000, 0.80000000, 0.83250000, 0.67333333, 0.97130000, 0.41100000, 0.25200000
0.09329090, 0.00000000, 0.60333333, 0.00000000, 0.71300000, 0.61850000, 0.98700000, 0.11308000, 0.80000000, 0.83250000, 0.67333333, 0.99225000, 0.45325000, 0.28200000
0.07526010, 0.00000000, 0.60333333, 0.00000000, 0.71300000, 0.64170000, 0.98300000, 0.10925000, 0.80000000, 0.83250000, 0.67333333, 0.76052500, 0.48275000, 0.26000000
0.05441140, 0.00000000, 0.60333333, 0.00000000, 0.71300000, 0.66550000, 0.98200000, 0.11776000, 0.80000000, 0.83250000, 0.67333333, 0.88822500, 0.44325000, 0.30400000
0.05090170, 0.00000000, 0.60333333, 0.00000000, 0.71300000, 0.62970000, 0.91800000, 0.11841000, 0.80000000, 0.83250000, 0.67333333, 0.96272500, 0.43175000, 0.32200000
0.08248090, 0.00000000, 0.60333333, 0.00000000, 0.71300000, 0.73930000, 0.99300000, 0.12263500, 0.80000000, 0.83250000, 0.67333333, 0.93967500, 0.41850000, 0.35600000
0.09513630, 0.00000000, 0.60333333, 0.00000000, 0.71300000, 0.67280000, 0.94100000, 0.12480500, 0.80000000, 0.83250000, 0.67333333, 0.01670000, 0.46775000, 0.29800000
0.04668830, 0.00000000, 0.60333333, 0.00000000, 0.71300000, 0.59760000, 0.87900000, 0.12903000, 0.80000000, 0.83250000, 0.67333333, 0.02620000, 0.47525000, 0.25400000
0.08200580, 0.00000000, 0.60333333, 0.00000000, 0.71300000, 0.59360000, 0.80300000, 0.13896000, 0.80000000, 0.83250000, 0.67333333, 0.00875000, 0.42350000, 0.27000000
0.07752230, 0.00000000, 0.60333333, 0.00000000, 0.71300000, 0.63010000, 0.83700000, 0.13915500, 0.80000000, 0.83250000, 0.67333333, 0.68052500, 0.40575000, 0.29800000
0.06801170, 0.00000000, 0.60333333, 0.00000000, 0.71300000, 0.60810000, 0.84400000, 0.13587500, 0.80000000, 0.83250000, 0.67333333, 0.99225000, 0.36750000, 0.40000000
0.03693110, 0.00000000, 0.60333333, 0.00000000, 0.71300000, 0.63760000, 0.88400000, 0.12835500, 0.80000000, 0.83250000, 0.67333333, 0.97857500, 0.36625000, 0.35400000
0.06654920, 0.00000000, 0.60333333, 0.00000000, 0.71300000, 0.63170000, 0.83000000, 0.13672000, 0.80000000, 0.83250000, 0.67333333, 0.99225000, 0.34975000, 0.39000000
0.05821150, 0.00000000, 0.60333333, 0.00000000, 0.71300000, 0.65130000, 0.89900000, 0.14008000, 0.80000000, 0.83250000, 0.67333333, 0.98455000, 0.25725000, 0.40400000
0.07839320, 0.00000000, 0.60333333, 0.00000000, 0.65500000, 0.62090000, 0.65400000, 0.14817000, 0.80000000, 0.83250000, 0.67333333, 0.99225000, 0.33050000, 0.42800000
0.03774980, 0.00000000, 0.60333333, 0.00000000, 0.65500000, 0.59520000, 0.84700000, 0.14357500, 0.80000000, 0.83250000, 0.67333333, 0.05502500, 0.42875000, 0.38000000
0.04422280, 0.00000000, 0.60333333, 0.00000000, 0.58400000, 0.60030000, 0.94500000, 0.12701500, 0.80000000, 0.83250000, 0.67333333, 0.82822500, 0.53300000, 0.38200000
0.15575700, 0.00000000, 0.60333333, 0.00000000, 0.58000000, 0.59260000, 0.71000000, 0.14542000, 0.80000000, 0.83250000, 0.67333333, 0.92185000, 0.45325000, 0.38200000
0.13075100, 0.00000000, 0.60333333, 0.00000000, 0.58000000, 0.57130000, 0.56700000, 0.14118500, 0.80000000, 0.83250000, 0.67333333, 0.99225000, 0.36900000, 0.40200000
0.04038410, 0.00000000, 0.60333333, 0.00000000, 0.53200000, 0.62290000, 0.90700000, 0.15496500, 0.80000000, 0.83250000, 0.67333333, 0.98832500, 0.32175000, 0.39200000
0.03568680, 0.00000000, 0.60333333, 0.00000000, 0.58000000, 0.64370000, 0.75000000, 0.14482500, 0.80000000, 0.83250000, 0.67333333, 0.98342500, 0.35900000, 0.46400000
0.04646890, 0.00000000, 0.60333333, 0.00000000, 0.61400000, 0.69800000, 0.67600000, 0.12664500, 0.80000000, 0.83250000, 0.67333333, 0.93670000, 0.29150000, 0.59600000
0.08055790, 0.00000000, 0.60333333, 0.00000000, 0.58400000, 0.54270000, 0.95400000, 0.12149000, 0.80000000, 0.83250000, 0.67333333, 0.88145000, 0.45350000, 0.27600000
0.04871410, 0.00000000, 0.60333333, 0.00000000, 0.61400000, 0.64840000, 0.93600000, 0.11526500, 0.80000000, 0.83250000, 0.67333333, 0.99052500, 0.46700000, 0.33400000
0.15023400, 0.00000000, 0.60333333, 0.00000000, 0.61400000, 0.53040000, 0.97300000, 0.10503500, 0.80000000, 0.83250000, 0.67333333, 0.87370000, 0.62275000, 0.24000000
0.10233000, 0.00000000, 0.60333333, 0.00000000, 0.61400000, 0.61850000, 0.96700000, 0.10852500, 0.80000000, 0.83250000, 0.67333333, 0.94925000, 0.45075000, 0.29200000
0.14333700, 0.00000000, 0.60333333, 0.00000000, 0.61400000, 0.62290000, 0.88000000, 0.09756000, 0.80000000, 0.83250000, 0.67333333, 0.95830000, 0.32775000, 0.42800000
0.05708180, 0.00000000, 0.60333333, 0.00000000, 0.53200000, 0.67500000, 0.74900000, 0.16658500, 0.80000000, 0.83250000, 0.67333333, 0.98267500, 0.19350000, 0.47400000
0.05731160, 0.00000000, 0.60333333, 0.00000000, 0.53200000, 0.70610000, 0.77000000, 0.17053000, 0.80000000, 0.83250000, 0.67333333, 0.98820000, 0.17525000, 0.50000000
0.02818380, 0.00000000, 0.60333333, 0.00000000, 0.53200000, 0.57620000, 0.40300000, 0.20491500, 0.80000000, 0.83250000, 0.67333333, 0.98230000, 0.26050000, 0.43600000
0.02378570, 0.00000000, 0.60333333, 0.00000000, 0.58300000, 0.58710000, 0.41900000, 0.18620000, 0.80000000, 0.83250000, 0.67333333, 0.92682500, 0.33350000, 0.41200000
0.05691750, 0.00000000, 0.60333333, 0.00000000, 0.58300000, 0.61140000, 0.79800000, 0.17729500, 0.80000000, 0.83250000, 0.67333333, 0.98170000, 0.37450000, 0.38200000
0.04835670, 0.00000000, 0.60333333, 0.00000000, 0.58300000, 0.59050000, 0.53200000, 0.15761500, 0.80000000, 0.83250000, 0.67333333, 0.97055000, 0.28625000, 0.41200000
0.00150860, 0.00000000, 0.92466667, 0.00000000, 0.60900000, 0.54540000, 0.92700000, 0.09104500, 0.13333333, 0.88875000, 0.67000000, 0.98772500, 0.45150000, 0.30400000
0.00183370, 0.00000000, 0.92466667, 0.00000000, 0.60900000, 0.54140000, 0.98300000, 0.08777000, 0.13333333, 0.88875000, 0.67000000, 0.86012500, 0.59925000, 0.14000000
0.00105740, 0.00000000, 0.92466667, 0.00000000, 0.60900000, 0.59830000, 0.98800000, 0.09340500, 0.13333333, 0.88875000, 0.67000000, 0.97527500, 0.45175000, 0.27200000
0.00111320, 0.00000000, 0.92466667, 0.00000000, 0.60900000, 0.59830000, 0.83500000, 0.10549500, 0.13333333, 0.88875000, 0.67000000, 0.99225000, 0.33375000, 0.40200000
0.00173310, 0.00000000, 0.32300000, 0.00000000, 0.58500000, 0.57070000, 0.54000000, 0.11908500, 0.20000000, 0.48875000, 0.64000000, 0.99225000, 0.30025000, 0.43600000
0.00279570, 0.00000000, 0.32300000, 0.00000000, 0.58500000, 0.59260000, 0.42600000, 0.11908500, 0.20000000, 0.48875000, 0.64000000, 0.99225000, 0.33975000, 0.49000000
0.00289600, 0.00000000, 0.32300000, 0.00000000, 0.58500000, 0.53900000, 0.72900000, 0.13993000, 0.20000000, 0.48875000, 0.64000000, 0.99225000, 0.52850000, 0.39400000
0.00268380, 0.00000000, 0.32300000, 0.00000000, 0.58500000, 0.57940000, 0.70600000, 0.14463500, 0.20000000, 0.48875000, 0.64000000, 0.99225000, 0.35250000, 0.36600000
0.00239120, 0.00000000, 0.32300000, 0.00000000, 0.58500000, 0.60190000, 0.65300000, 0.12045500, 0.20000000, 0.48875000, 0.64000000, 0.99225000, 0.32300000, 0.42400000
0.00177830, 0.00000000, 0.32300000, 0.00000000, 0.58500000, 0.55690000, 0.73500000, 0.11999500, 0.20000000, 0.48875000, 0.64000000, 0.98942500, 0.37750000, 0.35000000

Test data:


# boston_test.txt
# norm constants: 100, 100, 30, 1, 1, 10, 100, 20, 30, 800, 30, 400, 40, 50
#
# crime     zoning      indus       river       nox         rooms       oldness     dist        access     tax          pup_tch     black       low_stat    med_val
0.00006320, 0.18000000, 0.07700000, 0.00000000, 0.53800000, 0.65750000, 0.65200000, 0.20450000, 0.03333333, 0.37000000, 0.51000000, 0.99225000, 0.12450000, 0.48000000
0.00029850, 0.00000000, 0.07266667, 0.00000000, 0.45800000, 0.64300000, 0.58700000, 0.30311000, 0.10000000, 0.27750000, 0.62333333, 0.98530000, 0.13025000, 0.57400000
0.00224890, 0.12500000, 0.26233333, 0.00000000, 0.52400000, 0.63770000, 0.94300000, 0.31733500, 0.16666667, 0.38875000, 0.50666667, 0.98130000, 0.51125000, 0.30000000
0.00627390, 0.00000000, 0.27133333, 0.00000000, 0.53800000, 0.58340000, 0.56500000, 0.22493000, 0.13333333, 0.38375000, 0.70000000, 0.98905000, 0.21175000, 0.39800000
0.01251790, 0.00000000, 0.27133333, 0.00000000, 0.53800000, 0.55700000, 0.98100000, 0.18989500, 0.13333333, 0.38375000, 0.70000000, 0.94142500, 0.52550000, 0.27200000
0.00840540, 0.00000000, 0.27133333, 0.00000000, 0.53800000, 0.55990000, 0.85700000, 0.22273000, 0.13333333, 0.38375000, 0.70000000, 0.75855000, 0.41275000, 0.27800000
0.01130810, 0.00000000, 0.27133333, 0.00000000, 0.53800000, 0.57130000, 0.94100000, 0.21165000, 0.13333333, 0.38375000, 0.70000000, 0.90042500, 0.56500000, 0.25400000
0.00064170, 0.00000000, 0.19866667, 0.00000000, 0.49900000, 0.59330000, 0.68200000, 0.16801500, 0.16666667, 0.34875000, 0.64000000, 0.99225000, 0.24200000, 0.37800000
0.00033590, 0.75000000, 0.09833333, 0.00000000, 0.42800000, 0.70240000, 0.15800000, 0.27005500, 0.10000000, 0.31500000, 0.61000000, 0.98905000, 0.04950000, 0.69800000
0.00171420, 0.00000000, 0.23033333, 0.00000000, 0.44800000, 0.56820000, 0.33800000, 0.25502000, 0.10000000, 0.29125000, 0.59666667, 0.99225000, 0.25525000, 0.38600000
0.00088730, 0.21000000, 0.18800000, 0.00000000, 0.43900000, 0.59630000, 0.45700000, 0.34073500, 0.13333333, 0.30375000, 0.56000000, 0.98890000, 0.33625000, 0.39400000
0.00013110, 0.90000000, 0.04066667, 0.00000000, 0.40300000, 0.72490000, 0.21900000, 0.43483000, 0.16666667, 0.28250000, 0.59666667, 0.98982500, 0.12025000, 0.70800000
0.00149320, 0.25000000, 0.17100000, 0.00000000, 0.45300000, 0.57410000, 0.66200000, 0.36127000, 0.26666667, 0.35500000, 0.65666667, 0.98777500, 0.32875000, 0.37400000
0.00035840, 0.80000000, 0.11233333, 0.00000000, 0.39800000, 0.62900000, 0.17800000, 0.33057500, 0.13333333, 0.42125000, 0.53666667, 0.99225000, 0.11675000, 0.47000000
0.00088260, 0.00000000, 0.36033333, 0.00000000, 0.41300000, 0.64170000, 0.06600000, 0.26436500, 0.13333333, 0.38125000, 0.64000000, 0.95932500, 0.16800000, 0.48400000
0.00095120, 0.00000000, 0.42766667, 0.00000000, 0.43700000, 0.62860000, 0.45000000, 0.22513000, 0.16666667, 0.49750000, 0.62333333, 0.95807500, 0.22350000, 0.42800000
0.00041130, 0.25000000, 0.16200000, 0.00000000, 0.42600000, 0.67270000, 0.33500000, 0.27003500, 0.13333333, 0.35125000, 0.63333333, 0.99225000, 0.13225000, 0.56000000
0.00057350, 0.00000000, 0.14966667, 0.00000000, 0.44900000, 0.66300000, 0.56100000, 0.22188500, 0.10000000, 0.30875000, 0.61666667, 0.98075000, 0.16325000, 0.53200000
0.00046840, 0.00000000, 0.11366667, 0.00000000, 0.48900000, 0.64170000, 0.66100000, 0.15461500, 0.06666667, 0.33750000, 0.59333333, 0.98045000, 0.22025000, 0.45200000
0.00122040, 0.00000000, 0.09633333, 0.00000000, 0.44500000, 0.66250000, 0.57800000, 0.17476000, 0.06666667, 0.34500000, 0.60000000, 0.89495000, 0.16625000, 0.56800000
0.00148660, 0.00000000, 0.28533333, 0.00000000, 0.52000000, 0.67270000, 0.79900000, 0.13889000, 0.16666667, 0.48000000, 0.69666667, 0.98690000, 0.23550000, 0.55000000
0.00132620, 0.00000000, 0.28533333, 0.00000000, 0.52000000, 0.58510000, 0.96700000, 0.10534500, 0.16666667, 0.48000000, 0.69666667, 0.98512500, 0.41175000, 0.39000000
0.00107930, 0.00000000, 0.28533333, 0.00000000, 0.52000000, 0.61950000, 0.54400000, 0.13889000, 0.16666667, 0.48000000, 0.69666667, 0.98372500, 0.32500000, 0.43400000
0.00171340, 0.00000000, 0.33366667, 0.00000000, 0.54700000, 0.59280000, 0.88200000, 0.12315500, 0.20000000, 0.54000000, 0.59333333, 0.86227500, 0.39400000, 0.36600000
0.00068990, 0.00000000, 0.85500000, 0.00000000, 0.58100000, 0.58700000, 0.69700000, 0.11288500, 0.06666667, 0.23500000, 0.63666667, 0.97287500, 0.35925000, 0.44000000
0.00169020, 0.00000000, 0.85500000, 0.00000000, 0.58100000, 0.59860000, 0.88400000, 0.09964500, 0.06666667, 0.23500000, 0.63666667, 0.96255000, 0.37025000, 0.42800000
0.00340060, 0.00000000, 0.72966667, 0.00000000, 0.62400000, 0.64580000, 0.98900000, 0.10592500, 0.13333333, 0.54625000, 0.70666667, 0.98760000, 0.31500000, 0.38400000
0.00557780, 0.00000000, 0.72966667, 0.00000000, 0.62400000, 0.63350000, 0.98200000, 0.10553500, 0.13333333, 0.54625000, 0.70666667, 0.98667500, 0.42400000, 0.36200000
0.00290900, 0.00000000, 0.72966667, 0.00000000, 0.62400000, 0.61740000, 0.93600000, 0.08059500, 0.13333333, 0.54625000, 0.70666667, 0.97020000, 0.60400000, 0.28000000
0.02379340, 0.00000000, 0.65266667, 0.00000000, 0.87100000, 0.61300000, 1.00000000, 0.07095500, 0.16666667, 0.50375000, 0.49000000, 0.43227500, 0.69500000, 0.27600000
0.01656600, 0.00000000, 0.65266667, 0.00000000, 0.87100000, 0.61220000, 0.97300000, 0.08090000, 0.16666667, 0.50375000, 0.49000000, 0.93200000, 0.35250000, 0.43000000
0.03535010, 0.00000000, 0.65266667, 1.00000000, 0.87100000, 0.61520000, 0.82600000, 0.08727500, 0.16666667, 0.50375000, 0.49000000, 0.22002500, 0.37550000, 0.31200000
0.01273460, 0.00000000, 0.65266667, 1.00000000, 0.60500000, 0.62500000, 0.92600000, 0.08992000, 0.16666667, 0.50375000, 0.49000000, 0.84730000, 0.13750000, 0.54000000
0.02924000, 0.00000000, 0.65266667, 0.00000000, 0.60500000, 0.61010000, 0.93000000, 0.11417000, 0.16666667, 0.50375000, 0.49000000, 0.60040000, 0.24525000, 0.50000000
0.01207420, 0.00000000, 0.65266667, 0.00000000, 0.60500000, 0.58750000, 0.94600000, 0.12129500, 0.16666667, 0.50375000, 0.49000000, 0.73072500, 0.36075000, 0.34800000
0.00066640, 0.00000000, 0.13500000, 0.00000000, 0.51000000, 0.65460000, 0.33100000, 0.15661500, 0.16666667, 0.37000000, 0.55333333, 0.97740000, 0.13325000, 0.58800000
0.00065880, 0.00000000, 0.08200000, 0.00000000, 0.48800000, 0.77650000, 0.83300000, 0.13705000, 0.10000000, 0.24125000, 0.59333333, 0.98890000, 0.18900000, 0.79600000
0.00060470, 0.00000000, 0.08200000, 0.00000000, 0.48800000, 0.61530000, 0.68800000, 0.16398500, 0.10000000, 0.24125000, 0.59333333, 0.96777500, 0.32875000, 0.59200000
0.00090680, 0.45000000, 0.11466667, 0.00000000, 0.43700000, 0.69510000, 0.21500000, 0.32399000, 0.16666667, 0.49750000, 0.50666667, 0.94420000, 0.12750000, 0.74000000
0.00013810, 0.80000000, 0.01533333, 0.00000000, 0.42200000, 0.78750000, 0.32000000, 0.28242000, 0.13333333, 0.31875000, 0.48000000, 0.98557500, 0.07425000, 1.00000000
0.00017780, 0.95000000, 0.04900000, 0.00000000, 0.40300000, 0.71350000, 0.13900000, 0.38267000, 0.10000000, 0.50250000, 0.56666667, 0.96075000, 0.11125000, 0.65800000
0.00136420, 0.00000000, 0.35300000, 0.00000000, 0.48900000, 0.58910000, 0.22300000, 0.19727000, 0.13333333, 0.34625000, 0.62000000, 0.99225000, 0.27175000, 0.45200000
0.00174460, 0.00000000, 0.35300000, 1.00000000, 0.48900000, 0.59600000, 0.92100000, 0.19385500, 0.13333333, 0.34625000, 0.62000000, 0.98312500, 0.43175000, 0.43400000
0.00198020, 0.00000000, 0.35300000, 0.00000000, 0.48900000, 0.61820000, 0.42400000, 0.19727000, 0.13333333, 0.34625000, 0.62000000, 0.98407500, 0.23675000, 0.50000000
0.00358090, 0.00000000, 0.20666667, 1.00000000, 0.50700000, 0.69510000, 0.88500000, 0.14308500, 0.26666667, 0.38375000, 0.58000000, 0.97925000, 0.24275000, 0.53400000
0.00526930, 0.00000000, 0.20666667, 0.00000000, 0.50400000, 0.87250000, 0.83000000, 0.14472000, 0.26666667, 0.38375000, 0.58000000, 0.95500000, 0.11575000, 1.00000000
0.00537000, 0.00000000, 0.20666667, 0.00000000, 0.50400000, 0.59810000, 0.68100000, 0.18357500, 0.26666667, 0.38375000, 0.58000000, 0.94587500, 0.29125000, 0.48600000
0.00330450, 0.00000000, 0.20666667, 0.00000000, 0.50700000, 0.60860000, 0.61500000, 0.18259500, 0.26666667, 0.38375000, 0.58000000, 0.94187500, 0.27200000, 0.48000000
0.00113290, 0.30000000, 0.16433333, 0.00000000, 0.42800000, 0.68970000, 0.54300000, 0.31680500, 0.20000000, 0.37500000, 0.55333333, 0.97812500, 0.28450000, 0.44000000
0.00191330, 0.22000000, 0.19533333, 0.00000000, 0.43100000, 0.56050000, 0.70200000, 0.39774500, 0.23333333, 0.41250000, 0.63666667, 0.97282500, 0.46150000, 0.37000000
0.00140300, 0.22000000, 0.19533333, 0.00000000, 0.43100000, 0.64870000, 0.13000000, 0.36983500, 0.23333333, 0.41250000, 0.63666667, 0.99070000, 0.14750000, 0.48800000
0.00035480, 0.80000000, 0.12133333, 0.00000000, 0.39200000, 0.58760000, 0.19100000, 0.46101500, 0.03333333, 0.39375000, 0.54666667, 0.98795000, 0.23125000, 0.41800000
0.00540110, 0.20000000, 0.13233333, 0.00000000, 0.64700000, 0.72030000, 0.81800000, 0.10560500, 0.16666667, 0.33000000, 0.43333333, 0.98200000, 0.23975000, 0.67600000
0.00761620, 0.20000000, 0.13233333, 0.00000000, 0.64700000, 0.55600000, 0.62800000, 0.09932500, 0.16666667, 0.33000000, 0.43333333, 0.98100000, 0.26125000, 0.45600000
0.00299160, 0.20000000, 0.23200000, 0.00000000, 0.46400000, 0.58560000, 0.42100000, 0.22145000, 0.10000000, 0.27875000, 0.62000000, 0.97162500, 0.32500000, 0.42200000
0.00096040, 0.40000000, 0.21366667, 0.00000000, 0.44700000, 0.68540000, 0.42800000, 0.21336500, 0.13333333, 0.31750000, 0.58666667, 0.99225000, 0.07450000, 0.64000000
0.00035780, 0.20000000, 0.11100000, 0.00000000, 0.44290000, 0.78200000, 0.64500000, 0.23473500, 0.16666667, 0.27000000, 0.49666667, 0.96827500, 0.09400000, 0.90800000
0.00010960, 0.55000000, 0.07500000, 0.00000000, 0.38900000, 0.64530000, 0.31900000, 0.36536500, 0.03333333, 0.37500000, 0.51000000, 0.98680000, 0.20575000, 0.44000000
0.00035020, 0.80000000, 0.16500000, 0.00000000, 0.41100000, 0.68610000, 0.27900000, 0.25583500, 0.13333333, 0.30625000, 0.64000000, 0.99225000, 0.08325000, 0.57000000
0.00129320, 0.00000000, 0.46400000, 0.00000000, 0.43700000, 0.66780000, 0.31100000, 0.29802000, 0.13333333, 0.36125000, 0.53333333, 0.99225000, 0.15675000, 0.57200000
0.00044170, 0.70000000, 0.07466667, 0.00000000, 0.40000000, 0.68710000, 0.47400000, 0.39139000, 0.16666667, 0.44750000, 0.49333333, 0.97715000, 0.15175000, 0.49600000
0.00054790, 0.33000000, 0.07266667, 0.00000000, 0.47200000, 0.66160000, 0.58100000, 0.16850000, 0.23333333, 0.27750000, 0.61333333, 0.98340000, 0.22325000, 0.56800000
0.02635480, 0.00000000, 0.33000000, 0.00000000, 0.54400000, 0.49730000, 0.37800000, 0.12597000, 0.13333333, 0.38000000, 0.61333333, 0.87612500, 0.31600000, 0.32200000
0.00253560, 0.00000000, 0.33000000, 0.00000000, 0.54400000, 0.57050000, 0.77700000, 0.19725000, 0.13333333, 0.38000000, 0.61333333, 0.99105000, 0.28750000, 0.32400000
0.00167600, 0.00000000, 0.24600000, 0.00000000, 0.49300000, 0.64260000, 0.52300000, 0.22702000, 0.16666667, 0.35875000, 0.65333333, 0.99225000, 0.18000000, 0.47600000
0.00191860, 0.00000000, 0.24600000, 0.00000000, 0.49300000, 0.64310000, 0.14700000, 0.27079500, 0.16666667, 0.35875000, 0.65333333, 0.98420000, 0.12700000, 0.49200000
0.00045440, 0.00000000, 0.10800000, 0.00000000, 0.46000000, 0.61440000, 0.32200000, 0.29368000, 0.13333333, 0.53750000, 0.56333333, 0.92142500, 0.22725000, 0.39600000
0.00039610, 0.00000000, 0.17300000, 0.00000000, 0.51500000, 0.60370000, 0.34500000, 0.29926500, 0.16666667, 0.28000000, 0.67333333, 0.99225000, 0.20025000, 0.42200000
0.00061510, 0.00000000, 0.17300000, 0.00000000, 0.51500000, 0.59680000, 0.58500000, 0.24061000, 0.16666667, 0.28000000, 0.67333333, 0.99225000, 0.23225000, 0.37400000
0.00031130, 0.00000000, 0.14633333, 0.00000000, 0.44200000, 0.60140000, 0.48500000, 0.40068000, 0.10000000, 0.44000000, 0.62666667, 0.96410000, 0.26325000, 0.35000000
0.00062110, 0.40000000, 0.04166667, 0.00000000, 0.42900000, 0.64900000, 0.44400000, 0.43960500, 0.03333333, 0.41875000, 0.65666667, 0.99225000, 0.14950000, 0.45800000
0.00106590, 0.80000000, 0.06366667, 0.00000000, 0.41300000, 0.59360000, 0.19500000, 0.52928500, 0.13333333, 0.41750000, 0.73333333, 0.94010000, 0.13925000, 0.41200000
0.04541920, 0.00000000, 0.60333333, 0.00000000, 0.77000000, 0.63980000, 0.88000000, 0.12591000, 0.80000000, 0.83250000, 0.67333333, 0.93640000, 0.19475000, 0.50000000
0.04555870, 0.00000000, 0.60333333, 0.00000000, 0.71800000, 0.35610000, 0.87900000, 0.08066000, 0.80000000, 0.83250000, 0.67333333, 0.88675000, 0.17800000, 0.55000000
0.06538760, 0.00000000, 0.60333333, 1.00000000, 0.63100000, 0.70160000, 0.97500000, 0.06012000, 0.80000000, 0.83250000, 0.67333333, 0.98012500, 0.07400000, 1.00000000
0.19609100, 0.00000000, 0.60333333, 0.00000000, 0.67100000, 0.73130000, 0.97900000, 0.06581500, 0.80000000, 0.83250000, 0.67333333, 0.99225000, 0.33600000, 0.30000000
0.88976200, 0.00000000, 0.60333333, 0.00000000, 0.67100000, 0.69680000, 0.91900000, 0.07082500, 0.80000000, 0.83250000, 0.67333333, 0.99225000, 0.43025000, 0.20800000
0.16811800, 0.00000000, 0.60333333, 0.00000000, 0.70000000, 0.52770000, 0.98100000, 0.07130500, 0.80000000, 0.83250000, 0.67333333, 0.99225000, 0.77025000, 0.14400000
0.06962150, 0.00000000, 0.60333333, 0.00000000, 0.70000000, 0.57130000, 0.97000000, 0.09632500, 0.80000000, 0.83250000, 0.67333333, 0.98607500, 0.42775000, 0.30200000
0.08716750, 0.00000000, 0.60333333, 0.00000000, 0.69300000, 0.64710000, 0.98800000, 0.08628500, 0.80000000, 0.83250000, 0.67333333, 0.97995000, 0.42800000, 0.26200000
0.25046100, 0.00000000, 0.60333333, 0.00000000, 0.69300000, 0.59870000, 1.00000000, 0.07944000, 0.80000000, 0.83250000, 0.67333333, 0.99225000, 0.66925000, 0.11200000
0.67920800, 0.00000000, 0.60333333, 0.00000000, 0.69300000, 0.56830000, 1.00000000, 0.07127000, 0.80000000, 0.83250000, 0.67333333, 0.96242500, 0.57450000, 0.10000000
0.51135800, 0.00000000, 0.60333333, 0.00000000, 0.59700000, 0.57570000, 1.00000000, 0.07065000, 0.80000000, 0.83250000, 0.67333333, 0.00650000, 0.25275000, 0.30000000
0.18084600, 0.00000000, 0.60333333, 0.00000000, 0.67900000, 0.64340000, 1.00000000, 0.09173500, 0.80000000, 0.83250000, 0.67333333, 0.06812500, 0.72625000, 0.14400000
0.11087400, 0.00000000, 0.60333333, 0.00000000, 0.71800000, 0.64110000, 1.00000000, 0.09294500, 0.80000000, 0.83250000, 0.67333333, 0.79687500, 0.37550000, 0.33400000
0.15860300, 0.00000000, 0.60333333, 0.00000000, 0.67900000, 0.58960000, 0.95400000, 0.09548000, 0.80000000, 0.83250000, 0.67333333, 0.01920000, 0.60975000, 0.16600000
0.08492130, 0.00000000, 0.60333333, 0.00000000, 0.58400000, 0.63480000, 0.86100000, 0.10263500, 0.80000000, 0.83250000, 0.67333333, 0.20862500, 0.44100000, 0.29000000
0.11160400, 0.00000000, 0.60333333, 0.00000000, 0.74000000, 0.66290000, 0.94600000, 0.10623500, 0.80000000, 0.83250000, 0.67333333, 0.27462500, 0.58175000, 0.26800000
0.22051100, 0.00000000, 0.60333333, 0.00000000, 0.74000000, 0.58180000, 0.92400000, 0.09331000, 0.80000000, 0.83250000, 0.67333333, 0.97862500, 0.55275000, 0.21000000
0.10671800, 0.00000000, 0.60333333, 0.00000000, 0.74000000, 0.64590000, 0.94800000, 0.09939500, 0.80000000, 0.83250000, 0.67333333, 0.10765000, 0.59950000, 0.23600000
0.06717720, 0.00000000, 0.60333333, 0.00000000, 0.71300000, 0.67490000, 0.92600000, 0.11618000, 0.80000000, 0.83250000, 0.67333333, 0.00080000, 0.43600000, 0.26800000
0.04752370, 0.00000000, 0.60333333, 0.00000000, 0.71300000, 0.65250000, 0.86500000, 0.12179000, 0.80000000, 0.83250000, 0.67333333, 0.12730000, 0.45325000, 0.28200000
0.04812130, 0.00000000, 0.60333333, 0.00000000, 0.71300000, 0.67010000, 0.90000000, 0.12987500, 0.80000000, 0.83250000, 0.67333333, 0.63807500, 0.41050000, 0.32800000
0.03163600, 0.00000000, 0.60333333, 0.00000000, 0.65500000, 0.57590000, 0.48200000, 0.15332500, 0.80000000, 0.83250000, 0.67333333, 0.83600000, 0.35325000, 0.39800000
0.04348790, 0.00000000, 0.60333333, 0.00000000, 0.58000000, 0.61670000, 0.84000000, 0.15167000, 0.80000000, 0.83250000, 0.67333333, 0.99225000, 0.40725000, 0.39800000
0.06393120, 0.00000000, 0.60333333, 0.00000000, 0.58400000, 0.61620000, 0.97400000, 0.11030000, 0.80000000, 0.83250000, 0.67333333, 0.75690000, 0.60250000, 0.26600000
0.05824010, 0.00000000, 0.60333333, 0.00000000, 0.53200000, 0.62420000, 0.64700000, 0.17121000, 0.80000000, 0.83250000, 0.67333333, 0.99225000, 0.26850000, 0.46000000
0.03673670, 0.00000000, 0.60333333, 0.00000000, 0.58300000, 0.63120000, 0.51900000, 0.19958500, 0.80000000, 0.83250000, 0.67333333, 0.97155000, 0.26450000, 0.42400000
0.00207460, 0.00000000, 0.92466667, 0.00000000, 0.60900000, 0.50930000, 0.98000000, 0.09113000, 0.13333333, 0.88875000, 0.67000000, 0.79607500, 0.74200000, 0.16200000
0.00178990, 0.00000000, 0.32300000, 0.00000000, 0.58500000, 0.56700000, 0.28800000, 0.13993000, 0.20000000, 0.48875000, 0.64000000, 0.98322500, 0.44000000, 0.46200000
0.00224380, 0.00000000, 0.32300000, 0.00000000, 0.58500000, 0.60270000, 0.79700000, 0.12491000, 0.20000000, 0.48875000, 0.64000000, 0.99225000, 0.35825000, 0.33600000
0.00062630, 0.00000000, 0.39766667, 0.00000000, 0.57300000, 0.65930000, 0.69100000, 0.12393000, 0.03333333, 0.34125000, 0.70000000, 0.97997500, 0.24175000, 0.44800000
0.00045270, 0.00000000, 0.39766667, 0.00000000, 0.57300000, 0.61200000, 0.76700000, 0.11437500, 0.03333333, 0.34125000, 0.70000000, 0.99225000, 0.22700000, 0.41200000
0.00060760, 0.00000000, 0.39766667, 0.00000000, 0.57300000, 0.69760000, 0.91000000, 0.10837500, 0.03333333, 0.34125000, 0.70000000, 0.99225000, 0.14100000, 0.47800000
0.00109590, 0.00000000, 0.39766667, 0.00000000, 0.57300000, 0.67940000, 0.89300000, 0.11944500, 0.03333333, 0.34125000, 0.70000000, 0.98362500, 0.16200000, 0.44000000
0.00047410, 0.00000000, 0.39766667, 0.00000000, 0.57300000, 0.60300000, 0.80800000, 0.12525000, 0.03333333, 0.34125000, 0.70000000, 0.99225000, 0.19700000, 0.23800000
Posted in Machine Learning | Leave a comment

New Version of Matrix Pseudo-Inverse With QR Decomposition (Householder Algorithm) Using C#

Posted on August 7, 2026 by jamesdmccaffrey

I recently (yesterday) made major revisions to my personal C# matrix QR decomposition using the Householder algorithm. In my work environment, I use matrix QR decomposition for matrix pseudo-inverse of a matrix of training data. So, my next step after a new QR decomposition function, presented in yesterday’s blog post, was to implement a new pseudo-inverse function.

Suppose matrix X holds training data (typically a design matrix, where a leading column of 1.0 values has been added to deal with the model bias). Then, to find the pseudo-inverse of X, pinv(X):

1. X = Q * R
2. pinv(X) = inv(Q * R)
3. pinv(X) = inv(Q) * inv(R)
4. pinv(X) = tr(Q) * inv(R)

In step 1, the matrix of training data X is decomposed using QR-Householder decomposition. The QR decomposition applies to any shape matrix, but regular inverse applies only to square matrices. Because training data almost always has more rows than columns, it’s possible to use a relaxed form that assumes this condition.

In step 2, the matrix inverse is applied to both sides of the equation.

In step 3, the matrix property inv(A*B) = inv(B) * inv(A) is applied.

In step 4, the inverse of Q is the transpose of Q, which is very easy to calculate. This is a special property of the Q result of QR decomposition, and is why QR decomposition is used. The R matrix is upper-triangular, and that inverse is a special case which is easy to compute.

Expressed in code:

public static double[][] MatPinv(double[][] M)
{
  double[][] Q; double[][] R;
  MatDecompQR(M, out Q, out R);  // Householder
  double[][] Ri = MatInvUpperTri(R);
  double[][] Qi = MatTranspose(Q);
  double[][] result = MatProduct(Ri, Qi);
  return result;
}

Most of the work is done in the MatDecompQR() function.

Here’s output of a demo run:

Relaxed  MP pseudo-inverse using QR decomposition
 Householder algorithm

Source (tall) matrix A:
   1.0   2.0   3.0   4.0   5.0
   0.0  -3.0   5.0  -7.0   9.0
   2.0   0.0  -2.0   0.0  -2.0
   4.0  -1.0   5.0   6.0   1.0
   3.0   6.0   8.0   2.0   2.0
   5.0  -2.0   4.0  -4.0   3.0

Computing pinv
Done

pseudo-inverse =
   0.0911  -0.0561   0.1650  -0.0250  -0.0144   0.1445
   0.0921  -0.0417   0.0755  -0.1379   0.0939   0.0053
  -0.1618   0.0430  -0.1421   0.1041   0.0790  -0.0414
   0.0664  -0.0250  -0.0145   0.0753  -0.0372  -0.0454
   0.1808   0.0376   0.0624  -0.0541  -0.0470   0.0104

End demo

I partially validated my relaxed MP pseudo-inverse by feeding the same source matrix to the NumPy np.linalg.pinv() function to make sure I got the same result.

OK. Good fun. My next step will be to use this new relaxed MP pseudo-inverse via QR-Householder functionality to train a machine learning regression problem (probably quadratic regression or maybe linear regression).


I don’t collect things. I collect ideas, such as a new algorithm to compute a pseudo-inverse.

But many people collect things as a hobby. The 1940 Superman Trading Cards by Gum, Inc. were the first superhero trading cards. There were 72 cards in the set. Collectors will pay a huge amount of money for these cards — a complete set in good condition could fetch several hundred thousand dollars.

Three men did the card artwork. Joe Shuster was the co-creator of Superman and did most of the cards. Paul Cassidy and Wayne Boring each did several of the cards. They also did Superman newspaper strips and comic books.

Here is card #1.

Demo program. Replace “lt” (less than), “gt”, “lte”, “gte” with Boolean operator symbols (my blog editor chokes on them).

using System;
using System.IO;

namespace MatrixPseudoInverseQRHouseholder
{
  internal class Program
  {
    static void Main(string[] args)
    {
      Console.WriteLine("\nRelaxed  MP pseudo-inverse " +
        "using QR decomposition Householder algorithm ");
            
      double[][] A = new double[6][];
      A[0] = new double[] { 1, 2, 3, 4, 5 };
      A[1] = new double[] { 0, -3, 5, -7, 9 };
      A[2] = new double[] { 2, 0, -2, 0, -2 };
      A[3] = new double[] { 4, -1, 5, 6, 1 };
      A[4] = new double[] { 3, 6, 8, 2, 2 };
      A[5] = new double[] { 5, -2, 4, -4, 3 };

      Console.WriteLine("\nSource (tall) matrix A: ");
      MatShow(A, 1, 6);

      Console.WriteLine("\nComputing pinv ");
      double[][] Pinv = QRHouseholder.MatPinv(A);
      Console.WriteLine("Done ");

      Console.WriteLine("\npseudo-inverse = ");
      MatShow(Pinv, 4, 9);
   
      Console.WriteLine("\nEnd demo ");
      Console.ReadLine();
    } // Main

    // ------------------------------------------------------

    static void MatShow(double[][] M, int dec, int wid)
    {
      for (int i = 0; i "lt" M.Length; ++i)
      {
        for (int j = 0; j "lt" M[0].Length; ++j)
        {
          double v = M[i][j];
          Console.Write(v.ToString("F" + dec).
            PadLeft(wid));
        }
        Console.WriteLine("");
      }
    }

  } // class Program

  // ========================================================

  public class QRHouseholder
  {
    // container class for MP pseudo-inverse via Householder
    // A = Q * R
    // pinv(A) = inv(R) * inv(Q)  note order matters
    //         = inv upper tri (easy) * transpose (easy)

    public static double[][] MatPinv(double[][] M)
    {
      double[][] Q; double[][] R;
      MatDecompQR(M, out Q, out R);  // Householder
      double[][] Ri = MatInvUpperTri(R);
      double[][] Qi = MatTranspose(Q);
      double[][] result = MatProduct(Ri, Qi);
      return result;
    }

    // ------------------------------------------------------

    public static double[][] MatInvUpperTri(double[][] U)
    {
      int n = U.Length;  // must be square matrix

      double[][] result = MatMake(n, n);
      for (int i = 0; i "lt" n; ++i)
        result[i][i] = 1.0;
      for (int k = 0; k "lt" n; ++k)
      {
        for (int j = 0; j "lt" n; ++j)
        {
          for (int i = 0; i "lt" k; ++i)
          {
            result[j][k] -= result[j][i] * U[i][k];
          }
          result[j][k] /= U[k][k];
        }
      }
      return result;
    }

    // ------------------------------------------------------

    public static double[][] MatMake(int nRows, int nCols)
    {
      double[][] result = new double[nRows][];
      for (int i = 0; i "lt" nRows; ++i)
        result[i] = new double[nCols];
      return result;
    }

    // ------------------------------------------------------

    public static double[][] MatTranspose(double[][] M)
    {
      int nRows = M.Length;
      int nCols = M[0].Length;
      double[][] result = MatMake(nCols, nRows);
      for (int i = 0; i "lt" nRows; ++i)
        for (int j = 0; j "lt" nCols; ++j)
          result[j][i] = M[i][j];
      return result;
    }

    // ------------------------------------------------------

    public static double[][] MatProduct(double[][] A,
      double[][] B)
    {
      int aRows = A.Length; int aCols = A[0].Length;
      int bRows = B.Length; int bCols = B[0].Length;
      if (aCols != bRows)
        throw new Exception("Non-conformable matrices");

      double[][] result = new double[aRows][];
      for (int i = 0; i "lt" aRows; ++i)
        result[i] = new double[bCols];

      for (int i = 0; i "lt" aRows; ++i) // each row of A
        for (int j = 0; j "lt" bCols; ++j) // each col of B
          for (int k = 0; k "lt" aCols; ++k)
            result[i][j] += A[i][k] * B[k][j];

      return result;
    }

    // ------------------------------------------------------

    public static void MatDecompQR(double[][] A, 
      out double[][] Q,  out double[][] R)
    {
      int m = A.Length; int n = A[0].Length;
      if (m "lt" n)
        Console.WriteLine("FATAL: nRows must be gte nCols");

      double[][] QQ = MatMake(m, m); // working full Q
      for (int i = 0; i "lt" m; ++i)
        QQ[i][i] = 1.0;  // identity matrix

      double[][] RR = MatMake(m, n);
      for (int i = 0; i "lt" m; ++i)
        for (int j = 0; j "lt" n; ++j)
          RR[i][j] = A[i][j]; // copy of A is working R

      int k = Math.Min(m, n);  // or just use n
      for (int j = 0; j "lt" k; ++j) // main processing loop
      {
        int xn = m - j;
        double[] x = new double[xn];
        for (int i = 0; i "lt" xn; ++i)
          x[i] = RR[j + i][j];

        double ss = 0.0;
        for (int i = 0; i "lt" xn; ++i)
          ss += x[i] * x[i];
        double normX = Math.Sqrt(ss);

        // if (normX == 0.0) continue;
        if (Math.Abs(normX) "lt" 1.0e-12) continue;

        double sign;
        if (x[0] "gte" 0.0) sign = -1.0;
        else sign = 1.0; // counter-intuitive
      
        double[] u = new double[xn];
        for (int i = 0; i "lt" xn; ++i)
          u[i] = x[i] / (x[0] - sign * normX); // check div 0
        u[0] = 1.0;

        // compute scaling factor tau = 2 / (u^T * u)
        double tau = -sign * (x[0] - sign * normX) / normX;

        // dimensions for sub-matrices
        int nRowsSubR = m - j;   int nColsSubR = n - j;
        int nRowsSubQ = m;       int nColsSubQ = m - j;

        double[] vr = new double[nColsSubR];
        for (int c = 0; c "lt" nColsSubR; ++c)
        {
          double acc = 0.0;
          for (int r = 0; r "lt" nRowsSubR; ++r)
            acc += u[r] * RR[j + r][j + c];
          vr[c] = acc;
        }

        double[] vq = new double[nRowsSubQ];
        for (int r = 0; r "lt" nRowsSubQ; ++r)
        {
          double acc = 0.0;
          for (int c = 0; c "lt" nColsSubQ; ++c)
            acc += u[c] * QQ[r][j + c];
          vq[r] = acc;
        }

        // update sub-R
        for (int r = 0; r "lt" nRowsSubR; ++r)
          for (int c = 0; c "lt" nColsSubR; ++c)
            RR[j + r][j + c] -= tau * u[r] * vr[c];

        // update sub-Q
        for (int r = 0; r "lt" nRowsSubQ; ++r)
          for (int c = 0; c "lt" nColsSubQ; ++c)
            QQ[r][j + c] -= tau * vq[r] * u[c];
       
      } // j

      // extract QQ RR into out params
      Q = MatMake(m, n);
      for (int i = 0; i "lt" m; ++i)
        for (int j = 0; j "lt" n; ++j)
          Q[i][j] = QQ[i][j];

      R = MatMake(n, n);
      for (int i = 0; i "lt" n; ++i)
        for (int j = 0; j "lt" n; ++j)
          R[i][j] = RR[i][j];

      return;
    } // MatDecompQR

  } // class QRHouseholder

  // ========================================================

} // ns

Python validation code:

  import numpy as np

  A = np.array([
    [ 1, 2, 3, 4, 5 ],
    [ 0, -3, 5, -7, 9 ],
    [ 2, 0, -2, 0, -2 ],
    [ 4, -1, 5, 6, 1 ],
    [ 3, 6, 8, 2, 2 ],
    [ 5, -2, 4, -4, 3 ]], dtype=np.float64)

  print("\nA = "); print(A)

  Pinv = np.linalg.pinv(A)
  print("\nNumPy pseudo-inverse = "); print(Pinv)
Posted in Machine Learning | Leave a comment

Deep Neural Network Regression From Scratch Using JavaScript

Posted on August 6, 2026 by jamesdmccaffrey

One morning before work, I realized that I hadn’t written any JavaScript code for several weeks. For mental exercise, I decided to implement a regression system (to predict a single numeric value), using a neural network with exactly two hidden layers, from scratch, using JavaScript.

The effort was an interesting challenge. It didn’t take me very long (about 90 minutes) because I had recently coded the same system using both C# and Python, and all the ideas were still fresh in my head.

The output of my demo:

Begin JavaScript deep NN regression demo

Loading synthetic train (200) and test (40) data
Done

First three train X:
 -0.1660   0.4406  -0.9998  -0.3953  -0.7065
  0.0776  -0.1616   0.3704  -0.5911   0.7562
 -0.9452   0.3409  -0.1654   0.1174  -0.7192

First three train y:
0.4840
0.1568
0.8054

Creating 5-10-10-1 tanh() identity() neural network regressor
Done

Setting lrnRate = 0.05
Setting maxEpochs = 8000
epoch:      0   MSE =   0.0369   acc = 0.1050
epoch:    800   MSE =   0.0005   acc = 0.7700
epoch:   1600   MSE =   0.0004   acc = 0.8250
epoch:   2400   MSE =   0.0003   acc = 0.8550
epoch:   3200   MSE =   0.0002   acc = 0.9150
epoch:   4000   MSE =   0.0002   acc = 0.8950
epoch:   4800   MSE =   0.0002   acc = 0.8900
epoch:   5600   MSE =   0.0001   acc = 0.9000
epoch:   6400   MSE =   0.0001   acc = 0.9050
epoch:   7200   MSE =   0.0001   acc = 0.9100
Done

Evaluating model

Accuracy (10%) on training data = 0.9400
Accuracy (10%) on test data     = 0.9500

MSE on training data = 0.0001
MSE on test data     = 0.0002

Predicting y for train[0]
Predicted y = 0.4815

End demo

The accuracy results of the JavaScript implementation were essentially the same as the results from the C# and Python implementations. The JavaScript version was a bit slower than the C# version, but significantly faster than the Python version.

I used one of my standard synthetic datasets. The data looks like:

-0.1660,  0.4406, -0.9998, -0.3953, -0.7065,  0.4840
 0.0776, -0.1616,  0.3704, -0.5911,  0.7562,  0.1568
-0.9452,  0.3409, -0.1654,  0.1174, -0.7192,  0.8054
. . .

The first five values on each line are the predictors. The last value is the target to predict. There are 200 training items and 400 test items.

In theory (the Universal Approximation Theorem), any neural network regression system with two hidden layers can be implemented using a neural network with a single hidden layer. But in practice, using two hidden layers often leads to a better prediction model.


The JavaScript language is filled with ironies. For example, JavaScript is a language that was written in just 10 days as a quick utility for web animations, but is now arguably the most dominant and omnipresent programming language in the world.

Here are two examples of job-hiring irony.

Left: The “Accepting Resumes” sign for the Verizon mobile phone company is unfortunately placed relative to the garbage can. The woman pointing directly to the garbage can opening is a nice, added touch.

Right: The “Now Hiring” sign for the Sports Authority sporting goods store loses a lot of credibility due to the placement directly below a sign with a decidedly different message.

Demo program. Replace “lt” (less than), “gt”, “lte”, “gte” with Boolean operator symbols (my blog editor chokes on symbols).

// neural_net_deep_regression.js
// node.js  ES6

// NN with 2 hidden layers regression
// tanh, identity activations

let FS = require("fs")  // to load data from file

// ==========================================================

class NeuralNetDeepRegressor
{
  constructor(numInput, numHiddenA, numHiddenB, numOutput,
    seed)
  {
    this.rnd = new Erratic(seed);  // pseudo-pseudo-random

    this.numInput = numInput; 
    this.numHiddenA = numHiddenA;
    this.numHiddenB = numHiddenB;
    this.numOutput = numOutput;

    this.iNodes = vecMake(numInput, 0.0);
    this.aNodes = vecMake(numHiddenA, 0.0);
    this.bNodes = vecMake(numHiddenB, 0.0);  
    this.oNodes = vecMake(numOutput, 0.0);

    this.iaWeights = matMake(numInput, numHiddenA, 0.0);
    this.abWeights = matMake(numHiddenA, numHiddenB, 0.0);
    this.boWeights = matMake(numHiddenB, numOutput, 0.0);

    this.aBiases = vecMake(numHiddenA, 0.0);
    this.bBiases = vecMake(numHiddenB, 0.0);
    this.oBiases = vecMake(numOutput, 0.0); // [1] 
  }

  // --------------------------------------------------------
  // predict(), train(), MSE(), accuracy()
  // --------------------------------------------------------

  predict(x)
  {
    // copy input into iNodes
    for (let i = 0; i "lt" this.numInput; ++i)
      this.iNodes[i] = x[i];

    // compute hidden layer A
    for (let j = 0; j "lt" this.numHiddenA; ++j) {
      let sum = 0.0;
      for (let i = 0; i "lt" this.numInput; ++i)
        sum += this.iNodes[i] * this.iaWeights[i][j];
      sum += this.aBiases[j];
      this.aNodes[j] = this.hyperTan(sum);
    }

    // compute hidden layer B
    for (let j = 0; j "lt" this.numHiddenB; ++j) {
      let sum = 0.0;
      for (let i = 0; i "lt" this.numHiddenA; ++i)
        sum += this.aNodes[i] * this.abWeights[i][j];
      sum += this.bBiases[j];
      this.bNodes[j] = this.hyperTan(sum);
    }

    // compute output layer
    for (let j = 0; j "lt" this.numOutput; ++j) {
      let sum = 0.0;
      for (let i = 0; i "lt" this.numHiddenB; ++i)
        sum += this.bNodes[i] * this.boWeights[i][j];
      sum += this.oBiases[j];
      this.oNodes[j] = this.identity(sum);
    }

    return this.oNodes[0];  // a single scalar value
  }

  // --------------------------------------------------------

  train(trainX, trainY, lrnRate, maxEpochs)
  {
    // init weights
    let lo = -0.01; let hi = 0.01;

    for (let i = 0; i "lt" this.numInput; ++i)
      for (let j = 0; j "lt" this.numHiddenA; ++j)
        this.iaWeights[i][j] =
          (hi - lo) * this.rnd.next() + lo;

    for (let i = 0; i "lt" this.numHiddenA; ++i)
      for (let j = 0; j "lt" this.numHiddenB; ++j)
        this.abWeights[i][j] =
          (hi - lo) * this.rnd.next() + lo;

    for (let i = 0; i "lt" this.numHiddenB; ++i)
      for (let j = 0; j "lt" this.numOutput; ++j)
        this.boWeights[i][j] =
          (hi - lo) * this.rnd.next() + lo;

    // each weight and bias has a gradient
    let boGrads = matMake(this.numHiddenB,
      this.numOutput, 0.0);
    let abGrads = matMake(this.numHiddenA,
      this.numHiddenB, 0.0);
    let iaGrads = matMake(this.numInput,
      this.numHiddenA, 0.0);

    let oBiasGrads = vecMake(this.numOutput, 0.0);
    let bBiasGrads = vecMake(this.numHiddenB, 0.0);
    let aBiasGrads = vecMake(this.numHiddenA, 0.0);

    // each output and hidden node has a 'signal'
    //  which is gradient without associated input
    //  (lower case delta in Wikipedia)
    let oSignals = vecMake(this.numOutput, 0.0);
    let bSignals = vecMake(this.numHiddenB, 0.0);
    let aSignals = vecMake(this.numHiddenA, 0.0);

    let indices = vecMake(trainX.length, 0.0);
    for (let i = 0; i "lt" indices.length; ++i)
      indices[i] = i;

    let freq = Math.trunc(maxEpochs / 10); // progress freq
    for (let epoch = 0; epoch "lt" maxEpochs; ++epoch) {
      this.shuffle(indices);
      for (let ii = 0; ii "lt" trainX.length; ++ii) {
        let idx = indices[ii];
        let x = trainX[idx];
        let actualY = trainY[idx];
        let predY = this.predict(x);

        // output node signals depends on target values
        for (let k = 0; k "lt" this.numOutput; ++k) {
          let error = predY - actualY;  // standard form
          let derivative = 1.0; // identity activation
          oSignals[k] = error * derivative;
        }

        // signal for B nodes depends on output signals
        for (let j = 0; j "lt" this.numHiddenB; ++j) {
          let derivative =
            (1 + this.bNodes[j]) * (1 - this.bNodes[j]);
          let sum = 0.0;
          for (let k = 0; k "lt" this.numOutput; ++k)
            sum += oSignals[k] * this.boWeights[j][k];
          bSignals[j] = derivative * sum;
        }

        // signal for A nodes, depends on B signals
        for (let j = 0; j "lt" this.numHiddenA; ++j) {
          let derivative =
            (1 + this.aNodes[j]) * (1 - this.aNodes[j]);
          let sum = 0.0;
          for (let k = 0; k "lt" this.numHiddenB; ++k)
            sum += bSignals[k] * this.abWeights[j][k];
          aSignals[j] = derivative * sum;
        }

        // at this point, all signals have been computed
        // use signals to calculate gradients left-to-right

        for (let i = 0; i "lt" this.numInput; ++i)
          for (let j = 0; j "lt" this.numHiddenA; ++j)
            iaGrads[i][j] = this.iNodes[i] * aSignals[j];

        for (let i = 0; i "lt" this.numHiddenA; ++i)
          for (let j = 0; j "lt" this.numHiddenB; ++j)
            abGrads[i][j] = this.aNodes[i] * bSignals[j];

        for (let i = 0; i "lt" this.numHiddenB; ++i)
          for (let j = 0; j "lt" this.numOutput; ++j)
            boGrads[i][j] = this.bNodes[i] * oSignals[j];

        // compute bias gradients
        for (let j = 0; j "lt" this.numHiddenA; ++j)
          aBiasGrads[j] = 1.0 * aSignals[j];
        for (let j = 0; j "lt" this.numHiddenB; ++j)
          bBiasGrads[j] = 1.0 * bSignals[j];
        for (let j = 0; j "lt" this.numOutput; ++j)
          oBiasGrads[j] = 1.0 * oSignals[j];

        // use gradients to update all weights

        for (let i = 0; i "lt" this.numInput; ++i)
          for (let j = 0; j "lt" this.numHiddenA; ++j)
            this.iaWeights[i][j] -= iaGrads[i][j] * lrnRate;

        for (let i = 0; i "lt" this.numHiddenA; ++i)
          for (let j = 0; j "lt" this.numHiddenB; ++j)
            this.abWeights[i][j] -= abGrads[i][j] * lrnRate;

        for (let i = 0; i "lt" this.numHiddenB; ++i)
          for (let j = 0; j "lt" this.numOutput; ++j)
            this.boWeights[i][j] -= boGrads[i][j] * lrnRate;

        // update all biases

        for (let j = 0; j "lt" this.numHiddenA; ++j)
          this.aBiases[j] -= aBiasGrads[j] * lrnRate;

        for (let j = 0; j "lt" this.numHiddenB; ++j)
          this.bBiases[j] -= bBiasGrads[j] * lrnRate;

        for (let j = 0; j "lt" this.numOutput; ++j)
          this.oBiases[j] -= oBiasGrads[j] * lrnRate;

      } // ii each train item

      if (epoch % freq == 0) {
        let mse = 
          this.MSE(trainX, trainY).toFixed(4);
        let acc = 
          this.accuracy(trainX, trainY, 0.10).toFixed(4);

        let s1 = "epoch: " +
          epoch.toString().padStart(6, ' ');
        let s2 = "   MSE = " + 
          mse.toString().padStart(8, ' ');
        let s3 = "   acc = " + acc.toString();
        console.log(s1 + s2 + s3);
      }

    } // epoch

    return;
  }

  // --------------------------------------------------------

  MSE(dataX, dataY)
  {
    let sum = 0.0;
    let n = dataX.length;
    for (let i = 0; i "lt" n; ++i) {
      let x = dataX[i];
      let actualY = dataY[i];  // target
      let predY = this.predict(x); 
      sum += (predY - actualY) * (predY - actualY); 
    }
    return sum / n; 
  } 

  // --------------------------------------------------------

  accuracy(dataX, dataY, pctClose)
  {
    let n = dataX.length;
    let nCorrect = 0; let nWrong = 0;
    for (let i = 0; i "lt" n; ++i) { 
      let x = dataX[i];
      let actualY = dataY[i];
      let predY = this.predict(x);
      if ( Math.abs(actualY - predY) "lt" 
        Math.abs(actualY * pctClose) ) {
        ++nCorrect;
      }
      else {
        ++nWrong;
      }
    }
    return nCorrect / (nCorrect + nWrong);
  }

  // --------------------------------------------------------
  // helpers: shuffle(), hyperTan(), identity(), 
  // --------------------------------------------------------

  shuffle(v)
  {
    // Fisher-Yates
    let n = v.length;
    for (let i = 0; i "lt" n; ++i) {
      let r = this.rnd.nextInt(i, n);
      let tmp = v[r];
      v[r] = v[i];
      v[i] = tmp;
    }
  }

  // -------------------------------------------------------- 

  hyperTan(x)
  {
    if (x "lt" -8.0) {
      return -1.0;
    }
    else if (x "gt" 8.0) {
      return 1.0;
    }
    else {
      return Math.tanh(x);
    }

  } 

  // --------------------------------------------------------

  identity(x)
  {
    return x;
  } 

  // --------------------------------------------------------

} // class NeuralNetDeepRegressor

// ==========================================================

// helpers: loadTxt(), class Erratic, vecMake(), matMake(),
//  matToVec(), vecShow(), vecShow(), matShow()

// ----------------------------------------------------------

function loadTxt(fn, delimit, usecols, comment) {
  // efficient but mildly complicated
  let all = FS.readFileSync(fn, "utf8");  // giant string
  all = all.trim();  // strip final crlf in file
  let lines = all.split("\n");  // array of lines

  // count number non-comment lines
  let nRows = 0;
  for (let i = 0; i "lt" lines.length; ++i) {
    if (!lines[i].startsWith(comment))
      ++nRows;
  }
  let nCols = usecols.length;
  let result = matMake(nRows, nCols, 0.0); 
 
  let r = 0;  // into lines
  let i = 0;  // into result[][]
  while (r "lt" lines.length) {
    if (lines[r].startsWith(comment)) {
      ++r;  // next row
    }
    else {
      let tokens = lines[r].split(delimit);
      for (let j = 0; j "lt" nCols; ++j) {
        result[i][j] = parseFloat(tokens[usecols[j]]);
      }
      ++r;
      ++i;
    }
  }

  return result;
}

// ----------------------------------------------------------

class Erratic
{
  constructor(seed)
  {
    this.seed = seed + 0.5;  // avoid 0
  }

  next()
  {
    let x = Math.sin(this.seed) * 1000;
    let result = x - Math.floor(x);  // [0.0,1.0)
    this.seed = result;  // for next call
    return result;
  }

  nextInt(lo, hi)
  {
    let x = this.next();
    return Math.trunc((hi - lo) * x + lo);
  }
}

// ----------------------------------------------------------

function vecMake(n, val)
{
  let result = [];
  for (let i = 0; i "lt" n; ++i) {
    result[i] = val;
  }
  return result;
}

// ----------------------------------------------------------

function matMake(rows, cols, val)
{
  let result = [];
  for (let i = 0; i "lt" rows; ++i) {
    result[i] = [];
    for (let j = 0; j "lt" cols; ++j) {
      result[i][j] = val;
    }
  }
  return result;
}

// ----------------------------------------------------------

function matToVec(m)
{
  let r = m.length;
  let c = m[0].length;
  let result = 	vecMake(r*c, 0.0);
  let k = 0;
  for (let i = 0; i "lt" r; ++i) {
    for (let j = 0; j "lt" c; ++j) {
      result[k++] = m[i][j];
    }
  }
  return result;
}

// ----------------------------------------------------------

function vecShow(v, dec, len)
{
  for (let i = 0; i "lt" v.length; ++i) {
    if (i != 0 && i % len == 0) {
      process.stdout.write("\n");
    }
    if (v[i] "gte" 0.0) {
      process.stdout.write(" ");  // + or - space
    }
    process.stdout.write(v[i].toFixed(dec));
    process.stdout.write("  ");
  }
  process.stdout.write("\n");
}

// ----------------------------------------------------------

function vecShow(vec, dec, wid, nl)
{
  for (let i = 0; i "lt" vec.length; ++i) {
    let x = vec[i];
    if (Math.abs(x) "lt" 0.000001) x = 0.0  // avoid -0.00
    let xx = x.toFixed(dec);
    let s = xx.toString().padStart(wid, ' ');
    process.stdout.write(s);
    process.stdout.write(" ");
  }

  if (nl == true)
    process.stdout.write("\n");
}

// ----------------------------------------------------------

function matShow(m, dec, wid)
{
  let rows = m.length;
  let cols = m[0].length;
  for (let i = 0; i "lt" rows; ++i) {
    for (let j = 0; j "lt" cols; ++j) {
      if (m[i][j] "gte" 0.0) {
        process.stdout.write(" ");  // + or - space
      }
      process.stdout.write(m[i][j].toFixed(dec));
      process.stdout.write("  ");
    }
    process.stdout.write("\n");
  }
}

// ==========================================================

function main()
{
  // process.stdout.write("\033[0m");  // reset
  // process.stdout.write("\x1b[1m" + "\x1b[37m"); // white
  console.log("\nBegin JavaScript deep NN regression demo ");
  
  // 1. load data
  console.log("\nLoading synthetic train" +
    " (200) and test (40) data");
  let trainFile = ".\\Data\\synthetic_train_200.txt";
  let trainX = loadTxt(trainFile, ",", [0,1,2,3,4], "#");
  let trainY = loadTxt(trainFile, ",", [5], "#");
  trainY = matToVec(trainY);

  let testFile = ".\\Data\\synthetic_test_40.txt"
  let testX = loadTxt(testFile, ",", [0,1,2,3,4], "#");
  let testY = loadTxt(testFile, ",", [5], "#");
  testY = matToVec(testY);
  console.log("Done ");
  
  console.log("\nFirst three train X: ");
  for (let i = 0; i "lt" 3; ++i)
    vecShow(trainX[i], 4, 8, true);  // vec, dec, wid, nl

  console.log("\nFirst three train y: ");
  for (let i = 0; i "lt" 3; ++i)
    console.log(trainY[i].toFixed(4).toString().
      padStart(2, ' '));

  // 2. create network
  console.log("\nCreating 5-10-10-1 tanh()" +
    " identity() neural network regressor  ");
  let seed = 1;
  let nn = new NeuralNetDeepRegressor(5, 10, 10, 1, seed);
  console.log("Done ");

  // 3. train network
  let lrnRate = 0.05;
  let maxEpochs = 8000;
  console.log("\nSetting lrnRate = 0.05 ");
  console.log("Setting maxEpochs = 8000 ");
  nn.train(trainX, trainY, lrnRate, maxEpochs);
  console.log("Done ");

  // 4. evaluate model
  console.log("\nEvaluating model ");
  let trainAcc = nn.accuracy(trainX, trainY, 0.10);
  let testAcc = nn.accuracy(testX, testY, 0.10);
  console.log("\nAccuracy (10%) on training data = " +
    trainAcc.toFixed(4).toString()); 
  console.log("Accuracy (10%) on test data     = " +
    testAcc.toFixed(4).toString());

  let trainMSE = nn.MSE(trainX, trainY);
  let testMSE = nn.MSE(testX, testY);
  console.log("\nMSE on training data = " +
    trainMSE.toFixed(4).toString()); 
  console.log("MSE on test data     = " +
    testMSE.toFixed(4).toString());

  // 5. use trained model
  console.log("\nPredicting y for train[0] ");
  let x = trainX[0];
  let predY = nn.predict(x);
  console.log("Predicted y = " + 
    predY.toFixed(4).toString());
  //console.log(predY.toFixed(4).toString());

  //process.stdout.write("\033[0m");  // reset
  console.log("\nEnd demo");
}

main()

Training data:

# synthetic_train_200.txt
#
-0.1660,  0.4406, -0.9998, -0.3953, -0.7065,  0.4840
 0.0776, -0.1616,  0.3704, -0.5911,  0.7562,  0.1568
-0.9452,  0.3409, -0.1654,  0.1174, -0.7192,  0.8054
 0.9365, -0.3732,  0.3846,  0.7528,  0.7892,  0.1345
-0.8299, -0.9219, -0.6603,  0.7563, -0.8033,  0.7955
 0.0663,  0.3838, -0.3690,  0.3730,  0.6693,  0.3206
-0.9634,  0.5003,  0.9777,  0.4963, -0.4391,  0.7377
-0.1042,  0.8172, -0.4128, -0.4244, -0.7399,  0.4801
-0.9613,  0.3577, -0.5767, -0.4689, -0.0169,  0.6861
-0.7065,  0.1786,  0.3995, -0.7953, -0.1719,  0.5569
 0.3888, -0.1716, -0.9001,  0.0718,  0.3276,  0.2500
 0.1731,  0.8068, -0.7251, -0.7214,  0.6148,  0.3297
-0.2046, -0.6693,  0.8550, -0.3045,  0.5016,  0.2129
 0.2473,  0.5019, -0.3022, -0.4601,  0.7918,  0.2613
-0.1438,  0.9297,  0.3269,  0.2434, -0.7705,  0.5171
 0.1568, -0.1837, -0.5259,  0.8068,  0.1474,  0.3307
-0.9943,  0.2343, -0.3467,  0.0541,  0.7719,  0.5581
 0.2467, -0.9684,  0.8589,  0.3818,  0.9946,  0.1092
-0.6553, -0.7257,  0.8652,  0.3936, -0.8680,  0.7018
 0.8460,  0.4230, -0.7515, -0.9602, -0.9476,  0.1996
-0.9434, -0.5076,  0.7201,  0.0777,  0.1056,  0.5664
 0.9392,  0.1221, -0.9627,  0.6013, -0.5341,  0.1533
 0.6142, -0.2243,  0.7271,  0.4942,  0.1125,  0.1661
 0.4260,  0.1194, -0.9749, -0.8561,  0.9346,  0.2230
 0.1362, -0.5934, -0.4953,  0.4877, -0.6091,  0.3810
 0.6937, -0.5203, -0.0125,  0.2399,  0.6580,  0.1460
-0.6864, -0.9628, -0.8600, -0.0273,  0.2127,  0.5387
 0.9772,  0.1595, -0.2397,  0.1019,  0.4907,  0.1611
 0.3385, -0.4702, -0.8673, -0.2598,  0.2594,  0.2270
-0.8669, -0.4794,  0.6095, -0.6131,  0.2789,  0.4700
 0.0493,  0.8496, -0.4734, -0.8681,  0.4701,  0.3516
 0.8639, -0.9721, -0.5313,  0.2336,  0.8980,  0.1412
 0.9004,  0.1133,  0.8312,  0.2831, -0.2200,  0.1782
 0.0991,  0.8524,  0.8375, -0.2102,  0.9265,  0.2150
-0.6521, -0.7473, -0.7298,  0.0113, -0.9570,  0.7422
 0.6190, -0.3105,  0.8802,  0.1640,  0.7577,  0.1056
 0.6895,  0.8108, -0.0802,  0.0927,  0.5972,  0.2214
 0.1982, -0.9689,  0.1870, -0.1326,  0.6147,  0.1310
-0.3695,  0.7858,  0.1557, -0.6320,  0.5759,  0.3773
-0.1596,  0.3581,  0.8372, -0.9992,  0.9535,  0.2071
-0.2468,  0.9476,  0.2094,  0.6577,  0.1494,  0.4132
 0.1737,  0.5000,  0.7166,  0.5102,  0.3961,  0.2611
 0.7290, -0.3546,  0.3416, -0.0983, -0.2358,  0.1332
-0.3652,  0.2438, -0.1395,  0.9476,  0.3556,  0.4170
-0.6029, -0.1466, -0.3133,  0.5953,  0.7600,  0.4334
-0.4596, -0.4953,  0.7098,  0.0554,  0.6043,  0.2775
 0.1450,  0.4663,  0.0380,  0.5418,  0.1377,  0.2931
-0.8636, -0.2442, -0.8407,  0.9656, -0.6368,  0.7429
 0.6237,  0.7499,  0.3768,  0.1390, -0.6781,  0.2185
-0.5499,  0.1850, -0.3755,  0.8326,  0.8193,  0.4399
-0.4858, -0.7782, -0.6141, -0.0008,  0.4572,  0.4197
 0.7033, -0.1683,  0.2334, -0.5327, -0.7961,  0.1776
 0.0317, -0.0457, -0.6947,  0.2436,  0.0880,  0.3345
 0.5031, -0.5559,  0.0387,  0.5706, -0.9553,  0.3107
-0.3513,  0.7458,  0.6894,  0.0769,  0.7332,  0.3170
 0.2205,  0.5992, -0.9309,  0.5405,  0.4635,  0.3532
-0.4806, -0.4859,  0.2646, -0.3094,  0.5932,  0.3202
 0.9809, -0.3995, -0.7140,  0.8026,  0.0831,  0.1600
 0.9495,  0.2732,  0.9878,  0.0921,  0.0529,  0.1289
-0.9476, -0.6792,  0.4913, -0.9392, -0.2669,  0.5966
 0.7247,  0.3854,  0.3819, -0.6227, -0.1162,  0.1550
-0.5922, -0.5045, -0.4757,  0.5003, -0.0860,  0.5863
-0.8861,  0.0170, -0.5761,  0.5972, -0.4053,  0.7301
 0.6877, -0.2380,  0.4997,  0.0223,  0.0819,  0.1404
 0.9189,  0.6079, -0.9354,  0.4188, -0.0700,  0.1907
-0.1428, -0.7820,  0.2676,  0.6059,  0.3936,  0.2790
 0.5324, -0.3151,  0.6917, -0.1425,  0.6480,  0.1071
-0.8432, -0.9633, -0.8666, -0.0828, -0.7733,  0.7784
-0.9444,  0.5097, -0.2103,  0.4939, -0.0952,  0.6787
-0.0520,  0.6063, -0.1952,  0.8094, -0.9259,  0.4836
 0.5477, -0.7487,  0.2370, -0.9793,  0.0773,  0.1241
 0.2450,  0.8116,  0.9799,  0.4222,  0.4636,  0.2355
 0.8186, -0.1983, -0.5003, -0.6531, -0.7611,  0.1511
-0.4714,  0.6382, -0.3788,  0.9648, -0.4667,  0.5950
 0.0673, -0.3711,  0.8215, -0.2669, -0.1328,  0.2677
-0.9381,  0.4338,  0.7820, -0.9454,  0.0441,  0.5518
-0.3480,  0.7190,  0.1170,  0.3805, -0.0943,  0.4724
-0.9813,  0.1535, -0.3771,  0.0345,  0.8328,  0.5438
-0.1471, -0.5052, -0.2574,  0.8637,  0.8737,  0.3042
-0.5454, -0.3712, -0.6505,  0.2142, -0.1728,  0.5783
 0.6327, -0.6297,  0.4038, -0.5193,  0.1484,  0.1153
-0.5424,  0.3282, -0.0055,  0.0380, -0.6506,  0.6613
 0.1414,  0.9935,  0.6337,  0.1887,  0.9520,  0.2540
-0.9351, -0.8128, -0.8693, -0.0965, -0.2491,  0.7353
 0.9507, -0.6640,  0.9456,  0.5349,  0.6485,  0.1059
-0.0462, -0.9737, -0.2940, -0.0159,  0.4602,  0.2606
-0.0627, -0.0852, -0.7247, -0.9782,  0.5166,  0.2977
 0.0478,  0.5098, -0.0723, -0.7504, -0.3750,  0.3335
 0.0090,  0.3477,  0.5403, -0.7393, -0.9542,  0.4415
-0.9748,  0.3449,  0.3736, -0.1015,  0.8296,  0.4358
 0.2887, -0.9895, -0.0311,  0.7186,  0.6608,  0.2057
 0.1570, -0.4518,  0.1211,  0.3435, -0.2951,  0.3244
 0.7117, -0.6099,  0.4946, -0.4208,  0.5476,  0.1096
-0.2929, -0.5726,  0.5346, -0.3827,  0.4665,  0.2465
 0.4889, -0.5572, -0.5718, -0.6021, -0.7150,  0.2163
-0.7782,  0.3491,  0.5996, -0.8389, -0.5366,  0.6516
-0.5847,  0.8347,  0.4226,  0.1078, -0.3910,  0.6134
 0.8469,  0.4121, -0.0439, -0.7476,  0.9521,  0.1571
-0.6803, -0.5948, -0.1376, -0.1916, -0.7065,  0.7156
 0.2878,  0.5086, -0.5785,  0.2019,  0.4979,  0.2980
 0.2764,  0.1943, -0.4090,  0.4632,  0.8906,  0.2960
-0.8877,  0.6705, -0.6155, -0.2098, -0.3998,  0.7107
-0.8398,  0.8093, -0.2597,  0.0614, -0.0118,  0.6502
-0.8476,  0.0158, -0.4769, -0.2859, -0.7839,  0.7715
 0.5751, -0.7868,  0.9714, -0.6457,  0.1448,  0.1175
 0.4802, -0.7001,  0.1022, -0.5668,  0.5184,  0.1090
 0.4458, -0.6469,  0.7239, -0.9604,  0.7205,  0.0779
 0.5175,  0.4339,  0.9747, -0.4438, -0.9924,  0.2879
 0.8678,  0.7158,  0.4577,  0.0334,  0.4139,  0.1678
 0.5406,  0.5012,  0.2264, -0.1963,  0.3946,  0.2088
-0.9938,  0.5498,  0.7928, -0.5214, -0.7585,  0.7687
 0.7661,  0.0863, -0.4266, -0.7233, -0.4197,  0.1466
 0.2277, -0.3517, -0.0853, -0.1118,  0.6563,  0.1767
 0.3499, -0.5570, -0.0655, -0.3705,  0.2537,  0.1632
 0.7547, -0.1046,  0.5689, -0.0861,  0.3125,  0.1257
 0.8186,  0.2110,  0.5335,  0.0094, -0.0039,  0.1391
 0.6858, -0.8644,  0.1465,  0.8855,  0.0357,  0.1845
-0.4967,  0.4015,  0.0805,  0.8977,  0.2487,  0.4663
 0.6760, -0.9841,  0.9787, -0.8446, -0.3557,  0.1509
-0.1203, -0.4885,  0.6054, -0.0443, -0.7313,  0.4854
 0.8557,  0.7919, -0.0169,  0.7134, -0.1628,  0.2002
 0.0115, -0.6209,  0.9300, -0.4116, -0.7931,  0.4052
-0.7114, -0.9718,  0.4319,  0.1290,  0.5892,  0.3661
 0.3915,  0.5557, -0.1870,  0.2955, -0.6404,  0.2954
-0.3564, -0.6548, -0.1827, -0.5172, -0.1862,  0.4622
 0.2392, -0.4959,  0.5857, -0.1341, -0.2850,  0.2470
-0.3394,  0.3947, -0.4627,  0.6166, -0.4094,  0.5325
 0.7107,  0.7768, -0.6312,  0.1707,  0.7964,  0.2757
-0.1078,  0.8437, -0.4420,  0.2177,  0.3649,  0.4028
-0.3139,  0.5595, -0.6505, -0.3161, -0.7108,  0.5546
 0.4335,  0.3986,  0.3770, -0.4932,  0.3847,  0.1810
-0.2562, -0.2894, -0.8847,  0.2633,  0.4146,  0.4036
 0.2272,  0.2966, -0.6601, -0.7011,  0.0284,  0.2778
-0.0743, -0.1421, -0.0054, -0.6770, -0.3151,  0.3597
-0.4762,  0.6891,  0.6007, -0.1467,  0.2140,  0.4266
-0.4061,  0.7193,  0.3432,  0.2669, -0.7505,  0.6147
-0.0588,  0.9731,  0.8966,  0.2902, -0.6966,  0.4955
-0.0627, -0.1439,  0.1985,  0.6999,  0.5022,  0.3077
 0.1587,  0.8494, -0.8705,  0.9827, -0.8940,  0.4263
-0.7850,  0.2473, -0.9040, -0.4308, -0.8779,  0.7199
 0.4070,  0.3369, -0.2428, -0.6236,  0.4940,  0.2215
-0.0242,  0.0513, -0.9430,  0.2885, -0.2987,  0.3947
-0.5416, -0.1322, -0.2351, -0.0604,  0.9590,  0.3683
 0.1055,  0.7783, -0.2901, -0.5090,  0.8220,  0.2984
-0.9129,  0.9015,  0.1128, -0.2473,  0.9901,  0.4776
-0.9378,  0.1424, -0.6391,  0.2619,  0.9618,  0.5368
 0.7498, -0.0963,  0.4169,  0.5549, -0.0103,  0.1614
-0.2612, -0.7156,  0.4538, -0.0460, -0.1022,  0.3717
 0.7720,  0.0552, -0.1818, -0.4622, -0.8560,  0.1685
-0.4177,  0.0070,  0.9319, -0.7812,  0.3461,  0.3052
-0.0001,  0.5542, -0.7128, -0.8336, -0.2016,  0.3803
 0.5356, -0.4194, -0.5662, -0.9666, -0.2027,  0.1776
-0.2378,  0.3187, -0.8582, -0.6948, -0.9668,  0.5474
-0.1947, -0.3579,  0.1158,  0.9869,  0.6690,  0.2992
 0.3992,  0.8365, -0.9205, -0.8593, -0.0520,  0.3154
-0.0209,  0.0793,  0.7905, -0.1067,  0.7541,  0.1864
-0.4928, -0.4524, -0.3433,  0.0951, -0.5597,  0.6261
-0.8118,  0.7404, -0.5263, -0.2280,  0.1431,  0.6349
 0.0516, -0.8480,  0.7483,  0.9023,  0.6250,  0.1959
-0.3212,  0.1093,  0.9488, -0.3766,  0.3376,  0.2735
-0.3481,  0.5490, -0.3484,  0.7797,  0.5034,  0.4379
-0.5785, -0.9170, -0.3563, -0.9258,  0.3877,  0.4121
 0.3407, -0.1391,  0.5356,  0.0720, -0.9203,  0.3458
-0.3287, -0.8954,  0.2102,  0.0241,  0.2349,  0.3247
-0.1353,  0.6954, -0.0919, -0.9692,  0.7461,  0.3338
 0.9036, -0.8982, -0.5299, -0.8733, -0.1567,  0.1187
 0.7277, -0.8368, -0.0538, -0.7489,  0.5458,  0.0830
 0.9049,  0.8878,  0.2279,  0.9470, -0.3103,  0.2194
 0.7957, -0.1308, -0.5284,  0.8817,  0.3684,  0.2172
 0.4647, -0.4931,  0.2010,  0.6292, -0.8918,  0.3371
-0.7390,  0.6849,  0.2367,  0.0626, -0.5034,  0.7039
-0.1567, -0.8711,  0.7940, -0.5932,  0.6525,  0.1710
 0.7635, -0.0265,  0.1969,  0.0545,  0.2496,  0.1445
 0.7675,  0.1354, -0.7698, -0.5460,  0.1920,  0.1728
-0.5211, -0.7372, -0.6763,  0.6897,  0.2044,  0.5217
 0.1913,  0.1980,  0.2314, -0.8816,  0.5006,  0.1998
 0.8964,  0.0694, -0.6149,  0.5059, -0.9854,  0.1825
 0.1767,  0.7104,  0.2093,  0.6452,  0.7590,  0.2832
-0.3580, -0.7541,  0.4426, -0.1193, -0.7465,  0.5657
-0.5996,  0.5766, -0.9758, -0.3933, -0.9572,  0.6800
 0.9950,  0.1641, -0.4132,  0.8579,  0.0142,  0.2003
-0.4717, -0.3894, -0.2567, -0.5111,  0.1691,  0.4266
 0.3917, -0.8561,  0.9422,  0.5061,  0.6123,  0.1212
-0.0366, -0.1087,  0.3449, -0.1025,  0.4086,  0.2475
 0.3633,  0.3943,  0.2372, -0.6980,  0.5216,  0.1925
-0.5325, -0.6466, -0.2178, -0.3589,  0.6310,  0.3568
 0.2271,  0.5200, -0.1447, -0.8011, -0.7699,  0.3128
 0.6415,  0.1993,  0.3777, -0.0178, -0.8237,  0.2181
-0.5298, -0.0768, -0.6028, -0.9490,  0.4588,  0.4356
 0.6870, -0.1431,  0.7294,  0.3141,  0.1621,  0.1632
-0.5985,  0.0591,  0.7889, -0.3900,  0.7419,  0.2945
 0.3661,  0.7984, -0.8486,  0.7572, -0.6183,  0.3449
 0.6995,  0.3342, -0.3113, -0.6972,  0.2707,  0.1712
 0.2565,  0.9126,  0.1798, -0.6043, -0.1413,  0.2893
-0.3265,  0.9839, -0.2395,  0.9854,  0.0376,  0.4770
 0.2690, -0.1722,  0.9818,  0.8599, -0.7015,  0.3954
-0.2102, -0.0768,  0.1219,  0.5607, -0.0256,  0.3949
 0.8216, -0.9555,  0.6422, -0.6231,  0.3715,  0.0801
-0.2896,  0.9484, -0.7545, -0.6249,  0.7789,  0.4370
-0.9985, -0.5448, -0.7092, -0.5931,  0.7926,  0.5402

Test data:

# synthetic_test_40.txt
#
 0.7462,  0.4006, -0.0590,  0.6543, -0.0083,  0.1935
 0.8495, -0.2260, -0.0142, -0.4911,  0.7699,  0.1078
-0.2335, -0.4049,  0.4352, -0.6183, -0.7636,  0.5088
 0.1810, -0.5142,  0.2465,  0.2767, -0.3449,  0.3136
-0.8650,  0.7611, -0.0801,  0.5277, -0.4922,  0.7140
-0.2358, -0.7466, -0.5115, -0.8413, -0.3943,  0.4533
 0.4834,  0.2300,  0.3448, -0.9832,  0.3568,  0.1360
-0.6502, -0.6300,  0.6885,  0.9652,  0.8275,  0.3046
-0.3053,  0.5604,  0.0929,  0.6329, -0.0325,  0.4756
-0.7995,  0.0740, -0.2680,  0.2086,  0.9176,  0.4565
-0.2144, -0.2141,  0.5813,  0.2902, -0.2122,  0.4119
-0.7278, -0.0987, -0.3312, -0.5641,  0.8515,  0.4438
 0.3793,  0.1976,  0.4933,  0.0839,  0.4011,  0.1905
-0.8568,  0.9573, -0.5272,  0.3212, -0.8207,  0.7415
-0.5785,  0.0056, -0.7901, -0.2223,  0.0760,  0.5551
 0.0735, -0.2188,  0.3925,  0.3570,  0.3746,  0.2191
 0.1230, -0.2838,  0.2262,  0.8715,  0.1938,  0.2878
 0.4792, -0.9248,  0.5295,  0.0366, -0.9894,  0.3149
-0.4456,  0.0697,  0.5359, -0.8938,  0.0981,  0.3879
 0.8629, -0.8505, -0.4464,  0.8385,  0.5300,  0.1769
 0.1995,  0.6659,  0.7921,  0.9454,  0.9970,  0.2330
-0.0249, -0.3066, -0.2927, -0.4923,  0.8220,  0.2437
 0.4513, -0.9481, -0.0770, -0.4374, -0.9421,  0.2879
-0.3405,  0.5931, -0.3507, -0.3842,  0.8562,  0.3987
 0.9538,  0.0471,  0.9039,  0.7760,  0.0361,  0.1706
-0.0887,  0.2104,  0.9808,  0.5478, -0.3314,  0.4128
-0.8220, -0.6302,  0.0537, -0.1658,  0.6013,  0.4306
-0.4123, -0.2880,  0.9074, -0.0461, -0.4435,  0.5144
 0.0060,  0.2867, -0.7775,  0.5161,  0.7039,  0.3599
-0.7968, -0.5484,  0.9426, -0.4308,  0.8148,  0.2979
 0.7811,  0.8450, -0.6877,  0.7594,  0.2640,  0.2362
-0.6802, -0.1113, -0.8325, -0.6694, -0.6056,  0.6544
 0.3821,  0.1476,  0.7466, -0.5107,  0.2592,  0.1648
 0.7265,  0.9683, -0.9803, -0.4943, -0.5523,  0.2454
-0.9049, -0.9797, -0.0196, -0.9090, -0.4433,  0.6447
-0.4607,  0.1811, -0.2389,  0.4050, -0.0078,  0.5229
 0.2664, -0.2932, -0.4259, -0.7336,  0.8742,  0.1834
-0.4507,  0.1029, -0.6294, -0.1158, -0.6294,  0.6081
 0.8948, -0.0124,  0.9278,  0.2899, -0.0314,  0.1534
-0.1323, -0.8813, -0.0146, -0.0697,  0.6135,  0.2386
Posted in JavaScript, Machine Learning | Leave a comment

Yet Another Metric for Regression Model Evaluation: Mean Absolute Scaled Error (MASE) Implemented Using C#

Posted on August 5, 2026 by jamesdmccaffrey

The goal of a machine learning regression problem is to predict a single numeric value. For example, you might want to predict the price of a house in a particular area based on square footage, year built, number of bedrooms, and so on.

There are about a dozen metrics to evaluate a trained regression model, but most are rarely used. In my regression project scenarios, I use prediction mean squared error (MSE), prediction accuracy (to within a specified percentage of the true target value), and coefficient of determination (R2).

A model MSE value is a bit difficult to interpret because 1.) it heavily penalizes outlier predictions, and 2.) it depends on how the target y values are scaled. But MSE is useful because many regression techniques minimize MSE.

Accuracy is the most interpretable metric, but it requires an arbitrary closeness-to-actual-target parameter (like 5% or 10%). Plus accuracy is not very granular, and therefore can be misleading (such as when a model just barely predicts many y values within the specified percentage tolerance).

R2 is sort of a normalized accuracy (larger values are better). R2 doesn’t depend on how the target y values are scaled but R2 isn’t too easy to interpret. R2 is mostly useful to compare two entirely different regression models. R2 is the default “score” metric for scikit-learn regression models and classification models.

A metric called mean absolute scaled error (MASE) is common in the time series regression community, but MASE is rarely used in standard regression problem scenarios. MASE doesn’t heavily penalize outlier predictions like MSE does (because it uses absolute value of error instead of error-squared), and MASE doesn’t depend on how the target y data is scaled.

Note: The MASE implementation shown in this blog post works only for standard regression, not time series regression. The MASE version for TSR predicts a baseline value as the previous target y instead of the average of the target y values.

One Sunday morning, just to entertain myself, I decided to implement a MASE function using the C# language:

  public static double MASE(dynamic model,
    double[][] dataX, double[] dataY)
  {
    // Mean Absolute Scaled Error
    // error relative to baseline predict-mean target y
    // standard tabular data scenarios, not TSR scenarios

    // 1. compute model mean absolute error (MAE)
    int n = dataX.Length;
    double sum = 0.0;
    for (int i = 0; i "lt" n; ++i) // "lt" is less-than
    {
      double predY = model.Predict(dataX[i]);
      sum += Math.Abs(predY - dataY[i]);
    }
    double modelMAE = sum / n;

    // 2. compute mean of target y values
    double sumY = 0.0;
    for (int i = 0; i "lt" n; ++i)
      sumY += dataY[i];
    double meanY = sumY / n;

    // 3. compute baseline MAE (always predict mean y)
    double baseSum = 0.0;
    for (int i = 0; i "lt" n; ++i)
      baseSum += Math.Abs(meanY - dataY[i]);
    double baseMAE = baseSum / n;

    if (baseMAE "lt" 1.0e-12)
      return modelMAE;

    // 4. compute MASE as ratio of model to baseline
    double result = modelMAE / baseMAE;
    return result;
  }

The C# keyword “dynamic” allows the MASE() function to be used with any C# implementation of a regression model (LinearRegression, NearestNeighborsRegressor, QuadraticRegressor, etc.), as long as the model has a Predict() method. For example:

. . .
SomeRegressor model = new SomeRegressor(lrnRate, maxEpochs);
model.Train(trainX, trainY);
maseTrain = MASE(model, trainX, trainY);
Console.WriteLine("MASE for train data = " + maseTrain);
. . .

I put together a demo of MASE with one of my C# linear regression implementations. Output of a demo run:

Begin C# linear regression using MP pinv (QR-Householder)
 training with L2 regularization

Loading synthetic train (200) and test (40) data
Done

First three train X:
 -0.1660  0.4406 -0.9998 -0.3953 -0.7065
  0.0776 -0.1616  0.3704 -0.5911  0.7562
 -0.9452  0.3409 -0.1654  0.1174 -0.7192

First three train y:
  0.4840
  0.1568
  0.8054

Creating and training Linear Regression model
 using QR p-inverse with regularization
Setting L2 lamda = 1.0000
Done

Coefficients/weights:
-0.2618  0.0331  -0.0453  0.0353  -0.1132
Bias/constant: 0.3618

Evaluating model

Accuracy train (within 0.10) = 0.4700
Accuracy test (within 0.10) = 0.6500

MSE train = 0.0026
MSE test = 0.0019

MASE train = 0.2619
MASE test = 0.2402

End demo

Because MASE is an error metric, for MASE, lower values are better. The MASE metric can be as low as 0.0 (perfect predictions, no error — impossible in practice) up to 1.0 (the model predicts exactly like just predicting the average of the target y values for any input), or MASE can be greater than 1.0 which means the model predicts even worse than just predicting the average y for any input.

MASE has some nice properties. And MASE is very common in the time series regression community, but MASE is almost never used in the tabular data regression community. The main reason why R2 is nearly universal for tabular data regression is mostly because R2 is used by scikit-learn, and scikit-learn has a virtual monopoly on machine learning library code and usage patterns.

See https://jamesmccaffreyblog.com/2026/04/20/refactoring-my-demo-of-linear-regression-with-closed-form-training-using-csharp/ for one version of linear regression code, and the data.


I am a big fan of old science fiction movies from the 1950s and 1960s. The limited non-digital technology of the time meant that all movies were relatively expensive and difficult to create, and so there aren’t many sci-fi movies from those two decades. But starting in the 1990s, digital technology advanced to the point where a reasonably professional movie could be made for roughly $2.0 million dollars. This led to an explosion of sci-fi movies. Most of these movies are bad, but every now and then a low-budget sci-fi movie surprises me in a good way. Unlike a regression model where evaluation is objective, it’s difficult to compute a subjective evaluation metric for movies.

In “Terror Birds” (2016), a group of five young people go into an isolated forest-ranch to search for one of the group’s missing father. Unfortunately, a scientist is raising two prehistoric, very mean, very large, and very hungry birds. Surprisingly clever plot, excellent dialog, decent special effects, excellent acting, and a nice combination of humor and suspense. My grade = A- (but my personal quality bar is low).

In “Ice Spiders” (2007), a group of young skiers on an isolated Utah mountain run into giant genetically altered spiders that are the product of a secret government lab. This movie has a nice combination of interesting plot, pretty good acting, humor-vs-scary, and excellent special effects. The movie doesn’t try to be anything more than it is. My grade = solid B.

Posted in Machine Learning | Leave a comment

New Version of Linear Regression Trained Using MP Pseudo-Inverse via QR-Householder Using C#

Posted on August 4, 2026 by jamesdmccaffrey

There are three ways to train a basic linear regression model: 1.) using stochastic gradient descent, 2.) using left pseudo-inverse (normal equations) via Cholesky inverse, 3.) using relaxed Moore-Penrose pseudo-inverse via one of many possible inverses.

Each of the three techniques has several variations, and each variation has several implementation approaches. Briefly, 1.) SGD works for any size of test data but requires you to specify a learning rate and max iterations, which must be determined by trial and error. 2.) left pseudo-inverse works well for small-to-medium size training data but will fail for datasets with unlucky combinations of values (“ill-conditioned”), due to arithmetic overflow or underflow. 3.) MP pseudo-inverse works well for medium-to-large training data, but is more complicated to implement than SGD or left pseudo-inverse.

For MP pseudo-inverse there are six main algorithms (and many secondary algorithms): 1.) SVD (Jacobi algorithm), 2.) SVD (Golub-Kahan algorithm), 3.) SVD (bidiagonalization Householder algorithm), 4.) QR (Givens algorithm), 5.) QR (modified Gram-Schmidt algorithm), 6.) QR (Householder algorithm). If you find this moderately confusing, you are not alone.

I have implemented all of the six MP pseudo-inverse algorithms from scratch. My approach-of-choice is QR-Householder. I recently refactored my QR-Householder implementation to fix a couple of edge-case bugs and remove several inefficiencies. So, I put together a demo of linear regression, trained using my new QR-Householder implementation. Output of the demo:

Begin C# linear regression using MP pinv
 (QR-Householder) training

Loading synthetic train (200) and test (40) data
Done

First three train X:
 -0.1660  0.4406 -0.9998 -0.3953 -0.7065
  0.0776 -0.1616  0.3704 -0.5911  0.7562
 -0.9452  0.3409 -0.1654  0.1174 -0.7192

First three train y:
  0.4840
  0.1568
  0.8054

Creating and training Linear Regression 
 model using QR p-inverse
Done

Coefficients/weights:
-0.2656  0.0333  -0.0454  0.0358  -0.1146
Bias/constant: 0.3619

Evaluating model

Accuracy train (within 0.10) = 0.4600
Accuracy test (within 0.10) = 0.6500

MSE train = 0.0026
MSE test = 0.0020

Predicting for x =
  -0.1660   0.4406  -0.9998  -0.3953  -0.7065

Predicted y = 0.5329

End demo

The demo data is synthetic. It was generated by a neural network, and so linear regression cannot predict it very well. There are 200 training items and 40 test items so it’s a small dataset.

One minor detail. When using MP pseudo-inverse for training, there’s no easy way to introduce regularization. This isn’t a major issue, but if for some reason regularization is absolutely required, you must add a dim-by-dim matrix of zeros, with regularization constant on the diagonal, to the bottom of the training X matrix, and add dim 0.0 values to the end of training y vector. Not worth the trouble in any practical scenario.


I am fascinated to the point of obsession by mathematics, computer science, and machine learning. And for some reason, I love to watch for movie scenes where the closed captioning reads “chittering”. Whenever I find a chittering, I’m thrilled. I’m not joking.

Left: “Harry Potter and the Sorcerer’s Stone” (2001). The first movie in the series is my favorite. Here a large group of owls gather outside the Dursley’s house, attempting to deliver Harry’s invitation to Hogwarts school.

Right: “Harry Potter and the Prisoner of Azkaban” (2004). The third movie in the series is my second favorite. Here Ron’s rat “Scabbers” runs away from Prof. Lupin and Sirius Black. He is revealed to be the evil Peter Pettigrew in disguise.

Demo program. Replace “lt” (less than), “gt”, “lte”, “gte” with Boolean operator symbols (my blog editor chokes on them).

using System;
using System.IO;
using System.Collections.Generic;

namespace LinearRegressionPinvQRHouseholder
{
  internal class LinearRegressionPinvProgram
  {
    static void Main(string[] args)
    {
      Console.WriteLine("\nBegin C# linear regression" +
        " using MP pinv (QR-Householder) training ");

      // 1. load data
      Console.WriteLine("\nLoading synthetic train" +
        " (200) and test (40) data");
      string trainFile =
        "..\\..\\..\\Data\\synthetic_train_200.txt";
      int[] colsX = new int[] { 0, 1, 2, 3, 4 };
      double[][] trainX =
        MatLoad(trainFile, colsX, ',', "#");
      double[] trainY =
        MatToVec(MatLoad(trainFile,
        new int[] { 5 }, ',', "#"));

      string testFile =
        "..\\..\\..\\Data\\synthetic_test_40.txt";
      double[][] testX =
         MatLoad(testFile, colsX, ',', "#");
      double[] testY =
        MatToVec(MatLoad(testFile,
        new int[] { 5 }, ',', "#"));
      Console.WriteLine("Done ");

      Console.WriteLine("\nFirst three train X: ");
      for (int i = 0; i "lt" 3; ++i)
        VecShow(trainX[i], 4, 8);

      Console.WriteLine("\nFirst three train y: ");
      for (int i = 0; i "lt" 3; ++i)
        Console.WriteLine(trainY[i].ToString("F4").
          PadLeft(8));

      // 2. create and train model using pseudo-inverse
      Console.WriteLine("\nCreating and training" +
        " Linear Regression model using QR p-inverse ");
      LinearRegressor model = new LinearRegressor();
      model.Train(trainX, trainY);
      Console.WriteLine("Done ");

      // 2b. show model parameters
      Console.WriteLine("\nCoefficients/weights: ");
      for (int i = 0; i "lt" model.weights.Length; ++i)
        Console.Write(model.weights[i].
          ToString("F4") + "  ");
      Console.WriteLine("\nBias/constant: " +
        model.bias.ToString("F4"));

      // 3. evaluate model
      Console.WriteLine("\nEvaluating model ");
      double accTrain = model.Accuracy(trainX, trainY, 0.10);
      Console.WriteLine("\nAccuracy train (within 0.10) = " +
        accTrain.ToString("F4"));
      double accTest = model.Accuracy(testX, testY, 0.10);
      Console.WriteLine("Accuracy test (within 0.10) = " +
        accTest.ToString("F4"));

      double mseTrain = model.MSE(trainX, trainY);
      Console.WriteLine("\nMSE train = " +
        mseTrain.ToString("F4"));
      double mseTest = model.MSE(testX, testY);
      Console.WriteLine("MSE test = " +
        mseTest.ToString("F4"));

      // 4. use model to predict first training item
      double[] x = trainX[0];
      Console.WriteLine("\nPredicting for x = ");
      VecShow(x, 4, 9);
      double predY = model.Predict(x);
      Console.WriteLine("\nPredicted y = " +
        predY.ToString("F4"));

      Console.WriteLine("\nEnd demo ");
      Console.ReadLine();
    } // Main()

    // ------------------------------------------------------
    // helpers for Main(): MatLoad(), MatToVec(), VecShow()
    // ------------------------------------------------------

    static double[][] MatLoad(string fn, int[] usecols,
      char sep, string comment)
    {
      List"lt"double[]"gt" result = new List"lt"double[]"gt"();
      string line = "";
      FileStream ifs = new FileStream(fn, FileMode.Open);
      StreamReader sr = new StreamReader(ifs);
      while ((line = sr.ReadLine()) != null)
      {
        if (line.StartsWith(comment) == true)
          continue;
        string[] tokens = line.Split(sep);
        List"lt"double"gt" lst = new List"lt"double"gt"();
        for (int j = 0; j "lt" usecols.Length; ++j)
          lst.Add(double.Parse(tokens[usecols[j]]));
        double[] row = lst.ToArray();
        result.Add(row);
      }
      sr.Close(); ifs.Close();
      return result.ToArray();
    }

    static double[] MatToVec(double[][] mat)
    {
      int nRows = mat.Length;
      int nCols = mat[0].Length;
      double[] result = new double[nRows * nCols];
      int k = 0;
      for (int i = 0; i "lt" nRows; ++i)
        for (int j = 0; j "lt" nCols; ++j)
          result[k++] = mat[i][j];
      return result;
    }

    static void VecShow(double[] vec, int dec, int wid)
    {
      for (int i = 0; i "lt" vec.Length; ++i)
        Console.Write(vec[i].ToString("F" + dec).
          PadLeft(wid));
      Console.WriteLine("");
    }

  } // class Program

  // ========================================================

  public class LinearRegressor
  {
    public double[] weights;
    public double bias;
    private Random rnd;

    // ------------------------------------------------------

    public LinearRegressor(int seed = 0)  // ctor
    {
      this.weights = new double[0];
      this.bias = 0;
      this.rnd = new Random(seed); // not used this version
    }

    // ------------------------------------------------------
    // primary: Train(), Predict(), Accuracy(), MSE()
    // helpers: MatToDesign(), MatVecProd()
    // ------------------------------------------------------

    public double Predict(double[] x)
    {
      double result = 0.0;
      for (int j = 0; j "lt" x.Length; ++j)
        result += x[j] * this.weights[j];
      result += this.bias;
      return result;
    }

    // ------------------------------------------------------

    public void Train(double[][] trainX, double[] trainY)
    {
      // wts = pinv(designX) * y
      int dim = trainX[0].Length;
      this.weights = new double[dim];

      double[][] X = MatToDesign(trainX);  // design X
      // for regularization, add dim-by-dim with tiny alpha
      // on diagonal to bottom of trainX, and add
      // dim 0.0s to end of trainY
      double[][] Xpinv = QRHouseholder.MatPinv(X);
      double[] biasAndWts = MatVecProd(Xpinv, trainY);
      this.bias = biasAndWts[0];
      for (int i = 1; i "lt" biasAndWts.Length; ++i)
        this.weights[i - 1] = biasAndWts[i];
      return;
    }

    // ------------------------------------------------------

    public double Accuracy(double[][] dataX, double[] dataY,
      double pctClose)
    {
      int numCorrect = 0; int numWrong = 0;
      for (int i = 0; i "lt" dataX.Length; ++i)
      {
        double actualY = dataY[i];
        double predY = this.Predict(dataX[i]);
        if (Math.Abs(predY - actualY) "lt"
          Math.Abs(pctClose * actualY))
          ++numCorrect;
        else
          ++numWrong;
      }
      return (numCorrect * 1.0) / (numWrong + numCorrect);
    }

    // ------------------------------------------------------

    public double MSE(double[][] dataX, double[] dataY)
    {
      int n = dataX.Length;
      double sum = 0.0;
      for (int i = 0; i "lt" n; ++i)
      {
        double actualY = dataY[i];
        double predY = this.Predict(dataX[i]);
        sum += (actualY - predY) * (actualY - predY);
      }
      return sum / n;
    }

    // ------------------------------------------------------
    
    private static double[] MatVecProd(double[][] M,
      double[] v)
    {
      // return a regular vector
      int nRows = M.Length;
      int nCols = M[0].Length;
      int n = v.Length;
      if (nCols != n)
        throw new Exception("non-conform in MatVecProd");

      double[] result = new double[nRows];
      for (int i = 0; i "lt" nRows; ++i)
        for (int k = 0; k "lt" nCols; ++k)
          result[i] += M[i][k] * v[k];

      return result;
    }

    // ------------------------------------------------------

    private static double[][] MatToDesign(double[][] M)
    {
      // add a column of 1s
      int nRows = M.Length;
      int nCols = M[0].Length;
      double[][] result = new double[M.Length][];
      for (int i = 0; i "lt" nRows; ++i)
        result[i] = new double[nCols + 1];

      for (int i = 0; i "lt" nRows; ++i)
      {
        result[i][0] = 1.0;
        for (int j = 1; j "lt" nCols + 1; ++j)
          result[i][j] = M[i][j - 1];
      }
      return result;
    }

  } // class LinearRegressor

  // ========================================================

  public class QRHouseholder
  {
    // container for MP pseudo-inverse via QR-Householder
    // A = Q * R
    // pinv(A) = inv(R) * inv(Q)  note order matters
    //         = inv upper tri (easy) * transpose (easy)

    public static double[][] MatPinv(double[][] M)
    {
      double[][] Q; double[][] R;
      MatDecompQR(M, out Q, out R);  // Householder
      double[][] Ri = MatInvUpperTri(R);
      double[][] Qi = MatTranspose(Q);
      double[][] result = MatProduct(Ri, Qi);
      return result;
    }

    // ------------------------------------------------------

    public static double[][] MatInvUpperTri(double[][] U)
    {
      int n = U.Length;  // must be square matrix

      double[][] result = MatMake(n, n);
      for (int i = 0; i "lt" n; ++i)
        result[i][i] = 1.0;
      for (int k = 0; k "lt" n; ++k)
      {
        for (int j = 0; j "lt" n; ++j)
        {
          for (int i = 0; i "lt" k; ++i)
          {
            result[j][k] -= result[j][i] * U[i][k];
          }
          result[j][k] /= U[k][k];
        }
      }
      return result;
    }

    // ------------------------------------------------------

    public static double[][] MatMake(int nRows, int nCols)
    {
      double[][] result = new double[nRows][];
      for (int i = 0; i "lt" nRows; ++i)
        result[i] = new double[nCols];
      return result;
    }

    // ------------------------------------------------------

    public static double[][] MatTranspose(double[][] M)
    {
      int nRows = M.Length;
      int nCols = M[0].Length;
      double[][] result = MatMake(nCols, nRows);
      for (int i = 0; i "lt" nRows; ++i)
        for (int j = 0; j "lt" nCols; ++j)
          result[j][i] = M[i][j];
      return result;
    }

    // ------------------------------------------------------

    public static double[][] MatProduct(double[][] A,
      double[][] B)
    {
      int aRows = A.Length; int aCols = A[0].Length;
      int bRows = B.Length; int bCols = B[0].Length;
      if (aCols != bRows)
        throw new Exception("Non-conformable matrices");

      double[][] result = new double[aRows][];
      for (int i = 0; i "lt" aRows; ++i)
        result[i] = new double[bCols];

      for (int i = 0; i "lt" aRows; ++i) // each row of A
        for (int j = 0; j "lt" bCols; ++j) // each col of B
          for (int k = 0; k "lt" aCols; ++k)
            result[i][j] += A[i][k] * B[k][j];

      return result;
    }

    // ------------------------------------------------------

    public static void MatDecompQR(double[][] A, 
      out double[][] Q,  out double[][] R)
    {
      int m = A.Length; int n = A[0].Length;
      if (m "lt" n)
        Console.WriteLine("FATAL: nRows must be gte nCols");

      double[][] QQ = MatMake(m, m); // working full Q
      for (int i = 0; i "lt" m; ++i)
        QQ[i][i] = 1.0;  // identity matrix

      double[][] RR = MatMake(m, n);
      for (int i = 0; i "lt" m; ++i)
        for (int j = 0; j "lt" n; ++j)
          RR[i][j] = A[i][j]; // copy of A is working R

      int k = Math.Min(m, n);  // or just use n
      for (int j = 0; j "lt" k; ++j) // main processing loop
      {
        int xn = m - j;
        double[] x = new double[xn];
        for (int i = 0; i "lt" xn; ++i)
          x[i] = RR[j + i][j];

        double ss = 0.0;
        for (int i = 0; i "lt" xn; ++i)
          ss += x[i] * x[i];
        double normX = Math.Sqrt(ss);

        // if (normX == 0.0) continue;
        if (Math.Abs(normX) "lt" 1.0e-12) continue;

        double sign;
        if (x[0] "gte" 0.0) sign = -1.0;
        else sign = 1.0; // counter-intuitive
      
        double[] u = new double[xn];
        for (int i = 0; i "lt" xn; ++i)
          u[i] = x[i] / (x[0] - sign * normX); // check div 0
        u[0] = 1.0;

        // compute scaling factor tau = 2 / (u^T * u)
        double tau = -sign * (x[0] - sign * normX) / normX;

        // dimensions for sub-matrices
        int nRowsSubR = m - j;   int nColsSubR = n - j;
        int nRowsSubQ = m;       int nColsSubQ = m - j;

        double[] vr = new double[nColsSubR];
        for (int c = 0; c "lt" nColsSubR; ++c)
        {
          double acc = 0.0;
          for (int r = 0; r "lt" nRowsSubR; ++r)
            acc += u[r] * RR[j + r][j + c];
          vr[c] = acc;
        }

        double[] vq = new double[nRowsSubQ];
        for (int r = 0; r "lt" nRowsSubQ; ++r)
        {
          double acc = 0.0;
          for (int c = 0; c "lt" nColsSubQ; ++c)
            acc += u[c] * QQ[r][j + c];
          vq[r] = acc;
        }

        // update sub-R
        for (int r = 0; r "lt" nRowsSubR; ++r)
          for (int c = 0; c "lt" nColsSubR; ++c)
            RR[j + r][j + c] -= tau * u[r] * vr[c];

        // update sub-Q
        for (int r = 0; r "lt" nRowsSubQ; ++r)
          for (int c = 0; c "lt" nColsSubQ; ++c)
            QQ[r][j + c] -= tau * vq[r] * u[c];
       
      } // j

      // extract QQ RR into out params
      Q = MatMake(m, n);
      for (int i = 0; i "lt" m; ++i)
        for (int j = 0; j "lt" n; ++j)
          Q[i][j] = QQ[i][j];

      R = MatMake(n, n);
      for (int i = 0; i "lt" n; ++i)
        for (int j = 0; j "lt" n; ++j)
          R[i][j] = RR[i][j];

      return;
    } // MatDecompQR

  } // class QRHouseholder


  // ========================================================

} // ns

Training data:

# synthetic_train_200.txt
#
-0.1660,  0.4406, -0.9998, -0.3953, -0.7065,  0.4840
 0.0776, -0.1616,  0.3704, -0.5911,  0.7562,  0.1568
-0.9452,  0.3409, -0.1654,  0.1174, -0.7192,  0.8054
 0.9365, -0.3732,  0.3846,  0.7528,  0.7892,  0.1345
-0.8299, -0.9219, -0.6603,  0.7563, -0.8033,  0.7955
 0.0663,  0.3838, -0.3690,  0.3730,  0.6693,  0.3206
-0.9634,  0.5003,  0.9777,  0.4963, -0.4391,  0.7377
-0.1042,  0.8172, -0.4128, -0.4244, -0.7399,  0.4801
-0.9613,  0.3577, -0.5767, -0.4689, -0.0169,  0.6861
-0.7065,  0.1786,  0.3995, -0.7953, -0.1719,  0.5569
 0.3888, -0.1716, -0.9001,  0.0718,  0.3276,  0.2500
 0.1731,  0.8068, -0.7251, -0.7214,  0.6148,  0.3297
-0.2046, -0.6693,  0.8550, -0.3045,  0.5016,  0.2129
 0.2473,  0.5019, -0.3022, -0.4601,  0.7918,  0.2613
-0.1438,  0.9297,  0.3269,  0.2434, -0.7705,  0.5171
 0.1568, -0.1837, -0.5259,  0.8068,  0.1474,  0.3307
-0.9943,  0.2343, -0.3467,  0.0541,  0.7719,  0.5581
 0.2467, -0.9684,  0.8589,  0.3818,  0.9946,  0.1092
-0.6553, -0.7257,  0.8652,  0.3936, -0.8680,  0.7018
 0.8460,  0.4230, -0.7515, -0.9602, -0.9476,  0.1996
-0.9434, -0.5076,  0.7201,  0.0777,  0.1056,  0.5664
 0.9392,  0.1221, -0.9627,  0.6013, -0.5341,  0.1533
 0.6142, -0.2243,  0.7271,  0.4942,  0.1125,  0.1661
 0.4260,  0.1194, -0.9749, -0.8561,  0.9346,  0.2230
 0.1362, -0.5934, -0.4953,  0.4877, -0.6091,  0.3810
 0.6937, -0.5203, -0.0125,  0.2399,  0.6580,  0.1460
-0.6864, -0.9628, -0.8600, -0.0273,  0.2127,  0.5387
 0.9772,  0.1595, -0.2397,  0.1019,  0.4907,  0.1611
 0.3385, -0.4702, -0.8673, -0.2598,  0.2594,  0.2270
-0.8669, -0.4794,  0.6095, -0.6131,  0.2789,  0.4700
 0.0493,  0.8496, -0.4734, -0.8681,  0.4701,  0.3516
 0.8639, -0.9721, -0.5313,  0.2336,  0.8980,  0.1412
 0.9004,  0.1133,  0.8312,  0.2831, -0.2200,  0.1782
 0.0991,  0.8524,  0.8375, -0.2102,  0.9265,  0.2150
-0.6521, -0.7473, -0.7298,  0.0113, -0.9570,  0.7422
 0.6190, -0.3105,  0.8802,  0.1640,  0.7577,  0.1056
 0.6895,  0.8108, -0.0802,  0.0927,  0.5972,  0.2214
 0.1982, -0.9689,  0.1870, -0.1326,  0.6147,  0.1310
-0.3695,  0.7858,  0.1557, -0.6320,  0.5759,  0.3773
-0.1596,  0.3581,  0.8372, -0.9992,  0.9535,  0.2071
-0.2468,  0.9476,  0.2094,  0.6577,  0.1494,  0.4132
 0.1737,  0.5000,  0.7166,  0.5102,  0.3961,  0.2611
 0.7290, -0.3546,  0.3416, -0.0983, -0.2358,  0.1332
-0.3652,  0.2438, -0.1395,  0.9476,  0.3556,  0.4170
-0.6029, -0.1466, -0.3133,  0.5953,  0.7600,  0.4334
-0.4596, -0.4953,  0.7098,  0.0554,  0.6043,  0.2775
 0.1450,  0.4663,  0.0380,  0.5418,  0.1377,  0.2931
-0.8636, -0.2442, -0.8407,  0.9656, -0.6368,  0.7429
 0.6237,  0.7499,  0.3768,  0.1390, -0.6781,  0.2185
-0.5499,  0.1850, -0.3755,  0.8326,  0.8193,  0.4399
-0.4858, -0.7782, -0.6141, -0.0008,  0.4572,  0.4197
 0.7033, -0.1683,  0.2334, -0.5327, -0.7961,  0.1776
 0.0317, -0.0457, -0.6947,  0.2436,  0.0880,  0.3345
 0.5031, -0.5559,  0.0387,  0.5706, -0.9553,  0.3107
-0.3513,  0.7458,  0.6894,  0.0769,  0.7332,  0.3170
 0.2205,  0.5992, -0.9309,  0.5405,  0.4635,  0.3532
-0.4806, -0.4859,  0.2646, -0.3094,  0.5932,  0.3202
 0.9809, -0.3995, -0.7140,  0.8026,  0.0831,  0.1600
 0.9495,  0.2732,  0.9878,  0.0921,  0.0529,  0.1289
-0.9476, -0.6792,  0.4913, -0.9392, -0.2669,  0.5966
 0.7247,  0.3854,  0.3819, -0.6227, -0.1162,  0.1550
-0.5922, -0.5045, -0.4757,  0.5003, -0.0860,  0.5863
-0.8861,  0.0170, -0.5761,  0.5972, -0.4053,  0.7301
 0.6877, -0.2380,  0.4997,  0.0223,  0.0819,  0.1404
 0.9189,  0.6079, -0.9354,  0.4188, -0.0700,  0.1907
-0.1428, -0.7820,  0.2676,  0.6059,  0.3936,  0.2790
 0.5324, -0.3151,  0.6917, -0.1425,  0.6480,  0.1071
-0.8432, -0.9633, -0.8666, -0.0828, -0.7733,  0.7784
-0.9444,  0.5097, -0.2103,  0.4939, -0.0952,  0.6787
-0.0520,  0.6063, -0.1952,  0.8094, -0.9259,  0.4836
 0.5477, -0.7487,  0.2370, -0.9793,  0.0773,  0.1241
 0.2450,  0.8116,  0.9799,  0.4222,  0.4636,  0.2355
 0.8186, -0.1983, -0.5003, -0.6531, -0.7611,  0.1511
-0.4714,  0.6382, -0.3788,  0.9648, -0.4667,  0.5950
 0.0673, -0.3711,  0.8215, -0.2669, -0.1328,  0.2677
-0.9381,  0.4338,  0.7820, -0.9454,  0.0441,  0.5518
-0.3480,  0.7190,  0.1170,  0.3805, -0.0943,  0.4724
-0.9813,  0.1535, -0.3771,  0.0345,  0.8328,  0.5438
-0.1471, -0.5052, -0.2574,  0.8637,  0.8737,  0.3042
-0.5454, -0.3712, -0.6505,  0.2142, -0.1728,  0.5783
 0.6327, -0.6297,  0.4038, -0.5193,  0.1484,  0.1153
-0.5424,  0.3282, -0.0055,  0.0380, -0.6506,  0.6613
 0.1414,  0.9935,  0.6337,  0.1887,  0.9520,  0.2540
-0.9351, -0.8128, -0.8693, -0.0965, -0.2491,  0.7353
 0.9507, -0.6640,  0.9456,  0.5349,  0.6485,  0.1059
-0.0462, -0.9737, -0.2940, -0.0159,  0.4602,  0.2606
-0.0627, -0.0852, -0.7247, -0.9782,  0.5166,  0.2977
 0.0478,  0.5098, -0.0723, -0.7504, -0.3750,  0.3335
 0.0090,  0.3477,  0.5403, -0.7393, -0.9542,  0.4415
-0.9748,  0.3449,  0.3736, -0.1015,  0.8296,  0.4358
 0.2887, -0.9895, -0.0311,  0.7186,  0.6608,  0.2057
 0.1570, -0.4518,  0.1211,  0.3435, -0.2951,  0.3244
 0.7117, -0.6099,  0.4946, -0.4208,  0.5476,  0.1096
-0.2929, -0.5726,  0.5346, -0.3827,  0.4665,  0.2465
 0.4889, -0.5572, -0.5718, -0.6021, -0.7150,  0.2163
-0.7782,  0.3491,  0.5996, -0.8389, -0.5366,  0.6516
-0.5847,  0.8347,  0.4226,  0.1078, -0.3910,  0.6134
 0.8469,  0.4121, -0.0439, -0.7476,  0.9521,  0.1571
-0.6803, -0.5948, -0.1376, -0.1916, -0.7065,  0.7156
 0.2878,  0.5086, -0.5785,  0.2019,  0.4979,  0.2980
 0.2764,  0.1943, -0.4090,  0.4632,  0.8906,  0.2960
-0.8877,  0.6705, -0.6155, -0.2098, -0.3998,  0.7107
-0.8398,  0.8093, -0.2597,  0.0614, -0.0118,  0.6502
-0.8476,  0.0158, -0.4769, -0.2859, -0.7839,  0.7715
 0.5751, -0.7868,  0.9714, -0.6457,  0.1448,  0.1175
 0.4802, -0.7001,  0.1022, -0.5668,  0.5184,  0.1090
 0.4458, -0.6469,  0.7239, -0.9604,  0.7205,  0.0779
 0.5175,  0.4339,  0.9747, -0.4438, -0.9924,  0.2879
 0.8678,  0.7158,  0.4577,  0.0334,  0.4139,  0.1678
 0.5406,  0.5012,  0.2264, -0.1963,  0.3946,  0.2088
-0.9938,  0.5498,  0.7928, -0.5214, -0.7585,  0.7687
 0.7661,  0.0863, -0.4266, -0.7233, -0.4197,  0.1466
 0.2277, -0.3517, -0.0853, -0.1118,  0.6563,  0.1767
 0.3499, -0.5570, -0.0655, -0.3705,  0.2537,  0.1632
 0.7547, -0.1046,  0.5689, -0.0861,  0.3125,  0.1257
 0.8186,  0.2110,  0.5335,  0.0094, -0.0039,  0.1391
 0.6858, -0.8644,  0.1465,  0.8855,  0.0357,  0.1845
-0.4967,  0.4015,  0.0805,  0.8977,  0.2487,  0.4663
 0.6760, -0.9841,  0.9787, -0.8446, -0.3557,  0.1509
-0.1203, -0.4885,  0.6054, -0.0443, -0.7313,  0.4854
 0.8557,  0.7919, -0.0169,  0.7134, -0.1628,  0.2002
 0.0115, -0.6209,  0.9300, -0.4116, -0.7931,  0.4052
-0.7114, -0.9718,  0.4319,  0.1290,  0.5892,  0.3661
 0.3915,  0.5557, -0.1870,  0.2955, -0.6404,  0.2954
-0.3564, -0.6548, -0.1827, -0.5172, -0.1862,  0.4622
 0.2392, -0.4959,  0.5857, -0.1341, -0.2850,  0.2470
-0.3394,  0.3947, -0.4627,  0.6166, -0.4094,  0.5325
 0.7107,  0.7768, -0.6312,  0.1707,  0.7964,  0.2757
-0.1078,  0.8437, -0.4420,  0.2177,  0.3649,  0.4028
-0.3139,  0.5595, -0.6505, -0.3161, -0.7108,  0.5546
 0.4335,  0.3986,  0.3770, -0.4932,  0.3847,  0.1810
-0.2562, -0.2894, -0.8847,  0.2633,  0.4146,  0.4036
 0.2272,  0.2966, -0.6601, -0.7011,  0.0284,  0.2778
-0.0743, -0.1421, -0.0054, -0.6770, -0.3151,  0.3597
-0.4762,  0.6891,  0.6007, -0.1467,  0.2140,  0.4266
-0.4061,  0.7193,  0.3432,  0.2669, -0.7505,  0.6147
-0.0588,  0.9731,  0.8966,  0.2902, -0.6966,  0.4955
-0.0627, -0.1439,  0.1985,  0.6999,  0.5022,  0.3077
 0.1587,  0.8494, -0.8705,  0.9827, -0.8940,  0.4263
-0.7850,  0.2473, -0.9040, -0.4308, -0.8779,  0.7199
 0.4070,  0.3369, -0.2428, -0.6236,  0.4940,  0.2215
-0.0242,  0.0513, -0.9430,  0.2885, -0.2987,  0.3947
-0.5416, -0.1322, -0.2351, -0.0604,  0.9590,  0.3683
 0.1055,  0.7783, -0.2901, -0.5090,  0.8220,  0.2984
-0.9129,  0.9015,  0.1128, -0.2473,  0.9901,  0.4776
-0.9378,  0.1424, -0.6391,  0.2619,  0.9618,  0.5368
 0.7498, -0.0963,  0.4169,  0.5549, -0.0103,  0.1614
-0.2612, -0.7156,  0.4538, -0.0460, -0.1022,  0.3717
 0.7720,  0.0552, -0.1818, -0.4622, -0.8560,  0.1685
-0.4177,  0.0070,  0.9319, -0.7812,  0.3461,  0.3052
-0.0001,  0.5542, -0.7128, -0.8336, -0.2016,  0.3803
 0.5356, -0.4194, -0.5662, -0.9666, -0.2027,  0.1776
-0.2378,  0.3187, -0.8582, -0.6948, -0.9668,  0.5474
-0.1947, -0.3579,  0.1158,  0.9869,  0.6690,  0.2992
 0.3992,  0.8365, -0.9205, -0.8593, -0.0520,  0.3154
-0.0209,  0.0793,  0.7905, -0.1067,  0.7541,  0.1864
-0.4928, -0.4524, -0.3433,  0.0951, -0.5597,  0.6261
-0.8118,  0.7404, -0.5263, -0.2280,  0.1431,  0.6349
 0.0516, -0.8480,  0.7483,  0.9023,  0.6250,  0.1959
-0.3212,  0.1093,  0.9488, -0.3766,  0.3376,  0.2735
-0.3481,  0.5490, -0.3484,  0.7797,  0.5034,  0.4379
-0.5785, -0.9170, -0.3563, -0.9258,  0.3877,  0.4121
 0.3407, -0.1391,  0.5356,  0.0720, -0.9203,  0.3458
-0.3287, -0.8954,  0.2102,  0.0241,  0.2349,  0.3247
-0.1353,  0.6954, -0.0919, -0.9692,  0.7461,  0.3338
 0.9036, -0.8982, -0.5299, -0.8733, -0.1567,  0.1187
 0.7277, -0.8368, -0.0538, -0.7489,  0.5458,  0.0830
 0.9049,  0.8878,  0.2279,  0.9470, -0.3103,  0.2194
 0.7957, -0.1308, -0.5284,  0.8817,  0.3684,  0.2172
 0.4647, -0.4931,  0.2010,  0.6292, -0.8918,  0.3371
-0.7390,  0.6849,  0.2367,  0.0626, -0.5034,  0.7039
-0.1567, -0.8711,  0.7940, -0.5932,  0.6525,  0.1710
 0.7635, -0.0265,  0.1969,  0.0545,  0.2496,  0.1445
 0.7675,  0.1354, -0.7698, -0.5460,  0.1920,  0.1728
-0.5211, -0.7372, -0.6763,  0.6897,  0.2044,  0.5217
 0.1913,  0.1980,  0.2314, -0.8816,  0.5006,  0.1998
 0.8964,  0.0694, -0.6149,  0.5059, -0.9854,  0.1825
 0.1767,  0.7104,  0.2093,  0.6452,  0.7590,  0.2832
-0.3580, -0.7541,  0.4426, -0.1193, -0.7465,  0.5657
-0.5996,  0.5766, -0.9758, -0.3933, -0.9572,  0.6800
 0.9950,  0.1641, -0.4132,  0.8579,  0.0142,  0.2003
-0.4717, -0.3894, -0.2567, -0.5111,  0.1691,  0.4266
 0.3917, -0.8561,  0.9422,  0.5061,  0.6123,  0.1212
-0.0366, -0.1087,  0.3449, -0.1025,  0.4086,  0.2475
 0.3633,  0.3943,  0.2372, -0.6980,  0.5216,  0.1925
-0.5325, -0.6466, -0.2178, -0.3589,  0.6310,  0.3568
 0.2271,  0.5200, -0.1447, -0.8011, -0.7699,  0.3128
 0.6415,  0.1993,  0.3777, -0.0178, -0.8237,  0.2181
-0.5298, -0.0768, -0.6028, -0.9490,  0.4588,  0.4356
 0.6870, -0.1431,  0.7294,  0.3141,  0.1621,  0.1632
-0.5985,  0.0591,  0.7889, -0.3900,  0.7419,  0.2945
 0.3661,  0.7984, -0.8486,  0.7572, -0.6183,  0.3449
 0.6995,  0.3342, -0.3113, -0.6972,  0.2707,  0.1712
 0.2565,  0.9126,  0.1798, -0.6043, -0.1413,  0.2893
-0.3265,  0.9839, -0.2395,  0.9854,  0.0376,  0.4770
 0.2690, -0.1722,  0.9818,  0.8599, -0.7015,  0.3954
-0.2102, -0.0768,  0.1219,  0.5607, -0.0256,  0.3949
 0.8216, -0.9555,  0.6422, -0.6231,  0.3715,  0.0801
-0.2896,  0.9484, -0.7545, -0.6249,  0.7789,  0.4370
-0.9985, -0.5448, -0.7092, -0.5931,  0.7926,  0.5402

Test data:

# synthetic_test_40.txt
#
 0.7462,  0.4006, -0.0590,  0.6543, -0.0083,  0.1935
 0.8495, -0.2260, -0.0142, -0.4911,  0.7699,  0.1078
-0.2335, -0.4049,  0.4352, -0.6183, -0.7636,  0.5088
 0.1810, -0.5142,  0.2465,  0.2767, -0.3449,  0.3136
-0.8650,  0.7611, -0.0801,  0.5277, -0.4922,  0.7140
-0.2358, -0.7466, -0.5115, -0.8413, -0.3943,  0.4533
 0.4834,  0.2300,  0.3448, -0.9832,  0.3568,  0.1360
-0.6502, -0.6300,  0.6885,  0.9652,  0.8275,  0.3046
-0.3053,  0.5604,  0.0929,  0.6329, -0.0325,  0.4756
-0.7995,  0.0740, -0.2680,  0.2086,  0.9176,  0.4565
-0.2144, -0.2141,  0.5813,  0.2902, -0.2122,  0.4119
-0.7278, -0.0987, -0.3312, -0.5641,  0.8515,  0.4438
 0.3793,  0.1976,  0.4933,  0.0839,  0.4011,  0.1905
-0.8568,  0.9573, -0.5272,  0.3212, -0.8207,  0.7415
-0.5785,  0.0056, -0.7901, -0.2223,  0.0760,  0.5551
 0.0735, -0.2188,  0.3925,  0.3570,  0.3746,  0.2191
 0.1230, -0.2838,  0.2262,  0.8715,  0.1938,  0.2878
 0.4792, -0.9248,  0.5295,  0.0366, -0.9894,  0.3149
-0.4456,  0.0697,  0.5359, -0.8938,  0.0981,  0.3879
 0.8629, -0.8505, -0.4464,  0.8385,  0.5300,  0.1769
 0.1995,  0.6659,  0.7921,  0.9454,  0.9970,  0.2330
-0.0249, -0.3066, -0.2927, -0.4923,  0.8220,  0.2437
 0.4513, -0.9481, -0.0770, -0.4374, -0.9421,  0.2879
-0.3405,  0.5931, -0.3507, -0.3842,  0.8562,  0.3987
 0.9538,  0.0471,  0.9039,  0.7760,  0.0361,  0.1706
-0.0887,  0.2104,  0.9808,  0.5478, -0.3314,  0.4128
-0.8220, -0.6302,  0.0537, -0.1658,  0.6013,  0.4306
-0.4123, -0.2880,  0.9074, -0.0461, -0.4435,  0.5144
 0.0060,  0.2867, -0.7775,  0.5161,  0.7039,  0.3599
-0.7968, -0.5484,  0.9426, -0.4308,  0.8148,  0.2979
 0.7811,  0.8450, -0.6877,  0.7594,  0.2640,  0.2362
-0.6802, -0.1113, -0.8325, -0.6694, -0.6056,  0.6544
 0.3821,  0.1476,  0.7466, -0.5107,  0.2592,  0.1648
 0.7265,  0.9683, -0.9803, -0.4943, -0.5523,  0.2454
-0.9049, -0.9797, -0.0196, -0.9090, -0.4433,  0.6447
-0.4607,  0.1811, -0.2389,  0.4050, -0.0078,  0.5229
 0.2664, -0.2932, -0.4259, -0.7336,  0.8742,  0.1834
-0.4507,  0.1029, -0.6294, -0.1158, -0.6294,  0.6081
 0.8948, -0.0124,  0.9278,  0.2899, -0.0314,  0.1534
-0.1323, -0.8813, -0.0146, -0.0697,  0.6135,  0.2386
Posted in Machine Learning | Leave a comment

Checking Machine Learning Training Data for Multicollinearity Using VIF (Variance Inflation Factor) With From-Scratch JavaScript

Posted on August 3, 2026 by jamesdmccaffrey

In machine learning, if training data is multicollinear, the interpretability of the resulting model will likely be poor (the details are tricky and out of the scope of this post).

Multicollinear data has two or more rows that are highly correlated mathematically. For example, if a set of training data has a column (predictor variable) house-size (in square feet), and another column house-price, those two columns are likely highly correlated with each other.

Note: Multicollinear data is usually not a problem for neural network regression, and not a problem for tree-based regression (random forest, gradient boosting).

I have implemented VIF analysis using from-scratch C# and from-scratch Python. One morning before work, I figured I’d implement VIF analysis from-scratch JavaScript.

VIF is a value between 1.0 and positive infinity (actually, in weird scenarios, a VIF value could be less than one). Briefly, if all column VIF values are less than about 7.0, the data is probably OK.

if VIF is close to 1.0, the column is not correlated.
if VIF between 1.0 and 5.0, column is mildly correlated
if VIF between 5.0 and 10.0, column is highly correlated
if VIF greater than 10.0, column is extremely correlated

To compute the VIF for a specified column of training data, you use the specified column as the dependent y variable, and use the remaining columns as the independent predictor variables, and compute a linear regression model, and then compute the R2 (coefficient of determination) for the model. The VIF value for the column is 1.0 / (1.0 – R2).

Suppose that you have a set of training data X predictor values, and you use some column c as the dependent y variable, and all the other columns as predictors for c. After training the linear regression model, you compute R2 and it is 0.90 — which means column c is predicted very well by the other columns. The VIF value for column c is 1.0 / (1.0 – R2) = 1.0 / 0.10 = 10.0 which is large which is bad because column c is a linear combination of the other columns — the data is somewhat multicollinear.

Now, with the same setup, suppose R2 is 0.20 — which means column c cannot be predicted well by the other columns. The VIF value is 1.0 / (1.0 – 0.20) = 1.0 / 0.8 = 1.25 which is a small value, which is good, because column c is not a linear combination of the other columns, and therefore the data is not multicollinear.

I created two datasets. The first data set has five columns of predictors, followed by a column of target y values. The data is “normal” in the sense that there’s no multicollinearity. There are 20 items. It looks like:

-0.1660,  0.4406, -0.9998, -0.3953, -0.7065,  0.4840
 0.0776, -0.1616,  0.3704, -0.5911,  0.7562,  0.1568
-0.9452,  0.3409, -0.1654,  0.1174, -0.7192,  0.8054
. . .

The second dataset is highly multicollinear, where the third column is 2 times the first column, plus the second column, plus a small random value between 0.000 and 0.001. It looks like:

-0.1660,  0.4406,  0.1096, -0.3953, -0.7065, 0.4840
 0.0776, -0.1616, -0.0045, -0.5911,  0.7562, 0.1568
-0.9452,  0.3409, -1.5482,  0.1174, -0.7192, 0.8054
. . .

After about 45 minutes of coding, I had a JavaScript program up and running. The output of the demo program is:

Begin variance inflation factor (VIF) demo using JavaScript

Loading synthetic (20) normal non-collinear dataset from file

First two items:
  -0.1660    0.4406   -0.9998   -0.3953   -0.7065
   0.0776   -0.1616    0.3704   -0.5911    0.7562

Begin VIF analysis
col =  0 |  vif = 1.1980
col =  1 |  vif = 1.4591
col =  2 |  vif = 1.2345
col =  3 |  vif = 1.3025
col =  4 |  vif = 1.2120

Loading synthetic (20) highly collinear dataset from file
(col[2] = 2.0 * col[0] + col[1] + rnd)

First two items:
  -0.1660    0.4406    0.1096   -0.3953   -0.7065
   0.0776   -0.1616   -0.0045   -0.5911    0.7562

Begin VIF analysis
col =  0  |  vif = 25546262.9389
col =  1  |  vif = 6023299.3846
col =  2  |  vif = 30951889.0306
col =  3  |  vif = 1.2937
col =  4  |  vif = 1.2117

End VIF demo

As expected, the first dataset didn’t have any bad VIF values, but the VIF values for the second dataset show that columns [0], [1], [2] are highly correlated.

Behind the scenes, I train each linear regression model using a closed-form algorithm — relaxed Moore-Penrose pseudo-inverse via QR decomposition (Householder version). This could fail for very large datasets. In such cases, you can use stochastic gradient descent (SGD), which can handle arbitrarily large datasets.

No big moral to this blog post. Just an interesting exploration.


I’m a big fan machine learning, and old science fiction movies. In machine learning, highly correlated data columns are a bad thing. In old science fiction movies, highly correlated couples are a good thing.

Left: In “Earth vs. the Flying Saucers” (1956), scientist Dr. Russell Marvin and his new wife Carol, have good chemistry, even when they’re being stalked by a flying saucer, visible in the rear window of their car.

Right: In “Beginning of the End” (1957), scientist Dr. Ed Wainwright and romantic interest, news reporter Audrey Aimes, have good chemistry, even when they’re about to encounter huge, man-eating grasshoppers.

Demo program. Replace “lt” (lss than), “gt”, “lte”, “gte” with Boolean operator symbols (my blog editor chokes on symbols).

// variance_inflation_factor.js
// compute VIF: are columns of a dataset collinear?

let FS = require("fs")  // for loadTxt()

// ==========================================================

function varInfFactor(data, col)
{
  // treat col as dependent variable, other columns as preds
  let X = matDeleteCol(data, col);
  let y = matGetColumn(data, col)
  let model = new LinearRegressor();
  model.train(X, y);
  let r2 = model.R2(X, y);
  let vif = 1.0 / (1.0 - r2);
  return vif;
}

// helpers for varInfFactor()

function matDeleteCol(data, col)
{
  let nRows = data.length;
  let nCols = data[0].length;

  let result = matMake(nRows, nCols-1, 0.0);

  let k = 0; // into result
  for (let j = 0; j "lt" nCols; ++j) {
    if (j == col) continue;
    for (let i = 0; i "lt" nRows; ++i)
      result[i][k] = data[i][j];
    ++k;
  }
  return result;
}

function matGetColumn(data, col)
{
  let nRows = data.length;
  let nCols = data[0].length;
  let result = vecMake(nRows, 0.0);
  for (let i = 0; i "lt" nRows; ++i)
    result[i] = data[i][col];
  return result;
}

// minor helpers for the helpers

function matMake(nRows, nCols, val)
{
  let result = [];
  for (let i = 0; i "lt" nRows; ++i) {
    result[i] = [];
    for (let j = 0; j "lt" nCols; ++j) {
      result[i][j] = val;
    }
  }
  return result;
}

function vecMake(n, val)
{
  let result = [];
  for (let i = 0; i "lt" n; ++i) {
    result[i] = val;
  }
  return result;
}

// ==========================================================

class LinearRegressor
{
  // LR using closed-form MP pseudo-inverse training 

  constructor(seed)
  {
    this.weights;            // allocated in train()
    this.bias = 0.0;         // supplied in train()
  }

  // --------------------------------------------------------
  // methods: train(), predict(), R2()
  // --------------------------------------------------------

  predict(x)
  {
    let sum = 0.0;
    for (let i = 0; i "lt" x.length; ++i) {
      sum += x[i] * this.weights[i];
    }
    sum += this.bias;
    return sum;
  }

  // --------------------------------------------------------

  train(trainX, trainY)
  {
    // w = pinv(DX) * y
    let dim = trainX[0].length;         // number predictors
    this.weights = this.vecMake(dim, 0.0);  // allocate wts

    let DX = this.matToDesign(trainX);  // design matrix
    let Xpinv = this.matPinv(DX);       // pinv of design X
    let biasAndWts = this.matVecProd(Xpinv, trainY);

    this.bias = biasAndWts[0];
    for (let i = 1; i "lt" biasAndWts.length; ++i)
      this.weights[i-1] = biasAndWts[i];
    return;
  }

  // --------------------------------------------------------

  R2(dataX, dataY)
  {
    let n = dataX.length;
    let sum = 0.0;

    for (let i = 0; i "lt" n; ++i)
      sum += dataY[i];
    let meanY = sum / n;

    let ssRes = 0.0; // sum squared residuals
    let ssTot = 0.0; // sum squared total

    for (let i = 0; i "lt" n; ++i) {
      let predY = this.predict(dataX[i]);
      ssRes += (dataY[i] - predY) * (dataY[i] - predY);
      ssTot += (dataY[i] - meanY) * (dataY[i] - meanY);
    }
    return 1.0 - (ssRes / ssTot);
  }

  // --------------------------------------------------------
  // primary helpers for train()
  // matVecProd, matPinv, matToDesign
  // --------------------------------------------------------

  matVecProd(M, v)
  {
    // return a regular 1D vector
    let nRows = M.length;
    let nCols = M[0].length;
    let n = v.length;  // assume nCols = n

    let result = this.vecMake(nRows, 0.0);
    for (let i = 0; i "lt" nRows; ++i)
      for (let k = 0; k "lt" nCols; ++k)
        result[i] += M[i][k] * v[k];

    return result;
  }

  // --------------------------------------------------------

  matToDesign(M)
  {
    // add a leading column of 1.0s to M
    let nRows = M.length;
    let nCols = M[0].length;
    let result = this.matMake(nRows, nCols+1, 1.0);

    for (let i = 0; i "lt" nRows; ++i) {
      for (let j = 1; j "lt" nCols+1; ++j) {  // note 1s
        result[i][j] = M[i][j-1];
      }
    }
    return result;
  }

  // --------------------------------------------------------

  matPinv(M) // Moore-Penrose using QR-Householder decomp
  {
    // A = Q*R, pinv(A) = inv(R) * trans(Q) 
    let m = M.length; let n = M[0].length;
    if (m "lt" n)
      cosole.log("ERROR. works only m "gte" m");
    let QR = this.matDecomposeQR(M, true);  // reduced  
    let Rinv = this.matInvUpperTri(QR[1]);
    let Qinv = this.matTranspose(QR[0]);
    let result = this.matProduct(Rinv, Qinv);
    return result;
  }

  // --------------------------------------------------------

  // helpers for matPinv
  // matDecomposeQR, matInvUpperTri, matTranspose, matProduct
  // --------------------------------------------------------

  matDecomposeQR(M, reduced)
  {
    // QR decomposition, Householder algorithm.
    // result[0] = Q, result[1] = R
    let m = M.length;
    let n = M[0].length;

    if (m "lt" n)
      console.log("No rows less than cols");

    let Q = this.matIdentity(m); // working Q
    let R = this.matCopy(M); // working R

    let end = 0;
    if (m == n) end = n - 1;
    else end = n;

    for (let i = 0; i "lt" end; ++i) {
      let H = this.matIdentity(m);
      let a = this.vecMake(m-i, 0.0);
      let k = 0;
      for (let ii = i; ii "lt" m; ++ii) // corr
        a[k++] = R[ii][i];

      let normA = this.vecNorm(a);
      if (a[0] "lt" 0.0 && normA "gt" 0.0) // corr
        normA = -normA;
      else if (a[0] "gt" 0.0 && normA "lt" 0.0)
        normA = -normA;

      let v = this.vecMake(a.length, 0.0); 
      for (let j = 0; j "lt" v.length; ++j)
        v[j] = a[j] / (a[0] + normA);
      v[0] = 1.0;

      // Householder algorithm
      let h = this.matIdentity(a.length);
      let vvDot = this.vecDot(v, v);
      let A = this.vecToMat(v, v.length, 1);
      let B = this.vecToMat(v, 1, v.length);
      let AB = this.matProduct(A, B);

      for (let ii = 0; ii "lt" h.length; ++ii)
        for (let jj = 0; jj "lt" h[0].length; ++jj)
          h[ii][jj] -= (2.0 / vvDot) * AB[ii][jj];

      // copy h[][] into lower right corner of H[][]
      let d = m - h.length; // corr
      for (let ii = 0; ii "lt" h.length; ++ii)
        for (let jj = 0; jj "lt" h[0].length; ++jj)
          H[ii + d][jj + d] = h[ii][jj];

      Q = this.matProduct(Q, H);
      R = this.matProduct(H, R);
    } // i

    let result = [];

    if (reduced == false) {
      result[0] = Q;
      result[1] = R;
      return result;
    }
    else if (reduced == true) {
      let qRows = Q.length; let qCols = Q[0].length;
      let rRows = R.length; let rCols = R[0].length;
      // assumes m "gte" n !!

      // square-up R
      let dim = Math.min(rRows, rCols);
      let Rsquared = this.matMake(dim, dim, 0.0);
      for (let i = 0; i "lt" dim; ++i)
        for (let j = 0; j "lt" dim; ++j)
          Rsquared[i][j] = R[i][j];

      // Q needs same number columns as R
      // so that inv(R) * trans(Q) works
      let Qtrimmed = this.matMake(qRows, dim, 0.0);
      for (let i = 0; i "lt" qRows; ++i)
        for (let j = 0; j "lt" dim; ++j)
          Qtrimmed[i][j] = Q[i][j];

      result[0] = Qtrimmed;
      result[1] = Rsquared;
      return result;
    }
  } // matDecomposeQR()

  // --------------------------------------------------------

  matInvUpperTri(U)
  {
    let n = U.length;  // U must be square matrix
    let result = this.matIdentity(n);
      
    for (let k = 0; k "lt" n; ++k) {
      for (let j = 0; j "lt" n; ++j) {
        for (let i = 0; i "lt" k; ++i) {
          result[j][k] -= result[j][i] * U[i][k];
        }
        result[j][k] /= (U[k][k] + 1.0e-8); // avoid 0
      }
    }
    return result;
  }

  // --------------------------------------------------------

  matTranspose(M)
  {
    let nRows = M.length;
    let nCols = M[0].length;
    let result = this.matMake(nCols, nRows, 0.0);
    for (let i = 0; i "lt" nRows; ++i)
      for (let j = 0; j "lt" nCols; ++j)
        result[j][i] = M[i][j];
    return result;  
  }

  // --------------------------------------------------------

  matProduct(matA, matB)
  {
    let aRows = matA.length;
    let aCols = matA[0].length;
    let bRows = matB.length;
    let bCols = matB[0].length;

    let result = this.matMake(aRows, bCols, 0.0);

    for (let i = 0; i "lt" aRows; ++i)
      for (let j = 0; j "lt" bCols; ++j)
        for (let k = 0; k "lt" aCols; ++k) 
          result[i][j] += matA[i][k] * matB[k][j];

    return result;
  }

  // --------------------------------------------------------
  // misc helper methods/functions:
  // matMake, matCopy, matIdentity, vecMake,
  // vecToMat, vecNorm, vecDot
  //
  // all of these could be declared 'static'
  // --------------------------------------------------------

  matMake(nRows, nCols, val)
  {
    let result = [];
    for (let i = 0; i "lt" nRows; ++i) {
      result[i] = [];
      for (let j = 0; j "lt" nCols; ++j) {
        result[i][j] = val;
      }
    }
    return result;
  }

  // --------------------------------------------------------

  matCopy(M)
  {
    let nRows = M.length;
    let nCols = M[0].length;
    let result = this.matMake(nRows, nCols, 0.0);
    for (let i = 0; i "lt" nRows; ++i)
      for (let j = 0; j "lt" nCols; ++j)
        result[i][j] = M[i][j];
    return result;
  }

  // --------------------------------------------------------

  matIdentity(n)
  {
    let result = this.matMake(n, n, 0.0);
    for (let i = 0; i "lt" n; ++i)
      result[i][i] = 1.0;
    return result;
  }

  // --------------------------------------------------------

  vecMake(n, val)
  {
    let result = [];
    for (let i = 0; i "lt" n; ++i) {
      result[i] = val;
    }
    return result;
  }

  // --------------------------------------------------------

  vecToMat(vec, nRows, nCols)
  {
    let result = this.matMake(nRows, nCols, 0.0);
    let k = 0;
    for (let i = 0; i "lt" nRows; ++i)
      for (let j = 0; j "lt" nCols; ++j)
        result[i][j] = vec[k++];
    return result;
  }

  // --------------------------------------------------------

  vecNorm(vec)
  {
    let sum = 0.0;
    let n = vec.length;
    for (let i = 0; i "lt" n; ++i)
      sum += vec[i] * vec[i];
    return Math.sqrt(sum);
  }

  // --------------------------------------------------------

  vecDot(v1, v2)
  {
    let n = v1.length;  // assume len(v1) == len(v2)
    let sum = 0.0;
    for (let i = 0; i "lt" n; ++i)
      sum += v1[i] * v2[i];
    return sum;
  }

  // --------------------------------------------------------

  // --------------------------------------------------------
  // member methods for debugging: matShow, vecShow
  // --------------------------------------------------------

  matShow(M, dec, wid) // for debugging
  {
    let small = 1.0 / Math.pow(10, dec);
    let nr = M.length;
    let nc = M[0].length;
    for (let i = 0; i "lt" nr; ++i) {
      for (let j = 0; j "lt" nc; ++j) {
        let x = M[i][j];
        if (Math.abs(x) "lt" small) x = 0.0;
        let xx = x.toFixed(dec);
        let s = xx.toString().padStart(wid, ' ');
        process.stdout.write(s);
        process.stdout.write(" ");
      }
      process.stdout.write("\n");
    }
  }

  // --------------------------------------------------------

  vecShow(vec, dec, wid)  // for debugging
  {
    let small = 1.0 / Math.pow(10, dec);
    for (let i = 0; i "lt" vec.length; ++i) {
      let x = vec[i];
      if (Math.abs(x) "lt" small) x = 0.0  // avoid -0.00
      let xx = x.toFixed(dec);
      let s = xx.toString().padStart(wid, ' ');
      process.stdout.write(s);
      process.stdout.write(" ");
    }
    process.stdout.write("\n");
  }

} // end class LinearRegressor

// ==========================================================

// ----------------------------------------------------------
// helper functions for main()
// matShow, vecShow, matLoad, matToVec
// ----------------------------------------------------------

function vecShow(vec, dec, wid, nl)
{
  let small = 1.0 / Math.pow(10, dec);
  for (let i = 0; i "lt" vec.length; ++i) {
    let x = vec[i];
    if (Math.abs(x) "lt" small) x = 0.0  // avoid -0.00
    let xx = x.toFixed(dec);
    let s = xx.toString().padStart(wid, ' ');
    process.stdout.write(s);
    process.stdout.write(" ");
  }

  if (nl == true)
    process.stdout.write("\n");
}

// ----------------------------------------------------------

function matShow(A, dec, wid)
{
  let small = 1.0 / Math.pow(10, dec);
  let nr = A.length;
  let nc = A[0].length;
  for (let i = 0; i "lt" nr; ++i) {
    for (let j = 0; j "lt" nc; ++j) {
      let x = A[i][j];
      if (Math.abs(x) "lt" small) x = 0.0;
      let xx = x.toFixed(dec);
      let s = xx.toString().padStart(wid, ' ');
      process.stdout.write(s);
      process.stdout.write(" ");
    }
    process.stdout.write("\n");
  }
}

// ----------------------------------------------------------

function matToVec(M)
{
  let nr = M.length;
  let nc = M[0].length;
  let result = [];
  for (let i = 0; i "lt" nr*nc; ++i) {  // vecMake(r*c, 0.0);
    result[i] = 0.0;
  }
  let k = 0;
  for (let i = 0; i "lt" nr; ++i) {
    for (let j = 0; j "lt" nc; ++j) {
      result[k++] = M[i][j];
    }
  }
  return result;
}

// ----------------------------------------------------------

function matLoad(fn, delimit, usecols, comment)
{
  // efficient but mildly complicated
  let all = FS.readFileSync(fn, "utf8");  // giant string
  all = all.trim();  // strip final crlf in file
  let lines = all.split("\n");  // array of lines

  // count number non-comment lines
  let nRows = 0;
  for (let i = 0; i "lt" lines.length; ++i) {
    if (!lines[i].startsWith(comment))
      ++nRows;
  }
  let nCols = usecols.length;
  // let result = matMake(nRows, nCols, 0.0); 
  let result = [];
  for (let i = 0; i "lt" nRows; ++i) {
    result[i] = [];
    for (let j = 0; j "lt" nCols; ++j) {
      result[i][j] = 0.0;
    }
  }
 
  let r = 0;  // into lines
  let i = 0;  // into result[][]
  while (r "lt" lines.length) {
    if (lines[r].startsWith(comment)) {
      ++r;  // next row
    }
    else {
      let tokens = lines[r].split(delimit);
      for (let j = 0; j "lt" nCols; ++j) {
        result[i][j] = parseFloat(tokens[usecols[j]]);
      }
      ++r;
      ++i;
    }
  }

  return result;
}

// ==========================================================

function main()
{
  console.log("\nBegin variance inflation factor (VIF) " +
    "demo using JavaScript ");

  // non-collinear (good) dataset
  console.log("\nLoading synthetic (20) normal " +
    "non-collinear dataset from file ");

  let file1 = ".\\Data\\synthetic_train_20.txt";
  let train1 = matLoad(file1, ",", [0,1,2,3,4], "#");

  console.log("\nFirst two items: ");
  for (let i = 0; i "lt" 2; ++i)
    vecShow(train1[i], 4, 9, true);

  console.log("\nBegin VIF analysis ");
  for (let j = 0; j "lt" train1[0].length; ++j) {
    let z = varInfFactor(train1, j);
    console.log("col = " + j.toString().padStart(2) + 
      " |  vif = " + z.toFixed(4).toString() );
  }

  // collinear (bad) dataset
  console.log("\nLoading synthetic (20) highly collinear " +
    "dataset from file ");
  console.log("(col[2] = 2.0 * col[0] + col[1] + rnd) " );

  let file2 = ".\\Data\\synthetic_train_20_collinear.txt";
  let train2 = matLoad(file2, ",", [0,1,2,3,4], "#");

  console.log("\nFirst two items: ");
  for (let i = 0; i "lt" 2; ++i)
    vecShow(train2[i], 4, 9, true);

  console.log("\nBegin VIF analysis ");
  for (let j = 0; j "lt" train2[0].length; ++j) {
    let z = varInfFactor(train2, j);
    console.log("col = " + j.toString().padStart(2) +
      "  |  vif = " + z.toFixed(4).toString());
  }

  console.log("\nEnd VIF demo");
}

main();

First, normal, dataset:

# synthetic_train_20.txt
#
-0.1660,  0.4406, -0.9998, -0.3953, -0.7065,  0.4840
 0.0776, -0.1616,  0.3704, -0.5911,  0.7562,  0.1568
-0.9452,  0.3409, -0.1654,  0.1174, -0.7192,  0.8054
 0.9365, -0.3732,  0.3846,  0.7528,  0.7892,  0.1345
-0.8299, -0.9219, -0.6603,  0.7563, -0.8033,  0.7955
 0.0663,  0.3838, -0.3690,  0.3730,  0.6693,  0.3206
-0.9634,  0.5003,  0.9777,  0.4963, -0.4391,  0.7377
-0.1042,  0.8172, -0.4128, -0.4244, -0.7399,  0.4801
-0.9613,  0.3577, -0.5767, -0.4689, -0.0169,  0.6861
-0.7065,  0.1786,  0.3995, -0.7953, -0.1719,  0.5569
 0.3888, -0.1716, -0.9001,  0.0718,  0.3276,  0.2500
 0.1731,  0.8068, -0.7251, -0.7214,  0.6148,  0.3297
-0.2046, -0.6693,  0.8550, -0.3045,  0.5016,  0.2129
 0.2473,  0.5019, -0.3022, -0.4601,  0.7918,  0.2613
-0.1438,  0.9297,  0.3269,  0.2434, -0.7705,  0.5171
 0.1568, -0.1837, -0.5259,  0.8068,  0.1474,  0.3307
-0.9943,  0.2343, -0.3467,  0.0541,  0.7719,  0.5581
 0.2467, -0.9684,  0.8589,  0.3818,  0.9946,  0.1092
-0.6553, -0.7257,  0.8652,  0.3936, -0.8680,  0.7018
 0.8460,  0.4230, -0.7515, -0.9602, -0.9476,  0.1996

Second, multicollinear, dataset:

# synthetic_train_20_collinear.txt
# col [2] = 2*[0] + [1] + rand(0.001)
#
-0.1660,  0.4406,  0.1096, -0.3953, -0.7065, 0.4840
 0.0776, -0.1616, -0.0045, -0.5911,  0.7562, 0.1568
-0.9452,  0.3409, -1.5482,  0.1174, -0.7192, 0.8054
 0.9365, -0.3732,  1.5016,  0.7528,  0.7892, 0.1345
-0.8299, -0.9219, -2.5800,  0.7563, -0.8033, 0.7955
 0.0663,  0.3838,  0.5179,  0.3730,  0.6693, 0.3206
-0.9634,  0.5003, -1.4245,  0.4963, -0.4391, 0.7377
-0.1042,  0.8172,  0.6100, -0.4244, -0.7399, 0.4801
-0.9613,  0.3577, -1.5636, -0.4689, -0.0169, 0.6861
-0.7065,  0.1786, -1.2325, -0.7953, -0.1719, 0.5569
 0.3888, -0.1716,  0.6073,  0.0718,  0.3276, 0.2500
 0.1731,  0.8068,  1.1544, -0.7214,  0.6148, 0.3297
-0.2046, -0.6693, -1.0770, -0.3045,  0.5016, 0.2129
 0.2473,  0.5019,  0.9980, -0.4601,  0.7918, 0.2613
-0.1438,  0.9297,  0.6435,  0.2434, -0.7705, 0.5171
 0.1568, -0.1837,  0.1313,  0.8068,  0.1474, 0.3307
-0.9943,  0.2343, -1.7528,  0.0541,  0.7719, 0.5581
 0.2467, -0.9684, -0.4732,  0.3818,  0.9946, 0.1092
-0.6553, -0.7257, -2.0345,  0.3936, -0.8680, 0.7018
 0.8460,  0.4230,  2.1166, -0.9602, -0.9476, 0.1996
Posted in JavaScript, Machine Learning | Leave a comment

Machine Learning and Psychology: Sunk Cost Fallacy, Complexity Fallacy, and Obnoxious Researchers

Posted on July 31, 2026 by jamesdmccaffrey

In a nutshell, engineers and researchers:

a.) Have a tendency to continue investing time, money, and effort into a failing venture simply because they have already invested resources in it.

b.) Have a tendency to believe that a complex solution must be better than a simple solution even in the face of contradictory facts.

c.) When compensation and prestige is tied to being an expert, clarity is a liability, and researchers can become obnoxious jerks.

I spent a decade as a university professor (teaching math and computer science), and two decades at Microsoft Research (working on machine learning). Even though my career was in technical fields, my very first BA degree was in Psychology (from UC Irvine).

Recently, while I was working with tree-based regression (predict a single numeric value) systems, I noticed that in literally all of my experiments, AdaBoost.R2 and Gradient Boost regression with super-simple Extra Tree regressors as the weak learners worked better than AdaBoost.R2 and Gradient Boost regression with extremely-complex standard Decision Tree regressors as the weak learners.

However, all library implementations of AdaBoost.R2 and Gradient Boost regression use Decision Tree regressors as the default. Why? Because of the sunk cost fallacy combined with the complexity fallacy.

In research and engineering environments, team prestige, grant funding, patents, and thousands of commit histories are bound up in the systems people build.

The Sunk Cost Fallacy: Admitting that a random split heuristic (with Extra Trees) performs just as well, or better than, a hyper-optimized greedy tree algorithm requires confronting the uncomfortable truth that months of micro-optimizing split-criterion logic, dynamic programming, and search trees weren’t necessary for most real-world tasks.

The Complexity Fallacy: There is a pervasive bias in computer science and research that more complex algorithms must be superior. A mathematically intricate split optimizer (like XGBoost exact greedy algorithm or second-order gradient expansion) feels “smarter” than literally rolling a pair of dice to pick a threshold, even when the rolling-dice technique gives better practical results.

Standard Decision Trees had a 20-year head start in academia and production codebases before Extremely Randomized Trees were introduced in 2006. By then, the entire ecosystem (libraries, papers, benchmarks, hardware optimizations) was already anchored around greedy splits with standard Decision Tree learners.

On top of all this, there is a huge I-Know-More-Than-You-Do ego effect.

Arguably the most pervasive, unwritten political dynamic in technical organizations is complexity as job security and social status.

In research, the phrase “knowledge is power” takes on a very literal, cynical form. When an engineer or researcher’s compensation, title, and influence are tied to being an irreplaceable subject matter expert, clarity is a liability. If a solution is simple, elegant, and easily understood by a junior engineer or a product manager, it strips away the author’s aura of specialized genius.

This leads to preferring overly-complex techniques that can’t be easily understood by non-experts or even other experts. Put a bit more bluntly, although the vast majority of my colleagues were nice and collaborative, there were a lot of complete jerks and jerkettes too.

All of this is a sharp reminder of how technological defaults are shaped as much by human psychology and historical inertia as by pure engineering merit. A simpler method often works just as well or better because nature and real-world data are noisy, and simple random heuristics are surprisingly robust against noise.


The Elizabeth Holmes Effect

Posted in Miscellaneous | Leave a comment

New Version of Matrix QR Decomposition With the Householder Algorithm Using C#

Posted on July 30, 2026 by jamesdmccaffrey

In the back of my mind, I wasn’t entirely happy with my current version of matrix QR decomposition using the Householder algorithm. The current version worked fine, but the underlying code just didn’t feel quite right in some way that I couldn’t articulate.

So I decided to take another stab at matrix decomposition using Householder. For me, the main purpose of QR decomposition is to compute the relaxed Moore-Penrose pseudo-inverse, which in turn is used to train a linear regression model or a quadratic regression model.

After a day or so of work, I got a new version of QR-Householder decomposition up and running to my satisfaction. The output of a demo run:

Begin QR decomposition using Householder algorithm
More precise than QR w/ modified Gram-Schmidt and QR w/
Givens but more complicated.
Latest version (July 2026)

Source (tall) matrix A:
   1.0   2.0   3.0   4.0   5.0
   0.0  -3.0   5.0  -7.0   9.0
   2.0   0.0  -2.0   0.0  -2.0
   4.0  -1.0   5.0   6.0   1.0
   3.0   6.0   8.0   2.0   2.0
   5.0  -2.0   4.0  -4.0   3.0

Computing QR
Done

Q =
  -0.134840   0.258894   0.147094  -0.310903   0.869379
   0.000000  -0.410745   0.759588   0.274531   0.180705
  -0.269680  -0.029872  -0.526675   0.219131   0.300339
  -0.539360  -0.196660   0.116670  -0.738280  -0.260150
  -0.404520   0.776682   0.316260   0.255197  -0.226191
  -0.674200  -0.348511  -0.101841   0.412033   0.049823

R =
  -7.416198  -0.809040  -8.494918  -1.887760  -3.505839
   0.000000   7.303797   2.618812   5.678242  -2.031322
   0.000000   0.000000   7.998637  -2.988836   9.068775
   0.000000   0.000000   0.000000  -8.732743   1.486219
   0.000000   0.000000   0.000000  -0.000000   4.809501

End demo

I validated my QR decomp implementation by sending the same input matrix to the Python language np.linalg.qr() function, to make sure the results were the same.

There are a ton of details. My implementation workks only for matrices that have more rows than columns — such as training data. My implementation returns a ‘reduced’ Q and a ‘reduced’ R (needed for pseudo-inverse) rather than ‘full’ Q and R matrices.

OK. Good fun. Next, I’ll need to implement a relaxed Moore-Penrose pseudo-inverse using my new QR decomp code. And then after that, I’ll need to use the pseudo-inverse to train a linear regression model.


I’m not very good at articulating subjective things. Here are two screen captures from a short AI-generated video titled “Oracle”, from a guy called Anglomangler. I can’t explain why or how, but the nightmarish quality of the video really makes an impact on me.

Demo program. Replace “lt” (less than), “gt”, “lte”, “gte” with Boolean operator symbols (my blog editor chokes on them).

using System;
using System.IO;

namespace MatrixDecompQRHouseholder
{
  internal class Program
  {
    static void Main(string[] args)
    {
      Console.WriteLine("\nBegin QR decomposition using" +
        " Householder algorithm ");
      Console.WriteLine("More precise than QR w/ modified" +
        " Gram-Schmidt and QR w/ Givens but more " +
        "complicated. ");
      Console.WriteLine("Latest version (June 2026) ");
      
      double[][] A = new double[6][];
      A[0] = new double[] { 1, 2, 3, 4, 5 };
      A[1] = new double[] { 0, -3, 5, -7, 9 };
      A[2] = new double[] { 2, 0, -2, 0, -2 };
      A[3] = new double[] { 4, -1, 5, 6, 1 };
      A[4] = new double[] { 3, 6, 8, 2, 2 };
      A[5] = new double[] { 5, -2, 4, -4, 3 };

      Console.WriteLine("\nSource (tall) matrix A: ");
      MatShow(A, 1, 6);

      Console.WriteLine("\nComputing QR ");
      double[][] Q;
      double[][] R;
      QRHouseholder.MatDecompQR(A, out Q, out R);
      Console.WriteLine("Done ");

      Console.WriteLine("\nQ = ");
      MatShow(Q, 6, 11);
      Console.WriteLine("\nR = ");
      MatShow(R, 6, 11);

      Console.WriteLine("\nEnd demo ");
      Console.ReadLine();
    } // Main

    static void MatShow(double[][] M, int dec, int wid)
    {
      for (int i = 0; i "lt" M.Length; ++i)
      {
        for (int j = 0; j "lt" M[0].Length; ++j)
        {
          double v = M[i][j];
          Console.Write(v.ToString("F" + dec).
            PadLeft(wid));
        }
        Console.WriteLine("");
      }
    }

  } // class Program

  // ========================================================

  public class QRHouseholder
  {
    public static void MatDecompQR(double[][] A, 
      out double[][] Q,  out double[][] R)
    {
      int m = A.Length; int n = A[0].Length;
      if (m "lt" n)
        Console.WriteLine("FATAL: nRows must be gte nCols ");

      double[][] QQ = MatMake(m, m); // working full Q
      for (int i = 0; i "lt" m; ++i)
        QQ[i][i] = 1.0;  // identity matrix

      double[][] RR = MatMake(m, n);
      for (int i = 0; i "lt" m; ++i)
        for (int j = 0; j "lt" n; ++j)
          RR[i][j] = A[i][j]; // copy of A is working R

      int k = Math.Min(m, n);  // or just use n
      for (int j = 0; j "lt" k; ++j) // main processing loop
      {
        int xn = m - j;
        double[] x = new double[xn];
        for (int i = 0; i "lt" xn; ++i)
          x[i] = RR[j + i][j];

        double ss = 0.0;
        for (int i = 0; i "lt" xn; ++i)
          ss += x[i] * x[i];
        double normX = Math.Sqrt(ss);

        // if (normX == 0.0) continue;
        if (Math.Abs(normX) "lt" 1.0e-12) continue;

        double sign;
        if (x[0] "gte" 0.0) sign = -1.0;
        else sign = 1.0; // counter-intuitive
      
        double[] u = new double[xn];
        for (int i = 0; i "lt" xn; ++i)
          u[i] = x[i] / (x[0] - sign * normX); // check div 0
        u[0] = 1.0;

        // compute scaling factor tau = 2 / (u^T * u)
        double tau = -sign * (x[0] - sign * normX) / normX;

        // dimensions for sub-matrices
        int nRowsSubR = m - j;   int nColsSubR = n - j;
        int nRowsSubQ = m;       int nColsSubQ = m - j;

        double[] vr = new double[nColsSubR];
        for (int c = 0; c "lt" nColsSubR; ++c)
        {
          double acc = 0.0;
          for (int r = 0; r "lt" nRowsSubR; ++r)
            acc += u[r] * RR[j + r][j + c];
          vr[c] = acc;
        }

        double[] vq = new double[nRowsSubQ];
        for (int r = 0; r "lt" nRowsSubQ; ++r)
        {
          double acc = 0.0;
          for (int c = 0; c "lt" nColsSubQ; ++c)
            acc += u[c] * QQ[r][j + c];
          vq[r] = acc;
        }

        // update sub-R
        for (int r = 0; r "lt" nRowsSubR; ++r)
          for (int c = 0; c "lt" nColsSubR; ++c)
            RR[j + r][j + c] -= tau * u[r] * vr[c];

        // update sub-Q
        for (int r = 0; r "lt" nRowsSubQ; ++r)
          for (int c = 0; c "lt" nColsSubQ; ++c)
            QQ[r][j + c] -= tau * vq[r] * u[c];
       
      } // j

      // extract QQ RR into out params
      Q = MatMake(m, n);
      for (int i = 0; i "lt" m; ++i)
        for (int j = 0; j "lt" n; ++j)
          Q[i][j] = QQ[i][j];

      R = MatMake(n, n);
      for (int i = 0; i "lt" n; ++i)
        for (int j = 0; j "lt" n; ++j)
          R[i][j] = RR[i][j];

      return;  

    } // MatDecompQR

    // ------------------------------------------------------

    public static double[][] MatMake(int nRows, int nCols)
    {
      double[][] result = new double[nRows][];
      for (int i = 0; i "lt" nRows; ++i)
        result[i] = new double[nCols];
      return result;
    }

  } // class QR_Householder

  // ========================================================

} // ns

Python language validation program:

import numpy as np

def main():
  A = np.array([
    [ 1, 2, 3, 4, 5 ],
    [ 0, -3, 5, -7, 9 ],
    [ 2, 0, -2, 0, -2 ],
    [ 4, -1, 5, 6, 1 ],
    [ 3, 6, 8, 2, 2 ],
    [ 5, -2, 4, -4, 3 ]], dtype=np.float64)

  print("\nA = "); print(A)

  # call NumPy function
  print("\nNumPy QR: ")
  Q, R = np.linalg.qr(A, mode='reduced')
  print("\nNumPy Q = "); print(Q)
  print("\nNumPy R = "); print(R)

if name == "__main__":
  main()
Posted in Machine Learning | Leave a comment

Support Vector Regression With SGD Training Using JavaScript

Posted on July 29, 2026 by jamesdmccaffrey

The goal of a machine learning regression problem is to predict a single numeric value. For example, a bank might want to predict the maximum safe loan amount for a customer, based on age, account balance, annual income, and so on.

One of about a dozen common regression techniques is (kernel) support vector regression (SVR). I have implemented SVR using Python and C#, but one day before work, I realized that I had not implemented SVR from scratch, using JavaScript. So I figured I’d do so.

There are three main ways to train a kernel support regression model: quadratic programming (QP) optimization, the sequential minimal optimization (SMO) algorithm, and stochastic sub-gradient descent (SGD). I used SGD, which is far by the simplest SVR training technique.

The output of my demo program is:

Begin support vector regression (SVR) with SGD training
 using JavaScript

Loading train (200) and test (40) from file

First three train X:
 -0.1660   0.4406  -0.9998  -0.3953  -0.7065
  0.0776  -0.1616   0.3704  -0.5911   0.7562
 -0.9452   0.3409  -0.1654   0.1174  -0.7192

First three train y:
   0.4840
   0.1568
   0.8054

Setting RBF gamma = 0.3000
Setting epsilon = 0.007500
Setting C = 1.00

Setting SGD lrnRate = 0.0010
Setting SGD maxEpochs = 5000
Setting 0-weight tolerance = 0.000100

Creating and training SVR model using SGD
epoch =      0  MSE = 0.0853  acc = 0.0850
epoch =   1000  MSE = 0.0001  acc = 0.9800
epoch =   2000  MSE = 0.0001  acc = 0.9800
epoch =   3000  MSE = 0.0001  acc = 0.9800
epoch =   4000  MSE = 0.0001  acc = 0.9850
Done

Model weights:
 -0.9224   -0.0454   -0.0018   -0.6600  . . .   0.0005
  0.0066    0.3016    0.4597   -0.0423  . . .  -0.8236
. . .
 -0.9890   -0.0044   -0.0705    0.9898  . . .  -0.1284
  0.3902

Model bias = 0.4020

Number support vectors = 196

Computing model accuracy

Train acc (within 0.10) = 0.9850
Test acc (within 0.10) = 0.9250

Train MSE = 0.0001
Test MSE = 0.0001

Train R2 = 0.9985
Test R2 = 0.9948

Predicting for trainX[0] =
  -0.1660    0.4406   -0.9998   -0.3953   -0.7065
Predicted y = 0.4927

End demo

The demo data is synthetic. It was generated by a 5-10-1 neural network with random weights and bias values. The idea here is that the synthetic data does have an underlying, but complex, non-linear structure which can be predicted.

During training, the SVR model assigns one weight value, into a vector called alpha, to each training item, plus a special weight called the bias. After training, the SVR model determined that 6 of the 200 alpha weights were very close to zero, and so those 6 alpha values and their 6 associated training items were removed. This left 196 alpha weight values and training items, called the support vectors.

All of the parameter values must be determined by trial and error. The gamma parameter defines the RBF function that is used to measure the similarity between data item vectors. Larger values of gamma shrinks the radius of influence of individual training points. This tends to increase model accuracy at the expense of increased risk of model overfitting.

The epsilon value defines how close to correct a prediction must be to be considered a non-support vector. Larger values of epsilon create fewer support vectors.

The C value is used for model regularization, which prevents model alpha weights from becoming very large, which often leads to model overfitting. Larger values of C have a smaller regularization affect.

The lrnRate value controls how much alpha weight values change at each update during training. Larger values of lrnRate increase the speed of training, at the risk of jumping over good weight values.

The maxEpochs value controls how many iterations are performed during training. The effect of larger values of SVR maxEpochs can vary greatly.

The tol (“tolerance”) value controls pruning away training vectors to support vectors, by defining how to close to 0 an alpha weight value must be in order to be pruned away. Larger values of tol allow more alpha weights to be defined as zero, which reduces the number of support vectors.

The biggest weakness of support vector regression is the difficulty of tuning the hyperparameters. Small changes in parameter values can produce extremely large changes in the model, and the hyperparameters interact in complex ways.

Support vector regression had a brief surge of popularity in the late 1990s and early 2000s. However, data scientists realized that the closely related kernel ridge regression (KRR) has several significant advantages over SVR, and so the use of SVR declined to the point where it is not used very much today.

Specifically, SVR is more difficult to implement than KRR, SVR is much more difficult to tune than KRR (KRR can use true SGD, which is easier to tune than SVR sub-gradient descent), and SVR often gives slightly worse prediction accuracy than KRR (due mostly to the difficulty in parameter tuning). That said, there are some problem scenarios where kernel SVR is highly effective.


When I worked at Microsoft Research, I worked on the 4th floor of Building 99, along with the researchers who popularized support vector regression in the late 1990s (John P, Chris B, and others). In engineering, the mindset is to be presented with a specific problem and then find a solution to that problem. But in research, the mindset is to come up with interesting ideas/solutions and then determine if there are any problems the solutions can be applied to. So these potential solutions are examples of “From Research It Came”.

Left: “It Came from Outer Space” (1953). An astronomer and his girlfriend live in a remote desert town. They see a spacecraft land. Soon townspeople start acting strangely, as they are impersonated by the aliens. The aliens just want to repair their spacecraft. They do and there’s a happy ending. My grade = A-.

Center: “It Came From Beneath the Sea” (1955). Atomic testing in the South Pacific disturbs an aircraft carrier sized gigantic octopus. It heads towards San Francisco, wreaking havoc along the way. The menace is finally stopped with an experimental jet-powered torpedo. My grade = B.

Right: “From Hell It Came” (1957). South Pacific island + atomic testing + murdered villager + tree = a very bad tree. Yes, this is not a good movie but it has a certain charm (well, to me anyway). My grade = B-.

Demo program. Replace “lt” (less than), “gt”, “lte”, “gte” with Boolean operator symbols. (My blog editor chokes on symbols).

// svr_sgd.js
// support vector regression using SGD training
// not recommended for most scenarios -- use KRR instead
// node.js environment

let FS = require("fs")  // for loadTxt()

// ----------------------------------------------------------

class SVR
{
  constructor(gamma, epsilon, C, lrnRate, maxEpochs, tol)
  {
    this.gamma = gamma;
    this.epsilon = epsilon;
    this.C = C;

    this.suppX;  // support vectors
    this.suppY;  
    this.alpha;     // weights
    this.b;         // bias

    this.lrnRate = lrnRate;
    this.maxEpochs = maxEpochs;
    this.tol = tol;  // KKT tolerance

    this.seed = 0.5;  // default init
  }

  // --------------------------------------------------------
  // primary: train(), predict(), accuracy(), MSE(), R2()
  // helpers: makeK(), rbf(), shuffle(), next(), nextInt()
  // --------------------------------------------------------

  predict(x)
  {
    // x is a vector (not matrix as in scikit)
    let n = this.suppX.length;
    let sum = 0.0;
    for (let i = 0; i "lt" n; ++i) {
      let xx = this.suppX[i];
      let k = this.rbf(x, xx, this.gamma);
      sum += this.alpha[i] * k;
    }
    return sum + this.b;
  }

  // --------------------------------------------------------

  train(trainX, trainY)
  {
    this.suppX = trainX;
    this.suppY = trainY;
    let n = trainX.length;  // allocate model weights

    this.alpha = vecMake(n, 0.0);
    let lo = -0.10; let hi = 0.10;
    for (let i = 0; i "lt" n; ++i) {
      this.alpha[i] = (hi - lo) * this.next() + lo;
    }
    this.b = 0.0;

    // precompute all rbf values to K for fast train
    let K = this.makeK(trainX); 
    let freq = Math.trunc(this.maxEpochs / 5);  // progress
    let lamda = 1.0 / this.C;

    // set up indices for shuffling
    let indices = vecMake(n, 0);
    for (let i = 0; i "lt" n; ++i)
      indices[i] = i;

    // main sub-gradient processing loop
    for (let epoch = 0; epoch "lt" this.maxEpochs; ++epoch) {
      this.shuffle(indices);
      for (let i = 0; i "lt" indices.length; ++i) {
        let idx = indices[i];
        // let x = trainX[idx];
        let predY = 0.0;
        for (let j = 0; j "lt" this.alpha.length; ++j)
          predY += this.alpha[j] * K[idx][j];
        predY += this.b;
        // let predY = this.predict(x);  // slow
        let actualY = trainY[idx];
        let error = predY - actualY;

        let gradLoss = 0.0;
        let insideTube = false;
        if (error "gt" this.epsilon)
          gradLoss = 1.0;
        else if (error "lt" -this.epsilon)
          gradLoss = -1.0;
        else {
          gradLoss = 0.0;
          insideTube = true;
        }

        // local kernel regularization gradient
        let gradReg = this.alpha[idx] * K[idx][idx];
        //  decoupled updates to the active index
        this.alpha[idx] -= this.lrnRate * 
          (lamda * gradReg + gradLoss);
        this.b -= this.lrnRate * gradLoss;
        
        // force tiny weights to 0
        if (insideTube == true &&
          Math.abs(this.alpha[idx]) "lt" this.tol) {
            this.alpha[idx] = 0.0;
        }

        // in-loop clip to bound updates mid-flight
        if (this.alpha[idx] "lt" -this.C)
          this.alpha[idx] = -this.C;
        else if (this.alpha[idx] "gt" this.C)
          this.alpha[idx] = this.C;

      } // each training item

      if (epoch % freq == 0) // show progress
      {
        let mse = this.MSE(trainX, trainY);
        let acc = this.accuracy(trainX, trainY, 0.10);
        let s1 = "epoch = " +
          epoch.toString().padStart(6, ' ');
        let s2 = "  MSE = " +
          mse.toFixed(4).toString();
        let s3 = "  acc = " + acc.toFixed(4).toString();
        console.log(s1 + s2 + s3);
      }

    } // each epoch

    // final global clip
    for (let i = 0; i "lt" n; ++i) {
      if (this.alpha[i] "lt" -this.C)
        this.alpha[i] = -this.C;
      else if (this.alpha[i] "gt" this.C)
        this.alpha[i] = this.C;
    }

    // prune: store only explicit support vectors and alphas
    let svMask = [];
    for (let i = 0; i "lt" this.alpha.length; ++i) {
      if (Math.abs(this.alpha[i]) "gt" 1.0e-5)
        svMask.push(i);
    }

    this.suppX = matSelectRows(trainX, svMask);
    this.suppY = vecSelectItems(trainY, svMask);
    this.alpha = vecSelectItems(this.alpha, svMask);
 
  } // train()

  // --------------------------------------------------------

  makeK(X)
  {
    // Kernel-Gram matrix helper for train()
    // pre-compute all similarities, to avoid re-computes
    let n = X.length;
    let result = matMake(n, n);
    for (let i = 0; i "lt" n; ++i)
      for (let j = 0; j "lt" n; ++j)
        result[i][j] = 
          this.rbf(X[i], X[j]);
      return result;
    }

  // --------------------------------------------------------

  rbf(v1, v2)
  {
    let n = v1.length;
    let sum = 0.0;
    for (let i = 0; i "lt" n; ++i) {
      let diff = v1[i] - v2[i];
      sum += diff * diff;
    }
    return Math.exp(-1 * this.gamma * sum);
  }

  // --------------------------------------------------------

  accuracy(dataX, dataY, pctClose)
  {
    let nCorrect = 0; let nWrong = 0;
    let n = dataX.length;
    
    for (let i = 0; i "lt" n; ++i) {
      let x = dataX[i];
      let actualY = dataY[i];
      let predY = this.predict(x);
      if (Math.abs(predY - actualY) "lt" 
        Math.abs(pctClose * actualY)) {
        ++nCorrect;
      }
      else {
        ++nWrong;
      }
    }
    return (nCorrect * 1.0) / (nCorrect + nWrong);
  }

  // --------------------------------------------------------

  MSE(dataX, dataY)
  {
    let n = dataX.length;
    let sum = 0.0;
    for (let i = 0; i "lt" n; ++i) {
      let x = dataX[i];
      let actualY = dataY[i];
      let predY = this.predict(x);
      sum += (actualY - predY) * (actualY - predY);
    }
    return sum / n;
  }

  // --------------------------------------------------------

  R2(dataX, dataY)
  {
    let n = dataX.length;
    let ssRes = 0.0; let ssTot = 0.0;
    let meanY = vecMean(dataY);

    for (let i = 0; i "lt" n; ++i) {
      let x = dataX[i];
      let actualY = dataY[i];
      let predY = this.predict(x);
      ssRes += (actualY - predY) * (actualY - predY);
      ssTot += (actualY - meanY) * (actualY - meanY);
    }
    let result = 1.0 - (ssRes / ssTot);
    return result
  }

  // --------------------------------------------------------

  next()
  {
    let x = Math.sin(this.seed) * 1000;
    let result = x - Math.floor(x);  // [0.0,1.0)
    this.seed = result;  // for next call
    return result;
  }

  // --------------------------------------------------------

  nextInt(lo, hi)
  {
    let x = this.next();
    return Math.trunc((hi - lo) * x + lo);
  }

  // --------------------------------------------------------

  shuffle(indices)
  {
    // Fisher-Yates
    for (let i = 0; i "lt" indices.length; ++i) {
      let ri = this.nextInt(i, indices.length);
      let tmp = indices[ri];
      indices[ri] = indices[i];
      indices[i] = tmp;
      //indices[i] = i; // for testing
    }
  }

} // end class KRR

// ==========================================================

// ----------------------------------------------------------
// vector and matrix helper functions
// ----------------------------------------------------------

function vecMake(n, val)
{
  let result = [];
  for (let i = 0; i "lt" n; ++i) {
    result[i] = val;
  }
  return result;
}

// ----------------------------------------------------------

function matMake(rows, cols, val)
{
  let result = [];
  for (let i = 0; i "lt" rows; ++i) {
    result[i] = [];
    for (let j = 0; j "lt" cols; ++j) {
      result[i][j] = val;
    }
  }
  return result;
}

// ----------------------------------------------------------

function matSelectRows(X, rows)
{
  let nRowsSrc = X.length;
  let nColsSrc = X[0].length;
  let n = rows.length;
  let result = matMake(n, nColsSrc, 0.0);

  for (let i = 0; i "lt" n; ++i) { // i pts into result
    let srcRow = rows[i];
    for (let j = 0; j "lt" nColsSrc; ++j) {
      result[i][j] = X[srcRow][j];
    }
  }
  return result;
}

// ----------------------------------------------------------

function vecSelectItems(vec, idxs)
{
  let n = idxs.length;
  let result = vecMake(n, 0.0);
  for (let i = 0; i "lt" n; ++i) {
    result[i] = vec[idxs[i]];
  }
  return result;
}

// ----------------------------------------------------------

function vecMean(vec)
{
  let n = vec.length;
  let sum = 0.0;
  for (let i = 0; i "lt" n; ++i)
    sum += vec[i];
  let result = sum / n;
  return result;
}

// ----------------------------------------------------------

function vecShow(vec, dec, wid, nl)
{
  let small = 1.0 / Math.pow(10, dec);
  for (let i = 0; i "lt" vec.length; ++i) {
    let x = vec[i];
    if (Math.abs(x) "lt" small) x = 0.0  // avoid -0.00
    let xx = x.toFixed(dec);
    let s = xx.toString().padStart(wid, ' ');
    process.stdout.write(s);
    process.stdout.write(" ");
  }

  if (nl == true)
    process.stdout.write("\n");
}

// ----------------------------------------------------------

function matShow(A, dec, wid)
{
  let small = 1.0 / Math.pow(10, dec);
  let nr = A.length;
  let nc = A[0].length;
  for (let i = 0; i "lt" nr; ++i) {
    for (let j = 0; j "lt" nc; ++j) {
      let x = A[i][j];
      if (Math.abs(x) "lt" small) x = 0.0;
      let xx = x.toFixed(dec);
      let s = xx.toString().padStart(wid, ' ');
      process.stdout.write(s);
      process.stdout.write(" ");
    }
    process.stdout.write("\n");
  }
}

// ----------------------------------------------------------

function matToVec(m)
{
  let r = m.length;
  let c = m[0].length;
  let result = 	vecMake(r*c, 0.0);
  let k = 0;
  for (let i = 0; i "lt" r; ++i) {
    for (let j = 0; j "lt" c; ++j) {
      result[k++] = m[i][j];
    }
  }
  return result;
}

// ----------------------------------------------------------

function loadTxt(fn, delimit, usecols, comment)
{
  let all = FS.readFileSync(fn, "utf8");  // giant string
  all = all.trim();  // strip final crlf in file
  let lines = all.split("\n");  // array of lines

  // count number non-comment lines
  let nRows = 0;
  for (let i = 0; i "lt" lines.length; ++i) {
    if (!lines[i].startsWith(comment))
      ++nRows;
  }
  let nCols = usecols.length;
  let result = matMake(nRows, nCols, 0.0); 
 
  let r = 0;  // into lines
  let i = 0;  // into result[][]
  while (r "lt" lines.length) {
    if (lines[r].startsWith(comment)) {
      ++r;  // next row
    }
    else {
      let tokens = lines[r].split(delimit);
      for (let j = 0; j "lt" nCols; ++j) {
        result[i][j] = parseFloat(tokens[usecols[j]]);
      }
      ++r;
      ++i;
    }
  }

  return result;
}

// ----------------------------------------------------------
// ----------------------------------------------------------

function main()
{
  console.log("\nBegin support vector regression (SVR) " +
    "with SGD training using JavaScript ");

  // 1. load data
  console.log("\nLoading train (200) and" +
    " test (40) from file ");

  let trainFile = ".\\Data\\synthetic_train_200.txt";
  let trainX = loadTxt(trainFile, ",", [0,1,2,3,4], "#");
  let trainY = loadTxt(trainFile, ",", [5], "#");
  trainY = matToVec(trainY);
  
  let testFile = ".\\Data\\synthetic_test_40.txt";
  let testX = loadTxt(testFile, ",", [0,1,2,3,4], "#");
  let testY = loadTxt(testFile, ",", [5], "#");
  testY = matToVec(testY);

  console.log("\nFirst three train X: ");
  for (let i = 0; i "lt" 3; ++i)
    vecShow(trainX[i], 4, 8, true); // true: add newline

  console.log("\nFirst three train y: ");
  for (let i = 0; i "lt" 3; ++i)
    console.log(trainY[i].toFixed(4).toString().
    padStart(9, ' '));

  // 2. create and train KRR model
  let gamma = 0.30;    // RBF param
  let epsilon = 0.0075;  // SVR epsilon
  let C = 1.0;  // regularization

  let lrnRate = 0.001;
  let maxEpochs = 5000;
  let tol = 0.0001;
  let seed = 0;

  console.log("\nSetting RBF gamma = " +
    gamma.toFixed(4).toString());
  console.log("Setting epsilon = " +
    epsilon.toFixed(6).toString());
  console.log("Setting C = " +
    C.toFixed(2).toString());

  console.log("\nSetting SGD lrnRate = " +
    lrnRate.toFixed(4).toString());
  console.log("Setting SGD maxEpochs = " +
    maxEpochs.toString());
  console.log("Setting 0-weight tolerance = " +
    tol.toFixed(6).toString());

  console.log("\nCreating and training SVR" +
    " model using SGD ");
  let model = new SVR(gamma, epsilon, C,
    lrnRate, maxEpochs, tol, seed); 
  model.train(trainX, trainY);
  console.log("Done ");

  // 3. show trained model weights
  console.log("\nModel weights: ");
  vecShow(model.alpha, 4, 9, true);

  console.log("\nModel bias = " +
    model.b.toFixed(4).toString());

  let numSupp = model.alpha.length;
  console.log("\nNumber support vectors = " + 
    numSupp.toString());

  // 4. evaluate model
  console.log("\nComputing model accuracy ");
  let trainAcc = model.accuracy(trainX, trainY, 0.10);
  let testAcc = model.accuracy(testX, testY, 0.10);

  console.log("\nTrain acc (within 0.10) = " +
    trainAcc.toFixed(4).toString());
  console.log("Test acc (within 0.10) = " +
    testAcc.toFixed(4).toString());

  let trainMSE = model.MSE(trainX, trainY);
  let testMSE = model.MSE(testX, testY);

  console.log("\nTrain MSE = " +
    trainMSE.toFixed(4).toString());
  console.log("Test MSE = " +
    testMSE.toFixed(4).toString());

  let trainR2 = model.R2(trainX, trainY);
  let testR2 = model.R2(testX, testY);

  console.log("\nTrain R2 = " +
    trainR2.toFixed(4).toString());
  console.log("Test R2 = " +
    testR2.toFixed(4).toString());

  // 5. use model
  let x = trainX[0];
  console.log("\nPredicting for trainX[0] = ");
  vecShow(x, 4, 9, true);  // add newline

  let predY = model.predict(x);
  console.log("Predicted y = " + 
    predY.toFixed(4).toString());

  console.log("\nEnd demo");
}

main();

Training data

# synthetic_train_200.txt
#
-0.1660,  0.4406, -0.9998, -0.3953, -0.7065,  0.4840
 0.0776, -0.1616,  0.3704, -0.5911,  0.7562,  0.1568
-0.9452,  0.3409, -0.1654,  0.1174, -0.7192,  0.8054
 0.9365, -0.3732,  0.3846,  0.7528,  0.7892,  0.1345
-0.8299, -0.9219, -0.6603,  0.7563, -0.8033,  0.7955
 0.0663,  0.3838, -0.3690,  0.3730,  0.6693,  0.3206
-0.9634,  0.5003,  0.9777,  0.4963, -0.4391,  0.7377
-0.1042,  0.8172, -0.4128, -0.4244, -0.7399,  0.4801
-0.9613,  0.3577, -0.5767, -0.4689, -0.0169,  0.6861
-0.7065,  0.1786,  0.3995, -0.7953, -0.1719,  0.5569
 0.3888, -0.1716, -0.9001,  0.0718,  0.3276,  0.2500
 0.1731,  0.8068, -0.7251, -0.7214,  0.6148,  0.3297
-0.2046, -0.6693,  0.8550, -0.3045,  0.5016,  0.2129
 0.2473,  0.5019, -0.3022, -0.4601,  0.7918,  0.2613
-0.1438,  0.9297,  0.3269,  0.2434, -0.7705,  0.5171
 0.1568, -0.1837, -0.5259,  0.8068,  0.1474,  0.3307
-0.9943,  0.2343, -0.3467,  0.0541,  0.7719,  0.5581
 0.2467, -0.9684,  0.8589,  0.3818,  0.9946,  0.1092
-0.6553, -0.7257,  0.8652,  0.3936, -0.8680,  0.7018
 0.8460,  0.4230, -0.7515, -0.9602, -0.9476,  0.1996
-0.9434, -0.5076,  0.7201,  0.0777,  0.1056,  0.5664
 0.9392,  0.1221, -0.9627,  0.6013, -0.5341,  0.1533
 0.6142, -0.2243,  0.7271,  0.4942,  0.1125,  0.1661
 0.4260,  0.1194, -0.9749, -0.8561,  0.9346,  0.2230
 0.1362, -0.5934, -0.4953,  0.4877, -0.6091,  0.3810
 0.6937, -0.5203, -0.0125,  0.2399,  0.6580,  0.1460
-0.6864, -0.9628, -0.8600, -0.0273,  0.2127,  0.5387
 0.9772,  0.1595, -0.2397,  0.1019,  0.4907,  0.1611
 0.3385, -0.4702, -0.8673, -0.2598,  0.2594,  0.2270
-0.8669, -0.4794,  0.6095, -0.6131,  0.2789,  0.4700
 0.0493,  0.8496, -0.4734, -0.8681,  0.4701,  0.3516
 0.8639, -0.9721, -0.5313,  0.2336,  0.8980,  0.1412
 0.9004,  0.1133,  0.8312,  0.2831, -0.2200,  0.1782
 0.0991,  0.8524,  0.8375, -0.2102,  0.9265,  0.2150
-0.6521, -0.7473, -0.7298,  0.0113, -0.9570,  0.7422
 0.6190, -0.3105,  0.8802,  0.1640,  0.7577,  0.1056
 0.6895,  0.8108, -0.0802,  0.0927,  0.5972,  0.2214
 0.1982, -0.9689,  0.1870, -0.1326,  0.6147,  0.1310
-0.3695,  0.7858,  0.1557, -0.6320,  0.5759,  0.3773
-0.1596,  0.3581,  0.8372, -0.9992,  0.9535,  0.2071
-0.2468,  0.9476,  0.2094,  0.6577,  0.1494,  0.4132
 0.1737,  0.5000,  0.7166,  0.5102,  0.3961,  0.2611
 0.7290, -0.3546,  0.3416, -0.0983, -0.2358,  0.1332
-0.3652,  0.2438, -0.1395,  0.9476,  0.3556,  0.4170
-0.6029, -0.1466, -0.3133,  0.5953,  0.7600,  0.4334
-0.4596, -0.4953,  0.7098,  0.0554,  0.6043,  0.2775
 0.1450,  0.4663,  0.0380,  0.5418,  0.1377,  0.2931
-0.8636, -0.2442, -0.8407,  0.9656, -0.6368,  0.7429
 0.6237,  0.7499,  0.3768,  0.1390, -0.6781,  0.2185
-0.5499,  0.1850, -0.3755,  0.8326,  0.8193,  0.4399
-0.4858, -0.7782, -0.6141, -0.0008,  0.4572,  0.4197
 0.7033, -0.1683,  0.2334, -0.5327, -0.7961,  0.1776
 0.0317, -0.0457, -0.6947,  0.2436,  0.0880,  0.3345
 0.5031, -0.5559,  0.0387,  0.5706, -0.9553,  0.3107
-0.3513,  0.7458,  0.6894,  0.0769,  0.7332,  0.3170
 0.2205,  0.5992, -0.9309,  0.5405,  0.4635,  0.3532
-0.4806, -0.4859,  0.2646, -0.3094,  0.5932,  0.3202
 0.9809, -0.3995, -0.7140,  0.8026,  0.0831,  0.1600
 0.9495,  0.2732,  0.9878,  0.0921,  0.0529,  0.1289
-0.9476, -0.6792,  0.4913, -0.9392, -0.2669,  0.5966
 0.7247,  0.3854,  0.3819, -0.6227, -0.1162,  0.1550
-0.5922, -0.5045, -0.4757,  0.5003, -0.0860,  0.5863
-0.8861,  0.0170, -0.5761,  0.5972, -0.4053,  0.7301
 0.6877, -0.2380,  0.4997,  0.0223,  0.0819,  0.1404
 0.9189,  0.6079, -0.9354,  0.4188, -0.0700,  0.1907
-0.1428, -0.7820,  0.2676,  0.6059,  0.3936,  0.2790
 0.5324, -0.3151,  0.6917, -0.1425,  0.6480,  0.1071
-0.8432, -0.9633, -0.8666, -0.0828, -0.7733,  0.7784
-0.9444,  0.5097, -0.2103,  0.4939, -0.0952,  0.6787
-0.0520,  0.6063, -0.1952,  0.8094, -0.9259,  0.4836
 0.5477, -0.7487,  0.2370, -0.9793,  0.0773,  0.1241
 0.2450,  0.8116,  0.9799,  0.4222,  0.4636,  0.2355
 0.8186, -0.1983, -0.5003, -0.6531, -0.7611,  0.1511
-0.4714,  0.6382, -0.3788,  0.9648, -0.4667,  0.5950
 0.0673, -0.3711,  0.8215, -0.2669, -0.1328,  0.2677
-0.9381,  0.4338,  0.7820, -0.9454,  0.0441,  0.5518
-0.3480,  0.7190,  0.1170,  0.3805, -0.0943,  0.4724
-0.9813,  0.1535, -0.3771,  0.0345,  0.8328,  0.5438
-0.1471, -0.5052, -0.2574,  0.8637,  0.8737,  0.3042
-0.5454, -0.3712, -0.6505,  0.2142, -0.1728,  0.5783
 0.6327, -0.6297,  0.4038, -0.5193,  0.1484,  0.1153
-0.5424,  0.3282, -0.0055,  0.0380, -0.6506,  0.6613
 0.1414,  0.9935,  0.6337,  0.1887,  0.9520,  0.2540
-0.9351, -0.8128, -0.8693, -0.0965, -0.2491,  0.7353
 0.9507, -0.6640,  0.9456,  0.5349,  0.6485,  0.1059
-0.0462, -0.9737, -0.2940, -0.0159,  0.4602,  0.2606
-0.0627, -0.0852, -0.7247, -0.9782,  0.5166,  0.2977
 0.0478,  0.5098, -0.0723, -0.7504, -0.3750,  0.3335
 0.0090,  0.3477,  0.5403, -0.7393, -0.9542,  0.4415
-0.9748,  0.3449,  0.3736, -0.1015,  0.8296,  0.4358
 0.2887, -0.9895, -0.0311,  0.7186,  0.6608,  0.2057
 0.1570, -0.4518,  0.1211,  0.3435, -0.2951,  0.3244
 0.7117, -0.6099,  0.4946, -0.4208,  0.5476,  0.1096
-0.2929, -0.5726,  0.5346, -0.3827,  0.4665,  0.2465
 0.4889, -0.5572, -0.5718, -0.6021, -0.7150,  0.2163
-0.7782,  0.3491,  0.5996, -0.8389, -0.5366,  0.6516
-0.5847,  0.8347,  0.4226,  0.1078, -0.3910,  0.6134
 0.8469,  0.4121, -0.0439, -0.7476,  0.9521,  0.1571
-0.6803, -0.5948, -0.1376, -0.1916, -0.7065,  0.7156
 0.2878,  0.5086, -0.5785,  0.2019,  0.4979,  0.2980
 0.2764,  0.1943, -0.4090,  0.4632,  0.8906,  0.2960
-0.8877,  0.6705, -0.6155, -0.2098, -0.3998,  0.7107
-0.8398,  0.8093, -0.2597,  0.0614, -0.0118,  0.6502
-0.8476,  0.0158, -0.4769, -0.2859, -0.7839,  0.7715
 0.5751, -0.7868,  0.9714, -0.6457,  0.1448,  0.1175
 0.4802, -0.7001,  0.1022, -0.5668,  0.5184,  0.1090
 0.4458, -0.6469,  0.7239, -0.9604,  0.7205,  0.0779
 0.5175,  0.4339,  0.9747, -0.4438, -0.9924,  0.2879
 0.8678,  0.7158,  0.4577,  0.0334,  0.4139,  0.1678
 0.5406,  0.5012,  0.2264, -0.1963,  0.3946,  0.2088
-0.9938,  0.5498,  0.7928, -0.5214, -0.7585,  0.7687
 0.7661,  0.0863, -0.4266, -0.7233, -0.4197,  0.1466
 0.2277, -0.3517, -0.0853, -0.1118,  0.6563,  0.1767
 0.3499, -0.5570, -0.0655, -0.3705,  0.2537,  0.1632
 0.7547, -0.1046,  0.5689, -0.0861,  0.3125,  0.1257
 0.8186,  0.2110,  0.5335,  0.0094, -0.0039,  0.1391
 0.6858, -0.8644,  0.1465,  0.8855,  0.0357,  0.1845
-0.4967,  0.4015,  0.0805,  0.8977,  0.2487,  0.4663
 0.6760, -0.9841,  0.9787, -0.8446, -0.3557,  0.1509
-0.1203, -0.4885,  0.6054, -0.0443, -0.7313,  0.4854
 0.8557,  0.7919, -0.0169,  0.7134, -0.1628,  0.2002
 0.0115, -0.6209,  0.9300, -0.4116, -0.7931,  0.4052
-0.7114, -0.9718,  0.4319,  0.1290,  0.5892,  0.3661
 0.3915,  0.5557, -0.1870,  0.2955, -0.6404,  0.2954
-0.3564, -0.6548, -0.1827, -0.5172, -0.1862,  0.4622
 0.2392, -0.4959,  0.5857, -0.1341, -0.2850,  0.2470
-0.3394,  0.3947, -0.4627,  0.6166, -0.4094,  0.5325
 0.7107,  0.7768, -0.6312,  0.1707,  0.7964,  0.2757
-0.1078,  0.8437, -0.4420,  0.2177,  0.3649,  0.4028
-0.3139,  0.5595, -0.6505, -0.3161, -0.7108,  0.5546
 0.4335,  0.3986,  0.3770, -0.4932,  0.3847,  0.1810
-0.2562, -0.2894, -0.8847,  0.2633,  0.4146,  0.4036
 0.2272,  0.2966, -0.6601, -0.7011,  0.0284,  0.2778
-0.0743, -0.1421, -0.0054, -0.6770, -0.3151,  0.3597
-0.4762,  0.6891,  0.6007, -0.1467,  0.2140,  0.4266
-0.4061,  0.7193,  0.3432,  0.2669, -0.7505,  0.6147
-0.0588,  0.9731,  0.8966,  0.2902, -0.6966,  0.4955
-0.0627, -0.1439,  0.1985,  0.6999,  0.5022,  0.3077
 0.1587,  0.8494, -0.8705,  0.9827, -0.8940,  0.4263
-0.7850,  0.2473, -0.9040, -0.4308, -0.8779,  0.7199
 0.4070,  0.3369, -0.2428, -0.6236,  0.4940,  0.2215
-0.0242,  0.0513, -0.9430,  0.2885, -0.2987,  0.3947
-0.5416, -0.1322, -0.2351, -0.0604,  0.9590,  0.3683
 0.1055,  0.7783, -0.2901, -0.5090,  0.8220,  0.2984
-0.9129,  0.9015,  0.1128, -0.2473,  0.9901,  0.4776
-0.9378,  0.1424, -0.6391,  0.2619,  0.9618,  0.5368
 0.7498, -0.0963,  0.4169,  0.5549, -0.0103,  0.1614
-0.2612, -0.7156,  0.4538, -0.0460, -0.1022,  0.3717
 0.7720,  0.0552, -0.1818, -0.4622, -0.8560,  0.1685
-0.4177,  0.0070,  0.9319, -0.7812,  0.3461,  0.3052
-0.0001,  0.5542, -0.7128, -0.8336, -0.2016,  0.3803
 0.5356, -0.4194, -0.5662, -0.9666, -0.2027,  0.1776
-0.2378,  0.3187, -0.8582, -0.6948, -0.9668,  0.5474
-0.1947, -0.3579,  0.1158,  0.9869,  0.6690,  0.2992
 0.3992,  0.8365, -0.9205, -0.8593, -0.0520,  0.3154
-0.0209,  0.0793,  0.7905, -0.1067,  0.7541,  0.1864
-0.4928, -0.4524, -0.3433,  0.0951, -0.5597,  0.6261
-0.8118,  0.7404, -0.5263, -0.2280,  0.1431,  0.6349
 0.0516, -0.8480,  0.7483,  0.9023,  0.6250,  0.1959
-0.3212,  0.1093,  0.9488, -0.3766,  0.3376,  0.2735
-0.3481,  0.5490, -0.3484,  0.7797,  0.5034,  0.4379
-0.5785, -0.9170, -0.3563, -0.9258,  0.3877,  0.4121
 0.3407, -0.1391,  0.5356,  0.0720, -0.9203,  0.3458
-0.3287, -0.8954,  0.2102,  0.0241,  0.2349,  0.3247
-0.1353,  0.6954, -0.0919, -0.9692,  0.7461,  0.3338
 0.9036, -0.8982, -0.5299, -0.8733, -0.1567,  0.1187
 0.7277, -0.8368, -0.0538, -0.7489,  0.5458,  0.0830
 0.9049,  0.8878,  0.2279,  0.9470, -0.3103,  0.2194
 0.7957, -0.1308, -0.5284,  0.8817,  0.3684,  0.2172
 0.4647, -0.4931,  0.2010,  0.6292, -0.8918,  0.3371
-0.7390,  0.6849,  0.2367,  0.0626, -0.5034,  0.7039
-0.1567, -0.8711,  0.7940, -0.5932,  0.6525,  0.1710
 0.7635, -0.0265,  0.1969,  0.0545,  0.2496,  0.1445
 0.7675,  0.1354, -0.7698, -0.5460,  0.1920,  0.1728
-0.5211, -0.7372, -0.6763,  0.6897,  0.2044,  0.5217
 0.1913,  0.1980,  0.2314, -0.8816,  0.5006,  0.1998
 0.8964,  0.0694, -0.6149,  0.5059, -0.9854,  0.1825
 0.1767,  0.7104,  0.2093,  0.6452,  0.7590,  0.2832
-0.3580, -0.7541,  0.4426, -0.1193, -0.7465,  0.5657
-0.5996,  0.5766, -0.9758, -0.3933, -0.9572,  0.6800
 0.9950,  0.1641, -0.4132,  0.8579,  0.0142,  0.2003
-0.4717, -0.3894, -0.2567, -0.5111,  0.1691,  0.4266
 0.3917, -0.8561,  0.9422,  0.5061,  0.6123,  0.1212
-0.0366, -0.1087,  0.3449, -0.1025,  0.4086,  0.2475
 0.3633,  0.3943,  0.2372, -0.6980,  0.5216,  0.1925
-0.5325, -0.6466, -0.2178, -0.3589,  0.6310,  0.3568
 0.2271,  0.5200, -0.1447, -0.8011, -0.7699,  0.3128
 0.6415,  0.1993,  0.3777, -0.0178, -0.8237,  0.2181
-0.5298, -0.0768, -0.6028, -0.9490,  0.4588,  0.4356
 0.6870, -0.1431,  0.7294,  0.3141,  0.1621,  0.1632
-0.5985,  0.0591,  0.7889, -0.3900,  0.7419,  0.2945
 0.3661,  0.7984, -0.8486,  0.7572, -0.6183,  0.3449
 0.6995,  0.3342, -0.3113, -0.6972,  0.2707,  0.1712
 0.2565,  0.9126,  0.1798, -0.6043, -0.1413,  0.2893
-0.3265,  0.9839, -0.2395,  0.9854,  0.0376,  0.4770
 0.2690, -0.1722,  0.9818,  0.8599, -0.7015,  0.3954
-0.2102, -0.0768,  0.1219,  0.5607, -0.0256,  0.3949
 0.8216, -0.9555,  0.6422, -0.6231,  0.3715,  0.0801
-0.2896,  0.9484, -0.7545, -0.6249,  0.7789,  0.4370
-0.9985, -0.5448, -0.7092, -0.5931,  0.7926,  0.5402

Test data:

# synthetic_test_40.txt
#
 0.7462,  0.4006, -0.0590,  0.6543, -0.0083,  0.1935
 0.8495, -0.2260, -0.0142, -0.4911,  0.7699,  0.1078
-0.2335, -0.4049,  0.4352, -0.6183, -0.7636,  0.5088
 0.1810, -0.5142,  0.2465,  0.2767, -0.3449,  0.3136
-0.8650,  0.7611, -0.0801,  0.5277, -0.4922,  0.7140
-0.2358, -0.7466, -0.5115, -0.8413, -0.3943,  0.4533
 0.4834,  0.2300,  0.3448, -0.9832,  0.3568,  0.1360
-0.6502, -0.6300,  0.6885,  0.9652,  0.8275,  0.3046
-0.3053,  0.5604,  0.0929,  0.6329, -0.0325,  0.4756
-0.7995,  0.0740, -0.2680,  0.2086,  0.9176,  0.4565
-0.2144, -0.2141,  0.5813,  0.2902, -0.2122,  0.4119
-0.7278, -0.0987, -0.3312, -0.5641,  0.8515,  0.4438
 0.3793,  0.1976,  0.4933,  0.0839,  0.4011,  0.1905
-0.8568,  0.9573, -0.5272,  0.3212, -0.8207,  0.7415
-0.5785,  0.0056, -0.7901, -0.2223,  0.0760,  0.5551
 0.0735, -0.2188,  0.3925,  0.3570,  0.3746,  0.2191
 0.1230, -0.2838,  0.2262,  0.8715,  0.1938,  0.2878
 0.4792, -0.9248,  0.5295,  0.0366, -0.9894,  0.3149
-0.4456,  0.0697,  0.5359, -0.8938,  0.0981,  0.3879
 0.8629, -0.8505, -0.4464,  0.8385,  0.5300,  0.1769
 0.1995,  0.6659,  0.7921,  0.9454,  0.9970,  0.2330
-0.0249, -0.3066, -0.2927, -0.4923,  0.8220,  0.2437
 0.4513, -0.9481, -0.0770, -0.4374, -0.9421,  0.2879
-0.3405,  0.5931, -0.3507, -0.3842,  0.8562,  0.3987
 0.9538,  0.0471,  0.9039,  0.7760,  0.0361,  0.1706
-0.0887,  0.2104,  0.9808,  0.5478, -0.3314,  0.4128
-0.8220, -0.6302,  0.0537, -0.1658,  0.6013,  0.4306
-0.4123, -0.2880,  0.9074, -0.0461, -0.4435,  0.5144
 0.0060,  0.2867, -0.7775,  0.5161,  0.7039,  0.3599
-0.7968, -0.5484,  0.9426, -0.4308,  0.8148,  0.2979
 0.7811,  0.8450, -0.6877,  0.7594,  0.2640,  0.2362
-0.6802, -0.1113, -0.8325, -0.6694, -0.6056,  0.6544
 0.3821,  0.1476,  0.7466, -0.5107,  0.2592,  0.1648
 0.7265,  0.9683, -0.9803, -0.4943, -0.5523,  0.2454
-0.9049, -0.9797, -0.0196, -0.9090, -0.4433,  0.6447
-0.4607,  0.1811, -0.2389,  0.4050, -0.0078,  0.5229
 0.2664, -0.2932, -0.4259, -0.7336,  0.8742,  0.1834
-0.4507,  0.1029, -0.6294, -0.1158, -0.6294,  0.6081
 0.8948, -0.0124,  0.9278,  0.2899, -0.0314,  0.1534
-0.1323, -0.8813, -0.0146, -0.0697,  0.6135,  0.2386
Posted in JavaScript, Machine Learning | Leave a comment