One Sunday morning, I figured I’d take look at some examples of training a logistic regression binary classification model using stochastic gradient descent with batch training. I found a few examples, but none using the C# language which is the language most of my colleagues at work use. So, I decided to put together an example using raw C#.

For my demo, I used one of my standard synthetic datasets. The goal is to predict the sex of a person (male = 0, female = 1) from age, state of residence (Michigan, Nebraska, Oklahoma), income, and political leaning (conservative, moderate, liberal).

The raw data looks like:

F   24   michigan   29500.00   liberal
M   39   oklahoma   51200.00   moderate
F   63   nebraska   75800.00   conservative
M   36   michigan   44500.00   moderate
F   27   nebraska   28600.00   liberal
. . .

The encoded and normalized comma-delimited data looks like:

1, 0.24, 1, 0, 0, 0.2950, 0, 0, 1
0, 0.39, 0, 0, 1, 0.5120, 0, 1, 0
1, 0.63, 0, 1, 0, 0.7580, 1, 0, 0
0, 0.36, 1, 0, 0, 0.4450, 0, 1, 0
1, 0.27, 0, 1, 0, 0.2860, 0, 0, 1
. . .

Each data item is a person. The fields are sex (male = 0, female = 1), age (divided by 100), state (Michigan = 100, Nebraska = 010, Oklahoma – 001), income (divided by $100,000) and political leaning (conservative = 100, moderate = 010, liberal = 001). There are 200 training items and 40 test items.

Because I’m very familiar with logistic regression, and have implemented it from scratch many times, most of my time was spent implementing helper functions to load the training and test data into memory.

For my architecture, I used a straightforward design where the model is just a separate weights array and a bias variable — instead of using a probably better design with some sort of LogisticRegression class. I’ll code up a demo using that approach when I get a chance.

The results of using batch training (I used batch size = 10) were what I expected — pretty much the same as the results I get when I use so-called online training, which just means a batch size of 1. Batch training often works better than online training when the number of training items is large, say, more than 1,000.

A feature that I didn’t use in my demo was regularization. Regularization limits the magnitudes of the model weights in order to prevent model overfitting. The two main types of regularization are L2 and weight decay. In a very surprising result, for logistic regression, L2 and weight decay are mathematically equivalent but the implementations look very different. When I refactor this demo to an OOP design, I’ll probably add weight decay regularization (much easier to implement than L2).

A nice way to spend some time on a Sunday morning.

One of the most important principles of software design is the idea of levels of abstraction. My demo has a medium level of abstraction. I could have used a higher level of abstraction by encapsulating the logistic regression functionality into a LogisticRegression class.

I often think about levels of abstraction in art too. My liking, or not liking, of an illustration depends to some extent on the level of abstraction used. Here are three I like. Left: By artist Hans Jochem Bakker. Low level of abstraction, high level of detail. Center: By artist Barry Wilson. Medium level of abstraction and detail. Right: Anonymous stock art. High level of abstraction, low level of detail.

Demo program. Replace “lt” (less than), “gt”, “lte”, “gte”, “and” with Boolean operator symbols (my blog editor often chokes on symbols).

using System;
using System.IO;
// .NET 8

namespace PeopleLogisticRegressionBatch
{
  internal class LogisticRegressionProgram
  {
    static void Main(string[] args)
    {
      Console.WriteLine("\nLogistic regression with batch" +
        " training using raw C# demo ");

      // 0. get ready
      Random rnd = new Random(0);

      // 1. load train data
      Console.WriteLine("\nLoading People data ");
      string fn = "..\\..\\..\\Data\\people_train.txt";
      double[][] allTrain = MatLoad(fn, 200,
        [0,1,2,3,4,5,6,7,8], ',');
      double[][] xTrain = Extract(allTrain,
        [1,2,3,4,5,6,7,8]);
      double[] yTrain = MatToVec(Extract(allTrain, [0]));

      // 1b. load test data
      fn = "..\\..\\..\\Data\\people_test.txt";
      double[][] allTest = MatLoad(fn, 40,
        [0,1,2,3,4,5,6,7,8], ',');
      double[][] xTest = Extract(allTest,
        [1,2,3,4,5,6,7,8]);
      double[] yTest = MatToVec(Extract(allTest, [0]));

      // 2. create model
      Console.WriteLine("\nCreating logistic " +
        "regression model ");
      // the model is really the wts and the bias
      double[] wts = new double[8];
      double lo = -0.01; double hi = 0.01;
      for (int i = 0; i "lt" wts.Length; ++i)
        wts[i] = (hi - lo) * rnd.NextDouble() + lo;
      double bias = 0.0;

      // 2b. create grads to accumulate for batch training
      double[] grads = new double[8];
      double biasGrad = 0.0;

      // 3. train model
      double lrnRate = 0.01;
      int maxEpochs = 10000;
      int batSize = 10;
      Console.WriteLine("\nSetting batch size = " + batSize);
      Console.WriteLine("Training using SGD with lrnRate = " +
        lrnRate);

      int n = xTrain.Length;  // like 200
      int batchesPerEpoch = n / batSize;  // like 20
      int freq = maxEpochs / 10;  // to show progress

      int[] indices = new int[200];
      for (int i = 0; i "lt" indices.Length; ++i)
        indices[i] = i;

      for (int epoch = 0; epoch "lt" maxEpochs; ++epoch)
      {
        Shuffle(indices, rnd); // new order each epoch

        int ptr = 0;  // points into indices
        for (int batIdx = 0; batIdx "lt" batchesPerEpoch;
          ++batIdx) // 0, 1, . . 19
        {
          for (int i = 0; i "lt" batSize; ++i) // 0 . . 9
          {
            int ii = indices[ptr++];  // compute output
            double[] x = xTrain[ii];
            double y = yTrain[ii];
            double p = ComputeOutput(wts, bias, x);
            // accumulate gradients
            for (int j = 0; j "lt" wts.Length; ++j)
              grads[j] += x[j] * (p - y);
            biasGrad += 1.0 * (p - y);
          } // end of curr batch

          // time to update weights
          for (int j = 0; j "lt" wts.Length; ++j)
            wts[j] += -1 * lrnRate * grads[j];
          bias += -1 * lrnRate * biasGrad;

          // zero-out grads for next batch
          for (int j = 0; j "lt" wts.Length; ++j)
            grads[j] = 0.0;
          biasGrad = 0.0; 
        } // batches

        if (epoch % freq == 0)  // progress every few epochs
        {
          double loss = MSELoss(wts, bias, xTrain, yTrain);
          string s1 = "epoch = " +
            epoch.ToString().PadLeft(4) + " | ";
          string s2 = " loss = " + loss.ToString("F4");
          Console.WriteLine(s1 + s2);
    }
      } // for epochs
      Console.WriteLine("Done ");

      // 4. evaluate model 
      Console.WriteLine("\nEvaluating trained model ");
      double accTrain = Accuracy(wts, bias, xTrain, yTrain);
      Console.WriteLine("Accuracy on train data: " +
        accTrain.ToString("F4"));
      double accTest = Accuracy(wts, bias, xTest, yTest);
      Console.WriteLine("Accuracy on test data: " +
        accTest.ToString("F4"));

      // 5. use model
      Console.WriteLine("\nSetting x = [33, Nebraska, " +
        "$50,000, moderate]: ");
      double[] inpt = new double[] { 0.33, 0, 1, 0,
        0.50000, 0, 1, 0 };
      double pVal = ComputeOutput(wts, bias, inpt);
      Console.WriteLine("p-val = " + pVal.ToString("F4"));
      if (pVal "lt" 0.5)
        Console.WriteLine("class 0 (male) ");
      else
        Console.WriteLine("class 1 (female) ");

      // 6.TODO: save wts and bias to file

      Console.WriteLine("\nEnd logistic regression" +
        " raw C# demo ");
      Console.ReadLine();
    } // Main

    public static void Shuffle(int[] arr, Random rnd)
    {
      // Fisher-Yates algorithm
      int n = arr.Length;
      for (int i = 0; i "lt" n; ++i)
      {
        int ri = rnd.Next(i, n);  // random index
        int tmp = arr[ri];
        arr[ri] = arr[i];
        arr[i] = tmp;
      }
    }

    public static double ComputeOutput(double[] w, double b,
      double[] x)
    {
      double z = 0.0;
      for (int i = 0; i "lt" w.Length; ++i)
      {
        z += w[i] * x[i];
      }
      z += b;
      double p = 1.0 / (1.0 + Math.Exp(-z));
      return p;
    }

    public static double Accuracy(double[] w, double b,
      double[][] dataX, double[] dataY)
    {
      int nCorrect = 0; int nWrong = 0;
      for (int i = 0; i "lt" dataX.Length; ++i)
      {
        double[] x = dataX[i];
        double y = dataY[i];
        double p = ComputeOutput(w, b, x);
        if ((y == 0.0 "and" p "lt" 0.5) || 
          (y == 1.0 "and" p "gte" 0.5))
          nCorrect += 1;
        else
          nWrong += 1;
      }
      double acc = (nCorrect * 1.0) / (nCorrect + nWrong);
      return acc;
    }

    public static double MSELoss(double[] w, double b,
      double[][] dataX, double[] dataY)
    {
      double sum = 0.0;
      for (int i = 0; i "lt" dataX.Length; ++i)
      {
        double[] x = dataX[i];
        double y = dataY[i];
        double p = ComputeOutput(w, b, x);
        sum += (y - p) * (y - p);
      }
      double mse = sum / dataX.Length;
      return mse;
    }

    public static double[][] MatLoad(string fn, int nRows,
      int[] cols, char sep)
    {
      int nCols = cols.Length;
      double[][] result = MatCreate(nRows, nCols);
      string line = "";
      string[] tokens = null;
      FileStream ifs = new FileStream(fn, FileMode.Open);
      StreamReader sr = new StreamReader(ifs);

      int i = 0;
      while ((line = sr.ReadLine()) != null)
      {
        if (line.StartsWith("#") == true)
          continue;
        tokens = line.Split(sep);
        for (int j = 0; j "lt" nCols; ++j)
        {
          int k = cols[j];  // into tokens
          result[i][j] = double.Parse(tokens[k]);
        }
        ++i;
      }
      sr.Close(); ifs.Close();
      return result;
    }

    public static double[][] MatCreate(int rows, int cols)
    {
      double[][] result = new double[rows][];
      for (int i = 0; i "lt" rows; ++i)
        result[i] = new double[cols];
      return result;
    }

    public static void MatShow(double[][] mat, int dec,
      int wid)
    {
      int nRows = mat.Length;
      int nCols = mat[0].Length;
      for (int i = 0; i "lt" nRows; ++i)
      {
        for (int j = 0; j "lt" nCols; ++j)
        {
          double x = mat[i][j];
          Console.Write(x.ToString("F" + dec).PadLeft(wid));
        }
        Console.WriteLine("");
        // Utils.VecShow(mat[i], dec, wid);
      }
    }

    public static double[][] Extract(double[][] mat,
      int[] cols)
    {
      int nRows = mat.Length;
      int nCols = cols.Length;
      double[][] result = MatCreate(nRows, nCols);
      for (int i = 0; i "lt" nRows; ++i)
      {
        for (int j = 0; j "lt" nCols; ++j)  // idx into src cols
        {
          int srcCol = cols[j];
          int destCol = j;
          result[i][destCol] = mat[i][srcCol];
        }
      }
      return result;
    }

    public static double[] MatToVec(double[][] mat)
    {
      int nRows = mat.Length;
      int nCols = mat[0].Length;
      int nCells = nRows * nCols;
      double[] result = new double[nCells];
      int k = 0;
      for (int i = 0; i "lt" nRows; ++i)
        for (int j = 0; j "lt" nCols; ++j)
          result[k++] = mat[i][j];
      return result;
    }

  } // Program

} // ns

Training data:

# people_train.txt
# sex (0 = male, 1 = female) - dependent variable
# age, state (michigan, nebraska, oklahoma), income,
# politics type (conservative, moderate, liberal)
#
1, 0.24, 1, 0, 0, 0.2950, 0, 0, 1
0, 0.39, 0, 0, 1, 0.5120, 0, 1, 0
1, 0.63, 0, 1, 0, 0.7580, 1, 0, 0
0, 0.36, 1, 0, 0, 0.4450, 0, 1, 0
1, 0.27, 0, 1, 0, 0.2860, 0, 0, 1
1, 0.50, 0, 1, 0, 0.5650, 0, 1, 0
1, 0.50, 0, 0, 1, 0.5500, 0, 1, 0
0, 0.19, 0, 0, 1, 0.3270, 1, 0, 0
1, 0.22, 0, 1, 0, 0.2770, 0, 1, 0
0, 0.39, 0, 0, 1, 0.4710, 0, 0, 1
1, 0.34, 1, 0, 0, 0.3940, 0, 1, 0
0, 0.22, 1, 0, 0, 0.3350, 1, 0, 0
1, 0.35, 0, 0, 1, 0.3520, 0, 0, 1
0, 0.33, 0, 1, 0, 0.4640, 0, 1, 0
1, 0.45, 0, 1, 0, 0.5410, 0, 1, 0
1, 0.42, 0, 1, 0, 0.5070, 0, 1, 0
0, 0.33, 0, 1, 0, 0.4680, 0, 1, 0
1, 0.25, 0, 0, 1, 0.3000, 0, 1, 0
0, 0.31, 0, 1, 0, 0.4640, 1, 0, 0
1, 0.27, 1, 0, 0, 0.3250, 0, 0, 1
1, 0.48, 1, 0, 0, 0.5400, 0, 1, 0
0, 0.64, 0, 1, 0, 0.7130, 0, 0, 1
1, 0.61, 0, 1, 0, 0.7240, 1, 0, 0
1, 0.54, 0, 0, 1, 0.6100, 1, 0, 0
1, 0.29, 1, 0, 0, 0.3630, 1, 0, 0
1, 0.50, 0, 0, 1, 0.5500, 0, 1, 0
1, 0.55, 0, 0, 1, 0.6250, 1, 0, 0
1, 0.40, 1, 0, 0, 0.5240, 1, 0, 0
1, 0.22, 1, 0, 0, 0.2360, 0, 0, 1
1, 0.68, 0, 1, 0, 0.7840, 1, 0, 0
0, 0.60, 1, 0, 0, 0.7170, 0, 0, 1
0, 0.34, 0, 0, 1, 0.4650, 0, 1, 0
0, 0.25, 0, 0, 1, 0.3710, 1, 0, 0
0, 0.31, 0, 1, 0, 0.4890, 0, 1, 0
1, 0.43, 0, 0, 1, 0.4800, 0, 1, 0
1, 0.58, 0, 1, 0, 0.6540, 0, 0, 1
0, 0.55, 0, 1, 0, 0.6070, 0, 0, 1
0, 0.43, 0, 1, 0, 0.5110, 0, 1, 0
0, 0.43, 0, 0, 1, 0.5320, 0, 1, 0
0, 0.21, 1, 0, 0, 0.3720, 1, 0, 0
1, 0.55, 0, 0, 1, 0.6460, 1, 0, 0
1, 0.64, 0, 1, 0, 0.7480, 1, 0, 0
0, 0.41, 1, 0, 0, 0.5880, 0, 1, 0
1, 0.64, 0, 0, 1, 0.7270, 1, 0, 0
0, 0.56, 0, 0, 1, 0.6660, 0, 0, 1
1, 0.31, 0, 0, 1, 0.3600, 0, 1, 0
0, 0.65, 0, 0, 1, 0.7010, 0, 0, 1
1, 0.55, 0, 0, 1, 0.6430, 1, 0, 0
0, 0.25, 1, 0, 0, 0.4030, 1, 0, 0
1, 0.46, 0, 0, 1, 0.5100, 0, 1, 0
0, 0.36, 1, 0, 0, 0.5350, 1, 0, 0
1, 0.52, 0, 1, 0, 0.5810, 0, 1, 0
1, 0.61, 0, 0, 1, 0.6790, 1, 0, 0
1, 0.57, 0, 0, 1, 0.6570, 1, 0, 0
0, 0.46, 0, 1, 0, 0.5260, 0, 1, 0
0, 0.62, 1, 0, 0, 0.6680, 0, 0, 1
1, 0.55, 0, 0, 1, 0.6270, 1, 0, 0
0, 0.22, 0, 0, 1, 0.2770, 0, 1, 0
0, 0.50, 1, 0, 0, 0.6290, 1, 0, 0
0, 0.32, 0, 1, 0, 0.4180, 0, 1, 0
0, 0.21, 0, 0, 1, 0.3560, 1, 0, 0
1, 0.44, 0, 1, 0, 0.5200, 0, 1, 0
1, 0.46, 0, 1, 0, 0.5170, 0, 1, 0
1, 0.62, 0, 1, 0, 0.6970, 1, 0, 0
1, 0.57, 0, 1, 0, 0.6640, 1, 0, 0
0, 0.67, 0, 0, 1, 0.7580, 0, 0, 1
1, 0.29, 1, 0, 0, 0.3430, 0, 0, 1
1, 0.53, 1, 0, 0, 0.6010, 1, 0, 0
0, 0.44, 1, 0, 0, 0.5480, 0, 1, 0
1, 0.46, 0, 1, 0, 0.5230, 0, 1, 0
0, 0.20, 0, 1, 0, 0.3010, 0, 1, 0
0, 0.38, 1, 0, 0, 0.5350, 0, 1, 0
1, 0.50, 0, 1, 0, 0.5860, 0, 1, 0
1, 0.33, 0, 1, 0, 0.4250, 0, 1, 0
0, 0.33, 0, 1, 0, 0.3930, 0, 1, 0
1, 0.26, 0, 1, 0, 0.4040, 1, 0, 0
1, 0.58, 1, 0, 0, 0.7070, 1, 0, 0
1, 0.43, 0, 0, 1, 0.4800, 0, 1, 0
0, 0.46, 1, 0, 0, 0.6440, 1, 0, 0
1, 0.60, 1, 0, 0, 0.7170, 1, 0, 0
0, 0.42, 1, 0, 0, 0.4890, 0, 1, 0
0, 0.56, 0, 0, 1, 0.5640, 0, 0, 1
0, 0.62, 0, 1, 0, 0.6630, 0, 0, 1
0, 0.50, 1, 0, 0, 0.6480, 0, 1, 0
1, 0.47, 0, 0, 1, 0.5200, 0, 1, 0
0, 0.67, 0, 1, 0, 0.8040, 0, 0, 1
0, 0.40, 0, 0, 1, 0.5040, 0, 1, 0
1, 0.42, 0, 1, 0, 0.4840, 0, 1, 0
1, 0.64, 1, 0, 0, 0.7200, 1, 0, 0
0, 0.47, 1, 0, 0, 0.5870, 0, 0, 1
1, 0.45, 0, 1, 0, 0.5280, 0, 1, 0
0, 0.25, 0, 0, 1, 0.4090, 1, 0, 0
1, 0.38, 1, 0, 0, 0.4840, 1, 0, 0
1, 0.55, 0, 0, 1, 0.6000, 0, 1, 0
0, 0.44, 1, 0, 0, 0.6060, 0, 1, 0
1, 0.33, 1, 0, 0, 0.4100, 0, 1, 0
1, 0.34, 0, 0, 1, 0.3900, 0, 1, 0
1, 0.27, 0, 1, 0, 0.3370, 0, 0, 1
1, 0.32, 0, 1, 0, 0.4070, 0, 1, 0
1, 0.42, 0, 0, 1, 0.4700, 0, 1, 0
0, 0.24, 0, 0, 1, 0.4030, 1, 0, 0
1, 0.42, 0, 1, 0, 0.5030, 0, 1, 0
1, 0.25, 0, 0, 1, 0.2800, 0, 0, 1
1, 0.51, 0, 1, 0, 0.5800, 0, 1, 0
0, 0.55, 0, 1, 0, 0.6350, 0, 0, 1
1, 0.44, 1, 0, 0, 0.4780, 0, 0, 1
0, 0.18, 1, 0, 0, 0.3980, 1, 0, 0
0, 0.67, 0, 1, 0, 0.7160, 0, 0, 1
1, 0.45, 0, 0, 1, 0.5000, 0, 1, 0
1, 0.48, 1, 0, 0, 0.5580, 0, 1, 0
0, 0.25, 0, 1, 0, 0.3900, 0, 1, 0
0, 0.67, 1, 0, 0, 0.7830, 0, 1, 0
1, 0.37, 0, 0, 1, 0.4200, 0, 1, 0
0, 0.32, 1, 0, 0, 0.4270, 0, 1, 0
1, 0.48, 1, 0, 0, 0.5700, 0, 1, 0
0, 0.66, 0, 0, 1, 0.7500, 0, 0, 1
1, 0.61, 1, 0, 0, 0.7000, 1, 0, 0
0, 0.58, 0, 0, 1, 0.6890, 0, 1, 0
1, 0.19, 1, 0, 0, 0.2400, 0, 0, 1
1, 0.38, 0, 0, 1, 0.4300, 0, 1, 0
0, 0.27, 1, 0, 0, 0.3640, 0, 1, 0
1, 0.42, 1, 0, 0, 0.4800, 0, 1, 0
1, 0.60, 1, 0, 0, 0.7130, 1, 0, 0
0, 0.27, 0, 0, 1, 0.3480, 1, 0, 0
1, 0.29, 0, 1, 0, 0.3710, 1, 0, 0
0, 0.43, 1, 0, 0, 0.5670, 0, 1, 0
1, 0.48, 1, 0, 0, 0.5670, 0, 1, 0
1, 0.27, 0, 0, 1, 0.2940, 0, 0, 1
0, 0.44, 1, 0, 0, 0.5520, 1, 0, 0
1, 0.23, 0, 1, 0, 0.2630, 0, 0, 1
0, 0.36, 0, 1, 0, 0.5300, 0, 0, 1
1, 0.64, 0, 0, 1, 0.7250, 1, 0, 0
1, 0.29, 0, 0, 1, 0.3000, 0, 0, 1
0, 0.33, 1, 0, 0, 0.4930, 0, 1, 0
0, 0.66, 0, 1, 0, 0.7500, 0, 0, 1
0, 0.21, 0, 0, 1, 0.3430, 1, 0, 0
1, 0.27, 1, 0, 0, 0.3270, 0, 0, 1
1, 0.29, 1, 0, 0, 0.3180, 0, 0, 1
0, 0.31, 1, 0, 0, 0.4860, 0, 1, 0
1, 0.36, 0, 0, 1, 0.4100, 0, 1, 0
1, 0.49, 0, 1, 0, 0.5570, 0, 1, 0
0, 0.28, 1, 0, 0, 0.3840, 1, 0, 0
0, 0.43, 0, 0, 1, 0.5660, 0, 1, 0
0, 0.46, 0, 1, 0, 0.5880, 0, 1, 0
1, 0.57, 1, 0, 0, 0.6980, 1, 0, 0
0, 0.52, 0, 0, 1, 0.5940, 0, 1, 0
0, 0.31, 0, 0, 1, 0.4350, 0, 1, 0
0, 0.55, 1, 0, 0, 0.6200, 0, 0, 1
1, 0.50, 1, 0, 0, 0.5640, 0, 1, 0
1, 0.48, 0, 1, 0, 0.5590, 0, 1, 0
0, 0.22, 0, 0, 1, 0.3450, 1, 0, 0
1, 0.59, 0, 0, 1, 0.6670, 1, 0, 0
1, 0.34, 1, 0, 0, 0.4280, 0, 0, 1
0, 0.64, 1, 0, 0, 0.7720, 0, 0, 1
1, 0.29, 0, 0, 1, 0.3350, 0, 0, 1
0, 0.34, 0, 1, 0, 0.4320, 0, 1, 0
0, 0.61, 1, 0, 0, 0.7500, 0, 0, 1
1, 0.64, 0, 0, 1, 0.7110, 1, 0, 0
0, 0.29, 1, 0, 0, 0.4130, 1, 0, 0
1, 0.63, 0, 1, 0, 0.7060, 1, 0, 0
0, 0.29, 0, 1, 0, 0.4000, 1, 0, 0
0, 0.51, 1, 0, 0, 0.6270, 0, 1, 0
0, 0.24, 0, 0, 1, 0.3770, 1, 0, 0
1, 0.48, 0, 1, 0, 0.5750, 0, 1, 0
1, 0.18, 1, 0, 0, 0.2740, 1, 0, 0
1, 0.18, 1, 0, 0, 0.2030, 0, 0, 1
1, 0.33, 0, 1, 0, 0.3820, 0, 0, 1
0, 0.20, 0, 0, 1, 0.3480, 1, 0, 0
1, 0.29, 0, 0, 1, 0.3300, 0, 0, 1
0, 0.44, 0, 0, 1, 0.6300, 1, 0, 0
0, 0.65, 0, 0, 1, 0.8180, 1, 0, 0
0, 0.56, 1, 0, 0, 0.6370, 0, 0, 1
0, 0.52, 0, 0, 1, 0.5840, 0, 1, 0
0, 0.29, 0, 1, 0, 0.4860, 1, 0, 0
0, 0.47, 0, 1, 0, 0.5890, 0, 1, 0
1, 0.68, 1, 0, 0, 0.7260, 0, 0, 1
1, 0.31, 0, 0, 1, 0.3600, 0, 1, 0
1, 0.61, 0, 1, 0, 0.6250, 0, 0, 1
1, 0.19, 0, 1, 0, 0.2150, 0, 0, 1
1, 0.38, 0, 0, 1, 0.4300, 0, 1, 0
0, 0.26, 1, 0, 0, 0.4230, 1, 0, 0
1, 0.61, 0, 1, 0, 0.6740, 1, 0, 0
1, 0.40, 1, 0, 0, 0.4650, 0, 1, 0
0, 0.49, 1, 0, 0, 0.6520, 0, 1, 0
1, 0.56, 1, 0, 0, 0.6750, 1, 0, 0
0, 0.48, 0, 1, 0, 0.6600, 0, 1, 0
1, 0.52, 1, 0, 0, 0.5630, 0, 0, 1
0, 0.18, 1, 0, 0, 0.2980, 1, 0, 0
0, 0.56, 0, 0, 1, 0.5930, 0, 0, 1
0, 0.52, 0, 1, 0, 0.6440, 0, 1, 0
0, 0.18, 0, 1, 0, 0.2860, 0, 1, 0
0, 0.58, 1, 0, 0, 0.6620, 0, 0, 1
0, 0.39, 0, 1, 0, 0.5510, 0, 1, 0
0, 0.46, 1, 0, 0, 0.6290, 0, 1, 0
0, 0.40, 0, 1, 0, 0.4620, 0, 1, 0
0, 0.60, 1, 0, 0, 0.7270, 0, 0, 1
1, 0.36, 0, 1, 0, 0.4070, 0, 0, 1
1, 0.44, 1, 0, 0, 0.5230, 0, 1, 0
1, 0.28, 1, 0, 0, 0.3130, 0, 0, 1
1, 0.54, 0, 0, 1, 0.6260, 1, 0, 0

Test data:

0, 0.51, 1, 0, 0, 0.6120, 0, 1, 0
0, 0.32, 0, 1, 0, 0.4610, 0, 1, 0
1, 0.55, 1, 0, 0, 0.6270, 1, 0, 0
1, 0.25, 0, 0, 1, 0.2620, 0, 0, 1
1, 0.33, 0, 0, 1, 0.3730, 0, 0, 1
0, 0.29, 0, 1, 0, 0.4620, 1, 0, 0
1, 0.65, 1, 0, 0, 0.7270, 1, 0, 0
0, 0.43, 0, 1, 0, 0.5140, 0, 1, 0
0, 0.54, 0, 1, 0, 0.6480, 0, 0, 1
1, 0.61, 0, 1, 0, 0.7270, 1, 0, 0
1, 0.52, 0, 1, 0, 0.6360, 1, 0, 0
1, 0.30, 0, 1, 0, 0.3350, 0, 0, 1
1, 0.29, 1, 0, 0, 0.3140, 0, 0, 1
0, 0.47, 0, 0, 1, 0.5940, 0, 1, 0
1, 0.39, 0, 1, 0, 0.4780, 0, 1, 0
1, 0.47, 0, 0, 1, 0.5200, 0, 1, 0
0, 0.49, 1, 0, 0, 0.5860, 0, 1, 0
0, 0.63, 0, 0, 1, 0.6740, 0, 0, 1
0, 0.30, 1, 0, 0, 0.3920, 1, 0, 0
0, 0.61, 0, 0, 1, 0.6960, 0, 0, 1
0, 0.47, 0, 0, 1, 0.5870, 0, 1, 0
1, 0.30, 0, 0, 1, 0.3450, 0, 0, 1
0, 0.51, 0, 0, 1, 0.5800, 0, 1, 0
0, 0.24, 1, 0, 0, 0.3880, 0, 1, 0
0, 0.49, 1, 0, 0, 0.6450, 0, 1, 0
1, 0.66, 0, 0, 1, 0.7450, 1, 0, 0
0, 0.65, 1, 0, 0, 0.7690, 1, 0, 0
0, 0.46, 0, 1, 0, 0.5800, 1, 0, 0
0, 0.45, 0, 0, 1, 0.5180, 0, 1, 0
0, 0.47, 1, 0, 0, 0.6360, 1, 0, 0
0, 0.29, 1, 0, 0, 0.4480, 1, 0, 0
0, 0.57, 0, 0, 1, 0.6930, 0, 0, 1
0, 0.20, 1, 0, 0, 0.2870, 0, 0, 1
0, 0.35, 1, 0, 0, 0.4340, 0, 1, 0
0, 0.61, 0, 0, 1, 0.6700, 0, 0, 1
0, 0.31, 0, 0, 1, 0.3730, 0, 1, 0
1, 0.18, 1, 0, 0, 0.2080, 0, 0, 1
1, 0.26, 0, 0, 1, 0.2920, 0, 0, 1
0, 0.28, 1, 0, 0, 0.3640, 0, 0, 1
0, 0.59, 0, 0, 1, 0.6940, 0, 0, 1