Logistic regression is a technique for binary classification. For example, you might want to predict the sex of a person (0 = male, 1 = female) based on age, State, income, and political leaning. I enjoy writing code, so one rainy evening (I live in the Pacific Northwest so this wasn’t unusual), while I was waiting for the rain to stop so I could walk my dogs, I decided to implement logistic regression, from scratch, using C#.

In logistic regression, the prediction output is a pseudo-probability value between 0.0 and 1.0. If the p-value is less than 0.5, the prediction is class 0. If the p-value is greater than or equal to 0.5, the prediction is class 1. For any problem, the choice of which label is 0 and which is 1, is arbitrary.

The prediction equation for logistic regression is:

z = (wo * x0) + (w1 * x1) + . . . + b
p = 1.0 / (1.0 + exp(-z))

The xi are predictor values. The wi are model weights. The b is a bias constant. Training the model is the process of finding values for the weights and the bias so that computed output probabilities closely match the target y (0 / 1) values.

For my demo, I used one of my standard artificial datasets. The raw data looks like:

F, 24, michigan, 29500.00, liberal
M, 39, oklahoma, 51200.00, moderate
F, 63, nebraska, 75800.00, conservative
. . .

The normalized and encoded data looks like:

1, 0.24, 1,0,0, 0.2950, 0,0,1
0, 0.39, 0,0,1, 0.5120, 0,1,0
1, 0.63, 0,1,0, 0.7580, 1,0,0
. . .

There are only three State values. The output of my demo is:

Begin C# linear regression demo with Newton-Raphson training

Loading people train (200) and test (40) data
Done

First three train X:
  0.2400  1.00  0.00  0.00  0.2950  0.00  0.00  1.00
  0.3900  0.00  0.00  1.00  0.5120  0.00  1.00  0.00
  0.6300  0.00  1.00  0.00  0.7580  1.00  0.00  0.00

First three train y:
  1.0000
  0.0000
  1.0000

Creating Logistic Regression model
Done

Start Newton-Raphson train
epoch =     0  MSE =   0.1274
epoch =    40  MSE =   0.1145
epoch =    80  MSE =   0.1145
epoch =   120  MSE =   0.1145
epoch =   160  MSE =   0.1145
Done

Weights/coefficients:
81.4712 1.8590 1.8903 0.2694 -80.7779 3.9275 1.1614 -1.0702
Bias/constant: 4.0187

Evaluating model

Accuracy train = 0.8450
Accuracy test = 0.7250

MSE train = 0.1145
MSE test = 0.1305

Confusion matrix:
 actual   0 |     17     9  | 0.6538
 actual   1 |      2    12  | 0.8571
----------
predicted:         0     1  | 0.7250

End demo

There are dozens of design alternatives for logistic regression. I used Newton-Raphson training with binary cross entropy (BCE) loss gradient updates, combined with mean squared error loss progress messages.

Newton-Raphson training is appealing because it does not require any training parameters such as a learning rate and maximum epochs, that must be determined by trial and error. The main alternative, stochastic gradient descent (SGD), does require a learning rate and max epochs. Newton-Raphson computes an nxn Hessian matrix, which isn’t feasible for extrememly large datasets, but Newton-Raphson is very fast compared to SGD training. L-BFGS training is possible too, but it isn’t well-suited for scratch implementation.

I implemented code to compute and display a confusion matrix. This is more or less standard procedure for binary classification problems.

I validated my logistic regression system by running the demo data through the scikit-learn library LogisticRegression module. I got essentially the same results. See https://jamesmccaffreyblog.com/2023/02/10/logistic-regression-using-the-scikit-library-in-visual-studio-magazine/.

My implementation can be improved. Currently, the training loop iterates a fixed number of iterations (maxEpochs). It’s possible to monitor the change in model weights and bias, and early-exit training when the largest change is less than some small value, like 1.0e-6 or so. But that’s a task for another day.

Good fun on a rainy Pacific Northwest night.

I am a big fan of science fiction films of the 1950s. Some of these movies have really bad special effects, but pretty good acting and story, make them entertaining (to me anyway).

Right: In “The Giant Claw” (1957), a giant bird creature somehow manages to get to Earth from an antimatter galaxy, which makes it invulnerable to all missiles and weapons. Scientists devise a way to remove the creature’s antimatter protection, and kill it with rockets. This is a pretty good movie but the production ran out of money before the special effects could be added, and resorted to low-budget Mexican special effects company. My grade for the movie = B.

Demo program. Replace “lt” (less than), “gt”, “lte”, “gte” with Boolean operator symbols. My blog editor chokes on symbols.

using System;
using System.IO;
using System.Collections.Generic;

namespace LogisticRegressionPeopleNR
{
  internal class LogisticRegressionNRProgram
  {
    static void Main(string[] args)
    {
      Console.WriteLine("\nBegin C# linear regression" +
        " demo with Newton-Raphson training ");
      // sex age State income politics
      // 1, 0.24, 1,0,0, 0.2950, 0,0,1
      // 0, 0.39, 0,0,1, 0.5120, 0,1,0
      // 0    1   2 3 4     5    6 7 8
      //
      // predict sex from age State income politics

      // 1. load data
      Console.WriteLine("\nLoading people train" +
        " (200) and test (40) data");
      string trainFile =
        "..\\..\\..\\Data\\people_train_200.txt";
      int[] colsX = new int[] { 1, 2, 3, 4, 5, 6, 7, 8 };
      double[][] trainX =
        MatLoad(trainFile, colsX, ',', "#");
      double[] trainY =
        MatToVec(MatLoad(trainFile,
        new int[] { 0 }, ',', "#"));

      string testFile =
        "..\\..\\..\\Data\\people_test_40.txt";
      double[][] testX =
        MatLoad(testFile, colsX, ',', "#");
      double[] testY =
        MatToVec(MatLoad(testFile,
        new int[] { 0 }, ',', "#"));
      Console.WriteLine("Done ");

      Console.WriteLine("\nFirst three train X: ");
      for (int i = 0; i "lt" 3; ++i)
        VecShow(trainX[i], 4, 8);

      Console.WriteLine("\nFirst three train y: ");
      for (int i = 0; i "lt" 3; ++i)
        Console.WriteLine(trainY[i].ToString("F4").
          PadLeft(8));

      // 2. create and train model
      Console.WriteLine("\nCreating Logistic " +
        "Regression model ");
      LogisticRegression model =
        new LogisticRegression();
      Console.WriteLine("Done ");

      Console.WriteLine("\nStart Newton-Raphson train ");
      model.Train(trainX, trainY);
      Console.WriteLine("Done ");

      // 2b.show model parameters
      Console.WriteLine("\nWeights/coefficients: ");
      for (int i = 0; i "lt" model.weights.Length; ++i)
        Console.Write(model.weights[i].ToString("F4") + " ");
      Console.WriteLine("\nBias/constant: " +
        model.bias.ToString("F4"));

      // 3. evaluate model
      Console.WriteLine("\nEvaluating model ");

      double accTrain = model.Accuracy(trainX, trainY);
      Console.WriteLine("\nAccuracy train = " +
        accTrain.ToString("F4"));
      double accTest = model.Accuracy(testX, testY);
      Console.WriteLine("Accuracy test = " +
        accTest.ToString("F4"));

      double mseTrain = model.MSE(trainX, trainY);
      Console.WriteLine("\nMSE train = " +
        mseTrain.ToString("F4"));
      double mseTest = model.MSE(testX, testY);
      Console.WriteLine("MSE test = " +
        mseTest.ToString("F4"));

      Console.WriteLine("\nConfusion matrix: ");
      int[][] cm = model.ConfusionMatrix(testX, testY);
      model.ShowConfusion(cm);

      Console.WriteLine("\nEnd demo ");
      Console.ReadLine();

    } // Main

    // ------------------------------------------------------
    // helpers for Main()
    // ------------------------------------------------------

    static double[][] MatLoad(string fn, int[] usecols,
      char sep, string comment)
    {
      List"lt"double[]"gt" result = 
        new List"lt"double[]"gt"();
      string line = "";
      FileStream ifs = new FileStream(fn, FileMode.Open);
      StreamReader sr = new StreamReader(ifs);
      while ((line = sr.ReadLine()) != null)
      {
        if (line.StartsWith(comment) == true)
          continue;
        string[] tokens = line.Split(sep);
        List"lt"double"gt" lst = new List"lt"double"gt"();
        for (int j = 0; j "lt" usecols.Length; ++j)
          lst.Add(double.Parse(tokens[usecols[j]]));
        double[] row = lst.ToArray();
        result.Add(row);
      }
      sr.Close(); ifs.Close();
      return result.ToArray();
    }

    static double[] MatToVec(double[][] M)
    {
      int nRows = M.Length;
      int nCols = M[0].Length;
      double[] result = new double[nRows * nCols];
      int k = 0;
      for (int i = 0; i "lt" nRows; ++i)
        for (int j = 0; j "lt" nCols; ++j)
          result[k++] = M[i][j];
      return result;
    }

    static void VecShow(double[] vec, int dec, int wid)
    {
      for (int i = 0; i "lt" vec.Length; ++i)
        Console.Write(vec[i].ToString("F" + dec).
          PadLeft(wid));
      Console.WriteLine("");
    }

  } // class Program

  // ========================================================

  public class LogisticRegression
  {
    public double[] weights;
    public double bias;
    private Random rnd;

    public LogisticRegression(int seed = 0)
    {
      this.weights = new double[0]; // compiler happy
      this.bias = 0.0;
      this.rnd = new Random(seed);
    }

    // ------------------------------------------------------

    public int Predict(double[] x)
    {
      double p = ComputeP(x);
      if (p "lt" 0.5)
        return 0;
      else
        return 1;
    }

    // ------------------------------------------------------

    public double ComputeP(double[] x)
    {
      double z = 0.0;
      for (int i = 0; i "lt" x.Length; ++i)
        z += this.weights[i] * x[i];
      z += this.bias;
      double p = this.Sigmoid(z);
      return p;
    }

    // ------------------------------------------------------

    public double Sigmoid(double z)
    {
      if (z "lt" -6.0)
        return 0.0;
      else if (z "gt" 6.0)
        return 1.0;
      else
        return 1.0 / (1.0 + Math.Exp(-z));
    }

    // ------------------------------------------------------

    public void Train(double[][] trainX, double[] trainY)
    {
      // Newton-Raphson technique
      // initialize weights and bias
      int dim = trainX[0].Length;
      int n = trainX.Length;
      int m = dim + 1;  // include bias as final parameter

      this.weights = new double[dim];
      this.bias = 0.0;
      int maxEpochs = trainX.Length; // heuristic

      // Newton-Raphson iterations
      for (int epoch = 0; epoch "lt" maxEpochs; ++epoch)
      {
        // gradient and Hessian
        double[] grad = new double[m]; // features + bias
        double[][] H = new double[m][]; // H is mxm
        for (int i = 0; i "lt" m; ++i)
          H[i] = new double[m];

        // accumulate gradient and Hessian
        for (int i = 0; i "lt" n; ++i) // each train item
        {
          double[] x = trainX[i];
          double y = trainY[i];

          double p = this.ComputeP(x);
          double error = p - y; // order is important
          double v = p * (1.0 - p);

          // include bias as a final feature
          double[] xBias = new double[m];
          for (int j = 0; j "lt" dim; ++j)
            xBias[j] = x[j];
          xBias[dim] = 1.0;

          // gradient
          for (int j = 0; j "lt" m; ++j)
            grad[j] += error * xBias[j];

          // Hessian
          for (int j = 0; j "lt" m; ++j)
            for (int k = 0; k "lt" m; ++k)
              H[j][k] += v * xBias[j] * xBias[k];
        }

        // average gradient and Hessian
        for (int j = 0; j "lt" m; ++j)
        {
          grad[j] /= n;
          for (int k = 0; k "lt" m; ++k)
            H[j][k] /= n;
        }

        // small diagonal for numerical stability
        for (int j = 0; j "lt" m; ++j)
          H[j][j] += 1.0e-8;

        // Cholesky decomposition: H = L * L^T
        double[][] L = new double[m][];
        for (int i = 0; i "lt" m; ++i)
          L[i] = new double[m];

        for (int i = 0; i "lt" m; ++i)
        {
          for (int j = 0; j "lte" i; ++j)
          {
            double sum = H[i][j];

            for (int k = 0; k "lt" j; ++k)
              sum -= L[i][k] * L[j][k];

            if (i == j)
              L[i][j] = Math.Sqrt(sum);
            else
              L[i][j] = sum / L[j][j]; // not 0
          }
        } // each item

        // solve L * z = -gradient
        // (forward substitution)
        double[] z = new double[m];

        for (int i = 0; i "lt" m; ++i)
        {
          double sum = -grad[i];
          for (int j = 0; j "lt" i; ++j)
            sum -= L[i][j] * z[j];
          z[i] = sum / L[i][i];
        }

        // Solve L^T * delta = z
        // (back substitution)
        double[] delta = new double[m];

        for (int i = m - 1; i "gte" 0; --i)
        {
          double sum = z[i];
          for (int j = i + 1; j "lt" m; ++j)
            sum -= L[j][i] * delta[j];
          delta[i] = sum / L[i][i];
        }

        // update weights and bias
        for (int j = 0; j "lt" dim; ++j)
          this.weights[j] += delta[j];
        this.bias += delta[dim];

        // show progress
        if (epoch % Math.Max(1, maxEpochs / 5) == 0)
        {
          double mse = this.MSE(trainX, trainY);
          Console.WriteLine("epoch = " +
            epoch.ToString().PadLeft(5) +
            "  MSE = " + mse.ToString("F4").PadLeft(8));
        }
      } // each epoch
    } // Train()

    // ------------------------------------------------------

    public double MSE(double[][] dataX, double[] dataY)
    {
      int n = dataX.Length;
      double sum = 0.0;
      for (int i = 0; i "lt" n; ++i)
      {
        double[] x = dataX[i];
        double y = dataY[i];
        double p = this.ComputeP(x);
        sum += (p - y) * (p - y);
      }
      return sum / n;
    }

    // ------------------------------------------------------

    public double Accuracy(double[][] dataX, double[] dataY)
    {
      int n = dataX.Length;
      int numCorrect = 0; int numWrong = 0;
      for (int i = 0; i "lt" n; ++i)
      {
        double[] x = dataX[i];
        double y = (int)(dataY[i]);  // 0 or 1
        int predY = this.Predict(x);  // 0 or 1

        if (predY == y)
          ++numCorrect;
        else
          ++numWrong;
      }
      return (numCorrect * 1.0) / (numCorrect + numWrong);
    }

    // ------------------------------------------------------

    public int[][] ConfusionMatrix(double[][] dataX,
      double[] dataY)
    {
      // assumes binary classification
      int[][] result = new int[2][];  // 2x2
      result[0] = new int[2];
      result[1] = new int[2];
      int n = dataX.Length;
      for (int i = 0; i "lt" n; ++i) // each data item
      {
        int predY = this.Predict(dataX[i]); // 0 or 1
        int actualY = (int)(dataY[i]);  // 0 or 1

        ++result[actualY][predY];

        //if (actualY == 0 && predY == 0)
        //  ++result[0][0];
        //else if (actualY == 0 && predY == 1)
        //  ++result[0][1];
        //else if (actualY == 1 && predY == 0)
        //  ++result[1][0];
        //else if (actualY == 1 && predY == 1)
        //  ++result[1][1];
      }
      return result;
    }

    public void ShowConfusionX(int[][] cm)
    {
      // simple display version
      Console.Write("actual = 0: ");
      Console.Write(cm[0][0].ToString().PadLeft(4) + " ");
      Console.WriteLine(cm[0][1].ToString().PadLeft(4));
      Console.Write("actual = 1: ");
      Console.Write(cm[1][0].ToString().PadLeft(4) + " ");
      Console.WriteLine(cm[1][1].ToString().PadLeft(4));
    }

    public void ShowConfusion(int[][] cm)
    {
      int n = cm.Length; // number classes
      double[] accuracies = new double[n];
      for (int i = 0; i "lt" n; ++i)
      {
        int rowSum = 0;
        for (int j = 0; j "lt" n; ++j)
          rowSum += cm[i][j];
        accuracies[i] = (cm[i][i] * 1.0) / rowSum;
      }

      // assume binary classification
      int numCorrect = cm[0][0] + cm[1][1];
      int numItems = 
        cm[0][0] + cm[0][1] + cm[1][0] + cm[1][1];
      double overallAcc = (numCorrect * 1.0) / numItems;
     
      for (int i = 0; i "lt" n; ++i)
      {
        Console.Write(" actual " + 
          i.ToString().PadLeft(3) + " | ");
        for (int j = 0; j "lt" n; ++j)
          Console.Write(cm[i][j].ToString().PadLeft(6));
        Console.WriteLine("  | " + 
          accuracies[i].ToString("F4"));
      }
      Console.WriteLine("----------");
      Console.Write("predicted:    ");
      for (int j = 0; j "lt" cm[0].Length; ++j)
        Console.Write(j.ToString().PadLeft(6));
      Console.Write("  | " + overallAcc.ToString("F4"));
      Console.WriteLine("");
    }

  } // class LogisticRegression

  // ========================================================

} // ns

Training data:

# people_train_200.txt
# sex (0 = male, 1 = female)
# age, state (michigan, nebraska, oklahoma), income,
# politics type (conservative, moderate, liberal)
#
1, 0.24, 1, 0, 0, 0.2950, 0, 0, 1
0, 0.39, 0, 0, 1, 0.5120, 0, 1, 0
1, 0.63, 0, 1, 0, 0.7580, 1, 0, 0
0, 0.36, 1, 0, 0, 0.4450, 0, 1, 0
1, 0.27, 0, 1, 0, 0.2860, 0, 0, 1
1, 0.50, 0, 1, 0, 0.5650, 0, 1, 0
1, 0.50, 0, 0, 1, 0.5500, 0, 1, 0
0, 0.19, 0, 0, 1, 0.3270, 1, 0, 0
1, 0.22, 0, 1, 0, 0.2770, 0, 1, 0
0, 0.39, 0, 0, 1, 0.4710, 0, 0, 1
1, 0.34, 1, 0, 0, 0.3940, 0, 1, 0
0, 0.22, 1, 0, 0, 0.3350, 1, 0, 0
1, 0.35, 0, 0, 1, 0.3520, 0, 0, 1
0, 0.33, 0, 1, 0, 0.4640, 0, 1, 0
1, 0.45, 0, 1, 0, 0.5410, 0, 1, 0
1, 0.42, 0, 1, 0, 0.5070, 0, 1, 0
0, 0.33, 0, 1, 0, 0.4680, 0, 1, 0
1, 0.25, 0, 0, 1, 0.3000, 0, 1, 0
0, 0.31, 0, 1, 0, 0.4640, 1, 0, 0
1, 0.27, 1, 0, 0, 0.3250, 0, 0, 1
1, 0.48, 1, 0, 0, 0.5400, 0, 1, 0
0, 0.64, 0, 1, 0, 0.7130, 0, 0, 1
1, 0.61, 0, 1, 0, 0.7240, 1, 0, 0
1, 0.54, 0, 0, 1, 0.6100, 1, 0, 0
1, 0.29, 1, 0, 0, 0.3630, 1, 0, 0
1, 0.50, 0, 0, 1, 0.5500, 0, 1, 0
1, 0.55, 0, 0, 1, 0.6250, 1, 0, 0
1, 0.40, 1, 0, 0, 0.5240, 1, 0, 0
1, 0.22, 1, 0, 0, 0.2360, 0, 0, 1
1, 0.68, 0, 1, 0, 0.7840, 1, 0, 0
0, 0.60, 1, 0, 0, 0.7170, 0, 0, 1
0, 0.34, 0, 0, 1, 0.4650, 0, 1, 0
0, 0.25, 0, 0, 1, 0.3710, 1, 0, 0
0, 0.31, 0, 1, 0, 0.4890, 0, 1, 0
1, 0.43, 0, 0, 1, 0.4800, 0, 1, 0
1, 0.58, 0, 1, 0, 0.6540, 0, 0, 1
0, 0.55, 0, 1, 0, 0.6070, 0, 0, 1
0, 0.43, 0, 1, 0, 0.5110, 0, 1, 0
0, 0.43, 0, 0, 1, 0.5320, 0, 1, 0
0, 0.21, 1, 0, 0, 0.3720, 1, 0, 0
1, 0.55, 0, 0, 1, 0.6460, 1, 0, 0
1, 0.64, 0, 1, 0, 0.7480, 1, 0, 0
0, 0.41, 1, 0, 0, 0.5880, 0, 1, 0
1, 0.64, 0, 0, 1, 0.7270, 1, 0, 0
0, 0.56, 0, 0, 1, 0.6660, 0, 0, 1
1, 0.31, 0, 0, 1, 0.3600, 0, 1, 0
0, 0.65, 0, 0, 1, 0.7010, 0, 0, 1
1, 0.55, 0, 0, 1, 0.6430, 1, 0, 0
0, 0.25, 1, 0, 0, 0.4030, 1, 0, 0
1, 0.46, 0, 0, 1, 0.5100, 0, 1, 0
0, 0.36, 1, 0, 0, 0.5350, 1, 0, 0
1, 0.52, 0, 1, 0, 0.5810, 0, 1, 0
1, 0.61, 0, 0, 1, 0.6790, 1, 0, 0
1, 0.57, 0, 0, 1, 0.6570, 1, 0, 0
0, 0.46, 0, 1, 0, 0.5260, 0, 1, 0
0, 0.62, 1, 0, 0, 0.6680, 0, 0, 1
1, 0.55, 0, 0, 1, 0.6270, 1, 0, 0
0, 0.22, 0, 0, 1, 0.2770, 0, 1, 0
0, 0.50, 1, 0, 0, 0.6290, 1, 0, 0
0, 0.32, 0, 1, 0, 0.4180, 0, 1, 0
0, 0.21, 0, 0, 1, 0.3560, 1, 0, 0
1, 0.44, 0, 1, 0, 0.5200, 0, 1, 0
1, 0.46, 0, 1, 0, 0.5170, 0, 1, 0
1, 0.62, 0, 1, 0, 0.6970, 1, 0, 0
1, 0.57, 0, 1, 0, 0.6640, 1, 0, 0
0, 0.67, 0, 0, 1, 0.7580, 0, 0, 1
1, 0.29, 1, 0, 0, 0.3430, 0, 0, 1
1, 0.53, 1, 0, 0, 0.6010, 1, 0, 0
0, 0.44, 1, 0, 0, 0.5480, 0, 1, 0
1, 0.46, 0, 1, 0, 0.5230, 0, 1, 0
0, 0.20, 0, 1, 0, 0.3010, 0, 1, 0
0, 0.38, 1, 0, 0, 0.5350, 0, 1, 0
1, 0.50, 0, 1, 0, 0.5860, 0, 1, 0
1, 0.33, 0, 1, 0, 0.4250, 0, 1, 0
0, 0.33, 0, 1, 0, 0.3930, 0, 1, 0
1, 0.26, 0, 1, 0, 0.4040, 1, 0, 0
1, 0.58, 1, 0, 0, 0.7070, 1, 0, 0
1, 0.43, 0, 0, 1, 0.4800, 0, 1, 0
0, 0.46, 1, 0, 0, 0.6440, 1, 0, 0
1, 0.60, 1, 0, 0, 0.7170, 1, 0, 0
0, 0.42, 1, 0, 0, 0.4890, 0, 1, 0
0, 0.56, 0, 0, 1, 0.5640, 0, 0, 1
0, 0.62, 0, 1, 0, 0.6630, 0, 0, 1
0, 0.50, 1, 0, 0, 0.6480, 0, 1, 0
1, 0.47, 0, 0, 1, 0.5200, 0, 1, 0
0, 0.67, 0, 1, 0, 0.8040, 0, 0, 1
0, 0.40, 0, 0, 1, 0.5040, 0, 1, 0
1, 0.42, 0, 1, 0, 0.4840, 0, 1, 0
1, 0.64, 1, 0, 0, 0.7200, 1, 0, 0
0, 0.47, 1, 0, 0, 0.5870, 0, 0, 1
1, 0.45, 0, 1, 0, 0.5280, 0, 1, 0
0, 0.25, 0, 0, 1, 0.4090, 1, 0, 0
1, 0.38, 1, 0, 0, 0.4840, 1, 0, 0
1, 0.55, 0, 0, 1, 0.6000, 0, 1, 0
0, 0.44, 1, 0, 0, 0.6060, 0, 1, 0
1, 0.33, 1, 0, 0, 0.4100, 0, 1, 0
1, 0.34, 0, 0, 1, 0.3900, 0, 1, 0
1, 0.27, 0, 1, 0, 0.3370, 0, 0, 1
1, 0.32, 0, 1, 0, 0.4070, 0, 1, 0
1, 0.42, 0, 0, 1, 0.4700, 0, 1, 0
0, 0.24, 0, 0, 1, 0.4030, 1, 0, 0
1, 0.42, 0, 1, 0, 0.5030, 0, 1, 0
1, 0.25, 0, 0, 1, 0.2800, 0, 0, 1
1, 0.51, 0, 1, 0, 0.5800, 0, 1, 0
0, 0.55, 0, 1, 0, 0.6350, 0, 0, 1
1, 0.44, 1, 0, 0, 0.4780, 0, 0, 1
0, 0.18, 1, 0, 0, 0.3980, 1, 0, 0
0, 0.67, 0, 1, 0, 0.7160, 0, 0, 1
1, 0.45, 0, 0, 1, 0.5000, 0, 1, 0
1, 0.48, 1, 0, 0, 0.5580, 0, 1, 0
0, 0.25, 0, 1, 0, 0.3900, 0, 1, 0
0, 0.67, 1, 0, 0, 0.7830, 0, 1, 0
1, 0.37, 0, 0, 1, 0.4200, 0, 1, 0
0, 0.32, 1, 0, 0, 0.4270, 0, 1, 0
1, 0.48, 1, 0, 0, 0.5700, 0, 1, 0
0, 0.66, 0, 0, 1, 0.7500, 0, 0, 1
1, 0.61, 1, 0, 0, 0.7000, 1, 0, 0
0, 0.58, 0, 0, 1, 0.6890, 0, 1, 0
1, 0.19, 1, 0, 0, 0.2400, 0, 0, 1
1, 0.38, 0, 0, 1, 0.4300, 0, 1, 0
0, 0.27, 1, 0, 0, 0.3640, 0, 1, 0
1, 0.42, 1, 0, 0, 0.4800, 0, 1, 0
1, 0.60, 1, 0, 0, 0.7130, 1, 0, 0
0, 0.27, 0, 0, 1, 0.3480, 1, 0, 0
1, 0.29, 0, 1, 0, 0.3710, 1, 0, 0
0, 0.43, 1, 0, 0, 0.5670, 0, 1, 0
1, 0.48, 1, 0, 0, 0.5670, 0, 1, 0
1, 0.27, 0, 0, 1, 0.2940, 0, 0, 1
0, 0.44, 1, 0, 0, 0.5520, 1, 0, 0
1, 0.23, 0, 1, 0, 0.2630, 0, 0, 1
0, 0.36, 0, 1, 0, 0.5300, 0, 0, 1
1, 0.64, 0, 0, 1, 0.7250, 1, 0, 0
1, 0.29, 0, 0, 1, 0.3000, 0, 0, 1
0, 0.33, 1, 0, 0, 0.4930, 0, 1, 0
0, 0.66, 0, 1, 0, 0.7500, 0, 0, 1
0, 0.21, 0, 0, 1, 0.3430, 1, 0, 0
1, 0.27, 1, 0, 0, 0.3270, 0, 0, 1
1, 0.29, 1, 0, 0, 0.3180, 0, 0, 1
0, 0.31, 1, 0, 0, 0.4860, 0, 1, 0
1, 0.36, 0, 0, 1, 0.4100, 0, 1, 0
1, 0.49, 0, 1, 0, 0.5570, 0, 1, 0
0, 0.28, 1, 0, 0, 0.3840, 1, 0, 0
0, 0.43, 0, 0, 1, 0.5660, 0, 1, 0
0, 0.46, 0, 1, 0, 0.5880, 0, 1, 0
1, 0.57, 1, 0, 0, 0.6980, 1, 0, 0
0, 0.52, 0, 0, 1, 0.5940, 0, 1, 0
0, 0.31, 0, 0, 1, 0.4350, 0, 1, 0
0, 0.55, 1, 0, 0, 0.6200, 0, 0, 1
1, 0.50, 1, 0, 0, 0.5640, 0, 1, 0
1, 0.48, 0, 1, 0, 0.5590, 0, 1, 0
0, 0.22, 0, 0, 1, 0.3450, 1, 0, 0
1, 0.59, 0, 0, 1, 0.6670, 1, 0, 0
1, 0.34, 1, 0, 0, 0.4280, 0, 0, 1
0, 0.64, 1, 0, 0, 0.7720, 0, 0, 1
1, 0.29, 0, 0, 1, 0.3350, 0, 0, 1
0, 0.34, 0, 1, 0, 0.4320, 0, 1, 0
0, 0.61, 1, 0, 0, 0.7500, 0, 0, 1
1, 0.64, 0, 0, 1, 0.7110, 1, 0, 0
0, 0.29, 1, 0, 0, 0.4130, 1, 0, 0
1, 0.63, 0, 1, 0, 0.7060, 1, 0, 0
0, 0.29, 0, 1, 0, 0.4000, 1, 0, 0
0, 0.51, 1, 0, 0, 0.6270, 0, 1, 0
0, 0.24, 0, 0, 1, 0.3770, 1, 0, 0
1, 0.48, 0, 1, 0, 0.5750, 0, 1, 0
1, 0.18, 1, 0, 0, 0.2740, 1, 0, 0
1, 0.18, 1, 0, 0, 0.2030, 0, 0, 1
1, 0.33, 0, 1, 0, 0.3820, 0, 0, 1
0, 0.20, 0, 0, 1, 0.3480, 1, 0, 0
1, 0.29, 0, 0, 1, 0.3300, 0, 0, 1
0, 0.44, 0, 0, 1, 0.6300, 1, 0, 0
0, 0.65, 0, 0, 1, 0.8180, 1, 0, 0
0, 0.56, 1, 0, 0, 0.6370, 0, 0, 1
0, 0.52, 0, 0, 1, 0.5840, 0, 1, 0
0, 0.29, 0, 1, 0, 0.4860, 1, 0, 0
0, 0.47, 0, 1, 0, 0.5890, 0, 1, 0
1, 0.68, 1, 0, 0, 0.7260, 0, 0, 1
1, 0.31, 0, 0, 1, 0.3600, 0, 1, 0
1, 0.61, 0, 1, 0, 0.6250, 0, 0, 1
1, 0.19, 0, 1, 0, 0.2150, 0, 0, 1
1, 0.38, 0, 0, 1, 0.4300, 0, 1, 0
0, 0.26, 1, 0, 0, 0.4230, 1, 0, 0
1, 0.61, 0, 1, 0, 0.6740, 1, 0, 0
1, 0.40, 1, 0, 0, 0.4650, 0, 1, 0
0, 0.49, 1, 0, 0, 0.6520, 0, 1, 0
1, 0.56, 1, 0, 0, 0.6750, 1, 0, 0
0, 0.48, 0, 1, 0, 0.6600, 0, 1, 0
1, 0.52, 1, 0, 0, 0.5630, 0, 0, 1
0, 0.18, 1, 0, 0, 0.2980, 1, 0, 0
0, 0.56, 0, 0, 1, 0.5930, 0, 0, 1
0, 0.52, 0, 1, 0, 0.6440, 0, 1, 0
0, 0.18, 0, 1, 0, 0.2860, 0, 1, 0
0, 0.58, 1, 0, 0, 0.6620, 0, 0, 1
0, 0.39, 0, 1, 0, 0.5510, 0, 1, 0
0, 0.46, 1, 0, 0, 0.6290, 0, 1, 0
0, 0.40, 0, 1, 0, 0.4620, 0, 1, 0
0, 0.60, 1, 0, 0, 0.7270, 0, 0, 1
1, 0.36, 0, 1, 0, 0.4070, 0, 0, 1
1, 0.44, 1, 0, 0, 0.5230, 0, 1, 0
1, 0.28, 1, 0, 0, 0.3130, 0, 0, 1
1, 0.54, 0, 0, 1, 0.6260, 1, 0, 0

Test data:

# people_test_40.txt
#
0, 0.51, 1, 0, 0, 0.6120, 0, 1, 0
0, 0.32, 0, 1, 0, 0.4610, 0, 1, 0
1, 0.55, 1, 0, 0, 0.6270, 1, 0, 0
1, 0.25, 0, 0, 1, 0.2620, 0, 0, 1
1, 0.33, 0, 0, 1, 0.3730, 0, 0, 1
0, 0.29, 0, 1, 0, 0.4620, 1, 0, 0
1, 0.65, 1, 0, 0, 0.7270, 1, 0, 0
0, 0.43, 0, 1, 0, 0.5140, 0, 1, 0
0, 0.54, 0, 1, 0, 0.6480, 0, 0, 1
1, 0.61, 0, 1, 0, 0.7270, 1, 0, 0
1, 0.52, 0, 1, 0, 0.6360, 1, 0, 0
1, 0.30, 0, 1, 0, 0.3350, 0, 0, 1
1, 0.29, 1, 0, 0, 0.3140, 0, 0, 1
0, 0.47, 0, 0, 1, 0.5940, 0, 1, 0
1, 0.39, 0, 1, 0, 0.4780, 0, 1, 0
1, 0.47, 0, 0, 1, 0.5200, 0, 1, 0
0, 0.49, 1, 0, 0, 0.5860, 0, 1, 0
0, 0.63, 0, 0, 1, 0.6740, 0, 0, 1
0, 0.30, 1, 0, 0, 0.3920, 1, 0, 0
0, 0.61, 0, 0, 1, 0.6960, 0, 0, 1
0, 0.47, 0, 0, 1, 0.5870, 0, 1, 0
1, 0.30, 0, 0, 1, 0.3450, 0, 0, 1
0, 0.51, 0, 0, 1, 0.5800, 0, 1, 0
0, 0.24, 1, 0, 0, 0.3880, 0, 1, 0
0, 0.49, 1, 0, 0, 0.6450, 0, 1, 0
1, 0.66, 0, 0, 1, 0.7450, 1, 0, 0
0, 0.65, 1, 0, 0, 0.7690, 1, 0, 0
0, 0.46, 0, 1, 0, 0.5800, 1, 0, 0
0, 0.45, 0, 0, 1, 0.5180, 0, 1, 0
0, 0.47, 1, 0, 0, 0.6360, 1, 0, 0
0, 0.29, 1, 0, 0, 0.4480, 1, 0, 0
0, 0.57, 0, 0, 1, 0.6930, 0, 0, 1
0, 0.20, 1, 0, 0, 0.2870, 0, 0, 1
0, 0.35, 1, 0, 0, 0.4340, 0, 1, 0
0, 0.61, 0, 0, 1, 0.6700, 0, 0, 1
0, 0.31, 0, 0, 1, 0.3730, 0, 1, 0
1, 0.18, 1, 0, 0, 0.2080, 0, 0, 1
1, 0.26, 0, 0, 1, 0.2920, 0, 0, 1
0, 0.28, 1, 0, 0, 0.3640, 0, 0, 1
0, 0.59, 0, 0, 1, 0.6940, 0, 0, 1