For the past few months, I’ve been slowly exploring the idea of implementing a neural network using raw (no libraries) JavaScript. Once every few days, on my lunch break, I’d design and code a few modules. This morning I made a nice step forward by getting a train() function up and running.

I used the standard Iris Dataset example, where the goal is to predict species (setosa, versicolor, or virginica) from four numeric predictors (sepal length and width, petal length and width).

I’ve implemented neural networks using several different programming languages (C, C#, Python, R) but I had forgotten how tricky and difficult the code is. But in the end I got the training function to work.

In order to get the train() function to work, I needed to implement several helper functions. I wrote a loadTxt() function to read data into memory. I wrote a program-defined class to generate seedable pseudo-random numbers. I wrote a shuffle() function to generate a random order for processing the training data. I wrote a meanSqErr() function to compute the current mean squared error so I could monitor error during training. And there were a few more minor functions too.

In retrospect, one of the main challenges when implementing a neural network from scratch is that there are a lot of helper functions to deal with.

Implementing a simple neural network from scratch is feasible, but not easy. By simple I mean a single hidden layer. But for two or more hidden layers, implementation gets extremely difficult.

Doc Savage was a precursor to today’s superheroes. He didn’t have special powers but he was a super-brilliant scientist with amazing athletic skills. He faced many challenges but had five friends who helped him. In the 1930s and 40s he appeared in a monthly magazine, with stories by Lester Dent and art by Walter Baumhofer. The character was mostly forgotten in the 1950s but then had a second wave of popularity in the 1960s and 70s when the stories were re-published in paperback form. Same stories but attributed to a fictitious “Kenneth Robeson” and new art by James Bama featuring truly weird hair (which I didn’t like at all). I’ve read a few of the stories and found them mildly interesting from a historical/cultural perspective.