I wrote an article titled “Multi-Class Classification Using PyTorch: Defining a Network” in the December 2020 edition of the online Microsoft Visual Studio Magazine. See https://visualstudiomagazine.com/articles/2020/12/15/pytorch-network.aspx.

The article is the second in a four-part series where I give a complete end-to-end example of how to create a deep neural multi-class classifier using Python. The running example problem across the four articles is to predict a college student’s major from their sex, number of units completed so far, home state, and score on an admission test. The raw data looks like:

M  39.5  oklahoma  512  geology
F  27.5  nebraska  286  history
M  22.0  maryland  335  finance
. . .
M  59.5  oklahoma  694  history

The network definition presented in the article has a 6-(10-10)-3 architecture:

class Net(T.nn.Module):
  def __init__(self):
    super(Net, self).__init__()
    self.hid1 = T.nn.Linear(6, 10)  # 6-(10-10)-3
    self.hid2 = T.nn.Linear(10, 10)
    self.oupt = T.nn.Linear(10, 3)

    T.nn.init.xavier_uniform_(self.hid1.weight)
    T.nn.init.zeros_(self.hid1.bias)
    T.nn.init.xavier_uniform_(self.hid2.weight)
    T.nn.init.zeros_(self.hid2.bias)
    T.nn.init.xavier_uniform_(self.oupt.weight)
    T.nn.init.zeros_(self.oupt.bias)

  def forward(self, x):
    z = T.tanh(self.hid1(x))
    z = T.tanh(self.hid2(z))
    z = self.oupt(z)  # no softmax: CrossEntropyLoss() 
    return z

There are 6 input nodes rather than 4 because the home state predictor is one-hot encoded, for example, nebraska = 0,1,0.

There are many design decisions that must be made when creating a neural prediction model. These decisions include number of hidden layers, number of nodes in each layer, the initialization algorithms for nodes and biases, and the hidden layer and output layer activation functions.

Learning how to create, train, and use deep neural networks isn’t easy but it’s not rocket surgery either.

Two rocket surgery illustrations by artist Klaus Burgle (1926-2015).