I’ve been using the PyTorch neural code library since version 0.2 in early 2017 and I like PyTorch a lot. Even though PyTorch is slowly but surely stabilizing, there are still quite a few things about PyTorch that let you know it’s a young library. For example, why does PyTorch have three different softmax() functions? There is a tensor.softmax() method, a tensor.nn.functional.softmax() method, and a tensor.nn.Softmax() class.

As an open source library grows and evolves, there are constant changes in the library structure because it’s just not possible to anticipate what new features will be needed, and what features will be dropped. Additionally, as library contributors come and go, each guy can possibly make changes to the library structure.

So, a certain amount of structural chaos is pretty much inevitable in PyTorch. This makes learning PyTorch somewhat more difficult than it could be because there are several ways to do anything, and best practices and standard techniques won’t emerge and gel for a couple of years.

Here’s a short demo program that illustrates three ways to compute softmax() on a tensor:

# softmax_crazy.py
# PyTorch 1.4.0 Anaconda3 5.2.0 (Python 3.6.5)
# CPU, Windows, no dropout

import numpy as np
import torch as T
device = T.device("cpu")  # apply to Tensor or Module

def main():
  print("\nWhy are there 3 softmax() in PyTorch? \n")

  my_arr = np.array([1, 2, 3], dtype=np.float32)
  my_tensor = T.tensor(my_arr, dtype=T.float32).to(device)

  print("source tensor: ")
  print(my_tensor)
  print("")

  print("T.softmax(), T.nn.functional.softmax(), \
    T.nn.Softmax(): \n")

  t1 = T.softmax(my_tensor, dim=0).to(device)
  print(t1)
  print("")

  t2 = T.nn.functional.softmax(my_tensor, dim=0).to(device)
  print(t2)
  print("")

  C = T.nn.Softmax(dim=0)
  t3 = C(my_tensor).to(device)
  print(t3)
  print("")

if __name__ == "__main__":
  main()

The moral of the story is that if you are starting to learn PyTorch, or TensorFlow with Keras, or any deep neural technologies, don’t underestimate how difficult your learning path will be. But on the other hand, don’t get discouraged.

Just like there are multiple ways to do things in deep neural learning, there are multiple ways to spell at college football games. Left: University of Ohio Buckeyes flag team. Center: University of Notre Dame Fighting Irish cheerleader. Right: University of Tennessee Volunteers fans.