I wrote an article titled “Working With PyTorch Tensors” in the June 2020 edition of the online Visual Studio Magazine. See https://visualstudiomagazine.com/articles/2020/06/09/working-with-pytorch.aspx.

PyTorch is a Python language code library that can be used to create deep neural networks. The fundamental object in PyTorch is called a tensor. A tensor is essentially an n-dimensional array that can be processed using either a CPU or a GPU.

PyTorch tensors are surprisingly complex. One of the keys to getting started with PyTorch is learning just enough about tensors, without getting bogged down with too many details. With a basic knowledge of tensors in hand you can learn how to create and use neural networks in a spiral manner, adding knowledge of details with each new program you write.

The article presents what I consider (based on teaching many PyTorch training classes at my workplace) the fundamental concepts of tensors necessary to establish a solid foundation for learning how to create PyTorch neural networks.

I begin the article by presenting a minimal “hello world” PyTorch neural network. The idea is to provide readers with some context for how tensors are used in practice.

Then I present a relatively short demo program that illustrates most of the fundamental ideas for PyTorch tensors:

Creating tensors
Reshaping tensors
Converting tenors
Functions on tensors

Here are four example statements:

t = T.tensor([1.0, 2.0, 3.0], dtype=T.float32).to(device)
t2 = t1.reshape(4,3,2)   # vs. t1.view(4,3,2)
arr = t1.numpy()         # vs. t1.detach().numpy()
probs = T.softmax(t1, dim=1)

One of the conceptual mistakes that software developers who are new to PyTorch make is to not realize how complex a singe PyToch statement is. Each of the statements about is surpisingly complicated in terms of what happens behind the scenes.

A second pitfall is that PyTorch has an insane number of ways to perform any task. For example, the statement t = T.tensor([1.0, 2.0, 3.0], dtype=T.float32).to(device) has literally dozens of equivalent statements.

Now, all that said, I really like PyTorch a lot. But it has an extremely steep learning curve. I’m quite proud of the article I wrote — it feels right to me.

Three unexpected images returned by a search for “tensor”. Left: Appears to be some sort of a brain scan. I suspect the word “tensor” has something to do with the imaging technology. Center: Two men shopping at a very messy Target store. I suspect the words “tensor” and “target” must be related in some way. Right: A stock photo of a girl playing a tenor recorder. I suspect “tenor” was mislabeled as “tensor” in the photo metadata. Thank you Internet for being endlessly interesting.