Category Archives: PyTorch

Neural Network Comparison: From-Scratch C# vs. PyTorch

I was at a conference where I was scheduled to give a presentation “Introduction to Neural Networks Using C#”. I figured I’d create a PyTorch version of my from-scratch C# demo, just in case I had lots of extra time … Continue reading

Posted in PyTorch | Leave a comment

PyTorch Anomaly Detection for Sequential Data Using an Autoencoder with a Transformer Module and Numeric Pseudo-Embedding

Whew! This may be one of the worst blog titles I’ve ever written, but it was the best I could do. Let me try to explain. The idea starts with the data. Most ordinary data is not sequential. For example, … Continue reading

Posted in PyTorch, Transformers | Leave a comment

Binary Classification Using PyTorch with the Multi-Class Approach and NLLLoss

I’ve always known that, in theory, to create a PyTorch binary classification prediction model, you can just pretend the problem is a multi-class classification problem. Bottom line: the multi-class approach works but has no advantage over the standard binary approach. … Continue reading

Posted in PyTorch | Leave a comment

Creating Some Semi-Realistic Synthetic Medical Data

The goal of one of my ongoing projects is to find anomalies in medical data, where the data is ordered in some way. I imagine each line of data represents a hospital patient, and each value on a line is … Continue reading

Posted in PyTorch, Transformers | Leave a comment

Quantile Regression Using a PyTorch Neural Network with a Quantile Loss Function

Quantile regression is a machine learning technique to predict a single numeric value in situations where you care about under-prediction (or sometimes over-prediction). I’ll phrase the rest of this blog post in terms of scenarios where you mostly care about … Continue reading

Posted in PyTorch | Leave a comment

Poisson Neural Network Using PyTorch with PoissonNLLLoss

Poisson distributed data is often counts that arise when “things arrive” per some fixed unit of time. For example, if you observe the count of cars arriving at a traffic intersection each Monday between 11:00 and 11:05 PM, you might … Continue reading

Posted in PyTorch | Leave a comment

Predicting Poisson Distributed Target Data: PyTorch Neural Network vs scikit PoissonRegressor

Bottom line: I did a few experiments that compared predicting multivariate Poisson distributed target data using 1.) a standard PyTorch neural network regression system and 2.) using the scikit-learn PoissonRegressor() module. The results were roughly the same. Several weeks ago, … Continue reading

Posted in PyTorch | Leave a comment

PyTorch Neural Network Distillation Using the Teacher-Student Technique

The DeepSeek R1 large language model, announced on January 20 of this year, shocked the deep learning community because it was much less expensive to create and is much less expensive to use, compared to all other current LLMs. According … Continue reading

Posted in PyTorch | 3 Comments

An Experiment with Applying Attention to a PyTorch Regression Model on a Synthetic Dataset

The goal of a machine learning regression problem is to predict a single numeric value. Classical ML regression techniques include linear regression, Gaussian process regression, gradient boosting regression, and others. I’ve been experimenting with a new algorithm for machine learning … Continue reading

Posted in PyTorch, Transformers | Leave a comment

Explaining Why PyTorch Multi-Class Classification Neural Networks Use NLLLoss (Negative Log-Likelihood Loss)

I was preparing to teach a class on PyTorch neural networks at the large tech company I work for. To prepare, I wanted to mentally review PyTorch neural network basics, including the mysterious NLLLoss function. 1. Early Days: One-Hot Targets, … Continue reading

Posted in PyTorch | Leave a comment