I contributed to an article titled “Contrastive Loss Representation for Anomaly Detection Has Cybersecurity Implications” in the May 2022 edition of the online Pure AI Web site. See https://pureai.com/articles/2022/05/03/anomaly-detection.aspx.

The article describes a type of neural network architecture called contrastive loss representation (CLR). CLR was originally designed for image data, but the article describes how the technique was adapted for use with log files, for cybersecurity purposes.

Briefly, contrastive loss representation for image data accepts an image (such as a 32 x 32 color image of a dog) and generates a numeric vector that is an abstract representation of the image (such as a numeric array of 500 values). The abstract representation vector can be used for so-called “downstream” tasks such as creating an image classifier, with only a very small number of images that are labeled with the correct class.

The article first describes CLR for image data. The CIFAR-10 (Canadian Institute for Advanced Research, 10 classes) dataset has 50,000 training images and 10,000 test images. Each image is 32 x 32 pixels. Because the images are color, each image has three channels (red, green, blue). Each pixel-channel value is an integer between 0 and 255. Each image is one of 10 classes: plane (class 0), car, bird, cat, deer, dog, frog, horse, ship, truck (class 9). Using all 50,000 training images it’s relatively easy to create an image classification system that achieves about 90 percent accuracy.

Suppose you want to create an image classifier for a new dataset of 32 x 32 images where each image is one of three classes: bicycle, cow and rabbit. You only have 100 labeled training images for each class. If you create an image classifier from scratch using the 300 training images, your classifier will certainly have poor accuracy because you just don’t have enough training data.

However, your (bicycle, cow, rabbit) image data is similar in some intuitive sense to the CIFAR-10 image data. If you could construct an internal representation of the CIFAR-10 data, there’s a good chance you could use that representation to jump-start an image classifier for your data and get good accuracy even though you have a very limited amount of training data.

I am quoted in the article:

McCaffrey commented, “Applying contrastive loss representation to non-image data is a straightforward idea so I’m not surprised that the technique appears to work well.”

McCaffrey further observed, “Deep neural systems have made fantastic progress in many areas, notably natural language processing. But one area where these deep neural systems have not quite met expectations is in cybersecurity.”

McCaffrey also noted, “This research, and many other efforts, seem to be making good progress toward our ability to detect and defend against malicious attacks on computer systems.”

An Internet search for “contrastive” led to a high contrast portrait (left). This led to a “gel” portrait (center). And that led to a “jello hat” portrait. Thank you Internet, for endless entertainment.