Suppose you have some system that predicts the ordering (ranking) of items and you want to compute a measure of prediction accuracy. It’s a bit of a tricky problem. One approach is to use what’s called Discounted Cumulative Gain (DCG).

In my example below, there are six items to rank. The correct ordering is (1, 2, 3, 4, 5, 6) and the predicted ordering is (2, 1, 3, 4, 6 5) which is pretty close. Using DCG, the prediction accuracy is 0.8745.

First you compute DCG for the correct (ideal) ordering. We assume that 1st place has “relevance” 6, 2nd place has relevance 5, and so on. Then DCG is sum[ 2^rel-1 / log2(i+1) ] = 94.5903.

Next you compute the DCG for the predicted ordering using the relevance values. Here that value is 82.7188.

The final accuracy metric is the DCG of the predicted ranking divided by the DCG of the ideal ranking = 82.7188 / 94.5903 = 0.8745. This is called the normalized DCG, or nDCG.

Beauty contests are an example of an applied ranking problem. Here are seven participants in the 2018 Miss Universe contest. From left: Miss Canada, Miss Poland, Miss Philippines (she won the event), Miss Switzerland, Miss Columbia, Miss Bolivia, Miss Kyrgyzstan. Beauty is subjective so I don’t think DCG applies here.