Suppose a group of people each rank their preference of a set of options, from best to worst. The Kendall tau distance is a metric that compares how close any two sets of rankings are. If the K-t distance for two rankings is 0.0 then two rankings agree exactly. If the K-t distance is 1.0 then the two rankings have maximum disagreement.

Here’s an example. Suppose Joe and Ken list their preferences of five cities, from best to worst. The raw lists are:

   Joe's    Ken's
===================
1. Camden   Austin 
2. Austin   Eureka
3. Denver   Boston
4. Boston   Camden
5. Eureka   Denver

And so the rankings (1 = best, 2 = second best, etc.) are:

         Joe  Ken
==================
Austin    2    1
Boston    4    3
Camden    1    4
Denver    3    5
Eureka    5    2

To compute Kendall tau distance you look at each possible pair of options for each ranker, count the number of times the rankings differ, then divide by the total number of pairs:

        Joe   Ken   Differ?
============================
AB      >     >     
AC      <     >     x
AD      >     >
AE      >     >
BC      <     >     x
BD      <     >     x
BE      >     <     x
CD      >     >
CE      >     <     x
DE      >     <     x

The AB > > entry means both Joe and Ken agree that A(ustin) is better than B(oston). The AC < > entry means Joe thinks A(ustin) is worse than C(amden) but Ken thinks A(ustin) is better than C(amden). There are a total of ten comparison pairs and Joe and Ken disagree on 6 so the Kendall tau distance between their rankings is 6 / 10 = 0.60.

Notice that if Joe and Ken agreed perfectly, their would be no pairs where they differ and so K-t would be 0 / 10 = 0.0.

I don’t use Kendall tau distance very often but it’s a nice metric to remember whenever you have rankings of items.

Many rankings are subjective. Miss America 1946 contest and Miss America 1985 contest. I wonder what the judges’ Kendall tau scores were.