When I was a university professor, one of the most difficult topics I had to explain was the connection between logistic regression and log odds. There are several interrelated ideas, and although each idea is quite simple, when you combine them it becomes a bit difficult to understand the whole concept.

First, you have to understand odds. Suppose you have a binary problem where an outcome can be a success, with probability p, or a failure with probability 1-p. Suppose you roll a dice and success is getting a 1, 2, 3, or a 4, and failure is getting a 5 or 6. Therefore p = 2/3 and 1-p = 1/3.

The odds of success are p / (1-p) = (2/3) / (1/3) = 2 to 1 (often written 2:1). Notice that if p is less than 0.5 then the odds will be x to 1 where x is a number less than 1. If p = 0.5 the odds will be exactly 1:1. If p is greater than 0.5 then the odds of success will be x to 1 where x is a number greater than 1.

Now for logistic regression, the probability of a success is e^z / (1.0 + e^z) where z = b + (w0)(x0) + (w1)(x1) + . . . A minor problem is that it’s hard to interpret the wi weights — about all you can say is that an increase in a weight increases z by some amount.

But if you do a lot of algebra you can solve for 1-p and then 1 / (1-p) and get the odds of success are just e^z to 1. The e is a bit annoying so if you take the log of both sides you get log(p / (1-p)) = log(e^z) = z.

The point is that the b and w0, w1, etc. weights can now be (sort of) interpreted — as the value of a wi increases, the value of the log odds of success increase proportionally. (By the way, log odds is also called the “logit”).

Well, that’s all fine and dandy but not particularly useful in my opinion. The moral is that there is a mathematical connection between logistic regression prediction and log odds. But in my opinion, this connection isn’t very useful from a practical point of view, and is sort of an explanation in search of a problem.

Note that you can also use the idea of log odds to generate the logistic regression equation. If you assume log(p / (1-p)) is linear, i.e., log(p / (1-p)) = b + (w0)(x0) + (w1)(x1) + . . . then if you solve for p you get p = e^z / (1.0 + e^z) however, again, in my opinion this is interesting but not very useful in a practical sense.