GOTV - Part 11 - Bayesian modeling

by: Grebner

Wed Feb 09, 2011 at 04:51:44 AM EST

(Today's title was selected to scare away everybody but the hard-core.)

250 years ago, Thomas Bayes figured out a fundamental, but very hard to understand principle of probability theory, which has become known as Bayes' Theorem.  I'm not going to prove it, explain it, or even apply it in this essay.  Instead - like everybody else who claims to apply Bayesian theory - I'll just refer to it in a hand-waving way, and treat it as an inspiration.

Bayes figured out exactly how observing something should affect our understanding of the world.  First, he says, you have to believe something. It doesn't matter where that belief comes from, or even whether it's true or false, but you can't get anywhere if you don't have a basic idea of the world.  For example, in October 2009, I would have said "Dave Bing has about a 98% chance of winning re-election as Mayor of Detroit, and Tom Barrow has only a 2% chance".  That's called an "a priori estimate".  (Latin!)

Second, you observe an "event" which might occur in two or more different ways, which serves as evidence to strengthen or weaken your belief in each of the possible outcomes.  In this case, it was a poll by a previously unknown polling firm, showing Barrow with a wide lead:

 (http://www.michiganliberal.com/diary/15633/new-detroit-mayoral-poll-probably-fake) 

Third, we perform a set of calculations which depend on the probability connections between the original question (which candidate will win) and the evidence (the outcome of the poll).  The outcome is an adjusted probability.  In this case, because I was so sure Bing was leading, and because the poll seemed so flaky, I only slightly altered my opinion - in this case it was just enough to cause me to run a poll of my own, which proved conclusively to me that the Barrow poll was fraudulent - which it turned out to be.

In short, Bayes figured out exactly how to evaluate statistical evidence - his work has stood the test of time, and his insight permeats the physical and social sciences today.  If only he had figured out some method ordinary people could understand and apply!

Now, to apply Bayesian thinking to GOTV. 

Bayes first point is easily overlooked:  if we're going to study GOTV - or anything else - in a scientific way, first we need to assign baseline probabilities to each possible outcome.  Then we apply varying treatments, and evaluate how the outcomes compare to our initial guess.

Concretely, PPC (my firm, Practical Political Consulting, Inc.) assigns to every voter in Michigan before each election a probability that they will cast a vote.  Our initial assignment happens to be based on a mathematical model we've designed over the years, but it could just as well be based on guesswork.

About 60 days after the election, we obtain the actual turnout data from the Secretary of State, and we begin to explore who turned out better than we expected and who disappointed us.  We consider where our model was wrong, what unexpected phenomena drove some people to the polls, and which experimental interventions suceeded.

And our predictions aren't just who will vote, but such things as whether they'll use absentee ballots, and whether they'll vote Democratic.  (This latter, of course, can't be confirmed by using voter history data, but requires other sources, such as polling.)

For me, Bayes' essential point is the initial assignment of probability; figuring out how to adjust it later isn't hard.  What's hard is the beginning.

Let's use the recent gubernatorial election to illustrate.  To begin with, if Michigan has 7 million voters and we expect 3.5 million turnout, we could simply guess that each and every voter has exactly a 50% probability of voting - that's not very clever, but it's a place to begin.  But let's propose a second model:  that everybody who voted in 2008 (5 million) has 70% chance of voting, while the two million non-voters in 2008 have each a 0% chance of voting.  Using the turnout data from the 2010 election, we would quickly find out that this slightly more sophisticated model was more accurate in picking out the individual voters.

Now, let's add another variable:  voting in the 8/2010 primary.  Let's guess that the 1.5 million who voted in that election would have a 100% chance of voting in November, the remaining 3.5 million from 11/2008 would have a 50% probability, and we'll assign a 12.5% probability to the people who failed to vote in either 8/2010 or 11/2008.  This model turns out to be better still.  

We don't have to guess in order to create these models; there are statistical packages that can select the most powerful package of predictors when looking at previous elections, and we can then apply what we learn to the most recent problem.

Once we start, we can find thousands of questions to ask.  Where did we fall down?  Where did we succeed beyond our hopes?  Where was the opposition more effective than we expected?  Which of our canvassers turned out voters who would normally have slept in?

This process - guess, test, and then evaluate the results and update your guess - seems perfectly obvious.   But does anybody in politics actually do it?  Do we want to optimize our effectiveness over time, or only to repeat whatever wisdom we acquire from the past?  Are we driven by science or myth?

[Apology to Nazgul35: I know this wasn't what you meant when you repeatedly complained that I wasn't applying Bayesian methods.  Sorry.]