The Myth Of The Statistical Tie

-Submitted by David Drumm (Nal), Guest Blogger

This campaign season we’ll hear a lot about “statistical ties.” The “statistical tie” misnomer is used to refer to the situation where one candidate leads another candidate but that lead is within the margin of error (MOE). However, what we’re really interested in is the probability that one candidate leads the other candidate.

Since polling all voters is a costly and time consuming process, a random sample of voters is selected and, based on some assumptions, one can make a probabilistic judgement regarding the outcome of an election. Polling all voters yields the “true” percentage while the random sample can only estimate the “true” value.

Every time a sample is taken, a different (perhaps) estimate of the “true” value is obtained. The estimate plus and minus the MOE is called the confidence interval. A 95% confidence interval says that 95% of the sample estimates will lie within that interval. Also, a 95% confidence interval says that we are 95% certain that the “true” value lies within that interval.

Consider two candidates, Smith and Jones. We want to know what percentage of the voters prefer Smith and what percentage prefer Jones. The “true” percentage is unknown so we randomly sample the population to obtain an estimate. We want to be 95% confident that our estimate lies within the “true” percentage plus-or-minus three points. The figure at the right determines the sample size for different MOEs. Sample sizes usually assume an infinite population and a correction factor is only required when dealing with very small populations.

After our poll of 1067 likely voters, Smith leads Jones 49-46% with an MOE of 3%. Some would call this a “statistical tie” since the lead is within the MOE. However, the following table tells us that Smith’s lead is 84% probable.

H/T: Kevin Drum, Fritz Scheuren.

45 thoughts on “The Myth Of The Statistical Tie”

  1. Tony C. 1, October 7, 2012 at 3:30 pm

    What exactly would you propose as an actuarial approach to predicting the outcome of the election that does not involve polling people’s opinions of the candidates or their policies?
    ==========================

    Ask Al Gore. He got robbed.

  2. @Matt: Who are you yelling at? If it is me, a friend of mine from college became an actuary for an insurance company; he has no problem at all with political polling, and reads Nate Silver’s analyses regularly.

    What exactly would you propose as an actuarial approach to predicting the outcome of the election that does not involve polling people’s opinions of the candidates or their policies?

  3. The “doodling in math class” is a highly sophisticated production, one of many, by professional mathematician Victoria “Vi” Hart. It should surprise no one that she is the daughter of well known sculptor and mathematician George W. Hart.

  4. Do you know what actuaries are? They don’t rely on polls. Maybe you should get your head out of your ass.

  5. @mahtso: The selection is random; typically it is literally random numbers being dialed until somebody answers.

    A valid reason for weighting is that one can discover, from previous polls, that for whatever reason a definable segment of people do not participate as much in the polling as they do at the voting booth.

    For example, some segment may screen calls more than others, or may have only cell phones.

    So if you find out from past elections that your polling method under-selects Democrats and over-selects Republicans, you can find a weighting for the people you DO choose that brings the polling for those past elections as closely as possible into line with the actual outcomes; and apply that to the current polling. So you might increase the weight of the few young respondents you have, decrease the weight the elderly that are more likely to have land lines and free time to participate, increase the weight of the few working mothers that found the time to respond, and so forth.

    Unless the pollster is corrupt the weighting is not done in the interest of biasing the outcome, it is done in the interest of using past polling outcomes and experience to better predict the outcome.

    Built into those weights will be the implicit adjustment for people that say they intend to vote but do not, and vice versa.

  6. “Since polling all voters is a costly and time consuming process, a random sample of voters is selected and, based on some assumptions, one can make a probabilistic judgement regarding the outcome of an election. Polling all voters yields the “true” percentage while the random sample can only estimate the “true” value.”

    Are the samples actually random? I’ve seen reports that some of the recent polls have taken 10% more of one party than the other, which is not likely to happen in the presidential election.

    For polls before the election there are no “voters.” The statistics are not necessarily valid for reasons including that people may say they intend to vote, but then do not.

  7. Doodling in math class…..

    Are those really the words of young student?
    If so what is that person doing there? Why is he/she alone solving the integration of quantum mechanics with the general theory of relativity?

    David, what is a debit actuary card? Never met one here in Sweden. How old are you. I’m 76.

    Otteray,
    Thanks for the “doodling”. It was a strain to follow, but more inspiring than gaelic songs. Just to show I don’t praise you for all you do.

    To compare:
    The gaelic I heard expressed feelings round a warm fire when thoughts gone youth are recalled and deeds of ancient heroes come again to be told. Melancholy rules.

    The african one (the first by Malisha), echoes the cells of my body in their working together, they say, we can do it, and only when we srive together. And then my mind chimes in with a song in a minor key, remorse again…..we all die, in spite of our strivings.

  8. Oro Lee,

    And I who thought all women liked beer. I knew the ones who went out with me liked sex. They might drink white wine at home when they were in a prissy mood. And that could be fun, when her housemate goes out and returns in a topless bathing suit.

    Thanks for letting your hair down. So few do.
    I think it must be due to my constantly baring my bottom.
    Yours deserves a repeat:

    “Oro Lee
    1, October 6, 2012 at 9:48 pm
    The only statistic I ever paid much attention to is that a girl who says she likes beer is more likely than others to have sex on the first date. How much more likely, I’m not sure.

    My own experience indicates the difference might be from no way in hell to only if tomorrow is the end of the world.”

  9. OS

    watching “doodling in math class” makes me happy to know that teaching to tests aren’t completely killing this generations imagination.

    hope springs eternal.

  10. PPS. I aint 78 yet, that’s just what my debit actuary card says.
    Math is confusing but it does have logic. Some do, some don’t.
    My first doodle…..

    “Bitterness and Spite, .. Only its Owner….. Truly bites”.

    My opinion and experience, this is true

  11. David, I think you mean Piet Hein. Yes.

    Sometimes I think Vi Hart is channeling Piet Hein.

  12. OS, are you familiar with Pete Hein and GROOKS. He is awesome.

    Doodling should be encouraged for everyone. ….. Unless they are driving over the edge. Thank you, doodling is a great equalizer. …. Unless one is ANTI doodling …. then NO ONE suspects the anti doodling Inquisition.

  13. One of the best life actualizations I ever experienced …. is my infallibility, or lack of it. I am wrong often, Yahoo, I am not perfect. I can only admit to my opinion, I can learn and embarrass myself.
    Embarrassment is not utilized enough. Ego often prevents acknowledging it. I’m right your wrong,….. and I will defend my wrongness to prove it.
    Learning is a human tool. Learning wrong is a human condition. Admitting wrongness is embarrassing and a noble trait. Then I can move on to future errors and admittance of them.
    I would rather live and be fallible and learn, than live and know it all.

    F all the know it alls my new bumper sticker. F-all-the knowitalls.

    PS OH OH…………

  14. The margarine of error is how much transfat it has in it. Oh I don’t believe it’s butter, has been shouted to me on TV numerous times. If preachers got in their pulpit and professed IT IS REALLY BUTTER !! perhaps my choice would have been easier.

    Alas, my taste buds are my own. ….. Margarines of errors are left to the palate, until the final vote is counted. Too often, or always, it is the influence of environment that guides my pull, on the poll, at the poll.

    I want to vote for the equality of all humans in the human gene pool.
    Ghandi is dead, Christ is dead, Crist is a lobbyist and a health care multi millionaire. What’s a human to do…..?
    If profit at the expense of the equal substrate of the human race is legal where is the balance? My equal substrate allows me to undermine your equal substrate. My ability to bedazzle and befuddle you makes undermining your equality a profit motive. Well Yahoo, undermine away.

    This is a choice we all face. I can beat you, or I can be with you. Or I can be beaten by you. Baa Baa any sheep, do you have any wool ? Yes Mam’ Yes Sir’ 5 bags full.

    In my 78 years of life I will F- you …so my 78 years will be more comfortable than yours.
    Here’s a peanut be happy.

    Or we are all Equal…….. except for the HATE we chose to believe.

    How often is hate a disease that allows me to be better than you.
    PS. My left hand and right hand equally typed this rant. I don’t know which one is screwed up. Anyone think it was the left one ?????

    OH OH … I meant to press delete, but it seems I pressed post.
    …..again.

  15. The only statistic I ever paid much attention to is that a girl who says she likes beer is more likely than others to have sex on the first date. How much more likely, I’m not sure.

    My own experience indicates the difference might be from no way in hell to only if tomorrow is the end of the world.

  16. Go to Florida. Ask them how it works out. Is Jeb Bush still there? Don’t ask Al Gore. I would have made them pay.

  17. You also have the fact that the question is repeated so often, that one can combine polls.

    The number of samples required to meet a particular MOE is very close to 1/(MOE*MOE); so for example, to get to 2%:
    1/(.02 * .02) = 1/(.0004) = 2500. (2401 is the more accurate number, but this is close enough).

    Thus, if you are given the MOE, you can compute how many people were in the poll, and from the results compute how many said Romney, and how many said Obama. Then you can add up polls and have a “larger” poll, with a much smaller margin of error (which is computed as approx 1/ sqrt(N), where N is the total number of people surveyed.)

    More intuitively: After you see nine polls with basically the same result, the MOE is divided by 3.

Comments are closed.