Thursday, January 23, 2014

In which we engage in "voodoo math"

Two true stories.

A judge’s paradigm reads that he will not consider Theory arguments. My debater, who enjoys Theory arguments, knows better than to employ them in this round facing this judge. The opponent runs Theory, and my debater loses. The RFD? “I vote on Theory.”

More interesting: A panel is set. It’s 1/2, 1/1 and 4/3. It is balanced, but not great. One side comes to tab and asks if we can do any better. Not with mutuality, no. The best we could do is another 1/2 against their team. The protesters don’t mind that. They’d rather have a 2 judge than this particular 3 judge, and in fact, would prefer this even though they’ve now given the mathematical benefit to their opponent. Personally, I’m against it, but it’s their tournament, and the tab room decision is that this is now one-sided, since only the protestors are in the tab room, but if the other debater agrees, then it’s OK. I can live with that, so we go to the other debater. He is adamant: he wants the judge originally given to him by tab, even though the new judge would be a 1 for him instead of a 4. So, without agreement from both sides, the round stands. It’s a 2-1 decision for the adamant debater. The ballot he didn’t pick up is from the 4/3, or put another way, the judge who was protested against was the only ballot for the protestors. So much for the math.

I’m making two different points here. First of all, judges just do whatever the hell they do in any given round, and most written paradigms are just so much bush-wa. As a matter of fact, I would go so far as to say, the more specific the written paradigm, the more bush-wa within. Personal experience of a judge may be useful, because you can get a sense of some general things they might like or dislike, but even that goes only so far. Here’s a test. Before you next see your family, predict their answers to a question like, “What three albums would you take to a dessert island?” Your family now, your flesh and blood. Can you do it? I’ll bet you can’t. Better still, have them tell you their 3 dessert island albums, then predict in advance, from those answers, how they would feel about some album they’ve never heard before by someone they’ve never heard of. How do you think that’s going to work? Well, it’s the same as predicting how any judge will vote in any round. You might be able to do it, but how well? Are you willing to put money on it? Every time? How do you think that applies to the predictability of a judge’s vote in a debate round?

The second point is that, given the first point, organizing the potential judges into a mathematical equation is, at best, only marginally predictive. If you attach a number to a judge, that number is not the likelihood that that judge will or will not pick you up: it is your guess about the likelihood of whether that judge will or will not pick you up. Given that in any particular debate round you are not worried about trends, metadata or quantitative analysis of any sort across a pool of judges but instead in the very specific results of that round and the vote of one specific judge, this is, to paraphrase George H.W. Bush, voodoo math.

Judges are unpredictable, and cannot be ranked according to whether they will vote a certain way. A ranking of the judges is nothing more than a lot of wild-ass guessing. Assigning judges based on that ranking is, therefore, based on wild-ass guessing about unpredictability. The only benefit of MJP in this completely bogus mathematical wonderland is that it finds a match between two teams’ wild-ass guessing about unpredictability.

So we can now translate the statement, “I got a 3-3 judge,” into the more precise, “My opponent and I, based on the flimsiest of evidence that cannot predict outcomes, identically ranked the probability of one of those outcomes.”

MJP is a tool for matching teams’ opinions. No more, no less.

No comments: