Take a number – but proceed with caution
So, Spokane County residents use twice as much water as Washington residents in general. Of course there are explanatory circumstances, as many recent writers have pointed out: the climate’s drier here, we have larger lawns, we aren’t brainwashed socialists yet, etc.
There’d be little news and less talk without statistics, and in recent weeks I’ve been keeping track of how many and what kind of letters the S-R has received and printed. June is perhaps our slowest month for letters received, so it was a good time to warm up a new spreadsheet without being overwhelmed with data.
Since my only technical expertise in this area comes from a single semester of elementary statistics in college, my experiments in gathering statistics (this is my second shot at it) have been a crash course in the pitfalls of using and collecting them carelessly.
If my experience is any gauge, the vast majority of statistics are boring. The numbers I present here are, frankly, pedestrian. I can’t point to any striking aberrance that exposes a critical problem or solution affecting millions, thousands or hundreds around the globe.
I suspect most statistics are like that until a spin doctor gets a hold of the data. Watch a few hours of football and you’ll see the hay a good announcer can make from the fact that Receiver One is the first player to have at least two catches in eight consecutive games against the same team since Receiver Two did it eight years ago. With enough data-combing, couldn’t something this speciously important be said about any player in any game?
Some pitfalls of stats appear before the data are even gathered, in the rules by which the gatherers play. Sorting letters according to topic and content, I found that even a conscientious gatherer constantly makes subjective judgments that could affect the results.
A letter about who an elected Barack Obama might appoint to the Supreme Court, for instance, could defensibly be labeled “national politics,” “presidential election” or “courts.” Better or more specific classifications might eliminate some guesswork, but if they’re too specific they don’t mean anything either; there would be one letter in each. And if I had an interest in skewing the stats, I could do so without being particularly conscious of it just by how I choose to classify gray-area letters.
For similar reasons, the numbers in my report only roughly represent the reasons we don’t use a large part of the material that crosses the letters desk. If a letter is too long and also contains plagiarism, choosing one or the other as “the” reason we didn’t print it is misleading. But the obvious solution, listing and tracking more than one choice in each category, introduces a level of complexity to the process that I simply don’t have the time or training to handle.
Without knowing the methodology of any particular study, it is difficult to know if the same kinds of constraints affected its findings. A good statistical presentation can account for many of the red flags my amateur experience raised, but in general we rarely take the time or trouble to examine how the facts were gathered and how that might affect their interpretation. I’m more suspicious now of polls, surveys and research for which the methodology is not made available – especially, of course, when the source is a group with a vested interest in the results. Let the reader beware of subtly or not-so-subtly loaded questions.
In an extreme example of false dichotomy, one recent anonymous letter wrapped up a scathing critique of a national corporation thus: “This letter has been mailed to every single newspaper in U.S.A. … The only reason why you would not publish this will be either you receive some kind of kickbacks from this company or the Press is full of … Cowards.”
My guess is this writer has a job as a push-pollster: “Would you be more likely to support (A) lower taxes, better education and a guaranteed retirement or (B) the incumbent – who, by the way, once vivisected a bunny rabbit?”
When you pose these kinds of either/or questions, you’ll probably get the answers you’re hoping for. Unfortunately, you won’t get answers that mean anything.
Many reactions to the mayor’s watering plan may have been knee-jerks, but I’m encouraged by debate over what numbers mean or don’t mean. Approaching statistics with a healthy skepticism can guard against distorted claims to which numbers lend false weight.