Kind of obvious, right?
But I recently did choose to use data and analysis from an advocacy group—one that I strongly disagreed with no less.
My project was to research and find out how many Americans were eschewing driving cars and choosing public transportation instead. I came across a comprehensive survey created by the American Public Transit Association (APTA) that found that Americans took a record 10.8 billion trips in 2013, "the highest annual ridership in 58 years” the subtitle of the report read. According to its President and CEO the survey illustrated “a fundamental shift going on in the way we move about our communities…”
Then I found some additional commentary, on a blog that challenged the APTA’s notion that transit use was truly increasing. I quickly scanned down the page to find the source and discovered that it was The Heartland Institute—an organization with a “libertarian and conservative” agenda to “promote public policy based on individual liberty, limited government, and free markets.” I was, in fact, already familiar with and quite unsympathetic towards this group. For years it has actively been trying to convince the public to discount human created climate change, as well as pursue other politically “on the right” points of view that I personally do not ascribe to.
So my immediate reaction was to dismiss this blog piece as just more propaganda. But I decided to suspend judgment a bit, and see what else my research turned up and revisit the site further down the road. So later when it was time to pull together my research, I did take the time back to read the full blog piece. And what I found was in fact a convincing argument.
The reasons that I found the piece convincing were:
- The data it used to back up its points were from neutral and respected sources, including the Bureau of the Census, and the article linked back to those original sources.
- The argument that disputed the conclusions of the APTA study was logical, clear, and easy to understand.
- The tone of the piece itself was informative, and did not call on, refer to, or mix up its larger value related viewpoints with the data and argument.
Heartland’s argument itself was actually quite straightforward: while it was true that transit ridership was at an all-time high as the APTA data showed, that organization failed to put its data into two very important pieces of context: changes in population and mass transit use by one particular city. On the first point, the blog provided reputable data that between 2008-2013 the total US population rose from 304 million to 316 million. Once that data is taken into account, annual public transportation use, on a per person basis, actually falls from about 35 trips to 34. Furthermore, if one takes out New York City’s contribution—which represents 40% of all transit riders—American public transit use on a per capita basis actually falls for those years.
But although this new information was extremely valuable in putting the APTA data into the missing and necessary context, it still is just one more piece of data and still does not tell the “whole story.” Further research uncovered larger, macro trends supporting likely ongoing growth of public transportation in the U.S. due to forces like aging boomers looking to move to urban centers with nearby businesses; millennials lack of passion and interest in driving; and a renewed increase among certain cities towards building new light rail and public transport options, among other relevant trends. The President and CEO’s claim that Americans were in fact fundamentally shifting their way of travel looked to be correct, but not because of the total rise of transit use found in its survey.
That’s why any good researcher will find and evaluate multiple sources of information—including data from advocacy groups—and ultimately make his or her own conclusions.