I’m a self-professed data geek and I love the saying, “In God We Trust. All Others Bring Data!” For most of our projects, we rely on US Government statistics like those from the Census Bureau, Dept of Labor and Bureau of Economic Analysis. We augment those data with structured interviews, a method borrowed from case study analysis to gain detailed and nuanced information. Often, we also use survey data – in a current project we are using a survey to gather data about alternative concepts – a way to get information directly from a targeted audience.
But not all data is created equal. There’s another great saying: “”There are three kinds of lies: lies, damned lies, and statistics.” I think it’s really important to view any data with a critical eye. For instance, I recently ran a report on venture capital investments in biotechnology companies in Maine. At first glance, it looked like amazing news – there was one year recently with more than $16 million invested! When I dug deeper, there was just one investment, for $16 million. That leads to a completely different finding.
Another example are all of these indexes and cross-state comparisons that dominate the Internet. We’re click happy for anything that says “My State” is First in Anything! Or the Top 10 Anything! You want to dig into these to find out how solid is the data they are based on, how do the authors define the category, and how are various components weighted. So, a recent Kauffman Foundation finding that Maine was #1 in the country in first-year survival rate for entrepreneurs sounds really good. Kauffman calculates this rate from data that originates from the ES202 forms that all firms submit quarterly, a very robust source. However, a look at Maine’s ranking on this measure over the past twenty years shows that the 2017 number of 88% is a blip, not a sustained measure (yet). So, best to take it with a grain of salt.
My overall advice: Run the numbers. But understand where they come from, what assumptions are being made, and how sensitive your decisions are to any one data point.
Are you ready to learn more about innovation and entrepreneurship-focused economic development? Get started with this complementary pdf.