Warning: Be Wary of State Economic Development Rankings, Study Finds

/

New analysis from a national research organization is casting considerable doubt on the validity of the steady stream of state economic development rankings that routinely appear in the media.  State economic rankings cannot be taken at face value, according to an article published online by the Journal of Applied Research in Economic Development.

"How Can I Create My Favorite State Ranking?" points out that such rankings typically are subjective and often offer little meaningful information, despite their pervasiveness in the media, government and among economic analysts.

"Often, these ranstateskings are plagued by lack of clarity about what is being measured or whether the ranking accurately assesses what it claims to measure," said Yasuyuki Motoyama, Ph.D., senior scholar at the Ewing Marion Kauffman Foundation, who wrote the article with Jared Konczal, a senior analyst. "Further, the connection between these indexes and actual economic growth and performance at the state level is ambiguous, at best."

The proliferation of state rankings, the authors point out, can lead policymakers and economic development consultants to misuse them, either celebrating a conveniently positive ranking or initiating efforts to address a poor ranking, when neither action may be based on valid economic indicators.  The Kauffman Foundation website points out that the authors conducted a series of exercises to demonstrate how such rankings can be manipulated, using eight state-level indicators:

  • Self-employment rate
  • Kauffman Index of Entrepreneurial Activity
  • Business startup rate
  • Ratio of science and engineering bachelor degree holders to the total population
  • Patents per science and engineering workforce
  • Venture capital investment over Gross State Product
  • Research and development expenditures
  • Number of high-growth Inc. 500 firms  

The indicators were chosen because they typically are used to measure entrepreneurship and innovation, two vital indicators for every state's economic health. The subjectivity and limitations of state economic rankings led to four observations, according to the article:

  1. Policymakers should not rely on a single indicator to gauge economic conditions.
  2. Aggregating indicators does not provide solutions because indicators are highly variable.
  3. Policymakers should not focus on improving their states' rankings because the rankings lack meaning.
  4. Rather, they should employ a scorecard approach, which does not create a normative, quantified measure, but descriptively assesses various conditions of each state.  

Demonstrating that rankings can be developed in ways that favor any given state, the Kauffman researchers created a simulation analysis with randomly generatedkauffman-details-logo weights, which revealed that, among 1,000 different scenarios, five states were eligible to be No. 1, 16 were eligible for the Top Five and 22 could be ranked in the Top 10.

Depending upon how a range of variables were weighted, Connecticut placed in the top 10 in 41 of the scenarios, 17th overall among the states.  (Just ahead of Texas; just behind Maine).  The full report can be viewed on Journal of Applied Research in Economic Development. website.