Sometimes it seems that a new state ranking comes out every week. State business competitiveness ranking reports have become a popular tool for media, think tank and research center websites to attract viewers and users. Lists and rankings are popular in our new media world, as they work well and attract clicks for websites, drive social media traffic and are good fodder for opinion and debate focused news programs. These rankings, built on compilations and combinations of indicators, result in news stories touting the “best states for…” or “worst states for…” and the subsequent crowing or explaining depending on where your state ranks.
The rankings may be used to make political points or to support certain policy positions, either by the organization promoting the ranking or by advocates and critics in the state. Policy analysts also dive deep into the methodologies, to determine whether there is a relationship between the cumulative ranking or portion of the rankings and reality as they perceive it on the ground in the state. Importantly, site selection consultants do read many of these rankings, and may rule a state in or out depending on its performance. Finally, the media love rankings. They are great click-bait for internet sites and provide for provocative headlines and opportunities to interview people who agree or disagree with the ranking.
The variation in rankings is stark in some cases – for example, California ranks first in the US News Business Environment sub-index ranking, but 49th in the Tax Foundation Business Climate Survey. Similarly, Texas is second in the CNBC State Business Rankings while 38th in the US News Best States ranking. This demonstrates that differences in indicators chosen, weighting or indexing of those indicators, and even the issue orientation of the organization producing the ranking can make a significant difference in the final ranking scores and comparisons. Sometimes, there is even great contradictions between two different rankings on what appears to be the same area of focus.
Like public opinion polling, there is both art and science to the development of a state ranking. Which indicators are chosen, how they are combined into indices, and how they are weighted can have a significant impact on the resulting ranking. The methodology can reveal a great deal about the position of the sponsors of the ranking, particularly for those with a narrower focus.
So, for state policy makers and analysts, as well as media, pundits and the general public, it is important to understand the who, what, when, and why of the ranking before immediately reacting to either good or bad news from a ranking report. When reading any report, it is important to not just look at the ranking, but to review the methodology and then answer the following questions:
WHO compiled the ranking, and what is their general viewpoint on business competitiveness issues? Are they an advocacy organization for a specific viewpoint or industry, or are they generally neutral in their viewpoint? What is their reputation for accuracy, what are their credentials, and how long have they been doing this?
WHAT is the ranking trying to measure? How broad or narrow is the focus, and what is their general message on what makes for a good ranking and what makes for a poor ranking? What are the components of the ranking, how are various measures weighted, and how relevant are they for the state?
WHEN was the ranking completed? In the always-on internet era, ranking reports exist forever, so understanding the timing of the ranking – and the dates of the underlying ranking indicators – can make a difference in how to respond to a ranking. For example, if a ranking relies off federal data that is only collected periodically, recent policy actions could have happened after the data was collected. In another example, the ranking may have been compiled under a different administration or set of policies and allow for easy explanations as to new approaches and hoped for outcomes.
WHY does a state rank as it does? Which indicators or factors make the most difference in the ranking, and how do those relate to your state? For example, rankings that put a strong emphasis and weight on historic population growth are likely to result in a lower ranking for states like Pennsylvania, Ohio or Michigan, while those that emphasize university research funding may end up with higher rankings for states with strong University research centers, like Massachusetts, California or North Carolina.
Once you have answered these questions about a report, consider a few questions:
- How do you compare to the states with which you compete? Most states don’t compete with all fifty states – they are in true competition with a defined set for business and talent attraction purposes. Focus on those, and see how you stack up.
- What does the report say you are doing well? Most of the reports are broken into multiple sections and the overall score is a composite of many subsections. Emphasizing where you do well can lead to policy actions that allow you to you do more of that, or you can promote that aspect of your state more effectively and broadly, citing the report as a source.
- How has your ranking changed over time and what can you do about that? Understanding where you’ve fallen or where you’ve improved can allow you to learn how both the perception and reality of your state’s competitiveness has changed and give you important information to use in potential policy and legislative strategies.
By closely examining the who, what, when and why of ranking reports, and then using the report to learn more about your state and where it shines and where it might need help, you can turn the release of yet another ranking into an effective tool for state improvement.
Steve Wray is Vice President of Strategic Initiatives at ESI. In this role, Steve develops, oversees and helps implement programs and projects that support ESI’s vision and short- and long-term plans. He supports the work of the firm in developing new partnerships, expanding and building on existing practice areas, and integrating the firm’s strengths in economic analysis and thought leadership.