U.S. News & World Report released its annual compendium of rankings of undergraduate colleges and universities across the nation last week. This is usually an eagerly-awaited event at many institutions around the country, and one with which most of us in leadership positions have a love-hate relationship. On the one hand, we all decry how the process attempts to reduce what are in most cases large, complex, and multi-mission institutions into a single number. On the other, we all recognize the attention it generates and the potential impact it can have on the decisions of millions of college-going students.
I’ll cut to the chase and the news everyone is most interested in – USF maintained its top 100 position from the prior year, dropping just one spot from tied for #96 last year to tied for #97. But like the proverbial duck on the surface of the water, this small change masks a lot what is happening below the surface.
When the U.S. News rankings were first developed in 1983 (back when the publication itself was still a well-respected weekly news magazine, rather than its current state as an organization that makes its money largely from selling a variety of rankings of colleges, schools, hospitals, and the like), the formula it used rewarded institutions in large part for three characteristics: wealth, reputation, and selectivity. The rankings were heavily tilted toward measures of institutional wealth, including how much money the institutions spent, endowment size, and the like.
Peer assessment, i.e., what other college presidents thought of your institution carried a lot of weight, as did how selective you were in admissions. The more students you rejected, ceteris paribus, the higher your ranking would be. And being more selective meant that you were also generally enrolling students with higher SAT or ACT scores, which was also an important criterion. The general criticism was that the rankings were too focused on inputs, rather than the actual outcomes created by the institutions.
Over the years, U.S. News has tweaked its ranking methodology, sometimes in response to criticisms from those of us in higher education, but also in response to competition from the many other college rankings. These others include rankings from Washington Monthly, Money magazine, the Wall Street Journal, Forbes, and the Princeton Review just to name a few. While these other rankings gather attention from prospective students, their families, and the media, none can match the attention paid to the granddaddy of them all.
To the credit of the U.S. News editors, led by Bob Morse who has been at the helm for three decades, the rankings methodology today is much broader and inclusive of multiple dimensions of college performance than when it started. In response to the criticism that the rankings were generally just measuring institutional wealth and how well institutions serve students from rich families (in large part because of the strong and positive correlation between academic metrics, like test scores, and family income in our nation), the rankings in recent years have included metrics of how well institutions serve a broader swath of the socioeconomic distribution of students.
For example, the methodology today includes the following outcome measures:
- first-to-second year retention
- 6-year graduation rates
- graduation rate performance, or the difference between a school’s actual graduation rate and its predicted rate, based on the demographics and academics of its student body
- how well the institution does in graduating students who are recipients of Pell grants, the federal government’s need-based grant program that awards the vast majority of its funds to students who or whose families are in the bottom half of the income distribution
This is a positive move, which gives credit to universities (like USF) which strive to provide opportunities to students from groups who historically have been excluded or underrepresented from higher education.
Even with this change, however, the U.S. News rankings still are heavily weighted toward factors that provide a perverse set of incentives to universities. First, universities are given more credit for spending more money in educating their students, rather than being credited for operating more efficiently. Ten percent of an institution’s score is based on how much money the institution spends on “instruction, research, student support services, and related educational expenditures,” with more spending related to a better ranking. In addition, seven percent of the ranking is based on how much institutions spend on faculty salaries (adjusted for regional cost of living measurements). Again, institutions are given more credit for spending more money.
At USF, we have undergone a concerted effort in recent years to examine our expense budget and ensure that we are operating as efficiently and effectively as possible. We have used this process to free up funds from operations to invest in our financial budget, in order to ensure that our students receive adequate grant aid to supplement what’s available from the state and federal governments, private sources, and a student’s own resources, and to ensure that they can afford to attend USF. But under the U.S. News methodology, we get no credit in our ranking for our investments in financial aid, and in fact are penalized by not spending more on other categories of expenditures.
A second category that provides some perverse incentives for institutions is one labeled “expert opinion.” U.S. News surveys presidents, provosts, and admissions heads and asks them to rate all the other institutions in the same category (i.e., national universities) on a Likert scale. The conglomeration of these surveys provides an expert opinion score that totals 20 percent of an institution’s entire ranking.
What this portion of the methodology does is encourage colleges and universities to invest large sums of money in sending glossy brochures to presidents, provosts, and admissions heads extolling the virtues and accomplishments of the institution. In the weeks leading up to when the U.S. News survey is released for completion, I will generally receive multiple copies of these every day, which I promptly drop into the recycling bin (other than when saved for the quick photo opportunity on the right).
Rather than focusing on the overall ranking we receive – which is what most of the media attention focuses on – we here at USF are more interested in the metrics behind the rankings and some of the additional lists that U.S. News publishes. What we are most pleased about are the areas in which we outperform our overall ranking.
One example is the portion of the methodology that measures the outcomes of retention and graduation. While our overall ranking among national universities was 97th, we ranked #86 on the outcomes measurement. We also are very happy that we perform so well on graduation rate performance, which as noted above is the difference between a school’s actual graduation rate and its predicted rate, based on the demographics and academics of its student body. Our actual graduation rate was 9 points higher than our predicted graduation rate.
And we are able to achieve these outcomes with relatively fewer financial resources than many of our peer institutions. We ranked #132 on financial resources (described above) and #186 in alumni giving, another component of the rankings. In addition, USF is tied for fifth in the country (with Stanford) for the racial/ethnic diversity of our student body and tied for 23rd for the highest proportion of international students on our campus.
We are also included on the U.S. News list of “A-plus schools for B students,” a list that it describes as containing institutions, “which have strong ratings in the 2020 U.S. News Best Colleges rankings, [and] accept a significant number of students with nonstratospheric transcripts.” What this means is that many of our students do not have the test scores or high school grades that would qualify them for admission to one of the top 10 or 20 schools on the list, but they come to USF and do very well because of the support and attention they receive here.
As I stated in the opening, there is a lot to unpack when you examine the U.S. News rankings. I hope this has done just a little to help explain them in more detail.