Last spring, I posted about the challenge of calculating a return on investment (ROI) for individual colleges. The problem is with obtaining accurate data on the earnings of the graduates of a given college or university. I described there the problem with a ROI website created by the firm PayScale, which relies on self-reported data from individuals. There are a number of issues with PayScale’s methodology, but the biggest is that there is just no way to know how representative the respondents to their surveys are of the totality of the graduates for any single institution. There is also no way to judge the accuracy of the data reported.
Now there is another report out that uses the PayScale data to calculate ROI measures for community colleges. Last week, a study issued jointly by theNexus Research and Policy Center and the American Institutes for Research (AIR) took on the challenge of calculating the returns to associate’s degrees not just for individuals, but also for taxpayers in the form of the government subsidy provided to community colleges. There are a number of methodological problems with this study, which examined 579 community colleges around the country, but key among them is the study’s reliance on the PayScale data as the outcome measure for the ROI calculations. AIR is also the author of the College Measures website which, as I described in last spring’s post, shares the same flaw in using the PayScale data for some of its analyses.
Continue reading “A massive fail on ROI calculations for community colleges”