Ranking of hospital quality not 100% accurate

NEW YORK (Reuters Health) - Rankings of the top 50 "America's Best Hospitals" for heart care and heart surgery by the US News and World Report are for the most part in line with outcomes of patients.

However, according to a study published in the Archives of Internal Medicine Monday, they are not 100 percent accurate.

"The U.S. News & World Report ranking, which includes many of the nation's most prestigious hospitals, did identify a group of hospitals that was much more likely than non-ranked hospitals to have superb performance on 30-day mortality after acute myocardial infarction," Dr. Harlan M. Krumholz and his colleagues report. "However, our study also revealed that not all ranked hospitals had outstanding performance and that many non-ranked hospitals performed well."

The magazine's rankings for 2003 were based on three equally weighted measures: in-hospital death rates for cardiovascular conditions, reputation among surveyed cardiologists, and hospital infrastructure.

To assess the Report's accuracy, the investigative team, based at Yale University School of Medicine in New Haven, Connecticut, and at Harvard Medical School in Boston, used a risk model endorsed by the National Quality Forum that profiles hospitals on risk-standardized 30-day mortality rates after a heart attack.

They assessed 30-day death rates among 13,662 patients admitted to 50 hospitals ranked on the U.S. News list as the best in "Heart and Heart Surgery" and among 254,907 patients admitted to 3,813 unranked hospitals in 2003.

After the researchers factored in patient characteristics, the 30-day death rates were, on average, lower in ranked hospitals compared with non-ranked hospitals (16 percent vs. 17.9 percent).

But when the hospitals were divided into four groups based on these rates, 35 ranked hospitals (70 percent) were in the group with the fewest deaths, 11 (22 percent) were in the middle two groups and four (8 percent) were in the worst-performing group.

Moreover, 11 ranked hospitals (22 percent) and 28 non-ranked hospitals (0.73 percent) had standardized mortality ratios far less than one -- meaning that although ranked hospitals were more likely to have lower-than-expected death rates, non-ranked hospitals with favorable ratios outnumbered ranked hospitals with similar performance by nearly three to one.

"As a result, the U.S. News & World Report ranking list does not include many hospitals that have outstanding performances for the care of patients with acute myocardial infarction," or heart attack, the authors write.

The authors attribute the imprecision of the results to the "disproportionate influence of reputation" applied to the rankings.

Being named one of "America's Best Hospitals" is a "popular and coveted quality title," Dr. Eric D. Peterson and Dr. Sean Michael O'Brien write in a related editorial. However, they caution that the results need to be understood in light of potential inconsistency and fallibility of quality-ranking systems.

SOURCE: Archives of Internal Medicine, July 9, 2007.