Three top criticisms against CMS’ overall hospital star ratings

The Hospital Compare Overall Hospital Quality Star Ratings, published by CMS in July and updated in December 2016, are meant to provide transparency on hospital quality to help patients select the best site for their healthcare needs.

In reality, leading healthcare institutions are warning consumers that the ratings leave a lot of information out and may not paint a complete picture of a hospital’s qualifications.

“Patients should beware of making decisions about hospitals using the newly released star ratings. They are based on a deeply flawed methodology that does not take into account important differences in the patient populations and the complexity of conditions that teaching hospitals treat,” Association of American Medical Colleges (AAMC) President and CEO Darrell G. Kirch, MD, wrote in a statement when the ratings were released in July. “As a result, many of the nation’s leading teaching hospitals-institutions that provide the most advanced healthcare in the world-have been assigned lower ratings than other hospitals that treat patients with less complex conditions or that treat only certain conditions. This new system could have very serious consequences for patients who are choosing where to go for treatment, potentially driving them away from some of the best hospitals for their conditions.”

Methodology

A hospital’s overall quality star rating encompasses 64 of the more than 100 measures displayed on the Hospital Compare website. The methodology for the ratings was developed with the help of technical experts and public comment, according to CMS, and will be revised as needed, with ratings updated quarterly.

While CMS states on its website that the agency has received “numerous” letters of support from patient and consumer advocacy groups, many industry leaders are much less happy with the functionality of the ratings system.

The AAMC convened a panel of experts on quality reporting back in 2014 to help develop a set of guiding principles to help CMS create a consistent and meaningful ratings system. Those principles included recommendations about specifying target audiences for the ratings, and using measures and methodologies supported by the National Quality Forum, says the AAMC. CMS, however, did not utilize the work of the AAMC and continued with its “one-sized-fits-all-model,” according to the organization.

“The flawed methodology is unsuited to provide comprehensive ratings for a wide variety of hospitals and communities, and the ratings do not accurately reflect the full picture of hospital care in the United States,” AAMC leaders wrote in a statement in October 2016. “As healthcare professionals at our teaching hospitals continue to provide world-class healthcare and lead quality improvement efforts, the AAMC and our member institutions will urge CMS to modify the Star Ratings methodology to take into account patient populations and complex care provided by teaching hospitals.  Until the ratings are revised, it is patients who will suffer the most from the lack of clarity and transparency, who may choose a hospital for its rating over a hospital that would provide the best care for their condition.”

Even Congress questioned the ratings system, asking CMS in the spring of 2016 to review its methodology. The push from Congress caused CMS to delay the launch of the system, originally planned for April 2016, to July 2016.

Still, the ratings appear to be skewed in the favor of smaller, private and non-teaching hospitals that take fewer complex cases and have patients in higher socioeconomic classes, say critics.

Just 2.2%-102 in total-of U.S. hospitals earned a five-star rating in the first release of data from the new system, with 20.3% earning four stars, 38.5% earning three stars, 15.7% earning two stars and 2.9% earning one star. Another 20% of hospitals did not have adequate reporting data to be rated. Broken down by size, hospitals with fewer beds fared better, with hospitals with less than 99 beds having the highest percentage of five-star ratings and fewer ratings of three stars or less when compared to hospitals with 100 beds or more.

Criticism #1: Ratings are vague, generalized

Janis M. Orlowski, MD, chief healthcare officer at the AAMC, says the association supports a transparent ratings system that would make healthcare choices easier for the public, but that doesn’t mean that the star ratings are that system.

“We’re completely in favor of rating and making good information available to the public. The thing is that if you take a look at a large complex academic medical center or health system and give it a rating, what does that really mean?” Orlowski says.

The current ratings system does little to inform a patient seeking a liver transplant or a healthy pregnant woman where their needs would best be met, she says. Instead, the system gleans incomplete information from hospitals and places them under a generic rating.

The AAMC has suggested that the ratings be separated into service lines to provide better information.“The fact that it comes together under a single star rating is not the best way to present information to the public,” she says.

Criticism #2: Less reporting leads to higher ratings

Another major concern is that reporting is spotty, at best, says Orlowski. Seven domains are assessed, but half of the hospitals that received five-star ratings did not report on all those domains. Large teaching hospitals report everything, but there are many specialty or smaller hospitals that don’t report on mortality or morbidity, she explains.

“What we believe and what the data shows is that the more measures you reported on, the lower your score was. I think that’s a reflection that there are some measures that are easier to report on,” Orlowski says. “It’s not that those hospitals didn’t voluntarily report those measures, it’s that they didn’t have enough numbers to report. But we should compare like hospitals to like hospitals. Everyone was thrown into one pot and there were significant differences. The system is not set up the way it should be.”

Akin Demehin, director of policy at the American Hospital Association (AHA), echoes Orlowski’s concerns and says the ratings system does not achieve its intended goals.

“Transparency is certainly a goal we have long supported. Hospitals have been reporting quality data on hospital compare for over a decade now and the star ratings were supposed to help make the data easier to understand and more accessible,” Demehin says. “But we think the star rating that’s currently designed is deeply flawed and raises far more questions than answers.”

Like the AAMC, Demehin says the AHA found gross discrepancies in how hospitals were rating based on the number of measures that were reported.

“We see clear evidence that the more measures you report on at hospital compare, and the more you have included in the star rating, the worse you do,” Demehin says. “For large hospitals, they’re going to be the ones that are more likely to report on a significant number of measures. That’s a source of bias we think needs to be looked at.”

Next: Criticism #3

Criticism #3:Socioeconomic bias

The system also presents problems in terms of managing poorer populations, according to critics. Literature supports the fact that readmission rates are higher in low-income patient populations. Patients may not have pharmacies near their homes or the social support to adhere to prescribed medical regimens. But due to value-based reimbursement initiatives, hospitals that serve these patient populations may be penalized for higher readmission rates and funding may be diverted away from those hospitals to hospitals that serve patients with lower readmission rates.

“This an absolute crime. In regard to the value-based program, we’ve taken resources from hospitals that serve the poor and taken penalties from them, and given bonuses to hospitals in better areas,” Orlowski says. “I know CMS didn’t intent to shift funds from poorer to richer hospitals, but that’s what they’ve done.”

The AHA found similar discrepancies in how hospitals serving poorer populations were rated. “We’ve also done analysis that looks at how hospitals do by kind of a proxy or the sociodemographic status of the patient population they treat, and found that those hospitals caring for poorer patients fared worse,” Demehin says. “CMS needs to go back to the drawing board on the star ratings and comes up with a system that is fairer and more accurate.”

What’s next?

Prior to the launch of the ratings system, Orlowski says the AAMC urged CMS to continue working with stakeholders before making the system live to achieve better metrics and methodology. That did not happen, though, and now patients may be in a position to reject a hospital that can better serve their needs because of a faulty rating that didn’t give them a clear picture of the hospital’s worth.

“What ended up happening is penalties and public reporting were put in right away,” Orlowski says. “The last thing you want is for people to take this information and drive by a teaching hospital because they reported more metrics and go to a hospital that reported less. This isn’t a quality, but a data issue.”

Demehin says the star ratings should be suspended until changes can be made to avoid permanent damage to the reputations of good hospitals.

“We really think that the system needs to be improved to ensure that it’s more accurate. We know that CMS would need to work with stakeholders to make that happen. We have argued that ratings should be suspended until we can work out a better approach,” Demehin says. “The impact of star ratings is reputational. Because these data are public, because they are shared on a Medicare website, they garner some attention.”

In December 2016, the AHA sent a letter to then president-elect Donald Trump, outlining ideas for regulatory relief for hospitals and including proposals for suspending the star ratings until improvement can be made. There has been no answer yet, but the AHA and other associations continue to push CMS for change.

“We have made our concerns very well known to CMS. Before the ratings were launched, we joined with AAMC and two other hospital associations to ask them to delay and consider changes,” Demehin says. “On our agenda now is really taking a look at the regulations that affect hospitals and trying to identify those we think need to be changed and improve to better align with patient care and create more efficiency.”

When pressed by Managed Healthcare Executive about the criticisms of the ratings system, CMS officials say “substantial efforts” were made to involve stakeholders and hospitals in the development of the system, citing a July 2015 dry run of the system as well as numerous meetings and calls throughout 2016.

“CMS is committed to improving outcomes and working with stakeholders to improve individual quality measures, while minimizing unintended consequences for all facilities, regardless of the characteristics of the patients they serve,” the agency wrote in a statement to Managed Healthcare Executive. “We anticipate that a new (technical expert panel) will be empaneled by our contractors in early 2017 to discuss possible updates to the methodology. We look forward to continued discussions with stakeholders in late Spring/early Summer 2017.”

Rachael Zimlich is a writer in Columbia Station, Ohio.