Hospital ratings: Advancing or obscuring choices?

April 3, 2015

Article

Hospital rankings provide consumers with valuable information, but they're not always in agreement.

From televisions to toasters and pizza to plumbers, ratings have been the collective pulse of consumer satisfaction for product or service-buying decisions. Only recently, however, has healthcare joined the fray.

Now, multi-media promotion on billboards, bus posters, direct mail and more feature splashy headlines of local hospitals as “Ranked Highest,” “Rated #1,” or “Best Rated in Specialty.” Sounds like good marketing, but does it help the consumer?

In the recent article, “National Hospital Ratings Systems Share Few Common Scores and May Generate Confusion Instead of Clarity,”Health Affairs, March, 2015, authors took ratings to task and examined four national hospital ratings systems:

“Best Hospitals” from U.S. News and World Report,

“America’s Top 100 Hospitals” from Healthgrades,

“Annual Hospital Survey” from Leapfrog Group, and

Hospital ratings provided by Consumer Reports.

With the exception of Consumer Reports, ratings research and publication is self-financed by the respective groups, and hospitals are permitted to use the results in their marketing and advertising.

AustinLead study author J. Matt Austin, M.S., Ph.D., assistant professor at the Armstrong Institute for Patient Safety and Quality at Johns Hopkins Medicine, explains the study incentive. “There were a growing number of consumer-directed hospital ratings and we were curious to understand how much agreement there was across these systems--is there a consistency in rating? We were interested in what was the disagreement or agreement.”

Austin says the study found differences in the top performers’ definition. “Leapfrog Group uses an A,B,C,D hospital score; Healthgrades has ‘America’s Top 50 and 100 Hospitals’, U.S. News & World Report has an honor roll,” says Austin. “

Consumer Reports doesn’t necessarily have high or low performers, but uses a hospital safety rating system which includes proprietary and outside public rating data.

“One of the more interesting findings,” says Austin, “is in the case of 27 hospitals that were found high on one rating system, [but] were low on another. So what does that say? It seems that every hospital has bright spots and opportunities for improvement.”

With the exception of Consumer Reports, which only makes its information available to paid subscribers, the others are publicly available online.

NEXT: Comparing apples to pineapples

Comparing apples to pineapples

While the Affordable Care Act (ACA) has brought millions under the umbrella of coverage, Austin says the current and growing number of patients in high-deductible health plans “puts the onus on consumers to become savvy purchasers. They are purchasing expensive services out-of-pocket and they really do need information on quality and safety.”

Austin is heartened by recent emphasis on paying for the value of care, rather than simply volume. “We’re now moving toward value and volume, and away from volume alone.”

Marks

But can consumers discern value when ratings appear conflicted? Evan Marks, chief strategy officer of Healthgrades, agrees that the differences between survey instruments can be confusing to consumers, but he asks, “Why would they all evaluate the same thing?”

“Let’s take car ratings as an example. The National Highway Traffic and Safety Administration is going to review cars on test crash statistics; Car & Driver may test cars on road feel and handling; and Motorweek may test on costs and reliability. But if we line up each best-rated car, we’re going to have three different sets of cars, not one. The same thing is happening here, so yes, the report is correct in that if consumers don’t understand what is being evaluated some may be confused.”

Marks explains that Healthgrades develops ‘‘America’s Best 50 and 100 Hospitals” designation by evaluating outcomes data from U.S. Centers for Medicare and Medicaid Services (CMS) across 4,500 short term acute care hospitals in the U.S.. Healthgrades evaluates each hospital for risk adjusted mortality and complication outcomes in 32 of the most common procedures and conditions. Hospitals consistently performing in the top 5% of all hospitals each year, for a minimum of 6 years, are eligible to become recognized as among the nations’ 50 or 100 “best”.

According to Marks, Healthgrades also publishes in-hospital patient safety ratings across 13 patient safety indicators using software from the Agency for Health Research and Quality (AHRQ) as well as CMS HCAHPS patient experience scores for most short term acute care hospitals. All information is provided online free of charge or registration. Marks says all of their methodologies are available online at Healthgrades.com, which he boasts logs more than a million visits a day, and thirty million visits a month-“Twenty times more than CMS.”

Marks says the current driver of transparency and ratings popularity “is being driven by new healthcare delivery models fostered in part by the Affordable Care Act.

“Other factors driving this ‘need to know’ is that today, people are very mobile, changing their insurance plans. The docs are aligning with different systems as a result of the ACA, allegiances are changing, everyone is moving, looking and comparing.”

The result, says Marks, is “an explosion in need for consumers to have access to information to help them differentiate hospitals; in what they do, and what they specialize and excel in.” And the Internet is the galvanizing link to make it all accessible, he summarizes.

Disparity is no surprise

Like Marks, study author Robert M. Wachter, M.D., professor and associate chair in the Department of Medicine at the University of California, San Francisco, is similarly not surprised that the study found disparity among the four systems’ ratings for the same facility.

“In some ways, it’s the old story: you perceive the totality on the part of the animal you’re looking at. Four different systems are looking at hospitals four different ways, with four different core questions.”

Wachter says that the web is “democratizing” the information, but adds that searching for health service is very different from searching for a reliable washing machine.

“People have an urgency and an anxiety in this pursuit--you don’t search for healthcare unless you need it. So, there is a level of complexity here that ratings can help. You may partly trust an online consumer survey format, like Yelp, but you may want some validation from a reasonable set of criteria so that the rating is not just what people felt like when they walked in the door.”

Ratings, says Wachter, must provide information that is truly useful for patient decision-making. “We all want to know specifics--is the surgeon technically good? How good are they [physicians, hospitals] in treating Lupus? This goes beyond the view of any individual patient. Any kind of rating needs to blend things that are similar domains.”

Wachter notes that “adjusting for how sick people are when they are admitted” and whether “a hospital adjusts for preexisting factors,” for example, can lead to different outcomes which can affect overall ratings.

But the ratings process, he asserts, is still very much in a learning curve. “People might have expected a ‘good place’ gets it all right…[but] in healthcare, no place is that good--yet; and quality and patient experience also says something about the measures. We’re at an early stage using public data.”

NEXT: CMS' compare website

Bar charts for easy comparison

Leah Binder, chief executive officer of Washington, D.C.-based Leapfrog Group, says their system focuses primarily on patient safety.

Binder“Hospitals are extremely complex; as a patient you need to know a lot of different information before you walk in the door. Patients should look at hospital safety, errors and accidents.” However, Binder adds, “Our market research finds that consumers don’t know what safety is.”

Binder explains Leapfrog is highly transparent; they report not only a letter grade for their hospital review but also publish the methodology and the origin of the data for all 28 measures the survey team analyzes annually. Site users can access a see-at-a-glance bar chart for comparisons of hospitals in several domains. The charts denote the facility’s level in meeting Leapfrog safety standards, with links to details on what the individual standards are and how they are derived. If a facility fails to respond, the non-response is noted.

“We pilot-test our survey with hospitals, we make every effort to be aligned with the AHRQ’s principles of the National Quality Strategy (NQS), the Joint Commission and CMS--we have a very intensive scientific review,” says Binder.

The Internet is one piece “turning patients into consumers,” seeking ratings, but Binder echoes Marks in that, “the biggest influence is high deductible health plans.

“This is a fast-growing phenomena. Sixty percent of insured now have some form of high deductible, so basically [the cost of] more and more direct care is being shouldered by the consumer.”

She cautions that ratings are not a one-size-fits-all assessment. “Some hospitals have excellent cardiac care, but have challenged OB units. You need a certain kind of rating, and more than just one rating. Consumers need to look at what they do want--quality nursing care, specialty expertise--and what they don’t want, such as accident or infections.

“It is a positive thing that there are so many rating systems,” she adds, “but much room for proficiency. We are nowhere close to where we need to be--our markets say they want more and they deserve more.”

CMS’ compare website

Consumers also have another tool at their disposal: the CMS Hospital Compare website. Created in partnership with the Hospital Quality Alliance (HQA), the site began publishing core measures of care in 2005, adding data from the Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) survey in 2008. Hospital outpatient data including imaging efficiency data, emergency department and surgical care processes, was incorporated in 2009; 30-day readmissions statistics for heart attack, heart failure and pneumonia patients were added in 2010; and data from the CMS readmission reduction program along with hospital-submitted data from the American College of Surgeons National Surgical Quality Improvement Program was added in 2012. Recent additions are HCAHPS care transition data, outcomes data for chronic obstructive pulmonary disease and strokes, and Prospective Payment System-exempt cancer hospital measures data.

The classic case of "Caveat Emptor"

As the ratings process evolves to encompass more data and increase transparency, stakeholders agree that consumers wield the buying power. But, says Austin, consumers also have to do their homework.

“One of the recommendations we would offer is for consumers to really look under the hood a bit more. It falls upon the consumer to understand what is being measured--is it aligned with your values? You have to look at safety issues, best medical centers, and the facilities need to be tailored to the patient needs.

“I think the readers have to look at each rating system, and what is being measured to be informed, what the rating represents. There is definitely value in multiple perspectives on quality and safety.”

Barbara L. Hesselgrave is a freelance writer in Baltimore, Maryland.