School of Public Health at the University of Minnesota Paper: Physician-Rating Websites are Biased in Picking Doctors, Exaggerating Consumer Opinions, Says Paper to be Presented at INFORMS Healthcare Conference June 2011

For several years, Medical Justice has made the case that most doctor rating sites have several fundamental flaws; many lack statistically a significant sample size of patient ratings which impacts the quality of information available to consumers. Doctors see between 1,000 to 3,000 patients a year, depending on specialty. Yet, most doctors only have a handful of online ratings. A paper, “The Information Value of Online Physician Ratings”, being presented at a healthcare conference sponsored by the Institute for Operations Research and the Management Sciences (INFORMS®) supports this point.

“The Information Value of Online Physician Ratings” is co-authored by Ritu Agarwal, professor of information systems and director of the Center for Health Information and Decision Systems, Guodong “Gordon” Gao, assistant professor, and PhD candidate Brad Greenwood, of the University of Maryland’s Robert H. Smith School of Business, along with Jeffrey McCullough of the School of Public Health at the University of Minnesota.

The authors investigated potential biases among Internet users rating general practitioners on websites. In particular, the study looked at which doctors patients chose to rate, how they rated those doctors and the intensity of the patients’ opinions.

Agarwal states, that physicians are concerned whether “these ratings a true measure of clinical quality.” Or are they just the rantings of a disgruntled minority.

In a podcast on The Science of Better, Agarwal explains “patients may not always agree with their (doctor’s) recommendation. If you don’t agree and are disgruntled, you may go online and complain about a procedure that was perfectly valid given the medical condition that you had.”

Agarwal also addresses the validity of online ratings. “If people who are happy don’t bother to go online and provide a rating, then you are going to get a skewed distribution of ratings online and the impression that you would walk away with is the doctor is not a very good doctor.”

The study’s findings support these concerns about bias in the current online rating environment. Among the authors’ conclusions;

  • Patients posting their opinions about doctors on online ratings are more prone than offline populations to exaggerate their opinions
  • Online ratings are more informative when identifying low-quality physicians, but not as effective in discerning high quality physicians from median ones.
  • Patients are most likely to provide ratings for their most flagrant or negative experiences with physicians.

Agarwal states “Patients need high quality information about the most consequential service that they consume: healthcare.”

We agree.

That is why Medical Justice is working with select partners to help create a system that will balance the legitimate needs of patients with the reasonable concerns of doctors; a system that will provide much needed “quality information.” Medical Justice/Dental Justice is leading the charge to promote ratings systems that include a critical mass number of credible reviews to provide the most accurate picture possible of a doctor’s practice.