Can advice generated by AI shape doctors' diagnoses?

Written by:
Dr. Geoffrey W. Rutledge
Chief Medical Officer and Co-founder
Last updated on June 8, 2021 UTC

The beneficial effects of presenting a differential diagnosis to doctors who are making a diagnosis continues to generate interest within the medical field. At the forefront is the potential for a differential diagnosis generated by artificial intelligence (AI) to improve diagnostic accuracy.

Recently, a new study shed light on the doctor-AI relationship and extended current research into the impact of advice on doctors' diagnostic accuracy.

Researchers presented radiology cases — each case with a chest X-ray, a short clinical story, and diagnostic advice — to 138 physicians in a high-expertise group (radiologists) and 127 in a lower radiology expertise group (internal medicine and emergency medicine doctors [IM/EM]). The participants had to evaluate the quality of the advice presented and make a final diagnosis.

All diagnostic advice was generated by human experts, but some of the advice was labeled as generated by AI. Independently, some incorrect advice was labeled as from a human expert or from the AI.

  • The results demonstrated that both the radiologist and IM/EM groups responded to the advice they were given: Diagnostic accuracy went up when the doctors were given accurate advice, and it went down when they were given inaccurate advice.
  • On the whole, the radiologists were better able to recognize the inaccurate advice as low-quality.
  • IM/EM doctors were equally likely to value accurate AI advice as they were to value accurate advice from human experts. By contrast, radiologists rated human-generated accurate advice higher than they rated AI-generated accurate advice.

These results confirm our approach to offering clinical diagnostic decision support on HealthTap. We offer high-quality AI assessments and other intelligently determined, relevant details about the patient to the doctor, so they can keep in mind all relevant data and possible suggestions for explanations as they are considering a case.

In the end, the treating doctor must consider all relevant information and suggested alternative explanations, and the doctor must make the ultimate diagnostic assessment and treatment decisions.

HealthTap is proud to be a leader in virtual healthcare, enabling people across the U.S. to get urgent care, receive guidance and counsel, and manage primary care with board-certified physicians.

Share:

More articles in Doctor Insights

Not just a doctor —
your doctor