Abst​ract

In the spring of 2018, the Pennsylvania Patient Safety Authority convened an expert panel to discuss issues in diagnostic error and strategies for improvement. The 10 invited speakers were Paul Epner (Society to Improve Diagnosis in Medicine [SIDM]), Dr. Mark Graber (SIDM), Sue Sheridan (SIDM), Dr. Chris Goeschel (MedStar Health), Dana Siegal (CRICO), Dr. Hardeep Singh (Department of Veterans Affairs), Dr. Timothy Mosher (Milton S. Hershey Penn State Medical Center), Helen Haskell (Mothers Against Medical Error), Dr. Joyce Knestrick (American Association of Nurse Practitioners [AANP]), and Dr. Michael Consuelos (Hospital and Healthsystem Association of Pennsylvania [HAP]). The panel was moderated by Caitlyn Sidrane of the Pennsylvania Patient Safety Authority.

Discussion

What is diagnostic error? How does fear of malpractice claims influence provider judgment? What is the one thing facilities could do today to prevent a misdiagnosis? In the spring of 2018, the Patient Safety Authority convened a panel of the nation's top experts on diagnostic error to answer these questions.

The 10 invited speakers were (in the order in which they spoke) Paul Epner, Mark Graber, Sue Sheridan, Chris Goeschel, Dana Siegal, Hardeep Singh, Timothy Mosher, Helen Haskell, Joyce Knestrick, and Michael Consuelos. The panel was moderated by Caitlyn Sidrane. What follows are excerpts from the discussion.

CAITLYN SIDRANE: While diagnostic error is common and causes significant harm, leading to as many as 80,000 preventable U.S. deaths each year,² there is still no consistent, universally accepted definition. There's general agreement on the concept but difference on how to best operationalize it.² What is diagnostic error, and at what point is something considered delayed or a misdiagnosis?

PAUL EPNER: A layman's definition is whenever the diagnosis is wrong—i.e., I'm told one thing and it's really something different—or when it's avoidably delayed, and there are questions about that, certainly.

MARK GRABER: First, the definition in the IOM report is terrific, and it has been very well received. So for everyday purposes, I think we should all be trying to use that one. It's patient-focused, it emphasizes timeliness and accuracy issues and the importance of communication, so it really has a lot going for it.

Another thing is that patients have a lot of trouble recognizing that term and understanding that what happened to them is really a diagnostic error. We've done focus groups and we asked patients, "Who's had a diagnostic error?" and just a few hands go up. But if you explain, "This is about a diagnosis that should have been made a lot earlier, or a diagnosis that was wrong and was later found to be something else," then, all of a sudden, a lot more hands go up. So we have a problem with people understanding what a diagnostic error really is.

The question about at what point something is delayed is an excellent one, because it points out that we really have no standards or guidelines or appreciation in medicine today of how long it should take to diagnose anything.

There is a study about how long it takes to diagnose iron deficiency in women, and the answer is two years. That's crazy. How long does it take to document a diagnosis of asthma in children who are wheezing? Seven clinic visits.

We really see that there's a lot of work that needs to be done, and it can be done productively to get a better grip on how long diagnosis takes.

SUE SHERIDAN: I agree about the definition that was created by the IOM. It's not too complicated or academic; it's understandable, patient-centered, and I'm hoping that everybody will use that as a definition going forward. Many, many patients have experienced diagnostic error, and I myself, despite the severity of both the diagnostic errors in my family, didn't fully appreciate that they were both diagnostic errors until I got to know Mark [Graber]. Then I started to realize how what happened to my family fits in the IOM definition of diagnostic error.

CHRIS GOESCHEL: One of the things that we grappled with in our work that I believe is important is when a diagnosis is communicated to the patient. When do you know what you know, and when is it shared with patients? Timely communication is critical.

DANA SIEGAL: I have the very far end of this journey, and the malpractice claims I study provide all the hindsight in the world to look at diagnostic error. When we look back on these cases, we're able to see the initial missteps that resulted in a wrong initial diagnosis that then links to the delay of getting the right diagnosis. Because we code these cases for educational purposes, we agonized for years over whether it was wrong, delayed, or missed, but ultimately, we focused on the opportunity to better identify vulnerabilities and interventions for the broader diagnostic process, providing more clarity on what the common issues were, regardless of a delayed versus wrong initial diagnosis.

The critical piece that we see in the medical malpractice claims is that, more often than not, we have the right information, and to Chris [Goeschel's] point, it just wasn't communicated. That may be to the patient, or it may be to another provider who had accountability to move the ball forward. And that's where so much of the vulnerability is, not in the cognitive or clinical judgment process, but in the communication and delivery of information.

HARDEEP SINGH: After doing more than a decade of research in this area, we're still struggling with how to define this correctly. In our work, we have found there's not a lot of distinction between missed, delayed, or wrong diagnosis, so we try not to use those as buckets. They overlap almost all the time.

The concept that's most useful to us has been that of a missed opportunity; was there something different I could have done when I was seeing the patient at that time? We need to understand that this is not a black-and-white sort of a thing, it's really gray. Diagnosis evolves over time. There's a lot of uncertainty involved. We need to think of this as a phenomenon that is beyond a single provider's thinking. And when we take it from a more systems perspective, the definition becomes a little better and is more operational.

MR. EPNER: It also depends on the issue of whether there's a symptom that we're reacting to. If the U.S. Preventive Services Task Force has a guideline for screening and the patient is asymptomatic but skips an appropriate screening, and if that screening would have shown something, is that a diagnostic error with the error being attributed to the patient for not completing appropriate screening?

I'm not saying it is, but the notion of whether there's a symptomatic impetus to initiating the diagnostic process or a public health strategy that's not being followed, one could say from a public health perspective that either one could be a basis for describing a diagnostic error.

And then one also needs to consider, even if someone is symptomatic, if having a simple explanation for the symptom is sufficient, or does the patient's context—their genotype, their phenotype, other socioeconomic attributes, social determinants—need to be included in the diagnosis because of the potential implication for care? And then is failure to identify those social determinants a diagnostic error as well? So, again, if you want to go very broad, these are other things that should be considered.

MS. SIDRANE: Paul [Epner] brings up an interesting point about the public health aspect of this. Medical errors are a leading cause of preventable death, and misdiagnoses are a leading cause of medical error.³ Are there broad-based interventions that can and should be implemented?

TIMOTHY MOSHER: One of the things that I look at when viewing diagnostic error from a public health standpoint is the need to focus on what are systemic factors that are addressable that we can extrapolate across the whole healthcare system. And we're finding that there are quite a few.

Concepts such as human factors engineering. The way we're setting up the health system that drives cognitive error and leads to errors in communication. The lack of ability to get information between different electronic medical records and other information systems. The lack of good registry data to be able to look at where diagnostic errors are occurring. There are a lot of things that need to be addressed at the public health level for us to be able to really leverage what we're learning about what's occurring in diagnostic error at the individual patient provider level. So for us to really be able to move this forward, we have to raise this to the level of a public health problem.

HELEN HASKELL: Well, IHI [the Institute for Healthcare Improvement, where Ms. Haskell sits on the board of directors] is doing that. We're coordinating a national action plan on patient safety as a public health issue, and certainly diagnosis will be part of that, to actually have strategies and measurements on a broad scale that can be followed. When I think about public health, one of the things that's near and dear to patients' hearts is the idea of public education campaigns, making signs and symptoms of common ailments part of general education, not just heart attack and stroke—although that's certainly needed—but also acute illnesses like sepsis. That has begun, but it needs to be more broadly disseminated.

DR. SINGH: I agree with Helen [Haskell] about the context of putting this broadly. If you look at it from that perspective, there are lots of things we could do with broader healthcare issues such as access and workforce competency and high quality diagnostic testing, the use of health IT [information technology] involving patients and improving the way we deliver care in our clinics or emergency rooms, and letting doctors spend more time with patients. All of that is good and all of that can be put into the context of public health, but what it means for diagnosis specifically is also strengthening the overall health system, improving primary care, improving emergency room care, and working on broader delivery system issues, not just the thinking around making a diagnosis.

MR. EPNER: So let me take a slightly contradictory stand on that. According to the National Academy of Medicine, we're wasting $765 billion a year in healthcare.⁴ But that probably doesn't include things like the advanced stages of disease that require more expensive treatments versus early detection, lost employment, productivity, et cetera.

So if we're saying that public health implies that we're not focused on an individual patient but that we're trying to ensure that everyone has the best opportunity to be healthy and productive, and that society's resources are best used to help society, then there is a public health imperative to ensure patients are actively engaged and the systems are actively engaged in trying to find detectable health problems, even when someone may not have entered the health system yet.

MS. HASKELL: That sort of intervention can be really onerous, and it's something that I would be hesitant to advocate for, to go and seek out people, except in the cases of perhaps an acute or contagious illness.

DR. SINGH: I agree. We are not doing a very good job in diagnosing disease in patients who are presenting to us with symptoms that are fairly obvious and have red flags, so going out and looking for asymptomatics with disease that might present in 20, 25 years is probably not the way patient safety ought to move right now.

MR. EPNER: I'm certainly not suggesting that we go out and look for patients who have nothing wrong but maybe will in 20, 25 years, but certainly, diabetes is an underdiagnosed problem. There are many other infectious diseases, such as HIV, et cetera, where there is a public health value to screening, and that has been well demonstrated and well published.

There are other kinds of areas where, again, having people understand when they should access the system, and if the U.S. Preventive Services Task Force is right about appropriate screenings, it's in the interest of society and healthcare to reduce the economic and human burden in the long term.

MS. SIDRANE: What about patients and their responsibility in all of this? What role do you think patients should play? And what do you wish patients knew about the diagnostic process?

DR. GRABER: Given the complexity of diagnosis, it's amazing that we do get it right almost all the time. The diagnostic error movement and activities are not meant to beat up on doctors or on healthcare organizations: it's to think about how we can do a better job. And what patients don't appreciate is how complicated it is and how many pitfalls there are.

MS. SHERIDAN: What I wish patients and family members knew about the diagnostic process is that it is a process, it's not a static event, and there are many potential pitfalls and breakdowns in that process. I wish that patients and family members knew the magnitude and impact of diagnostic errors.

Unfortunately, patients believe healthcare systems are infallible, and patients need to understand that they are not. I'd like patients to know that no news is no news, regarding test results, regarding their diagnoses. Never assume anything just because you haven't heard anything.

Regarding the responsibility of the patient, that's the wrong question. It all starts with the clinician. I struggle with the concept of "patient responsibility" regarding getting the right diagnosis. A lot of times it's very hard for patients to step into this diagnostic process. That clinician needs to issue an invitation to the patient to join the doctor in the diagnostic process.

And so the question is not what's the role of the patient, but has the clinician informed the patient and family member about the diagnostic process, so that he or she is crystal clear about the importance of the history and physical, and what symptoms to look for, how to report them, to whom they should report them, and within what time frame. It's better framed as, what can clinicians do to encourage them to participate in their diagnostic process?

The clinician also needs to encourage the patient to collect his or her documents, use the portal, ask questions, and be informed as much as possible. Knowledge is power. Patients will go to the web, and we want them to, so clinicians can help identify the most credible websites where their patients and family members can do their research.

JOYCE KNESTRICK: I don't know what other people have been seeing, but I've noticed that patients are becoming more active members about their diagnosis. I see more people bringing things in from the internet. "I think I have this, what do you think?" But it's also important to make sure that you can develop that trusting relationship with the patient so that they tell you everything that they need to tell you to help you to make a good diagnosis. And I think that is the biggest part that they could play, and sometimes it's difficult because they don't want to tell you everything.

MS. SIDRANE: There was a recent study by U.S. News & World Report that quantifies how much patients really don't want to tell us. They found that up to 30 percent of patients have either told a white lie or intentionally withheld information from their healthcare provider.⁵

MS. HASKELL: It's really important that patients understand the concept of uncertainty. Tests and providers don't necessarily have all the answers. Diagnoses can change. They really have to be part of the process and bring in their own information. It's critical for patients to do research. And in terms of the information that they give to the provider, I take a pretty different perspective on this. The obligation goes one way. Medicine is a service industry: healthcare providers have an obligation to do their best to help patients and to inform them, but patients don't have an obligation to tell everything to their healthcare providers.

Patients are concerned that their information isn't secure, and with insurance being so up in the air, it could even be used against them. And while they might be happy for their providers to have it, they don't know what's going to happen to it after that. Doctors and advanced care practitioners aren't always aware of how little control they have over that information and where it might go after them. Patients need to know what information to convey to a clinician about what they think they might have. They're not going to share everything, so they need to know what they need to convey.

MR. EPNER: The notion for patients to decide by themselves what's important would require them to know everything the physician knows that's going into establishing the diagnosis. We do have a lack of trust and lack of control of private information, and that is a problem that does need to be solved. But to say that the physician needs to read the mind of the patient or it's okay if you don't give them enough clues; the ultimate consequence of a wrong diagnosis because of insufficient information will be felt by the patient.

MS. HASKELL: I agree with your last statement, but what I meant was that doctors' offices collect a lot of information on patients, a certain amount of which is not necessarily relevant to any given situation. Similarly, patients may provide too much information because they are not sure what is important, and they want to be sure nothing is missed. While all this information should in theory be grist for the mill, in practice it can create a high noise-to-signal ratio that is fertile ground for diagnostic error. The fact that it also may not be secure is simply icing on the cake.

The point I was trying to make is that clinicians should be more targeted about what they collect, and patients should be more targeted about what they communicate. Doctors are concerned about this, too. Dr. John Ely has a tool, "Eight Characteristics of a Symptom,"⁶ that is intended to help with exactly this problem of imprecise communication.

MS. SIDRANE: How does the payment system impact diagnostic error? How much influence do payers have in this, and what changes need to be made to the payment system?

DR. GRABER: The payment system has a major impact on diagnostic quality in big ways and in little ways. Our current payment systems aren't focused on quality, they're focused on productivity, and that's an enormous problem. The biggest complaint we hear from doctors is they just don't have enough time for diagnosis. It takes time to get a history and do a physical and think about what might be going on, and if you only have 10 or 15 minutes to see a new patient with 15 problems, that's not going to be good for diagnosis. We would like to see a payment system that rewards quality and accuracy and timeliness, and we don't have that at all now.

And there are little things. For example, if you're seeing a patient, there's pressure to assign a code so that encounter can be billed. So before you're even sure what's going on, the patient is now labeled, and the next person seeing that patient may not think it through completely, and that's bad, too. Premature labeling is one of the major problems we see in terms of what causes diagnostic errors.

MS. SHERIDAN: I agree with Mark [Graber]. Our payment system has a tremendous impact on diagnostic errors. Case in point, when my husband's malignant pathology failed to be communicated to the ordering doctor, I asked the pathologist why in the world didn't he pick up the phone and call the doctor. And he said, "Sue, we're not paid for that. That's not my job." We need to ensure that we have a payment system, as Mark [Graber] said, that incentivizes quality communication and teamwork.

Also, it is important to remove financial incentives that inadvertently delay the diagnosis and appropriate treatment, especially when diagnostic delays can result in serious patient harm. I'll give you an example regarding newborn jaundice management. During the peak of managed care in the 1990s, bundled payments to hospitals for procedures—such as newborn delivery—gave hospitals incentives to limit their costs for those procedures. A negative consequence was that many hospitals discontinued routine laboratory bilirubin tests for newborns and instead relied on less expensive and less effective visual assessment of jaundice levels, delaying accurate diagnosis and treatment of newborn jaundice.

Furthermore, hospitals received an additional payment when newborns would be readmitted for eventual bilirubin testing and appropriate treatment. This delay in diagnosing and treating newborn jaundice resulted in the reemergence of kernicterus [brain damage from jaundice] in the USA. I think that has changed, but we have to really look at how our payment models are structured to ensure that there are no incentives that could inadvertently delay a diagnosis or actually harm the patient.

MICHAEL CONSUELOS: I have a question for the group. Physicians have been under an RVU-based [Medicare's relative value units] or production-based model for decades unless you're in some sort of salaried position. Is it more about the regulatory stuff to make sure there's documentation for billing or is it really the productivity piece?

DR. GRABER: That's a good point and I think it's a little of both. To pile on the problems with the current payment system, there are no more autopsies in the United States, by and large, and that's completely because it's not paid for anymore by CMS [the Centers for Medicare and Medicaid Services] or by private insurance. You have to pay for it out of pocket, and that has led to a problem with feedback.

There's a lot of overconfidence out there, and that's a problem. There used to be somebody called a clinical pathologist in every laboratory who the clinicians could call up and say, "Hey, what's the best diagnostic workup for my patient who's jaundiced?" and get some advice on how to interpret tests or come up with the best strategies. That person is gone, and again, it's because it's not paid for, it's not covered in any modest payment model. Very bad for diagnosis to have lost that individual.

DR. CONSUELOS: Organizations should be able to spend their time and effort and resources to find out where their greatest risks are, and where diagnostic error is most likely to occur. To really look for solutions. And maybe the money should be spent around allowing physicians and other providers to spend more time thinking about the patient's diagnosis and treatment plan and freeing them up from the burdens that are not related to patient care.

MS. SIDRANE: Speaking about burdens on physicians, what about medical malpractice? Do you think that provider judgment is influenced by concerns about malpractice?

MS. SIEGAL: That sounds like that's coming right at me. This is a question we've asked often. On the one hand, there is always the question about over-diagnosis and are we doing too much, and are we getting into trouble by the more we do? I'm not sure that over-diagnosis or even just doing more tests—that might seem appropriate to some but not others—is always driven by the fear of malpractice. Our providers truly have a desire and a commitment to do the right thing by their patient and to get the right answer as quickly as they can.

That said, I do think that there are providers who would have trusted their gut and said, you know what, I'm not sure we need that MRI [magnetic resonance imaging scan] right now, let's hold on this for a little bit longer. But then, they decide to do it anyway, for fear that they are missing a positive finding and might be sued. I do believe there is some unavoidable fear of retribution, or of being sued for not doing everything possible in the moment, that drives the volume of testing providers do.

If anything, we might be reducing diagnostic error if they were going all-out for everything they thought would be helpful. I'm just not convinced fear of lawsuits is the only thing driving that.

DR. GRABER: The goal of malpractice originally was to ensure that the quality of medical care is at a high standard, and from my perspective, it has failed pretty miserably in achieving that goal. It's not good for learning; almost every case is concluded with settlements that are sealed. You can't really find out what happened. There's very little learning that derives from this.

It's certainly not good for patients. If you talk to patients who have gone through the malpractice mill, even the ones who win their case, they're disappointed because they didn't ever really get an apology; they didn't really find out what happened. There was usually very little commitment from the organization to fix things that are broken, so it's a disappointment to them. And in terms of getting them compensation, it's horrible. There are so many people who are harmed from diagnostic error, but just the very tiniest minority get any compensation from the current malpractice system.

And it's not even good at identifying bad doctors. There are a few out there, and we could do a much better job at finding them and finding different things for them to do besides trying to diagnose patients.

MS. SIEGAL: Thanks, Mark [Graber]. I do think there are many challenges in the litigation process, but I would have to disagree with your comments about limited learning opportunities. Yes, with reference to the laws and litigation processes, I would completely agree with you about its vulnerabilities. But with respect to the learning that's available from studying malpractice cases, it is far more valuable than you've credited it for.

Perhaps an opportunity exists for us to expose the data more and begin to see the value we can gain from studying the malpractice claims in large volumes that show the repeated system failures, communication failures, patterns of behavior failures that contribute to diagnostic failure. There's much to be learned, and perhaps we haven't positioned it yet in a way that people can hear and benefit from it. But I would beg to differ with you on what there is to learn. We are learning valuable lessons. Our entire risk management programs are driven by our learnings from our malpractice cases.

DR. GRABER: You're absolutely right, and we can learn so much from closed claims analyses and the deep kind of learning that you all are doing at CRICO and other organizations. I was talking about all those cases that are finalized with nondisclosure agreements and are taken off the market in terms of learning.

MS. SIEGAL: They may be off the "public" market, but they all live in our claims management systems; and for those insurers that share in our national malpractice database [CRICO's Comparative Benchmarking System (CBS)], we have every claim—open and closed, dismissed, or settled—available for analysis.

MS. GOESCHEL: Dana [Siegal], I agree with you, but something that you said at the end is critically important. For people who know to look at CRICO or to search for things, there's wealth of learning. But there is a larger audience. And you talked about positioning the information differently, leveraging it differently. There is a significant need to do that for the betterment of people who have interacted with healthcare and of those who have not yet had the fortune or misfortune to do so.

MS. SHERIDAN: I doubt that patients really know about what you learned in your database. Given confidentiality agreements that patients sign, it is hard for the public to benefit from the lessons learned, This is a problem. However, there are more and more patients who are refusing to sign such agreements in hopes of driving change and improvement across the healthcare system. Studies show that is the number one thing that patients want after experiencing harm in our healthcare system. Also, there's a really nice phenomena going on where patients who are engaged in litigation are sitting down with the hospital and the providers and agreeing to improvements and new policies in that hospital to prevent similar events, and they share it with other hospitals.

That's something that we can foment and encourage. And, frankly, I was one of the litigants 15 years ago, and it wasn't all about money. This is about policy change, and it did happen in both healthcare systems. We need to encourage that dialogue more and more—that we don't fear it and hide it. All the kernicterus cases that came before my son had been sealed and I was never aware of them. The fact that newborns were suffering brain damage from the failure to test jaundice levels never rose to the public eye, and that's what we need to make sure happens.

MS. SIEGAL: There is a difference between learning from the malpractice cases, which are somewhat removed from the event, and something else Mark [Graber] mentioned that's really critical. And, Sue [Sheridan], you're alluding to it in some way, too: the movement for apology and disclosure has got to go bigger, broader, and more aggressive, because we are still not embracing our patients as openly and as honestly as we should.

And to your point, allowing them the opportunity to participate in individual events that we can better learn from in real time is essential. We're so afraid of the litigation that we're not having honest, transparent conversations. And that's a problem. But apology and disclosure are a very different way and a very important way of learning in real time, versus the hindsight of data analysis that tells us where to go back and look in our systems. We need both.

MS. SIDRANE: A frequently cited concern regarding diagnostic error is the difficulty of quantifying it and the lack of adequate forms of measurement.⁷

DR. GRABER: Measurement is our weakest link. It's true that there's not a single healthcare organization in the country that's measuring their rate of diagnostic error, and that's because the tools that healthcare organizations use now to monitor patient safety problems don't readily find or detect diagnostic errors.

We need new ways to find these errors, and it isn't going to be that difficult. All we have to do is more effectively ask the patient, did we get it right at an appropriate point in time after they've been through a diagnostic experience, or ask the doctors.

I'm very impressed by a study that Bob Trowbridge did at Maine Medical Center. He's a hospitalist. He just told his hospitalist colleagues that he was interested in diagnostic error and could they bring cases to his attention. In a six-month period, they collected 36 cases of diagnostic error, not a single one of which would have been detected by the ongoing patient surveillance tools that the hospital used. That's a model for how we can find diagnostic errors going forward, in terms of getting physicians more interested in reporting them.

Beyond that, there are many ways to start measuring different aspects of the diagnostic process. Take timeliness. How long does it take between when you present with a lung nodule and a biopsy is done? How long does it take to diagnose anemia? There are all sorts of things we could start looking at, in terms of timeliness. It has also been pointed out by many people that even if you can't measure it, there are many opportunities to try different kinds of improvement efforts. And until we have good measurements, there are a lot of simple things that could be put in place now that would improve the safety and quality of diagnosis.

MR. EPNER: It also depends on if we're talking about what I call macro or micro measures. Macro measures are national measures that help guide policy—what's the total burden to patients, the total harm, the total cost?—so we know how to prioritize the allocation of resources. And then at the patient level or the physician level or the health system level, the micro level, how do we know whether there's been a suboptimum outcome for a patient at the real frontline level? There's a lot more work for both types. We're making progress, but we're not there yet.

DR. SINGH: I agree with Mark [Graber] that measurement is our weakest link. But something that is getting closer to maturing as a possible safety indicator in this area is how often abnormal test results are followed up within an institution. It's very quantifiable. We've done some work to figure out how we can use health IT and electronic health record data to try to identify patients who have missed follow-up. That could become, with a bit more time and thinking towards implementation, one measure where we could actually try to figure out how we're doing. But again, I want to emphasize, this is going to be hard because we have to balance the many measures that are being sprung on clinicians and hospitals. Some of them are just not worth anything, and we're often still measuring stuff just for the sake of measurement. It's not actionable measurement, it doesn't lead to anything concrete or any improvement.

DR. MOSHER: To follow up on Hardeep [Singh's] comment that we need measures that actually drive improvement: In radiology, we have American College of Radiology's (ACR) RADPEER, which is a national program in which we do double reads, and we look at discrepancy rates between readers. We have good data on national discrepancy rates (between one and three percent for what we consider clinically significant errors). And although that might sound like we're being quantitative and robust about it, we're finding that it doesn't drive change. It gives you the zone of quality that everybody else is demonstrating—therefore, I'm good. And we're also finding that physicians aren't buying into it because they realize it has very little value on improving their practice.

The trend now is really to focus on participation in group learning. Where are the opportunities where we see we're making mistakes? Let's be more focused and deliberate on trying to learn from those mistakes as opposed to simply trying to quantify them. I would just say, we need to be looking at actually trying to improve the process, and not simply measuring the process.

MR. EPNER: And just one more point to emphasize for this issue of measurement. Just Wednesday morning, I was in San Francisco for a meeting with [a donor], and I made the point that not all things that are important can be measured. And [he] said, "Yeah, we can't solve all important issues, so we're only going to focus on the ones that can be measured." So measurement is going to be an important issue in driving change, because I think his views reflect the views of many others as well.

MS. SIDRANE: A recurring topic in the literature is the importance of following up on test results and how the care team can prevent information from falling through the cracks.⁸

DR. SINGH: We've developed several best practices in the area. The problem is implementation of these best practices and adoption of these best practices among healthcare systems. One example is the Office of the National Coordinator for Health IT's (ONC) Safety Assurance Factors for EHR Resilience (SAFER) Guides⁹ that we developed back in 2014. They were updated last year again. They are available on HealthIT.gov's website. They're free, but few health systems use them.

In addition to closing the loop on test results, there is one on improving communication, which might be relevant to diagnosis. We've also developed several tools within the VA [Veterans Administration] that offer best practices, policies, procedures, and workflows^10,11 that are fairly generalizable and worth disseminating. And AHRQ [Agency for Healthcare Research and Quality] has developed an outpatient testing process toolkit¹² that is also available freely on their website. So there are definitely lots of best practices, especially in the area of test results and communication, but the issue is adoption and implementation.

MR. EPNER: Sometimes we try to make this too difficult. That's not to say implementation is easy, but Amazon and FedEx and UPS—they don't have the problems with closing the loop on delivery. And they know how to do it, they know how to keep the people informed along the way, they know when to raise the alarms when things are missing and don't happen along the way.

So it's not a technology discovery, it's an application issue. It's development, not research that's needed to just use the same tools that are already used in other industries to ensure that information flows, gets to the appropriate spot, and is acted upon. And there may be multiple appropriate spots, both healthcare professionals as well as patients, to do that if we just develop the will to do it. It's not a big leap, technologically.

MS. HASKELL: That's a great point, Paul [Epner]. Having test results go to the patient automatically, unless the patient has requested otherwise, is a big aid in this. Patients won't always see them, they won't always look, but it can help make sure that things don't get missed.

DR. KNESTRICK: Patients want to have their results, whether they're positive or negative, so it's important to let them know that it's OK to call and say, "I haven't gotten my results, what were they?" We have a lot of systems in place now with patient portals and other devices to do that, but not everybody is able to access information that way. So we need to leave the door open for the patient to be involved with their test results.

DR. MOSHER: One of the things that we found, regarding the closing the loop on incidental findings test results, is the real need to be proactive and actually have a nurse coordinator who really manages all of those test results and communicates with the patient.

We verify then that any follow-up studies have been ordered, and make sure that the provider is in the loop. We found that if we took a more passive role, that while we were communicating to the patient by sending them letters, that loop wasn't actually being closed. So if I were to say one thing as far as the best practice, it is that you really need verification, along the lines of what Amazon and FedEx have, of making sure that loop is actually truly being closed by somebody who manages and oversees that cycle.

MS. SIDRANE: What happens when the loop is closed but before a provider has an opportunity to discuss it with the patient, especially for something that's a more sensitive diagnosis, like cancer? Is that a situation in which it would be better for the providers to have a conversation ahead of time with the patient, or should all diagnoses be treated the same?

DR. KNESTRICK: The loop isn't closed just because the patient saw that information on the patient portal. I had that happen to me with someone who had a brain tumor and saw it on the patient portal before—over the weekend, and no one was there to talk to her about what the testing was. As soon as I realized that she had accessed that information, we called her right away and got her in for an appointment. But she had fretted all weekend about that information and not knowing, was looking things up on the internet.

In some ways I'm happy that she had access to it, but other ways holding off a little bit before it was posted there would have been better. But I still think we have the responsibility to make sure that feedback loop is closed. Just because she got the information didn't mean that the feedback loop was closed.

MR. EPNER: There are two key points here. One is that we should not be patronizing to patients in deciding what's good for them or bad for them. However, information that's out of context is not necessarily good for them. And I think what most systems do is embargo the results for 48 or 72 hours or something, and in some cases it's only for specific results, such as sexually transmitted diseases or potentially life-threatening things.

But, at the same time, if we don't at some point make sure the patient has access, we run the risk of something falling through a crack and patients feeling the consequence. That's why I recommend allowing a physician or a clinician, clinical team, to have 24, 48 hours to create context and contact the patient, but then at the end of that period if nothing has happened, then the communication goes directly to the patient. That seems to be the right balance of responsibility in making sure patients do have the opportunity to know what's going on.

DR. SINGH: We've found that patients may get anxious and have negative emotions even with normal results. It's a very anxiety-provoking experience, more than we think. And there are no national best practices on what the best timing of releasing test results is. When do you release abnormal imaging? When do you release labs?

We've heard anecdotally of patients getting abnormal results and taking detrimental action, including somebody who made funeral plans before they talked to the doctor based on what they saw on the test results. This is not an easy thing, just directly releasing all test results to patients and expecting them to follow up and interpret and manage them on their own. We're going to need to develop strategies to make sure we're doing this right. We recently called for a national platform of creating some best practices and standards in this area. I don't think we need to have different standards in Rochester versus Houston on when we should release imaging results that are abnormal to patients.

MS. SIDRANE: If someone called you tomorrow and asked what they could do to improve diagnosis in their own facility, what would you tell them?

MR. EPNER: Get started.

DR. KNESTRICK: Include all members of the team.

DR. GRABER: My number one recommendation to healthcare organizations is that they should start to identify and learn from diagnostic errors in their own patients. These errors are happening every day in every hospital, and there's a lot of harm. We estimate that there's 10 deaths every year in every hospital, and if places started to see that it's affecting their own patients, it would make it real, and it would also show them that this is an actionable problem. There's a lot that could be done to address this.

MS. GOESCHEL: I second that. And the thing that I would add is to acknowledge and to be transparent about it, not only within their own organization but outside of the organization. Patient safety picked up traction when everybody started saying, yeah, it happens here too. When we begin to own that this stuff happens everywhere, some of the leverage of folks who want to point fingers or say we're better than that will dissipate. Owning that this really is ubiquitous, that it does happen everywhere.

MS. SIEGAL: I couldn't agree more. And having been a clinician some years ago and on the teaching side now, if I had one message for leadership within organizations it's exactly that; talk about it openly and honestly, talk about it at your board meetings, talk about it at your staff meetings, talk about it when you're doing your rounds. It's up to leadership to make sure that everyone in the organization knows that it's a critical topic.

If you never hear about the diagnostic issues that we're having, or never hear that you're part of a diagnostic team, you will never begin to embrace it yourself. They've got to start talking about it at every level in their organization. Leaders need to lead the charge in order for everyone else to recognize that that's a direction they need to walk in.

DR. CONSUELOS: The prior folks have captured it. It's about the conversations with our leadership. And the one thing I would add is that I would ask our leaders to hire for this. This should be part of the ever-growing softer side of healthcare leadership and hiring for their junior staff based on their philosophy around this issue and quality and other important patient care aspects. It's not just about productivity, it's not just about research. Hire for these kind of competencies; hire leaders who think this way, who talk about improving quality and reducing errors.

MS. HASKELL: People should adhere to the principles of a high-reliability organization. In a facility, so much happens that is between providers, and my concern there is care coordination. So you have people making sure that the voice of the patient is heard, and that things are being followed up on and things are being noticed, so that when there are changes it doesn't fall through the cracks.

DR. MOSHER: I'd say one of the things that we need is to push it down to specific work units, where people are working around common symptoms or common problems, and they need to determine where there are opportunities to improve diagnosis. Start off with where you have a large degree of variability in diagnosis.

Give you an example in ours, I'm a musculoskeletal radiologist, and we found that we were having high variability when we were talking about our diagnosis of osteomyelitis in diabetic feet using an MRI. That gave us an opportunity to really come together, and by having a focused area that was a target opportunity for us to work in, we were able to see some real changes in a relatively short period of time.

It's easy to go in and say, look at this big massive problem, and you're quickly going to become overwhelmed. Focus on the day-to-day practice. What matters to you, what matters to your patients. Focus on those because then you're going to have the motivation to really carry these through to completion.

MS. GOESCHEL: And to Sue [Sheridan's] point earlier, it's crucial to raise the awareness of diagnostic challenges not only in our own organizations, but outside of our organizations, to embrace our patient community, foster very active patient and family advisory committees, and to be sure that our thought process constantly recognizes that as we build our teams and our partnerships for a safer world that our patients are key leaders in that journey as well.

_______
Definition of diagnostic error, per the National Academy of Medicine (NAM; formerly the Institute of Medicine [IOM]): the failure to (a) establish an accurate and timely explanation of the patient's health problem(s) or (b) communicate that explanation to the patient.¹

Notes

1. Committee on Diagnostic Error in Health Care; Board on Health Care Services, Institute of Medicine, The National Academies of Sciences, Engineering, and Medicine. Overview of Diagnostic Error in Health Care. Chapter 3. In: Balogh EP, Miller BT, Ball JR, editors. Improving diagnosis in health care. Washington (DC): National Academies Press; 2015. Overview of diagnostic error in health care. Also available: https://www.ncbi.nlm.nih.gov/books/NBK338594/. 
2. Leape LL, Berwick DM, Bates DW. Counting deaths due to medical errors—reply. JAMA. 2002;288(19):2405. Also available: https://jamanetwork.com/journals/jama/fullarticle/1845204 doi:10.1001/jama.288.19.2405-JLT1120-2-3
3. Committee on the Learning Health Care System in America; Institute of Medicine. Smith M, Saunders R, Stuckhardt L, McGinnis JM, editors. Best care at lower cost. The path to continuously learning health care in America. Washington (DC): National Academies Press; 2013 May. Also available: https://www.ncbi.nlm.nih.gov/books/NBK207225/?report=classic
4. Institute of Medicine (US) Roundtable on Evidence-Based Medicine. Yong PL, Saunders RS, Olsen L, editors. The healthcare imperative. Lowering costs and improving outcomes. Workshop series summary. Washington (DC): National Academies Press; 2010. Also available: https://www.ncbi.nlm.nih.gov/books/NBK53920/
5. Esposito L. Many patients lie to their doctors, survey finds. U.S. News and World Report. 2015 Nov 10. Also available: https://health.usnews.com/health-news/health-wellness/articles/2015/11/10/many-patients-lie-to-their-doctors-survey-finds
6. Frieden J. Teach patients how to communicate symptoms better --patients should learn the eight characteristics of a symptom. MedPage Today. 2015 Oct. Available https://www.medpagetoday.com/practicemanagement/practicemanagement/53833
7. Graber ML. The incidence of diagnostic error in medicine. BMJ Qual Saf. 2013 Oct 22 Suppl 2:ii21-ii27. Epub 2013 Jun 15. Also available: https://qualitysafety.bmj.com/content/22/Suppl_2/ii21.long doi: 10.1136/bmjqs-2012-001615
8. Singh H, Spitzmueller C, Petersen NJ, Sawhney MK, Sittig DF. Information overload and missed test results in electronic health record–based settings. JAMA Intern Med. 2013 Apr 22;173(8):702–4. https://jamanetwork.com/journals/jamainternalmedicine/fullarticle/1657753 doi:10.1001/2013.jamainternmed.61
9. Office of the National Coordinator for Health Information Technology (ONC), U.S. Department of Health and Human Services. SAFER Guides. Last reviewed 2018 Apr 4 [accessed 2018 Oct 20]. https://www.healthit.gov/topic/safety/safer-guidesOffi
10. Singh H. A checklist to improve CPRS 'View Alert' notifications: recommendations for VA facility leadership and staff. 2018 [cited 2018 Oct 5]. https://www.researchgate.net/publication/325439240_A_Checklist_to_Improve_CPRS_%27View_Alert%27_Notifications_Recommendations_for_VA_Facility_Leadership_and_Staff doi: 10.13140/rg.2.2.29002.88007
11. Singh H, Wilson L, Reis B, Sawhney MK, Espadas D, Sittig DF. Ten strategies to improve management of abnormal test result alerts in the electronic health record. J Patient Saf. 2010 Jun;6(2):121-3. Also available: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2885732/ doi: 10.1097/PTS.0b013e3181ddf652
12. Eder M, Smith SG, Cappelman J, Hickner J, Elder N, Singh G. Improving your office testing process. Toolkit for rapid-cycle patient safety and quality improvement. AHRQ Publication No. 13-0035. Rockville, MD: Agency for Healthcare Research and Quality; 2013 Aug [cited 2018 Oct 5]. Also available: https://www.ahrq.gov/sites/default/files/publications/files/officetesting-toolkit.pdf