When Charts Cry Wolf

EHR drug-drug interaction warnings may ultimately save lives, but does the collateral damage of “alert fatigue” negate the benefit?

EHR drug-drug interaction warnings may ultimately save lives, but does the collateral damage of “alert fatigue” negate the benefit?

“Warning: There is a SEVERE interaction between aspirin and heparin. Are you sure you want to proceed?”

ADVERTISEMENT

This is something my ED information system warns me about – often within a few minutes of telling me my patient has an elevated troponin. I’m forced to acknowledge a pop-up window if I want to give a drug, and the medication appears in a distractingly red box.

At some point, someone thought this drug-drug interaction warning (and hundreds like it) was a good idea for patient safety. Given the reports about rampant medication errors, the thinking was that software could help tired, overworked doctors avoid potentially harming patients.

More drugs were introduced and more interactions reported. And now, after years of this, most of us working with CPOE (computerized physician order entry) suffer from “alert fatigue” – we’re bombarded with so many warnings every time we place an innocuous medication order, we’re numbed to their effect. We just absent-mindedly click “OK” over and over, potentially skipping that rare interaction we forgot about, and negating the original intent of the warnings.

ADVERTISEMENT

Researchers have taken up the question: is CPOE and its associated drug-drug interaction warnings actually helping patients? And if so, how big a threat is alert fatigue to this benefit?

Unfortunately, most of the literature in this field is focused on the inpatient and ambulatory clinic environments, not the ED. And even within the hospital and outpatient reports, most results focus on operational metrics and error rates, rather than clinically significant outcomes and impact on mortality.

The lack of CPOE research in the ED environment is especially disappointing, given the high priority and framework for future efforts placed on these studies in a 2004 consensus conference paper by Handler and others (Handler 2004). CPOE, however, has been shown recently to reduce ED length of stay (Spalding 2011) and lab turnaround time (Westbrook 2009, Baumlin 2010). As a proxy for CPOE, Bizovi and others found that computer-assisted electronic prescriptions from the ED were significantly less likely to contain errors or require pharmacist clarification (Bizovi 2002).

ADVERTISEMENT

The Outpatient Setting
We can learn a few things from our colleagues in the ambulatory care world. One especially notable study in this environment, by Weingart and others (Arch Intern Med. 2009), looked at over a quarter-million prescription alerts generated over six months in Massachusetts. They calculated 402 adverse drug events that were avoided thanks to clinicians heeding interaction warnings (49 were judged as serious and 3 were potentially fatal, with dozens of likely prevented hospitalizations and ED visits).

On the flip side, they calculated that it took over 2700 warnings to prevent one serious adverse drug event, and that over 90% of alerts were overridden. Moreover, it seems that just 10% of the alerts accounted for the majority of adverse event prevention and cost savings. These results prompted the authors to wonder “whether the juice is worth the squeeze” (their words, but worth repeating). While acknowledging the limitations of the study in showing harder-to-quantify benefits of electronic prescribing (such as standardized dosing selections and formulary adherence), the authors concluded that the technology is worthwhile, but something’s got to be done about the volume of clinically insignificant alerts.

To that end, this group published the drug interactions most likely to be accepted by outpatient clinicians (go to www.dana-farber.org and search for “potentially dangerous drug combinations”).

Even the best “high-severity” alerts were only accepted between 25-45% of the time, while the worst “high-severity” alerts had a 2-8% acceptance rate (www.dana-farber.org). Hopefully, efforts like this will lead vendors and pharmacists to scale back the volume of clinically insignificant alerts, improving the signal-to-noise ratio and preventing alert fatigue. Interface improvements and usability studies will also help in this effort.

ADVERTISEMENT

Inpatient CPOE studies and the Hard Stop
It’s worth noting, however, the effectiveness (and downsides) of one of the worst interface elements – the dreaded “hard stop” alert that prevents doctors from moving forward on the electronic chart. At two academic centers in Philadelphia, Strom and others (Strom 2010) set up one of most reasonable drug interaction warnings: when clinicians tried ordering Bactrim to patients on Coumadin (or vice versa). But they used a “hard stop” – the interface would force users to either type in the indication for pneumocystis carinii treatment and acknowledge a personalized warning to proceed, or pick up the phone and talk to a pharmacist as the only way of getting the order to go through. The hard stop worked, compared to a control group (in which orders went through easily and hospital pharmacists would phone doctors later with concerns). The hard stop forced more than half of providers to use different medications. However, the trial was stopped early because of an unintended consequence in four patients. Two patients had day-long delays in receiving much-needed TMP/Sulfa, and two patients didn’t get warfarin in a timely fashion.

The clear lesson here is that CPOE and usability decisions can impact care in unpredictable ways. However, more data from the inpatient world suggests the benefits outweigh the harms. Longhurst and others (Longhurst 2010) studied a CPOE implementation at an academic children’s center (Stanford) and found an associated decrease in mortality over 18 months, suggesting 36 deaths might have been prevented with CPOE. This study stands in contrast to the earlier, shocking study by Han in Pittsburgh, (Han 2005), where CPOE was associated with a rise in pediatric deaths.

In Reckmann’s 2007 systematic review of 12 inpatient studies on CPOE and prescribing errors (Reckmann 2009), nine studies showed a significant decrease in errors with CPOE. The severity of the errors was inconsistently described, however, and many seemed minor. To be fair, the authors also note new errors associated with user/CPOE interface mistakes, such as duplicate orders, or incorrect selection of dosages or delivery methods. These new kinds of mistakes haven’t yet been categorized for rigorous analysis.

Enter Meaningful Use
We’re all a lot more likely to see CPOE and drug-drug interaction warnings, warts and all, in the near future. This is because of CMS’ plan for Meaningful Use incentives. As we’ve been covering in this series, Medicare and Medicaid incentives for demonstrating Meaningful Use of electronic records is spurring a lot of activity in emergency departments, hospitals and doctors’ offices. Places that had delayed implementation of electronic health records (EHR) for years are jumping on board – in part to catch some of the stimulus dollars, but also because these carrots are set to become sticks in a few years. Planned penalties will take the form of decreased Medicare and Medicaid reimbursements if providers or organizations don’t adopt EHR.

To qualify for stage one meaningful use incentives (beginning this year), hospitals have to attest that clinicians have ordered at least one medication via computer on 30% of unique inpatients or ED patients. The EHR also must have drug-drug and drug-allergy interaction checking in place for these orders.

Given the speed at which these incentives went from proposal to rule to collection period (CMS has already started distributing checks to provider
s and hospitals, less than two years after the ARRA stimulus bill passed) it seems naïve to expect EHR vendors to have developed more intelligent drug alerts or less frustrating CPOE experiences.

But there are several reasons for optimism. First, with widespread adoption of CPOE, complaints regarding usability problems and alert fatigue will only grow louder. The reporting (some mandatory, some not) and research that will emerge from all this EHR implementation should drive improvement in the CPOE arena.

Moreover, with Meaningful Use, a framework for evaluating and certifying EHRs has been developed. While usability and alert fatigue haven’t been part of the criteria for certification thus far, it stands to reason that the bar for certification will be raised with time, and considerations important to clinicians should be heard.

Stage two and three are still just proposed, with a period of public comment and reevaluation planned. But already it seems that drug-drug interaction checking will have to get smarter, perhaps incorporating a patient’s age, weight, or pregnancy status before producing an alert.

Finally, the development I’m most excited about: the rise of smart Clinical Decision Support systems and tools. Really, drug interactions are the simplest, most common, but also most frustrating form of “clinical decision support” (CDS). The thing is, CDS at its best can be quite sophisticated and effective. There’s a growing body of case reports of successful implementations that promote evidence-based guideline adherence, even in the ED (Melnick 2010). Melnick and others suggest future clinical practice guidelines be developed with an eye toward easier incorporation in electronic CDS systems (Melnick 2010).

Clinical calculators and links to online resources can be baked into CDS tools, gently prompting clinicians to learn about new standards. Studies in the inpatient and ambulatory care world have shown that the level of care can be raised (consider Riggio 2009, Schnipper 2010 or Weber 2008). Some trials underway may demonstrate cost savings or impact ED care.

CDS is also baked into the core requirements for demonstrating meaningful use of EHR. For stage one, just one rule needs to be implemented – a common choice among hospitals is DVT prophylaxis reminders for inpatients. But for stage two and beyond, it’s likely that more CDS will be required, and it’s been proposed that these rules be sensitive to patient context, and relevant to high-priority disease states. Furthermore, these aids should show up in a timely manner that promotes an efficient workflow, and (naturally) be evidence-based.

My dream is that someday, EHRs will have developed to the point where drug alerts are only displayed on truly significant interactions specific to my patients, and that clinical decision support is advanced enough to incorporate elements from my patient’s vitals, diagnoses, and lab results before nudging me to improve my orders. Who knows? The day may not be far off when my EHR is smart enough to know my patient is having an MI, rather than harassing me about mixing aspirin and heparin.

References

• Handler J et al. Computerized Physician Order Entry and Online Decision Support. Acad EM. Nov 2004. 11(11); pg. 1135-42.
• Spalding S, et al. Impact of computerized physician order entry on ED patient length of stay. Am J EM 2011  Feb;29(2):207-11.
• Baumlin K, et al. Clinical information system and process redesign improves emergency department efficiency. Jt Comm J Qual Patient Saf. 2010 Apr;36(4):179-85.
• Bizovi K, et al. The effect of computer-assisted prescription writing on emergency department prescription errors. Acad Emerg Med. 2002 Nov;9(11):1168-75.
• Weingart S, et al. An empirical model to estimate the potential impact of medication safety alerts on patient safety, health care utilization, and cost in ambulatory care. Arch Intern Med. 2009;169(16):1465-147.
• Strom B, et al. Unintended effects of a computerized physician order entry nearly hard-stop alert to prevent a drug interaction: a randomized controlled trial. Arch Intern Med. 2010 Sep 27;170(17):1578-83.
• Melnick E, et al. Knowledge translation of the American College of Emergency Physicians’ clinical policy on syncope using computerized clinical decision support. Int J Emerg Med. 2010 Jun 1;3(2):97-104.