Merely utter the term “patient satisfaction” among most emergency physicians and you’ll quickly see us retreat into a world of skepticism and exasperation. The term has become symbolic of a never-ending battle over flawed data and an inaccurate evaluation of our performance.
How our new cloud-based patient sat surveying techniques have increased participation tenfold and created game-changing performance metrics.
Merely utter the term “patient satisfaction” among most emergency physicians and you’ll quickly see us retreat into a world of skepticism and exasperation. The term has become symbolic of a never-ending battle over flawed data and an inaccurate evaluation of our performance. Between the imbalanced power of a single complaint and the volatility of our scores (how did we drop 80 points in one month?), it’s clear to most physicians that there are flaws in the measurement process too grave to tolerate. But while current approaches to assessing patient satisfaction in the emergency department (ED) have significant limitations, we can’t throw out the concept entirely. We should embrace new and innovative approaches to patient satisfaction that will actually enable delivery of better healthcare.
The statistical shortcomings of current approaches to assessing the patient experience have been well described in this publication already. What is often lost in the argument about the quality of patient satisfaction data is the power of the patient experience in improving clinical care. A growing body of scientific evidence shows that feedback from patients and families represents a unique dimension of quality that is otherwise impossible to objectively measure and is associated with better outcomes, such as reduced mortality and 30-day readmission rates. Patients directly observe their interpersonal care experiences, including patient–provider communication and care coordination, each of which represent unique dimensions of quality. For example, in a study of patients with acute myocardial infarction at U.S. hospitals, we found that service lines with higher patient experience scores had lower risk-adjusted in-hospital mortality rates (Glickman et al, Circulation: Cardiovascular Quality and Outcomes, 2010). And, importantly, these measures largely captured communication and care coordination with nurses and physicians, rather than non-care “aesthetic” aspects of the patient experience, such as room features and meals.
These strong relationships between the patient experience and more objective measures of quality clearly extend to the emergency department as well. As emergency physicians, we’ve all ordered an extra test or written a prescription for a patient based on their request, not necessarily because it was “good medicine.” However, a growing body of research shows that patients aren’t always seeking more tests or drugs, just better communication from their physicians and a shared approach to decision making. For example, in a recent randomized study of patients at low risk for acute coronary syndrome (Hess et al, Circulation: Cardiovascular Quality and Outcomes, 2012), a shared decision making approach led to decreased imaging utilization and higher levels of patient satisfaction.
Major Shortcomings of Current Measurement Approaches in the ED
A major flaw of current ED patient satisfaction surveys is that the sampling strategy does not align with how our time is actually spent in clinical practice. Recently, I spent the first three hours of a shift solely focused on resuscitating a previously healthy, middle-aged woman who presented to the emergency department with life-threating hyperkalemia, bradycardia, and respiratory failure. She survived, and following a 2-day course in the ICU was transferred to the floor in stable condition. Just before I left her bedside in the ED her husband hugged me and said he would be forever grateful. I then walked into the room of a patient who greeted me with “I’ve been waiting for three hours to see you – this is ridiculous.” As it turned out, her condition was relatively minor and she was discharged home shortly thereafter. I feel certain that this disgruntled patient has already mailed in her negative review while the patient I admitted to the ICU will never even receive a survey about the care I provided. Does this sound familiar?
We’ve also all suffered from the attribution problem, where negative comments assigned to us are really about some other aspect of the patient’s experience over which we have limited control, such as wait times or triage issues.
And then there’s the problem of the precision of the data. The statistical validity of ED satisfaction surveys has been discussed repeatedly. Not only is there a sampling bias created due to surveying only discharged patients, but many organizations also choose to only survey a limited sample of their patients due to the high cost of current survey methods. One of the authors of this piece (RV) actually discovered that his individual scores jumped from the 1st to the 99th percentile in one year, based solely on the input from fewer than 10 completed surveys (despite personally treating several thousand patients during that time period). He then estimated that it would take six years of data collection to achieve a statistically significant result. When reviewing our Press Ganey ED patient satisfaction reports with our clinical operations group monthly we’ve found the data only marginally more valuable than that of a random number generator.
Embracing a New Measurement Approach in the ED
My colleagues and I believe this entire measurement construct needs re-thinking. We want to improve the patient experience, and we must have better data to make that happen. Our new company, Bivarus, has taken a step in that direction by creating a cloud-based system that leverages email and smartphones to deliver brief, customized surveys to ED patients shortly after their visits. We’ve re-engineered patient satisfaction surveying in 4 simple yet critical ways. First is timing: the immediacy of our online survey allows patients to share feedback while the visit is still fresh, and facilitates prompt intervention on the provider side when indicated. Next, validity: the Bivarus concept was developed within an advanced, Bayesian mathematical framework to ensure data accuracy and precise assessment of individual providers. Third, the Bivarus surveys have been painstakingly researched based on the input of multiple stakeholder groups in emergency medicine. And finally, we’ve emphasized customization. We allow physicians and managers to prioritize and customize content specific to their clinical care environments, versus relying upon an outside body to dictate questions without benefit of our unique insight and experience. For example, at UNC we’ve decided to focus heavily on issues related to patient safety, including medication reconciliation and care transitions during shift change.
Through our efforts we have shown we can achieve high patient response rates and deliver statistically valid reporting back to our clinical managers. Approximately 75% of patients provide us with either an email address or cell phone number and we survey every single one of them. On an average month in the ED we receive almost 1,000 patient responses, almost 10 times the number UNC used to receive with our traditional vendor using mailed surveys. Our response rates average 30% in the ED and 50% in other ambulatory settings, ensuring an accurate sample and statistically valid data. Importantly, we are also hearing back from a high proportion of our patients in traditionally underserved populations, including non-English speaking Latinos, African Americans, and the advanced elderly. Among our respondents who use smartphones, almost half (48%) are underinsured (no insurance or Medicaid only). Bivarus has been so effective, in fact, that it is now the vendor of choice in the UNC ED. Our monthly meetings now focus on how to respond to the data we are receiving in real time, not arguments about the data itself. The results have been rich– driving meaningful improvements virtually every day in care coordination, service integration, and patient safety (including areas like acute pain management, medication reconciliation, pharmacy stocking in the ED, environmental services management, and infection control). For example, we recently received feedback from a patient who experienced a long delay in analgesia for acute biliary colic during a very busy ED shift. Based on her feedback and that of several other patients who had experienced similar issues, our operations group teamed with our anesthesia group to develop a new protocol for the treatment of acute pain in the ED. And for the first time in many of the ED attendings’ careers, we are receiving not just negative but a tremendous amount of positive feedback from patients.
It’s time that as a specialty we move past our arguments about the legitimacy of patient satisfaction data and focus our efforts on finding practical and innovative solutions for improving patient care. Our new science around measurement of the patient experience allows us to translate patient feedback into immediate frontline improvements and convert the mysteries around provider evaluation into scientifically valid, comprehensive measures.
Thankfully, Bivarus isn’t alone in this fight, judging from the recent proliferation of new companies in the market. For example, companies such as SMART-ER and Emergency Excellence have developed innovative approaches to patient callbacks in the emergency department. And hopefully others will continue to follow suit. We finally have the tools at our fingertips to catalyze this long overdue paradigm shift. It is now up to the community of emergency physicians to advocate for our right to use these new tools in our practices. It is also up to the broader healthcare community to embrace this transformation as a means of achieving their goal of improved performance from the health care delivery system.
Seth Glickman, MD is an Assistant Professor of EM at the University of North Carolina. He is cofounder of Bivarus, Inc. Abhi Mehrotra, MD is the Assistant Medical Director and Director of Quality and Performance, Department of EM, University of North Carolina. He is an equity-holder in Bivarus, Inc. Rob Vissers, MD is Medical Director, Legacy Emanuel ED; Adjunct Associate Professor of EM at Oregon Health Sciences University
1 Comment
Great article and thanks for mentioning Smart-ER. We have been working to perfect perfect our system for real-time alerting of serious patient problems (e.g., worsening condition or refused follow-up) as well as relaying service complaints. As you mentioned, all this leads to a far better patient experience, which is vital as we move toward full-fledged accountable care. Further information on the advantages of a robust patient recontact system can be found at http://smart-er.net/research/.