CMS Benchmarking: I’ll Show you My Numbers if I Can See Yours

No Comments

altAs the director of a single-coverage community ED with about 22,000 visits a year and about 20% admissions, I’ve been fortunate to be left alone by the administration. The nursing director and I – along with a team of scribes, advance practice clinicians and staff, have largely been free to do whatever we could to improve throughput in the ED.

As of 2013, CMS is beginning to flex its muscle in the ED and is expressing interest in just how efficiently patients are being managed

As the director of a single-coverage community ED with about 22,000 visits a year and about 20% admissions, I’ve been fortunate to be left alone by the administration. The nursing director and I – along with a team of scribes, advance practice clinicians and staff, have largely been free to do whatever we could to improve throughput in the ED.

We had a door-to-provider time (not door-to-triage nurse) of about 30 minutes – not fabulous but a lot better than most EDs. Our door-to-discharge time used to be 2.5 hours for patients going home, but gradually it has crept up to about 3 hours. Our door-to-admission time was by no means exemplary – about 5 hours – but this was largely due to factors outside of our control. When we proposed Peter Viccellio’s full capacity protocol (sending patients to the halls on the admitting floors) I remember the response of the nursing VP – “over my dead body.”

But in many ways the ED nurse manager and I had the freedom to internally control how the ED operated (short of hiring more staff). California law, largely championed by state nursing unions, mandates a staffing ratio of one nurse per four beds. That’s one nurse per four beds – not four patients, so you can imagine the money wasted on overstaffing during the middle of the night. What we really wanted were more techs – lots more – to facilitate the work of the nurses and do the zillion chores performed by nurses that do not require their specialized training.

But besides these problems, we had a cool ED. We only had triage when the beds were full. We developed a “rapid entry” program so that patients were brought back to empty beds without the triage interrogation (Are your immunizations up to date? Is you spouse beating you? What is your preferred way to learn – visual, auditory? You get the idea.)

In 1993 we implemented a computer system we had developed (and which we licensed to other hospitals) that allowed, during the process of coding CPT and ICD-9 data, all manner of clinical and throughput information to be pulled off the charts. Bottom line – there was not a question we couldn’t answer with our system, yet the doctors and nurses did no data entry. It was all done after the fact by a coder who captured all nursing and physician charges as well as every test, drug and key time.

One of the frustrations experienced by the ED leadership was that, despite having all manner of data on how our ED operated (and in many ways it was exemplary), when the Joint Commission came around, none of our process metrics seemed to matter. We had the best metrics in our five hospital system, yet we never got any pats on the head by the JC because process and throughput metrics were not on their list of things to assess.

Instead, the JC was hot to see our policy and procedure manuals (which we would dust off every three years) and our quality assurance process. They also wanted to check that we took “time outs” to make sure we didn’t suture the wrong laceration or reduce the wrong shoulder. And God forbid we wrote MS rather than morphine sulfate so that no patient would be given magnesium sulfate for their broken leg.  

But now things are changing – finally. As of 2013 CMS is requiring a ton of information regarding ED process metrics and utilization. CMS, being a major payor of hospital and ED services, is now flexing its muscle in the ED and is expressing interest in just how efficiently patients are being managed.

As a result, ED leaders around the country are getting nervous. Providing this information can be a two-edged sword. Armed with new data, will CEOs provide the resources to allow EDs to improve their processes or assure that patients with long bone fractures will get decent analgesia promptly? Or will CEOs simply use this information to whip ED staffs into working faster and harder, without providing the needed resources to improve.

The risk is particularly high for contracted emergency physicians. Already it is not uncommon to hear CEOs mandating increased physician staffing when the real problem is inadequate nursing and ancillary staffing. And who is paying for the scribes because the invention of electronic medical records has been imposed on the ED staff?

Below is a list of information that CMS currently is, or will soon be, requiring of EDs:

  • Median time from ED arrival to departure for discharged patients
  • Median time from door to diagnostic evaluation by a “qualified” provider
  • Median time to fibrinolysis
  • Percentage of AMI patients receiving fibrinolytic therapy within 30 minutes
  • Median time to transfer to another facility for acute cardiac intervention
  • Percentage receiving aspirin on arrival for suspected ACS patients
  • Median time to ECG for chest pain patients
  • Percentage getting MRI for low back pain without prior conservative treatment
  • Percentage getting abdominal CTs with both contrast and no contrast in the same patient
  • Percentage getting simultaneous use of brain and sinus CT
  • Percentage getting brain CTs for atraumatic headache
  • Median ED time for pain management of long bone fracture
  • Percentage of ED patients left without being seen
  • Percentage getting head CT results back within 45 minutes of arrival for patients with ischemic or hemorrhagic stroke
  • Median time to PCI
  • Percentage receiving primary PCI within 90 minutes of arrival
  • Percentage getting blood cultures prior to initial ED antibiotics for ICU pneumonia admissions
  • Percentage of pneumonia patients given the most appropriate initial antibiotics
  • Percentage of patients considered for TPA treatment in stroke patients

As can be seen, the required information reflects both process measures and clinical measures. Currently there are no “right” answers. CMS wants the information and doesn’t say what are good numbers and what are bad numbers – but clearly this will come. And it is likely that more metrics will be required to be reported in the future. For example, it is one thing to give pain management for a long bone fracture quickly, but it is another to provide adequate pain management (in my view, treating a femur fracture with ketorolac just shouldn’t cut it). So, like so many metrics, the devil is in the details and, as such, there are many opportunities to challenge the process.

One of the metrics that some emergency physicians have strenuously objected to relates to reporting the use of CTs in headache patients. Go onto the internet and search “ACEP, CT, Headache, CMS” and you will find many articles complaining about this metric. One such item is a letter dated August 28th, 2012, from then ACEP President David Seaberg to CMS’s acting administrator, Marilyn Tavenner. Seaberg details concerns with the CMS rulings in no uncertain terms:

“We must restate our strong objections to proposed measure OP-15: “Use of Brain CT in the Emergency Department of A-traumatic Headache” and urge CMS to remove this measure from the final rule.” We appreciate the serious concern around the use and overuse of CT scanning and ACEP has established a clinical guideline in this area. However, this guideline is not meant to include older adults or adults on anticoagulant medications. In fact, four clinical practice guidelines and several studies note that new onset headaches in patients aged > 50, >60, or aged >65 are “red flags” that indicate that a brain CT should be performed. Other risk factors such as new onset headache in patients who are hypertensive, taking anticoagulants, HIV positive, presenting with a history of cancer, fever, neck stiffness, or rash have also been outlined in the literature. In a recent study, ACEP members from 20 hospitals across 12 states performed chart abstraction of OP-15 on 748 cases, 83 percent of which had either a measure exclusion or an evidence-based clinical indication for head CT imaging, which was not accounted for in the administrative data collected by CMS. This imaging efficiency measure has not been endorsed by NQF, because it is not reliable, valid or accurate, and it should be removed from the final rule.”

As noted in the letter, emergency physicians appreciate the overuse of CT scanning. Clearly there are patients who need CTs for headaches, but there is a limit. And remember, the “correct” percentage has not been determined – CMS just wants the data for now. The study by Schuur and colleagues that is referenced in Dr. Seaberg’s letter demonstrated some of the weaknesses of solely using ICD-9 codes and other administrative data to differentiate those who appropriately need a CT from those who don’t.

So, maybe the methodology to assess the use of CT scans in headaches needs to be refined (and it sounds like there is clearly room for improvement), but we shouldn’t throw the baby out with the bath water. The methodology proposed to measure CT usage in headaches is not without some merit and the area of interest – utilization of CT scans for headaches – is very important.

To help put this issue into perspective, let’s look at a couple of studies. The principle author in the first is Raja, MD. He’s with the Department of Emergency Medicine, Center for Evidence-Based Imaging at Brigham & Women’s Hospital. He is also one of the authors in the Schuur study referenced above. The Raja paper assessed NHAMCS data and claimed that neuroimaging was responsible for the greatest increase in diagnostic imaging in American emergency departments. Seems CMS would be perfectly appropriate in trying to get a handle on this utilization. The study estimated that 6.7% of all ED visits had a head CT in 2007 (clearly we must be doing more in 2013). And, as is typical, factors associated with receiving a head CT often had some socioeconomic overtones – less CTs in non-Hispanic blacks – and hospital location and ownership were factors in CT utilization.

Raja, A.S., et al, Arch Intern Med 171(3):260, February 14, 2011

BACKGROUND: Neuroimaging with head CT scanning and/or MRI is responsible for the greatest increase in advanced diagnostic imaging in EDs in the US. One study estimated that head CT scanning performed in 2007 in the US may result in 4,000 future cancers.

METHODS: The authors, from Brigham & Women’s Hospital in Boston, analyzed neuroimaging in US EDs based on the 2007 National Hospital Ambulatory Medical Care Survey (NHAMCS).

RESULTS: Based on 35,490 ED visits reported in the 2007 NHAMCS, it was estimated that head CT scanning was performed in 6.7% of the approximately 117 million ED visits nationwide during that period, and that MRI of the head was performed in 0.26%. The three primary reasons for head CT scanning included trauma (18.1%), headache (13.0%) and dizziness (6.1%), and the three leading discharge diagnosis categories were trauma (20.5%), headache (9.2%) and epilepsy/convulsions (5.2%). Independent patient-related predictors of reduced utilization of head CT scanning included decreasing age and non-Hispanic black race/ethnicity, and independent hospital-related predictors included rural location (odds ratio [OR] 0.75 vs. urban hospitals) and state or local government ownership (OR 0.63 vs. nonprofit ownership) (95% CIs below 1.0). Compared with patients aged 31-40, odds ratios (ORs) for head CT scanning were 0.38 in children aged 0-2 years but 1.80 (95% CI 1.33-2.44) in those aged 71-80 and 2.57 (95% CI 1.92-3.34) in those over the age of 80. When compared with Caucasians, the OR among non-Hispanic blacks was 0.85 (95% CI 0.72-0.99).

CONCLUSIONS: In 2007, one of every 14 ED patients underwent head CT scanning. The authors cite the need for identification of appropriate indications for advanced neuroimaging. 8 references ( – no reprints)
Copyright 2011 by Emergency Medical Abstracts – All Rights Reserved 8/11 – #30

But what about a look at head CT utilization at an academic, tertiary care center. Not just any academic center but, in fact, the same one where Drs. Shuurs and Raja practice – Brigham and Women’s in Boston – where the principle author of the following study also practices.

Now for some fun. This was a study of head CT utilization by staff physicians in their department in 2009.  Of the 38 physicians profiled, experience ranged from 0-30 years. Unlike the above study in which these folks found head CT utilization of 6.7%, the utilization at Brigham and Woman’s was 8.9%. Don’t forget that the data is two years newer and, as many speculate, the performance of CTs will continue to increase year after year. The bigger story is that the magnitude of variation in physician utilization of head CTs was huge. And this is a tertiary care center teaching residents how to practice emergency medicine. The unadjusted utilization rate varied between 4.4% and a highly risk averse 16.9%! And after attempting to adjust for confounders the spread was still disheartening – 6.5%-13.5%.  

Now for the jackpot – utilization in atraumatic headaches. The range was almost unbelievable: 15.2% to 61.7%! Adjusting for confounders narrowed the difference from “ridiculous” to “ghastly”: 21.2% to 60.1%. To make matters worse, level of physician experience was not associated with the rate of ordering. Given that this is happening at an academic center, I shiver to think what the residents are being taught. Most likely the residents are being taught: “Don’t miss anything,” or “Dr. X always orders them so you should too,” or even “patients think your care is better when you order tests.”

Prevedello, L.M., et al, Am J Med 125(4):356, April 2012

BACKGROUND: Medicare expenditures for high-cost imaging increased by an average of 17% per year from 2000 to 2006. Such imaging has been identified as one of the key drivers of increasing healthcare costs. Rates of head CT scanning have been noted to vary substantially between individual facilities, but the degree of variation between physicians within a single institution has not been explored.

METHODS: This study, from Brigham & Women’s Hospital in Boston, examined variability between 38 emergency physicians (29% female, post-residency practice 0-30 years) in rates of requests for head CT scanning in 2009.

RESULTS: Head CT scanning was performed in 8.9% of the ED visits overall (4,919 of 55,281 visits), and tended to be more frequent in males, older patients, those presenting with head trauma, and patients with more urgent presentations. Unadjusted rates of scanning varied between physicians from 4.4% to 16.9%. After adjustment for potential confounders, requests for head CT scanning varied between physicians by about two-fold (6.5-13.5%). Similar patterns were observed in the subgroup of patients with atraumatic headaches, in whom rates of head CT scanning ordered by physicians ranged from 15.2% to 61.7%. After adjustment for confounders, there was a nearly three-fold variation between physicians in head CT scanning in this patient subgroup (21.2-60.1%). Differences in rates of head CT scanning were not influenced by physician gender or level of experience.

CONCLUSIONS: This study demonstrates substantial variability between emergency physicians in a single institution in the use of head CT scanning, and the importance of identifying methods to decrease this variability and promote the appropriate use of imaging. 27 references ( for reprints)

Richard Bukata, MD
Editor of Emergency Medical Abstracts (


Leave A Reply