Another JAMA study claims ALS beats BLS. But while it raises important questions, this study has too many biases to form concrete conclusions.
“Patients who are having a heart attack, stroke or other serious health emergency have a greater chance of surviving if they’re taken to the hospital in a basic life-support ambulance rather than one loaded with sophisticated equipment.”
So reads the dramatic take home message from a piece recently published in the Washington Post. The article, based off of a recent study published by Sanghavi in the Annals of Internal Medicine [1], begs the question: Have we finally proven less is better in prehospital care and “advanced” life support is actually deadly? The newspaper says “you bet.” We think this warrants a closer look.
Specific ALS skills such as the use of pain medications and prehospital twelve-lead EKGs can be shown to improve com- fort or speed to treatment, yet there are no definitive clinical studies confirming an overall advantage of ALS over BLS. A head-to-head comparison of ALS versus BLS care was initially undertaken in the landmark “OPALS” study published in 2007 [2]. The prospective, randomized, multicenter, controlled trial was specifically designed to address whether ALS care added upon existing BLS care led to better outcomes. The study concluded that, in this Canadian model, ALS care had no survival advantage except for respiratory emergencies. However, despite a large sample and advanced statistics, there are still significant limitations to the study’s conclusions. The Canadian health care system has fundamental differences from its US counterpart and therefore outcomes may not be generalizable.
Definitions and Study Population
The methodology of Sanghavi’s follow-up study seems impressive at first glance. This was an observational study using a 20% sample of hospital billings of Medicare beneficiaries from non-rural counties between 2006 and 2011 and having major trauma, stroke, AMI, or respiratory failure as final diagnoses. End-points were neurological functioning and survival to 30 days, 90 days, 1 year and 2 years. The information was distilled from Medicare claims by linking ambulance rides to in- patient and outpatient claims. This study relied upon a battery of statistical tests and their related assumptions.
CMS billing information (ICD codes) was used to define whether a patient was treated by BLS or ALS methods. BLS is billed when the treatment is supportive and interventions are typically limited, such as administering oxygen, splinting, providing oral glucose, or aiding in a patient in taking their own medications. ALS is billed when care is more advanced. CMS is strict regarding the documentation required to support the billing level. When documentation does not support ALS – even if ALS care is provided – the care is more likely billed as BLS. EMS managers continually struggle with getting their crews to adequately document to support higher returns.
Credibility
Sanghavi’s Annals of Internal Medicine study has an impressive n of 396,675. Since it was an observational and not a randomized study, the authors used a propensity score (a statistical matching technique that attempts to estimate the effect of a treatment or other intervention by accounting for the covariates that predict receiving the treatment) and compared patients only within each county, for non-rural counties. As an observational study, it allows for correlations to be made with the data, but it cannot lead to conclusions of causality.
The authors carefully detailed potential confounding variables such as hospital care and differences between EMS systems (distance to hospitals, differences in protocols, etc) and tried to address those variables with some guidance from EMS directors from around the country. While the quality of controlling for these variables is debatable, the authors clearly thought many of them out and attempted to address most of them.
Shortcomings
On-scene time
The authors also mention on-scene time, stating that ALS crews spend more time on scene with a “stay and play” model instead of the “scoop and go” model of BLS. Even though this is a potentially major confounding variable, they don’t take the time to delve further into the claim, analyze on-scene times, or suggest it as further study. More time on scene means more time away from definitive care, which can easily account for the increased morbidity and mortality the study found associated with ALS. The authors’ implication is that the scene time itself may be a problem, but they do not consider the possibility that scene stabilization for sicker patients takes more time.
Inability to distinguish between pre-hospital care and in-hospital care
The study also compares ICD codes in an effort to com- pare levels of care to reach its conclusions. However ICD codes don’t differentiate between what that level of care entails, what treatments were given or not given, and whether or not those providers were competent. The outcomes in this study were also based upon both pre-hospital and hospital care together, as opposed to just pre-hospital care. Although the authors attempted to control for this con- founding variable by comparing outcomes from patients receiving EMS care to outcomes from directly-admitted patients, there was no way to determine whether a patient’s outcomes were due to pre-hospital or due to inpatient hospital care. Therefore, the effects of hospital care inevitably would bias the conclusions.
Selection bias
The study looked at only Medicare beneficiaries. Part of the difficulty of any study is finding a representative population to study. The authors had a large sample, but they were all Medicare beneficiaries, which of course do not represent the general US population. The applicability of the results is therefore limited to patients who are in the Medicare system and are therefore generally 65 or older.
ALS care involves sicker patients
Our EMS systems are set up to specifically transport patients who are less ill or injured by BLS. The more severe cases are transported by ALS even if the ICD coded diagnosis ends up the same. The “obvious” stroke is transported by ALS whereas the subtle stroke is transported BLS even though both end up with an ICD diagnosis code of “stroke.” In both of these examples, the same provider could have easily provided BLS care to one patient and ALS care to another, both of whom eventually code to the same diagnosis. Propensity scoring cannot negate the inherent selection bias based upon the fact the patient got ALS or BLS. The assertion that ALS care may be more dangerous than BLS care rings as hollow as saying more people who go the ICU die than those who go to the floor. Sicker patients get ALS care (or ICU care), so having “worse outcomes” makes sense.
Author and Reviewer Inexperience
One criticism of the study is the authors themselves. It’s difficult to know how to control for variables, parse apart ICD codes, and understand how the whole billing process works even for experienced EMS providers. The researchers, however, are non-clinical, PhD scientists without any disclosed EMS experience, making it even more likely they may have overlooked some clinically relevant details. Since this article was published in the Annals of Internal Medicine, it is likely that the paper’s peer reviewers also lack EMS experience. While clinical experience in the field targeted for statistical analysis is not absolutely required, informed conclusions from a complex data analysis are more likely to result from informed design done by those with experience in the field.
Interventions may inappropriately define level of care
Another source of bias that cannot be revealed by culling data sets involves coding according to interventions. CMS defines ALS care based upon interventions, most of which are truly not proven to be of benefit – for example, placing an IV. The IV may be placed because the patient is identified as potentially being sicker as a precaution and there- fore that patient moves from BLS care to ALS care adding to the “sick burden” in that cohort but is not accompanied by any additional therapeutic opportunities for the provider. If the same crew had not put in an IV as a precaution, the trip would have been coded and analyzed as a BLS trip by Sanghavi’s study.
The Bottom Line
Yet another study challenges us to question the efficacy ALS care. ALS and BLS care are different in more than just capabilities, however. As the authors mention, time on scene (time away from definitive care) is an important difference between ALS and BLS, one that undoubtedly affected the study’s results. BLS care is also usually simplistic enough that treatments provided can be inferred if they bill at a BLS level. However, a transport billed at an ALS level introduces many more variables not represented in a simple ICD code. An ICD code won’t tell us whether someone in respiratory distress was treated with a non-rebreather mask, CPAP, or one of many advanced airways, and the study therefore cannot compare ALS treatments and their efficacy, and therefore comparing ICD codes to determine efficacy is not reliable.
Sanghavi’s study has too many biases to accurately com- pare outcomes between ALS and BLS care. The study certainly raises good questions, but it seems a retrospective claims review is just not the way to answer them. Because it is nearly impossible to create a randomized controlled trial in emergency care, we may never have a reliable answer to the debate about outcomes between ALS and BLS care. Nevertheless, we should continue to question and improve the efficacy of our prehospital interventions. The EMS Compass Project, an initiative for designing and testing EMS performance measures and the Evidence-Based EMS Guidelines Project, supported by a NHTSA grant and with broad engagement by stakeholders may help provide practical answers as to what does and does not work in EMS.
REFERENCES
- Sanghavi, Prachi et al. “Outcomes Of Basic Versus Advanced Life Support For Out-Of-Hospital Medical Emergencies”.Annals of Internal Medicine 163.9 (2015): 681. Web. 16 Dec. 2015.
- Stiell,IG et al “Advanced Life Support for Out of Hospital Respiratory Distress”. NEJM 356(21) 2156-64
2 Comments
Another volley in the great ALS vs. BLS debate…this study, much like its predecessors, is fraught with confounding variables and relies heavily on advanced statistical calculations that muddle the reader’s ability to garner a clinical relevance from the conclusion. As a proponent of physician based “retrieval medicine” I believe there is still a big role for ALS and beyond in the prehospital setting. I would bet that rural studies would show a much greater benefit for ALS providers. Furthermore, the late John Hinds, always talked about the concept of “meaningful interventions”. In his patient population of motorcycle trauma this included the automatic placement of bilateral chest tubes and his save rate was exceptional. We need a way of analyzing interventions for specific patient populations to see which field interventions provide the best “ROI”. I do believe that the EMS Compass initiative is a great foundation for this. On a final point, I really wonder how much of a role the lack of EMS research and evidence based guidance (in comparison to hospital based EBM) has played in the proposed lack of field ALS efficacy on patient M&M? Cheers!
Too heterogeneous study to make solid conclusions. Each case is different and interventions are needed at different times. I don’t see how a study can make conclusions of the usefulness (or lack of) with out recruiting very many thousands of patients from multiple settings.