Last week, I spent a fascinating hour and a half with two members of Physio-Control’s technical team going over the 12-Leads I discussed in a recent post. I was extremely impressed with their knowledge, their desire to improve the computer’s algorithms, and their commitment to patient care.
I went back over each of the charts to determine the discharge diagnoses of each case and any other relevant data, and was also able at their suggestion to obtain the 12-lead’s PCO file, which is available through CODE-Stat software, that provides a far more detailed look at what the computer is actually seeing when it makes its call. Unfortunately, I only had access to the PCO files to the most recent third of the transmissions.
From the original 58 transmissions, I excluded 9 -12-leads due to the following reasons, 3-patients went to other hospitals and had unconfirmable diagnoses, 3-12-leads from 1 patient who was a patient 4 times (all 4 times, his 12-lead triggered a false Acute MI reading), 1 patient who was a DNR and may not have been considered for the cath lab, 1 -12 lead from a patient who had a second 12-lead recorded several minutes later, but who had been listed as two separate patients, and one due to clear arm lead reversal that was later corrected in untransmitted 12-leads. I also recategorized one of the missed STEMIs as a confirmed STEMI when the PCO file revealed, all arm motion in the transmitted ECG and all three subsequent, but nontransmitted ECGs correctly called the ECG as a STEMI. And I recategorized another inappropriately labeled STEMI as a correct STEMI when I discovered a coding area in the chart.
So here then is the revised bottom line:
49 Transmitted ECGs (49 Patients that were either machine called STEMIs or clinical STEMIs not called by the machine. Patients who had nondiagnostic 12-leads were excluded).
Of the 42 called STEMI by the machine, 21 went to cath lab (50%), 19 were classified STEMI (45%), two had clean arteries.
23 ECGS incorrectly called STEMI (55%), 21 that did not go to cath lab (50%).
Of the 26 confirmed clinical STEMIs in the batch, 7 were missed by the machine interpretation (27%). The machine correctly captured captured 19 (73%).
If this small back of the envelope sample holds true, then you could say the computer will identify 73% of STEMIs, but miss 27%.
Likewise, if it were to hold true, if the machine does call a 12-lead a STEMI, there is a 50% chance they will go to the cath lab, and a 45% chance that it is actually a STEMI.
These are better figures than what I first reported, but still concerning if the computer interpretation’s limitations are not properly understood and accounted for in patient care algorithms.
Had we been able to view the PCO files from all the 12-leads, other small adjustments may have been made.
Keep in mind that this was not a rigorous scientific study. Such a study could and should be done, preferably at an institution with a high volume of transmitted ECGs.
That aside, here then are the lessons we should take from this.
1. Systems that require the computer to interpret the 12-Lead as a STEMI for activation put patients at risk. If the computer interpretation alone buys you a trip to the cath lab, some people will be cathed unneccessarily.
Also, if you can only preactivate the cath lab if the machine calls it a STEMI, many people having obvious STEMIs will have their care delayed.
2. The machine is only as good as the quality of the data. Failure to put the electrodes in the right place, to see that they are well affixed and that patient is not moving can lead to a false reading.
The fact that many paramedics I have talked to, have recognized this, and then redone a 12-lead suggests an even higher computer miscall rate, but it also suggests that vigilance to this, and perhaps requiring a 2nd 12-lead of high data quality, might lower the miscall rate substantially.
Paramedics should be very careful when acquiring 12-leads and if they distrust the result, should consider getting a 2nd 12-lead of higher quality after trouble-shooting any movement issues. Systems likewise should consider the 12-lead they are receiving may not reflect pristine positioning and lack of movement and may be treated with some skepticism.
3. There are two well defined examples that appear to mislead the machine – aflutter (which has been recognized by Tom Bouthillet at EMS 12-Lead and hyperactute T waves in the anterior leads. Of the 7 missed STEMIs in this small survey 5 had hyperactute T waves in the anterior leads. Of the 23 inaccurately called STEMIs, 4 had atrial flutter as an underlying rhythm.
Here are some examples of some of the hyperacute T wave misses.
4. Physio-control is committed to providing the best product possible. They will be sending many of the PCO files I shared with them to Glascow, which produces the algorithm, for analysis.
5. Ultimately, no computer will be perfect. The software should always just be used as an adjunct, a valuable adjunct, but not a requirement replacing human consideration.
On a final note, I think EMS has an obligation to do more than complain about the computer error. We should
a. Insisit on proper ECG placement and data quality
b. Identify patterns of errors and share this information with the developers of the algorithms so they can improve on them for our mutual goal of improved patient care and outcomes.