TY - JOUR

T1 - Effect of Machine Learning on Dispatcher Recognition of Out-of-Hospital Cardiac Arrest During Calls to Emergency Medical Services

T2 - A Randomized Clinical Trial

AU - Blomberg, Stig Nikolaj

AU - Christensen, Helle Collatz

AU - Lippert, Freddy

AU - Ersbøll, Annette Kjær

AU - Torp-Petersen, Christian

AU - Sayre, Michael R.

AU - Kudenchuk, Peter J.

AU - Folke, Fredrik

PY - 2021

Y1 - 2021

N2 - Importance: Emergency medical dispatchers fail to identify approximately 25% of cases of out-of-hospital cardiac arrest (OHCA), resulting in lost opportunities to save lives by initiating cardiopulmonary resuscitation. Objective: To examine how a machine learning model trained to identify OHCA and alert dispatchers during emergency calls affected OHCA recognition and response. Design, Setting, and Participants: This double-masked, 2-group, randomized clinical trial analyzed all calls to emergency number 112 (equivalent to 911) in Denmark. Calls were processed by a machine learning model using speech recognition software. The machine learning model assessed ongoing calls, and calls in which the model identified OHCA were randomized. The trial was performed at Copenhagen Emergency Medical Services, Denmark, between September 1, 2018, and December 31, 2019. Intervention: Dispatchers in the intervention group were alerted when the machine learning model identified out-of-hospital cardiac arrest, and those in the control group followed normal protocols without alert. Main Outcomes and Measures: The primary end point was the rate of dispatcher recognition of subsequently confirmed OHCA. Results: A total of 169 049 emergency calls were examined, of which the machine learning model identified 5242 as suspected OHCA. Calls were randomized to control (2661 [50.8%]) or intervention (2581 [49.2%]) groups. Of these, 336 (12.6%) and 318 (12.3%), respectively, had confirmed OHCA. The mean (SD) age among of these 654 patients was 70 (16.1) years, and 419 of 627 patients (67.8%) with known gender were men. Dispatchers in the intervention group recognized 296 confirmed OHCA cases (93.1%) with machine learning assistance compared with 304 confirmed OHCA cases (90.5%) using standard protocols without machine learning assistance (P = .15). Machine learning alerts alone had a significantly higher sensitivity than dispatchers without alerts for confirmed OHCA (85.0% vs 77.5%; P < .001) but lower specificity (97.4% vs 99.6%; P < .001) and positive predictive value (17.8% vs 55.8%; P < .001). Conclusions and Relevance: This randomized clinical trial did not find any significant improvement in dispatchers' ability to recognize cardiac arrest when supported by machine learning even though artificial intelligence did surpass human recognition. Trial Registration: ClinicalTrials.gov Identifier: NCT04219306.

AB - Importance: Emergency medical dispatchers fail to identify approximately 25% of cases of out-of-hospital cardiac arrest (OHCA), resulting in lost opportunities to save lives by initiating cardiopulmonary resuscitation. Objective: To examine how a machine learning model trained to identify OHCA and alert dispatchers during emergency calls affected OHCA recognition and response. Design, Setting, and Participants: This double-masked, 2-group, randomized clinical trial analyzed all calls to emergency number 112 (equivalent to 911) in Denmark. Calls were processed by a machine learning model using speech recognition software. The machine learning model assessed ongoing calls, and calls in which the model identified OHCA were randomized. The trial was performed at Copenhagen Emergency Medical Services, Denmark, between September 1, 2018, and December 31, 2019. Intervention: Dispatchers in the intervention group were alerted when the machine learning model identified out-of-hospital cardiac arrest, and those in the control group followed normal protocols without alert. Main Outcomes and Measures: The primary end point was the rate of dispatcher recognition of subsequently confirmed OHCA. Results: A total of 169 049 emergency calls were examined, of which the machine learning model identified 5242 as suspected OHCA. Calls were randomized to control (2661 [50.8%]) or intervention (2581 [49.2%]) groups. Of these, 336 (12.6%) and 318 (12.3%), respectively, had confirmed OHCA. The mean (SD) age among of these 654 patients was 70 (16.1) years, and 419 of 627 patients (67.8%) with known gender were men. Dispatchers in the intervention group recognized 296 confirmed OHCA cases (93.1%) with machine learning assistance compared with 304 confirmed OHCA cases (90.5%) using standard protocols without machine learning assistance (P = .15). Machine learning alerts alone had a significantly higher sensitivity than dispatchers without alerts for confirmed OHCA (85.0% vs 77.5%; P < .001) but lower specificity (97.4% vs 99.6%; P < .001) and positive predictive value (17.8% vs 55.8%; P < .001). Conclusions and Relevance: This randomized clinical trial did not find any significant improvement in dispatchers' ability to recognize cardiac arrest when supported by machine learning even though artificial intelligence did surpass human recognition. Trial Registration: ClinicalTrials.gov Identifier: NCT04219306.

U2 - 10.1001/jamanetworkopen.2020.32320

DO - 10.1001/jamanetworkopen.2020.32320

M3 - Journal article

C2 - 33404620

AN - SCOPUS:85100280242

VL - 4

SP - e2032320

JO - JAMA Network Open

JF - JAMA Network Open

SN - 2574-3805

IS - 1

ER -