Collective Intelligence Increases Diagnostic Accuracy in a General Practice Setting - Blanchard, M. D.; Herzog, S. M.; Kammer, J. E.; Zoller, N.; Kostopoulou, O.; Kurvers, Rhjm (Med Decis Making, 2024)
BACKGROUND: General practitioners (GPs) work in an ill-defined environment where diagnostic errors are prevalent. Previous research indicates that aggregating independent diagnoses can improve diagnostic accuracy in a range of settings. We examined whether aggregating independent diagnoses can also improve diagnostic accuracy for GP decision making. In addition, we investigated the potential benefit of such an approach in combination with a decision support system (DSS). METHODS: We simulated virtual groups using data sets from 2 previously published studies. In study 1, 260 GPs independently diagnosed 9 patient cases in a vignette-based study. In study 2, 30 GPs independently diagnosed 12 patient actors in a patient-facing study. In both data sets, GPs provided diagnoses in a control condition and/or DSS condition(s). Each GP's diagnosis, confidence rating, and years of experience were entered into a computer simulation. Virtual groups of varying sizes (range: 3-9) were created, and different collective intelligence rules (plurality, confidence, and seniority) were applied to determine each group's final diagnosis. Diagnostic accuracy was used as the performance measure. RESULTS: Aggregating independent diagnoses by weighing them equally (i.e., the plurality rule) substantially outperformed average individual accuracy, and this effect increased with increasing group size. Selecting diagnoses based on confidence only led to marginal improvements, while selecting based on seniority reduced accuracy. Combining the plurality rule with a DSS further boosted performance. DISCUSSION: Combining independent diagnoses may substantially improve a GP's diagnostic accuracy and subsequent patient outcomes. This approach did, however, not improve accuracy in all patient cases. Therefore, future work should focus on uncovering the conditions under which collective intelligence is most beneficial in general practice. HIGHLIGHTS: We examined whether aggregating independent diagnoses of GPs can improve diagnostic accuracy.Using data sets of 2 previously published studies, we composed virtual groups of GPs and combined their independent diagnoses using 3 collective intelligence rules (plurality, confidence, and seniority).Aggregating independent diagnoses by weighing them equally substantially outperformed average individual GP accuracy, and this effect increased with increasing group size.Combining independent diagnoses may substantially improve GP's diagnostic accuracy and subsequent patient outcomes.
Keywords: journalArticle; *General Practice/methods; Artificial Collective Intelligence in Open-Ended Decision Making").; Clinical Decision-Making/methods; Computer Simulation; Decision Support Systems, Clinical; Diagnostic Errors/statistics & numerical data; European Commission (Horizon Europe grant 101070588 "HACID: Hybrid Human; Female; General Practitioners; Humans; Male; collective intelligence; decision support systems; diagnostic accuracy; general practice; medical diagnostics; publication of this article: RHJMK and SMH acknowledge financial support from the; receipt of the following financial support for the research, authorship, and/or; research, authorship, and/or publication of this article. The authors disclosed; wisdom of crowds