Presenting artificial intelligence, deep learning, and machine learning studies to clinicians and healthcare stakeholders: an introductory reference with a guideline and a Clinical AI Research (CAIR) checklist proposal

Authors

  • Jakub Olczak Institute of Clinical Sciences, Danderyd University Hospital, Karolinska Institute, Sweden
  • John Pavlopoulos Department of Computer and System Sciences, Stockholm University, Sweden
  • Jasper Prijs Flinders University, Adelaide, Australia; Department of Trauma Surgery, University Medical Center Groningen, University of Groningen, Groningen, The Netherlands
  • Frank F A Ijpma Department of Trauma Surgery, University Medical Center Groningen, University of Groningen, Groningen, The Netherlands; The Machine Learning Consortium
  • Job N Doornberg Flinders University, Adelaide, Australia; Department of Trauma Surgery, University Medical Center Groningen, University of Groningen, Groningen, The Netherlands; The Machine Learning Consortium
  • Claes Lundström Center for Medical Image Science and Visualization, Linköping University, Sweden
  • Joel Hedlund Center for Medical Image Science and Visualization, Linköping University, Sweden
  • Max Gordon Institute of Clinical Sciences, Danderyd University Hospital, Karolinska Institute, Sweden

DOI:

https://doi.org/10.1080/17453674.2021.1918389

Abstract

Background and purpose — Artificial intelligence (AI), deep learning (DL), and machine learning (ML) have become common research fields in orthopedics and medicine in gen- eral. Engineers perform much of the work. While they gear the results towards healthcare professionals, the difference in competencies and goals creates challenges for collabora- tion and knowledge exchange. We aim to provide clinicians with a context and understanding of AI research by facilitat- ing communication between creators, researchers, clinicians, and readers of medical AI and ML research.

Methods and results — We present the common tasks, considerations, and pitfalls (both methodological and ethi- cal) that clinicians will encounter in AI research. We discuss the following topics: labeling, missing data, training, testing, and overfitting. Common performance and outcome mea- sures for various AI and ML tasks are presented, including accuracy, precision, recall, F1 score, Dice score, the area under the curve, and ROC curves. We also discuss ethi- cal considerations in terms of privacy, fairness, autonomy, safety, responsibility, and liability regarding data collecting or sharing.

Interpretation — We have developed guidelines for reporting medical AI research to clinicians in the run-up to a broader consensus process. The proposed guidelines consist of a Clinical Artificial Intelligence Research (CAIR) check- list and specific performance metrics guidelines to present and evaluate research using AI components. Researchers, engineers, clinicians, and other stakeholders can use these proposal guidelines and the CAIR checklist to read, present, and evaluate AI research geared towards a healthcare setting.

Downloads

Download data is not yet available.

Downloads

Additional Files

Published

2021-05-14

How to Cite

Olczak, J. ., Pavlopoulos, J., Prijs, J., Ijpma, F. F. A. ., Doornberg, J. N., Lundström, C., … Gordon, M. (2021). Presenting artificial intelligence, deep learning, and machine learning studies to clinicians and healthcare stakeholders: an introductory reference with a guideline and a Clinical AI Research (CAIR) checklist proposal. Acta Orthopaedica, 92(5), 513–525 . https://doi.org/10.1080/17453674.2021.1918389