Skip to content
Back to outputs

Artificial Intelligence in clinical decision support: challenges for evaluating AI and practical implications

Research output: Contribution to journalArticlepeer-review

Standard

Artificial Intelligence in clinical decision support : challenges for evaluating AI and practical implications. / Magrabi, Farah; Ammenwerth, Elske; McNair, Jytte Brender; De Keizer, Nicolet F; Hyppönen, Hannele; Nykänen, Pirkko; Rigby, Michael; Scott, Philip J; Vehko, Tuulikki; Wong, Zoie Shui-Yee; Georgiou, Andrew.

In: IMIA Yearbook of Medical Informatics, 25.04.2019.

Research output: Contribution to journalArticlepeer-review

Harvard

Magrabi, F, Ammenwerth, E, McNair, JB, De Keizer, NF, Hyppönen, H, Nykänen, P, Rigby, M, Scott, PJ, Vehko, T, Wong, ZS-Y & Georgiou, A 2019, 'Artificial Intelligence in clinical decision support: challenges for evaluating AI and practical implications', IMIA Yearbook of Medical Informatics. https://doi.org/10.1055/s-0039-1677903

APA

Magrabi, F., Ammenwerth, E., McNair, J. B., De Keizer, N. F., Hyppönen, H., Nykänen, P., Rigby, M., Scott, P. J., Vehko, T., Wong, Z. S-Y., & Georgiou, A. (2019). Artificial Intelligence in clinical decision support: challenges for evaluating AI and practical implications. IMIA Yearbook of Medical Informatics. https://doi.org/10.1055/s-0039-1677903

Vancouver

Magrabi F, Ammenwerth E, McNair JB, De Keizer NF, Hyppönen H, Nykänen P et al. Artificial Intelligence in clinical decision support: challenges for evaluating AI and practical implications. IMIA Yearbook of Medical Informatics. 2019 Apr 25. https://doi.org/10.1055/s-0039-1677903

Author

Magrabi, Farah ; Ammenwerth, Elske ; McNair, Jytte Brender ; De Keizer, Nicolet F ; Hyppönen, Hannele ; Nykänen, Pirkko ; Rigby, Michael ; Scott, Philip J ; Vehko, Tuulikki ; Wong, Zoie Shui-Yee ; Georgiou, Andrew. / Artificial Intelligence in clinical decision support : challenges for evaluating AI and practical implications. In: IMIA Yearbook of Medical Informatics. 2019.

Bibtex

@article{80537b5e285943aeb39497cde2450ecb,
title = "Artificial Intelligence in clinical decision support: challenges for evaluating AI and practical implications",
abstract = "Objectives - This paper draws attention to: i) key considerations for evaluating artificial intelligence (AI) enabled clinical decision support; and ii) challenges and practical implications of AI design, development, selection, use, and ongoing surveillance.Method - A narrative review of existing research and evaluation approaches along with expert perspectives drawn from the International Medical Informatics Association (IMIA) Working Group on Technology Assessment and Quality Development in Health Informatics and the European Federation for Medical Informatics (EFMI) Working Group for Assessment of Health Information Systems.Results - There is a rich history and tradition of evaluating AI in healthcare. While evaluators can learn from past efforts, and build on best practice evaluation frameworks and methodologies, questions remain about how to evaluate the safety and effectiveness of AI that dynamically harness vast amounts of genomic, biomarker, phenotype, electronic record, and care delivery data from across health systems. This paper first provides a historical perspective about the evaluation of AI in healthcare. It then examines key challenges of evaluating AI-enabled clinical decision support during design, development, selection, use, and ongoing surveillance. Practical aspects of evaluating AI in healthcare, including approaches to evaluation and indicators to monitor AI are also discussed.Conclusion - Commitment to rigorous initial and ongoing evaluation will be critical to ensuring the safe and effective integration of AI in complex sociotechnical settings. Specific enhancements that are required for the new generation of AI-enabled clinical decision support will emerge through practical application.",
author = "Farah Magrabi and Elske Ammenwerth and McNair, {Jytte Brender} and {De Keizer}, {Nicolet F} and Hannele Hypp{\"o}nen and Pirkko Nyk{\"a}nen and Michael Rigby and Scott, {Philip J} and Tuulikki Vehko and Wong, {Zoie Shui-Yee} and Andrew Georgiou",
note = "IMIA and Georg Thieme Verlag KG.",
year = "2019",
month = apr,
day = "25",
doi = "10.1055/s-0039-1677903",
language = "English",
journal = "IMIA Yearbook of Medical Informatics",
issn = "0943-4747",
publisher = "International Medical Informatics Association",

}

RIS

TY - JOUR

T1 - Artificial Intelligence in clinical decision support

T2 - challenges for evaluating AI and practical implications

AU - Magrabi, Farah

AU - Ammenwerth, Elske

AU - McNair, Jytte Brender

AU - De Keizer, Nicolet F

AU - Hyppönen, Hannele

AU - Nykänen, Pirkko

AU - Rigby, Michael

AU - Scott, Philip J

AU - Vehko, Tuulikki

AU - Wong, Zoie Shui-Yee

AU - Georgiou, Andrew

N1 - IMIA and Georg Thieme Verlag KG.

PY - 2019/4/25

Y1 - 2019/4/25

N2 - Objectives - This paper draws attention to: i) key considerations for evaluating artificial intelligence (AI) enabled clinical decision support; and ii) challenges and practical implications of AI design, development, selection, use, and ongoing surveillance.Method - A narrative review of existing research and evaluation approaches along with expert perspectives drawn from the International Medical Informatics Association (IMIA) Working Group on Technology Assessment and Quality Development in Health Informatics and the European Federation for Medical Informatics (EFMI) Working Group for Assessment of Health Information Systems.Results - There is a rich history and tradition of evaluating AI in healthcare. While evaluators can learn from past efforts, and build on best practice evaluation frameworks and methodologies, questions remain about how to evaluate the safety and effectiveness of AI that dynamically harness vast amounts of genomic, biomarker, phenotype, electronic record, and care delivery data from across health systems. This paper first provides a historical perspective about the evaluation of AI in healthcare. It then examines key challenges of evaluating AI-enabled clinical decision support during design, development, selection, use, and ongoing surveillance. Practical aspects of evaluating AI in healthcare, including approaches to evaluation and indicators to monitor AI are also discussed.Conclusion - Commitment to rigorous initial and ongoing evaluation will be critical to ensuring the safe and effective integration of AI in complex sociotechnical settings. Specific enhancements that are required for the new generation of AI-enabled clinical decision support will emerge through practical application.

AB - Objectives - This paper draws attention to: i) key considerations for evaluating artificial intelligence (AI) enabled clinical decision support; and ii) challenges and practical implications of AI design, development, selection, use, and ongoing surveillance.Method - A narrative review of existing research and evaluation approaches along with expert perspectives drawn from the International Medical Informatics Association (IMIA) Working Group on Technology Assessment and Quality Development in Health Informatics and the European Federation for Medical Informatics (EFMI) Working Group for Assessment of Health Information Systems.Results - There is a rich history and tradition of evaluating AI in healthcare. While evaluators can learn from past efforts, and build on best practice evaluation frameworks and methodologies, questions remain about how to evaluate the safety and effectiveness of AI that dynamically harness vast amounts of genomic, biomarker, phenotype, electronic record, and care delivery data from across health systems. This paper first provides a historical perspective about the evaluation of AI in healthcare. It then examines key challenges of evaluating AI-enabled clinical decision support during design, development, selection, use, and ongoing surveillance. Practical aspects of evaluating AI in healthcare, including approaches to evaluation and indicators to monitor AI are also discussed.Conclusion - Commitment to rigorous initial and ongoing evaluation will be critical to ensuring the safe and effective integration of AI in complex sociotechnical settings. Specific enhancements that are required for the new generation of AI-enabled clinical decision support will emerge through practical application.

U2 - 10.1055/s-0039-1677903

DO - 10.1055/s-0039-1677903

M3 - Article

C2 - 31022752

JO - IMIA Yearbook of Medical Informatics

JF - IMIA Yearbook of Medical Informatics

SN - 0943-4747

ER -

ID: 13855507