TY - GEN
T1 - GenAI summaries of articles
T2 - 11th International Workshop on Socio-Technical Perspectives in Information Systems, STPIS 2025
AU - Bednar, Peter
N1 - Publisher Copyright:
© 2025 Copyright for this paper by its author(s).
PY - 2025/12/20
Y1 - 2025/12/20
N2 - There are many discussions promoting the use of GenAI tools to create summaries of research papers and texts both for students as well as academics. These are often mainly focusing on potential benefits such as saving time and effort, including the advice and how-to instructions. Their expected value however is dependent on in how factually reliable and trustworthy generated summaries are, which is a topic that tends to be undermined by vague explorations of limitations and pitfalls. This paper looks into snapshot examples of generated summaries and compares their factual propositions with that of the original sources. Summaries were based on content from 7 articles and 5 books. The systematic evaluation of content is looking at 39 examples of factual errors in propositions including some examples of citation errors. The purpose of the research presented in this paper is to glean both evidence as well as some understanding of the factual reliability and trustworthiness of generated summaries. The paper concludes with a more balanced discussion regarding the use of AI generated summaries.
AB - There are many discussions promoting the use of GenAI tools to create summaries of research papers and texts both for students as well as academics. These are often mainly focusing on potential benefits such as saving time and effort, including the advice and how-to instructions. Their expected value however is dependent on in how factually reliable and trustworthy generated summaries are, which is a topic that tends to be undermined by vague explorations of limitations and pitfalls. This paper looks into snapshot examples of generated summaries and compares their factual propositions with that of the original sources. Summaries were based on content from 7 articles and 5 books. The systematic evaluation of content is looking at 39 examples of factual errors in propositions including some examples of citation errors. The purpose of the research presented in this paper is to glean both evidence as well as some understanding of the factual reliability and trustworthiness of generated summaries. The paper concludes with a more balanced discussion regarding the use of AI generated summaries.
KW - AI generated summaries
KW - Sociotechnical Perspectives on GPT use
UR - https://www.scopus.com/pages/publications/105039173826
M3 - Conference contribution
AN - SCOPUS:105039173826
T3 - CEUR Workshop Proceedings
SP - 1
EP - 19
BT - STPIS 2025 Socio-Technical Perspectives in Information Systems 2025
A2 - Topuzovska Latkovikj, Marija
A2 - Bednar, Peter
A2 - Rajanen, Mikko
A2 - Kävrestad, Jaokim
A2 - Vallo Hult, Helena
A2 - Elbanna, Amany
PB - CEUR Workshop Proceedings
Y2 - 17 September 2025 through 18 September 2025
ER -