TY - GEN
T1 - Improving Medical Image Segmentation Using Gaze Guidance from Non-Professionals
AU - Fang, Yinfeng
AU - Ma, Wenlong
AU - Yu, Xixia
AU - Liu, Qingsong
AU - Wang, Yuxi
AU - Peng, Yong
AU - Zhou, Dalin
AU - Ju, Zhaojie
N1 - Publisher Copyright:
© 2025 IEEE.
PY - 2025/12/22
Y1 - 2025/12/22
N2 - Medical image segmentation using artificial intelligence (AI) has significantly improved diagnosis and treatment, enhancing patient outcomes. However, AI's reliance on large amounts of labeled data and expert annotations limits its scalability. This paper explores the use of eye movement data from non-professionals to improve segmentation accuracy. A novel gaze-enhanced image segmentation model (GISM) is proposed, consisting of two main components: a transformer-based gaze feature extraction block (TGFB) and a multi-head gazeattention block (MHGB). The MHGB fuses gaze and image data by using gaze features as queries within a transformer framework. The colon polyp dataset, is applied to assess the model's performance. The outcomes reveal that 1) gaze data from non-professionals enhances segmentation accuracy, 2) TGFB outperforms other methods in fusing gaze information, and 3) TGFB is highly adaptable to encoder/decoder-based segmentation frameworks.
AB - Medical image segmentation using artificial intelligence (AI) has significantly improved diagnosis and treatment, enhancing patient outcomes. However, AI's reliance on large amounts of labeled data and expert annotations limits its scalability. This paper explores the use of eye movement data from non-professionals to improve segmentation accuracy. A novel gaze-enhanced image segmentation model (GISM) is proposed, consisting of two main components: a transformer-based gaze feature extraction block (TGFB) and a multi-head gazeattention block (MHGB). The MHGB fuses gaze and image data by using gaze features as queries within a transformer framework. The colon polyp dataset, is applied to assess the model's performance. The outcomes reveal that 1) gaze data from non-professionals enhances segmentation accuracy, 2) TGFB outperforms other methods in fusing gaze information, and 3) TGFB is highly adaptable to encoder/decoder-based segmentation frameworks.
UR - https://www.scopus.com/pages/publications/105031768056
U2 - 10.1109/ICARM65671.2025.11293518
DO - 10.1109/ICARM65671.2025.11293518
M3 - Conference contribution
AN - SCOPUS:105031768056
SN - 9798331503086
T3 - IEEE ICARM Proceedings
SP - 100
EP - 105
BT - 2025 10th IEEE International Conference on Advanced Robotics and Mechatronics, ICARM 2025
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 2025 10th IEEE International Conference on Advanced Robotics and Mechatronics, ICARM 2025
Y2 - 1 August 2025 through 3 August 2025
ER -