We are very happy to share that the registration for the MIDL Virtual Event Workshop on Medical Data Annotation Practices is now open. This workshop is organized by Maria Galanty (member of the qurAI group) and Dieuwertje Luitse (Faculty of Humanities, UvA) and will be online on May 14, 2025, at 14:00–17:00 CEST (UTC+2). You can register here.

Medical data annotation is a crucial yet challenging aspect of developing reliable deep learning models for healthcare. Errors, biases, and inconsistencies frequently affect commonly used datasets, particularly in biomedical image analysis, where data is often limited, inter-rater variability is high, and annotation styles differ among experts. The lack of standardized methods for defining “ground truth” and the inherent ambiguity of medical data further complicate the annotation process.

This workshop will bring together researchers to explore the complexities of medical dataset annotation through lectures, case study presentations, and discussions. A key focus will be on the importance of high-quality annotations. We will touch upon issues connected to label noise, confounders, and systematic biases, which can compromise model performance. The workshop will highlight how improving annotation processes can help address these issues. By tackling these challenges, the workshop aims to advance best practices, foster collaboration, and drive improvements in medical AI development.

Speakers

• Dr. Veronika Cheplygina (IT University of Copenhagen) will present Curious Findings about Medical Image Datasets. Her talk will highlight issues such as label noise, shortcuts, and confounders in medical image datasets.
• Tim Rädsch (German Cancer Research Center, DKFZ) will speak on Getting it Right: Unlocking Better Annotation Data through Improved Instructions and QA. He will present insights from large-scale studies examining how better labeling instructions and quality assurance practices can improve annotation performance.
• Dr. Coen de Vente (qurAI, University of Amsterdam, Amsterdam UMC) will discuss Lessons from the AIROGS Challenge: Collecting Reliable Annotations for Glaucoma Screening. His talk will share insights from the AIROGS challenge, which focused on developing robust AI models for glaucoma detection.