TY  - JOUR
AU  - Herman Bernardim Andrade, Gabriel
AU  - Yada, Shuntaro
AU  - Aramaki, Eiji
PY  - 2024
DA  - 2024/7/2
TI  - Is Boundary Annotation Necessary? Evaluating Boundary-Free Approaches to Improve Clinical Named Entity Annotation Efficiency: Case Study
JO  - JMIR Med Inform
SP  - e59680
VL  - 12
KW  - natural language processing
KW  - named entity recognition
KW  - information extraction
KW  - text annotation
KW  - entity boundaries
KW  - lenient annotation
KW  - case reports
KW  - annotation
KW  - case study
KW  - medical case report
KW  - efficiency
KW  - model
KW  - model performance
KW  - dataset
KW  - Japan
KW  - Japanese
KW  - entity
KW  - clinical domain
KW  - clinical
AB  - Background: Named entity recognition (NER) is a fundamental task in natural language processing. However, it is typically preceded by named entity annotation, which poses several challenges, especially in the clinical domain. For instance, determining entity boundaries is one of the most common sources of disagreements between annotators due to questions such as whether modifiers or peripheral words should be annotated. If unresolved, these can induce inconsistency in the produced corpora, yet, on the other hand, strict guidelines or adjudication sessions can further prolong an already slow and convoluted process. Objective: The aim of this study is to address these challenges by evaluating 2 novel annotation methodologies, lenient span and point annotation, aiming to mitigate the difficulty of precisely determining entity boundaries. Methods: We evaluate their effects through an annotation case study on a Japanese medical case report data set. We compare annotation time, annotator agreement, and the quality of the produced labeling and assess the impact on the performance of an NER system trained on the annotated corpus. Results: We saw significant improvements in the labeling process efficiency, with up to a 25% reduction in overall annotation time and even a 10% improvement in annotator agreement compared to the traditional boundary-strict approach. However, even the best-achieved NER model presented some drop in performance compared to the traditional annotation methodology. Conclusions: Our findings demonstrate a balance between annotation speed and model performance. Although disregarding boundary information affects model performance to some extent, this is counterbalanced by significant reductions in the annotator’s workload and notable improvements in the speed of the annotation process. These benefits may prove valuable in various applications, offering an attractive compromise for developers and researchers. 
SN  - 2291-9694
UR  - https://medinform.jmir.org/2024/1/e59680
UR  - https://doi.org/10.2196/59680
DO  - 10.2196/59680
ID  - info:doi/10.2196/59680
ER  -