JMIR Publications

ABSTRACT

Background:

Cerebral hemorrhage is a critical medical condition which necessitates a rapid and precise diagnosis for timely medical intervention including emergency operation. Computed Tomography (CT) is essential for identifying cerebral hemorrhage, while its effectiveness is limited by the availability of experienced radiologists, especially in resource-constrained regions or when shorthanded during holidays or night. Despite advancements in artificial intelligence (AI)-driven diagnostic tools, most of which require technical expertise, posing a challenge for widespread adoption in radiological imaging. The introduction of advanced natural language processing (NLP) models such as GPT-4, which can annotate and analyze images without extensive algorithmic training, offers a potential solution. This study investigates GPT-4's capability to identify and annotate cerebral hemorrhage in cranial CT scans, a novel application of NLP models in radiological imaging.

Objective:

CT scans with six types of cerebral hemorrhage collected at Ren Ji Hospital, Shanghai Jiao Tong University School of Medicine between January and September in 2023.

Methods:

In this retrospective analysis, we collected 208 CT scans with six types of cerebral hemorrhage at Ren Ji Hospital, Shanghai Jiao Tong University School of Medicine between January and September in 2023. All CT images were randomly fed into GPT-4 for the analysis and annotation of cerebral hemorrhage. The outputs were subsequently examined using Photoshop and evaluated by experienced radiologists on a 4-point scale to assess the identification completeness, accuracy and success.

Results:

The overall identification completeness percentage for six types of cerebral hemorrhage was 72.59 ± 18.62%. Specifically, GPT-4 achieved higher identification completeness percentages in epidural and intraparenchymal hemorrhages (89.02 ± 19.01%, 86.86 ± 17.69%, repectively), yet its identification completeness percentage in chronic subdural hemorrhages was very low (37.35 ± 37.50%). The misidentification percentages for complex hemorrhage, epidural hemorrhage and subarachnoid hemorrhage were relatively high (54.00 ± 28.04%, 50.25 ± 22.65%, 50.54 ± 29.18%, respectively), whereas they were relatively low for acute subdural hemorrhage, chronic subdural hemorrhage and intraparenchymal hemorrhage (32.61 ± 26.27%, 40.34 ± 27.19%, 26.24 ± 23.85%, respectively). The identification completeness percentages in both massive and minor bleeding showed no significant difference. However, the misidentification percentage in recognizing massive bleeding was significantly lower than that for minor bleeding. The identification completeness percentage and misidentification percentage for cerebral hemorrhage at different locations showed no significant differences. At last, radiologists showed relative acceptance regarding the identification completeness, accuracy and success (3.60 ± 0.54, 3.30 ± 0.65, 3.38 ± 0.64, respectively).

Conclusions:

GPT-4, a standout among NLP models, exhibits both promising capabilities and certain limitations in the realm of radiological imaging, particularly when it comes to identifying cerebral hemorrhages in CT scans. This opens up new directions and insights for the future development of NLP models in radiology. Clinical Trial: This retrospective study was registered at ClinicalTrials.gov (NCT06230419) and approved by the Ethics Committee of Ren Ji Hospital, Shanghai Jiao Tong University School of Medicine.