BURExtract-Llama: An LLM for Clinical Concept Extraction in Breast Ultrasound Reports

Authors: Yuxuan Chen, Haoyan Yang, Hengkai Pan, Fardeen Siddiqui, Antonio Verdone, Qingyang Zhang, Sumit Chopra, Chen Zhao, and Yiqiu Shen.

Breast ultrasound reports contain clinically valuable information, but variations in language and formatting make systematic extraction challenging. We present a pipeline that uses GPT-4 to generate training annotations and fine-tunes Llama 3–8B to convert these reports into structured JSON containing 16 lesion attributes.

Evaluated against clinician annotations, BURExtract-Llama achieves an average F1 score of 84.6%, comparable to GPT-4. This approach demonstrates how institutions can develop an in-house clinical information extraction model while reducing reliance on proprietary services and keeping inference within their own infrastructure.