Artificial intelligence assisted ultrasound report generation

Table 3 Challenges and proposed solutions in visual language model -based ultrasound report generation

Challenge	Proposed solution
Poor accuracy in text generation related to measurement results	Extract numerical values from ultrasound images using tools like TrOCR[15] and insert them into the report
Suboptimal handling of correspondence between text and images	Annotate the correspondence between text and images and design mechanisms to learn these relationships
Ineffective utilization of report templates	Use report templates as input, treat template prediction as an intermediate task, or have the model learn to modify templates
Issues with training data volume	Split existing reports into text-image pairs and reassemble them to create pseudo-cases for training
Ineffective utilization of historical reports	Use historical reports along with current ultrasound images as input
Neglect of image selection task	Explicitly model the image selection process to choose representative images for the report
Lack of utilization of ultrasound-related expertise	Fine-tune LLM models to learn this prior knowledge
Lack of exploration of predictive tasks	Conduct in-depth research on ultrasound examination scenarios to define effective predictive tasks

LLM: Large language model.