Machine-Learning May Assist in Classifying Knee MRIs

Natural language processing (NLP) may assist physicians to classify free-text knee MRI reports, according to a study in the April issue of the American Journal of Roentgenology. Researchers from Stanford University School of Medicine in California and Duke University Medical Center in Durham, N.C., sought to evaluate the performance of a NLP system in classifying a database of free-text knee MRI reports at two separate academic radiology practices. The researchers used the NLP trained and tested on expert-classified knee MRI reports from two major health care organizations. Radiology reports were modeled in the training set as vectors, and a support vector machine framework was used to train the classifier. A separate test set from each organization was used to evaluate the performance of the system.  Measures of efficacy of the classification system included standard evaluation metrics, such as accuracy, precision, recall, and F1 score (i.e., the weighted average of the precision and recall), and their respective 95 percent CIs were used to measure the efficacy of the classification system. The results showed accuracy for radiology reports that belonged to the model's clinically significant concept classes after training data from the same institution was good. The F1 score was greater than 90 percent. Performance of the classifier on cross-institutional application without institution-specific training data yielded F1 scores of 77.6 percent and 90.2 percent at the two organizations studied. “The study’s results show excellent accuracy by the NLP machine learning classifier in classifying free-text knee MRI reports, supporting the institution-independent reproducibility of knee MRI report classification,” the researchers concluded.  They also noted the machine learning classifier performed well on free-text knee MRI reports from another institution, supporting the feasibility of multi-institutional classification of radiologic imaging text reports with a single machine learning classifier without requiring institution-specific training data.