Can AI Foundation Models Bolster the Amount of Imaging Data Obtained from Brain MRI?: An Interview with Benjamin Kann, MD
Sharing his perspective as well as insights from recently published research, Benjamin Kann, M.D., discussed the utility of AI foundation models for accessing an array of diagnostic and prognostic data from brain MRI scans.
In a recent interview with Diagnostic Imaging, Benjamin Kann, M.D., an associate professor of radiation oncology at Harvard Medical School, discussed new research looking at the use of the Brain Imaging Adaptive Core (BrainIAC) model for optimizing data from brain magnetic resonance imaging (MRI) scans and the potential of AI foundation models for maximizing imaging data.
Q: So many AI tools are geared toward a specific, more narrow purpose. What prompted you and your team to look at a broader AI foundation model for brain MRI?
Benjamin Kann, M.D.: Yeah, that's absolutely right. So particularly in the brain MRI space, there are many preexisting models that are looking at particular things. Maybe it's stroke prediction or something about a brain tumor. … (However), we're getting millions of scans on these patients every year, and there's a lot of data that's being left on the table. There are a lot of tasks that perhaps individual institutions or research groups … don't have quite enough labeled data to develop a powerful model on one particular task.
We felt that, as we're seeing with large language models rolling out many foundation models, there could really be a role for vision-based foundation models in the major biomedical imaging modalities such as MRI. So you could sort of teach the model to have a baseline foundational knowledge about how a brain MRI structure looks, and then take that knowledge and with a lot fewer cases of labeled data, be able to develop a powerful model that can adapt to many different situations.
We feel (it) lowers the burden of annotated data that's needed for these variety of tasks. It allows other groups, whether they're in oncology, neurology or other (provider) types, that are getting an MRI of the brain to learn something from that image and then adapt it to a different task.
Q: What went into developing that brainAIC model? How long did it take to put that together?
Dr. Kann: It took many months. Despite the fact that we don't need a lot of labeled data to train these models, we still need a lot of unlabeled data that is easier to come by. It's still involved (with) aggregating about 50,000 scans from over 30 datasets that we could kind of harmonize, curate, and then train with what we call a self-supervised learning pipeline to develop that baseline core knowledge. That process of just curating the data, developing the architecture and training that model did take months.
Q: Can you talk a little bit more about the concept of self-supervised learning with this model?
Dr. Kann: Self-supervised learning is what really powers the foundation model, and what it allows us to do is take unlabeled data, just take MRIs and teach the model to learn just about the representation of that MRI without any particular task.
In this case, we used a pipeline called SimCLR, which is a popular self-supervised learning technique that (originated with) different researchers in the tech industry. It uses something called contrastive learning, where we take an image of an MRI and then we manipulate it in some way, and we simply ask the model to tell us: Are we looking at the same scan when we manipulate it, and does that actually represent the same scan of the original or not?
By teaching it to identify which scans match and which scans don't match, we're implicitly teaching that model to learn about various structures within the MRI and different contrast between tissues and orientation of the MRI. We iterate over that thousands and thousands of times, and that's what allows us to have this baseline knowledge that we can then train to predict individual tasks with a lot less data than we needed before.
Q: What kind of results did you see specifically with brain age prediction with this model?
Dr. Kann: With brain age prediction, we found that we were able to accurately estimate the chronological age of a patient within a couple years. And the key there was also, without using this foundational model approach, if we were to just try to train a model from scratch, we found that the performance was really a lot a lot worse, and the predictions were a lot less accurate.
Q: What did the model show in regard to brain cancer mutational subtypes?
Dr. Kann: Yeah, so we looked at a particular mutational subtype called the IDH (isocitrate dehydrogenase) mutation, which is a common mutation in adult gliomas. What we found is that when we use the brainIAC (model) as a baseline and then adapt it to that task with only a few hundred cases, we actually were able to get a pretty good signal as far as area under the curve … to predict that mutation. That gain was about 10 percent higher than if we were to use a model trained from scratch. We also compared it to some of the preexisting models, sort of older approaches of self-supervised learning and things like transfer learning as well.
Q: What results did you see with time to stroke prediction?
Dr. Kann: With time to stroke prediction, we were able to estimate the length of time between the patient's stroke and the image that was taken, and that the error between that estimation was reduced by about 10 to 15 days versus the other models.
Q: What about directions for future research?
Dr. Kann: I think that's where we're really excited about the potential for the brainIAC model to spur research in a lot of different areas. We've provided the model open source to the community and that will enable people to do research at various institutions. They may have an interesting application they want to look at from brain MRIs in their own patient population, and they can take the brainIAC core and then curate maybe a few dozen, maybe a couple hundred cases of labeled data and fine-tune the model to make what we think will be a powerful prediction model for that task.
It also enables us to explore and expand upon foundation models in general in the brain MRI space. I think this is an area where we're just starting to see some traction, and I expect that there's going to be improvements made on the brainIAC model itself, which is what we're working on. But this will also inspire other groups to look at different self-supervised methodologies and different foundation model approaches to develop models that can roll out in various scenarios.
Q: You touched upon this earlier, but what kind of clinical impact do you possibly see with this model moving forward?
Dr. Kann: As far as clinical impact, I think when we talk about things like mutation, prediction of a brain tumor, or the prediction of the development of dementia or stroke, these are really powerful pieces of information that I think we're going to be able to provide patients just from their routine brain MRI, and really uncover a lot of data and a lot of clinical insight from these MRIs that we just could not unlock before. I think that models like BrainIAC and other foundation models are just going to lower the barrier to do that in a variety of scenarios.
Q: Is there anything else you wanted to add about the study?
Dr. Kann: I think we're just excited for the community to investigate technologies like BrainIAC and test it out for themselves. (We) hope that it also just really pushes the field forward in terms of our ability to extract data from MRI at the point of care and be able to provide that information to the clinicians and patients.
(Editor’s note: This interview is also available as a video at:
Related Content
