Frontier AI models are trained mostly on public internet text and images, but clinical brain scans such as MRI and CT rarely appear in those datasets because they contain identifiable facial features. As a result, general-purpose vision models perform poorly when asked to interpret neuroimaging data, limiting their usefulness in radiology workflows and research.
A team from the University of Michigan introduced NeuroVFM, a visual foundation model built specifically for medical imaging. It was trained on 5.24 million MRI and CT volumes drawn from the UM‑NeuroImages dataset, which represents over two decades of routine care at Michigan Medicine. The approach, called health‑system learning, leverages the uncurated images generated during normal clinical operations, eliminating the need for paired radiology reports or disease‑specific labeling.
At the core of NeuroVFM is Vol‑JEPA, a self‑supervised vision‑only algorithm that extends the I‑JEPA and V‑JEPA frameworks to three‑dimensional medical scans. Instead of reconstructing raw pixels, Vol‑JEPA predicts latent representations of masked patches. The process tokenizes each volume into non‑overlapping 4×16×16‑voxel patches, splits them into a small visible context and a larger masked target, encodes the context with a student network, predicts the target latents with a predictor, and compares those predictions to ground‑truth latents produced by an exponential moving‑average teacher. A smooth L1 loss drives learning, while gradients are stopped through the teacher to keep the target stable.
Foreground‑focused masking, using precomputed head masks, encourages the model to learn shared neuroanatomy rather than background shortcuts. Context ratios are set to 25 % for MRI and 20 % for CT, with 20 % patch dropout.
Because NeuroVFM learns directly from everyday clinical scans without manual annotation, it provides a ready‑to‑use generalist backbone that can be fine‑tuned for specific brain‑imaging tasks such as tumor detection, lesion segmentation, or disease classification. Hospitals and research labs can adopt this model to improve AI assistance in radiology, reduce reliance on labeled datasets, and accelerate the deployment of robust neuroimaging tools.
#AI #MachineLearning #HealthAI #Neuroimaging #DeepLearning #MedTech