Catalyst and TReNDS have been working together on applications of deep learning for neuroimaging and brain dynamics. A recent product of this collaboration is Catalyst.Neuro, which reimplements the brain segmentation pipeline in End-to-end learning of brain tissue segmentation from imperfect labeling and An (almost) instant brain atlas segmentation for large-scale studies. This post describes the fundamental concepts implemented in Catalyst.Neuro and introduces different deep learning models comparison on brain segmentation task.
Note. Catalyst is a PyTorch framework for Deep Learning research and development. You get a training loop with metrics, model checkpointing, advanced logging and distributed training support without the boilerplate. Intro to Catalyst can be found here.
Structural magnetic resonance imaging (sMRI) is a non-invasive technique for examining the anatomy and the pathology of the brain. Segmenting a structural MRI into tissue types or functional regions is an important processing step that enables subsequent inferences about tissue changes in development, aging, and disease. Schizophrenia, multiple sclerosis, and dementia are just a few of the psychiatric conditions associated with abnormal degeneration of brain regions that reflect in their volume changes relative to healthy control subjects of similar age and gender.
Brain segmentation has previously been accomplished with a pipeline of iterative statistical methods including Markov Random Fields (FreeSurfer). However, it takes hours to process a single brain and can output different results based on different initializations. With recent advances in deep learning, medical imaging has seen a large number of applications for classification (benign vs. malignant tumors), regression (biological age estimation), and segmentation (nuclei segmentation). The current standard medical segmentation model (U-Net) is a convolutional neural network architecture consisting of an encoder and decoder.
Catalyst.Neuro implements a brain segmentation pipeline using the Mindboggle dataset to compare U-Net with the MeshNet (Dilated 3D CNN) architecture. With minimal preprocessing, MeshNet performs inference up to 9x faster and is >300x smaller while maintaining a competitive DICE score vs. a U-Net baseline. Interactive tutorials are available via Google Colab for training and inference and are located here.
Expert manual labeling is the gold standard for labeling brain segments for MRI's. Complete labeling of a single MRI scan can take up to 2-3 days for an expert. Manual labeling is also prone to inconsistency resulting in intra and inter-observer variability.
Manual segmentations of multiple observers of a colorectal liver metastasis on an axial slice of a CT scan.
We use Mindboggle to demonstrate our MRI segmentation pipeline. Mindboggle is the largest and most complete set of free, publicly accessible, manually labeled human brain images.
You can download Mindboggle MRI files and labels from the Open Science Framework using osfclient below
osf -p 9ahyp clone Mindboggle_data
cp -r Mindboggle_data/osfstorage/Mindboggle101_volumes/ data/Mindboggle_data
Mindboggle developers also have some impressive visualization projects including http://roygbiv.mindboggle.info/roygbiv/web/index.html where you can visualize a scan in 3d and get associated volumetric statistics.
