iSpine

Challenges

Complex appearance inhomogeneity
- Inter-modality intensity difference
- Intra-modality appearance variation
Great boundary variations
- Diverse boundary shape variation
- Local weak/no boundary^[1]
Inter-class overlapping when classifying neural foramina as a result of extreme diverse neural foramina images.^[2]
Multiple targets
- There are at least 17 target organs in the lumbar spine.^[3]
Multiple tasks
- Simultaneous localization and diagnosis of all lumbar organs.^[3]

Research

Boundary Regression Model

We propose a novel boundary regression segmentation framework for fully automated, multi-modal segmentation and area estimation of neural foramina.

Combines multiple output support vector regression and multiple kernel learning.^[1]

Synchronized Superpixels Representation (SSR)

synchronized superpixel representation model

Fig. 1. SSR model, implemented by integrating class label (0:normal,1:stenosed) into manifold alignment, provides a discriminative feature space (called SSR space) for reliable classification. (a) the class overlapping problem in original image space; (b) stenosed SSR; (c) normal SSR; (d) SSR space.^[2]

Deep Multiscale Multitask Learning Network (DMML-Net)

We propose a newly designed DMML-Net integrating multi-output learning and multitask regression learning into a fully convolutional network.^[3]

Robustly represents spinal structures and reinforces the salience of target organs due to a feature boosting module merging semantic multi-level features.
Capable of scale-invariance for various organs due to a k-means clustering method finding the rule of organs.
Promotes mutual benefit between inter-task and intra-task due to multitask regression and multitask loss modules.^[3]

Fig. 2. DMML-Net directly localizes and grades all lumbar organs after importing clinical lumbar MRI scans. Multiscale multi-output learning predicts target organs at many convolutional layers of an FCN. Multitask regression learning links multiple tasks with parameter sharing for mutual benefit^[3]

Approach

Boundary Regression Model

Formulates the segmentation task as a boundary regression model to fully leverage the advancement of machine learning in a holistic fashion for seeking the optimal segmentation, which simultaneously preserves accuracy, robustness, and efficiency.

The propose framework contains the following two components:

Training - The multi-kernel multi-output support vector regressor (MKMOSVR) learns the optimal parameters from the training set.
Testing - The output Y for every input X can be computed using the mathematical formulation of MKMOSVR.^[1]

Fig. 3. The overview of our regression segmentation framework.^[1]

Fig. 4. The superiority over conventional model-based methods. (a) Unsatisfied results achieved by conventional model-based methods; (b) accurate results achieved by our regression segmentation.^[1]

Synchronized Superpixels Representation (SSR)

The propose framework contains the following three components:

Spectral Graph - graph structure for pairwise similarities among all pixels within an image.
Spectral Bases - eigenvectors of a spectral matrix.
Superpixels - clusters obtained from grouping image pixels based on spectral bases, which approximate manifold of an image.^[2]

Fig. 2. The overview of our automated diagnosis framework.^[2]

Fig. 3. Accurate diagnosis results in multiple subjects with diverse appearance, size, and shape.^[2]

Deep Multiscale Multitask Learning Network (DMML-Net)

deep multiscale multitask learning network architecture

Fig. 4. The main workflow of DMML-Net. The shadow region covers the multiscale multi-output learning architecture, while the unshaded region covers the multitask regression learning.^[3]

Fig. 5. Feature boosting module merges selected convolutional layers to generate output layers. It enhances the global context information and exchange information between scales, thus it is capable of the scale-invariance for various target organs.^[3]

Fig. 6. The main procedures of the multitask regression learning formulation.^[3]

Fig. 7. Default box proposal module is responsible for hypothesizing locations of organs. a An input MRI scan with part ground truth boxes which are only needed in training phase. b Default box proposal module embeds k-means clustering results into default boxes proposal at each location of the 64 × 64 feature map. More refined locations (loc) of default boxes will be predicted by multitask regression module.^[3]

Fig. 9. The combination of multitask regression module and multitask loss module has combated the huge challenges from the localization and grading of all lumbar organs. These stable precision-recall curves also demonstrate the strengths of DMML-Net in the pathogenesisbased diagnosis of LNFS since both normal and abnormal neural foramina (NF) have high recall and precision.^[3]

Validation

Boundary regression model was validated on a dataset including 912 MR and 306 CT images collected from 152 subjects. Two types of metrics were used for evaluation: metric in segmentation, the dice similarity coefficiet, and a metric in area estimation, boundary distance.^[1]

SSR validation was conducted on a dataset including 110 mid-sagittal MR spine images collected from 110 subjects. Training images were manually cropped and labelled by physicians. Classification accuracy, specificity, and sensitivity were reported.^[2]

DMML-Net validation was conducted on a dataset including 1200 neural formina (518 normal, 682 abnormal), 1200 discs (627 normal, 573 abnormal), and 1000 lumbar vertebrae (690 normal, 310 abnormal) from 200 patients (average 60 years). Standard five-fold cross-validation was employed for performancec evaluation and comparison. Data augmentation, such as random distortion, random flip, random brightness, and contrast adjustment was also employed to enhance generalization. To demonstrate the DMML-Net's sensitivity and interpretability a standard precision-recall curve and mean average precision were reported.^[3]