To provide a solid basis for more general spine image analysis problems.
We propose a novel deep learning architecture called Transformed Deep Convolution Network (TDCN) used as a method for multi-modal vertebra recognition. This new architecture can fuse image features from different modalities, unsupervised, and automatically correct the pose of the vertebra. The TDCN-based recognition system allows us to simultaneously identify the locations, labels, and poses of vertebra structures in both MR and CT.[1]
The task of the automatic vertebra recognition is to identify the global spine and local vertebra structural information, such as spine shape, vertebra location and pose.
We propose a novel anatomy-inspired Hierarchical Deformable Model (HDM) that implements a comprehensive cross-modality vertebra framework. The framework provides simultaneous identification of local and global spine information in arbitrary image views. The HDM stimulates the local/global structures of the spine to perform deformable matching of spine images.[2]
A deep-learning-based supervised detection approach
Advantages:
(1) cross-modality
(2) feature enhancement
Transformed DCN: a novel invariant deep network
The TDCN automatically extracts the best representative and invariant features for MR/CT. It employs MR–CT feature fusion to enhance the feature discriminativity, and applies alignment transforms for input data to generate invariant representation. This resolves the modality and pose variation problems in vertebra recognition.[1]
A computational anatomy approach: Hierarchical Deformable Model
A three stage recognition approach: landmark detection, global shape registration, and local pose adjustment. The three stage approach is a comprehensive recognition method that provides simultaneous identification of local and global spine structures, with each stage individually implemented by the Hierarchical Deformable Model.
Tri-planar template matching (Step A)
(1) Apply 2D template matching on sagittal, axial, coronal view using deep features
Global shape registration (Step B)
(1) Reduce to point-set registration
(2) Registration adaptively driven by matching
with the tri-planar models
(3) ‘Anchor’ vertebrae (i.e., S1) can prevent
translational mis-alignment of other vertebrae
Local pose alignment (Step C)
(1) Congealing in multi-slices and different image views
(2) Recover 3D poses via back-projection of the aligned 2D planar poses
Validation was performed on cross-modality MR-CT datasets containing a total of 150 volumes with varying pathologies. The SVM was trained for vertebra/non-vertebra classification, and a set of 1150 patches, sampled from a combined total of 10 volumes from MR and CT, were used to train the TDCN system. The ground truth of the testing data and slice selection followed the standard radiology protocol of spine physician, and are in separated manual processes. Testing was conducted using 110 MR and CT sagittal slices from 90 MR-CT volumes, excluding training volumes. Lumbar scans and whole spine MR and CT slices were tested to show generality of our method.[1]
Validation was performed on T1/T2 MR and CT modalities using a combined total of 140 MR and CT samples from three different datasets. Data collected covers from lumbar, thoracic, cervical, and the whole spine. The initial HDM model was constructed from MR+CT image patches collected from different spine sections/views. The deep network of local appearance module was trained using randomly sampled planar patches. HDM planar templates were constructed from lumbar and thoracic patches. The HDM 3D spine model was manually built. Single slice processing was tested. Specific slices were sampled for 3D volume data and multiple-slice data. Ground truth values were used for comparison to evaluate pose accuracy, and the correct labelling rate and rate of vertebra/non-vertebra classification were evaluated.[2]
1. Yunliang Cai, Mark Landis, David T. Laidley, Anat Kornecki, Andrea Lum, and Shuo Li. Multi-Modal Vertebrae Recognition using Transformed Deep Convolution Network. CMIG. 2016.
2. Yunliang Cai, Said Osman, Manas Sharma, Mark Landis, and Shuo Li. Multi-Modality Vertebra Recognition in Arbitrary Views using 3D Deformable Hierarchical Model. IEEE TMI. 2015.