Research and Publications

Medical Image Computing (MIC)

The major research goal of MIC is to extract clinically relevant information or knowledge from medical images. We have expertise and strong interests in applying advanced machine learning, deep learning, shape modeling, spectral graph theory, etc, to study anatomical structures of human organs, and analyze brain connectivity patterns in various population groups.

Hippocampus in the brain.
  • Kevin H. Hobbs, Pin Zhang, Bibo Shi, Charles D. Smith and Jundong Liu: Quad-mesh based radial distance biomarkers for Alzheimer's Disease, ISBI 2016: 19-23 pdf.
  • Chen, Y., Shi, B., Zhang, P., Smith, C., Wang, Z., Liu, J. (2017). Hippocampus Segmentation through Multi-view Ensemble ConvNets, 2017 IEEE International Symposium on Biomedical Imaging (ISBI). [pdf]

Machine Learning and Deep Learning

We have developed a set of deep neural network solutions to extract anatomical structures from MR images. The strategies we explore include multi-view ensemble to improve segmentation accuracy [Chen ISBI 2017], integration of CNN and RNN to enhance segm entation consistency [Chen MIMI 2017], and adoptive weight learning to remedy the label imbalance issue, which boosts the handling of small tumors [Chen PR submission]. We also employ a hierarchical architecture to combine multiple CNN components for brain lesion detection/segmentation. The coarse level CNN trains the model to provide an initial prediction of class labels, which is ne tuned by the lower level CNNs. An additional weighting layer is added to generate spatially varying weights assigned to diferent classes, through which the detection of small lesions is improved [Chen PR submission]. We are also exploring attention models to simulate human experts’ activity in labeling white matter hyperintensities, aiming to boost the overall segmentation performance, as well as reduce the missing of small lesions.

U-Net Architecture.
  • Bibo Shi, Yani Chen, Kevin Hobbs, Charles D. Smith and Jundong Liu (2015). Nonlinear Metric Laerning for Alzheimer's Disease Diagnosis with Integration of Longitudinal Neuroimaging Features, BMVC 2015: 138.1-138.13, [pdf].

Metric Learning

In this project, we developed a series of nonlinear geometric transformation based solution to improve supervised [Shi BMVC 2015, Shi PR 2017, Shi NC submission], semi-supervised classi cation, and unsupervised clustering tasks. Geometric transformations including Thin plate splines (TPS) and Coherent Point Drifting (CPD) are utilized to transform the feature space, aiming to improve the performance of the classi er that follows. Our solutions are the rst attempt that uses smooth, dense and spatially varying transformation in distance metric learning. We have applied our methods to a variety of datasets and applications, including neuroimages on Alzheimer’s Disease diagnosis, and improvements over the state-of-the-art solutions have been demonstrated.

Computer Vision

Street scene analysis with high consistency: frame-based street scene segmentation solutions tend to su er from temporal inconsistency, which is evident in some state-of-the-art deep learning solutions, including SegNet. We address this problem by supplementing fully convolutional networks (FCN) with the power of sequence modeling. On top of FCNs for individual frames, the video a convolutional LSTM (CLSTM) recurrent neural networks processes the video sequence end-to-end, leading to improved label consistency among neighboring frames. Inadequate delineation of small-sized regions, often due to label imbalance, is addressed through an adaptive weighting-layer scheme.