Medical Image Analysis

Self-supervised representation learning can boost the performance of a pre-trained network on downstream tasks for which labeled data is limited. A popular method based on this paradigm, known as contrastive learning, works by constructing sets of positive and negative pairs from the data, and then pulling closer the representations of positive pairs while pushing apart those of negative pairs. Although contrastive learning has been shown to improve performance in various classification tasks, its application to image segmentation has been more limited...

Augmented Reality (AR) from preoperative data is a promising approach to improve intraoperative tumour localisation in Laparoscopic Liver Resection (LLR). Existing systems register the preoperative tumour model with the laparoscopic images and render it by direct camera projection, as if the organ were transparent. However, a simple geometric reasoning shows that this may induce serious surgeon misguidance. This is because the tools enter in a different keyhole than the laparoscope. As AR is particularly important for deep tumours, this problem potentially hinders the whole interest of AR guidance...

The variation in histologic staining between different medical centers is one of the most profound challenges in the field of computer-aided diagnosis. The appearance disparity of pathological whole slide images causes algorithms to become less reliable, which in turn impedes the wide-spread applicability of downstream tasks like cancer diagnosis. Furthermore, different stainings lead to biases in the training which in case of domain shifts negatively affect the test performance. Therefore, in this paper we propose MultiStain-CycleGAN, a multi-domain approach to stain normalization based on CycleGAN...

In recent years, deep learning (DL) has shown great potential in the field of dermatological image analysis. However, existing datasets in this domain have significant limitations, including a small number of image samples, limited disease conditions, insufficient annotations, and non-standardized image acquisitions. To address these shortcomings, we propose a novel framework called DermSynth3D. DermSynth3D blends skin disease patterns onto 3D textured meshes of human subjects using a differentiable renderer and generates 2D images from various camera viewpoints under chosen lighting conditions in diverse background scenes...

Three-dimensional (3D) ultrasound imaging has contributed to our understanding of fetal developmental processes by providing rich contextual information of the inherently 3D anatomies. However, its use is limited in clinical settings, due to the high purchasing costs and limited diagnostic practicality. Freehand 2D ultrasound imaging, in contrast, is routinely used in standard obstetric exams, but inherently lacks a 3D representation of the anatomies, which limits its potential for more advanced assessment...

The state-of-the-art multi-organ CT segmentation relies on deep learning models, which only generalize when trained on large samples of carefully curated data. However, it is challenging to train a single model that can segment all organs and types of tumors since most large datasets are partially labeled or are acquired across multiple institutes that may differ in their acquisitions. A possible solution is Federated learning, which is often used to train models on multi-institutional datasets where the data is not shared across sites...

Quantitative susceptibility mapping (QSM) is a post-processing technique for deriving tissue magnetic susceptibility distribution from MRI phase measurements. Deep learning (DL) algorithms hold great potential for solving the ill-posed QSM reconstruction problem. However, a significant challenge facing current DL-QSM approaches is their limited adaptability to magnetic dipole field orientation variations during training and testing. In this work, we propose a novel Orientation-Adaptive Latent Feature Editing (OA-LFE) module to learn the encoding of acquisition orientation vectors and seamlessly integrate them into the latent features of deep networks...

Pancreatic cancer (PC) is a severely malignant cancer variant with high mortality. Since PC has no obvious symptoms, most PC patients are belatedly diagnosed at advanced disease stages. Recently, artificial intelligence (AI) approaches have demonstrated promising prospects for early diagnosis of pancreatic cancer. However, certain non-causal factors (such as intensity and texture appearance variations, also called confounders) tend to induce spurious correlation with PC diagnosis. This undermines the generalization performance and the clinical applicability of the AI-based PC diagnosis approaches...

Focused cardiac ultrasound (FoCUS) is a valuable point-of-care method for evaluating cardiovascular structures and function, but its scope is limited by equipment and operator's experience, resulting in primarily qualitative 2D exams. This study presents a novel framework to automatically estimate the 3D spatial relationship between standard FoCUS views. The proposed framework uses a multi-view U-Net-like fully convolutional neural network to regress line-based heatmaps representing the most likely areas of intersection between input images...

Recognition of mitotic figures in histologic tumor specimens is highly relevant to patient outcome assessment. This task is challenging for algorithms and human experts alike, with deterioration of algorithmic performance under shifts in image representations. Considerable covariate shifts occur when assessment is performed on different tumor types, images are acquired using different digitization devices, or specimens are produced in different laboratories. This observation motivated the inception of the 2022 challenge on MItosis Domain Generalization (MIDOG 2022)...

Deep learning methods show great potential for the efficient and precise estimation of quantitative parameter maps from multiple magnetic resonance (MR) images. Current deep learning-based MR parameter mapping (MPM) methods are mostly trained and tested using data with specific acquisition settings. However, scan protocols usually vary with centers, scanners, and studies in practice. Thus, deep learning methods applicable to MPM with varying acquisition settings are highly required but still rarely investigated...

Recently, functional magnetic resonance imaging (fMRI) based functional connectivity network (FCN) analysis via graph convolutional networks (GCNs) has shown promise for automated diagnosis of brain diseases by regarding the FCNs as irregular graph-structured data. However, multiview information and site influences of the FCNs in a multisite, multiatlas fMRI scenario have been understudied. In this paper, we propose a Class-consistency and Site-independence Multiview Hyperedge-Aware HyperGraph Embedding Learning (CcSi-MHAHGEL) framework to integrate FCNs constructed on multiple brain atlases in a multisite fMRI study...

Nuclei detection and segmentation in hematoxylin and eosin-stained (H&E) tissue images are important clinical tasks and crucial for a wide range of applications. However, it is a challenging task due to nuclei variances in staining and size, overlapping boundaries, and nuclei clustering. While convolutional neural networks have been extensively used for this task, we explore the potential of Transformer-based networks in combination with large scale pre-training in this domain. Therefore, we introduce a new method for automated instance segmentation of cell nuclei in digitized tissue samples using a deep learning architecture based on Vision Transformer called CellViT...

The availability of big data can transform the studies in biomedical research to generate greater scientific insights if expert labeling is available to facilitate supervised learning. However, data annotation can be labor-intensive and cost-prohibitive if pixel-level precision is required. Weakly supervised semantic segmentation (WSSS) with image-level labeling has emerged as a promising solution in medical imaging. However, most existing WSSS methods in the medical domain are designed for single-class segmentation per image, overlooking the complexities arising from the co-existence of multiple classes in a single image...

Cardiac cine magnetic resonance imaging (MRI) is a commonly used clinical tool for evaluating cardiac function and morphology. However, its diagnostic accuracy may be compromised by the low spatial resolution. Current methods for cine MRI super-resolution reconstruction still have limitations. They typically rely on 3D convolutional neural networks or recurrent neural networks, which may not effectively capture long-range or non-local features due to their limited receptive fields. Optical flow estimators are also commonly used to align neighboring frames, which may cause information loss and inaccurate motion estimation...

In the context of automatic medical image segmentation based on statistical learning, raters' variability of ground truth segmentations in training datasets is a widely recognized issue. Indeed, the reference information is provided by experts but bias due to their knowledge may affect the quality of the ground truth data, thus hindering creation of robust and reliable datasets employed in segmentation, classification or detection tasks. In such a framework, automatic medical image segmentation would significantly benefit from utilizing some form of presegmentation during training data preparation process, which could lower the impact of experts' knowledge and reduce time-consuming labeling efforts...

Analyzing functional brain networks (FBN) with deep learning has demonstrated great potential for brain disorder diagnosis. The conventional construction of FBN is typically conducted at a single scale with a predefined brain region atlas. However, numerous studies have identified that the structure and function of the brain are hierarchically organized in nature. This urges the need of representing FBN in a hierarchical manner for more effective analysis of the complementary diagnostic insights at different scales...

Ultrasound is a promising medical imaging modality benefiting from low-cost and real-time acquisition. Accurate tracking of an anatomical landmark has been of high interest for various clinical workflows such as minimally invasive surgery and ultrasound-guided radiation therapy. However, tracking an anatomical landmark accurately in ultrasound video is very challenging, due to landmark deformation, visual ambiguity and partial observation. In this paper, we propose a long-short diffeomorphism memory network (LSDM), which is a multi-task framework with an auxiliary learnable deformation prior to supporting accurate landmark tracking...

The brain development during the perinatal period is characterized by rapid changes in both structure and function, which have significant impact on the cognitive and behavioral abilities later in life. Accurate assessment of brain age is a crucial indicator for brain development maturity and can help predict the risk of neonatal pathology. However, evaluating neonatal brains using magnetic resonance imaging (MRI) is challenging due to its complexity, multi-dimension, and noise with subtle alterations. In this paper, we propose a multi-modal deep learning framework based on transformers for precise post-menstrual age (PMA) estimation and brain development analysis using T2-weighted structural MRI (T2-sMRI) and diffusion MRI (dMRI) data...

Decoding brain states under different cognitive tasks from functional magnetic resonance imaging (fMRI) data has attracted great attention in the neuroimaging filed. However, the well-known temporal dependency in fMRI sequences has not been fully exploited in existing studies, due to the limited temporal-modeling capacity of the backbone machine learning algorithms and rigid training sample organization strategies upon which the brain decoding methods are built. To address these limitations, we propose a novel method for fine-grain brain state decoding, namely, group deep bidirectional recurrent neural network (Group-DBRNN) model...