Advancing machine learning for MR image reconstruction with an open competition: Overview of the 2019 fastMRI challenge.
Ontology highlight
ABSTRACT: Purpose
To advance research in the field of machine learning for MR image reconstruction with an open challenge.Methods
We provided participants with a dataset of raw k-space data from 1,594 consecutive clinical exams of the knee. The goal of the challenge was to reconstruct images from these data. In order to strike a balance between realistic data and a shallow learning curve for those not already familiar with MR image reconstruction, we ran multiple tracks for multi-coil and single-coil data. We performed a two-stage evaluation based on quantitative image metrics followed by evaluation by a panel of radiologists. The challenge ran from June to December of 2019.Results
We received a total of 33 challenge submissions. All participants chose to submit results from supervised machine learning approaches.Conclusions
The challenge led to new developments in machine learning for image reconstruction, provided insight into the current state of the art in the field, and highlighted remaining hurdles for clinical adoption.
SUBMITTER: Knoll F
PROVIDER: S-EPMC7719611 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Accelerating MRI scans is one of the principal outstanding problems in the MRI research community. Towards this goal, we hosted the second fastMRI competition targeted towards reconstructing MR images with subsampled k-space data. We provided participants with data from 7,299 clinical brain scans (de-identified via a HIPAA-compliant procedure by NYU Langone Health), holding back the fully-sampled data from 894 of these scans for challenge evaluation purposes. In contrast to the 2019 challenge, we focused our radiologist evaluations on pathological assessment in brain images. We also debuted a new Transfer track that required participants to submit models evaluated on MRI scanners from outside the training set. We received 19 submissions from eight different groups. Results showed one team scoring best in both SSIM scores and qualitative radiologist evaluations. We also performed analysis on alternative metrics to mitigate the effects of background noise and collected feedback from the participants to inform future challenges. Lastly, we identify common failure modes across the submissions, highlighting areas of need for future research in the MRI reconstruction community.
| S-EPMC8428775 | biostudies-literature
Project description:PURPOSE:Numerous image reconstruction methodologies for positron emission tomography (PET) have been developed that incorporate magnetic resonance (MR) imaging structural information, producing reconstructed images with improved suppression of noise and reduced partial volume effects. However, the influence of MR structural information also increases the possibility of suppression or bias of structures present only in the PET data (PET-unique regions). To address this, further developments for MR-informed methods have been proposed, for example, through inclusion of the current reconstructed PET image, alongside the MR image, in the iterative reconstruction process. In this present work, a number of kernel and maximum a posteriori (MAP) methodologies are compared, with the aim of identifying methods that enable a favorable trade-off between the suppression of noise and the retention of unique features present in the PET data. METHODS:The reconstruction methods investigated were: the MR-informed conventional and spatially compact kernel methods, referred to as KEM and KEM largest value sparsification (LVS) respectively; the MR-informed Bowsher and Gaussian MR-guided MAP methods; and the PET-MR-informed hybrid kernel and anato-functional MAP methods. The trade-off between improving the reconstruction of the whole brain region and the PET-unique regions was investigated for all methods in comparison with postsmoothed maximum likelihood expectation maximization (MLEM), evaluated in terms of structural similarity index (SSIM), normalized root mean square error (NRMSE), bias, and standard deviation. Both simulated BrainWeb (10 noise realizations) and real [18 F] fluorodeoxyglucose (FDG) three-dimensional datasets were used. The real [18 F]FDG dataset was augmented with simulated tumors to allow comparison of the reconstruction methodologies for the case of known regions of PET-MR discrepancy and evaluated at full counts (100%) and at a reduced (10%) count level. RESULTS:For the high-count simulated and real data studies, the anato-functional MAP method performed better than the other methods under investigation (MR-informed, PET-MR-informed and postsmoothed MLEM), in terms of achieving the best trade-off for the reconstruction of the whole brain and PET-unique regions, assessed in terms of the SSIM, NRMSE, and bias vs standard deviation. The inclusion of PET information in the anato-functional MAP method enables the reconstruction of PET-unique regions to attain similarly low levels of bias as unsmoothed MLEM, while moderately improving the whole brain image quality for low levels of regularization. However, for low count simulated datasets the anato-functional MAP method performs poorly, due to the inclusion of noisy PET information in the regularization term. For the low counts simulated dataset, KEM LVS and to a lesser extent, HKEM performed better than the other methods under investigation in terms of achieving the best trade-off for the reconstruction of the whole brain and PET-unique regions, assessed in terms of the SSIM, NRMSE, and bias vs standard deviation. CONCLUSION:For the reconstruction of noisy data, multiple MR-informed methods produce favorable whole brain vs PET-unique region trade-off in terms of the image quality metrics of SSIM and NRMSE, comfortably outperforming the whole image denoising of postsmoothed MLEM.
| S-EPMC6899618 | biostudies-literature
Project description:Magnetic resonance image (MRI) reconstruction using SENSitivity Encoding (SENSE) requires regularization to suppress noise and aliasing effects. Edge-preserving and sparsity-based regularization criteria can improve image quality, but they demand computation-intensive nonlinear optimization. In this paper, we present novel methods for regularized MRI reconstruction from undersampled sensitivity encoded data--SENSE-reconstruction--using the augmented Lagrangian (AL) framework for solving large-scale constrained optimization problems. We first formulate regularized SENSE-reconstruction as an unconstrained optimization task and then convert it to a set of (equivalent) constrained problems using variable splitting. We then attack these constrained versions in an AL framework using an alternating minimization method, leading to algorithms that can be implemented easily. The proposed methods are applicable to a general class of regularizers that includes popular edge-preserving (e.g., total-variation) and sparsity-promoting (e.g., l(1)-norm of wavelet coefficients) criteria and combinations thereof. Numerical experiments with synthetic and in vivo human data illustrate that the proposed AL algorithms converge faster than both general-purpose optimization algorithms such as nonlinear conjugate gradient (NCG) and state-of-the-art MFISTA.
| S-EPMC3081617 | biostudies-literature
Project description:PurposeThis work presents an end-to-end open-source MR imaging workflow. It is highly flexible in rapid prototyping across the whole imaging process and integrates vendor-independent openly available tools. The whole workflow can be shared and executed on different MR platforms. It is also integrated in the JEMRIS simulation framework, which makes it possible to generate simulated data from the same sequence that runs on the MRI scanner using the same pipeline for image reconstruction.MethodsMRI sequences can be designed in Python or JEMRIS using the Pulseq framework, allowing simplified integration of new sequence design tools. During the sequence design process, acquisition metadata required for reconstruction is stored in the MR raw data format. Data acquisition is possible on MRI scanners supported by Pulseq and in simulations through JEMRIS. An image reconstruction and postprocessing pipeline was implemented into a Python server that allows real-time processing of data as it is being acquired. The Berkeley Advanced Reconstruction Toolbox is integrated into this framework for image reconstruction. The reconstruction pipeline supports online integration through a vendor-dependent interface.ResultsThe flexibility of the workflow is demonstrated with different examples, containing 3D parallel imaging with controlled aliasing in volumetric parallel imaging (CAIPIRINHA) acceleration, spiral imaging, and B0 mapping. All sequences, data, and the corresponding processing pipelines are publicly available.ConclusionThe proposed workflow is highly flexible and allows integration of advanced tools at all stages of the imaging process. All parts of this workflow are open-source, simplifying collaboration across different MR platforms or sites and improving reproducibility of results.
| S-EPMC10054460 | biostudies-literature
Project description:This paper examines a combined supervised-unsupervised framework involving dictionary-based blind learning and deep supervised learning for MR image reconstruction from under-sampled k-space data. A major focus of the work is to investigate the possible synergy of learned features in traditional shallow reconstruction using adaptive sparsity-based priors and deep prior-based reconstruction. Specifically, we propose a framework that uses an unrolled network to refine a blind dictionary learning-based reconstruction. We compare the proposed method with strictly supervised deep learning-based reconstruction approaches on several datasets of varying sizes and anatomies. We also compare the proposed method to alternative approaches for combining dictionary-based methods with supervised learning in MR image reconstruction. The improvements yielded by the proposed framework suggest that the blind dictionary-based approach preserves fine image details that the supervised approach can iteratively refine, suggesting that the features learned using the two methods are complementary.
| S-EPMC8672324 | biostudies-literature
Project description:Recent advancements in deep learning for automated image processing and classification have accelerated many new applications for medical image analysis. However, most deep learning algorithms have been developed using reconstructed, human-interpretable medical images. While image reconstruction from raw sensor data is required for the creation of medical images, the reconstruction process only uses a partial representation of all the data acquired. Here, we report the development of a system to directly process raw computed tomography (CT) data in sinogram-space, bypassing the intermediary step of image reconstruction. Two classification tasks were evaluated for their feasibility of sinogram-space machine learning: body region identification and intracranial hemorrhage (ICH) detection. Our proposed SinoNet, a convolutional neural network optimized for interpreting sinograms, performed favorably compared to conventional reconstructed image-space-based systems for both tasks, regardless of scanning geometries in terms of projections or detectors. Further, SinoNet performed significantly better when using sparsely sampled sinograms than conventional networks operating in image-space. As a result, sinogram-space algorithms could be used in field settings for triage (presence of ICH), especially where low radiation dose is desired. These findings also demonstrate another strength of deep learning where it can analyze and interpret sinograms that are virtually impossible for human experts.
| S-EPMC6820559 | biostudies-literature
Project description:Anatomically-driven image reconstruction algorithms have become very popular in positron emission tomography (PET) where they have demonstrated improved image resolution and quantification. This work, consider the effect of spatial inconsistency between MR and PET images in hot and cold regions of the PET image. We investigate these effects on the kernel method from machine learning, in particular, the hybrid kernelized expectation maximization (HKEM). These were applied to Jaszczak phantom and patient data acquired with the Biograph Siemens mMR. The results show that even a small shift can cause a significant change in activity concentration. In general, the PET-MR inconsistencies can induce the partial volume effect, more specifically the 'spill-in' of the affected cold regions and the 'spill-out' from the hot regions. The maximum change was about 100% for the cold region and 10% for the hot lesion using KEM, against the 37% and 8% obtained with HKEM. The findings of this work suggest that including PET information in the kernel enhances the flexibility of the reconstruction in case of spatial inconsistency. Nevertheless, accurate registration and choice of the appropriate MR image for the creation of the kernel is essential to avoid artifacts, blurring, and bias.
| S-EPMC7596768 | biostudies-literature
Project description:To describe how integrated gradient nonlinearity (GNL) correction can be used within noniterative partial Fourier (homodyne) and parallel (SENSE and GRAPPA) MR image reconstruction strategies, and demonstrate that performing GNL correction during, rather than after, these routines mitigates the image blurring and resolution loss caused by postreconstruction image domain based GNL correction.Starting from partial Fourier and parallel magnetic resonance imaging signal models that explicitly account for GNL, noniterative image reconstruction strategies for each accelerated acquisition technique are derived under the same core mathematical assumptions as their standard counterparts. A series of phantom and in vivo experiments on retrospectively undersampled data were performed to investigate the spatial resolution benefit of integrated GNL correction over conventional postreconstruction correction.Phantom and in vivo results demonstrate that the integrated GNL correction reduces the image blurring introduced by the conventional GNL correction, while still correcting GNL-induced coarse-scale geometrical distortion. Images generated from undersampled data using the proposed integrated GNL strategies offer superior depiction of fine image detail, for example, phantom resolution inserts and anatomical tissue boundaries.Noniterative partial Fourier and parallel imaging reconstruction methods with integrated GNL correction reduce the resolution loss that occurs during conventional postreconstruction GNL correction while preserving the computational efficiency of standard reconstruction techniques. Magn Reson Med 75:2534-2544, 2016. © 2015 Wiley Periodicals, Inc.
| S-EPMC4716004 | biostudies-literature
Project description:PURPOSE:To develop and evaluate a novel deep learning-based reconstruction framework called SANTIS (Sampling-Augmented Neural neTwork with Incoherent Structure) for efficient MR image reconstruction with improved robustness against sampling pattern discrepancy. METHODS:With a combination of data cycle-consistent adversarial network, end-to-end convolutional neural network mapping, and data fidelity enforcement for reconstructing undersampled MR data, SANTIS additionally utilizes a sampling-augmented training strategy by extensively varying undersampling patterns during training, so that the network is capable of learning various aliasing structures and thereby removing undersampling artifacts more effectively and robustly. The performance of SANTIS was demonstrated for accelerated knee imaging and liver imaging using a Cartesian trajectory and a golden-angle radial trajectory, respectively. Quantitative metrics were used to assess its performance against different references. The feasibility of SANTIS in reconstructing dynamic contrast-enhanced images was also demonstrated using transfer learning. RESULTS:Compared to conventional reconstruction that exploits image sparsity, SANTIS achieved consistently improved reconstruction performance (lower errors and greater image sharpness). Compared to standard learning-based methods without sampling augmentation (e.g., training with a fixed undersampling pattern), SANTIS provides comparable reconstruction performance, but significantly improved robustness, against sampling pattern discrepancy. SANTIS also achieved encouraging results for reconstructing liver images acquired at different contrast phases. CONCLUSION:By extensively varying undersampling patterns, the sampling-augmented training strategy in SANTIS can remove undersampling artifacts more robustly. The novel concept behind SANTIS can particularly be useful for improving the robustness of deep learning-based image reconstruction against discrepancy between training and inference, an important, but currently less explored, topic.
| S-EPMC6660404 | biostudies-literature
Project description:As we age, our brain structure changes and our cognitive capabilities decline. Although brain aging is universal, rates of brain aging differ markedly, which can be associated with pathological mechanism of psychiatric and neurological diseases. Predictive models have been applied to neuroimaging data to learn patterns associated with this variability and develop a neuroimaging biomarker of the brain condition. Aiming to stimulate the development of more accurate brain-age predictors, the Predictive Analytics Competition (PAC) 2019 provided a challenge that included a dataset of 2,640 participants. Here, we present our approach which placed between the top 10 of the challenge. We developed an ensemble of shallow machine learning methods (e.g., Support Vector Regression and Decision Tree-based regressors) that combined voxel-based and surface-based morphometric data. We used normalized brain volume maps (i.e., gray matter, white matter, or both) and features of cortical regions and anatomical structures, like cortical thickness, volume, and mean curvature. In order to fine-tune the hyperparameters of the machine learning methods, we combined the use of genetic algorithms and grid search. Our ensemble had a mean absolute error of 3.7597 years on the competition, showing the potential that shallow methods still have in predicting brain-age.
| S-EPMC7738323 | biostudies-literature