Enhancing Volumetric Medical Image Translation with a 3D Multi-resolution Attention-Dense U-Net

Juhyung (Tony) Ha, Jong Sung Park, Eleftherios Garyfallidis, David Crandall, Xuhong Zhang,
Luddy Computer Science Department, Indiana University,

Abstract

Medical image translation is a process of converting one imaging modality to another, aiming to reduce the necessity for multiple image acquisitions and enhance the efficiency of treatment planning. In this study, we introduce a Generative Adversarial Network (GAN)-based framework for 3D medical image translation, using a 3D multi-resolution Dense-Attention UNet (3D-mDAUNet) as the generator and a 3D multi-resolution UNet as the discriminator. Our model is optimized with a unique combination of loss functions, including voxel loss, voxel-wise relativistic GAN loss, and 2.5D perception loss. This approach yields promising results in volumetric image quality assessment (IQA) across a variety of imaging modalities, body regions, and age groups. Furthermore, we propose a synthetic-to-real applicability assessment as an additional evaluation to assess the effectiveness of synthetic data in downstream applications. Overall, our results not only indicate that our framework produces high-quality synthetic medical images but also demonstrates the potential utility of synthetic data, with extensive testing across various experimental settings.

3D Multi-resolution Dense-Attention U-Net (3D-mDAUNet)

Results

The figures below present source to target image translation. Left image is a source used as an input to our model. Right image is a translated output image generated by our model.

The figures below present the zoomed comparison among different methods.

The figures below present the holistic comparison among different methods.