back to list

Project: Identify and visualize domain shift in image-2-image models

Description
Introduced in 2015 by Ronneberger et al. [1], The U-Net model is one of the most powerful neural networks for image translation. This process allows an input image to be translated into an (possibly very different) image efficiently and effectively. It has also been used for image segmentation, pixel-wise classification and many other image enhancement techniques such as denoising, super resolution and artifact removal.

For image-2-image models, such as U-Net, the direct application of the techniques developed for classification is not an effective approach. The reason for it is the distributed nature of the model, due to the skip connections, and the fact that the domain shift can happen in localized region of the image (the domain shift can happen only in a small patch of the global image).

In this project we want to investigate the domain shift in image to image models. Specifically, we want to investigate if it is possible to detect a domain shift in the input of reconstruction models for MR images [3]. We expect that a combination of Visual Analytics and unsupervised learning approaches can shed a light on this problem, enabling machine learning developers to create robust models with respect to the domain shift.

Details
Student
Sihan Zhu
Supervisor
Anna Vilanova
Secondary supervisor
NP
Nicola Pezzotti, Vidya Prasad