back to list

Project: Visualization of 3D reconstruction uncertainty in the context of X-ray coronary angiography

Description

Coronary artery disease (CAD) involves the narrowing or blockage of coronary arteries due to plaque buildup or stenosis, reducing blood flow to the heart. To diagnose CAD, X-ray coronary angiograms (CAG) are acquired to display the coronary artery structure from a 2D perspective. Typically, only 1-2 CAGs are acquired in the clinic, but these sparse views limit accurate diagnosis and treatment. 3D reconstruction of these 2D CAGs could improve diagnosis and treatment by providing a more comprehensive view. One application is view-planning, where the 3D reconstruction is projected to a new viewpoint, giving clinicians an idea of how the view would look without additional X-rays.

However, 3D reconstruction from 1-2 images is an ill-posed problem due to the limited information available. Neural Radiance Field (NeRF) [1], a promising deep learning method for 3D reconstruction, has shown potential for reconstructing from such few images and has been applied to CAGs by our group [2]. Nevertheless, utilizing a machine learning model to reconstruct from limited data inherently leads to uncertainty, as multiple valid solutions may exist given the sparse information. While some works have addressed uncertainty computation in NeRFs for natural scenes, this has yet to be fully explored for medical scenes, which behave differently.

NeRF for view-planning applications utilizing uncertainty measures have been proposed outside the medical domain [3]. However, these applications do not directly translate to the clinical context, both in terms of user perspective as well as information available. Can we build on these ideas to create a visualization tool that guides clinicians to perform view-planning?


Project goals:

(1) Compute an uncertainty estimation for NeRF fitting the X-ray scene

(2) Create a visualization tool that can guide clinicians with view-planning utilizing these uncertainty estimations


Requirements:

  • Good programming skills
  • Ability to use and apply ML models
  • Visualization background
  • Eagerness to work with clinicians / understand the medical problem


References

[1] Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R. (2021). Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1), 99-106.

[2] Maas, K. W., Pezzotti, N., Vermeer, A. J., Ruijters, D., & Vilanova, A. (2023, September). Nerf for 3d reconstruction from x-ray angiography: Possibilities and limitations. In VCBM 2023: Eurographics Workshop on Visual Computing for Biology and Medicine (pp. 29-40). Eurographics Association.

[3] Previous master project: https://vis.win.tue.nl/masterprojects/106/

Details
Supervisor
Kirsten Maas
Secondary supervisor
Anna Vilanova
Interested?
Get in contact