Project: Visualization of 3D reconstruction uncertainty in the context of X-ray coronary angiography
Description
Coronary artery disease (CAD) involves the narrowing or blockage of coronary arteries due to plaque buildup or stenosis, reducing blood flow to the heart. To diagnose CAD, X-ray coronary angiograms (CAG) are acquired to display the coronary artery structure from a 2D perspective. Typically, only 1-2 CAGs are acquired in the clinic, but these sparse views limit accurate diagnosis and treatment. 3D reconstruction of these 2D CAGs could improve diagnosis and treatment by providing a more comprehensive view. One clinical application is view-planning, where the 3D reconstruction is projected to a new viewpoint, giving clinicians an idea of how the view would look without additional X-rays. The figure gives a schematic overview of this scenario.
However, 3D
reconstruction from 1-2 images is an ill-posed problem due to the limited
information available. Neural Radiance Field (NeRF) [1], a promising deep
learning method for 3D reconstruction, has shown potential for reconstructing
from such few images and has been applied to CAGs by our group [2]. Nevertheless,
utilizing a machine learning model to reconstruct from limited data inherently
leads to uncertainty, as multiple valid solutions may exist given the sparse
information. While some works have addressed uncertainty computation in NeRFs
for natural scenes, this has yet to be fully explored for medical scenes.
NeRF for view-planning applications utilizing uncertainty measures have been proposed outside the medical domain [3]. However, these applications do not directly translate to the clinical context, both in terms of user perspective as well as data perspective. Can we build on these ideas to create a visualization tool that guides clinicians to perform view-planning?
Project
goals:
(1) Compute
an uncertainty estimation for NeRF fitting the X-ray scene
(2) Create
a visualization tool that can guide clinicians with view-planning utilizing these
uncertainty estimations
Requirements:
- Good programming skills
- Ability to use and apply ML models
- Visualization background
- Eagerness to understand the medical problem
References
[1] Mildenhall,
B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R., & Ng, R.
(2021). Nerf: Representing scenes as neural radiance fields for view synthesis.
Communications of the ACM, 65(1), 99-106.
[2] Maas, K. W., Pezzotti,
N., Vermeer, A. J., Ruijters, D., & Vilanova, A. (2023, September). Nerf
for 3d reconstruction from x-ray angiography: Possibilities and limitations. In VCBM 2023: Eurographics Workshop on Visual Computing for Biology and
Medicine (pp. 29-40). Eurographics Association.
[3] Previous master project: https://research.tue.nl/en/studentTheses/uncertainty-visualization-for-neural-radiance-field
Details
- Supervisor
-
Kirsten Maas
- Secondary supervisor
-
Anna Vilanova
- Interested?
- Get in contact