Project: Gaussian splatting for the 3D/4D reconstruction of X-ray angiography data
Description
The field
of 3D reconstruction from 2D imaging using deep learning is rapidly advancing. Neural
Radiance Fields (NeRFs) [1] have shown great promise in this area for both
natural and medical imaging. To address the slow training and inference of
these models, Gaussian Splatting [2] has been introduced, reducing
reconstruction times from hours to minutes.
X-ray
coronary angiography, which images the coronary arteries, stands to benefit
from these improvements. Some studies have explored using Gaussian Splatting
for this data [3], but they rely on time-consuming segmentations, limiting
clinical use. Our group recently proposed a NeRF-based method that reconstructs
directly from raw dynamic X-ray angiography data without segmentation [4], though it
remains too slow for clinical application. While moving from NeRFs to Gaussian
Splatting would improve computational speed, no solution currently applies this
technique directly to raw angiography data.
In this project, you will develop a fast method for 3D reconstruction of X-ray coronary angiography using the latest Gaussian Splatting techniques. We expect you to have a strong technical background, as you will be working with computer graphics techniques and deep learning models.
Video from [4]
[1] https://www.matthewtancik.com/nerf;
Mildenhall, B., Srinivasan, P. P., Tancik, M., Barron, J. T., Ramamoorthi, R.,
& Ng, R. (2021). Nerf: Representing scenes as neural radiance fields for
view synthesis. Communications of the ACM, 65(1), 99-106.
[2] https://repo-sam.inria.fr/fungraph/3d-gaussian-splatting/;
Kerbl, B., Kopanas, G., Leimkühler, T., & Drettakis, G. (2023). 3D Gaussian Splatting for Real-Time
Radiance Field Rendering. ACM Trans. Graph., 42(4), 139-1.
[3] Fu,
X. et al. (2024). 3DGR-CAR: Coronary Artery Reconstruction
from Ultra-sparse 2D X-Ray Views with a 3D Gaussians
Representation. In: Linguraru, M.G., et al. Medical Image
Computing and Computer Assisted Intervention – MICCAI 2024. MICCAI 2024. Lecture Notes in Computer Science, vol 15007. Springer,
Cham. https://doi.org/10.1007/978-3-031-72104-5_2
[4] https://kirstenmaas.github.io/nerfca/;
Maas, K. W. H., Ruijters, D., Vilanova, A., & Pezzotti, N. (2024). NeRF-CA:
Dynamic Reconstruction of X-ray Coronary Angiography with Extremely
Sparse-views. arXiv preprint arXiv:2408.16355.
Details
- Supervisor
-
Kirsten Maas
- Secondary supervisor
-
Anna Vilanova
- Interested?
- Get in contact