CaiYuanhao/X-Gaussian

Radiative Gaussian Splatting for Efficient X-ray Novel View Synthesis

ECCV 2024

5 CT scenes + 3 real scans · X-ray novel view synthesis · Direct 3D CT reconstruction

Paper · Code · Models · Data

Point Cloud Visualization

Training Process Visualization

GIF demonstrations are from the official X-Gaussian repository. The available CT and real-scan scenes are listed below.

Model Description

X-Gaussian represents X-ray attenuation using radiative 3D Gaussians. Introduced in Radiative Gaussian Splatting for Efficient X-ray Novel View Synthesis (ECCV 2024), it supports efficient X-ray novel view synthesis. These checkpoints use the current joint 2D/3D implementation, which also directly reconstructs a CT volume from the same Gaussian field while retaining the original Gaussian point attributes.

This repository provides eight scene-specific trained models: Chest, Abdomen, Foot, Head, Pancreas, and the real FIPS scans Pine, Seashell, and Walnut, paired with the X-Gaussian dataset. Each model represents its corresponding scene; reconstructing another scene requires training from that scene's projections and scanner geometry.

Available Checkpoints

All weights are the final iteration 31,000 checkpoints from the eight runs reported in the code repository.

Scene	Gaussian checkpoint	Dataset	Gaussians
Chest	point_cloud.ply	synthetic/chest_50.pickle	500,000
Abdomen	point_cloud.ply	synthetic/abdomen_50.pickle	59,756
Foot	point_cloud.ply	synthetic/foot_50.pickle	77,682
Head	point_cloud.ply	synthetic/head_50.pickle	116,013
Pancreas	point_cloud.ply	synthetic/pancreas_50.pickle	244,511
Pine	point_cloud.ply	real/pine/	80,067
Seashell	point_cloud.ply	real/seashell/	80,734
Walnut	point_cloud.ply	real/walnut/	177,247

Each scene folder preserves the layout expected by the project's render.py:

<synthetic|real>/<scene>/
├── config.json
├── metrics.json
└── point_cloud/
    └── iteration_31000/
        └── point_cloud.ply

config.json contains the training settings, decoder configuration, and original voxel geometry. Its data path is relative to the project root (data/synthetic/<scene>_50.pickle or data/real/<scene>). Outputs follow the same layout: output/synthetic/<scene>/ or output/real/<scene>/. The PLY includes global decoder and geometry metadata and retains the original 62 per-point columns. It can be used independently for 3D volume export; 2D rendering also needs the corresponding camera geometry. manifest.json records file sizes and SHA-256 checksums.

Training and Evaluation

Each model was trained from 50 X-ray projections at the released dataset resolution, using training-only FDK initialization, 30,000 base training steps, and 1,000 volume-ray refinement steps. Training jointly uses projection loss, 3D total variation, and volume-ray consistency. Reference CT volumes were used only for evaluation.

Performance

X-ray Novel View Synthesis

Results on the original five CT scenes, averaged over 50 held-out views per scene:

Scene	2D PSNR (dB)	2D SSIM
Chest	47.21	0.9937
Abdomen	48.16	0.9968
Foot	44.40	0.9869
Head	51.66	0.9979
Pancreas	47.65	0.9923

Each predicted and reference projection is normalized by its own maximum.

Original paper comparisons

The figures reproduce the original paper results; the table above evaluates the hosted joint 2D/3D checkpoints.

Direct 3D CT Reconstruction

Results measured on the five released models after 30,000 training steps and 1,000 refinement steps, without downsampling:

Scene	CT resolution	3D PSNR (dB)	3D SSIM
Chest	128 × 128 × 128	31.94	0.8895
Abdomen	512 × 512 × 463	33.33	0.8917
Foot	256 × 256 × 256	31.88	0.8795
Head	256 × 256 × 128	40.21	0.9732
Pancreas	512 × 512 × 240	22.22	0.8750

3D metrics use a fixed range of 1 on the original voxel grid, without clipping or intensity correction. SSIM averages nonempty reference slices along each axis, then across axes. CT ground truth is used only for evaluation.

Real Data

Real FIPS scans use 50 training and 100 held-out views, 560 × 560 detector images, and a 256³ reconstruction grid, without additional downsampling.

Scene	2D PSNR (dB)	2D SSIM	3D PSNR vs. FDK (dB)	3D SSIM vs. FDK
Pine	39.18	0.9777	37.79	0.9399
Seashell	32.54	0.9749	42.62	0.9593
Walnut	34.24	0.9140	30.32	0.6810

2D metrics average 100 held-out measured projections per scene, each normalized by its own maximum. The 3D reference is full-view FDK, not measured CT ground truth. 3D metrics use a fixed range of 1 without clipping or intensity correction. Each model uses 30,000 training and 1,000 refinement steps; the reference volume and test views are not used for training or initialization.

Left to right: measured X-ray, rendered X-ray, full-view FDK pseudo-reference, and X-Gaussian CT. CT previews use the same fixed 0–1 intensity range.

See the real-data sources and CC BY 4.0 attribution.

Usage

Please refer to the X-Gaussian repository for environment setup and model usage, including 2D projection rendering and direct 3D CT reconstruction.

Reconstruction Previews

Central orthogonal slices from the hosted models. In each image, the top row shows the reference CT, the middle row the reconstruction, and the bottom row the absolute error. Displayed CT intensities use a fixed range of 0–1 and errors use 0–0.1; metrics above use the unmodified arrays.

Chest

Abdomen

Foot

Head

Pancreas

Pine (real; full-view FDK reference)

Seashell (real; full-view FDK reference)

Walnut (real; full-view FDK reference)

Citation

If you use these models, please cite X-Gaussian and the SAX-NeRF dataset source:

@inproceedings{x_gaussian,
  title={Radiative gaussian splatting for efficient x-ray novel view synthesis},
  author={Yuanhao Cai and Yixun Liang and Jiahao Wang and Angtian Wang and Yulun Zhang and Xiaokang Yang and Zongwei Zhou and Alan Yuille},
  booktitle={ECCV},
  year={2024}
}

@inproceedings{sax_nerf,
  title={Structure-Aware Sparse-View X-ray 3D Reconstruction},
  author={Yuanhao Cai and Jiahao Wang and Alan Yuille and Zongwei Zhou and Angtian Wang},
  booktitle={CVPR},
  year={2024}
}