Decoding VGGT Features in 3D with Sparse Convolution

  Project Report (PDF)

Overview

This project explores decoding VGGT / AnySplat features for 3D Gaussian Splatting using sparse 3D convolutions, replacing the original MLP-based decoder with a ResNet-style architecture that better exploits voxelized spatial structure.

Contributions

• Feed-forward Gaussian Splatting refinement pipeline built on top of the VGGT/AnySplat backbone, introducing a ResNet-style sparse 3D convolutional decoder for voxelized feature aggregation.
• Kernel normalization for sparse 3D convolutions to stabilize feature aggregation under sparse voxel occupancy, addressing the inconsistent receptive field problem caused by varying numbers of active voxels.

Results

Improved 3D reconstruction quality on standard benchmarks:

Stack

PyTorch · SpConv · VGGT / AnySplat · Gaussian Splatting