About this video Over deze video
Volumetric videos offer immersive 4D experiences, but remain difficult to reconstruct, store, and stream at scale. Existing Gaussian Splatting based methods achieve high-quality reconstruction but break down on long sequences, temporal inconsistency, and fail under large motions and disocclusions. Moreover, their outputs are typically incompatible with conventional video coding pipelines, preventing practical applications. Rai et al. introduce PackUV, a novel 4D Gaussian representation that maps all Gaussian attributes into a sequence of structured, multi-scale UV atlas, enabling compact, image-native storage. To fit this representation from multi-view videos, they propose PackUV-GS, a temporally consistent fitting method that directly optimizes Gaussian parameters in the UV domain. An optical flow-guided Gaussian labeling and video keyframing module identifies dynamic Gaussians, stabilizes static regions, and preserves temporal coherence even under large motions and disocclusions. The resulting UV atlas format is the first unified volumetric video representation fully compatible with standard video codecs (e.g., FFV1) without losing quality, enabling efficient streaming within existing multimedia infrastructure. To evaluate long-duration volumetric capture, they present PackUV-2B, the largest multi-view video dataset to date, featuring more than 50-90 synchronized cameras, substantial motion, and frequent disocclusions across 100+ sequences and over 2B (billion) frames. Extensive experiments demonstrate that the method surpasses existing baselines in rendering fidelity while scaling to sequences up to 30 minutes with consistent quality.
Aashish Rai is a Ph.D. student in Computer Science at Brown University, advised by Srinath Sridhar. His research focuses on efficient methods for 3D and 4D novel view synthesis, reconstruction, and generative world modeling. He has also worked at Meta Reality Labs, developing methods that leverage 2D foundation models for 3D asset synthesis. Previously, he was a Research Assistant at Carnegie Mellon University’s Robotics Institute, working with Fernando De la Torre on realistic 3D face generation using 2D models.
This session is brought to you by the Cohere Labs Open Science Community - a space where ML researchers, engineers, linguists, social scientists, and lifelong learners connect and collaborate with each other. We'd like to extend a special thank you to Benedict Emoekabu and Mayank Bhaskar, Leads of our Computer Vision group for their dedication in organizing this event.
If you’re interested in sharing your work, we welcome you to join us! Simply fill out the form at https://forms.gle/ALND9i6KouEEpCnz6 to express your interest in becoming a speaker.
Join the Cohere Labs Open Science Community to see a full list of upcoming events (https://tinyurl.com/CohereLabsCommunityApp).
