MOSAIC-GS: MOnocular Scene Reconstruction via Advanced Initialization for Complex Dynamic Environments

Svitlana Morkva1, Maximum Wilder-Smith1, Michael Oechsle2, Alessio Tonioni2, Marco Hutter1, Vaishakh Patil1
1ETH Zürich 2Google Zürich
Accepted to CVPR 2026
Paper arXiv Video Github

MOSAIC-GS: monocular dynamic scene reconstruction via geometry-guided Gaussian Splatting.
High visual fidelity reconstruction at 180 FPS, trained in under 11 minutes — with dynamic instance segmentation and editing applications for free.

Abstract

We present MOSAIC-GS, a fully explicit and computationally efficient approach for high-fidelity dynamic scene reconstruction from monocular videos using Gaussian Splatting. To overcome the ill-posed nature of monocular reconstruction and the lack of multiview constraints, we leverage a suite of geometric cues—including depth, optical flow, dynamic segmentation, and point tracking. By integrating these with rigidity-based motion constraints, we estimate 3D scene dynamics during an initialization stage, reducing the reliance on ambiguous visual appearance for motion inference.

To ensure real-time rendering and compact motion encoding, the scene is disentangled into static and dynamic components. Trajectories in the dynamic partition are represented as time-dependent Poly-Fourier curves, enabling parameter-efficient support for non-rigid deformations. MOSAIC-GS achieves significantly faster optimization and rendering speeds compared to existing methods, while maintaining reconstruction quality on par with state-of-the-art monocular dynamic scene benchmarks.

The main contributions of our work can be summarized as follows:

  • Advanced Pre-processing: A framework for dynamic Gaussian Splatting that accelerates training and improves reconstruction fidelity.
  • Robust Monocular Pipeline: An integrated system for precise dynamic object detection and tracking across video sequences.
  • Trajectory Refinement: A novel method for extracting and optimizing 3D motion using per-instance rigidity constraints.
  • State-of-the-art Efficiency: High-speed training and inference achieved through scene disentanglement, compact deformation representation, and accurate motion initialization.

Pipeline

MOSAIC-GS features a four-step pre-processing pipeline: detecting dynamic regions via optical flow and epipolar error, tracking instances to extract rigid transformations, and encoding 3D motion as Poly-Fourier trajectories. Our precise initialization and disentangled Gaussian optimization enable high-fidelity reconstruction at a fraction of the computational cost of competing frameworks.

Novel View Reconstruction Results

Novel view reconstructions across scenes from the DyCheck and NVIDIA Dynamic Scene benchmarks, including various dynamic objects. Each video shows the rendered output from a held-out camera viewpoint not seen during training.

Qualitative Reconstruction Comparison

Drag the sliders to compare MOSAIC-GS against the ground truth, MoSca, and Gaussian Flow on dynamic scenes from the DyCheck dataset.

Space Out

Ground Truth Ours MoSca Gaussian Flow

Wheel

Ground Truth Ours MoSca Gaussian Flow

Block

Ground Truth Ours MoSca Gaussian Flow

Windmill

Ground Truth Ours MoSca Gaussian Flow

Training & Rendering Efficiency

Training time and rendering speed averaged across the DyCheck dataset. Reported training times include full pre-processing; the photometric optimization alone takes approximately 5 minutes.

Method Training Time ↓ Rendering Speed ↑
Gaussian Marbles* 5–9h 200 FPS
Gaussian Flow 23 min 52 FPS
MoSca 50 min 38 FPS
Ours (MOSAIC-GS) 10.5 min Best 180 FPS 2nd
* Values for Gaussian Marbles are taken from the original paper. All other methods were benchmarked on one NVIDIA RTX 4090 GPU.

Additional Applications

Beyond high-fidelity reconstruction, MOSAIC-GS enables temporally consistent scene segmentation and dynamic editing. By assigning segmentation IDs to dynamic Gaussians during initialization, the system allows for precise control over scene elements without any additional computational cost.

Reconstruction

Block Reconstruction

The core of MOSAIC-GS is the efficient, high-fidelity reconstruction of dynamic scenes, providing a foundation for further complex visual tasks.

Segmentation

Block Segmentation

Using point tracking and temporal refinement, the system ensures consistent mask identifiers over time. This avoids identity swaps and produces clean segmentation of dynamic objects as a natural by-product of reconstruction.

Editing

Scene Editing

Users can perform diverse editing operations, such as removing objects (e.g., the dark gray cube), changing object colors, adjusting motion/position, or isolating dynamic elements on a blank background.

BibTeX

@inproceedings{morkva2026mosaicgs,
  title     = {MOSAIC-GS: Monocular Scene Reconstruction via Advanced Initialization for Complex Dynamic Environments},
  author    = {Svitlana Morkva and Maximum Wilder-Smith and Michael Oechsle and Alessio Tonioni and Marco Hutter and Vaishakh Patil},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year      = {2026},
}