PhD Defense

We introduce SCONE, a novel mathematical framework for surface coverage-based exploration using depth sensors. Unlike previous approaches that rely on reinforcement learning, this supervised learning approach enables efficient exploration of arbitrarily large scenes with unrestricted camera motion. The framework's purely geometric considerations allow for better generalization to unseen scenes, making it particularly valuable for real-world applications.

We present MACARONS, which eliminates the need for depth sensors and explicit 3D ground truth by leveraging self-supervised learning from RGB images alone. This advancement enables online learning in large unknown environments, making autonomous exploration more practical and cost-effective. MACARONS learns to generate 3D reconstructions directly from RGB images and uses these reconstructions to supervise its coverage predictions, creating a self-improving system suitable for deployment in real-world scenarios.

We propose SuGaR, a method that combines explicit surface representation with Gaussian splatting for fast, high-quality mesh reconstruction. SuGaR first introduces a regularization strategy for extracting explicit surface meshes from Gaussian Splatting representations. Then, SuGaR introduces a hybrid representation that binds Gaussians to the extracted surface meshes and refines the structure through differentiable rendering. This approach achieves photorealistic rendering within remarkably short optimization times (less than an hour) and enables unprecedented fine-grained editing capabilities based on meshes, including surface sculpting, character animation, and scene compositing—features previously unavailable in radiance field models.

We extend SuGaR with Gaussian Frosting, which significantly improves material handling and rendering efficiency while maintaining SuGaR's interactive features. The method introduces a novel occlusion culling strategy and achieves rendering quality comparable to vanilla Gaussian Splatting. Notably, Frosting excels at reconstructing challenging materials like hair, fur, and grass, which are traditionally difficult to represent with pure surface-based methods but essential for building photorealistic scenes and virtual avatars. To democratize these capabilities, we developed a dedicated Blender add-on that enables artists and content creators to edit, sculpt, combine, and animate their reconstructions without programming expertise.

Finally, we introduce MAtCha, which tackles the challenge of sparse-view reconstruction. By leveraging learned priors, MAtCha enables high-quality mesh reconstruction from just 3-10 input images—a significant advancement over previous methods that required hundreds of images and complex preprocessing steps. The method produces sharp, detailed meshes of both foreground and background elements while maintaining high geometric quality across the entire scene. This capability might make high-quality 3D reconstruction more accessible to everyday users, potentially democratizing the technology for widespread adoption.