InfoNorm: Mutual Information Shaping of Normals for Sparse-View Reconstruction

Xulong Wang 1,2*, Siyan Dong3*, Youyi Zheng1†, Yanchao Yang3†,
1State Key Lab of CAD&CG, Zhejiang University
2Chohotech co. ltd.
3Institute of Data Science, The University of Hong Kong
4Department of Electrical and Electronic Engineering, The University of Hong Kong
European Conference on Computer Vision ECCV 2024
*Indicates Equal Contribution
†Corresponding authors
2This work was done during the author's internship at Chohotech Co. ltd..
Paper Supplementary Code arXiv
Interpolate start reference image.

An overview of the pipeline. We encode geometry-aware mutual information into a subset of the parameters to constrain the density field learning for better surface quality.


Abstract

3D surface reconstruction from multi-view images is essential for scene understanding and interaction. However, complex indoor scenes pose challenges such as ambiguity due to limited observations. Recent implicit surface representations, such as Neural Radiance Fields (NeRFs) and signed distance functions (SDFs), employ various geometric priors to resolve the lack of observed information. Nevertheless, their performance heavily depends on the quality of the pre-trained geometry estimation models. To ease such dependence, we propose regularizing the geometric modeling by explicitly encouraging the mutual information among surface normals of highly correlated scene points. In this way, the geometry learning process is modulated by the second-order correlations from noisy (first-order) geometric priors, thus eliminating the bias due to poor generalization. Additionally, we introduce a simple yet effective scheme that utilizes semantic and geometric features to identify correlated points, enhancing their mutual information accordingly. The proposed technique can serve as a plugin for SDF-based neural surface representations. Our experiments demonstrate the effectiveness of the proposed in improving the surface reconstruction quality of major states of the arts.



Results

ScanNet++

Interpolate start reference image.

Each row shows a comparison with a different baseline, from top to bottom, NeuS+, VolSDF+, GeoNeuS+, I2-SDF+, NeuRIS+, MonoSDF+, and Neuralangelo+. Red boxes are overlaid to help the contrast.


BibTeX

@misc{wang2024infonormmutualinformationshaping,
    title={InfoNorm: Mutual Information Shaping of Normals for Sparse-View Reconstruction}, 
    author={Xulong Wang and Siyan Dong and Youyi Zheng and Yanchao Yang},
    year={2024},
    eprint={2407.12661},
    archivePrefix={arXiv},
    primaryClass={cs.CV},
    url={https://arxiv.org/abs/2407.12661}, 
}