NeuralHumanFVV: Real-Time Neural Volumetric Human Performance Rendering
using RGB Cameras

CVPR 2021

4D reconstruction and rendering of human activities using 6 rgb cameras.

Xin Suo, Yuheng Jiang, Pei Lin, Yingliang Zhang,
Minye Wu, Kaiwen Guo, Lan Xu


4D reconstruction and rendering of human activities is critical for immersive VR/AR experience. Recent advances still fail to recover fine geometry and texture results with the level of detail present in the input images from sparse multi-view RGB cameras. In this paper, we propose NeuralHumanFVV, a real-time neural human performance capture and rendering system to generate both high-quality geometry and photo-realistic texture of human activities in arbitrary novel views. We propose a neural geometry generation scheme with a hierarchical sampling strategy for real-time implicit geometry inference, as well as a novel neural blending scheme to generate high resolution (e.g., 1k) and photo-realistic texture results in the novel views. Furthermore, we adopt neural normal blending to enhance geometry details and formulate our neural geometry and texture rendering into a multi-task learning framework. Extensive experiments demonstrate the effectiveness of our approach to achieve high-quality geometry and photo-realistic free view-point reconstruction for challenging human performances.


@inproceedings{suo2021neuralhumanfvv, title={NeuralHumanFVV: Real-time neural volumetric human performance rendering using RGB cameras}, author={Suo, Xin and Jiang, Yuheng and Lin, Pei and Zhang, Yingliang and Wu, Minye and Guo, Kaiwen and Xu, Lan}, booktitle={Proceedings of the IEEE/CVF conference on computer vision and pattern recognition}, pages={6226--6237}, year={2021} }