A Convolutional Neural Deferred Shader for Physics Based Rendering

We introduces pbnds+: a novel physics-based neural deferred shading pipeline utilizing convolution neural networks to decrease the parameters and improve the performance in shading and relighting tasks; Energy regularization is also proposed to restrict the model reflection during dark illumination.

Our convolutional neural deferred shader overcomes drawbacks of the existing work in computation complexity and speed. And the proposed energy regularization enhances the rendering performance under dark scenario.

We quantify the shading performance through comparing the common used metrics (PSNR, SSIM, LPIPS, FID). Extensive experiments demonstrate that our approach outperforms classical baselines, a state-of-the-art neural shading model, and a diffusion-based method.

The neural shading process: Image rendering models the interaction between light and surfaces, aiming to solve the rendering equation:

In this task, our goal is to use a convolutional neural network (CNN) to regress output of the rendering equation as the following process:

To make the neural rendering process be compatible with convolution neural network, we define the following operations to implement shading through convolution calculation. The input tensor is designed as a 5 dimentional tensor consisting of batch, sampled lights, feature channels, height and width dimenstions. A convolution neural network would regress the shading input to the coresponding color reuslt among each pixel. The batch and light dimenstion would be combined for the compatible with the convolution operations.

Following table exhibites the quantitative reuslts of our approach. And the comparison with other state-of-the-art methods.

Following examples exhibite the shading reuslts of our approach. And the comparison with other state-of-the-art methods.

Following examples exhibite the relighting reuslts of our approach. And the comparison with other state-of-the-art methods.