Efficient Rendering of Layered Materials
using an Atomic Decomposition with Statistical Operators

Laurent Belcour

Last Year ...

Rendered in Unity

Last Year ...

Rendering thin-film iridescence
- Using a clear-coat plugin in Mitsuba
- But no clear-coat available in Unity 😭
I wanted to show the beetle live
- One solution: code one in time!
Turn out we can do much more
- Multiple rough interfaces
- Energy conservation
- ...

Rendered in Mitsuba

With this car shot, I wanted to reproduce this Mitsuba rendering from last year's paper but in real-time. click This rendering used the clear-coat plugin on top of the iridescent one. The idea behind the clear-coat plugin is quite simple. The view ray is bent with respect to Snell's law and is used to sample the base BSDF. All those reflected rays are then bent to get out of the clear-coat. While this fails to account for multiple scattering in the clear-coat, it correctly follows light transport. click However, getting a physically based clear-coat in a real-time renderer is much more challenging. There, because IBL and area-lights are preintegrated so you cannot bent ray like mitsuba does. So people do a lot of hacks that are ... what they are. While they are efficient, they seem to take an different definition of physically based rendering. click

That didn't stop me from trying to make it. Only, I had to build it from the ground up and in time. click

But while doing it, I wanted to go further than a Mitsuba-like smooth clear-coat. I felt that an energy preserving rough coating was at reach. And fortunately, I could go even further than what I expected. And this model proved to work as well for offline rendering. So I guess that a nice first take home message would be that the constraints of real-time rendering can lead you to very interesting solutions, even for offline.

Layered Materials

Layered Materials

$\boldsymbol{\omega}_i$

$\boldsymbol{\omega}_o$

In this talk, we restrict layered materials to be made of a stack of rough dieletric or conducting interfaces *click* that are parallel and planar at the macroscopic scale. *click* The number of interfaces is however arbitrary. *click*

We also assume that the material is viewed from sufficiently far away such that light enter and exits the layered structure approximately at the same position. However, for clarity, the illustrations will depict different outgoing position. Keep in mind that those are not accounted in the end. *click*

The challenge here is the following: Given an incident beam of light, we want to describe light transport in the layered structure *click* for any number of scattering events *click*. Note that those interaction can be fairly complicated *click*.

On issue is that *click* light can bounce many times between two interfaces or can even *click* bounces between many interfaces before reaching the outside.

Layered Materials

Brute-force rendering is not possible
- At least not in real-time graphics
Precomputation is not statisfactory
- Forbid to use textured assets
- Memory limitation on GPUs
Our solution: tight approximation

LayerLab data - 1.7GiB [Jakob 2014]

Our Idea: Sum of GGX Lobes

$\mathbf{\omega}_i$

$\mathbf{n}$

$$\rho(\mathbf{\omega}_i, \mathbf{\omega}_o) = \sum_{k} w_k \, \rho_k(\mathbf{\omega}_i, \mathbf{\omega}_o)$$

Our model works as follow: starting from an incident direction, we virtually group together paths that interact with a specific interface. For example, *click* we group together directly reflected paths, *click* or paths that interact with the second interface but not below, and finally in this case *click* all the paths that interact a least once with the bottom interface. *click*

However, we do not explicitly compute those paths and instead work with statistical summary of light transport. In the end, each group of paths is described by its zero, first and second order statistics. *click*

From each set of statistics, we instantiate a GGX microfacet model that approximate those statistics. In this talk, I will refer to those instantiated models as GGX lobes. *click* And our approximate BRDF *click* consists in the sum of those models.

Statistical Analysis: Mapping

We study BSDF statistics
- In the orthographicaly projected disc
- There, GGX is almost rotationaly symmetric

Orthographic projection
GGX lobe with $\alpha = 0.01$

Statistical Analysis: Mapping

We study BSDF statistics
- In the orthographicaly projected disc
- There, GGX is almost rotationaly symmetric
To find a mapping
- From the three moments (energy, mean, variance)
- To a BRDF lobe parameters (albedo, view, roughness)

Orthographic projection
Equivalent Statistics

$\mathbf{\omega}_i$

$(e, \mathbf{\mu}, \sigma)$

Statistical Analysis: Mapping

We study BSDF statistics
- In the orthographicaly projected disc
- There, GGX is almost rotationaly symmetric
To find a mapping
- From the three moments (energy, mean, variance)
- To a BRDF lobe parameters (albedo, view, roughness)
Can we find the statistics of layered materials?

Layered configuration

Statistical Analysis: Framework

Infer statistics atomically
- Details in the paper
- Update $e$, $\mu$, and $\sigma$
Example: refraction operator
- Shift, scales and convolves the incident lobe

$$ e_t = \tilde{\mbox{F}} \, e_i $$

$$ \mu_t = - \eta_{12} \, \mu_i $$

$$ \sigma_t = \eta_{12} \, \color{blue}{\sigma_i} + \color{red}{s} $$

Reflection

Refraction

Scattering

Now we have to answer the following question: what are the statistics of light transport in a layered structure? To answer this question, we can build a statistical framework that allows to infer the statistics of radiance after atomic operations such as surface reflection, surface refraction and medium scattering. Those operators are fully detailed in the paper. In this talk, it is sufficient to know that each operator update the statistics of the incident radiance. *click*

Let's take the example of light refracted by a dielectric surface. *click*

The refraction ... *click*

scales the energy due to the average Fresnel term in the incident directional footprint. *click*

shift the mean with respect to the index of refraction ratio. This is a simple translation of Snell's law to the directional statistics. *click*

Another impact is that the incident variance (here in blue) is scaled by the change of interface and increased by a term (here in red) that depends on the surface's roughness.

I encourage you to read the paper for more details on those operators.

Statistical Analysis: Validation

Interactively testing atomic operators

incident radiance

transmitted radiance

Statistical Analysis: Framework

Multiple layers: chain operators

Now that we can summarize light transport at every interface, we need to be able to express the transport for specific groups of light paths. *click*

Lets take the following example of a group of light paths refractive two times before being reflected by the bottom interface and leaving the layered structure after two more refractions. *click*

In our framework, we use the central path to estimate the statistics. *click* This central path is constructed by converting all interfaces as smooth ones. Thus it correspond to a purely specular path. *click*

We start at the first vertex of this path and set the statistics to match a dirac in the incident direction. *click*

Then, we apply the first operator which in this case is a refraction. *click* This will specify the statistics of the outgoing radiance. *click* We convert those statistics into incident statistics by mirroring the statistics. and proceed with the next interface. *click* We continue this process for the remaining vertices of the path until the outgoing direction is reached. *click until end* Note that this example is here to give the intuition of how operators chain together, there exist an infinite number of specular paths and in practice we can not explicitly evaluate them all. Instead, we rely on three mechanisms for efficient evaluation: merging of statistics, analytic form for multiple scattering, and adding-doubling.

Statistical Analysis: Framework

Merging statistics

$(e_1, \mu_1, \sigma_1)$

$(e, \mu, \sigma)$

$(e, \mu, \sigma) = \left(e_1+e_2, \mu, \dfrac{e_1}{e}\sigma_1+\dfrac{e_2}{e}\sigma_2\right)$

$(e_2, \mu_2, \sigma_2)$

$+$

Statistical Analysis: Framework

Merging statistics
Multiple scattering
- Closed-form statistics
- Arithmetico-Geometric series

Statistical Analysis: Framework

Merging statistics
Multiple scattering
- Closed-form statistics
- Arithmetico-Geometric series

Adding-Doubling
- Adding-Doubling on variance [van de Hulst 1962]
- Average upper interaction as well

We associate this finding with a new adding-doubling method. The idea behind the adding-doubling method is to estimate the scattering statistics of a stack of layers by iteratively applying the multiple scattering formula on pairs of layers.

I will illustrate here how it works. Let say we want to evaluate the BRDF of the layered structure. Given the incident direction, *click* we start by computing the reflected and transmitted directional statistics due to the first interface and record the reflected statistics as our first BRDF lobe. *click* Next, we evaluate the multiple scattering between the first two interfaces and record the reflected statistics as our second BRDF lobe. *click* Next, we virtually merge these two interfaces of this layer into a single interface. This virtual interface response to an incident beam is defined by the reflected and transmitted statistics. *click* Finally, we evaluate the multiple scattering equation using this new layer and record the reflected statistics as the third lobe. Note that the adding-doubling will account for all the multiple scattering happening between the top layers and the current layer considered.

Offline Validation

Mitsuba renderer
- Both opaque and transparent plugins
- Varying number of textured layers
- Multiple Importance Sampling with the lobes

Offline Validation

Mitsuba renderer
Comparison with stochastic reference

Ours Reference	Ours Reference	Ours Reference
Metal foil	Rough metal	Gold Coated

Offline Validation

Mitsuba renderer
Comparison with stochastic reference
Layered method of Weidlich & Wilkie [2007]

Ours

Reference

[WW07]

Reference

Offline Validation

Mitsuba renderer
Comparison with stochastic reference
Layered method of Weidlich & Wilkie [2007]
Multiple scattering


$R + TRT$	$R + TR^+T$	Ours

Offline Results: Textures


Textured base $\alpha$	Textured top $\eta$	Textured top $\alpha$

Offline Results: Robot Bust

Two layer configuration
Multiple textured layers
- Base and top Index of Refraction
- Top roughness

Offline Results: Robot Bust

Offline Results: Dragon

Gold metal dragon
Adding a medium layer
- Simulate dust
- Increase the haze

Real-Time Results in Unity

Limitations: High Roughnesses

$\alpha = 0.3$

Ours

Reference

$\alpha = 0.6$

Ours

Reference

$\alpha = 0.9$

Ours

Reference

Summary

A novel BSDF model for layered structures
- Accurate for low roughnesses
- Accounts from multiple scattering
- No parameter dependent precomputation
- Compatible with real-time scenario
Our contributions
- Statistical analysis of light transport in layers
- New adding-doubling scheme for variance

Thank you for your attention


paper	supp. mat.	code	HDRP StackLit

available at belcour.github.io/blog

Efficient Rendering of Layered Materials using an Atomic Decomposition with Statistical Operators

Last Year ...

Last Year ...

Layered Materials

Layered Materials

Layered Materials

Our Idea: Sum of GGX Lobes

Statistical Analysis: Mapping

Statistical Analysis: Mapping

Statistical Analysis: Mapping

Statistical Analysis: Framework

Statistical Analysis: Validation

Statistical Analysis: Framework

Statistical Analysis: Framework

Statistical Analysis: Framework

Statistical Analysis: Framework

Offline Validation

Offline Validation

Offline Validation

Offline Validation

Offline Results: Textures

Offline Results: Robot Bust

Offline Results: Robot Bust

Offline Results: Dragon

Real-Time Results in Unity

Limitations: High Roughnesses

Summary

Thank you for your attention

Efficient Rendering of Layered Materials
using an Atomic Decomposition with Statistical Operators