Reflectance Transfer for Material Editing and Relighting
We present a new approach to diffuse reflectance estimation
for dynamic scenes.
Non-parametric image statistics are used to transfer
reflectance properties from a static example set to a
dynamic image sequence.
The approach allows diffuse reflectance estimation
for surface materials with inhomogeneous appearance,
such as those which commonly occur with patterned or
textured clothing. Material editing is also possible by
transferring edited reflectance properties.
Material reflectance properties are initially estimated
from static images of the subject under multiple
directional illuminations using photometric stereo.
The estimated reflectance together with the corresponding
image under uniform ambient illumination
form a prior set of reference material observations.
Material reflectance properties are then estimated
for video sequences of a moving person captured under
uniform ambient illumination by matching the
observed local image statistics to the reference observations.
Results demonstrate that the transfer of
reflectance properties enables estimation of the dynamic
surface normals and subsequent relighting combined
with material editing. This approach overcomes
limitations of previous work on material transfer
and relighting of dynamic scenes which was limited
to surfaces with regions of homogeneous reflectance.
We evaluate our approach for relighting
3D model sequences reconstructed from multiple
view video. Comparison to previous model relighting
demonstrates improved reproduction of detailed texture
and shape dynamics.
Shape From Shading
JVRB - Journal of Virtual Reality and Broadcasting