ACM SIGGRAPH 2008: Lighting and material of Halo 3 (Hao Chen)

Lighting and
Material of Halo 3
Hao Chen [email protected]
Xinguo Liu [email protected]
Microsoft Research Asia/Zhejiang University
Talk Overview
• Introduction
• Halo3 Lighting
• Halo3 Material Model
• Results
• Conclusions
• Future Work
Motivation
• global illumination
• handle variety of environments
• consistent lighting everywhere
• render bump maps “correctly”
• complex materials under complex lighting
• HDR
Related Work
• Irradiance Volume [GSHG98][Oat05]
• SH Irradiance Environment Map [RamamoorthiHanrahan01]
• Pre-computed Radiance Transfer [SKS02]
• SH Light Maps [GoodTaylor05]
• Sky and Atmosphere [PSS99][HoffmanPreetham02]
• Reflectance Models [CookTorrance81][Schlick94]
• Low Frequency Glossy Material [KSS02][SHHS03]
• Frequency Space Environment Map [RamamoorthiHanrahan02]
Rendering Equation
I (V )  
f
(
V
,
L
)

(

)
cos(

)
d


BRDF
Incident angle
Distant Lighting
Spherical Harmonics
• A complete set of orthogonal basis for L2 functions
on a unit sphere
m

Y
 ,   
 2  1   m P m
4
 m

 cos   eim
Orthogonality:
2

 
0
0
Ym ( ,  )Ym' '* ( ,  )sin  d d    ' mm '
Spherical Harmonics
• Real Spherical Harmonics
ym  ,  
 2 Re Ym  ,    , m  0




m
0
y  ,    
Y  ,  
, m0

m
, m0


2
Im
Y

,







Spherical Harmonics
• SH expansion
f  ,     f y  ,  
m m


 ,m
where
f 
m


 0
2

0
f  ,   y  ,   sin  d d
m

Spherical Harmonics
• SH Rotation by euler angles:  ,  , 









f '00 

1 
f '1 

0 
f '1  RSH ( ,  ,  ) 


1
f '1 


 

f 00 
1 
f1 
0 
f1

1
f1 
 
RSH ( ,  ,  )  Z X90 Z  X90 Z
[Green03]
Spherical Harmonics
• Order-3 Real
Spherical Harmonics
y22
15
2 
xy
0
0
y
y11
 3
y
2 
y10
y21
 15
yz
2 
y20
[Sloan08]
1
2 
3
2 
5
4 
3z
2
z

1
y11
 3
x
2 
y12
 15
xz
2 
y22
15
4 
x
2
 y2

Spherical Harmonics
• Order-3 Real SH
– Rotation
1






XZ90
X

90








00c 110
101 001
00s 000
00s
000
11c
0c 2 0 0 0 0 11
0 0 1 c 0 0 00
0 0 0 0  12 1 00
0 0 0  s 0 0 00
11 s2 0 0  233 2300
c  cos( ) s  sin( )
c2  cos(2 ) s2  sin(2 )






0 00s2

s 00 0

0 23230

c 00 0

11 
0 22c2
Spherical Harmonics
• A fast rotation code for pixel shader
/*
rotation [ r00 r01 r02 ] -> x-tagent of the local frame
matrix R = [ r10 r11 r12 ] -> y-tagent of the local frame
[ r20 r21 r22 ] -> z/normal of the local frame
r[] = { r00, r01, r02, r10, r11, r12, r20, r21, r22 };
*/
void sh_rot( const double r[], const double pIn[9], double pOut[9] )
{
// DC
pOut[0] = pIn[0];
// Linear
pOut[1] = r[3]*pIn[3] + r[4]*pIn[1] + r[5]*(-pIn[2]) ;
pOut[2] = -(r[6]*pIn[3] + r[7]*pIn[1] + r[8]*(-pIn[2]));
pOut[3] = r[0]*pIn[3] + r[1]*pIn[1] + r[2]*(-pIn[2]) ;
// Quadratics
pOut[4] = -(
( r[0]*r[4] + r[1]*r[3] ) * ( -pIn[4] ) +
( r[1]*r[5] + r[2]*r[4] ) * ( pIn[5] ) +
( r[2]*r[3] + r[0]*r[5] ) * ( pIn[7] ) +
( r[0]*r[3]
)*(
-pIn[8] ) +
( r[1]*r[4]
)*(
pIn[8] ) +
( r[2]*r[5]
) * ( -v173*pIn[6] ) );
pOut[5] =
( r[3]*r[7] + r[4]*r[6] ) * ( -pIn[4] ) +
( r[4]*r[8] + r[5]*r[7] ) * ( pIn[5] ) +
( r[5]*r[6] + r[3]*r[8] ) * ( pIn[7] ) +
( r[3]*r[6]
)*(
-pIn[8] ) +
( r[4]*r[7]
)*(
pIn[8] ) +
( r[5]*r[8]
) * ( -v173*pIn[6] ) ;
pOut[7] =
( r[0]*r[7] + r[1]*r[6] ) * ( -pIn[4] ) +
( r[1]*r[8] + r[2]*r[7] ) * ( pIn[5] ) +
( r[2]*r[6] + r[0]*r[8] ) * ( pIn[7] ) +
( r[0]*r[6]
)*(
-pIn[8] ) +
( r[1]*r[7]
)*(
pIn[8] ) +
( r[2]*r[8]
) * ( -v173*pIn[6] ) ;
pOut[6] =-v173*(( r[7]*r[6]
) * ( -pIn[4] ) +
( r[8]*r[7]
) * ( pIn[5] ) +
( r[6]*r[8]
) * ( pIn[7] ) +
0.5f*( r[6]*r[6] ) * (
-pIn[8]) +
0.5f*( r[7]*r[7] ) * (
pIn[8]) +
0.5f*( r[8]*r[8] ) * ( -v173*pIn[6]) +
0.5f*( 1.f/3.f ) * ( v173*pIn[6]) );
pOut[8] = -(
( r[1]*r[0] - r[4]*r[3] ) * ( -pIn[4] ) +
( r[2]*r[1] - r[5]*r[4] ) * ( pIn[5] ) +
( r[0]*r[2] - r[3]*r[5] ) * ( pIn[7] ) +
0.5f*( r[0]*r[0] - r[3]*r[3] ) * (
-pIn[8] ) +
0.5f*( r[1]*r[1] - r[4]*r[4] ) * (
pIn[8] ) +
0.5f*( r[2]*r[2] - r[5]*r[5] ) * ( -v173*pIn[6] ) );
}
SH Irradiance Env Map
[Ramamoorthi00]
Llm   L( ,  )Ylm ( ,  ) sin( )dd
 ,
diff solid
distant radiance
SH at
basis
given
evaluated at given
angle
direction
direction
Irradiance Volumes
[GSHG98]
• Spatially divide volume into cells.
• irradiance volume per cell.
• Interpolate between samples.
• Sharp shadow boundaries?
• Bump maps?
[Oat05]
Spherical Harmonics Light Map
• Parameterize over
geometry surface.
• Each texel is a SH
Vector.
• 9 textures for
quadratic SH.
• Highly compressed.
Halo3 Lighting Pipeline
Parameterize
Rendering
GI Solver
Compression
Parameterization
• UVAtlas (MSRA)
• Small Charts
• Long and thin charts
• > 80% utilization.
• Vastly improved over
Halo2.
Photon Mapping
Initialization
Direct
Illumination
Photon
Cast
Final
Gather
Exit
Illumination
Radiance
Estimate
Signal
Compression
DXT
Compression
Compression
• Two Stage Process.
• Signal Based Optimization.
– Sub-chart optimization.
• DXT HDR compression.
– Use 2 DXT5 to compress each floating point
texture.
Signal Based Optimization
• Compute gradient of
pixels in log space.
• Resize charts based
on gradients.
Signal Optimization Result
Before: 1024x1024
After: 512x512
Subchart Optimization
• Charts with only a few
high freq pixels can
still get large area.
• Solution: cut out the
high frequency area
into a separate chart.
DXT HDR Compression
• Use two DXT5 textures to compress the
SH coefficients (HDR, positive/negative)
4 x 4 block:
DXT[0]:
Alpha (Luminance) /
64 bits
RGB / 64 bits
DXT[1]:
Alpha (Luminance) /
64 bits
RGB / 64 bits
“Luvw” Color Space
• Each SH coefficient is a RGB vector and is
converted to Luvw color space.
• L: magnitude of the RGB vector.
– Non negative.
– Stored in DXT5 x 2’s alpha channels for higher precision.
• uvw: normalized vector.
– Good coherence, we store in the rgb channels of DXT5.
Luminance
• Store square root of L for higher dynamic range.
• Similar to log space, but cheaper to decode.
1.2
1
0.8
0.6
0.4
0.2
0
log
square root
Luminance Compression
• Use 2nd DXT alpha to compensate for the error.
Alpha channel of a 4 x 4 block
DXT[0]:
Block[0] = DXT_compress ( sqrt ( L_block ) )
DXT[1]:
Block[1] = DXT_compress ( L_block /
Decompress ( Block[0] ) ) )
Luminance Decoding
• L= Alpha0 * Alpha1 * Max_Luminance
2 DXT 5 alpha blocks
Just 1 alpha block
UVW compression
• Similar, use 2nd DXT5 to minimize error.
RGB channel of a 4 x 4 block
DXT[0]: Block[0] = DXT_compress ( RGB_block / 2 +0.5 )
DXT[1]:
Block[1] = DXT_compress ( ( RGB_block –
Decompress ( Block[0] ) ) / 2 + 0.5 )
• Decode: UVW= (rgb0 + rgb1) * 2 - 2
Compression Summary
• Signal compression re-packs charts based on
signal frequency. (4: 1 compression ratio).
– Sub-chart optimization break up charts if desired.
• DXT HDR compression compresses the raw
floating point value. (3: 1 compression ratio)
• Overall 12 : 1 compression ratio, quality loss is
perceivably small.
Rendering Diffuse Lighting
• Static Geometry.
– SH light map is bound as a surface textures.
– Per pixel: evaluate normal with the SH vector.
– Exactly like in Spherical Harmonics Irradiance Env Map.
• Dynamic Objects.
– Sample the SH light maps based on object position.
– Render object using PRT and the SH vector.
Diffuse Lighting Examples
Optimization
• Reduce storage and ALU count.
• Pull out dominant light, store intensity.
E
 (  cY (d )) , E
2
i  0 ,... 8
c
i
i
'
0
 ( Y (d )) /  Y (d )
i  0 ,...8
i i
i  0 ,...8
2
i
• Store linear SH instead of quadratic.
• In shader, do (N * L + sh_eval(sh[] – c * Y(d), N).
Comparing “SH 2.5” and SH 3
SH quadratic
N . L + SH linear
Material Motivation
• BRDF expressiveness.
• Real time performance.
• Compatible with Halo 3 lighting model.
• Requires low storage.
Halo3 Material Model
• Separate reflectance into separate, low to high
frequency parts.
– Diffuse.
– Low frequency glossy (area specular).
– Mid frequency glossy (environment map).
– High frequency glossy (analytical specular).
• Handle each with different techniques.
Cook Torrance BRDF
f (V , L)  kd Rd  k s  F  Rm
view direction
light direction diffuse
diff & specular
lobe
Fresnel
scaler specular lobe
DG
Rm(V , L) 
 ( N  L)( N V )
D: microfacet distribution function
G: geomery term
[CookTorrance81]
Cook Torrance BRDF
0.5
He
Cook-Torrance
Lafortune
Ward
Blinn-Phong
log(Err/Max Albedo)
0
-0.5
-1
-1.5
[Ngan05]
-2
10
20
30
40
50
Material
60
70
80
90
100
Analytical Specular
• Evaluate BRDF directly
in pixel shader (point
light)
• Use the dominant light
direction and intensity.
• HLSL listing in paper.
Environment Map
• Render cube map at
discrete locations.
• Filter out high
frequency.
• Pre-divide by area
specular.
• Multiply back in shader.
Area Specular
• Main idea:
– Express BRDF model
itself in SH.
– Pre-integrate key terms
into 2D textures.
– Evaluate BRDF in
shader with SH light.
– Low frequency only.
Light Integration
I (V )   f (V , L)cos ( ) L( )d
k d Rd  cos ( ) L( )d  k s  FRm (V , L)cos ( ) L( )d
??
SH irradiance env. map
Light Integration in SH
I s (V )  k s  FRm (V , L)cos ( ) L( )d
8
L( )   iYi ( )
Project light into SH basis.
i 0
Project BRDF and cosine term in SH basis
F
Bm,i (V )   Rm (V , L)cos ( )Yi ( )d
F0
8
I s (V )  K s F0  i Bm,i (V )
i 0
Dot product to convolve
Light Integration in SH Cont.
F  F0  (1  F0 )(1  ( L  H ))5
[Schilick94]
F0  (1  F0 )(1  ( L  H ))5
Bm,i (V )  
Rm (V , L)cos ( )Yi ( )d
F0
Cm,i (V )   Rm (V , L)cos ( )Yi ( )d
Dm,i (V )   (1  ( L  H )5 ) Rm (V , L)cos ( )Yi ( )d
Bm,i (V )  Cm,i (V ) 
1  F0
Dm,i (V )
F0
Preintegration
Pre-integration
N
Z
Z/N
V
Reflective symmetry means:
Y’
V’
Y
X
X’
Cm,i (V )  Dm,i (V )  0, i  1,4,5.
Isotropic BRDF = any coordinate frame 16 m values, and 8 V directions is enough.
C (i=0,2,3,6)
D (i=0,2,3,6)
C,D (i=7,8)
Rendering Area Specular
•
Build a local frame.
•
Look up C and D texture.
•
Rotate SH light vector into local frame.
•
Do SH Dot product.
•
HLSL Listing in course note.
Results
Conclusions
• SH light map is a natural extension to the traditional lightmapping pipeline.
• Separating material into layers is a good approximation for
all frequency reflectance.
• Area specular is critical for achieving seamless lighting
and material integration.
• ALU is cheap, and will get cheaper, take maximum
advantage of it.
Future Work
• Global Illumination with local, moving lights.
• GI for dynamic and semi-dynamic scenes.
• Better lighting basis (less ringing, higher frequency).
• Area specular model with complex transport.
• Measured BRDF.
• Non photo-realistic rendering.
Acknowledgement
• Authors
– Bungie: Hao Chen, Ben Wallace, Chris Tchou, David
Cook, Xi Wang, Yaohua Hu.
– MSRA: Xinguo Liu, Zhipeng Hu, Xin Huang, Minmin
Gong, Kun Zhou.
• Special Thanks
– Bungie artists.
– Peter Pike Sloan, Baining Guo, Harry Shum
– Kutta Srinivasan, Matt Lee, Mikey Wetzel.
Want to team up with the Master Chief?
Bungie is hiring.
http://www.bungie.net/jobs