Illustrative Rendering in Team Fortress 2 Jason Mitchell Moby Francke Dhabih Eng Outline • • • • Motivations and related work Environments Characters and interactive shading Future work Team Fortress 2 • Class-based multiplayer combat game which will be released this fall • Unique visual style • Differentiation - multiplayer combat games tend to embrace a contemporary photorealistic look • Gameplay -Team Fortress has always featured cartoonish, over-the-top situations • Readability - Class differentiation is the core of Team Fortress 2, hence we needed to be able to clearly differentiate classes visually Environment Design Principles • Value contrast • Simple forms • • No unnecessarily off-kilter shapes Minimize visual noise • Texture and geometric • Minimize repetition Contrasting Team Properties • Red • • Warm colors • Natural materials • Angular geometry Blue • Cool colors • Industrial materials • Orthogonal forms Blue base in 2fort map Red base in 2fort map World Rendering • Photorealistic techniques from our other games • Radiosity-generated light maps • Special effects such as reflection and refraction • Hand-painted textures with minimal noise, applied directly to 3D geometry • Loose details with visible brush strokes • Inherent solidity and frame-to-frame coherence • Hold up under magnification better than photoreference • Brush strokes appear in perspective, not in the 2D image plane [Miyazaki02] • High frequency detail in photorealistic games can overpower design Color Palette Miyazaki – Brush Width Foreshortened • Can easily imagine a 3D camera move between these 2D views of the same space Background plates from Spirited Away Neutral Entities • Variations in hue and saturation are used to differentiate neutral entities in the game world • A hue other than red or blue creates disassociation from either team color • Increased saturation makes these important entities stand out in the desaturated environment • Equally beneficial or dangerous to either team • Beneficial green / cyan health pickups • Dangerous yellow train yard gates Character Design Goals • Easily visible against environment • Characters must be readable quickly by other players • Communicate shape via shading and silhouette under all lighting conditions Gooch, 1998 • Hue and luminance shifts indicate surface Conventional Gooch Shading Shading orientation relative to light • Blend between warm and cool based upon unclamped Lambertian term, underlying albedo and some free parameters • Extreme lights and darks are reserved for edge lines and highlights ( ) ( ) ⎛ ⎛1 1 ⎞⎞ 1⎞ ⎛1 ˆ ˆ ⎜ nˆ ⋅ l + ⎟(kblue + αk d ) + ⎜⎜1 − ⎜ nˆ ⋅ l + ⎟ ⎟⎟(k yellow + β k d ) 2 ⎠⎠ 2⎠ ⎝2 ⎝ ⎝2 Lake, 2000 • Lake used a 1D texture lookup based upon the Lambertian term to simulate the limited color palette cartoonists use for painting cels • Also allows for the inclusion of a view-independent pseudo specular highlight by including a small number of bright texels at the “lit” end of the 1D texture map N·L Barla, 2006 • Barla has extended this technique by using a 2D texture lookup to incorporate view-dependent and level-of-detail effects. • Fresnel-like creates a hard “virtual backlight” which is essentially a rim-lighting term, though this term is not designed to correspond to any particular lighting environment. |N·V|r N·L |N·V|r N·L Early 20th Century Commercial Illustration • Chose to adopt specific conventions of the commercial illustrator J. C. Leyendecker: • Shading obeys a warm-to-cool hue shift. Shadows go to cool, not black • Saturation increases at the terminator with respect to a given light source. The terminator is often reddened. • On characters, interior details such as clothing folds are chosen to echo silhouette shapes • Silhouettes are often emphasized with rim highlights rather than dark outlines Rim Highlights Red Terminator J.C. Leyendecker Arrow collar advertisement, 1929 J.C. Leyendecker Swimmin’ Hole, 1935 Clothing Folds J.C. Leyendecker Thanksgiving 1628-1928 J.C. Leyendecker Tally-Ho, 1930 Engineer Concept Rim Highlighting: Before Rim Highlighting: After • Players must be able to quickly identify other players by team, class and selected weapon at a variety of distances and viewpoints • We think of this in terms of a visual “read hierarchy” • Design Goals • Team – Friend or Foe? • Color • Class – Run or Attack? • Distinctive silhouettes • Body proportions • Weapons • Shoes, hats and clothing folds • Selected weapon – What’s he packin’? • Highest contrast at chest level, where weapon is held • Gradient from dark feet to light chest Color Palette Character Lighting Equation View independent (( ) ) L γ ⎤ ⎡ ⎛ k d ⎢a(nˆ ) + ∑ci w⎜ α nˆ ⋅ lˆi + β ⎞⎟⎥ + ⎝ ⎠⎦ i =1 ⎣ ∑[c k max( f (vˆ ⋅ rˆ ) L i s s i k spec , f r k r (vˆ ⋅ rˆi ) k rim )]+ (nˆ ⋅ uˆ ) f k a(vˆ) i =1 View-dependent r r View Independent Terms (( ) ) L γ ⎤ ⎡ ⎛ k d ⎢a(nˆ ) + ∑ci w⎜ α nˆ ⋅ lˆi + β ⎞⎟⎥ ⎝ ⎠⎦ i =1 ⎣ • Spatially-varying directional ambient View Independent Terms (( ) ) L γ ⎤ ⎡ ⎛ k d ⎢a(nˆ ) + ∑ci w⎜ α nˆ ⋅ lˆi + β ⎞⎟⎥ ⎝ ⎠⎦ i =1 ⎣ • Spatially-varying directional ambient • Modified Lambertian terms View Independent Terms (( ) ) L γ ⎤ ⎡ ⎛ k d ⎢a(nˆ ) + ∑ci w⎜ α nˆ ⋅ lˆi + β ⎞⎟⎥ ⎝ ⎠⎦ i =1 ⎣ • Spatially-varying directional ambient • Modified Lambertian terms • Unclamped Lambertian term View Independent Terms (( ) ) L γ ⎤ ⎡ ⎛ k d ⎢a(nˆ ) + ∑ci w⎜ α nˆ ⋅ lˆi + β ⎞⎟⎥ ⎝ ⎠⎦ i =1 ⎣ • Spatially-varying directional ambient • Modified Lambertian terms • Unclamped Lambertian term • Scale, bias and exponent View Independent Terms (( ) ) L γ ⎤ ⎡ ⎛ k d ⎢a(nˆ ) + ∑ci w⎜ α nˆ ⋅ lˆi + β ⎞⎟⎥ ⎝ ⎠⎦ i =1 ⎣ • Spatially-varying directional ambient • Modified Lambertian terms • Unclamped Lambertian term • Scale, bias and exponent • Warping function • Albedo ( ) 1 1 nˆ ⋅lˆ + 2 2 View Independent Terms (( ) ) L γ ⎤ ⎡ ⎛ k d ⎢a(nˆ ) + ∑ci w⎜ α nˆ ⋅ lˆi + β ⎞⎟⎥ ⎝ ⎠⎦ i =1 ⎣ • Spatially-varying directional ambient • Modified Lambertian terms • Unclamped Lambertian term • Scale, bias and exponent • Warping function View Independent Terms (( ) ) L γ ⎤ ⎡ ⎛ k d ⎢a(nˆ ) + ∑ci w⎜ α nˆ ⋅ lˆi + β ⎞⎟⎥ ⎝ ⎠⎦ i =1 ⎣ • Spatially-varying directional ambient • Modified Lambertian terms • Unclamped Lambertian term • Scale, bias and exponent • Warping function • Albedo View Independent Terms (( ) ) L γ ⎤ ⎡ ⎛ k d ⎢a(nˆ ) + ∑ci w⎜ α nˆ ⋅ lˆi + β ⎞⎟⎥ ⎝ ⎠⎦ i =1 ⎣ • Spatially-varying directional ambient • Modified Lambertian terms • Unclamped Lambertian term • Scale, bias and exponent • Warping function • Albedo Ambient Cube • Grounds characters in game worlds • Pre-compute irradiance samples • • • • throughout the environment Variable density irradiance volume [Greger98] where each sample defines an irradiance environment map [Ramamoorthi01] Directional ambient term which includes only indirect light Lights beyond the first four can be added to the ambient cube Used in a novel way in rim lighting, which we’ll discuss in a moment Directional Ambient Constant Ambient View-dependent Terms ∑[c k max( f (vˆ ⋅ rˆ ) L i s i =1 s i k spec , f r k r (vˆ ⋅ rˆi ) k rim )]+ (nˆ ⋅ uˆ ) f k a(vˆ) r r View-dependent Terms ∑[c k max( f (vˆ ⋅ rˆ ) L i s s i k spec i =1 • Multiple Phong terms per light , f r k r (vˆ ⋅ rˆi ) k rim )]+ (nˆ ⋅ uˆ ) f k a(vˆ) r r View-dependent Terms ∑[c k max( f (vˆ ⋅ rˆ ) L i s s i k spec i =1 • Multiple Phong terms per light • krim broad, constant exponent , f r k r (vˆ ⋅ rˆi ) k rim )]+ (nˆ ⋅ uˆ ) f k a(vˆ) r r View-dependent Terms ∑[c k max( f (vˆ ⋅ rˆ ) L i s s i k spec i =1 • Multiple Phong terms per light • krim broad, constant exponent • kspec exponent (constant or texture) , f r k r (vˆ ⋅ rˆi ) k rim )]+ (nˆ ⋅ uˆ ) f k a(vˆ) r r View-dependent Terms ∑[c k max( f (vˆ ⋅ rˆ ) L i s s i k spec i =1 • Multiple Phong terms per light • krim broad, constant exponent • kspec exponent (constant or texture) • fs artist tuned Fresnel term , f r k r (vˆ ⋅ rˆi ) k rim )]+ (nˆ ⋅ uˆ ) f k a(vˆ) r r View-dependent Terms ∑[c k max( f (vˆ ⋅ rˆ ) L i s s i k spec i =1 • Multiple Phong terms per light • krim broad, constant exponent • kspec exponent (constant or texture) • fs artist tuned Fresnel term • fr rim Fresnel term, (1-(n·v))4 , f r k r (vˆ ⋅ rˆi ) k rim )]+ (nˆ ⋅ uˆ ) f k a(vˆ) r r View-dependent Terms ∑[c k max( f (vˆ ⋅ rˆ ) L i s s i k spec i =1 • Multiple Phong terms per light • krim broad, constant exponent • kspec exponent (constant or texture) • fs artist tuned Fresnel term • fr rim Fresnel term, (1-(n·v))4 • kr rim mask texture , f r k r (vˆ ⋅ rˆi ) k rim )]+ (nˆ ⋅ uˆ ) f k a(vˆ) r r View-dependent Terms ∑[c k max( f (vˆ ⋅ rˆ ) L i s s i k spec i =1 • Multiple Phong terms per light • krim broad, constant exponent • kspec exponent (constant or texture) • fs artist tuned Fresnel term • fr rim Fresnel term, (1-(n·v))4 • kr rim mask texture • ks specular mask texture , f r k r (vˆ ⋅ rˆi ) k rim )]+ (nˆ ⋅ uˆ ) f k a(vˆ) r r View-dependent Terms ∑[c k max( f (vˆ ⋅ rˆ ) L i s s i k spec i =1 • Multiple Phong terms per light • krim broad, constant exponent • kspec exponent (constant or texture) • fs artist tuned Fresnel term • fr rim Fresnel term, (1-(n·v))4 • kr rim mask texture • ks specular mask texture , f r k r (vˆ ⋅ rˆi ) k rim )]+ (nˆ ⋅ uˆ ) f k a(vˆ) r r View-dependent Terms ∑[c k max( f (vˆ ⋅ rˆ ) L i s s i k spec i =1 • Multiple Phong terms per light • krim broad, constant exponent • kspec exponent (constant or texture) • fs artist tuned Fresnel term • fr rim Fresnel term, (1-(n·v))4 • kr rim mask texture • ks specular mask texture , f r k r (vˆ ⋅ rˆi ) k rim )]+ (nˆ ⋅ uˆ ) f k a(vˆ) r r View-dependent Terms ∑[c k max( f (vˆ ⋅ rˆ ) L i s s i k spec i =1 • Multiple Phong terms per light • krim broad, constant exponent • kspec exponent (constant or texture) • fs artist tuned Fresnel term • fr rim Fresnel term, (1-(n·v))4 • kr rim mask texture • ks specular mask texture , f r k r (vˆ ⋅ rˆi ) k rim )]+ (nˆ ⋅ uˆ ) f k a(vˆ) r r View-dependent Terms ∑[c k max( f (vˆ ⋅ rˆ ) L i s s i k spec , f r k r (vˆ ⋅ rˆi ) i =1 • • Multiple Phong terms per light • krim broad, constant exponent • kspec exponent (constant or texture) • fs artist tuned Fresnel term • fr rim Fresnel term, (1-(n·v))4 • kr rim mask texture • ks specular mask texture Dedicated rim lighting • a(v) Directional ambient evaluated with v k rim )]+ (nˆ ⋅ uˆ ) f k a(vˆ) r r View-dependent Terms ∑[c k max( f (vˆ ⋅ rˆ ) L i s s i k spec , f r k r (vˆ ⋅ rˆi ) i =1 • • Multiple Phong terms per light • krim broad, constant exponent • kspec exponent (constant or texture) • fs artist tuned Fresnel term • fr rim Fresnel term, (1-(n·v))4 • kr rim mask texture • ks specular mask texture Dedicated rim lighting • a(v) Directional ambient evaluated with v • kr same rim mask k rim )]+ (nˆ ⋅ uˆ ) f k a(vˆ) r r View-dependent Terms ∑[c k max( f (vˆ ⋅ rˆ ) L i s s i k spec , f r k r (vˆ ⋅ rˆi ) i =1 • • Multiple Phong terms per light • krim broad, constant exponent • kspec exponent (constant or texture) • fs artist tuned Fresnel term • fr rim Fresnel term, (1-(n·v))4 • kr rim mask texture • ks specular mask texture Dedicated rim lighting • a(v) Directional ambient evaluated with v • kr same rim mask • fr same rim Fresnel k rim )]+ (nˆ ⋅ uˆ ) f k a(vˆ) r r View-dependent Terms ∑[c k max( f (vˆ ⋅ rˆ ) L i s s i k spec , f r k r (vˆ ⋅ rˆi ) i =1 • • Multiple Phong terms per light • krim broad, constant exponent • kspec exponent (constant or texture) • fs artist tuned Fresnel term • fr rim Fresnel term, (1-(n·v))4 • kr rim mask texture • ks specular mask texture Dedicated rim lighting • a(v) Directional ambient evaluated with v • kr same rim mask • fr same rim Fresnel • n·u term that makes rim highlights tend to come from above (u is up vector) k rim )]+ (nˆ ⋅ uˆ ) f k a(vˆ) r r View-dependent Terms ∑[c k max( f (vˆ ⋅ rˆ ) L i s s i k spec , f r k r (vˆ ⋅ rˆi ) i =1 • • Multiple Phong terms per light • krim broad, constant exponent • kspec exponent (constant or texture) • fs artist tuned Fresnel term • fr rim Fresnel term, (1-(n·v))4 • kr rim mask texture • ks specular mask texture Dedicated rim lighting • a(v) Directional ambient evaluated with v • kr same rim mask • fr same rim Fresnel • n·u term that makes rim highlights tend to come from above (u is up vector) k rim )]+ (nˆ ⋅ uˆ ) f k a(vˆ) r r View-dependent Terms ∑[c k max( f (vˆ ⋅ rˆ ) L i s s i k spec , f r k r (vˆ ⋅ rˆi ) i =1 • • Multiple Phong terms per light • krim broad, constant exponent • kspec exponent (constant or texture) • fs artist tuned Fresnel term • fr rim Fresnel term, (1-(n·v))4 • kr rim mask texture • ks specular mask texture Dedicated rim lighting • a(v) Directional ambient evaluated with v • kr same rim mask • fr same rim Fresnel • n·u term that makes rim highlights tend to come from above (u is up vector) k rim )]+ (nˆ ⋅ uˆ ) f k a(vˆ) r r View-dependent Terms ∑[c k max( f (vˆ ⋅ rˆ ) L i s s i k spec , f r k r (vˆ ⋅ rˆi ) i =1 • • Multiple Phong terms per light • krim broad, constant exponent • kspec exponent (constant or texture) • fs artist tuned Fresnel term • fr rim Fresnel term, (1-(n·v))4 • kr rim mask texture • ks specular mask texture Dedicated rim lighting • a(v) Directional ambient evaluated with v • kr same rim mask • fr same rim Fresnel • n·u term that makes rim highlights tend to come from above (u is up vector) k rim )]+ (nˆ ⋅ uˆ ) f k a(vˆ) r r Future Work • More flexible specular • Anisotropic highlights [Heidrich98] [Gooch98] [Heidrich98] Future Work • More flexible specular • Anisotropic highlights [Heidrich98] [Gooch98] [Gooch98] Future Work • More flexible specular • Anisotropic highlights [Heidrich98] [Gooch98] • Shaping highlights [Anjyo03] [Anjyo03] Future Work • More flexible specular • Anisotropic highlights [Heidrich98] [Gooch98] • Shaping highlights [Anjyo03] • More reliable rim term Future Work • More flexible specular • Anisotropic highlights [Heidrich98] [Gooch98] • Shaping highlights [Anjyo03] • More reliable rim term • Image-space contrast enhancement [Luft06] [Luft06] Future Work • More flexible specular • Anisotropic highlights [Heidrich98] [Gooch98] • Shaping highlights [Anjyo03] • More reliable rim term • Image-space contrast enhancement [Luft06] • Abstracted shadows [DeCoro07] [DeCoro07] Conclusion • Motivations and related work • Environments • Characters and interactive shading • Future work Questions?