Concept Art/Style Guide Stylized Lighting Breakdown

using shader lighting techniques and physical light behavior to develop custom lighting specs

CONTENT

• notes

• process summary

• essential style features

• overall lighting features analysis

• diffuse lighting behavior

• specular lighting behavior

• special lighting behaviors

• final recommendations

Style Guide Reference #1

Style Guide Reference #2

NOTES

• Note - Use of AI Generation

  The way I used Stable Diffusion was as a proxy for an Artist’s output. To that end, I deliberately tried to minimize prompting that overly specified lighting and art style. Instead the focus was on general subject characteristics like features, clothing, equipment, and demographics.

  I did specify, through the use of LORAs, to make some aspects of output physiology and lighting more consistent. That said, this workflow is capable of multiple rendering and lighting styles, but, as an Art Director might, samples were included based on their aesthetic and narrative merit; NOT for ease of development and replication.

• Note - Treatment as organic output

  The output samples were all treated as “okay-ed” concept pieces by an Art Director, and the product of a human artist’s labor. In that spirit, minor details were all treated as intentional and desirable, so an earnest attempt was made to replicate the style as faithfully as possible.

  Where contradictions between pieces were present, I attempted to resolve them in such a way as to assume that both were possible given contextual clues. Where that was not possible, the contradictions were noted to be spec’d into shader specs so that they could both be possible and evaluated once the entire pipeline’s outputs could be judged in aggregate.

• Note - Stable Diffusion

  Stable diffusion (SD), as used for this project, is a still image generation technology. The output of SD is dependent on the model data provided, and the prompts and other parameters that steer SD’s navigation of the model.

  SD models are trained on volumes of data, and it’s outputs will be in the style of the data it was trained on.

  Prompts and other parameters seed the approach on how it traverses its neural network, basically generating a result out of all possibilities contained in the model.

• Note - "Anime style"

  The homebrewed model used in this project was trained on anime shows’ and movies’ stills, frames from mangas including color spreads, and various digital artworks in the “anime style”.

  Contemporary anime-style shares a lot of physiological homogeneity, but the different sources have very different designs regarding lighting and detail.

  FOR ALL EXAMPLES LOOK AT THE BOTTOM

  Broadcast animes need to be made inexpensively, and as such, they have heavily simplified lighting styles. This style frequently will use as few as a single tone for a given material, sometimes a secondary shading tone, and infrequently a specular highlight shape, typically reserved for hair, water, glass, and metals. Due to the massive simplification, the greatest artistic license is taken regarding physical light behavior.

  Mangas are typically black and white and limited to linework. Since mangas are usually the output of a single artist, color is rare, and panels usually comprise of line and hatch-work. However, color spreads can be made by principle artists in collaboration with a coloring artist, or even commissioned, and because the volume is lower, much higher levels of detail and quality can be given to a single image. Manga color spreads may use more suble techniques like dithering, gradients, and other more subtle techniques and has a generally more “premium” quality to it than an anime still.

  Lastly are digital paintings by various artists. As the approach with the least emphasis on efficiency, images can have extreme levels of fine detail, the greatest range of lighting techniques ranging from graphic and stylized all the way to rendered using physically based rendering techniques.

  Common vernacular usage of “anime style” doesn’t distinguish between these very different sources of origin so a more precise language is used to accurately spec the details of the source pieces.

PROCESS SUMMARY

The purpose of a style guide is to unify inter-disciplinary execution of a vision, however, since the development of the style guide will precede the development of asset pipelines and tools development, they require interpretation or negotiation. This is exacerbated if the beginnings of the style guide start with concept art in which it is the artists responsibility to prioritize aesthetics and expression over procedural accuracy.

In this example, I work off the output of Stable Diffusion AI output based on a home-brewed blend of different artistic models, based on prompts used to evoke styles in alignment with an Art Direction strategy doc I developed ahead of the project. I treat all of this output as if it were example pieces from the project’s Art Director, and interpret the pieces ahead of shader development.

The goals of this exercise is to:

• Analyze the elements that make up the examples’ art styles

• Identify the shader behavior necessary to replicate that style for real time rendering use in-game

• Deliminate elements of the style that require negotiation between the technical execution and the artistic vision that are either impossible, expensive, or require specialized features

The product of this exercise is what might be brought to a project’s Art Director and Producers to weigh in on and judge cost-benefits.

ESSENTIAL STYLE FEATURES

At the highest level, the style can be distilled to these features.

“Chibi” proportions - source clip-studio.com

Hyung Tae Kim - Supernormal Proportion Design - source gamedeveloper.com

• Character faces have conventional broadcast anime proportions and level of detail. Eyes are minimally oversized and “jeweled”.

• Character hair is more detailed than animated designs, and are closer to still sources like character portrait assets, or color spreads.

• Character bodies are more “realistically” proportioned and detailed. Limb lengths are minimally shortened in a “chibi” style, nor lengthened in a super-normal style. Characters bodies include contouring in their limbs and body that convey underlying musculature and are not simplified.

• Characters have “idealized” lean physiques and have “tall” proportions.

• Distant details approach more painterly techniques like anime backgrounds frequently feature.

• Backgrounds use fogging-like behavior to diminish fine details and flatten colors.

Broadcast anime example - two tone diffuse lighting only

Manga color spread example - multi tone shading with gradients, and specular highlighting

Regarding the lighting, this style blends the stylized conventions of broadcast anime and is embellished with some stylized physical light behaviors.

Guweiz - Anime inspired digital illustration with complex rendering effects

• Lighting conventions vary significantly by material type with skin being close to broadcast anime standards, and other materials ranging from color-spread to digital painting levels of detail.

• Clothing is generally highly detailed with respect to bunching and draping, stitching, and differentiation in degrees of smoothness/fineness, and specularity.

• Material texturing is generally reserved for static objects.

• Metals are highly reflective.

OVERALL LIGHTING BEHAVIOR ANALYSIS

The overall style lands somewhere between anime-style cel shading for animation and anime influenced digital paintings.

Looking at the provided samples from the “style guide” here are a subset of a few essential features I noted for later specification for the shader.

• Generally the lighting style follows a simple consistent behavior in “lit” regions, and more complex stepped shading in unlit regions.

• [Skin] In both examples, exposed skin tends to follow the broadcast cel-shading approach where predominantly contiguous areas are either represented in the base “lit” tone, or darkened in a “shaded” tone. In both examples, a third darker shaded tone seems to be reserved for skin facing near perfectly away from the light source or facing generally downward. Also for areas of ambient occlusion (AO) the third tone is used if surrounded by the main unlit tone, or the darkest tone with a value close to the color of adjacent linework is used when adjacent to regions already using the third tone.

• [Hair] The hair in both cases both break from the conventions established in the skin. In the female presenting character, a translucence approaching the incident light color is visible in the long curtains of hair behind the character. The male presenting character on the other hand, has a lit region color close to the incident light color, and reserves its primary tone to the shaded region. Both have a slight specular highlight resembling the contribution of a “hair light” or rim light creating a “halo” effect across the bulk of the hair over the scalp. Very minor gradient shading is applied around this halo region. Like skin, both have an AO region close in color to linework for that region.

• [Cloth] In alignment with the convention establish by the hair, clothing in each sample either shows a lit region in its primary color, similar to the concept of an “albedo” color, or in a color approaching the incident light color. The only exception being regions that should be ambiently occluded, where those areas are then conveyed in the primary shading color. In both cases, lit areas have generally flattened colors. Primary shaded regions have much more intricate lighting behaviors.

  • Shaded regions have a secondary shading step, however, unlike hair, employ a significant gradient in it’s further shading. The shading is primarily diffuse, but some areas accentuate regions of expected AO.

  • Shaded regions also exhibit minor specular reflections depending on “smoothness”. For example, the female characters tights look like they could be either synthetic or leather-like, and exhibit a specular reflection presuming a light source roughly in the same position as the camera.

  • Stitching and minor details can be conveyed through explicit linework in the style of AO shading.

• [Dielectrics] All non-skin, non-hair, non-cloth dielectrics broadly exhibit a mixture of hair and cloth-like behaviors. They are primarily differentiated as observable in the following examples.

• Foliage in the foreground behaves most like cloth, but also exhibits some translucence like one of the hair examples.

• Stone and other background opaque objects can have additional details including material color variation. These objects also contrast more rectlinear geometry with softer gradient shading.

• Clouds behave primarily like other opaque geometry, but include gradient effects from ground and height fog like sources, and more granular shading values with softened stepping.

• [Metals] Metals exhibit a combination of diffuse and specular shading, varying from object to object even within a single sample. More matte sandblasted or scuffed metal parts have a higher emphasis on diffuse lighting. The predominant color variation within metal objects follow specular highlighting conventions, which like cloth, presume a primary light source near the camera/view’s perspective.

  • The color follows metal specular behavior where incident light color suppresses the absorbed color spectrum, which for the brass in the samples is light from the green-blue spectrum.

  • The next brightest reflection source are rim light sources roughly perpendicular to the view direction.

  • Regions facing the characters’ clothing are typically darkened to suggest it is reflecting the featured character, or at a minimum, occludes the rim source on that side. In lieu of ambient occlusion, acutely angled areas are strongly shaded in a “hall of mirrors” like self-reflective darkening.

  • Mixed in with intermediately darkened areas are ambient color tones.

DIFFUSE LIGHTING BEHAVIOR

• Diffuse Lighting Description

  Diffuse lighting behavior describes the phenomenon where a surface that will diffuse light in many different directions will exhibit an averaged effect where the surfaces that are pointing towards a light source will have more incident light to diffuse and will appear brighter. This is like the way that a basketball on the ground of a court will appear brightest on the portion of the ball that face the sun.

  Diffuse lighting calculations use a surface’s normal direction to determine how much it is facing in the direction of a light source — the areas facing directly facing being lit the brightest, and areas perpendicular to facing away from the source receiving no lighting.

• Common Use in Anime Style

  Anime and other cel shading techniques typically use a cut-off, and determine an arbitrary midpoint at which any surface that should be more lit is conveyed in one color, and any surface less lit is conveyed in another. This two tone shading is the basis of rudimentary toon shaders.

In the samples, skin has the simplest diffuse behavior with a single tone for lit region. Non-diffuse lit regions exhibit up to 3 tones.

• Either stacked darkening or 4+ tones may need to be specified. A primary shaded tone can be darkened one step for either facing downward or opposite the light source, or darkened two steps if both.

• Recreating this style will require controls for determining stacked shading effects to be maximally faithful to the source material.

Between all dielectric materials, the shaded regions of diffuse lit areas must be configurable for stepped shading to softer gradients.

• Skin has a sharp step, but clothing in shaded regions need to allow for gradients.

• Additionally, beyond the typical 0 to 1 range for typical diffuse calculation, (-)1 to (+)1 range may be necessary to add further shading.

For the Art Director

• All diffuse light based effect are replicable to the style guide.

• The approach for stepped vs stacked shading should be the principle priority.

• A diffuse approach with more than 3 steps will be fast to develop, but harder to work with.

  • This approach will give a more “graphic” appearance which can look more “procedural” and less “hand drawn”, though will be the fastest to develop.

  • It will also require extending the diffuse dot-product range beyond a clamped 0 to 1 range. This will make fine tuning of meshes more time consuming, and trickier to maintain for a wide variety of lighting conditions.

• A diffuse approach with stacked shading will be less faithful to the source material, but will be more flexible.

  • Constraining the diffuse behavior to 3 regions, can result in less authorial control of shading on the part of artists, but the first three regions are the most physically realistic, and will therefore be the most replicable across the widest range of lighting conditions.

  • Stacking can be used to create the impression of that 4+th step, with the stacking being the product of a combination of screen space and texture-based AO shading. The downside is that these stacked effect will be, by default more gradient-like.

SPECULAR LIGHTING BEHAVIOR

• Specular Lighting Description

  Specular reflections describe the behavior where a surface directly reflects the light bouncing off of a surface at the viewer, like seeing the sun off of the surface of water, or the way a the face of a watch can create a bright spot when it redirects sunlight through a window. Specular calculations use a surfaces half vector to determine which parts of a surface should be reflecting any given light source at the camera. Rough and irregular surfaces would reduce this effect because the light would be more scattered, so typically surface roughness/smoothness values can be used to lower/raise a specular reflection.

• Common Use In Anime Style

  Specular reflections are used fairly sparingly in broadcast anime, and are used sparingly on materials that warrant it. When they are used, they’re used deliberately to accentuate the smoothness or wetness of a surface, like a character’s glasses, water, metallic objects, or the paint of a nice car.

Like many other designs in the anime style, specular light reflection is used to create a brightened halo along the crown of the hair, which is used to create a subtle brightening to convey luster, as well as to add a detail element to increase visual interest.

• In the samples, both characters have specular reflection in the hair. However, the specular region is divorced from any meaningful light sources in the scene.

• Recreating this effect will require negotiation regarding fidelity and complexity, for what may be a minor effect in motion and at distance.

Specular sheen is also visible in the “leather” like surfaces like the male presenting character’s boots, and the female presenting character’s leggings and sleeves. Both softer and rougher surfaces like clothing and stone don’t have that effect. Smooth surfaces like leaves in the foreground and metallic surfaces make extensive use of the effect.

• Specular control will be needed in the shader, but due to the artistic liberty of the specular light behavior on the part of the artists, the fidelity of the implementation will track directly inversely with the complexity of the execution.

• A physically based approach could be jarring next to the diffuse behavior.

• A spherical harmonic style approach could be adequate to approximate the artistic choices made in the samples. Two colors could be defined representing an environmental tint, and a contextual/primary reflection source color. This would potentially get closest in fidelity to the samples, but will require additional work on the part of an artist for every highly specular object, and to determine its contextual accent. Additionally it would not be adaptive from environment to environment, or from lighting condition to lighting condition.

• A highly simplified specular tinting effect could be used using the camera view direction as the principle input. This effect would be the fastest to both deliver and execute, and might prove to be sufficient at gameplay distances.

For the Art Director

• Specular lighting, broadly, is a relatively minor effect, and will require the most complexity to deliver faithfully to the style guide.

  • In the case of hair, the effect is suggestive of 3-point studio lighting conditions and not environmental interaction.

  • In the case of clothing, the specularity seems to suggest the presence of a light source close to the view source.

  • For metallic object, they exhibit behavior that are like simplified physically based reflections. Recreating a variation of this effect that is more physically based could undercut the anime style, and stylized approaches will require potentially significant additional labor somewhere in the pipeline, again for a relatively minor effect.

SPECIAL LIGHTING BEHAVIOR

Fresnel/Rim Lighting

Rim lighting example

• Rim-lighting can be calculated using a Fresnel calculation that highlights regions where the dot product between the view direction and the surface normal are close to zero.

• Some rim-lighting-like regions are evident in the samples, though their proximity to diffuse lit geometry might make the effect unnecessary.

• Rim-lighting maybe helpful to make individual objects pop out of the background more, but may work against the overall dark and foggy aesthetic at the heart of the design.

Translucence and Sub-Surface Scattering

• Light transmission effects are visible in the style-guide in the female presenting character’s hair, and in a particular piece of foliage.

• Given the game’s dark aesthetic, it would be unlikely to be relevant in most in-game lighting conditions and its absence would likely not be missed.

• If considered essential, it would have potential implications for maximum density of foliage, and character mesh design.

Ambient Occlusion

• Ambient occlusion is a close analog for certain shading choices by the artist.

• The samples make use of AO-based shading choices inconsistently by material.

  • For skin, AO is a potential justification for the darkest shaded regions.

  • For clothing, it coincides with areas represented as linework, darker shaded regions, or the basis of shading in certain lit regions.

  • A baked AO texture could be used to replicate the choices in a dynamic way to varying lighting conditions.

    • Baked AO will require additional manual processing by an artist in order to very faithfully adhere to the style guide.

    • A calculated AO texture could be had essentially for free if a subdivision level based sculpting approach is used and AO is baked down.

    • A wide variety of approaches to generating a AO texture are available with differing pipeline cost considerations, but texture based AO will likely be essential to capture the level of detail present in the style guide’s clothing examples.

  • Screen-space AO could be used as part of a stacked shading approach with a high degree of uniformity across all scene objects.

FINAL RECOMMENDATIONS

Diffuse behavior

• Diffuse lighting should serve as principle lighting behavior.

  • Lit regions and unlit/shaded regions should have different behaviors.

  • Lit regions should generally present as a uniform color, though controls could be exposed to interpolate between a surface’s albedo value and the incident light color.

  • Unlit/shaded regions will exhibit more complex behavior, to include controls for how sharp transition in values between a shaded tone and a shadow color are.

  • How subsequent lighting effects behave between lit and unlit regions will be detailed in their respective sections.

• Specular lighting should probably start with the most simplified implementation. No unified approach is represented in the style guide, and the overall effect in most surfaces are subtle.

  • I recommend that the hair specular reflection be omitted because executing to the example of the style guide would require significant additional work beyond what is derivative for free from mesh geometry, and normal map details.

  • Specular reflections in dielectric materials could be made available using the camera position as a light source, and using half-angle for masking, and exposed sliders to control intensity and spread/falloff.

  • Specular behavior for metallic materials will require experimentation since the style guide’s implementations take extensive artistic liberties. The order of escalation I recommend are: 1) the dielectric implementation, 2) additional half-vector based faked reflections, 3) a spherical harmonic approach, 4) a processed-physically based approach

    • These recommendations are in order of escalating development cost, and rendering performance implications, and potential to undermine the hand-drawn aesthetic of the style established in the guide.

  • RE Diffuse lit vs unlit regions:

    • Specular reflections in lit regions will be ignored.

    • Specular reflections in unlit regions will be controllable for tint color, spread/falloff, and intensity.

    • A more granularly controllable approach could be had by baking information onto one of the color channels available in the AO texture.

• Fresnel/Rim Lighting

  • I recommend that the effect be omitted.

  • I would advise that this option exists for later consideration if legibility issues arise where objects inadequately pop from the environment.

• Translucence and Sub-Surface Scattering

  • I recommend it be omitted at first since it isn’t core to the style, and the added complexity to develop with this feature in mind may not be visible in light of the game’s intended aesthetic.

• Ambient Occlusion

  • I recommend that both baked AO texture and screen space AO be tested for suitability. Though both will potentially be a departure from the style guide’s implementation, these approaches are physically based so will look right in the most lighting conditions, and feel natural on characters in motion.

  • RE Diffuse lit vs unlit regions:

    • AO shaded areas from the baked texture in lit regions will be shaded to the first shaded tone.

    • AO shaded areas from the baked texture in unlit regions will subtract from the derived diffuse value with a slider to increase or decrease this effect. Additionally they could have a separate shading effect independent of the diffuse value so a stacked and stepped effect could be achieved.

    • Screen space AO in all regions will be globally applied to the scene. A balance will need to be struck between its impact on environmental assets (which make heavy use of the effect) and its impact on characters (which the style guide depicts inconsistent applications by material).