Advanced render in flash pt.1

Rendering light with pixel bender pt.1

How to implement current 3D engine light rendering techniques in Flash with pixel bender.

Introduction

Since the release of Flash Player 9 we can manipulate the pixels in AS3. This technique is commonly used in current 3D game engine (thanks to the shaders). Unfortunately, the manipulation of these pixels within Flash is quite slow and do not apply in real time.

But since Flash Player 10, we can also use Pixel Bender technology (low level language close to GLSL) so we can create image filters.The current 3D rendering techniques are using massively Normal maps, Specular reflection and occlusion effect to be able to achieve very realistic images. This is what we ‘ll do in Flash. Our goal is to make it in 2D while keeping an optimal framerate of 30 fps.

We’ll use CrazyBump to generate the images needed.

The rendering technologies

Normal map

A normal map is used to store the pixel’s normals (direction and magnitude) of an image. To do this, we need to store 3D vector coordinates to know the pixel orientation. So we can change pixels brightness for each pixels (dot product of the normal and light vector).The conventional technique used in 3D engines is using red for X axis, green for Y and blue for Z axis.

Flash solution: Our pixel bender use the normal map generated by CrazyBump to modulate the brightness of each pixel.

Specular map

A specular highlight is the bright spot of light that appears on shiny objects when illuminated. This allows to change the material’s appearance and make reflective surface more realistic such as plastic and wet surfaces.

Flash solution: We will store a single bits plan for specularity (no specularity for black pixels, full effect for white pixels) and this info will be used by the pixel bender to make light reflections.

Screen Space Ambient Occlusion (SSAO)

SSAO is a rendering technique for efficiently approximating the well-known computer graphics ambient occlusion effect in real time. It was used for the first time in a video game in the 2007 PC game Crysis made by Crytek. In modern 3d game engine, the SSAO is calculated in real-time on the GPU by analyzing the depth buffer.

Flash Solution: Although January Frischmuth wrote a pixel bender to calculate in real time SSAO, the amount of calculation is too large to use it in Flash. I used a static single bits plan map so our pixel bender can shades some pixels.


Let’s do it in Flash

The demo

You can watch the demo here. Wait for loading (2mo) and click to focus and control.

So, a quick resume: First we need the diffuse colors (3 planes) , than the normals (3 planes), and the details (2 planes) : the specular map (red) and occlusion infos (green) for a total of 8 bits planes (3 images needed)

diffuse (3), specular (1), normals (3), occlusion (1)

Here a quick resume for each light rendering result:

  1. No light : A simple 300 x 300 copyPixels.
  2. Added pixel bender light intensity
  3. Added normal map
  4. Added specular light (use normal map only)
  5. Specular light (use specular map) and occlusion map added

the demo

The pixel bender

This pixel bender takes 3 input images and only one parameter: the light position (float2). We compute the dot product of the normal and light vector then we add diffuse color, specular and occlusion factor. The result is affected by light intensity.


<languageVersion : 1.0;>

kernel SuperTexture
<
    namespace : "com.whiteflashwhitehit";
    vendor : "rDad";
    version : 1;
    description : "Light rendering 4 : light intensity + normal map + specular reflection map + occlusion map";
>
{

    input image4 diffuseMap;
    input image4 normalMap;
    input image4 detailMap;
    output pixel4 result;

    parameter float3 lightPosition
    <
        minValue: float3(0.0, 0.0, 0.0);
        maxValue: float3(+800.0, +600.0, 200.0);
        defaultValue: float3(400.0, 150.0, 60.0);
    >;

    void
    evaluatePixel()
    {
        float2 positionHere = outCoord();
        float3 toLight      = normalize(float3(lightPosition.x-positionHere.x,lightPosition.y-positionHere.y,lightPosition.z));

        float lenght        = distance(lightPosition.xy, positionHere);
        float intensity     = clamp(((200.0 - lenght) / 100.0),0.0,1.0);

        float4 normal       = sampleNearest(normalMap, positionHere);
        float4 detail       = sampleNearest(detailMap, positionHere);
        float normalFactor  = dot(((normal.rgb * 2.0) - 1.0) * intensity, toLight);

        float specular      = pow(normalFactor, 30.0) * pow(detail.r,4.);
        float occlusion     = (1. - detail.g) * .2;

        result              = sampleNearest(diffuseMap, positionHere);
        result.rgb          = (result.rgb * normalFactor) + (specular - occlusion);
    }
}

Conclusion

Thanks to pixel bender, we can implement advanced rendering techniques in Flash. The optimization of each step is necessary (Everything run at optimal 30 fps but the last rendering demo drop to 20 fps) but pixel bender open new graphics perspectives to indie game developpers.

Softwares used