From GPWiki
Jump to: navigation, search

Single Pass Environment Mapping

Introduction

Another technique for increasing the realism of a computer generated image is to take into account an object's surroundings when rendering it. For example, when rendering an object with a mirrored surface you must use the object's surroundings to calculate the correct color at each pixel that the object covers. In ray tracing, this is quite literally the case where rays are reflected off of a particular surface to see what other objects are visible in the reflected image.

However, as you may know ray tracing is typically a very computationally and memory intensive process which is not easily implemented for use in real-time rendering. However, as is always the case there are certain assumptions and simplifications that can be made to allow a similar algorithm to approximate a ray tracing technique. This chapter will focus on one class of algorithms that attempt to do just this: environment mapping.

Environment mapping utilizes an image buffer to store the appearance information of all the objects surrounding one particular point in the scene. Then, when rendering the desired reflective object, this buffer is used to look up the color of the object that would be visible in the reflection of the mirrored surface for each pixel. This reduces the per-pixel rendering procedure from tracing a ray through the scene to only a texture lookup, at the additional cost of generating the environment map before performing the final rendering pass.

The fact that all of the scene information is gathered around a particular point is one of the primary simplifications that allow the algorithm to operate significantly faster than ray tracing. This is, however, also a potential source of problems with the algorithm. Since we are generating the environment maps around a point, there are potential accuracy issues with using those maps to represent all of the light surrounding a 3D geometric object. In addition, the reflective object itself can't appear in the environment map since it would completely envelop the simulation point.

Many variants of environment mapping have been proposed to counteract these limitations while still retaining some degree of the speed of the algorithm, and this chapter will investigate three of these variants: sphere mapped environment mapping, cube mapped environment mapping, and dual-paraboloid environment mapping. All three of these techniques store the scene information as viewed from a single point in world space. Each of these techniques uses different parameterizations to store the scene information with differing formats.

The three techniques presented in this chapter provide a trade-off between memory usage, rendering speed, and visual quality. Cube mapping utilizes a total of six 2D render targets to store the scene, while dual paraboloid mapping uses only two, and sphere mapping uses only one. However, the quality of each technique is also partially dependent on how the memory is used. Deciding which technique is appropriate for a given situation requires a careful comparison of each ones strength's and weaknesses. In this chapter, we will explore each of these techniques, how to implement them efficiently with Direct3D 10, and provide a demo program to experiment with and modify.

Algorithm Theory

Each of the three variants of environment mapping that we will investigate require a method of generating the environment map as well as a method of accessing the map once it is created. The key to understanding and implementing these two procedures is to understand the parameterization that is used to convert scene geometry to a form that can be stored in a render target. For all of these methods, the parameterization is based around defining a point that the environment map is surrounding. For the following discussion we will name this point C. Point C is typically located at the center of the object that will be rendered with the reflective material to give the best approximation of the current environment.

Sphere Mapping Parameterization

The simplest of these three parameterization techniques is called sphere mapping. It uses a single 2D render target to represent the environment surrounding point C. During the generation of the sphere map, the input geometry is converted from world-space Cartesian coordinates to a new coordinate system that represents how the geometry would look if we were to view the scene by looking directly at a perfectly reflective sphere.

Once this conversion has been performed, each world space vertex position will have been transformed to a new set of 3D coordinates. The x and y coordinates identify the x and y locations on the reflecting sphere where the corresponding vertex would appear when viewed from the current camera location. You can picture the reflective sphere as being centered in the render target, making the range of valid x and y coordinates [-1,1]. The z coordinate simply represents the distance from point C to that vertex. This parameterization essentially uses these new coordinates to determine where in the environment map each vertex should be placed during the creation of the map. This is shown in the following figure:

Figure 2 Spherical coordinates in a 2D buffer.png
Figure 1: How spherical coordinates relate to a 2D texture buffer.

While the location that the vertex will reside in the environment map is determined by the x and y coordinates, the z coordinate specifies the distance from point C to that vertex. By using this depth information in the output vertex position, the depth sorting hardware of the GPU (i.e. the z-buffer) will automatically determine what object is the closest to point C. This makes sense if you consider what each texel in the output buffer represents. Each texel can be thought of as the color that would be seen if you traced a ray from the viewer to point C, and then outward in the direction that the sphere would reflect it to. This implicitly requires that the closest object to point C is seen at that texel.

The mathematics of making the conversion from world space vertex positions to sphere map locations are based on the concept of viewing a reflective sphere. If you consider the output space of the vertex/geometry shaders, also known as clip space, there is a range of [-1,1] for the x and y coordinates and a range of [0,1] for the z coordinate. We need to find an algorithm for filling this output space with the sphere map data while utilizing as much of the available volume as possible.

To do this, we will consider the reflective sphere to be positioned at the origin and have unit size. This effectively makes the sphere touch the clip space boundary in the x and y directions at both the positive and negative sides. This also means that the z-values of the sphere will be in the [-1,1] range, but we will artificially set the z coordinate based on the normalized distance from point C which will produce the required [0,1] z range.

Let’s assume for now that point C is at the origin of world space. This means that point C is located right in the center of the unit sphere. Now consider a single vertex of the geometry in the environment surrounding point C. To find the location on the unit sphere’s surface where that vertex would be visible to point C, we simply normalize the vector representing that vertex’s position (this works because point C is at the origin). This vector represents the direction of the viewing ray that would be reflected by the unit sphere. From the initial description of the sphere mapping parameterization, we know that the view is coming from a single direction, and is reflected in all directions by the unit sphere. This is shown in the following figure:

Figure 3 Visualization of how a sphere reflects the environment.png
Figure 2: How a sphere reflects a view to the surrounding environment.

Since we know the incident view vector is always in the same direction, along the positive z-axis, then we now know the incident and reflected vectors for this particular vertex. If a photon were to originate at the vertex and travel to the sphere, then reflect off of the sphere towards the viewer, then the direction of that reflected photon would be the opposite of the view vector – which means it would be along the negative z-axis.

Given these two vectors, we can solve for the normal vector on the unit sphere’s surface that would have generated this reflection. Both the incident and reflected vectors are unit length, so the vector between them can be found by adding the two vectors, and then normalizing their size by dividing by the resulting vector’s magnitude. Please note that both of these vectors are defined as starting from Point C and travelling outward - thus adding them and normalizing produces a vector between the incident and reflected vectors. This process is shown in the following equations:

Single Pass Environment Mapping Equation 1 and 2.png

With the normalized normal vector found, we simply take the x and y coordinates of the normal vector to represent the x and y position in clip space that this vertex will be placed at. You can visualize this as projecting the normal vector onto the z = 0 plane in clip space. The output z coordinate is found by calculating the distance from the origin to the original vertex world space location. With these three pieces of information, we can consider the vertex to be transformed into the sphere map, and repeat the process for all remaining vertices in the surrounding environment. A sample sphere map is shown here for your reference:

Figure 4 Sample sphere map.png
Figure 3: A sample sphere map generated by the demo for this chapter.

Once the environment map has been generated, the object that point C is representing can be rendered as normal. A view vector is created from the camera point to the current pixel, and is reflected about the normal vector of the object's surface at the current pixel. After the vector has been reflected, its direction can be converted to sphere map coordinates in the same manner that was used to generate the map. These coordinates are then used to sample the sphere map and determine what color the output pixel should be.

The sphere mapping parameterization is relatively simple, but suffers from a singularity point at the far side of the sphere with respect to the viewer. This is due to the fact that there are many paths to the far side of the sphere, but only one point to represent the endpoint of all of these paths. In essence, the entire outer ring of the sphere map maps to the single point on the far side of the sphere. This results in a stretching artifact in the sphere map around the local area of the singularity point.

This also means that each texel of the sphere map does not correspond to the same amount of surface area when looking out at the environment from point C. Since the reflective object will likely use most of the sphere map at some location of its surface, the clarity of the reflected environment will vary depending on where the sample falls within the sphere map. This has the potential effect of distorting the object’s surface appearance unevenly.

One other caveat with sphere mapping involves the scene geometry location with respect to the singularity point. With one point representing multiple path ways to that point, it is possible for a single small triangle to be transformed to a much larger size in the sphere map if its vertices happen to surround the singularity. This occurs because the triangle is a linear object that is being warped into this spherical space. If the triangle were to be subdivided many times, the resulting triangles would appear along the outer rim of the sphere map instead of covering a large portion of it. This issue can be overcome (either by detecting the triangles that cross the singularity and removing them or ensuring that there are no triangles there to begin with) but introduces a limitation on the uses of this technique.

Cube Mapping Parameterization

Cube mapping is another parameterization that improves on sphere mapping. The overall concept is that instead of representing the surrounding scene with a spherical coordinate space you would use the 3D vector pointing out from point C to intersect a cube instead of a sphere. This is shown in the following figure:

Figure 5 Visualization of a cube map.png
Figure 4: A visualization of a cube map, and how it unfolds to individual render targets.

Each face of the cube is a 2D surface, which is easily accommodated with standard rendering techniques. The procedure for generating a cube map is simply to render the scene into each of the six faces of a cube with the proper projection, and then use the reflected view vector to look up the environment color in the appropriate cube face. Since a 3D reflection vector is used, there are no strange conversions to new non-linear coordinate spaces. Due to the popularity of this technique, there are even special hardware instructions available that select the appropriate cube face and sampling location, making this parameterization very easy to implement.

There are several significant differences to note between sphere mapping and cube mapping. The first is that cube mapping requires six rendering passes, one for each cube face, to generate the environment map. This is a significant increase in rendering workload that must be taken into consideration. Also, similarly to sphere mapping, the texel to environment surface area ratios are not uniform across each of the cube faces. However, the ratio is significantly more stable than sphere mapping and there are no singularity points – all areas of the environment can be relatively accurately represented without any special case considerations. Cube mapping thus offers higher image quality at the expense of additional rendering complexity.

Even though there is additional rendering complexity, we will see in the implementation section how to utilize several interesting techniques for reducing the rendering cost of this parameterization that have only recently been available with the introduction Direct3D 10.

Dual Paraboloid Parameterization

The two previous parameterization techniques both have different advantages as well as disadvantages. Sphere mapping is simple and efficient to implement, and yet it has certain image quality limitations. Cube mapping provides better image quality at the expense of additional rendering complexity. Now we will look at a third parameterization, dual-paraboloid mapping, which attempts to blend the attributes of the previous two techniques.

The concept behind dual paraboloid mapping is to use two opposing paraboloid surfaces as the basis for the parameterization of the environment instead of a sphere as used in sphere mapping. A paraboloid is the surface that you would create if you were to revolve a parabola about an axis that passes through its center point perpendicular to any tangent vector at that point.

If you imagine looking at the cone side of a mirrored surface paraboloid, then your view would reflect off of the surface of the paraboloid over one hemisphere of the volume surrounding it. If you consider a second paraboloid positioned in the opposite orientation to the first one, then it would cover the other hemisphere surrounding point C. This is illustrated in Figure 5:

Figure 6 Visualization of how two paraboloids reflect the environment.png
Figure 5: A visualization of how two paraboloids reflect the environment in two directions.

Thus with two paraboloid surface elements we can describe the entire environment surrounding point C. In this case, the two surface elements are represented by two 2D render targets. The parameterization uses the same methodology as sphere mapping to perform the conversion from world/view space to the paraboloid space – by finding the normal vector that would reflect the view vector to a particular location in the surrounding environment. However, the mathematics of how to determine normal vector are somewhat different for the paraboloid than it is for the sphere.

To find the normal vector of the paraboloid we first need to consider the mathematic construct that represents the paraboloid. We will use the following equation to define the shape of the paraboloid:

Single Pass Environment Mapping Equation 3.png

Now we will define a point on the surface of the paraboloid as follows:

Single Pass Environment Mapping Equation 4.png

A common technique for finding a normal vector on a given surface is to find two tangent vectors at the desired point and then take the cross product of those vectors. To do this, we can evaluate the partial derivatives for our point equation above:

Single Pass Environment Mapping Equation 5 6 and 7.png

Another way to find the normal vector is to consider the incident and reflected vectors again. If both vectors are normalized (or at least the same length), we can simply add them and the resulting vector has the same direction as the normal vector. The reflection vector is opposite for the two paraboloids since they are reflecting the scene into two opposite directions. Regardless of the reflection vector direction, we can perform the following equality to find the normal vector:

Single Pass Environment Mapping Equation 8.png

Once the sum of the incident and reflected vectors is known, we can trivially find the normal vector by dividing the entire vector by its z component:

Single Pass Environment Mapping Equation 9.png

With the normal vector found, we can directly use the x and y components of the vector to index into the paraboloid map. This process will be performed for each piece of scene geometry twice – once to generate the forward facing paraboloid and once to generate the backward facing paraboloid. Sampling the paraboloid maps is done in the same manner – for each reflection vector we find the normal that produced it and sample the appropriate texture. The following figure shows a sample paraboloid map.

Figure 7 Sample paraboloid maps.png
Figure 6: A sample pair of paraboloid maps rendered in wireframe.

You may be wondering what makes dual paraboloid mapping any more or less desirable than the other two parameterizations that we have discussed. The first point is that the environment representation requires two render targets, and consequently two rendering passes, instead of cube mapping's six render targets. At the same time, the texel to surface area ratio is significantly more stable in dual paraboloid mapping, and there is no singularity point in the environment map as opposed to sphere mapping. Thus the paraboloid parameterization can be considered a trade-off between the image quality of sphere mapping and the rendering complexity of cube mapping.

Implementation

With a solid understanding of the theory behind each of these parameterizations, we can now investigate how to implement them while utilizing some of the unique new features available in Direct3D 10. We will look at the implementation of the sphere, cube, and paraboloid parameterizations in turn, including the resources required and the rendering sequences used.

Sphere Mapping Implementation

As we have discussed above, sphere mapping utilizes a single 2D render target. Thus the render target and the depth target used to generate the sphere map can be created in the same manner as normal.

// Create depth stencil texture.
dstex.Width = ENVMAPSIZE;
dstex.Height = ENVMAPSIZE;
dstex.MipLevels = 1;
dstex.ArraySize = 1;
dstex.SampleDesc.Count = 1;
dstex.SampleDesc.Quality = 0;
dstex.Format = DXGI_FORMAT_D32_FLOAT;
dstex.Usage = D3D10_USAGE_DEFAULT;
dstex.BindFlags = D3D10_BIND_DEPTH_STENCIL;
dstex.CPUAccessFlags = 0;
dstex.MiscFlags = 0;
 
V_RETURN( pd3dDevice->CreateTexture2D( &dstex, NULL, &g_pSphereEnvDepthMap ));
 
 
// Create the sphere map render target
dstex.Format = DXGI_FORMAT_R8G8B8A8_UNORM;
dstex.MipLevels = MIPLEVELS;
dstex.ArraySize = 1;
dstex.BindFlags = D3D10_BIND_RENDER_TARGET | D3D10_BIND_SHADER_RESOURCE;
dstex.MiscFlags = D3D10_RESOURCE_MISC_GENERATE_MIPS;
 
V_RETURN( pd3dDevice->CreateTexture2D( &dstex, NULL, &g_pSphereEnvMap ));

With the render and depth targets created, we can also create the resource views that we will need to connect them to the rendering pipeline. The depth target will require only a depth stencil view, while the render target will require both a render target view as well as a shader resource view. This is also shown in the bind flags used to create the respective resources.

// Create the depth stencil view for the sphere map
DescDS.Format = DXGI_FORMAT_D32_FLOAT;
DescDS.ViewDimension = D3D10_DSV_DIMENSION_TEXTURE2D;
DescDS.Texture2D.MipSlice = 0;
 
V_RETURN( pd3dDevice->CreateDepthStencilView( g_pSphereEnvDepthMap, &DescDS, &g_pSphereEnvDepthMapDSV ));
 
 
// Create the render target view for the sphere map
DescRT.Format = DXGI_FORMAT_R8G8B8A8_UNORM;
DescRT.ViewDimension = D3D10_RTV_DIMENSION_TEXTURE2D;
DescRT.Texture2D.MipSlice = 0;
 
V_RETURN( pd3dDevice->CreateRenderTargetView( g_pSphereEnvMap, &DescRT, &g_pSphereEnvMapRTV ));
 
 
// Create the shader resource view for the sphere map
ZeroMemory( &SRVDesc, sizeof(SRVDesc) );
SRVDesc.Format = DXGI_FORMAT_R8G8B8A8_UNORM;
SRVDesc.ViewDimension = D3D10_SRV_DIMENSION_TEXTURE2D;
SRVDesc.Texture2D.MipLevels = MIPLEVELS;
 
V_RETURN( pd3dDevice->CreateShaderResourceView( g_pSphereEnvMap, &SRVDesc, &g_pSphereEnvMapSRV ));

Once the resources are created, we can turn our attention to generating the sphere map for use in the subsequent final rendering pass. This particular implementation utilizes all three programmable shader stages in the Direct3D 10 pipeline. To begin with, the vertex shader will transform all of the incoming geometry into a ‘view’ space in which the viewer is actually located at point C. This has the advantage that it places point C at the origin of this space, making the transformations required further down the pipeline simpler (you will see in the geometry shader how this helps). The texture coordinates of each vertex are simply passed through to the output structure. The vertex shader listing is shown here:

GS_ENVMAP_IN VS_SphereMap( VS_ENVMAP_IN input )
{
    GS_ENVMAP_IN output = (GS_ENVMAP_IN)0.0f;
 
    // Compute world position
    float4 PosWS = mul( input.Pos, mWorld );
 
    // Compute view position
    float4 PosVS = mul( PosWS, g_mViewCM[0] );
 
    // Output the view space position
    output.Pos = PosVS;
 
    // Propagate tex coord
    output.Tex = input.Tex;
 
    return output;
}

The bulk of the work in generating the sphere map occurs in the geometry shader. The geometry shader will iterate over the three vertices of the current triangle being rendered into the sphere map as shown here:

// Compute sphere map coordinates for each vertex
for( int v = 0; v < 3; v++ )
{
    .....
    .....
}

For each of these vertices, we begin by finding a unit length vector from point C to the vertex. Since the incoming vertices were transformed to view space in the vertex shader, we can simply normalize the input vertex position to obtain the vector.

// Find the normalized direction to the vertex
float3 SpherePos = normalize( input[v].Pos.xyz );

Next, we will add the view vector to the position vector to obtain the un-normalized normal vector of the sphere. Then we normalize the result to produce a normal vector that resides within the unit sphere.

// Add the view direction vector --> always (0,0,-1)
// This produces a vector in the same direction as the
// normal vector between the incident & reflected
SpherePos.z -= 1;
 
// Re-normalize the vector for unit length
SpherePos = normalize( SpherePos );

This normal vector provides the x and y coordinates for the output clip space position of the vertex. The z coordinate is found by calculating the distance from point C to the view space position of the vertex, and there is no perspective warping so the w coordinate can safely be set to 1.

// Output the x,y coordinates of the normal to locate 
// the vertices in clip space
 
output.Pos.x = SpherePos.x;
output.Pos.y = SpherePos.y;
 
// Output the view space distance from point C to vertex
output.Pos.z = length( input[v].Pos.xyz ) / 100.0f;
 
// There is no perspective warping, so w = 1
output.Pos.w = 1;

The input texture coordinates are simply passed through to the pixel shader, and then the vertex is appended to the output stream.

// Propagate the texture coordinates to the pixel shader
output.Tex = input[v].Tex;
 
CubeMapStream.Append( output );

The pixel shader only samples the object’s texture to place the appropriate color into the sphere map where the geometry shader positioned it.

The second half of the sphere map algorithm is to be able to access the appropriate location in the sphere map when rendering the final scene. This is performed primarily in the pixel shader, while the vertex shader and geometry shader are only used for vertex transformation and normal vector setup. While accessing the sphere map, we initially have the world space surface normal vector of the object being rendered. We also have the current pixel location in world space as well as the view position – which allows us to create the view vector. Together with the normal vector we can then create the reflected view vector, which indicates the direction in the scene that we wish to find in the sphere map.

// Vector from eye to pixel in WS
float3 I = vin.wPos.xyz - vEye;
 
// Reflected eye vector in WS
float3 wR = I - 2.0f * dot( I, wN ) * wN;</code>
 
The first task is to convert the reflected view vector to the same view space that was used while generating the sphere map.
 
<code type="hlsl">
// Reflected eye vector in front paraboloid basis
float3 wRf = normalize( mul( wR, (float3x3)g_mViewCM[0] ) );

With this reflection vector in the correct view space, we can follow the same process that we did in generating the sphere map – find the normal vector that would reflect the view vector in the direction of the reflection vector. We do this by adding the view vector and renormalizing the resulting vector.

// Add the view vector to the reflected vector
wRf.z -= 1.0f;
 
// Re-normalize the length of the resulting vector
float size = length( wRf.xyz );
wRf.x = wRf.x / size;
wRf.y = wRf.y / size;

These coordinates are now in the same clip space that we discussed earlier – meaning that the x and y coordinates are in the range of [-1,1]. Since we are now accessing a texture, we need to map these coordinates into texture space, with the x range of [0,1] and the y range of [0,1] with the origin at the top left corner of the texture.

// Remap the coordinates to texture space
float2 coords;
coords.x = ( wRf.x / 2 ) + 0.5;
coords.y = 1 - ( ( wRf.y / 2 ) + 0.5);

Finally, we sample the sphere map with a standard 2D texture sampler.

// Sample the sphere map
float4 SphereSampleFront = g_txSphereMap.Sample( g_samLinear, coords );

This value can then be directly used or blended with other parameters to appear partially reflective.

Cube Mapping Implementation

Next we will investigate the cube mapping implementation. The cube map resources are significantly different than the resources that we created for sphere mapping. First we will look at the creation of the render and depth targets.

// Create cubic depth stencil texture
D3D10_TEXTURE2D_DESC dstex;
dstex.Width = ENVMAPSIZE;
dstex.Height = ENVMAPSIZE;
dstex.MipLevels = 1;
dstex.ArraySize = 6;
dstex.SampleDesc.Count = 1;
dstex.SampleDesc.Quality = 0;
dstex.Format = DXGI_FORMAT_D32_FLOAT;
dstex.Usage = D3D10_USAGE_DEFAULT;
dstex.BindFlags = D3D10_BIND_DEPTH_STENCIL;
dstex.CPUAccessFlags = 0;
dstex.MiscFlags = D3D10_RESOURCE_MISC_TEXTURECUBE;
 
V_RETURN( pd3dDevice->CreateTexture2D( &dstex, NULL, &g_pCubeEnvDepthMap ));
 
// Create the cube map for env map render target
dstex.Format = DXGI_FORMAT_R8G8B8A8_UNORM;
dstex.BindFlags = D3D10_BIND_RENDER_TARGET | D3D10_BIND_SHADER_RESOURCE;
dstex.MiscFlags = D3D10_RESOURCE_MISC_GENERATE_MIPS | D3D10_RESOURCE_MISC_TEXTURECUBE;
dstex.MipLevels = MIPLEVELS;
 
V_RETURN( pd3dDevice->CreateTexture2D( &dstex, NULL, &g_pCubeEnvMap ));

The cube map resources make use of one of the new features in Direct3D 10 – texture arrays. The idea here is to create a single resource that is interpreted as multiple individual textures. These can then be individually indexed within one of the programmable shader stages in the rendering pipeline. In addition to being able to access the textures individually, you can also bind the texture array to the pipeline as a render target. Then each primitive can specify a system value in its output structure (identified with the semantic SV_RenderTargetArrayIndex) that will determine which texture in the array will receive that particular primitive. Consider what techniques this ability can allow – this provides a significant freedom for sorting or multiplying geometric data.

In this case, we will be using six textures in the texture array – one for each of the six directions in the cube map. This is specified with the dstex.ArraySize parameter, which we have set to six. Another point of interest is the dstex.MiscFlags parameter. In both the render and depth targets we specify the D3D10_RESOURCE_MISC_TEXTURECUBE flag. This allows the texture array resource to be bound to the rendering pipeline as a cube texture instead of just a simple texture array. With a cube texture, we can utilize the hardware instructions specifically added for accessing cube maps. We’ll see how to utilize these instructions later on in this section. Next we create the resource views needed for binding the render and depth targets to the pipeline.

// Create the depth stencil view for the entire cube
D3D10_DEPTH_STENCIL_VIEW_DESC DescDS;
DescDS.Format = DXGI_FORMAT_D32_FLOAT;
DescDS.ViewDimension = D3D10_DSV_DIMENSION_TEXTURE2DARRAY;
DescDS.Texture2DArray.FirstArraySlice = 0;
DescDS.Texture2DArray.ArraySize = 6;
DescDS.Texture2DArray.MipSlice = 0;
 
V_RETURN( pd3dDevice->CreateDepthStencilView( g_pCubeEnvDepthMap, &DescDS, &g_pCubeEnvDepthMapDSV ));
 
// Create the 6-face render target view
D3D10_RENDER_TARGET_VIEW_DESC DescRT;
DescRT.Format = dstex.Format;
DescRT.ViewDimension = D3D10_RTV_DIMENSION_TEXTURE2DARRAY;
DescRT.Texture2DArray.FirstArraySlice = 0;
DescRT.Texture2DArray.ArraySize = 6;
DescRT.Texture2DArray.MipSlice = 0;
 
V_RETURN( pd3dDevice->CreateRenderTargetView( g_pCubeEnvMap, &DescRT, &g_pCubeEnvMapRTV ));
 
 
// Create the shader resource view for the cubic env map
D3D10_SHADER_RESOURCE_VIEW_DESC SRVDesc;
ZeroMemory( &SRVDesc, sizeof(SRVDesc) );
SRVDesc.Format = dstex.Format;
SRVDesc.ViewDimension = D3D10_SRV_DIMENSION_TEXTURECUBE;
SRVDesc.TextureCube.MipLevels = MIPLEVELS;
SRVDesc.TextureCube.MostDetailedMip = 0;
 
V_RETURN( pd3dDevice->CreateShaderResourceView( g_pCubeEnvMap, &SRVDesc, &g_pCubeEnvMapSRV ));

Here again, we specify that we will be using these resources as a cube texture. The ViewDimension parameter specifies the actual usage for each of the resource views. Notice that the depth target and render target views specify D3D10_DSV_DIMENSION_TEXTURE2DARRAY and D3D10_RTV_DIMENSION_TEXTURE2DARRAY while the shader resource view specifies D3D10_SRV_DIMENSION_TEXTURECUBE. This is because while we are generating the cube map we want to interpret the resources as texture arrays and while we are accessing the cube map we want to interpret the resources as a cube texture. Prior to Direct3D 10, this type of multiple interpretations were not possible!

With the resources ready to go, we can start the algorithm by generating the cube map. In pre-Direct3D 10 implementations, we would need to render the geometry of the scene into each face of the cube map manually. However, with the texture array resource we can render the geometry once and then multiply the data in the geometry shader – producing one copy of the geometry for each of the six cube faces. To get started, the vertex shader is used to convert the input object space geometry to world space.

// Compute world position
output.Pos = mul( input.Pos, mWorld );

Next, the geometry shader receives the world space geometry one primitive at a time. For each primitive, the geometry shader iterates over it six times and transforming it into the appropriate viewing space for each of the cube faces. This is shown in the following code listing.

for( int f = 0; f < 6; ++f )
{
    // Compute screen coordinates
    PS_ENVMAP_IN output;
 
    output.RTIndex = f;
 
    for( int v = 0; v < 3; v++ )
    {
        output.Pos = mul( input[v].Pos, g_mViewCM[f] );
        output.Pos = mul( output.Pos, mProj );
        output.Tex = input[v].Tex;
 
        CubeMapStream.Append( output );
    }
 
    CubeMapStream.RestartStrip();
}

Notice how each output vertex specifies its output.RTIndex to determine which render target it ends up being rendered into. Also notice how the CubeMapStream.RestartStrip() function is called after each time the triangle’s three vertices have been passed into the stream. This allows the geometry to be rendered into each of the cube faces individually, enabling each of the render targets to perform the proper primitive clipping without interfering with the rendering of the other render targets. The pixel shader is invoked for each of the triangles as they are rasterized, storing the appropriate color into each face of the cube map.

In the second phase of the algorithm, the cube map is sampled using to produce the appropriate environment color. This is performed by calculating the world space reflected view vector. To find the vector, we simply need the position of the current fragment in world space, its normal vector, and the position of the viewer. The reflected vector is calculated as shown in the following listing.

// Find the world space surface normal vector
float3 wN = HighOrderInterpolate( vin.Normals, vin.Bary.x, vin.Bary.y );
 
wN = mul( wN, (float3x3)mWorld );
 
// Calculate the reflected view vector
float3 I = vin.wPos.xyz - vEye;
float3 wR = I - 2.0f * dot( I, wN ) * wN;

Once the reflected vector has been found, we utilize the cube map sampling hardware that is available in all modern GPUs. The input to the sampling function of the cube map takes a sampler structure to determine the filtering states as well as a 3D vector, which is used to determine which of the six cube faces to sample and where to sample within that face.

// Sample the cube map
float4 CubeSample = g_txEnvMap.Sample( g_samCube, wR );

This output color represents the environment color that the viewer would see on the reflective object.

Dual Paraboloid Implementation

The final parameterization that we will implement is the dual paraboloid parameterization. The dual paraboloid environment map will also use a texture array as its resource, but will use it somewhat differently than the cube map parameterization. Here is the code listing to create the depth and render targets for the dual paraboloid map.

// Create cubic depth stencil texture.
dstex.Width = ENVMAPSIZE;
dstex.Height = ENVMAPSIZE;
dstex.MipLevels = 1;
dstex.ArraySize = 2;
dstex.SampleDesc.Count = 1;
dstex.SampleDesc.Quality = 0;
dstex.Format = DXGI_FORMAT_D32_FLOAT;
dstex.Usage = D3D10_USAGE_DEFAULT;
dstex.BindFlags = D3D10_BIND_DEPTH_STENCIL;
dstex.CPUAccessFlags = 0;
dstex.MiscFlags = 0;
 
V_RETURN( pd3dDevice->CreateTexture2D( &dstex, NULL, &g_pParabEnvDepthMap ));
 
 
// Create the paraboloid map for env map render target
dstex.Format = DXGI_FORMAT_R8G8B8A8_UNORM;
dstex.MipLevels = MIPLEVELS;
dstex.ArraySize = 2;
dstex.BindFlags = D3D10_BIND_RENDER_TARGET | D3D10_BIND_SHADER_RESOURCE;
dstex.MiscFlags = D3D10_RESOURCE_MISC_GENERATE_MIPS;
 
V_RETURN( pd3dDevice->CreateTexture2D( &dstex, NULL, &g_pParabEnvMap ));

If you recall the discussion of the paraboloid parameterization, we need to create two 2D render targets. Thus, we will create a texture array with two texture elements. However, as opposed to the creation of the cube map, we do not specify the D3D10_RESOURCE_MISC_TEXTURECUBE miscellaneous flag since we will not be using the cube texture hardware. The resource views used to bind this resource to the pipeline are created next.

// Create the depth stencil view for the entire cube
DescDS.Format = DXGI_FORMAT_D32_FLOAT;
DescDS.ViewDimension = D3D10_DSV_DIMENSION_TEXTURE2DARRAY;
DescDS.Texture2DArray.FirstArraySlice = 0;
DescDS.Texture2DArray.ArraySize = 2;
DescDS.Texture2DArray.MipSlice = 0;
 
V_RETURN( pd3dDevice->CreateDepthStencilView( g_pParabEnvDepthMap, &DescDS, &g_pParabEnvDepthMapDSV ));
 
// Create the 6-face render target view
DescRT.Format = DXGI_FORMAT_R8G8B8A8_UNORM;
DescRT.ViewDimension = D3D10_RTV_DIMENSION_TEXTURE2DARRAY;
DescRT.Texture2DArray.ArraySize = 2;
DescRT.Texture2DArray.FirstArraySlice = 0;
DescRT.Texture2DArray.MipSlice = 0;
 
V_RETURN( pd3dDevice->CreateRenderTargetView( g_pParabEnvMap, &DescRT, &g_pParabEnvMapRTV ));
 
// Create the shader resource view for the cubic env map
ZeroMemory( &SRVDesc, sizeof(SRVDesc) );
SRVDesc.Format = DXGI_FORMAT_R8G8B8A8_UNORM;
SRVDesc.ViewDimension = D3D10_SRV_DIMENSION_TEXTURE2DARRAY;
SRVDesc.Texture2DArray.ArraySize = 2;
SRVDesc.Texture2DArray.MipLevels = MIPLEVELS;
 
V_RETURN( pd3dDevice->CreateShaderResourceView( g_pParabEnvMap, &SRVDesc, &g_pParabEnvMapSRV ));

With the resources in hand, we can now begin generating the paraboloid maps. The generation phase utilizes a similar technique to what the cube map uses. At the outermost level, each triangle is multiplied twice – once for each of the two paraboloid maps. Then, each vertex is transformed into the view space of point C. An additional per-vertex parameter, output.ZValue, is also added to the output structure which is the view space z component. This value determines if the current vertex is in the ‘front’ paraboloid or if it should be in the ‘back’ paraboloid.

Next, the paraboloid coordinates are calculated by normalizing the view space position, adding the view vector (which is always (0,0,1)) and then dividing by the z component (please see the algorithm theory section for further details). Finally, the output depth value is calculated as distance from point C, with the sign of the view space z component. This is needed to ensure that geometry from one half-space doesn’t interfere with the geometry destined to appear in the other half-space.

for( int f = 0; f < 2; ++f )
{
    // Compute screen coordinates
    PS_PARABMAP_IN output;
 
    output.RTIndex = f;
 
    for( int v = 0; v < 3; v++ )
    {
        // Transform the geometry into the appropriate view space
        output.Pos = mul( input[v].Pos, g_mViewCM[f+2] );
 
        // Output the view space depth for determining 
        // which paraboloid this vertex is in
        output.ZValue = output.Pos.z;
 
        // Normalize the view space position to get its direction
        float L = length( output.Pos.xyz );
        output.Pos.xyz = output.Pos.xyz / L;
 
        // Add the view vector to the direction
        output.Pos.z += 1;
 
        // Divide by 'z' to generate paraboloid coordinates
        output.Pos.x = output.Pos.x / output.Pos.z;
        output.Pos.y = output.Pos.y / output.Pos.z;
 
        // Set the depth value for proper depth sorting
        output.Pos.z = L * sign(output.ZValue) / 500;
        output.Tex = input[v].Tex;
 
        CubeMapStream.Append( output );
    }
 
    CubeMapStream.RestartStrip();
}

To access the paraboloid map, we follow a similar procedure as when we created it. This occurs primarily in the pixel shader, as shown in the following listing.

// Calculate the world space surface normal
float3 wN = HighOrderInterpolate( vin.Normals, vin.Bary.x, vin.Bary.y );
wN = mul( wN, (float3x3)mWorld );
 
// Calculate the reflected view vector
float3 I = vin.wPos.xyz - vEye;
float3 wR = I - 2.0f * dot( I, wN ) * wN;
 
// Calculate two versions of this vector - one for
// the 'front' paraboloid and one for the 'back' paraboloid
float3 wRf = normalize( mul( wR, (float3x3)g_mViewCM[2] ) );
float3 wRb = normalize( mul( wR, (float3x3)g_mViewCM[3] ) );
 
// Calculate the front paraboloid coordinates
float3 front;
front.x = ( wRf.x / ( 2*(1+wRf.z) ) ) + 0.5;
front.y = 1 - ( (wRf.y / ( 2*(1+wRf.z) ) ) + 0.5);
front.z = 0;
 
// Calculate the back paraboloid coordinates
float3 back;
back.x = ( wRb.x / ( 2*(1+wRb.z) ) ) + 0.5;
back.y = 1 - ( (wRb.y / ( 2*(1+wRb.z) ) ) + 0.5);
back.z = 1;
 
// Sample the paraboloid maps
float4 ParaboloidSampleFront = g_txParabFront.Sample( g_samParaboloid, front );
float4 ParaboloidSampleBack = g_txParabFront.Sample( g_samParaboloid, back );
 
float fLight = saturate( dot( skyDir, wN ) ) + 0.2f;
 
// Combine both paraboloid maps
float4 outCol = 0.3*vMaterialDiff*fLight + 
1.5 * vMaterialSpec * max( ParaboloidSampleFront, ParaboloidSampleBack );
outCol.a = vMaterialDiff.a;//preserve alpha
 
return outCol;

The access is performed by first calculating the world space surface normal, reflecting the view vector, and transforming the reflected view vector to both of the two paraboloids view spaces. Once this is complete, we then divide the x and y coordinates by one plus the z component of the reflected vector (which corresponds to the view vector plus the reflection vector, and then dividing by the z component) and scale and bias the result to access texture space. The resulting texture coordinates are used to sample their respective paraboloid maps. In this case, we take the sum of the two samples since the samplers are configured for a black border color. Only one paraboloid map should have non-black results due to the nature of the half-space divide between them. Similarly to cube and sphere mapping, the final resulting color can be used directly or blended with other rendering techniques for more advanced effects.

Demo and Algorithm Performance

Demo Download: Single_Pass_Environment_Mapping_Demo.zip

The implementation is based on the DirectX SDK sample named 'CubeMapGS'. The demo has been simplified somewhat to only render a reflective sphere with its surroundings rendered as they were originally. A set of three radial buttons allow the user to choose the parameterization to use during the interactive viewing of the scene.

The results reflect the performance and quality characteristics that were discussed in the algorithm theory section. The quality of the environment maps are best when using cube mapping, then paraboloid mapping, and finally sphere mapping. However, the opposite order applies to rendering speed – sphere mapping is the fastest, followed by paraboloid mapping and then cube mapping. The percentage increases will vary from machine to machine, but the speed change is approximately proportional to the number of render targets used.

Improving Sphere Map Implementation

While working with the demo, you will notice two different negative effects while using sphere mapping. The first effect is that the sphere rendering has a large warping location at one point on its surface. This point corresponds to the ‘singularity’ that was discussed in the sphere mapping theory section. If you click and drag around the sphere you can rotate the scene from other angles that cannot see the singularity – hence this issue can be overcome by updating the sphere map’s ‘view space’ that is used during the generation of the map. By continually shifting the sphere map’s forward direction you can effectively hide the singularity.

The second and more critical rendering artifact can be seen when the scanner arm passes behind the singularity point. In this case, the scanner arm is the closest object to point C, and will thus be rendered on top of all of the other objects in the scene. Since it is passing through the singularity point it should ideally be rendered as a ring around the outermost portion of the sphere map, while the inner portion of the sphere map should contain the rest of the environment that isn’t directly behind the singularity point.

The problem here is that the scanner arm’s triangles are linearly interpolated, and don’t take into account any form of wrapping around the back of the cube map. The following figure shows this effect. On the left hand side, the scanner arm is shown after being transformed to world space. On the right hand side, you can see that there are many instances of the geometry being wrapped around the sphere instead of around the outside of the sphere.

Figure 8 Sphere map incorrect rasterization.png
Figure 7: Incorrect rasterization of a triangle on the front of the sphere.

This is not a simple problem to solve. The purely mathematic solution would be to detect when there is a polygon overlapping the singularity, and then dynamically tessellating the polygon to remove the area surrounding the singularity. This is shown in the following figure.

Figure 9 Dynamically tesselated hole.png
Figure 8: A dynamically tessellated ‘hole’ in the triangle.

Due to the complexity of this type of operation, it is not the most efficient to perform in a geometry shader. Another potential solution would be to place the reflective object in such an orientation that there are no polygons overlapping the singularity. In either case, care must be taken while using sphere mapping to ensure that the artifacts are not visible to the end user!

Improving Paraboloid Map Implementation

You may also notice artifacts in the paraboloid environment mapping demo as well. The two paraboloids are positioned such that one of them is pointing upward and the other is pointing downward. In the space where the two paraboloids meet, you can see a thin black line separating the two. This is also caused by the triangles being rasterized in a linear space instead of the non-linear paraboloid interpolation. Consider the following image of a paraboloid map:

Figure 10 Paraboloid linear rasterization.png
Figure 9: A wireframe view of a paraboloid map showing the linear rasterization of triangles (taken from the upward facing paraboloid in the demo program).

The linear points between the vertices of the outermost triangles cause this line to appear between the paraboloids. In this case, we must also dynamically detect when a triangle crosses the paraboloid boundary and then dynamically tessellate those triangles until the resulting error would be smaller than a single texel. I hope to include an implementation of this dynamic tessellation in a future revision of the demo.

One other possibility is to choose a random axis to split the paraboloid on. This would cause a small visual discontinuity in a random direction. Then the results from the current frame and the previous frame could be used together to remove the holes. Regardless of the technique used, this issue must also be addressed to effectively use paraboloid mapping.

Conclusion

In this chapter, we have made a thorough investigation of sphere mapping, cube mapping, and paraboloid mapping. After reviewing the theory behind each technique, we have implemented each of them for an interactive demo that can be used to compare their various properties. The correct choice of a parameterization requires knowledge of the situations that it will be used in. For cases that require high quality rendering, cube maps are probably the technique to use. However, if rendering speed is the most important consideration, then sphere mapping may be more appropriate. If a good blend between the two makes more sense, then dual paraboloid mapping would be a logical choice. In any of these three cases, environment mapping should be within the reach of modern real-time rendering system.