Java:Tutorials:VolatileImage

From GPWiki
Jump to: navigation, search

Introduction

What is a VolatileImage?

VolatileImage is a relatively new object in the Java API. It differs from other Image variants in that if possible, VolatileImage is stored in Video RAM (VRAM). This means that instead of keeping the image in the system memory with everything else, it is kept on the memory local to the graphics card. This allows for much faster drawing-to and copying-from operations. There is a price to pay for this advantage though. On certain occasions, the image contents can be lost. Knowing this, the Java developers gave us ways to handle this.

Why Would a VolatileImage be Lost?

"It's true: VolatileImages can (and quite often will) go away and the API for VolatileImage was developed specifically to deal with that problem. All of the loss situations currently occur only on Windows.

You would think that we would actually be notified prior to these problems, so that we could prevent the loss, or backup the image, or something. But you'd be wrong; we only find out from Windows that there is a problem the next time we actually try to use the image. Surface loss situations arise from situations such as:

  • Another app going into fullscreen mode
  • The display mode changing on your screen (whether caused by your application or the user)
  • TaskManager being run
  • A screensaver kicking in
  • The system going into or out of StandBy or Hibernate mode
  • Your kid just yanked the power cord to the machine

Okay, so that last one applies to all platforms. And there's not much that the VolatileImage API can do to help you there. Try locking the door to your office." [1]

Why Should I use a VolatileImage?

Chances are if you're reading this, you're a game programmer. In most cases, this will mean that you want to be able to squeeze as much performance out of your system as possible. Using a VolatileImage helps you to do that by letting your video card take care of some of the graphical processing. Not only does a VolatileImage keep the information closer to where it's needed, but it also avoids having to send it over the system bus.

"Imagine trying to render 60 frames per second in an animation application, with a screen size of 1280x1024 and 32bpp screen depth; that's about 5 megs of data per frame * 60 frames = 300 MBytes of data moving across the bus, just to copy the buffer to the screen, not including anything else that needs to happen over the bus or through the CPU/cache/memory."[2]

How to Use VolatileImage

There are three main methods you need to worry about:

createCompatibleVolatileImage()

A VolatileImage is meant to optimize performance. In order to achieve this, the image data must be compatible with the hardware and the settings. That is, if the user's display is set to use 32-bit colour, then a 32-bit VolatileImage must be created so that it doesn't need to be converted to a 32-bit image later. That's why if the display mode changes, your VolatileImage will be lost.

The createCompatibleVolatileImage() method does not belong to VolatileImage, but rather GraphicsConfiguration. The GraphicsConfiguration object represents the display settings of the user's hardware. In order to create a VolatileImage, you need to obtain the current GraphicsConfiguration. Code Example 1 shows the syntax to retrieve the current GraphicsConfiguration.

Code Example 1

import java.awt.*;
import java.awt.image.VolatileImage;
 
...
 
GraphicsEnvironment ge = GraphicsEnvironment.getLocalGraphicsEnvironment();
GraphicsConfiguration gc = ge.getDefaultScreenDevice().getDefaultConfiguration();
 
...

Important Decisions

Once you have your GraphicsConfiguration, you can create your VolatileImage. At this point, you have a couple of choices to make:

  • How big will the VolatileImage be?
  • Do you want it to be hardware-accelerated? (Yes, hopefully.)
  • Will your image have transparent regions?
  • Will your image be translucent?

These choices are encapsulated in the various overridden versions of the method: createCompatibleVolatileImage(int width, int height)

  • Returns a VolatileImage with a data layout and color model compatible with this GraphicsConfiguration.[3]

createCompatibleVolatileImage(int width, int height, ImageCapabilities caps)

  • Returns a VolatileImage with a data layout and color model compatible with this GraphicsConfiguration, using the specified image capabilities.[3]

createCompatibleVolatileImage(int width, int height, ImageCapabilities caps, int transparency)

  • Returns a VolatileImage with a data layout and color model compatible with this GraphicsConfiguration, using the specified image capabilities and transparency value.[3]

createCompatibleVolatileImage(int width, int height, int transparency)

  • Returns a VolatileImage with a data layout and color model compatible with this GraphicsConfiguration, using the tranparency value.[3]

The ImageCapabilities object seems to only exist to specify/determine if the VolatileImage is accelerated. You probably don't need to worry about it.

The transparency parameter can be used to specify if the VolatileImage will be opaque, transparent, or translucent. These can be specified with the constants, Transparency.OPAQUE, Transparency.BITMASK, Transparency.TRANSLUCENT, respectively.

OPAQUE should be used if there will be no transparency. BITMASK should be used if there are transparent regions, but the image is otherwise opaque. TRANSLUCENT images can have pixels that are fully or partially transparent. Translucent images are a little bit trickier, so they have their own section.

validate()

Now that the VolatileImage has been created, we need to make sure we can use it. The validate() method has two uses:

  • After creation, validate() puts the VolatileImage into a known state, effectively clearing it, so it can be drawn to without artifacts showing up.
  • Before any operation is done to the VolatileImage, it makes sure that the image is still there and usable.


validate() takes one parameter, a GraphicsConfiguration object. This is to ensure that the VolatileImage is still compatible with the current settings.


validate() can return three different results:

  • VolatileImage.IMAGE_OK
    • The VolatileImage is still okay, and can be used.
  • VolatileImage.IMAGE_RESTORED
    • Something happened to the VolatileImage that may have affected its contents. If you're just going to clear the contents and draw over it, this doesn't matter. If you're copying from it, you should redraw the image first.
  • VolatileImage.IMAGE_INCOMPATIBLE
    • The VolatileImage is no longer in a usable state. You should recreate the VolatileImage.

Depending on what validate() returned, you might have to redraw or recreate your image. It can be easier to just recreate the image in both cases. Code Example 2 gives an example of creating a VolatileImage.

Code Example 2

Creating a VolatileImage.

import java.awt.*;
import java.awt.image.VolatileImage;
 
...
 
private VolatileImage createVolatileImage(int width, int height, int transparency) {	
	GraphicsEnvironment ge = GraphicsEnvironment.getLocalGraphicsEnvironment();
	GraphicsConfiguration gc = ge.getDefaultScreenDevice().getDefaultConfiguration();
	VolatileImage image = null;
 
	image = gc.createCompatibleVolatileImage(width, height, transparency);
 
	int valid = image.validate(gc);
 
	if (valid == VolatileImage.IMAGE_INCOMPATIBLE) {
		image = this.createVolatileImage(width, height, transparency);
		return image;
	}
 
	return image;
}
 
...

You might notice that the potential for infinite recursion exists. In practice, if the VolatileImage can't be created in VRAM, then it will resort to using system memory, which will never be incompatible. Once we cover contentsLost(), a more inclusive example can be shown.

Code Example 3

Copying from a VolatileImage.

import java.awt.*;
import java.awt.image.VolatileImage;
 
// From the Code Example 2.
VolatileImage vimage = createVolatileImage(800, 600, Transparency.OPAQUE);
 
...
 
public void draw(Graphics2D g, int x, int y) {
	GraphicsEnvironment ge = GraphicsEnvironment.getLocalGraphicsEnvironment();
	GraphicsConfiguration gc = ge.getDefaultScreenDevice().getDefaultConfiguration();
 
	// Since we're copying from the VolatileImage, we need it in a good state.
	if (vimage.validate(gc) != VolatileImage.IMAGE_OK) {
		vimage = createVolatileImage(vimage.getWidth(), vimage.getHeight(), vimage.getTransparency());
		render(); // This is coming up in Code Example 4.
	}
 
	g.drawImage(vimage,x,y,null);	
}
 
...

contentsLost()

The contents of a VolatileImage can be lost at anytime, so what happens if it happens while you're drawing to it? You'd like to know so that it can be redrawn. This is where contentsLost() comes into play. It will tell you if anything has happened to the VolatileImage since you last called validate(). This method should be called at the end of your drawing routine.

It is common to use a do...while loop in for this. If the contents are lost, then you'll have to redraw. Using a do...while loop rather than a while loop ensures that the image is drawn atleast once, and as many times as necessary before continuing.

It looks like this could cause an infinite loop, but in practice, it is very unlikely. Considering that your drawing routine will likely run fast enough so that it runs about 60 times per second, and what causes the contents to be lost, you can see that the contents being lost during your drawing routine will be a rare occurrence. Code Example 4 shows a drawing routine.

Code Example 4

Drawing to a VolatileImage.

import java.awt.*;
import java.awt.image.VolatileImage;
 
// From the Code Example 2.
VolatileImage vimage = createVolatileImage(800, 600, Transparency.OPAQUE);
 
...
 
public void render() {
	GraphicsEnvironment ge = GraphicsEnvironment.getLocalGraphicsEnvironment();
	GraphicsConfiguration gc = ge.getDefaultScreenDevice().getDefaultConfiguration();
 
	Graphics2D g = null;
 
	do {
 
		int valid = vimage.validate(gc);
 
		if (valid == VolatileImage.IMAGE_INCOMPATIBLE) {
			vimage = createVolatileImage(width, height, transparency);
		}
 
		try {
			g = vimage.createGraphics();
 
			mySprite.draw(g); // This is assumed to be created somewhere else, and is only used as an example.
		} finally {
			// It's always best to dispose of your Graphics objects.
			g.dispose();
		}
	} while (vimage.contentsLost());	
}

VolatileImage Techniques

Using a VolatileImage isn't straight-forward, even after you make sure its content is valid. Here are some techniques have been found useful.

Loading a VolatileImage from a File

The Toolkit is old. The guys at Sun don't really want us to use it anymore, in favour of ImageIO. Unfortunately, the read() methods in ImageIO return BufferedImage. There's no way to get around that. However, you can load a file into a BufferedImage, and then draw the contents of the BufferedImage onto a VolatileImage. Code Example 5 shows how to do this.

Code Example 5

How to load a file into a VolatileImage.

import java.io.*;
import java.awt.*;
import java.awt.image.VolatileImage;
import java.awt.image.BufferedImage;
 
public VolatileImage loadFromFile(String filename) {
	GraphicsEnvironment ge = GraphicsEnvironment.getLocalGraphicsEnvironment();
	GraphicsConfiguration gc = ge.getDefaultScreenDevice().getDefaultConfiguration();
 
	// Loads the image from a file using ImageIO.
	BufferedImage bimage = ImageIO.read( new File(filename) );
 
	// From Code Example 2.
	VolatileImage vimage = createVolatileImage(bimage.getWidth(), bimage.getHeight(), Transparency.OPAQUE);
 
	Graphics2D g = null;
 
	try {
		g = vimage.createGraphics();
 
		g.drawImage(bimage,null,0,0);
	} finally {	
		// It's always best to dispose of your Graphics objects.
		g.dispose();
	}
 
	return vimage;
}

How to Make Transparent Regions in a VolatileImage

Once you try the code from Code Example 5, you might notice that you get white in the regions that are supposed to be transparent. Did you notice that the transparency parameter is Transparency.OPAQUE? Good. It still doesn't work if you try Transparency.BITMASK, does it? When validate() is called in the createVolatileImage() method, it sets the VolatileImage to a known state. This means that it will clear it to white, in most cases. In order to get the transparency, you need to clear the VolatileImage to transparent. Code Example 6 demonstrates this.

Code Example 6

Clearing a VolatileImage to transparent before drawing to it.

// This uses the same code as from Code Example 5, but replaces the try block.
try {
	g = vimage.createGraphics();
 
	// These commands cause the Graphics2D object to clear to (0,0,0,0).
	g.setComposite(AlphaComposite.Src);
	g.setColor(Color.black);
	g.clearRect(0, 0, vimage.getWidth(), vimage.getHeight()); // Clears the image.
 
	g.drawImage(bimage,null,0,0);
} finally {	
	// It's always best to dispose of your Graphics objects.
	g.dispose();
}

Translucent Images

"To turn on acceleration of translucent images in 1.4 and 1.5 (using the DirectDraw/Direct3D pipeline):

-Dsun.java2d.translaccel=true
-Dsun.java2d.ddforcevram=true //unnecessary as of 1.5 Beta 1

When both these properties are true, the Java 2D system attempts to put translucent images into VRAM and use Direct3D for rendering (compositing) them to the screen or to a VolatileImage. Only translation transforms are supported (no rotation, scaling, and so on). Before 1.5, to be accelerated a translucent image had to be created in one of the following ways:

  • GraphicsConfiguration.createCompatibleImage(w,h, Transparency.TRANSLUCENT)
  • Images loaded with Toolkit.getImage() that have a translucent color model

As of 1.5, translucent images created with a BufferedImage constructor can also be accelerated. To find out whether an image can be accelerated on a particular device, you can use the Image getCapabilities method (added in 1.5) to get an ImageCapabilities object, which you can query using the isAccelerated method. Note that a managed image gets accelerated only after a certain number of copies to the screen or to another accelerated surface.

The following code fragment illustrates the use of accelerated images. The fragment assumes that back buffer is a VolatileImage. BufferStrategy can be used as well.

Image translucentImage = gc.createCompatibleImage(w, h, Transparency.TRANSLUCENT);
//...
Graphics2D g2 = (Grahics2D)backBuffer.getGraphics();
g2.drawImage(translucentImage, x, y, null);

Compositing with extra alpha with SRC_OVER rule (which is the default Graphics2D compositing rule) is accelerated, so the following code will allow the use of hardware for compositing:

Image translucentImage = gc.createCompatibleImage(w, h, Transparency.TRANSLUCENT);
// ...
float ea = 0.5;
Graphics2D g2 = (Grahics2D)backBuffer.getGraphics();
g2.setComposite(AlphaComposite.getInstance(AlphaComposite.SRC_OVER, ea));
g2.drawImage(translucentImage, x, y, null);

Note: The translaccel property has no effect if the OpenGL pipeline is in use."[4]

The cited reference fails to mention that in order to use "-Dsun.java2d.translaccel=true", it must be used as a VM option. For example:

java -Dsun.java2d.translaccel=true MyProgram

But I Need BufferedImage.getSubImage()!

BufferedImage has a handy method, getSubImage(). Unfortunately, that's not available for VolatileImage. There are ways to get around this if you only need part of a VolatileImage though:

Method 1

You're probably loading an image from a file and then drawing to a VolatileImage, so if it is something like a character sprite, where there are multiple poses in one file, you can split it up into individual poses when you load the image, rather than taking part of the image when it's needed. An array or Vector can be used to store the individual images.

Method 2

If you have a large image that you wish draw part of to the screen, like a background, it's hard have it split into smaller pictures. Instead, you can start drawing the entire image offscreen such that the part you want ends up on the screen.

There is some inherent potential performance loss in this case, as you waste time drawing to something that isn't the screen.

More Information

References