BD-J Images and Memory Management

Introduction

In an embedded device like a Blu-ray player, careful memory management by applications is essential for reliability. In a typical BD-J xlet, images are often the largest items in memory. The java.awt APIs for loading and disposing of images are a little bit quirky, so their correct use isn't always obvious. The need to explicitly manage image memory isn't always apparent to Java developers, particularly those who come from a desktop background.

Loading images and other issues are discussed in the HD Cookbook starting on page 17-9, in the sections "Working with Images" and "Combining Small Images." The more advanced subject of freeing image memory is not discussed in the cookbook. In this wiki entry, we explore best practices and APIs for managing image memory in an effective manner.

The Problem

Generally speaking, in Java the memory associated with objects is freed automatically, by the garbage collector. The same is true for image objects, but on an embedded device effectively freeing image memory presents special difficulties for the platform implementor. Images are usually stored in native memory (e.g. using malloc()). In a good quality implementation, garbage collection will be triggered when image memory begins running out, even if it stored outside the Java heap, e.g. in malloc space. This can be difficult to achieve, so in practice implementation quality varies. On some players, it's possible you might see failure due to lack of image buffer memory, even if that memory could be made available by GCing unreferenced images and reclaiming/compacting image buffer memory.

The Solution

An application can help the platform greatly by explicitly destroying images, thus freeing the native resources associated with those images. The API for doing this is a little bit confusing, partly because such explicit deallocation of images isn't necessary on a desktop platform with generous amounts of memory. Explicit deallocation of images wasn't supposed to be necessary on embedded devices, but pragmatically speaking, application authors really should do everything they can to help the platform.

Normal Images

The class java.awt.Image has a method called flush() defined. For a normal image, like a PNG image loaded from a disc, this method frees the pixmap data, so it works great for cleaning up native memory. As an application author, you might need to do reference-counting to ensure that you always flush images that are no longer used, and you never flush images that are stil used. You should also be sure to create images with the createImage() family of methods defined on java.awt.Toolkit, and not the getImage() methods. That's because getImage() tries to share image instances. getImage() can even result in two xlets sharing the same instance of an image, so that flushing an image in one xlet might corrupt the state of another xlet!

In the HD cookbook library, the class ManagedImage in com.hdcookbook.grin.util handles reference counting, and flushing images when not in use.

Images Buffers

Image buffers present a special problem. In the original design of Image, flush() was specified to flush only cached information that can be reconstructed. With a normal image, you can throw away the pixmap, and if you need the image again, you can always decode the .png file again (assuming, of course, that the data source of the .png file is available, but the image API makes this assumption). With an image buffer like java.awt.image.BufferedImage, however, the pixmap is the only copy of the image data. For that reason, flush() on a BufferedImage is effectively a NOP - it does not free the large native pixmap buffer.

In java.awt from Personal Basis Profile, garbage collection will of course eventually free a BufferedImage, but there is no API that immediately frees the pixmap buffer. This is unfortunate. This is solved for BD-J and other GEM platforms with a different buffered image class called DVBBufferedImage in the package org.dvb.ui. DVBBufferedImage defines a new dispose() method that frees the pixmap. Note that DVBBufferedImage also has the flush() method inherited from java.awt.Image, but this method does not flush the pixmap. That's because the contract of flush() only allows flushing cached data that can be reconstructed.

In the HD Cookbook menu application, we use DVBBufferedImage in this way. See AssetFinder in com.hdcookbook.util and MenuAssetFinder in com.hdcookbook.bookmenu.menu.MenuAssetFinder to see how this is done. It's a little more complicated than just using DVBBufferedImage because we want our code to also work on desktop java, which doesn't include this class. For this reason we go through the AssetFinder to delegate image buffer operations to a place that can be overridden in the BD-J xlet. The default DVBBufferedImage image type is functionally equivalent to what you get from the AWT createCompatibleImage() method defined on java.awt.GraphicsConfiguration.

Note that DVBBufferedImage was originally written for pJava (which didn't have Graphics2D), but it was written to coexist harmoniously with PBP (which does). It is guaranteed that the results of calling DVBBufferedImage.createGraphics() is an instanceof Graphics2D. This means that you can directly cast this value to the type java.awt.Graphics2D. This is specified in MHP, which requires that all instances of DVBGraphics be instances of Graphics2D. The specification for this is in the class description of DVBGraphics. To cast the Graphics object, the following should work:

    import java.awt.graphics2D;
    import org.dvb.DVBBufferedImage;
    ...
        DVBBufferedImage buffer;
        buffer = new DVBBufferedImage(1920, 1080);
        Graphics2D g = (Graphics2D) buffer.createGraphics();

        DVBBufferedImage buffer;
        buffer = new DVBBufferedImage(1920, 1080);
        Object o = buffer.createGraphics();
        Graphics2D g = (Graphics2D) o;

Summary

When writing an application, be sure that you

• Call flush() on normal images when you're done with them
• Use DVBBufferedImage instead of AWT's BufferedImage, so you can call dispose() on it when you're done

By doing this, you help the player free image memory as soon as possible. Since the image pixmap memory is typically native, this helps the player and the application avoid running out of memory.

-- Main.billf - 25 Apr 2008
-- Main.billf - 26 Jun 2008