Friday, August 15, 2014

DWA compression in OpenEXR 2.2

OpenEXR 2.2 was just released, and it features a new lossy codec created by Karl Rasche of DreamWorks Animation. Simply called DWA, it applies techniques common in lossy image compression like the discrete cosine transform and Huffman coding, but it uses them to great effect. If you can accept a little loss in your HDR images, DWA can shrink them to a fraction of their lossless size.

Like other DCT compression formats such as JPEG, DWA lets you set the amount of compression you want, so you can find a balance between loss and file size that best suits your particular task. This is a first for OpenEXR, or any other HDR format to my knowledge. The DWA Compression Level parameter is a floating point number, with higher values leading to more loss and smaller files.

The default Compression Level setting is 45.0, which creates a visually lossless image that is often less than half the size of its truly lossless Piz counterpart. In my tests, boosting that value to 100.0 still had no visible artifacts, but was further reduced to less than a third of the Piz version. Artifacts do start to appear around 200.0, but only if you zoom in to look and by that time your file is about one eighth the size of Piz. Of course, the artifacts do eventually become noticable. If you'd like to see what they look like, I've made a short video showing them—download MP4 or WebM.

(Of course, viewing DWA's lossy compression artifacts in some other lossy movie format is inherently problematic, but…)

Like other DCT codecs, re-compressing DWA over and over again does not incur much additional generational loss, so long as you use the same settings. Another thing that makes DWA significant is that it is fast, very fast. Most computers will have no problem decoding 2K frames at 24fps. The performance is aided by significant SIMD optimization, but it's very quick even without them. Bravo, Karl!

DWA in OpenEXR is actually two compression options, DWAA and DWAB. They use the same algorithm, but DWAA compresses pixels in blocks of 32 scanlines, while DWAB does 256 scanlines. Bundling more lines together lets DWAB compress more efficiently and increases performance when the entire entire image is read at once, as most programs do. Meanwhile, DWAA's smaller bundles make it more efficient for reading just a few scanlines at a time, as Nuke usually does.

I have re-compiled ProEXR 1.9 with OpenEXR 2.2 if you want to experiment with DWA a little. The Photoshop and After Effects plug-ins don't yet have a slider for setting the Compression Level, so it'll be fixed at the default value of 45.0. The Premiere plug-in does have a slider though, and it works fine in Adobe Media Encoder too.

As a new EXR codec, a file using DWA compression will not be compatible with programs that have not yet upgraded to OpenEXR 2.2, which as I write this is pretty much all of them. Some programs may take years to update. So feel free to experiment with DWA, but be careful when using it in production, making sure that your intended destination can accept it.


Anonymous said...

thanks, this is great! would you have the space to host the original exr test image sequences for more detailed analysis?

Brendan said...

I don't really have the space. I could provide the command line utility I wrote to generate them though. The source images were mostly just the sample EXR files.