Ben Karel
23 March 2007
email: {my first name}%{this domain}

Because Bandwidth Is Expensive:

Adding JPEG2000 support to Firefox

Synopsis

JPEG2000 is a standard for compressing photographic images. It produces higher-quality images at smaller filesizes than the current web standard, JPEG, but the only browser to offer native support is Safari. Because of this, it is unfeasible for web sites to use JPEG2000, since they must assume their images would not be rendered.

I propose to write a new extension that will add JPEG2000 support to Firefox. Hopefully, if accepted, this will be the first step towards seeing ubiquitous support for JPEG2000 on the web.

Incentives

Why would anyone want support for inline viewing of JPEG 2000 images in their web browser? More to the point, why would Mozilla want inline rendering of such images in Firefox? I assert that all parties involved gain benefits from such a feature.

For web sites like Google:

It's a fact of life for a company that lives on the web: bandwidth is expensive. Text is cheap; images are not. Here's a back-of-the-envelope calculation:

First, let us extrapolate from recent history with PNG transparency, tabs, and CSS support and assume that the Internet Explorer team at Microsoft tends to add those features already enjoying widespread support in the most popular competing browser. Therefore, JPEG 2000 support in IE will not appear until Firefox supports it.

Suppose that Google pays, oh, fifty cents per gigabyte for high-speed, high-reliability bandwidth. How much might Google save from ubiquitous browser support for rendering inline JPEG2000 images? Let's look at just ONE of Google's many online assets -- Google Images. An average results page from an Images search returns a page with 18 thumbnails totaling somewhere around 60 KB. Let's say Images serves up a page like that 15 million times per day around the world, 887 gigabytes. But some of those people will also click to see the cached full-size versions. Rounding off, let's say that Google Images serves a terabyte every day of JPEG thumbnails and images: $182,700 per year.

How much does Google save if it can serve JPEG2000 images instead? For the thumbnails, savings will be minimal -- let's say 10%, which reduces bandwidth by 90 GB/day. For the cached versions of the full images, JPEG2000 can provide greater compression. At 50% savings over JPEG, that's another sixty gigs a day saved. In total, we reduce Images' bandwidth bill by 15%, for a savings of $27,000 per year.

For Mozilla, the benefits are less concrete, but they're still real. There are factors of parity and popularity.

Safari already does it thanks to QuickTime integration, and it looks good if Firefox isn't missing cutting-edge features offered by other browsers.
As shown in the link, JPEG2000 support is one of the most-voted-on bugs in Bugzilla. It would be nice to resolve that seven-year-old bug.
There is also further validation of the extensibility of the extensions model used by Firefox: if something that seems as fundamental to the browser as an image decoder can be implemented as an extension, well, that's a pretty neat trick!
For pavlov specifically, I'll add that this might also set a good precedent for the libmng people. If they see that JPEG2000 can be written as an extension, perhaps they'll do the same and stop bugging you to re-add it to the trunk. Just a thought...

The web community at large also benefits. JPEG2000 gives higher-quality images in smaller files. Thus, webmasters can afford to make their images at close-to-lossless quality without increasing the size of the images they serve. Or, they can maintain roughly the same image quality at a significantly reduced filesize. Either way, the experience for the end user is better, because they get faster downloads, prettier pictures, or both.

Deliverables

At the end of the summer, I will have created a new extension to allow Firefox 3 (and possibly 2) to render JPEG2000 images as part of any webpage.

Details

See the Schedule section below for how I expect work to proceed on the extension. As I work, I will maintain a public Subversion repository with checkins several times per week. I'll also write roughly-weekly status updates and post them online for the world to see.

Links supporting the project:

Schedule

Week by week schedule:

Begin converting existing decoder to extension
Finish first extension (using libjasper)
Optimize libjasper code for small executable size
Write a second extension using the OpenJPEG library
Test both extensions for code size, speed, and correctness of rendering on Linux, Windows XP, and Mac OS X
Consult with Stuart Parmenter to determine whether the code will be adopted by Mozilla or remain a third-party extension. In case of the former, prepare code for eventually being checked in to the trunk.
In case of extra time, I will explore the possibility of creating an image-decoder finding service, to mirror the existing plugin finder service. This would allow for JPEG2000 support to be delivered on an as-needed basis in Firefox 3. I will also investigate the possibility of supporting Firefox 2 and possibly Firefox 1.5 in addition to the still-alpha Firefox 3.
Wrapup: submit best extension to addons.mozilla.org, write mentor evaluation

I won't have many other commitments over the summer. I may take one class at the University of Delaware, which would require about three hours a week of my time -- not enough to impact productivity on Summer of Coding.

Biography

Eligibility

I am finishing my second year of undergraduate study in Computer Science at the University of Delaware.

Experience

From 2004-2005, I spent hundreds of hours working on a sadly-unreleased update to the popular Adblock extension. From that time, I gained significant experience with the development process for Firefox extensions. In addition, as a warmup for this summer's project, I created a mostly-functional libpr0n jp2 decoder for trunk builds of Firefox. (screenshot, patch).

Reasoning

I've chosen this project for a number of reasons. One, I think it's the appropriate scope for a Summer of Code project -- not too big as to be unrealistic, but not so small as to be trivial. Two, I have a special place in my heart for Firefox, because of the massive potential it has for making regular people's lives better on a day-to-day basis. Three, I read CodingHorror.

I think I should be chosen for this project because I have already gained experience with the relevant portions of Mozilla's source code and architecture. Speaking from experience, XPCOM isn't the sort of thing you can just walk up to and know how to use right off the bat. QueryInterface is mysterious black magic when you first see it used. But, thanks to my stint on Adblock and pre-SoC work, I've scaled the first hump in familiarizing myself with Firefox's code and build process, which is one less thing I'll need to do to get started working this summer.

Also, I emailed Stuart Parmenter, showing him the above screenshot, and he said "It sounds like you've done a lot of great work so far and I'd love to see support for it as an extension." Considering that Stuart is the owner of ImageLib and co-owner of the gfx module, that counts for something, I think.

I am not applying for any other Summer of Code projects.