PNGStore - Embedding compressed CSS & JavaScript in PNGs
By Cal Henderson, August 22nd 2010.
Updated 2010-08-24: There are now some additional results.
Alex Le used a really interesting technique (originally proposed by Jacob Seidelin) for pushing the limits of the 10k Apart Contest - storing JavaScript and CSS in PNG files. The idea is pretty simple - since the contest limits the size of the final files, you can include more code by putting text inside PNGs and taking advantage of the PNG's internal compression.
So I wondered - can this technique be used to make regular web sites faster for very slow connections?
The Variables
The main way in which the real world differs from the contest is that we care about how many bytes go over the wire, not how many are stored on disk. This means that we can use GZip compression to serve compressed text (CSS & JavaScript) anyway. If this technique is going to be any good, it needs to beat GZipped content for size.
While Alex was using a fairly straight forward PNG-8 encoding, I wondered what would happen if I tried to put more bytes per pixel - storing 3 bytes per pixel in a PNG-24 or even 4 bytes per pixel in a PNG-32. This would avoid needing the PLTE (palette) block at the start of the image.
Alex used a very tall image, one pixel wide. I seemed to remember something about image dimensions mattering for PNG size, so try wide, tall and square images and checking for differences seems like a good idea.
Since JavaScript & CSS both tend to only use 7-bit ASCII, can we compress things down either further? With 7 bits of data per character, we could pretty easily fit 8 characters into 7 bytes, by just spreading out the bytes:
11111112 22222233 33333444 44445555 55566666 66777777 78888888
With all of these changes, the issue isn't information density, but final output size. Because of the way internal PNG compression works, some kinds of data will just compress better than others.
The Implementation
I hacked together a quick image generator using PHP and GD. For various input files, create output images with a combination of shape, bit depth and encoding. It's available on GitHub. The file bake.php creates all of the test images, while unpack.htm checks that you can decode them correctly using Canvas. I calculate MD5 sums of the data returned to check that it's exactly as expected.
As further bit depth and compressed encodings are used, the image dimensions shrink. It's not practical to view the wide and tall images, since they are several thousand pixels in a single dimension. Here are some of the square image encodings of the jQuery library:
 ASCII 8bit |
 ASCII 24bit |
 ASCII 32bit |
 8-in-7 8bit |
 8-in-7 24bit |
 8-in-7 32bit |
The visual representation tells you some interesting things about the code. The 8bit images are red because they only store data in the red channel. Data is more uniformly distributed in the 8-in-7 images, since the ASCII images never set the high bit on any of the bytes, so only really use half of the possible values for each pixel (the dark ones). The 32bit images are harder to see, because the transparency makes the whole image faint.
The image dimensions are reduced as the bit depth and encoding packing are increased.
As a final step, I passed the images through Yahoo's Smush.it to remove extra cruft that's not needed to be able to decode the data inside the files.
The Results
Here are the full results for the jQuery library, in table form:
| Mode | Dimensions | Raw Size | Smushed | Notes |
|---|---|---|---|---|
| Raw | 71,807 | |||
| GZipped | 24,281 | |||
| ASCII 8bit Wide | 71807 x 1 | 25,195 | - | Failed to load in browser |
| ASCII 8bit Tall | 1 x 71807 | 29,838 | - | Failed to load in browser |
| ASCII 8bit Square | 267 x 269 | 25,826 | - | No extra compression possible |
| ASCII 24bit Wide | 23936 x 1 | 42,615 | 24,487 | Smallest, still bigger than GZip |
| ASCII 24bit Tall | 1 x 23936 | 56,433 | 34,670 | |
| ASCII 24bit Square | 154 x 156 | 64,588 | 24,847 | |
| ASCII 32bit Wide | 17952 x 1 | 55,897 | 34,149 | Failed |
| ASCII 32bit Tall | 1 x 17952 | 62,415 | 36,666 | Failed |
| ASCII 32bit Square | 133 x 135 | 67,974 | 34,482 | Failed |
| Seq8 8bit Wide | 62732 x 1 | 42,923 | 42,335 | |
| Seq8 8bit Tall | 1 x 62732 | 52,529 | 52,392 | |
| Seq8 8bit Square | 250 x 252 | 43,517 | 42,863 | |
Analysis
As expected, the image dimensions don't map directly onto final file size.
The very wide and tall 8bit images failed to load correctly. PNGs have an upper dimension limit of 65535 (0xFFFF), so for input files larger than 64k, the smaller dimension will need to be increased. This limit applies to the dimensions, not the pixels, since the square 8bit image worked fine.
The 32bit images failed to work. It's unclear if this is because PHP-GD doesn't allow them to be set correctly (it only accepts values on a 0-127 scale instead of 0-255) or if there's something special about extracting the values in canvas (compositing mode seems to be set correctly). Regardless, these images came out larger than the 8bit or 24bit versions.
The 8-in-7 packing produced the images with the smallest dimensions, but the largest final files. This is because the highly uniform data is the hardest to compress; the files are the smallest if you turn off PNG compression.
Wide images always did better than tall ones, with square ones a little bit behind wide ones. For large files, square makes more sense since it doesn't break at large input. I suspect this might be related to the way pixels are stored in the IDAT block and perhaps making sure lines are aligned to 8 byte boundaries might be optimal.
The smallest output file was the Wide 24bit ASCII version. However, it was still a couple of hundred bytes larger than the GZipped original and that doesn't take into account the JavaScript needed to extract the code from the image. PNGs use Deflate/zLib compression which works differently to GZip, but it seems as though there is no big saving to be made here.
I additionally tried each PNG with every different combination of the PNG compression filters - Sub, Up, Average and Paeth. The results were for some smaller file sizes for the larger files, but no improvement at the low end.
Conclusion
Storing CSS & JavaScript data in PNGs is a neat hack, but GZipping your source files will get you a larger improvement and greatly simplify your production build process. But still, pretty cool.
Updated 2010-08-24: There are now some additional results.
Post-Script
I did this late one night, so there are probably mistakes. If you spot any of them, drop me an email and teach me: cal [at] iamcal.com
All of the source code can be downloaded from the GitHub repo. Bonus points for fixing the 32bit image encoding. This whole thing was inspired by Alex Le.
Copyright © 2010 Cal Henderson.
The text of this article is all rights reserved. No part of these publications shall be reproduced, stored in a retrieval system, or transmitted by any means - electronic, mechanical, photocopying, recording or otherwise - without written permission from the publisher, except for the inclusion of brief quotations in a review or academic work.
All source code in this article is licensed under a Creative Commons Attribution-ShareAlike 3.0 License. That means you can copy it and use it (even commerically), but you can't sell it and you must use attribution.
37 people have commented
[1] according to <a href="http://www.yuiblog.com/blog/2008/02/06/iphone-cacheability/">www.yuiblog.com/blog/2008/02/06/iphone-cacheability</a> at least
Mark: Very interesting! I wonder if the unpacking is fast enough to use in iPhone WebKit. Need to do some benchmarking.
Still bigger than gzip. I compressed jquery-1.4.2.min.js with GZRepack (kzip+7z+deflopt) and got it down to 23,866 bytes.
By the way, running Deflopt over that PNG produced by PNGOUT removes another 10 bytes. But then it's still 198 bytes bigger.
I seem to remember Steve Souders did a list somewhere.
Patches for any of the outstanding issues would be great.
>is the scenarios in which gzip doesn't work [...]
Ye, one should compare it with Packer for example. The PNG method should be smaller and the decoding should be also quicker (since the decompression itself is handled by a highly optimized native library).
so, to summarize :
- there is no size gain over gzip
- it can take a lot of CPU
- if CSS is encoded, it spares one HTTP request
- it could be used where gzip is not used : some cheap hosting company and the 15% of people that come to our sites without Gzip headers
- there is bugs to be aware of
what if we encode with base64 images in CSS, then encode the CSS in the PNG ? :)
do we have a whole site in one single file ?
My only suggestion would be to test this stuff:
a) other JS files, you shouldn't take conclusions from just one single case.
b) brute force test all widths for the best candidate (either 1b gray or 24b). The jQuery test didn't trigger any PNG filter on PNGOut, but maybe other files or widths can benefit from filtering a few lines.
@ChriS:
PNG recompressors optimize the palettes by default, so if only 1 color out of the 255 entries is used, then the output file will only have that one color.
Recompressed: <a href="http://i.imgur.com/KZZv5.png">i.imgur.com/KZZv5.png</a> (23,722 bytes)
However, PNG filters are 2D compressors that work for certain patterns of pixels, so changing the file dimensions is likely to yield a couple of combinations that win out. However, that requires you generate every possible size of PNG (about 70k versions for some of these encodings) optimize them all individually and see what you get.
But in situations where you can't use gzip or want to do something else fancy, it gets pretty close to gzip in size.
Consider having a sprite png with your site's pixel graphics (icons, logos etc) and then appending to that your site's JS and CSS.
You will be saving considerable amount of resources (esp. time) from just the HTTP headers not sent due to them all being shipped as one.
It would be interesting to actually benchmark this.
Really looking forward to when Chrome gets WebP with the new lossless format (recently introduced) and trying this.....the lossless mode gets better compression than a png put through pngout/crush.
Regards
Really looking forward to when Chrome gets WebP with the new lossless format (recently introduced) and trying this.....the lossless mode gets better compression than a png put through pngout/crush.
Consider having a sprite png with your site's pixel graphics (icons, logos etc) and then appending to that your site's JS and CSS.
You will be saving considerable amount of resources (esp. time) from just the HTTP headers not sent due to them all being shipped as one.
It would be interesting to actually benchmark this.
<...>april fool day pranks<...>
<...>april fool day 2016 images<...>
Good and full of knowledge
Leave your own comment
Comments have been disabled
# August 23, 2010 - 2:48 am PST