OVERHEAD

Log entry (25/09/2023)

As the shimmering veils of teal and violet parted, a horizon peaked through the starry clouds, and I returned a curious glare. This new planet, dubbed “Zopfli”, appeared to me like a buoy in rough waters, promising progress safe and certain, but here in the Oculorum nebula, sight should scarcely be trusted-

So we’re doing this intro-thing every time? Fine.

Modified “scanners”

Since the last post, I’ve discovered new options for the oxipng and cwebp tools which further optimise the output:

• cwebp -z 9 -mt -alpha_filter best “$original_image” -o “$path_to_cwebp_output”
  • In testing, the -alpha_filter best option made no difference to the output size, however, some people have reported that it makes a difference and, given there doesn’t seem to be a risk of the file size increasing from using this option, I’ve added it to the benchmarks.
• oxipng --opt max --strip all --alpha “$original_image” --out “$path_to_oxi_output”
  • Similarly, the --alpha option never increased the file size, so its now included as well, albeit this option tends to shrink the output size more frequently than the -alpha_filter best option does.

And finally, per the name-sake of this blog post:

• oxipng --opt max --strip all --alpha --zopfli “$original_image” --out “$path_to_oxi_zf_output”
  • The --zopfli option makes a significant difference to the output file size, and at least one other variable that we’ll get to later, up to the point that this command now has its own separate entry in the benchmarks: “OxiPNG-ZF”.

Zopfli is a very effective, but comparatively slow compression algorithm developed by Google, whose compression stream, the series of bytes that are output by the algorithm, is compatible with any DEFLATE decompressor. And, since the PNG format uses the DEFLATE algorithm internally, Zopfli can be used in its place for more compact PNG images while staying fully spec compliant.

Data analysis

Typical case

Let’s look at the two examples from the previous blog post, now with these modified options and the new --zopfli entry:

Filename:		'Commercial Break.png'
Original Filesize:	3733779 bytes
PNGQuant Filesize:	3733779 bytes
OxiPNG Filesize:	3098589 bytes
OxiPNG-ZF Filesize:	3063186 bytes
Oxi-Quant Filesize:	3098589 bytes
CWebP Filesize:		2222816 bytes
Smallest encoding:	2222816 bytes, CWebP

For images like ‘Commercial Break.png’, which don’t use transparency, the new alpha-related options predictably did nothing to the output file size. But Zopfli makes a noticeable difference! (well, ~1.2% smaller, but I’m aiming for perfection here, so these savings are important)

And for pixel art images, the results now look like this:

Filename:		'post-10k.png'
Original Filesize:	7418 bytes
PNGQuant Filesize:	5540 bytes
OxiPNG Filesize:	5470 bytes
OxiPNG-ZF Filesize:	5279 bytes
Oxi-Quant Filesize:	5470 bytes
CWebP Filesize:		5122 bytes
Smallest encoding:	5122 bytes, CWebP

This image does use transparency, but, --alpha isn’t guaranteed to make a difference to the output file size regardless. And again, Zopfli saves some bytes, but cwebp is still the best option in the vast majority of cases.

…

It happened again, didn’t it?

Outliers

Filename:		'series-deciphering_hugo.png'
Original Filesize:	3819 bytes
PNGQuant Filesize:	1990 bytes
OxiPNG Filesize:	1885 bytes
OxiPNG-ZF Filesize:	1850 bytes
Oxi-Quant Filesize:	1885 bytes
CWebP Filesize:		1918 bytes
Smallest encoding:	1850 bytes, OxiPNG-ZF

…sigh. Well, it’s basically the same situation as last time, with a variant of OxiPNG being the smallest encoding, only this time Zopfli furthered cemented the victory.

Filename:		'tag-css.png'
Original Filesize:	4780 bytes
PNGQuant Filesize:	2474 bytes
OxiPNG Filesize:	2258 bytes
OxiPNG-ZF Filesize:	2256 bytes
Oxi-Quant Filesize:	2256 bytes
CWebP Filesize:		2054 bytes
Smallest encoding:	2054 bytes, CWebP

Gasp. The “OxiPNG-ZF” output matched the “Oxi-Quant” output in size! Does that mean-

Filename:		'Invert.png'
Original Filesize:	1542935 bytes
PNGQuant Filesize:	1586364 bytes
OxiPNG Filesize:	1506742 bytes
OxiPNG-ZF Filesize:	1502437 bytes
Oxi-Quant Filesize:	1502382 bytes
CWebP Filesize:		1356018 bytes
Smallest encoding:	1356018 bytes, CWebP

OH COME ON! “Oxi-Quant” can beat Zopfli compression!?

If you don’t know what the “Oxi-Quant” entry is, see the ‘Data analysis’ section of the Captain’s log post for more information.

Fine, I guess we can at least always use --zopfli when using OxiPNG then-

Filename:		'tag-hugo.png'
Original Filesize:	998 bytes
PNGQuant Filesize:	517 bytes
OxiPNG Filesize:	482 bytes
OxiPNG-ZF Filesize:	483 bytes
Oxi-Quant Filesize:	482 bytes
CWebP Filesize:		418 bytes
Smallest encoding:	418 bytes, CWebP

…

So, here we are again; we need to use all three tools, a combo, and now the Zopfli variant of OxiPNG on top of that, to find the smallest possible image encoding. Extra unfortunately, Zopfli is VERY COMPUTATIONALLY EXPENSIVE!

real	52m55.662s
user	596m33.458s
sys	0m37.799s

The real time is the actual amount of time that passed, with user representing the amount of time across all threads that was spent running the script.

Yikes. Thankfully, it doesn’t have to take this long, because among the images I’ve tested, there’s a pattern that all of them follow:

• If the input image is large, cwebp always yields the smallest output, and by a large margin.
• If the input image is small, cwebp only yields the smallest output by a small margin compared to OxiPNG and its variants, and even yields suboptimal output file sizes on infrequent occasions.

To take advantage of this trend, I’ve set the benchmarking script to only test the “OxiPNG-ZF” output for input images that are less than or equal to 16,384 bytes in size, which is roughly four times larger than the largest input image where cwebp wasn’t the winner (aka. ‘series-deciphering_hugo.png’), which greatly improves the runtime:

real	0m40.114s
user	6m33.520s
sys	0m1.833s

That’s much faster! And if the remaining OxiPNG variants yield output file sizes which are uncomfortably close to the “CWebP” output file size, I can always check those images manually just in case of even more edge-cases.

I still need three goddamn tools to find the smallest output, but an improvement is an improvement.

An unexpected find

Last time, I had said that for the gif2webp tool the output was substantially larger than what gifsicle or gif2apng could achieve, but as it turns out, this isn’t always true.

The following commands were used to obtain the results you’re about to see:

• gifsicle --optimize=3 --optimize=keep-empty “$original_image” -o “$path_to_gifsicle_output”
  • Link to ‘Gifsicle’ webpage.
• gif2apng -z2 -i10 “$original_image” “$path_to_apng_output”
  • Link to gif2apng webpage.
• gif2webp -quiet -mt -min_size -m 6 -metadata none “$original_image” -o “$path_to_webp_output”
  • Link to ’libwebp’ webpage.

Additionally, the exiftool -all= “$path_to_apng_output” command saved 36 bytes for the original GIF2APNG output, and this optimised file size is what is used below. See the ‘EXIF data’ section of the Crunch animated pixel art post for more information about the tool.

Filename:		'tag-scripting.gif'
Original Filesize:	18345 bytes
Gifsicle Filesize:	10532 bytes
GIF2APNG Filesize:	6144 bytes
GIF2WebP Filesize:	5322 bytes
Smallest encoding:	5322 bytes, GIF2WebP

My theory is that GIF input images that are best encoded in the APNG format can typically be further shrunked down when encoded as animated WebP images, however, if the smallest encoding of a GIF is still a GIF, this will rarely be the case.

I’ll need to gather a wider set of GIFs to confirm this though, as the theory is based solely on this one result.

Additional notes

• From experimenting with various values for the --filters and --fast options, I’ve found that the most effective way of finding the smallest possible encoding of an image, with no concerns for the extra time spent, is to use the --opt max option’s defaults of trying all filters in parallel (aka. --filters 0-9) without trying to heuristically eliminate filters ahead of time (aka. No --fast option).
• OxiPNG is due for a release at some point, which will include many new optimisations and tweaks both from itself and also its updated dependencies, and this will almost certainly change the results I’ve gathered so far (hopefully eliminating the need for pngquant once and for all).