Instead of using a new PNG standard, I'd still rather use JPEG XL just because it has progressive decoding. And you know, whilst looking like png, being as small as webp, supporting HDR and animations, and having even faster decoding speed.
https://dennisforbes.ca/articles/jpegxl_just_won_the_image_w...
JPEG XL definitely has advantages over PNG but there is one serious seemingly insurmountable obstacle:
Nothing really supports it. Latest Safari at least has support for it not feature-flagged or anything, but it doesn't support JPEG XL animations.
To be fair, nothing supports a theoretical PNG with Zstandard compression either. While that would be an obstacle to using PNG with Zstandard for a while, I kinda suspect it wouldn't be that long of a wait because many things that support PNG today also support Zstandard anyways, so it's not a huge leap for them to add Zstandard support to their PNG codecs. Adding JPEG-XL support is a relatively bigger ticket that has struggled to cross the finish line.
The thing I'm really surprised about is that you still can't use arithmetic coding with JPEG. I think the original reason is due to patents, but I don't think there have been active patents around that in years now.
Every new image codec faces this challenge. PNG + Zstandard would look similar. The ones that succeeded have managed it by piggybacking off a video codec, like https://caniuse.com/avif.
It is possible to polyfill an image format, this was done with FLIF¹². Not that it mean FLIF got the traction required to be used much anywhere outside its own demos…
It is also possible to detect support and provide different formats (so those supporting a new format get the benefit of small data transfer or other features) though this doesn't happen as it isn't usually an issue enough to warrant the extra complication.
----
[1] Main info: https://flif.info/
[2] Demo with polyfill: https://uprootlabs.github.io/poly-flif/
Any polyfill requires JavaScript which is a dealbreaker for something as critical as image display, IMO.
Would be interesting if you could provide a decoder for <picture> tags to change the formats it supports but I don't see how you could do that without the browser first downloading the PNG/JPEG version first, thus negating any bandwidth benefits.
Depending on the site it might be practical to detect JS on first request and set a cookie to indicate that the new format (and polyfill) can be sent on subsequent requests instead of the more common format.
Or for a compiled-to-static site just use <NOSCRIPT> to let those with no JS enabled to go off to the version compiled without support/need for such things.
Why would PNG + ZStandard have a harder time than AVIF? In practice, AVIF needs more new code than PNG + ZStandard would.
I'm just guessing, but bumping a library version to include new code cam integrating a separate library might be the differentiating factor.
The zstd library is already included by most major browsers since it is a supported content encoding. Though I guess that does leave out Safari, but Safari should probably support Zstd for that, too. (I would've preferred that over Brotli, but oh well.)
Btw, could you 'just' use no compression on this level in the PNG, and let the transport compression handle it?
So on paper (and on disk) your PNG would be larger, but the number of bits transmitted would be almost the same as using Zstd?
EDIT: similarly, your filesystem could handle the on-disk compression.
This might work for something like PNG, but would work less well for something like JPG, where the compression part is much more domain specific to image data (as far as I am aware).
If there is a particular reason why that wouldn't work, I'm not aware of it. Seems like you would eat a very tiny cost for deflate literal overhead (a few bytes per 65,535 bytes of literal data?) but maybe you would wind up saving a few bytes from also compressing the headers.
5 bytes per block or 0.000076 overhead.
> but there is one serious seemingly insurmountable obstacle
It can be surmounted with WebAssembly: https://github.com/niutech/jxl.js/
Single thread demo: https://niutech.github.io/jxl.js/
Multithread demo: https://niutech.github.io/jxl.js/multithread/
Maybe for websites like Instagram that consist primarily of images. For everywhere else you have to amortize the cost of the download over the savings for the number of images in an average browsing session, as browsers segment the cache so you can't assume it will be available locally hot.
Actually I wonder, why in general not more decoders are just put into webassembly and are actually kept 'hot' on demand. Couldn't this also be an extension? Wouldn't that reduce the attack surface? I remember a time when most video and flash was a plugin. People would download the stuff. On the other hand using a public CDN at least would keep the traffic down for the one hosting the website.
As I understand it JPEG XL has a lot of interest in medical imaging and is coming to DICOM. After it's in DICOM, whichever browser supports it best will rule hospitals.
Ha, yeah right, hospitals are still running IE11 in some places in the US
Yes, you're right. It'd be lovely to not need to install a DICOM viewer plugin thing.
It's better when the way it works is "this is format is good, therefore we will support it" rather than "people support this format, therefore it is good".
> I kinda suspect it wouldn't be that long of a wait
Yeah... guess again. It took Chrome 13 years to support animated PNG - the last major change to PNG.
APNG wasn't part of PNG itself until very recently, so I'd argue it's kind-of neither here nor there.
Maybe they were focused on Webp?
>he thing I'm really surprised about is that you still can't use arithmetic coding with JPEG
Or AVC YUV44 with Firefox (https://bugzilla.mozilla.org/show_bug.cgi?id=1368063). Fortunately, AV1 YUV444 seems to be supported.
That's because people have allowed the accumulation of power and control by Big Tech. Features in and capabilities of end user operating systems and browsers are gate kept by a handful of people in Big Tech. There is no free market there. Winners are picked by politics, not merit. Switching costs are extreme due to vendor lock in and carefully engineered friction.
The justification for WebP in Chrome over JPEG-XL was pure hand waving nonsense not technical merit. The reality is they would not dare cede any control or influence to the JPEG-XL working group.
Hell the EU is CONSIDERING mandatory attestation driven by whitelisted signed phone firmwares for certain essential activities. Freedom of choice is an illusion.
Webp is a lot older than jpg xl, right?
It was behind a feature flag and then removed? I guess that's where the skepticism comes from
It's also because supporting features is work that takes time away from other bug fixes and other features
> The thing I'm really surprised about is that you still can't use arithmetic coding with JPEG.
I was under the impression libjpeg added support in 2009 (in v7). I'd assume most things support it by now.
Believe it or not, last I checked, many browsers and some other software (file managers, etc.) still couldn't do anything with JPEG files that have arithmetic coding. Apparently, although I haven't tried this myself, Adobe Photoshop also specifically doesn't support it.
Arithmetic coding decodes 1 bit at a time, usually in such a way that you can’t do two bits or more with SIMD instructions. So it will be slow and energy inefficient.
Deompression is limited by memory bandwidth IME, which means that more efficient compression is (almost) always more power-efficient too.
(I don't have numbers for this, but it was generally agreed by the x264 team at one point.)
this isn't necessarily true. zstd uses an ans which is a type of arithmetic coding which is very efficient to decode
Nice to learn about. It’s good to know the field has progressed, however the context focused on JPEG, where my point does apply.
JPEG XL support will probably resemble JPEG 2000 support after enough time has passed:
Except JXL has actual value unlike J2K which wasn't that much more efficient than JPEG and much slower.
So far, it is following the same pattern. Safari adopts it, no one else does and then one day, Safari drops it. It is currently on step 2. When step 3 occurs, the cycle will be complete.
> Nothing really supports it.
Everything supports it, except web browsers.
JPEG-XL is supported by a lot of the most important parts of the ecosystem (image editors and the major desktop operating systems) but it is a long way away from "everything". Browsers are the most major omission, but given their relative importance here it is not a small one. JPEG-XL is dead in the water until that problem can be resolved.
If Firefox is anything to go off of, the most rational explanation here seems to just be that adding a >100,000 line multi-threaded C++ codebase as a dependency for something that parses untrusted user inputs in a critical context like a web browser is undesirable at this point in the game (other codecs remain a liability but at least have seen extensive battle-testing and fuzzing over the years.) I reckon this is probably the main reason why there has been limited adoption so far. Apple seems not to mind too much, but I am guessing they've just put so much into sandboxing Webkit and image codecs already that they are relatively less concerned with whether or not there are memory safety issues in the codec... but that's just a guess.
Apple also adopted JPEG-XL across their entire software stack. It's supported throughout the OS, and by pretty much every application they develop, so I'm guessing they sunk a fair bit of time/money into hardening their codec
On Linux the browser can and should link dynamically against the system library for image formats.
> >100,000 line multi-threaded C++
W. T. F. Yeah, if this is the state of the reference implementation, then I'm against JPEG-XL just on moral grounds.
Only because it's both the reference encoder and decoder, and the encoder tends to be a lot more complex than the decoder. (Source: I have developed a partial JPEG XL decoder in the past, and it was <10K lines of C code.)
> reference
They aren't going to give you two problems to solve/consider: clever code and novel design.
webp still got a vulnerability
You can use a polyfill.
I don't like progressive decoding. Lots of people don't realize that it's a thing, and complain that my photo is blurry when it simply hasn't loaded fully yet. If it just loaded normally from top to bottom, it would be obvious whether it has loaded or not, and people will be able to more accurately judge the quality of the image. That's why I always save my JPEGs as baseline encoding.
Is there no good progress indicator that gets you the best of both worlds - instant image recognition and the ability to wait and get better quality?
Web browsers already have code in place for webp (lossless,vp8) and avif (av1, which also supports animation), as well as classic jpeg and png, and maybe also HEIC (hevc/h265)... what benefit do we have by adding yet another file format if all the use cases are already covered by the existing formats? That said, I do like JPEG-XL, but I also kind of understand the hesitation to adopt it too. I imagine if Apple's push for it continues, then it is just a matter of time to get supported more broadly in Chrome etc.
Avif is cute but using that as an excuse to not add jxl is a travesty. At the time either one of those could have been added, jxl should have been the choice.
The biggest benefit is that it's actually designed as an image format. All the video offshoots have massive compromises made so they can be decoded in 15 milliseconds in hardware.
The ability to shrink old jpegs with zero generation loss is pretty good too.
The benefits are better quality, higher speed, and more features like progressive decoding. JXL is a single multi-trick pony unlike the others
Good summary https://cloudinary.com/blog/time_for_next_gen_codecs_to_deth...
Doesn't PNG have progressive decoding? I.e. adam7 algorithm
It does, using Adam7: https://en.wikipedia.org/wiki/Adam7_algorithm
The recently released PNG 3 also supports HDR and animations: https://www.w3.org/TR/png-3/
> The recently released PNG 3 also supports HDR and animations: https://www.w3.org/TR/png-3/
APNG isn't recent so much as the specs were merged together. APNG will be 21 years old in a few weeks.
True, but https://news.ycombinator.com/item?id=44802079 presumably holds the opinion that APNG != PNG, so I mentioned PNG 3 to counteract that. Animated PNGs being officially PNG is recent.
Adam7 is interlacing, not progressive decoding (i.e. it cannot be used to selectively decode a part of the image). It also interacts extremely poorly with compression; there is no good reason to ever use it.
Comparison of "zpng" (PNG wth zstd) and WebP lossless, with current PNG. From https://github.com/WangXuan95/Image-Compression-Benchmark :
Compressed format Compressed size (bytes) Compress Time Decompress Time
WEBP (lossless m5) 1,475,908,700 1,112 49
WEBP (lossless m1) 1,496,478,650 720 37
ZPNG (-19) 1,703,197,687 1,529 20
ZPNG 1,755,786,378 26 24
PNG (optipng -o5) 1,899,273,578 27,680 26
PNG (optipng -o2) 1,905,215,734 4,395 27
PNG (optimize=True) 1,935,713,540 1,120 29
PNG (optimize=False) 2,003,016,524 335 34
"Only slightly faster in decompression time."
m5 vs -19 is nearly 2.5x faster to decompress; given that most data is decompressed many many more times (often thousands or millions of times more, often by devices running on small batteries) than it is compressed, that's an enormous win, not "only slightly faster".
The way in which it might not be worth it is the larger size, which is a real drawback.
The difference is barely noticeable in real-world cases, in terms of performance or battery. Decoding images is a small part of loading an entire webpage from the internet. And transferring data isn't free either, so any benefits need to be offset against the larger file size and increased network usage.
When you talk about images over HTTP, you need to consider most web servers and browsers already support zstd compression on the transport, so the potential bandwidth win provided by zstd is already being made use of today.
I'm not sure how that's relevant for a new "ZPNG" format vs. lossless WebP?
you have to do the math - do you have more bandwidth or storage or cpu?
Not related to images, but I remember compressing packages of executables and zstd was a clear winner over other compression standards.
Some compression algorithms can run in parallel, and on a system with lots of cpus it can be a big factor.
Win how?
More efficiency will inevitably only lead to increased usage of the CPU and in turn batteries draining faster.
So someone is going to load 2.5x as many images because it can be decoded 2.5x faster? The paradox isn't a law of physics, it's an interesting observation about markets. (If this was a joke it was too subtle for me)
Might as well just shoot yourself if that's how you look at improvements. The only way to do something good it to stop existing. (this is a general statement, not aimed at you or anyone in particular)
Am I reading those numbers right? That's like 25x faster compression than WEBP-M1, there's probably a use case for that.
The numbers seem small enough that it will rarely matter, but I suppose there might be a use case somewhere?
But lets be real here: this is basically just a new image format. With more code to maintain, fresh new exciting zero-days, and all of that. You need a strong use case to justify that, and "already fast encode is now faster" is probably not it.
I don’t think it’s quite as bad, though? It’s using a known compression library that (from reading other comments) has seen use and testing. The rest of PNG would remain unchanged, as the decompression format is a plugin.
I know it needs to be battle tested as a single entity but it’s not the same as writing a new image format from scratch.
Considering both zstandard and PNG are already web facing technologies, would the combination of both really increase the attack surface?
And compresses significantly faster than regular png too.
That’s the major advantage of zstd, fast compression. Not particularly relevant in web use cases, but would be great for saving screenshots.
Is webp really losses here? As far as i remember its capped at 4:2:0 and can't do 4:4:4 files without loosing some of the color data
I looked it up and lossy mode only supports 4:2:0 but the lossless mode uses RGBA.
The developer who asked for the faster compression formats has later solved the problem himself:
https://github.com/richgel999/fpng
It turns out that deflate can be much faster when implemented specifically for PNG data, instead general-purpose compression (while still remaining 100%-standard-compatible).
Note he also expects a worse compression as tradeoff. I think he implements RLE in terms of zlib:
[...]Deflate compressor which was optimized for simplicity over high ratios. The "parser" only supports RLE matches using a match distance of 3/4 bytes, [...]One of the interesting features of ZStandard is the support for external dictionaries. It supports "training" a dictionary on a set of samples, of whatever size (16KiB, 64 KiB, etc.), then applying that dictionary as a separate input file for compression and decompression. This lets you compress short content much more effectively.
I doubt it would apply to PNG because of the length and content doesn't seem to be dictionary-friendly, but it would be interesting to try from some giant collection of scraped PNGs. This approach was important enough for Brotli to include a "built-in" dictionary covering HTML.
This has been applied to minecraft region files in a fork of paper, which is a type of minecraft server.
https://github.com/UltraVanilla/paper-zstd/blob/main/patches...
from the author of this patch on discord - the level 9 for compression isn't practical and is too slow for a real production server but it does show the effectiveness of zstd with a shared dictionary.
So you start off with a 755.2 MiB world (in this test, it is a section of an existing DEFLATE-compressed world that has been lived in for a while). If you recreate its regions it will compact it down to 695.1 MiB
You set region-file-compression=lz4 and run --recreateRegionFiles and it turns into a 998.9 MiB world. Makes sense, worse compression ratios but less CPU is what mojang documented in the changelog. Neat, but I'm confused as to what the benefits are as I/O increasingly becomes the more constrained thing nowadays. This is just a brief detour from what I'm really trying to test
You set region-file-compression=none and it turns into a 3583.0 MiB world. The largest region file in this sample was 57 MiB
Now, you take this world, and compress each of the region files individually using zstd -9, so that the region files are now .mca.zst files. And you get a world that is 390.2 MiBAuthor here -- the solution I discussed in that message isn't quite the same solution as the one linked. The `paper-zstd` repository is the one using dictionary compression on individual chunks. But in the `.mca.zst` solution I'm not using dictionaries at all. It's more like a glorified LinearPaper -- just take the region file, decompress the individual chunks, and recompress the entire 1024 chunk container together. It breaks random access to individual chunks, but it's great for archival or cloud storage offloading of infrequently visited parts of a MC world, which is what I'm using it for.
I don't remember the exact compression ratios for the dictionary solution in that repo, but it wasn't quite as impressive (IIRC around a 5% reduction compared to non-dictionary zstd at the same level). And the padding inherent to the region format takes away a lot of the ratio benefit right off the bat, though it may have worked better in conjunction with the PaperMC SectorFile proposal, which has less padding, or by rewriting the storage using some sort of LSM tree library that performs well at compactly storing blobs of varying size. I've dropped the dictionary idea for now, but it definitely could be useful. More research is needed.
> You set region-file-compression=lz4 and run --recreateRegionFiles and it turns into a 998.9 MiB world. Makes sense, worse compression ratios but less CPU is what mojang documented in the changelog. Neat, but I'm confused as to what the benefits are as I/O increasingly becomes the more constrained thing nowadays. This is just a brief detour from what I'm really trying to test
Might make sense if the region files are on a fast SSD and the server is more CPU-constrained? I assume the server reads from and writes to the region files during activity, a 3.5x increase in IO throughput at very little CPU cost (both ways) is pretty attractive. IIRC at lower compression levels deflate is about an order of magnitude more expensive than lz4.
zstd --fast is also quite attractive, but I'm always confused as to what the level of parallelism is in benchmarks, as zstd is multithreaded by default and benchmarks tend to show wallclock rather than CPU seconds.
Note that each region file contains 1024 chunks that are designed to be (but probably aren't) accessed at random, so compressing a region file is like a solid archive with a solid block size of 1024 files.
> I doubt it would apply to PNG because of the length and content doesn't seem to be dictionary-friendly
Correct - I wouldn't expect this to be useful for PNG. Compression dictionaries are applicable in situations where a group of documents contain shared patterns of literal content, like snippets of HTML. This is very uncommon in PNG image data, especially since any difference in compression settings, like the use of a different color palette, or different row filtering algorithms, will make the pattern unrecognizable.
Years ago I built a slippy map (google maps-style) tile server for non-image data. One of the use cases was to be able to quickly sample elevation data at an arbitrary lat/lng in a few milliseconds.
The data set is so large that you obviously want to delay decompression as long as possible. I turned to 16-bit grayscale PNGs, because PNG is a widely-used a standard. These were fine, but I wasn't close to my target latency.
After some experimentation, I was surprised to discover two things:
1. Deflate, this widely used standard, is just super slow compared to other algorithms (at least, in Go's native PNG decoder)
2. Tool and library support for formats other than ARGB32 is pretty lacking
So I turned to some bespoke integer compression algorithms like Snappy and Simple8b, and got a 20x decompression speedup, with maybe 20% worse compression ratios. This, along with some other tricks, got me where I needed to go.
Maybe there are some niche file formats out there that would've solved this. But in total we're not even talking about that much code, so it was easier to just invent my own.
I was impressed at how much Jart squeezed out of simple run-length-encoding of their binaries, and decoding only took 14 bytes of code [0]
When looking at that code:
aa 1: stosb e2 fd loop 1b
Why doesnt jart simply use rep stosb? It would take 1 less byte and even be slightly more idiomatic.
ZSTD is a great compression algorithm but an important PNG (v1.2) advantage is that implementations are available in almost all actively used operating systems and in most popular languages. The same cannot be said about ZSTD with very few implementations except https://github.com/facebook/zstd
I'm not even sure there is a good pure Java (no JNI) and Go (without Cgo) implementations for ZSTD. And it definitely would require more powerful hardware - some micro-controllers which can use PNG are too small for ZSTD.
For Go, we've been using this library which supports ZSTD. https://github.com/klauspost/compress
Very reasonable.
I've recently experimented with the methods of serving bitmaps out of the database in my project[1]. One option was to generate PNG on the fly, but simply outputting an array of pixel color values over HTTP with Content-Encoding: zstd has won over PNG.
Combined with the 2D-delta-encoding as in PNG, it will be even better.
I think there is a benefit to knowing that if you have a png file it works everywhere that supports png.
Better to make the back compat breaks be entirely new formats.
Related: what's the status of content negotiation? Any browsers use it seriously, and has it been successful? If so, then why not zpng.
Zstandard gets a lot of attention but I love LZ4 for speed. At least in the .NET world we have a fast LZ4 compressor/decompressor that is barely slower than memcpy. If you're already copying the data, you might as well LZ4 it. It's great over the wire when I control both the server and the client.
Does deflate lead the pack in any metric at all anymore? Only one I can think of is extreme low spec compression (microcontrollers).
The only metric deflate leads on is widespread support. By any other metric, it has been superseded.
I'd assume memory usage as well, because it has a tiny context window compared to zstd
You can change the context window of zstd if you want. But yes, the default context window size for zstd is 8MB, versus 32k.
Even there, LZ4 is probably better.
You think LZ4 is more portable than zlib? I'm gonna need some citations on that.
zlib is 30 years old, according to Wikipedia. And that's technically wrong since 'zlib' was factored out of gzip (nearly 33 years old) for use in libpng, which is also 30 years old.
A basic LZ4 decompressor is on the order of a few dozen lines of code. It's exceptionally easy to implement.
Just because its old doesn't mean it's more portable. If anything it makes me think it's even less portable.
not more portable, but probably faster in resource constrained environments
(2021)
In my opinion PNG doesn't need fixing. Being ancient is a feature. Everything supports it. As much as I appreciate the nerdy exercise, PNG is fine as it is. My only gripe is that some software writes needlessly bloated files (like adding a useless alpha channel, when it's not needed). I wish we didn't need tools like OptiPNG etc.
Yes. One of the best features of png is that I don't have to wonder if it's going to work somewhere. Throwing that away in favor of a bit of premature optimization seems like a big loss. Especially as this wouldn't be the only modernized image compression format out there. Why use this over, eg, lossless webp?
I don't think I have ever noticed the decode time of a png.
>I don't think I have ever noticed the decode time of a png.
When it was developed, 200 Mhz Pentium was the current tech. Back of the envelope numbers (ie. chatgpt) says my current desktop CPU (i7-14700K) decodes 7000x faster.
PNG is fully serial, no? So 7000x seems like a big over exaggeration
I assumed decoding multiple files in parallel. I think very few websites have just one PNG file.
Most of the comments on that issue are from this year.
The reason we have a world full of .gif today is that the .png committee rejected animation back when everyone was saying PNG would be the "GIF killer". Just sayin'. Don't hold your breath.
Remember this: https://burnallgifs.org/
We ought consider using QOI instead.
QOI is often equivalent or better compression than PNG, _before_ you even compress it with something like LZ4 etc.
Compressing QOI with something like LZ4 would generally outperform PNG.
QOI has some pretty major downsides. it only supports 8 bit SRGB, and is optimized for images with 8 bit transparency. Also, the handsome it uses seems to harm compression when entropy compression is used. Also, the focus on streaming means that the algorithm can't take advantage of 2d locality.
QOI is really cool, but I think the author cut the final version of the spec too early, and intentionally closed it off to a future version with more improvements. With another year or 2 of development, I think it probably works have become ~10% more efficient and suitable for more usecases.
> Compressing QOI with something like LZ4 would generally outperform PNG.
https://github.com/nigeltao/qoir has some numbers comparing QOIR (which is QOI-inspired-with-LZ4) vs PNG.
QOIR has better decode speed and comparable compression ratio (depending on which PNG encoder you use).
QOIR's numbers are also roughly similar to ZPNG.