Τ-coder Performance Evaluation
Τ-coder is an advanced JPEG encoder combining open source technology with proprietary optimizations resulting in significant compression performance improvements.
Τ-coder's performance is evaluated on the CID22 dataset using objective performance metrics (VMAF and PSNR) against two state-of-the-art open source encoders: MozJPEG developed by Mozilla and Jpegli developed by Google.
CID22 Dataset
The CID22 dataset was introduced by Cloudinary in 2022 and since then became a popular choice in the image compression community. It includes 250 losslessly captured images with content typical for web — people and their environment, natural, rural and urban scenes as well and computer generated graphics.
Typically, only a subset of 50 "validation" images is used for performance evaluation, assuming that the other 200 can be used for "training", however since there was no training on CID22 perfromed for τ-coder, the entire 250 image set was used for performance evaluation to improve result stability.
Quality Metrics
Quality metrics are algorithms that assess the impact of JPEG compression on image quality. Two metrics were used for this evaluation — PSNR and VMAF. PSNR is simply a measure of how much original data was preserved. From the encoder's prospective, it gives an idea of its ability to "squeeze" input data and doesn't take into account any perceptual aspects. VMAF, on the other hand, was developed specifically to address perceptual quality and is known to have a high correlation with MOS. Together these two metrics should give an idea of encoder's algorithmic compression efficiency and its ability to preserve more perceptually important data.
Bjontegaard Delta Rate
The Bjontegaard Delta Rate (BD-rate) is a way to compare efficiency of two lossy encoders, A and B. It shows the percentage difference in the amount of data needed by B compared A to achieve the same quality level, as measured by metrics like PSNR or VMAF. For example, a BD-rate of +10% means the encoder B uses 10% more data for the same quality, making it less efficient. If the BD-rate is negative, it means the encoder B is more efficient, needing less data than A for the same quality.
Testing Process
For each encoder — MozJPEG, Jpegli and τ-coder — the input dataset was compressed at different quality levels (points) so that the entire quality range is approximately matched between encoders, with MozJPEG being used as a baseline. All encoders used 4:2:0 chroma sampling and progressive JPEG encoding. Next, the two quality metrics — PSNR and VMAF — were calculated for each point, averaged over the entire dataset. Finally, VMAF and PSNR BD rates were calculated from bitrates and quality values.
Results
Table below presents quality and bitrates for each encoder. Each table row presents total size and average bitrate, PSNR and VMAF produced by coresponding encoder at a given quality point over the entire CID22 dataset (250 images).
Encoder | Total size, bytes | Average Bits per pixel | Average PSNR, dB | Average VMAF |
---|---|---|---|---|
MozJPEG | 5275724 | 0.08 | 34.92 | 88.16 |
6124893 | 0.09 | 35.70 | 90.06 | |
7334747 | 0.11 | 36.75 | 91.92 | |
10904076 | 0.17 | 38.69 | 93.90 | |
Jpegli | 5257753 | 0.08 | 35.00 | 89.05 |
5926776 | 0.09 | 35.75 | 90.47 | |
6842390 | 0.10 | 36.74 | 91.92 | |
9346389 | 0.14 | 38.97 | 94.18 | |
Τ-coder | 4910348 | 0.07 | 34.91 | 89.44 |
5499697 | 0.08 | 35.61 | 90.76 | |
6144486 | 0.09 | 36.36 | 91.93 | |
9446084 | 0.14 | 39.52 | 94.52 |
NOTE: A pre-release version of τ-coder was used for this comparison, so these values may differ insignificantly from those returned by the service.
Table below presents BD-rate values of pair-wise comparisons of τ-coder vs MozJPEG and Jpegli.
Encoder A | Encoder B | PSNR BD rate, % | VMAF BD rate, % |
---|---|---|---|
MozJPEG | Τ-coder | -13.45 | -18.06 |
Jpegli | Τ-coder | -6.05 | -10.27 |
As noted previously, the negative values mean that the encoder B requires a lower bit-rate to achieve the same quality as measured by corresponding metrics.
Discussion
As can seen from the BD rate table, τ-coder does a better job compressing images than the other encoders.
The PSNR BD rate indicates that τ-coder provides better coding tools — algorithms that pack input into a compressed bitstream — requiring on average 6 (compared to Jpegli) to 13 (compared to MozJPEG) percent less bitrate to carry the same amount of the original image data.
The higher VMAF BD rate indicates that τ-coder is also better at deciding what image details should be preserved and what can be sacrificed to minimize perceptual impact at a given quality level, requiring on average 10 to 18 percent less bitrate for the same perceptual quality.
Conclusion
Τ-coder requires up to 18% less bitrate to achieve the same perceptual quality as state-of-the-art open-source encoders.
Final Remarks
Originally it was planned to include into this evaluation some image optimization services from Google search but random four of them taken from the first page of Google results performed so poor that it was decided to focus on MozJPEG and Jpegli which generally show state-of-the-art performance.
Gallery
The gallery presents samples from the CID22 dataset compressed with three encoders: MozJPEG, Jpegli and τ-coder.
Two cases are considered: comparison of objective and perceptual quality for similarly sized output, and comparison of output size for similar objective quality.
Since it's impossible to match neither size not quality precisely, the preference was given to MozJPEG and Jpegli: for the quality comparison case, τ-coder's size is slightly smaller than the other two encoders, and for the size comparison, τ-coder's quality is slightly higher.
pexels-photo-3586798.png
Medium detail scene. Similar objective quality. Τ-coder produces about 9% smaller output than both MozJPEG and Jpegli.
Medium detail scene. Similar objective quality. Τ-coder produces about 9% smaller output than both MozJPEG and Jpegli.
Jpegli. 25,294 Bytes. VMAF: 90.08
Click on image to flip
Click on image to flip
τ-coder. 23,085 Bytes. VMAF: 90.17
MozJPEG. 25,339 Bytes. VMAF: 88.99
Click on image to flip
Click on image to flip
pexels-photo-7078476.png
High detail scene. Similar objective quality. Τ-coder produces about ~9% smaller output than both MozJPEG and Jpegli.
High detail scene. Similar objective quality. Τ-coder produces about ~9% smaller output than both MozJPEG and Jpegli.
Jpegli. 38,729 Bytes. VMAF: 88.55
Click on image to flip
Click on image to flip
τ-coder. 35,263 Bytes. VMAF: 89.58
MozJPEG. 38,378 Bytes. VMAF: 88.18
Click on image to flip
Click on image to flip
1173777.png
Contrast detail scene. Similar objective quality. Τ-coder prduces about ~5% smaller output than both MozJPEG and Jpegli.
Contrast detail scene. Similar objective quality. Τ-coder prduces about ~5% smaller output than both MozJPEG and Jpegli.
Jpegli. 38,248 Bytes. VMAF: 89.21
Click on image to flip
Click on image to flip
τ-coder. 36,732 Bytes. VMAF: 89.67
MozJPEG. 38,961 Bytes. VMAF: 87.62
Click on image to flip
Click on image to flip
pexels-photo-3586798.png (fragment)
Medium detail scene. Similar size. Τ-coder produces over 1 unit higher VMAF quality than both MozJPEG and Jpegli. Subjectively, τ-coder preserves more details around lashes and lips.
Medium detail scene. Similar size. Τ-coder produces over 1 unit higher VMAF quality than both MozJPEG and Jpegli. Subjectively, τ-coder preserves more details around lashes and lips.
Jpegli. 25,294 Bytes. VMAF: 90.08
Click on image to flip
Click on image to flip
τ-coder. 24,819 Bytes. VMAF: 90.99
MozJPEG. 25,339 Bytes. VMAF: 88.99
Click on image to flip
Click on image to flip
pexels-photo-2811087.png (fragment)
Medium detail scene. Similar size. Τ-coder produces over 1 unit higher VMAF quality than both MozJPEG and Jpegli. Subjectively, τ-coder preserves more details around lips and hair.
Medium detail scene. Similar size. Τ-coder produces over 1 unit higher VMAF quality than both MozJPEG and Jpegli. Subjectively, τ-coder preserves more details around lips and hair.
Jpegli. 19,362 Bytes. VMAF: 89.39
Click on image to flip
Click on image to flip
τ-coder. 19,321 Bytes. VMAF: 90.69
MozJPEG. 19,560 Bytes. VMAF: 88.95
Click on image to flip
Click on image to flip
pexels-photo-2811087.png (fragment)
High detail scene. Similar size. Τ-coder produces over 1 unit higher VMAF quality than both MozJPEG and Jpegli. Subjectively, τ-coder preserves slightly more fur details.
High detail scene. Similar size. Τ-coder produces over 1 unit higher VMAF quality than both MozJPEG and Jpegli. Subjectively, τ-coder preserves slightly more fur details.
Jpegli. 22,441 Bytes. VMAF: 88.55
Click on image to flip
Click on image to flip
τ-coder. 21,848 Bytes. VMAF: 89.58
MozJPEG. 21,960 Bytes. VMAF: 88.18
Click on image to flip
Click on image to flip
Get your τ-coder API key now!
Opt-in for occasional updates