Kin of the Stars

Please sign in to post.
Kin of the Stars

A community for the fans of Crest of the Stars, Abh culture, anime, technology, science fiction, video games, and friendly conduct.


    Webp, WebM and other new compression formats

    Share
    avatar
    Almael
    Imperial Admiral
    Imperial Admiral

    Number of posts : 2385
    Imperial Credits : 12903
    Registration date : 2009-03-18

    Webp, WebM and other new compression formats

    Post by Almael on 2/26/2017, 4:01 pm

    Imho it's a bit early but I've seen files in webp format popping up more frequently
    so here are some information I have gathered.

    In the past view years media transfer over the web has dramatically increased so it comes at no surprise a bunch of new media and compression formats have appeared to reduce the transfer load.
    The Webp is google's attempt at replacing jpeg, gif and png images.
    https://en.wikipedia.org/wiki/WebP
    https://www.philipstorry.net/thoughts/bpg-vs-jpeg-vs-webp-vs-jpeg-xr
    It offers a lossless as well as a lossy compression, and all other features found in the format it replaces.
    The WebM is the video counterpart.
    https://en.wikipedia.org/wiki/WebM

    I'm not interested in the lossy part as quality doesn't matter when you already have decided to accept losses. I'm most interested in WebP vs. PNG at maximum compression and no cheating/cutting etc.

    So far Webp does create a smaller sized file than png. However, the thing I noticed is it does not seem to save the print resolution (in dpi). This might be a problem if you want to print an image and don't know the original size, hence, resolution needed. It actually saves a different print size.
    Edit: The resolution seems to be fixed at 72 dpi and the result is based on that conversion.

    https://developers.google.com/speed/webp/docs/webp_study
    https://developers.google.com/speed/webp/faq
    https://optimus.keycdn.com/support/png-to-webp/
    https://www.smashingmagazine.com/2015/10/webp-images-and-performance/
    https://pngmini.com/vs-webp/
    https://www.andrewmunsell.com/blog/png-vs-webp/

    Widespread compatibility is still an issue and it's still in development. I don't feel it's reliable. So my bet is rather on an even better format in about 5 years. (10 years after HEVC / X.265 when X.266 comes out)

    --------------
    FLIF is another lossless format which claims to be better than all the others
    http://flif.info/
    https://en.wikipedia.org/wiki/Free_Lossless_Image_Format
    It's based on the Adam7 algorithm.
    https://en.wikipedia.org/wiki/Adam7_algorithm
    It does not mind of bits and pieces are missing after a transfer...
    To be honest this is like a fake compression from a long time ago which recursively deletes half the data each round => making any compression size is possible.
    I'm skeptical because it accepts any amount of losses.

    ----------
    Brotli is google's new general web compression
    https://en.wikipedia.org/wiki/Brotli
    I expect it to be replaced soon enough in another 5 years.


    -----------
    http://www.htmlgoodies.com/tutorials/web_graphics/the-resolutions-too-low-printing-web-graphics.html
    In case you don't know 300 dpi is standard for printing (including your printer). Keep this in mind when you scan photos. This is the minimum.


    Last edited by Almael on 2/27/2017, 12:29 pm; edited 3 times in total
    avatar
    Almael
    Imperial Admiral
    Imperial Admiral

    Number of posts : 2385
    Imperial Credits : 12903
    Registration date : 2009-03-18

    Re: Webp, WebM and other new compression formats

    Post by Almael on 2/27/2017, 10:24 am

    More technical analysis on the FLIF (from the author himself)
    http://cloudinary.com/blog/one_pixel_is_worth_three_thousand_words
    http://cloudinary.com/blog/a_one_color_image_is_worth_two_thousand_words
    As you can see FLIF cheats by generalizing the information for these simple images.
    Analysis for random image data is still missing.

    from the author:


    Optical comparison between PNG and FLIF: choose a file and use the mouse to drag the separation line between the images.
    https://uprootlabs.github.io/poly-flif/
    avatar
    Almael
    Imperial Admiral
    Imperial Admiral

    Number of posts : 2385
    Imperial Credits : 12903
    Registration date : 2009-03-18

    Re: Webp, WebM and other new compression formats

    Post by Almael on 3/2/2017, 5:11 pm

    As it turns out WebP doesn't store 8 bit color depth PNG or gif the same way. It store them all as 24 bit images so when you convert them back you get huge files (3x).
    The developers have said to use zopflipng to optimize and minimize the back-conversion.
    I use 8 bit PNG for B&W document scans to save space. You can use paint.net for creating them.

    https://blog.andreetoledo.me/implementing-png-optimization-with-zopflipng-d729cef66ab3#.mzg5d4obr
    https://github.com/jibsen/pngwolf-zopfli/releases
    http://www.diyphotography.net/8-bit-vs-16-bit-color-depth-use-matters/

    zopfli is an older google compression algorithm
    https://github.com/google/zopfli
    brothli is a recent google compression algorithm replacing zopfli
    https://github.com/google/brotli

    I will run an automated large scale conversion test over night. The result should be done by tomorrow.

    ----------
    PNG -> WebP test
    74 files, 24 bit 256 BW colors (incompetent), non-optimized PNG, 177517153 bytes => lossless WebP 109458168 bytes
    109458168/177517153=0.61660614847738122523855483419115

    arc 112821194 by PeaZip
    7zip 110700724 ultra LZMA2@1536MB word273
    rar5 110684374 best@1024MB
    rar 110675369
    arc 110663292 by arctool 0.9.425 half files compressed
    zip 110632297 by winrar
    a typical result

    TIFF -> WebP test
    M101 galaxy
    https://www.noao.edu/image_gallery/images/d5/02881.tiff
    24 bit color TIFF 173783458 => PNG 89635418 bytes => WebP 68920952 bytes
    back conversion from PNG/WebP => 173721890 bytes
    TIFF
    => zip 88714427 bytes
    => rar5 83115804 bytes
    => rar 81556889 bytes
    WebP
    => zip 68926638 bytes
    => rar5 68921130 bytes
    => rar 68921032 bytes
    atypical due to tiff color specifics

    the center of the Milky Way
    http://chandra.harvard.edu/photo/2009/galactic/
    48 bit color TIFF 64413092 => PNG 9455342 => WebP 7918028
    back conversion from PNG/WebP => 25246026 24 bit => color loss!

    avatar
    Almael
    Imperial Admiral
    Imperial Admiral

    Number of posts : 2385
    Imperial Credits : 12903
    Registration date : 2009-03-18

    Re: Webp, WebM and other new compression formats

    Post by Almael on 3/3/2017, 11:44 am

    All right the result are in:

    4378 files, 24 bit color art random mix, max compression 9 PNG, 3390155392 bytes

    using irfanview to convert
    => lossless WebP 2548059564
    2548059564/3390155392=0,75160553702430404700458049092282 => 75.2 %

    using googles "cwebp -lossless -q 100 -m 6" to convert via batch file
    => lossless WebP 2488098544
    2488098544/3390155392=0,73391873123909005761586045905945 => 73.4%

    The difference may not be much but for large database even 1% is worth it.

    The developers told me brotli doesn't work well and even using an uncompressed PNG would only achieve 60% of what webP can do.

    So my conclusion is WebP is good but you should only used it for the unimportant images. It's not as worth the effort for optimized document scans or manga since you need to do more than back conversion if needed. As far as packing goes the results are relatively the same as with packing jpeg.

    WebP is still some ways from the desired version I wish for but since the update last saturday and more browsers are going to support it it's definitely time to go for it.


    FLIF is still too under-developed but if it works as well and secure it might be worth it in another 5 years. By that time I expect the first glimpses of X.266 to be released and google to adapt it for another new compression algorithm.
    avatar
    Almael
    Imperial Admiral
    Imperial Admiral

    Number of posts : 2385
    Imperial Credits : 12903
    Registration date : 2009-03-18

    Re: Webp, WebM and other new compression formats

    Post by Almael on 3/15/2017, 6:17 pm

    There seem to be some trouble with crunchyroll.
    In any case because of this someone posted a long discussion about video encoding with X.264/H.264. Seriously this is outdated. MKV format support has been added to TV about 8-9 years ago. HEVC about the same time. HEVC2 is now standard hardware support. And in about 3-5 years the next standard (X.266) will come out. I can understand the reluctance or laziness to change to X265/HEVC now but the next generation should definitely be adopted.
    https://medium.com/@Daiz/crunchyrolls-reduced-video-quality-is-deliberate-cost-cutting-at-the-expense-of-paying-customers-c86c6899033b#.b8lcum7pv
    avatar
    Almael
    Imperial Admiral
    Imperial Admiral

    Number of posts : 2385
    Imperial Credits : 12903
    Registration date : 2009-03-18

    Re: Webp, WebM and other new compression formats

    Post by Almael on 3/17/2017, 2:27 pm

    Google just released a newer encode for JPEG:
    Guetzli
    https://github.com/google/guetzli/
    https://arxiv.org/abs/1703.04421
    35% smaller at better quality, they always seem to prefer it to be blurred rather than keeping detail contrast as with JPEG. Modern JPEG-wavelet which is by design optimized to keep structure by approaching fractals. I guess these days gamers tend to call this kind of technology procedural.

    Sponsored content

    Re: Webp, WebM and other new compression formats

    Post by Sponsored content


      Current date/time is 7/26/2017, 12:41 pm