[digiKam-users] Thumbnails are in Progessive Graphics File (PGF)

[Tac Tacelosky]

After digging around a bit, I think I have the answer to the question about how the thumbnails are stored, which is PGF:

https://en.wikipedia.org/wiki/Progressive_Graphics_File

Unfortunately, I cannot for the life of me find something in PHP that will convert from PGF to JPEG or PNG. 

Not surprisingly, it takes up a lot of space, but is wasted bandwidth for using a remote database.

Any suggestions on how to convert it?

Tac

[Gilles Caulier-4]

Hi,

ImageMAgick do not have a PGF support yet.

The libpgf project provide a CLI converter based on freeImage library :

https://sourceforge.net/projects/libpgf/files/libpgf/7.19.3-latest/

Best 

Gilles Caulier

Le mer. 10 avr. 2019 à 20:42, Tac Tacelosky <[hidden email]> a écrit :

After digging around a bit, I think I have the answer to the question about how the thumbnails are stored, which is PGF:

https://en.wikipedia.org/wiki/Progressive_Graphics_File

Unfortunately, I cannot for the life of me find something in PHP that will convert from PGF to JPEG or PNG. 

Not surprisingly, it takes up a lot of space, but is wasted bandwidth for using a remote database.

Any suggestions on how to convert it?

Tac

[Gerhard Hoogterp]

On woensdag 10 april 2019 20:35:58 CEST Tac Tacelosky wrote:
> After digging around a bit, I think I have the answer to the question about
> how the thumbnails are stored, which is PGF:
>
> https://en.wikipedia.org/wiki/Progressive_Graphics_File
>
> Unfortunately, I cannot for the life of me find something in PHP that will
> convert from PGF to JPEG or PNG.
 
Probably a stupid question, but... if piwigo stores its thumbnails in a
database in a format that nobody uses, including browsers AND I see online the
thumbnails being jpg, which makes sense since otherwise they would be useless,
wouldn't that mean that in the piwigo code somewhere there's a conversion?
And as I see this code on github it should be there somewhere?

Not a piwigo user or anything, just wondering?

--
https://gerhardhoogterp.nl/latest

[danielx386]

Don't quote me on this but I think piwigo stores thumbnail on the file system and not database.

---

From: Gerhard Hoogterp <[hidden email]>
Sent: Thursday, 11 April 2019 5:11 am
To: [hidden email]
Subject: Re: [digiKam-users] Thumbnails are in Progessive Graphics File (PGF)

On woensdag 10 april 2019 20:35:58 CEST Tac Tacelosky wrote:
> After digging around a bit, I think I have the answer to the question about
> how the thumbnails are stored, which is PGF:
>
> https://en.wikipedia.org/wiki/Progressive_Graphics_File
>
> Unfortunately, I cannot for the life of me find something in PHP that will
> convert from PGF to JPEG or PNG.

Probably a stupid question, but... if piwigo stores its thumbnails in a
database in a format that nobody uses, including browsers AND I see online the
thumbnails being jpg, which makes sense since otherwise they would be useless,
wouldn't that mean that in the piwigo code somewhere there's a conversion?
And as I see this code on github it should be there somewhere?

Not a piwigo user or anything, just wondering?

--
https://gerhardhoogterp.nl/latest

Le 10/04/2019 à 21:11, Gerhard Hoogterp a écrit :

> Not a piwigo user or anything, just wondering?
>
better ask on piwigo forum :-)

jdd

--
http://dodin.org

[Tac Tacelosky]

Not piwigo, but DigiKam.  But since digiKam is a desktop app, it's normally completely self-contained.  But the discussion was about accessing the MySQL database in digiKam so that it could be usable on the web.  And that's where having an esoteric format is problematic.  If the thumbnails were stored in PNG or JPEG, it would literally be a few lines of code to render them in the web browser.

Tac

On Wed, Apr 10, 2019 at 3:11 PM Gerhard Hoogterp <[hidden email]> wrote:

On woensdag 10 april 2019 20:35:58 CEST Tac Tacelosky wrote:
> After digging around a bit, I think I have the answer to the question about
> how the thumbnails are stored, which is PGF:
>
> https://en.wikipedia.org/wiki/Progressive_Graphics_File
>
> Unfortunately, I cannot for the life of me find something in PHP that will
> convert from PGF to JPEG or PNG.

Probably a stupid question, but... if piwigo stores its thumbnails in a
database in a format that nobody uses, including browsers AND I see online the
thumbnails being jpg, which makes sense since otherwise they would be useless,
wouldn't that mean that in the piwigo code somewhere there's a conversion?
And as I see this code on github it should be there somewhere?

Not a piwigo user or anything, just wondering?

--
https://gerhardhoogterp.nl/latest

[Gilles Caulier-4]

PGF is not an esoteric image compression format. It's an open source compressor, working with wavelets.  In opposite that you said, the file size is very optimized compared to JPEG and especially PNG. The image quality is so far better than JPEG, and is very fast.

To store thumbnail in database we need quality and size optimization, else we will explose the storage.

similar compression format are :

- JPEG 2000 : slow to compress/decompress (require float computation), not fully open source (patents).

- JPEG-XR : Microsoft, patents, not open source.

- Webp : Google, opensource, but not yet finalized when we needs to choice the DB thumbs storage..

- Heif : Apple, patents, partially open source through Nokia, very recent.

Web technology, use archaic image compression algorithm as JPEG. PNG compression is zip like... Wevelets compression is the future, better, powerfull. 

So before to judge the choice done about thumbnail store in digiKam database, take in consideration all technical levels. Thumbnails database storage is not designed for the Web.

PGF format have been selected in 2008 if i remember.

So you need a converter. PHP use ImageMagick in background, it's the best place to have the codec for the Web.

Best  

Gilles Caulier

  

Le mer. 10 avr. 2019 à 23:08, Tac Tacelosky <[hidden email]> a écrit :

Not piwigo, but DigiKam.  But since digiKam is a desktop app, it's normally completely self-contained.  But the discussion was about accessing the MySQL database in digiKam so that it could be usable on the web.  And that's where having an esoteric format is problematic.  If the thumbnails were stored in PNG or JPEG, it would literally be a few lines of code to render them in the web browser.

Tac

On Wed, Apr 10, 2019 at 3:11 PM Gerhard Hoogterp <[hidden email]> wrote:

On woensdag 10 april 2019 20:35:58 CEST Tac Tacelosky wrote:
> After digging around a bit, I think I have the answer to the question about
> how the thumbnails are stored, which is PGF:
>
> https://en.wikipedia.org/wiki/Progressive_Graphics_File
>
> Unfortunately, I cannot for the life of me find something in PHP that will
> convert from PGF to JPEG or PNG.

Probably a stupid question, but... if piwigo stores its thumbnails in a
database in a format that nobody uses, including browsers AND I see online the
thumbnails being jpg, which makes sense since otherwise they would be useless,
wouldn't that mean that in the piwigo code somewhere there's a conversion?
And as I see this code on github it should be there somewhere?

Not a piwigo user or anything, just wondering?

--
https://gerhardhoogterp.nl/latest

[Gerhard Hoogterp]

On woensdag 10 april 2019 22:56:15 CEST [hidden email] wrote:
> Le 10/04/2019 à 21:11, Gerhard Hoogterp a écrit :
> > Not a piwigo user or anything, just wondering?
>
> better ask on piwigo forum :-)
>
> jdd

Nah, from the maindiscussion  on this subject I got the idea it was piwigo
that stored it's thumbnails in a weird format. Not digikam.. My bad..



--
https://gerhardhoogterp.nl/latest

[Gerhard Hoogterp]

On woensdag 10 april 2019 23:08:14 CEST Tac Tacelosky wrote:
> Not piwigo, but DigiKam.  But since digiKam is a desktop app, it's normally
> completely self-contained.  But the discussion was about accessing the
> MySQL database in digiKam so that it could be usable on the web.  And
> that's where having an esoteric format is problematic.  If the thumbnails
> were stored in PNG or JPEG, it would literally be a few lines of code to
> render them in the web browser.

In that case half the needed code should be in the digikam code After all it
reads and presents the thumbnails on screen which is more or less half way to
saving it as something else?


--
https://gerhardhoogterp.nl/latest

[Gilles Caulier-4]

yes absolutly There is a unit test here :

https://cgit.kde.org/digikam.git/tree/core/tests/fileio/qtpgftest.cpp

This code take a PGF file and load in Qt::QImage container, and save it as PNG.

Code relevant from digiKam core (including libpgf as well) :

https://cgit.kde.org/digikam.git/tree/core/libs/pgfutils

Gilles Caulier

Le jeu. 11 avr. 2019 à 17:19, Gerhard Hoogterp <[hidden email]> a écrit :

On woensdag 10 april 2019 23:08:14 CEST Tac Tacelosky wrote:
> Not piwigo, but DigiKam.  But since digiKam is a desktop app, it's normally
> completely self-contained.  But the discussion was about accessing the
> MySQL database in digiKam so that it could be usable on the web.  And
> that's where having an esoteric format is problematic.  If the thumbnails
> were stored in PNG or JPEG, it would literally be a few lines of code to
> render them in the web browser.

In that case half the needed code should be in the digikam code After all it
reads and presents the thumbnails on screen which is more or less half way to
saving it as something else?


--
https://gerhardhoogterp.nl/latest

[Gilles Caulier-4]

Hi,

I just created a entry in ImageMagick bugzilla to support PGF :

https://github.com/ImageMagick/ImageMagick/issues/1548

Best

Gilles Caulier

Le mer. 10 avr. 2019 à 20:57, Gilles Caulier <[hidden email]> a écrit :

Hi,

ImageMAgick do not have a PGF support yet.

The libpgf project provide a CLI converter based on freeImage library :

https://sourceforge.net/projects/libpgf/files/libpgf/7.19.3-latest/

Best 

Gilles Caulier

Le mer. 10 avr. 2019 à 20:42, Tac Tacelosky <[hidden email]> a écrit :

After digging around a bit, I think I have the answer to the question about how the thumbnails are stored, which is PGF:

https://en.wikipedia.org/wiki/Progressive_Graphics_File

Unfortunately, I cannot for the life of me find something in PHP that will convert from PGF to JPEG or PNG. 

Not surprisingly, it takes up a lot of space, but is wasted bandwidth for using a remote database.

Any suggestions on how to convert it?

Tac

[Tac Tacelosky]

Wahoo!  Thanks, Gilles.  This will really open up using the database.

Tac

On Mon, Apr 15, 2019 at 8:19 AM Gilles Caulier <[hidden email]> wrote:

Hi,

I just created a entry in ImageMagick bugzilla to support PGF :

https://github.com/ImageMagick/ImageMagick/issues/1548

Best

Gilles Caulier

Le mer. 10 avr. 2019 à 20:57, Gilles Caulier <[hidden email]> a écrit :

Hi,

ImageMAgick do not have a PGF support yet.

The libpgf project provide a CLI converter based on freeImage library :

https://sourceforge.net/projects/libpgf/files/libpgf/7.19.3-latest/

Best 

Gilles Caulier

Le mer. 10 avr. 2019 à 20:42, Tac Tacelosky <[hidden email]> a écrit :

After digging around a bit, I think I have the answer to the question about how the thumbnails are stored, which is PGF:

https://en.wikipedia.org/wiki/Progressive_Graphics_File

Unfortunately, I cannot for the life of me find something in PHP that will convert from PGF to JPEG or PNG. 

Not surprisingly, it takes up a lot of space, but is wasted bandwidth for using a remote database.

Any suggestions on how to convert it?

Tac

[Gilles Caulier-4]

Le lun. 15 avr. 2019 à 14:44, Tac Tacelosky <[hidden email]> a écrit :

Wahoo!  Thanks, Gilles.  This will really open up using the database.

Now, please comment this report in ImageMagick to convince developers to include PGF support.

Best

Gilles Caulier

[Tac Tacelosky]

Can you also put in a request for GD, that's what I'm currently using, but I'd switch to whichever library implements it.

https://github.com/libgd/libgd

Thanks,

Tac

On Mon, Apr 15, 2019 at 8:53 AM Gilles Caulier <[hidden email]> wrote:

Le lun. 15 avr. 2019 à 14:44, Tac Tacelosky <[hidden email]> a écrit :

Wahoo!  Thanks, Gilles.  This will really open up using the database.

Now, please comment this report in ImageMagick to convince developers to include PGF support.

Best

Gilles Caulier

[Striper]

Hi Gilles,

The link to the pgf loader that you mentioned in your ImageMagick issue,
does not exist anymore in the digikam 7.0 branch. Does this mean that pgf
support is dropped in Digikam 7 or did you just restructure the code?

https://cgit.kde.org/digikam.git/tree/core/libs/dimg/loaders/pgfloader.cpp
=> does not exist

https://cgit.kde.org/digikam.git/tree/libs/dimg/loaders/pgfloader.cpp?h=development/6.0.0
=> does exist

Best regards,
Barry



--
Sent from: http://digikam.1695700.n4.nabble.com/digikam-users-f1735189.html

[Gilles Caulier-4]

Hi,

Code is restructured in fact. All image loader are now plugin :

https://invent.kde.org/kde/digikam/tree/master/core/dplugins/dimg/pgf

PGF still used to store thumbnails in database, and there is no better eq to change this. HEIF is slow, pattented to write data (not to read) and WEBP has possible patents with Google stuff.

Other point, PGF library is very small compared to HEIF or WEBP. So, PGF is perfect. Library is there in digiKam core :

https://invent.kde.org/kde/digikam/tree/master/core/libs/pgfutils

Best

Gilels Caulier

Le mer. 29 janv. 2020 à 03:26, Striper <[hidden email]> a écrit :

Hi Gilles,

The link to the pgf loader that you mentioned in your ImageMagick issue,
does not exist anymore in the digikam 7.0 branch. Does this mean that pgf
support is dropped in Digikam 7 or did you just restructure the code?

https://cgit.kde.org/digikam.git/tree/core/libs/dimg/loaders/pgfloader.cpp
=> does not exist

https://cgit.kde.org/digikam.git/tree/libs/dimg/loaders/pgfloader.cpp?h=development/6.0.0
=> does exist

Best regards,
Barry



--
Sent from: http://digikam.1695700.n4.nabble.com/digikam-users-f1735189.html

[Rafael Linux]

I didn't know about PGF and how Digikam stores image thumbnails, but after
reading about that format, I think it's a very , at the moment of writing
this post, that PGF choice was a wise choice. In fact, I can't understand
why this format was not adopted as the standard for web images by W3C, cause
it have even multiple resolution representation and progressive
transmission!!! I can't believe this format exist 20 years ago and is not
being used by most of GNU/GPL image manipulation software.

I expect this 2020 year finally AVIF (AV1 image format) will give us a final
version to have a perfect format to store images.



--
Sent from: http://digikam.1695700.n4.nabble.com/digikam-users-f1735189.html

Le 29/01/2020 à 11:59, Rafael Linux a écrit :

> this post, that PGF choice was a wise choice. In fact, I can't understand
> why this format was not adopted as the standard for web images by W3C, cause
> it have even multiple resolution representation and progressive
> transmission!!!

why change things that works? jpg is known as far as 1986

https://fr.wikipedia.org/wiki/Joint_Photographic_Experts_Group

and the people that makes the decision are camera makers...

jdd


--
http://dodin.org

[Gilles Caulier-4]

Le mer. 29 janv. 2020 à 05:55, Rafael Linux <[hidden email]> a écrit :

I didn't know about PGF and how Digikam stores image thumbnails, but after
reading about that format, I think it's a very , at the moment of writing
this post, that PGF choice was a wise choice. In fact, I can't understand
why this format was not adopted as the standard for web images by W3C, cause
it have even multiple resolution representation and progressive
transmission!!! I can't believe this format exist 20 years ago and is not
being used by most of GNU/GPL image manipulation software.

I expect this 2020 year finally AVIF (AV1 image format) will give us a final
version to have a perfect format to store images.

Exactly. But Google, Apple, Microsoft prefer to buy code/license or re-invent the rules instead to promote and use a free software which do already the job...

Best

Gilles

[Striper]

Although I understand your arguments and reasoning, I do not fully agree with
you. PGF may be the best technical solution for storing thumbnails (or any
graphic image for that matter), it's definitely not the best solution from
an interoperability standpoint. The simple fact that Google expands its
search results for "PGF" with hits on "PDF", means that this format is not
widely used nor supported. Except for the C++ implementation on SourceForge
(libPGF), there is not a single library or utility to be found that can
decode or encode this format. In other words, unless you are developing in
C++, it's useless.

There are countless examples of inferior technology that became the standard
over a superior technology (Betamax vs VHS is a classic). For techies like
myself, that hurts, but it is a fact of life that we have to live with.

The great thing about open source is that everybody can use it the way they
want to. Digikam is a great product and I love it, but using this kind of
exotic formats hampers the "openness", in my opinion. Would anybody really
notice the difference in performance or quality when the thumbnails are
stored as JPG or PNG? Would a few extra bytes really matter when we are
already storing gigabytes of RAW and JPG images?

FYI, I got to this thread because I was looking for a way to connect a nice
looking web frontend (not Piwigo!) to my Digikam library. Most of the
solutions out there require my photos to be duplicated to their data
structure and try to extract metadata and thumbnails on their own. Wouldn't
it be great if other software can simply tap into a Digikam database and use
the wealth of information that is in there? While that may still work for
most of the data, thumbnails are apparently out of the picture (pun
intended).

Thanks and best regards, Barry



--
Sent from: http://digikam.1695700.n4.nabble.com/digikam-users-f1735189.html