[Bug 211758] New: Saving TIFF-16 compressed using exiv2 0.18 shows odd results

https://bugs.kde.org/show_bug.cgi?id=211758


Olleg Samoylov <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|[hidden email]                |




--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=211758





--- Comment #20 from Marcel Wiesweg <marcel wiesweg gmx de>  2009-12-07 16:53:04 ---
Created an attachment (id=38900)
 --> (http://bugs.kde.org/attachment.cgi?id=38900)
Metadata extracted from NEF with exiv2

Andreas, find attached metadata from a NEF file extracted with exiv2.
Then I have taken a TIFF image
(http://digikam3rdparty.free.fr/TEST_IMAGES/TIFF/Solar_Spectrum.tiff) and
inserted the metadata with "exiv2 in".
Afterwards, GIMP gives the typical error messages:

/home/marcel/Solar_Spectrum.tiff: invalid TIFF directory; tags are not sorted
in ascending order
/home/marcel/Solar_Spectrum.tiff: wrong data type 7 for "Photoshop"; tag
ignored
/home/marcel/Solar_Spectrum.tiff: wrong data type 7 for "Photoshop"; tag
ignored

/home/marcel/Solar_Spectrum.tiff: unknown field with tag 11 (0xb) encountered
/home/marcel/Solar_Spectrum.tiff: unknown field with tag 18246 (0x4746)
encountered
/home/marcel/Solar_Spectrum.tiff: unknown field with tag 18249 (0x4749)
encountered
/home/marcel/Solar_Spectrum.tiff: unknown field with tag 11 (0xb) encountered
/home/marcel/Solar_Spectrum.tiff: unknown field with tag 18246 (0x4746)
encountered
/home/marcel/Solar_Spectrum.tiff: unknown field with tag 18249 (0x4749)
encountered
TIFF-Bild: /home/marcel/Solar_Spectrum.tiff: wrong data type 7 for "Photoshop";
tag ignored

The former three in a dialog, the latter on the console.
The picture itself can be opened by gimp, but there are other example images in
some of the four bug reports merged here that are more severely broken.
So keep in mind there could be two distinct problems, but we should assume it's
one problem and fix these warnings.

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=211758





--- Comment #21 from Andreas Huggel <ahuggel gmx net>  2009-12-08 12:02:38 ---
Marcel,

Thanks for the reproducer. Some preliminary analysis, below are the tags which
cause the warnings, if I delete all four of them Gimp is happy. It looks like
the complaint about tags not sorted in ascending order is a side-effect. I
think libtiff simply doesn't know the first three and for the last one it
expects a different type ('byte' instead of 'undefined' - which makes no
difference - but I don't remember where I have 'undefined' from, will
investigate).

Andreas


Unknown field
0x000b Exif.Image.ProcessingSoftware                Ascii      34
digiKam-0.10.0-rc1 (rev.: 891281)
0x4746 Exif.Image.Rating                            SLong       1  0
0x4749 Exif.Image.RatingPercent                     SLong       1  0

Photoshop tag
0x8649 Exif.Image.ImageResources                    Undefined 160  (Binary
value suppressed)

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=211758





--- Comment #22 from Marcel Wiesweg <marcel wiesweg gmx de>  2009-12-15 20:14:20 ---
Andreas, as an "emergency" measurement to work around this for our 1.0 release,
should we just remove these four tags when saving a tiff? Or should we rather
keep metadata saving disabled?

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=211758





--- Comment #23 from Gilles Caulier <caulier gilles gmail com>  2009-12-15 20:23:01 ---
Marcel,

I had already disabled tiff metadata saving in digiKam and kipi-plugins...

Gilles Caulier

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=211758





--- Comment #24 from Andreas Huggel <ahuggel gmx net>  2009-12-16 05:34:43 ---
Marcel, Gilles,

For the first three of these tags, there is nothing wrong if digiKam writes
these [1].
For the ImageResources tag, exiv2 seems to use the wrong type, although I have
yet to find the specification. It's easy to fix in exiv2 if necessary. This tag
may contain information that the users want to keep (written by Photoshop), it
would be interesting to know if Photoshop still reads it with the changed type.

So these warnings are relatively harmless. They do not explain the corrupted
images that some users have encountered.

-ahu.


[1] TIFF 6.0 Specification, Section 7: "Other fields. TIFF readers must be
prepared to encounter fields other than those required in TIFF files. TIFF
writers are allowed to write optional fields such as Make, Model, and DateTime,
and TIFF readers may use such fields if they exist. TIFF readers must not,
however, refuse to read the file if such optional fields do not exist. TIFF
readers must also be prepared to encounter and ignore private fields not
described in the TIFF specification."

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=211758





--- Comment #25 from Andreas Huggel <ahuggel gmx net>  2009-12-18 10:47:09 ---
> For the ImageResources tag, exiv2 seems to use the wrong type, although I have
> yet to find the specification. It's easy to fix in exiv2 if necessary.

Changed to BYTE: http://dev.exiv2.org/issues/show/661

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Leonardo Giordani-2]

https://bugs.kde.org/show_bug.cgi?id=211758





--- Comment #26 from Leonardo Giordani <giordani leonardo gmail com>  2010-01-23 10:15:25 ---
(In reply to comment #25)
> > For the ImageResources tag, exiv2 seems to use the wrong type, although I have
> > yet to find the specification. It's easy to fix in exiv2 if necessary.
>
> Changed to BYTE: http://dev.exiv2.org/issues/show/661

This last bugfix will solve the problem in opening digikam TIFFs with The GIMP?
Is this only available in SVN? On my Kubuntu Karmic I have exiv2-0.18.2-1, may
I just compile the SVN source and install it or are there compatibility issues?

Thanks

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=211758





--- Comment #27 from Marcel Wiesweg <marcel wiesweg gmx de>  2010-01-23 21:20:20 ---
No, this bug is not solved.

Repeated testcase from comment #20: Extracted metadata from a NEF and inserting
this into Solar_spectrum.tiff, using the exiv2 command line tool, current exiv2
SVN.

Error message from gimp:
invalid TIFF directory; tags are not sorted in ascending order

Console output:
/home/marcel/Solar_Spectrum.tiff: unknown field with tag 11 (0xb) encountered
/home/marcel/Solar_Spectrum.tiff: unknown field with tag 18246 (0x4746)
encountered
/home/marcel/Solar_Spectrum.tiff: unknown field with tag 18249 (0x4749)
encountered

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=211758





--- Comment #28 from Andreas Huggel <ahuggel gmx net>  2010-04-06 16:53:06 ---
Marcel,

The issues mentioned in comment #27 are not due to exiv2 - they can easily be
reproduced with any other tool.

As for tag 0xb, try the following:

1 - Take a (minimal) TIFF image.tif
2 - Add the tag with any program that can do this properly, e.g., with exiftool
-ProcessingSoftware=exiftool image.tif
3 - Open in GIMP

And you'll get that "invalid TIFF directory; tags are not sorted in ascending
order" error. The error itself is wrong - wherever it comes from. The image is
perfectly ok, and the tags are correctly sorted in ascending order.

As for the warnings for the other two tags - 0x4746 and 0x4749 - refer to quote
from the TIFF specs in comment #24. A TIFF-reader needs to be able to deal with
unknown tags. Issuing a warning is fine I guess but the warning can be ignored,
the image is fine.

It would cripple exiv2 if I'd disallow writing non-standard TIFF tags. Instead
if you decide they should not occur in a TIFF image, digiKam needs to suppress
them.

Andreas

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=211758





--- Comment #29 from Andreas Huggel <ahuggel gmx net>  2010-04-06 17:44:23 ---
Next is the error 'TIFF directory is missing required "ImageLength" field.'

The root cause for that message is the JPEG thumbnail which is included in
these digiKam generated TIFFs as a second "page" (IFD1). Presumably these tags
were copied from the Exif metadata of a JPEG with an Exif thumbnail.

This can be fixed by deleting all Exif.Thumbnail.* tags (the 2nd "page") from
the TIFF image (or not writing them in the first place). That can be done by
digiKam or exiv2, not sure yet if it should be in exiv2 (as part of
http://dev.exiv2.org/issues/show/668, will find out when it comes to TIFF-like
RAW images).

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=211758





--- Comment #30 from Andreas Huggel <ahuggel gmx net>  2010-04-06 19:02:16 ---
Finally, the "emboss effect":

Apparently 'Adobe Deflate' compression requires the 'Predictor' field, which
seems to get removed in the libkexiv2/Exiv2 call. The broken images can be
fixed by adding the tag again with a Short value 2, e.g.,

exiv2-0.19 -M'set Exif.Image.0x013d Short 2' MINOLTA-DYNAX7D-04.tif

using the sample from Gilles in comment #9.

This issue is fixed in the current Exiv2 trunk revision - Exiv2 now
differentiates between image tags and metadata, and does not modify any image
tags (http://dev.exiv2.org/issues/show/668). And the 'Predictor' tag is an
image tag.

(Note that also means the command above won't work with the current exiv2 from
trunk - it will ignore the command because it attempts to set an image tag.)

Gilles, following the steps from comment #9 you should not be able to reproduce
this emboss effect anymore using exiv2 from the trunk. Can you confirm that?

Andreas

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=211758





--- Comment #31 from Gilles Caulier <caulier gilles gmail com>  2010-04-06 19:06:29 ---
To Andreas,

- comment #28 : "invalid TIFF directory; tags are not sorted in ascending
order" error come from libtiff. gimp and digiKam delegate all tiff loading code
to libtiff.

- comment #30 : I will check it tomorrow morning.

Gilles

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=211758





--- Comment #32 from Marcel Wiesweg <marcel wiesweg gmx de>  2010-04-06 20:08:41 ---
Thanks Andreas for having a look into this. It's an important bug for us.

Two questions:

- do you "recommend" that we remove Exif.Thumbnail.* when saving a TIFF with
our TIFF loader? In that case, we'll do that. I dont know better than you.
- is there a workaround for the Predictor tag in our TIFF loader for older
exiv2 versions? Or is the field removed so early that we can't know it's been
there?

I'm still a bit unsure about the Gimp warnings from libtiff, but I can't easily
know where they come from either.

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=211758





--- Comment #33 from Andreas Huggel <ahuggel gmx net>  2010-04-07 04:55:59 ---
Re comment #32:

> - do you "recommend" that we remove Exif.Thumbnail.* when saving a TIFF with our TIFF loader?

Yes, that way it will also work with older versions of exiv2.

> - is there a workaround for the Predictor tag in our TIFF loader

If the TIFF is written as an 'Adobe deflated' compressed TIFF, i.e., the TIFF
image written by libtiff has an Exif.Image.Compression tag with a Short value 8
then digiKam can add (or prevent deletion of) the Predictor tag. It seems to
have a Short value of 2 (but I've only determined that empirically with just a
few TIFFs, so not sure if it's really always 2 - I'm not into TIFF image data
compression algorithms).

More general, when older versions of exiv2 are in use, digiKam could prevent
modifications to the image tags of TIFF images itself, similar to what exiv2 is
doing in Exiv2::Internal::TiffHeader::isImageTag() -
http://dev.exiv2.org/repositories/entry/exiv2/trunk/src/tiffimage.cpp#L1643
(this is the newer version, IIRC you once implemented something like this based
on the previous code)

I believe this issue is also related to bug #183171 and may go away once that
bug is fixed.

Andreas

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=211758





--- Comment #34 from Marcel Wiesweg <marcel wiesweg gmx de>  2010-04-11 17:04:11 ---
SVN commit 1113706 by mwiesweg:

Remove all Exif.Thumbnail.* tags when writing TIFF.

CCBUG: 211758

 M  +10 -1     tiffloader.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1113706

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=211758





--- Comment #35 from Marcel Wiesweg <marcel wiesweg gmx de>  2010-04-11 17:06:48 ---
183171 should now be fixed.
To test if this actually fixes the problem with the Predictor tag I need to use
an older version from exiv2 trunk? Do you know an SVN revision to downgrade to?

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=211758





--- Comment #36 from Andreas Huggel <ahuggel gmx net>  2010-04-11 17:55:54 ---
> 183171 should now be fixed.

Yeah!

> To test if this actually fixes the problem with the Predictor tag I need to use
> an older version from exiv2 trunk? Do you know an SVN revision to downgrade to?

Try exiv2 r2036 (just before the 1st commit for #668)

-ahu.

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=211758





--- Comment #37 from Gilles Caulier <caulier gilles gmail com>  2010-04-11 21:48:12 ---
This want mean that TIFF writter drop Exif thumbnail creation ?

It's will be generated by Exiv2 after libtiff code instead ?

I think Exif thumbnail creation is important there, to follow properly TIFF/EP
paper from ISO.

Or i forget something there ?

Gilles Caulier

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=211758





--- Comment #38 from Marcel Wiesweg <marcel wiesweg gmx de>  2010-04-11 22:28:17 ---
Gilles: I dont know about this. I let Andreas answer this question.

Andreas: I'm not sure how to test this. Perhaps the original reporter here can
give a hint if the problem is fixed with latest versions.

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel