[Digikam-devel] [Bug 130525] New: Saving large (>5M) jpg's result in corrupt file

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=130525         




------- Additional Comments From ahuggel gmx net  2006-07-13 11:43 -------
Florencio has a good point with his second statement: The size of a JPEG segment is limited to 64k (because the size of the segment is stored in a 16 bit number). So saving a >64k Thumbnail in an APP13 segment is definitely not possible.

Now, as far as Exiv2 is concerned, it should allow to set IPTC data which is larger than 64k (the limit for the Preview field is 256000 bytes according to the specs) and it should allow to copy that IPTC data data buffer in a binary format. However, it should NOT allow to write an IPTC block larger than 64k to a JPEG image (using Image::writeMetadata). I'll check if it follows this specs later but does digikam actually use writeMetadata or does it write the binary block obtained from Exiv2 using its own logic? (I know, it's probably the famous DMetadata class again :) but I can't check it out right now)

Andreas
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[J.Westveer]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=130525         




------- Additional Comments From jwest melaque com  2006-07-13 18:02 -------
I tried comment #17 from Florencio Guimaraes, with "JPEG", 50.
This did not work with my test file (good.jpg).  However "JPEG", 20 did work,
and the image is once again editable/saveable.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=130525         




------- Additional Comments From ahuggel gmx net  2006-07-13 19:00 -------
Ok, this is a bug in Exiv2. In the (old) version of dmetaloader.cpp that I have here, digikam uses (Jpeg)Image::writeMetadata and that function does not check the size of the segment (around line 500 of jpegimage.cpp); the result is a corrupted JPEG image as reported.
The fix in Exiv2 will probably simply be to throw an exception and digikam will need to either proactively check the size of the IPTC data or handle the exception.

Unfortunately, I don't have time to fix this immediately (Gilles please feel free to do so if you have more time), I can only do it next week.

Andreas
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Florencio Guimaraes-2]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=130525         




------- Additional Comments From florencio omeu com br  2006-07-14 04:55 -------
Created an attachment (id=16982)
 --> (http://bugs.kde.org/attachment.cgi?id=16982&action=view)
my local-temporary-fix-workaround

This way, we get the best quality given the 64K limit to preview.
Additionally, get rid off a KTempFile.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[J.Westveer]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=130525         




------- Additional Comments From jwest melaque com  2006-07-14 07:26 -------
diff patch works here.  tnx.

jjw
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=130525         




------- Additional Comments From caulier.gilles free fr  2006-07-14 09:40 -------
Andreas,

I'm not at home this week (hollidays). I just read my mail, and others internet stuff.

Next week, with Marcel, we will prepare the 0.9.0-beta1 release. I will disable IPTC preview record with JPEG target file (and JPEG target only because it work fine with PNG and TIFF) for this release until an Exiv2 issue will be find.

Friendly

Gilles
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=130525         
caulier.gilles free fr changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |aironmail gmail com



------- Additional Comments From caulier.gilles free fr  2006-07-14 10:02 -------
*** Bug 130728 has been marked as a duplicate of this bug. ***
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=130525         




------- Additional Comments From marcel.wiesweg gmx de  2006-07-15 17:10 -------
As I understand this, there is a bug in libexiv2 because it does not check the size, but that's a minor problem, it is digikam's responsibility to check the size of the preview. Even if libexiv2 is fixed, digikam still needs to make sure that if the image is a JPEG, the preview may not be larger than 64k.

DMetadata does not know the image format, so the limit needs to be passed to the setImagePreview() method. This method is called from DImgInterface, which knows the file format.

For other formats, PNG, TIFF, the size is not limited?
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=130525         




------- Additional Comments From caulier.gilles free fr  2006-07-16 09:15 -------
Marcel,

To have checked libPNG, i have not seen any size restriction with text chuck used to save Exif and IPTC byte-array.

About tiff file, the spectification do not said about any restrictions (look the url links annoted in Digikam::DImg::TiffLoader), but i'm not sure... Andreas, can you confirm this point ?

Gilles
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=130525         




------- Additional Comments From caulier.gilles free fr  2006-07-16 11:12 -------
SVN commit 562917 by cgilles:

digikam from trunk : disabled temporally the IPTC preview tag recoring in JPEG target file duing an uncompatibility between the IPTC preview tag size (256K max) and the JPEG segment size used to store IPTC byte array (64K).

A solution need to be found in Exiv2 to prevent this problem!

CCMAIL: digikam-devel kde org, ahuggel gmx net
CCBUGS: 130525

 M  +13 -2     dimginterface.cpp  


--- trunk/extragear/graphics/digikam/utilities/imageeditor/canvas/dimginterface.cpp #562916:562917
 @ -514,9 +514,20  @
     meta.setIptc(d->image.getIptc());
 
     // Update Iptc preview.
-    QImage preview = d->image.smoothScale(800, 600, QSize::ScaleMin).copyQImage();
-    meta.setImagePreview(preview);
 
+    // TODO: see B.K.O #130525. The a JPEG segment is limited to 64K. If IPTC byte array
+    // bigger than 64K duing of image preview tag size, the target JPEG image will be
+    // broken. Note that IPTC image preview tag is limited to 256K!!!
+    // Temp. solution to disable IPTC preview record in JPEG file until a right solution
+    // will be found into Exiv2.
+    // Note : There is no limitation with TIFF and PNG about IPTC byte array size.
+
+    if ( !mimeType.upper() == QString("JPG") || !mimeType.upper() == QString("JPEG") )
+    {
+        QImage preview = d->image.smoothScale(800, 600, QSize::ScaleMin).copyQImage();
+        meta.setImagePreview(preview);
+    }
+
     // Update Exif thumbnail.
     QImage thumb = preview.smoothScale(160, 120, QImage::ScaleMin);
     meta.setExifThumbnail(thumb);
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=130525         




------- Additional Comments From marcel.wiesweg gmx de  2006-07-16 22:24 -------
Created an attachment (id=17010)
 --> (http://bugs.kde.org/attachment.cgi?id=17010&action=view)
patch to adapt preview size

Inspired by the patch from Florencio, this patch adapts the preview file size
if the image is a JPEG and if the 64k limit is exceeded.

It trades off size and/or quality and makes five attempts at reaching the
limit. The exact parameters may be tuned, I am currently unsure which preview
sizes and which quality factors are still acceptable.
A quality of 0 is not acceptable btw, I tried that and the whole image is a
block artefact. I would prefer to downsize the preview then a bit.

Even if libexiv2 is fixed in the future, it will simply throw an exception, I
think we still need to do something like this if we really want to embed a
preview.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=130525         
marcel.wiesweg gmx de changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
  Attachment #17010|0                           |1
        is obsolete|                            |



------- Additional Comments From marcel.wiesweg gmx de  2006-07-16 22:30 -------
Created an attachment (id=17011)
 --> (http://bugs.kde.org/attachment.cgi?id=17011&action=view)
this version compiles
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=130525         




------- Additional Comments From caulier.gilles free fr  2006-07-16 22:48 -------
Marcel,

This is my suggestions :

- using a jpeg compression ratio value under 50 give a lot of artifact.
- current preview size is 800x600. A size of 640*480 is also right to render on the screen (it used by Minolta camera).
- using a preview size under 640*480 is not adapted. To optimize in all case, i recommend to use 640*480 by default instead 800x600 with JPEG files.
- unforget that IPTC is used also to store other metadata like comments, tags, rating, and others stuff. We need some place for that in iptc array. And this is the most important than preview.

The iptc spec is available here :

http://www.iptc.org/std/IIM/4.1/specification/IIMV4.1.pdf

Look page 45 and 47.

Gilles
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=130525         




------- Additional Comments From caulier.gilles free fr  2006-07-18 09:26 -------
SVN commit 563680 by cgilles:

digikam from trunk : missing to disable temporally IPTC preview record during download from camera to prevent broken JPEG image if autorotation is used.

Marcel, we need to unforget to fix this part too when a right solution will be found.

CCBUGS: 130525


 M  +7 -1      exifrotate.cpp  


--- trunk/extragear/graphics/digikam/libs/jpegutils/exifrotate.cpp #563679:563680
 @ -230,7 +230,13  @
 
             // Update the image preview.
             QImage preview = img.scale(800, 600, QImage::ScaleMin);
-            metaData.setImagePreview(preview);
+            
+            // TODO: see B.K.O #130525. The a JPEG segment is limited to 64K. If IPTC byte array
+            // bigger than 64K duing of image preview tag size, the target JPEG image will be
+            // broken. Note that IPTC image preview tag is limited to 256K!!!
+            // Temp. solution to disable IPTC preview record in JPEG file until a right solution
+            // will be found into Exiv2.
+            // metaData.setImagePreview(preview);
 
             // Update the image thumbnail.
             QImage thumb = preview.scale(160, 120, QImage::ScaleMin);
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=130525         




------- Additional Comments From ahuggel gmx net  2006-07-22 05:35 -------
This is fixed in Exiv2 rev 847. Exiv2 now throws Exiv2::Error(37) if any JPEG APP segment is >64k but it allows to work with metadata which is larger.
http://dev.robotbattle.com/cmtinfo_svn.php?r=10&v=847

Please check the use of Exiv2::Photoshop::locateIptcIrb and locateIrb in digikam: the signatures of these members were fixed to use uint32_t from uint16_t

#26 : I also can't find any evidence that the size is limited in TIFF format.

Since IPTC is first packed into a Photoshop IRB (specified by Adobe) before it is written to the JPEG APP segment, I am wondering what Adobe applications do in this situation. The available Adobe specs are not clear.

So if anyone can test with Photoshop or similar Adobe app, I'd be interested to hear what happens if you try to save IPTC data >64k to a JPEG image, in particular is it split into two APP13 segments?

-ahu.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=130525         
caulier.gilles free fr changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|general                     |Image Editor
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=130525         




------- Additional Comments From caulier.gilles free fr  2006-08-18 09:17 -------
Is this possible to split into two APP13 segments if size of IPTC is > than 64K ?

Note : ImageMagick remove completly the IPTC segment if the size is > 64K. To test, i have generated a PNG file with an IPTC profile embeded (85Ko) and i have converted it to JPEG. ImageMagick said that IPTC segemnt exit but do not put it in target JPEG file...

Gilles
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-2]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=130525         




------- Additional Comments From caulier.gilles kdemail net  2007-01-04 16:59 -------
Marcel,

I think than i havethe ultimate solution : using JPEG2000 instead JPEG like preview image format embeded.

If you read the IPTC spec, the preview support JPEG2000 format. Since we have a native  JPEG2000 encoder in digiKam core, we can do it easily. Advantage :

- Better quality.
- Better compression ratio.
- And the most important : we can specify to JPEG2000 codec the target image data size in bytes instead the image quality. The quality settings is automaticly computed by the codec. This way is not possible with JPEG codec. Like this we can limit preview data size to 10Kb for example. We will have 54Kb to store the rest of IPTC data. I think it will always be right.

To decode JPEG2000 preview image with Qt is not a problem. Since KDE 3.3, JPEG2000 decoder (not encoder) is provided by KDE API. Normally, we will change nothing in the preview image loader implementation.

What do you think about ?

Gilles
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=130525         




------- Additional Comments From marcel.wiesweg gmx de  2007-01-04 23:27 -------
It's in the IPTC spec, so its standard-compliant, easy to code, and better quality for less size. That sounds great!
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-2]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=130525         




------- Additional Comments From caulier.gilles kdemail net  2007-02-06 22:53 -------
Marcel,

I have implemented the JPEG2000 Preview in my computer. It work fine, the quality and the size of the preview image are better (small and good), but there is a major problem with this solution :

It tooooooo long to decode JPEG2000 image preview with jasper library.

This is want mean than JPEG format is the only solution that we can use for preview. JPEG decoder is really more optimized than JPEG2000.

I will review again your patch to include it in libkexiv2. Perhaps we need to use a dedicaced and optimized JPEG codec to render a small and good JPEG preview file (instead to use the simple QT codec)

All suggestion welcome for that (link to implementation, libjpeg options optimization etc...)

Gilles
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel