[Bug 233222] New: Thumbnails displayed are from incorrect images

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=233222





--- Comment #19 from Marcel Wiesweg <marcel wiesweg gmx de>  2010-05-20 14:53:08 ---
Are you absolutely sure that digikam was not running in the background while
you copied these images?
That would be quite important here because I indeed assume something is
happening while scanning. Scanning a file which has a file system entry but is
not completely copied yet would nicely explain some problems.

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Simon-2]

https://bugs.kde.org/show_bug.cgi?id=233222





--- Comment #20 from Simon <simon eu gmail com>  2010-05-20 15:04:44 ---
Couldn't this easily be checked by deleting the according image entry from the
database and let digikam re-scan it?

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=233222





--- Comment #21 from  <bugreports familie-becker eu>  2010-05-20 15:24:58 ---
@Simon: That will most likely produce a new correct entry in the DB, as it
seems that the problem has nothing to do with the images itself. See for
example #16, touching the file and restarting digikam does solve the problem
for that file.

@Marc: I'm nearly 100% sure that Digikam was not running while copying. But
besides... it does not make a difference. At least here Digikam does not scan
"live". If I copy (cp, not digikam-copy) a new file to an album directory,
Digikam does not react in any way (just tested this here to be sure). To get
the new pic into the album, I have to scan for new pictures explicitely (which
is by default done on digikam startup here).

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=233222





--- Comment #22 from Marcel Wiesweg <marcel wiesweg gmx de>  2010-05-24 16:59:59 ---
Yes KDirWatch is notoriously broken (or crashes, alternatively) so digikam may
not react to extern file operations. Nonetheless I dont see any reason why at
digikam startup, file scanning should see files with size 0 or 8192 unless they
are currently being written and really have this size. In a short test, the cp
command seems to use a buffer size of 8192. Looking at the code of uniqueHash
generation, it will return a null array if it fails to read any bytes from a
file.
The scanning code in digikam is identical for startup scan or scanning while
running.

So this looks like a race condition between a cp operation and a simultaneously
running file scanner. But I'm unsure how to prevent this. Short research
revealed no reliable method to find out if a file is currently being written
to.

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=233222





--- Comment #23 from Marcel Wiesweg <marcel wiesweg gmx de>  2010-05-24 18:01:22 ---
SVN commit 1130153 by mwiesweg:

Regard a file as modified when its file size has changed, even if the
modification
date did not change.

This seems necessary to prevent problems like bug 233222 to persist: Now the
file
may still be wrongly scanned during the copy operation, but at least at next
rescan it will be corrected. Moreover it seems reasonable to assume a changed
file
when the file size changed (some operations will restore the modification
date).

Overhead: For my collections of 30000 images, scanning took 1550 msec without
the change
and 1750 including the check, with hot caches. With cold caches, the scan took
9300 msec in both cases, no significant difference here.

Note: At first start with this check, there may be a good number of files
rescanned.

CCBUG: 233222
CCMAIL: [hidden email]

 M  +10 -0     collectionscanner.cpp  
 M  +5 -6      imagescanner.cpp  


WebSVN link: http://websvn.kde.org/?view=rev&revision=1130153

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=233222





--- Comment #24 from PCB <bugreports familie-becker eu>  2010-05-25 11:24:38 ---
Hm, I'm still not convinced that the cp operation is the reason for the
error... still sure that cp operation and sync to HDD were already finished in
my case. But as long as Digikams behaviour gets better this way and the false
thumbnails are gone, I'm happy :-) Question is: When will Digikam discover that
the file size has changed? Ok, at least at next rescan. But what if I open up
Digikam, new pics are scanned in, possibly with wrong filesize and thumb again,
and then I open one of the new albums and click on a picture. Is this
sufficient?

Additional comments and thoughts:
1. Is it in any way possible to find out whether a certain file is currently
opened for write operations by another process? I didn't find anything like
that in "man 2 stat", but only had a short look. One other solution could be
that Digikam opens each _new_ file for writing and immediatly closes it again
without modification. As long as only one process may open each file for
writing at a time, this would result in the appropriate error code that could
be checked. Of course, there are (at least) two preconditions: mtime of the
file must not be modified, and it mustn't slow down Digikams scanning
operation. And, of course, this cannot be applied to read-only file systems
(different error code?).

2. Assume we have a file with 0 bytes length and name x.jpg. Even with the
version patched by Marcel this will result in a DB entry without uniqueHash and
with wrong thumbnail. Especially the latter is cleary an error and just mustn't
happen. What about a special treatment for corrupt jpg files (or whatever other
picture format)? I think the thumbnail error could be avoided if every file,
even empty ones, get a uniqueHash. Though I don't know how unique this will be
for empty files... at least this way they could perhaps point to a special
thumbnail indicating "this file is corrupt".

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=233222





--- Comment #25 from Marcel Wiesweg <marcel wiesweg gmx de>  2010-05-25 18:37:44 ---
> Question is: When will Digikam discover that
> the file size has changed? Ok, at least at next rescan. But what if I open up
> Digikam, new pics are scanned in, possibly with wrong filesize and thumb again,
> and then I open one of the new albums and click on a picture. Is this
> sufficient?

For now, pressing F5 (Refresh) should do the trick.

> Additional comments and thoughts:
> 1. Is it in any way possible to find out whether a certain file is currently
> opened for write operations by another process? I didn't find anything like
> that in "man 2 stat", but only had a short look. One other solution could be
> that Digikam opens each _new_ file for writing and immediatly closes it again
> without modification. As long as only one process may open each file for
> writing at a time, this would result in the appropriate error code that could
> be checked. Of course, there are (at least) two preconditions: mtime of the
> file must not be modified, and it mustn't slow down Digikams scanning
> operation. And, of course, this cannot be applied to read-only file systems
> (different error code?).

I have also researched this problem on the net, and your suggestion can be
found there. I'm not completely convinced that it will work, maybe we try. I
dont know about modification date in that case.

>
> 2. Assume we have a file with 0 bytes length and name x.jpg. Even with the
> version patched by Marcel this will result in a DB entry without uniqueHash and
> with wrong thumbnail. Especially the latter is cleary an error and just mustn't
> happen. What about a special treatment for corrupt jpg files (or whatever other
> picture format)? I think the thumbnail error could be avoided if every file,
> even empty ones, get a uniqueHash. Though I don't know how unique this will be
> for empty files... at least this way they could perhaps point to a special
> thumbnail indicating "this file is corrupt".

Yes, a file of size 0 should be a clear warning sign to the scanning process.
We need some mechanisms in place there to handle such problems. (for whatever
reason, this problem never came up over the years. I was wondering why but
accepted that it worked. Obviously, it does not)

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=233222





--- Comment #26 from Gilles Caulier <caulier gilles gmail com>  2010-06-09 00:38:27 ---
*** Bug 217422 has been marked as a duplicate of this bug. ***

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=233222


Marcel Wiesweg <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]




--- Comment #27 from Marcel Wiesweg <marcel wiesweg gmx de>  2010-07-29 10:37:43 ---
*** Bug 240945 has been marked as a duplicate of this bug. ***

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=233222





--- Comment #28 from Alexandre Racine <alexandreracine gmail com>  2010-10-20 02:47:12 ---
Created an attachment (id=52694)
 --> (http://bugs.kde.org/attachment.cgi?id=52694)
WrongSize thumbnail incomplete

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=233222





--- Comment #29 from Alexandre Racine <alexandreracine gmail com>  2010-10-20 02:48:03 ---
Hi all, I am from bug #240945, and now using digikam 1.4 with Ubuntu 10.10.

This does not occurs all the time, but let say one in a while.

I edit a picture with GIMP, when I am back from it, I see something like the
attachement. The original picture is ok, and for the diff one, the thumbnail is
half there, and the size is not correct.

Using the F5 key, does nothing.

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

https://bugs.kde.org/show_bug.cgi?id=233222





--- Comment #30 from Alexandre Racine <alexandreracine gmail com>  2010-12-09 03:42:53 ---
My comment #29 still occur with 1.6

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=233222


Marcel Wiesweg <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]




--- Comment #31 from Marcel Wiesweg <marcel wiesweg gmx de>  2011-06-11 22:51:27 ---
*** Bug 272725 has been marked as a duplicate of this bug. ***

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=233222


Gilles Caulier <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]




--- Comment #32 from Gilles Caulier <caulier gilles gmail com>  2011-12-15 08:49:05 ---
Quentin,

This file still valid using digiKam 2.4 ?

Gilles Caulier

--
Configure bugmail: https://bugs.kde.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

https://bugs.kde.org/show_bug.cgi?id=233222

Marcel Wiesweg <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|UNCONFIRMED                 |RESOLVED
         Resolution|---                         |WAITINGFORINFO

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=233222

Gilles Caulier <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Version Fixed In|                            |2.6.0

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=233222

--- Comment #33 from Gilles Caulier <[hidden email]> ---
New digiKam 4.11.0 is available :

https://www.digikam.org/node/740

Can you reproduce the problem with this release ?

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-4]

https://bugs.kde.org/show_bug.cgi?id=233222

Gilles Caulier <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Version Fixed In|2.6.0                       |4.12.0
         Resolution|WAITINGFORINFO              |FIXED

--- Comment #34 from Gilles Caulier <[hidden email]> ---
Not reproducible with 4.12.0

--
You are receiving this mail because:
You are the assignee for the bug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=233222

[hidden email] changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
          Component|Database                    |Database-Thumbs

--
You are receiving this mail because:
You are the assignee for the bug.