[digiKam-users] Read Error in windows version but not in linux version

[cliver]

Hi all,
I'm new to Digikam and image management in general so please bear with me.

I am trying to organise all my old photos and maintain things ongoing. I
have a collection of older images, mainly jpegs and some newer RAW images
taken from a Nikon D7200 camera. All stored on a local NAS.

I have 2 instances of Digikam installed.
Version 5.6.0 on a laptop running Ubuntu Mate
Version 5.9.0 on an different Laptop running Windows 10

I installed the Ubuntu version first and set it up and it seems to be
running OK. However my windows 10 laptop is far more powerful so I decided
to try the windows version.
On this instance when I set up Digikam the same and add the same NAS  folder
as a collection I get a load of errors when I run Digikam and when I try to
open any of the RAW images downloaded from the camera.

The errors I am getting are:
  Read Error at Scanline xxxxxxx, got 252 bytes, expected 8160
  Unknown Field with tag 36867 (0x9003) encountered
  ASCII value for TAG 'Copyright' contains null byte in value
  Value incorrectly truncated during reading due to implementation
limitations

These errors despite reading exactly the same files in both instances.

So my question:
Have I missed something in the setup on the windows version?
Or is this a bug in the Windows version of Digikam?
Or perhaps a bug in either the Nikon camera software or even windows (heaven
forbid ;)

I am contemplating setting the windows machine to dual boot linux but it's
brand new and still under warranty so would like to at least delay that if I
can.

Can anyone guide me as to the best way forward in getting it to run on
windows?

Thanks and Regards
Clive




--
Sent from: http://digikam.1695700.n4.nabble.com/digikam-users-f1735189.html

[Gilles Caulier-4]

Hi,

This message do not come from digiKam directly, but from Exiv2 shared library. It's not a bug as the message is clear : it's a limitation.

Now the question is why it appear under Windows and not Linux. I don't know. The Question must be posted to Exiv2 team, to be clear.

Best

Gilles Caulier

2018-07-08 13:40 GMT+02:00 cliver <[hidden email]>:

Hi all,
I'm new to Digikam and image management in general so please bear with me.

I am trying to organise all my old photos and maintain things ongoing. I
have a collection of older images, mainly jpegs and some newer RAW images
taken from a Nikon D7200 camera. All stored on a local NAS.

I have 2 instances of Digikam installed.
Version 5.6.0 on a laptop running Ubuntu Mate
Version 5.9.0 on an different Laptop running Windows 10

I installed the Ubuntu version first and set it up and it seems to be
running OK. However my windows 10 laptop is far more powerful so I decided
to try the windows version.
On this instance when I set up Digikam the same and add the same NAS  folder
as a collection I get a load of errors when I run Digikam and when I try to
open any of the RAW images downloaded from the camera.

The errors I am getting are:
  Read Error at Scanline xxxxxxx, got 252 bytes, expected 8160
  Unknown Field with tag 36867 (0x9003) encountered
  ASCII value for TAG 'Copyright' contains null byte in value
  Value incorrectly truncated during reading due to implementation
limitations

These errors despite reading exactly the same files in both instances.

So my question:
Have I missed something in the setup on the windows version?
Or is this a bug in the Windows version of Digikam?
Or perhaps a bug in either the Nikon camera software or even windows (heaven
forbid ;)

I am contemplating setting the windows machine to dual boot linux but it's
brand new and still under warranty so would like to at least delay that if I
can.

Can anyone guide me as to the best way forward in getting it to run on
windows?

Thanks and Regards
Clive




--
Sent from: http://digikam.1695700.n4.nabble.com/digikam-users-f1735189.html

[Simon Cropper]

In my mind this seems to imply that your directory, file names and maybe even your associated metadata have characters not shared between Linux and Windows.

Linux uses UTF-8 while Windows CP-1252. They are different code pages and when characters unique to one or other code page is used in a system that is not code-page aware -- like Windows, the programs get upset.

The only way to resolve the issue is to fix the offending paths, file names or metadata.

This is applicable for any program and is not unique to DigiKam.

Although this may not be the main problem, my understanding is that the path of files are stored as UUID+Path. I am not sure how robust the conversion routines are converting paths between Linux/UNIX format and Windows Format. Some of the issues may relate to this. Certainly most people report problems when moving their data between drives as this changes the UUID.

If I have the time, to test, I'd create one directory with a small number of files. Check all of them for any character that is are not a-z, A-Z, 0-9, space and _

Duplicate on your NAS.

Try creating a database in Linux using one of the directories.
Try creating a database in Windows using the other directory.

Try adding the Linux directory to Windows install and visa versa. I suspect this won't work. If it does, then the issue you are reporting is probably just code page incompatibility and you need to seek out the offending characters.

@Gilles -- This list regularly gets queries about how to setup a central photo repositories and have multiple clients looking into the a single database. In most cases, like Clive its involves mixed operating systems. I have never seen a clear response from the developers if this is even feasible. Can you please clarify?

On 09/07/18 02:01, Gilles Caulier wrote:

Hi,

This message do not come from digiKam directly, but from Exiv2 shared library. It's not a bug as the message is clear : it's a limitation.

Now the question is why it appear under Windows and not Linux. I don't know. The Question must be posted to Exiv2 team, to be clear.

Best

Gilles Caulier

2018-07-08 13:40 GMT+02:00 cliver <[hidden email]>:

Hi all,
I'm new to Digikam and image management in general so please bear with me.

I am trying to organise all my old photos and maintain things ongoing. I
have a collection of older images, mainly jpegs and some newer RAW images
taken from a Nikon D7200 camera. All stored on a local NAS.

I have 2 instances of Digikam installed.
Version 5.6.0 on a laptop running Ubuntu Mate
Version 5.9.0 on an different Laptop running Windows 10

I installed the Ubuntu version first and set it up and it seems to be
running OK. However my windows 10 laptop is far more powerful so I decided
to try the windows version.
On this instance when I set up Digikam the same and add the same NAS  folder
as a collection I get a load of errors when I run Digikam and when I try to
open any of the RAW images downloaded from the camera.

The errors I am getting are:
  Read Error at Scanline xxxxxxx, got 252 bytes, expected 8160
  Unknown Field with tag 36867 (0x9003) encountered
  ASCII value for TAG 'Copyright' contains null byte in value
  Value incorrectly truncated during reading due to implementation
limitations

These errors despite reading exactly the same files in both instances.

So my question:
Have I missed something in the setup on the windows version?
Or is this a bug in the Windows version of Digikam?
Or perhaps a bug in either the Nikon camera software or even windows (heaven
forbid ;)

I am contemplating setting the windows machine to dual boot linux but it's
brand new and still under warranty so would like to at least delay that if I
can.

Can anyone guide me as to the best way forward in getting it to run on
windows?

Thanks and Regards
Clive




--
Sent from: http://digikam.1695700.n4.nabble.com/digikam-users-f1735189.html

[Gilles Caulier-4]

2018-07-09 2:57 GMT+02:00 Simon Cropper <[hidden email]>:

In my mind this seems to imply that your directory, file names and maybe even your associated metadata have characters not shared between Linux and Windows.

Linux uses UTF-8 while Windows CP-1252. They are different code pages and when characters unique to one or other code page is used in a system that is not code-page aware -- like Windows, the programs get upset.

The only way to resolve the issue is to fix the offending paths, file names or metadata.

This is applicable for any program and is not unique to DigiKam.

Although this may not be the main problem, my understanding is that the path of files are stored as UUID+Path. I am not sure how robust the conversion routines are converting paths between Linux/UNIX format and Windows Format. Some of the issues may relate to this. Certainly most people report problems when moving their data between drives as this changes the UUID.

If I have the time, to test, I'd create one directory with a small number of files. Check all of them for any character that is are not a-z, A-Z, 0-9, space and _

Duplicate on your NAS.

Try creating a database in Linux using one of the directories.
Try creating a database in Windows using the other directory.

Try adding the Linux directory to Windows install and visa versa. I suspect this won't work. If it does, then the issue you are reporting is probably just code page incompatibility and you need to seek out the offending characters.

@Gilles -- This list regularly gets queries about how to setup a central photo repositories and have multiple clients looking into the a single database. In most cases, like Clive its involves mixed operating systems. I have never seen a clear response from the developers if this is even feasible. Can you please clarify?

Sharing the database between different Operating system is problematic. Characters encoding is one, event if this can be fixed to use UTF8 everywhere. Qt is able to converter from one encoding to another one easily.

But the most important is enable database lock mechanisms to prevent concurrent access at the same time from different computers. It's can be complex and introduce limitation certainly in this kind of use. We have already few entries in bugzilla about this topic.

In others words, i don't recommend to play in this configuration for the moment, or at least in only this case :

- same type of OS access to a centralized the database (Mysql)

- only one computer use the database, no more than one at the same time.

- collection are shared on the network, and UUID to identify the collection are the same everywhere (else data will be duplicated in database).

Best

Gilles Caulier

[Simon Cropper]

Gilles, Thanks for clarifying the situation. Cheers Simon

On 10/07/18 06:11, Gilles Caulier wrote:

2018-07-09 2:57 GMT+02:00 Simon Cropper <[hidden email]>:

In my mind this seems to imply that your directory, file names and maybe even your associated metadata have characters not shared between Linux and Windows.

Linux uses UTF-8 while Windows CP-1252. They are different code pages and when characters unique to one or other code page is used in a system that is not code-page aware -- like Windows, the programs get upset.

The only way to resolve the issue is to fix the offending paths, file names or metadata.

This is applicable for any program and is not unique to DigiKam.

Although this may not be the main problem, my understanding is that the path of files are stored as UUID+Path. I am not sure how robust the conversion routines are converting paths between Linux/UNIX format and Windows Format. Some of the issues may relate to this. Certainly most people report problems when moving their data between drives as this changes the UUID.

If I have the time, to test, I'd create one directory with a small number of files. Check all of them for any character that is are not a-z, A-Z, 0-9, space and _

Duplicate on your NAS.

Try creating a database in Linux using one of the directories.
Try creating a database in Windows using the other directory.

Try adding the Linux directory to Windows install and visa versa. I suspect this won't work. If it does, then the issue you are reporting is probably just code page incompatibility and you need to seek out the offending characters.

@Gilles -- This list regularly gets queries about how to setup a central photo repositories and have multiple clients looking into the a single database. In most cases, like Clive its involves mixed operating systems. I have never seen a clear response from the developers if this is even feasible. Can you please clarify?

Sharing the database between different Operating system is problematic. Characters encoding is one, event if this can be fixed to use UTF8 everywhere. Qt is able to converter from one encoding to another one easily.

But the most important is enable database lock mechanisms to prevent concurrent access at the same time from different computers. It's can be complex and introduce limitation certainly in this kind of use. We have already few entries in bugzilla about this topic.

In others words, i don't recommend to play in this configuration for the moment, or at least in only this case :

- same type of OS access to a centralized the database (Mysql)

- only one computer use the database, no more than one at the same time.

- collection are shared on the network, and UUID to identify the collection are the same everywhere (else data will be duplicated in database).

Best

Gilles Caulier