Pentax PEF vs DNG in digikam

[Rinus]

Hi list,

If I convert my PEF files to DNG, they look very different. DNG looks
darker.
It is both the same image, both RAW and both lossless. I think it should
look the same. Can someone explain why it does not.

TIA!

Have a nice day,
Rinus
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Karl Günter Wünsch]

On 09/27/11 14:37, sleepless wrote:
> Hi list,
>
> If I convert my PEF files to DNG, they look very different. DNG looks
> darker.
> It is both the same image, both RAW and both lossless. I think it should
> look the same. Can someone explain why it does not.
Because the conversion PEF to DNG loses meta data which will determine
how the image will be processed. In this context the conversion is a
lossy one!
regards
Karl Günter
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Rinus]

Op 27-09-11 14:53, Karl Günter Wünsch schreef:

> On 09/27/11 14:37, sleepless wrote:
>> Hi list,
>>
>> If I convert my PEF files to DNG, they look very different. DNG looks
>> darker.
>> It is both the same image, both RAW and both lossless. I think it should
>> look the same. Can someone explain why it does not.
> Because the conversion PEF to DNG loses meta data which will determine
> how the image will be processed. In this context the conversion is a
> lossy one!

Thanks! That will only affect the embedded jpg I guess. Well indeed it
appears I am viewing ¨half size raw preview¨ from pef and embedded jpg
from DNG.
How can I make the behaviour for both the same way?
Rinus
> regards
> Karl Günter
> _______________________________________________
> Digikam-users mailing list
> [hidden email]
> https://mail.kde.org/mailman/listinfo/digikam-users
>

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Karl Günter Wünsch]

On 09/27/11 16:55, sleepless wrote:
> Thanks! That will only affect the embedded jpg I guess.
No, it also will affect your ability to edit the RAW images - in some
cases certain flaws or oversights in the DNG specification and/or bugs
in the RAW to DNG converter meant that you suffer a severe loss of
quality in the process going from native RAW to DNG.
DNG isn't much more than a stopgap measure for cameras for which there
isn't a proper native RAW developer anymore, it is not a proper
alternative for existing native RAW files if there is a way of
developing directly from them.
> How can I make the behaviour for both the same way?
You can't. DNG has different behaviour from your native RAW format.
regards
Karl Günter
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Gilles Caulier-4]

2011/9/27 Karl Günter Wünsch <[hidden email]>:

> On 09/27/11 14:37, sleepless wrote:
>> Hi list,
>>
>> If I convert my PEF files to DNG, they look very different. DNG looks
>> darker.
>> It is both the same image, both RAW and both lossless. I think it should
>> look the same. Can someone explain why it does not.
> Because the conversion PEF to DNG loses meta data which will determine
> how the image will be processed. In this context the conversion is a
> lossy one!

Definitively NO.

DNG is not lossy format, especially metadata. All is preserved, as markernotes.

The problem is the way  to demosaicing and post process DNG. It sound
like not all process is done in DNG against PEF. This is why image is
dark (no gamma and WB adjustements). These informations are taken in
makernotes, of course.

How to solve it. libraw used by digiKam must be patched in to use
makernotes, for all possible camera makers. This is a huge job, not
simple.

Gilles Caulier
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Rinus]

Thanks Günter and Gilles. It is clear to me. I will stay away from DNG,
no confusions anymore.
Rinus
Op 27-09-11 17:37, Gilles Caulier schreef:

> 2011/9/27 Karl Günter Wünsch<[hidden email]>:
>> On 09/27/11 14:37, sleepless wrote:
>>> Hi list,
>>>
>>> If I convert my PEF files to DNG, they look very different. DNG looks
>>> darker.
>>> It is both the same image, both RAW and both lossless. I think it should
>>> look the same. Can someone explain why it does not.
>> Because the conversion PEF to DNG loses meta data which will determine
>> how the image will be processed. In this context the conversion is a
>> lossy one!
> Definitively NO.
>
> DNG is not lossy format, especially metadata. All is preserved, as markernotes.
>
> The problem is the way  to demosaicing and post process DNG. It sound
> like not all process is done in DNG against PEF. This is why image is
> dark (no gamma and WB adjustements). These informations are taken in
> makernotes, of course.
>
> How to solve it. libraw used by digiKam must be patched in to use
> makernotes, for all possible camera makers. This is a huge job, not
> simple.
>
> Gilles Caulier
> _______________________________________________
> Digikam-users mailing list
> [hidden email]
> https://mail.kde.org/mailman/listinfo/digikam-users
>

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Karl Günter Wünsch]

On Tuesday 27 September 2011, Gilles Caulier wrote:
> DNG is not lossy format, especially metadata. All is preserved, as
markernotes.
This most certainly is wrong. Even (or especially) the Adobe RAW to DNG
converter at times has reinterpreted (I would go as far as saying
misinterpreted) some of the metadata - while preserving the original maker
notes which no one can do anything useful with in a DNG - resulting in for
example wrong black value data which couldn't be corrected because that data
had been falsely derived from some discarded rows and columns of the sensor...
>
> The problem is the way  to demosaicing and post process DNG. It sound
> like not all process is done in DNG against PEF. This is why image is
> dark (no gamma and WB adjustements). These informations are taken in
> makernotes, of course.
They are supposedly taken from the transcribed (from native to DNG standard)
maker notes. As I read the DNG standard, the original maker notes are still
available but must not be used if the DNG standard ones are in existence.
>
> How to solve it. libraw used by digiKam must be patched in to use
> makernotes, for all possible camera makers. This is a huge job, not
> simple.
It's a job which is a useless waste of effort as the existance of solutions
like digikam, ufraw or rawtherapee (all open source and fast evolving
applications) simply make DNG as a format obsolete. There simply is no need
anymore to convert your images from one propriarity file format to a one
propriarity file format (DNG is just a propriarity file format from a
manufacturer who doesn't produce a camera) because the ability to process the
original RAW files may be lost...
regards
Karl Günter
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Gilles Caulier-4]

2011/9/27 Karl Günter Wünsch <[hidden email]>:
> On Tuesday 27 September 2011, Gilles Caulier wrote:
>> DNG is not lossy format, especially metadata. All is preserved, as
> markernotes.
> This most certainly is wrong. Even (or especially) the Adobe RAW to DNG
> converter at times has reinterpreted (I would go as far as saying
> misinterpreted) some of the metadata - while preserving the original maker
> notes which no one can do anything useful with in a DNG - resulting in for
> example wrong black value data which couldn't be corrected because that data
> had been falsely derived from some discarded rows and columns of the sensor...

I know that Adobe converter do not backup makernote properly
(especially for Olympus camera).

This is not true for digiKam DNG converter (i know, i written code (:=))) :

https://projects.kde.org/projects/extragear/graphics/kipi-plugins/repository/revisions/master/entry/dngconverter/dngwriter/dngwriter.cpp#L1034

The problem is raw processor (libraw), which do not use this backup
properly for the moment. At least NO data are lost in DNG

Gilles Caulier
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Karl Günter Wünsch]

On 09/28/11 09:10, Gilles Caulier wrote:
> The problem is raw processor (libraw), which do not use this backup
> properly for the moment.
That backup is worthless in this context as the DNG specification
mandates that the RAW processor is supposed to work solely on the
transcribed subset of maker notes.

> At least NO data are lost in DNG
Maybe in the version you programmed behaves like that but what about the
Adobe one? I have tried DNG back when I still had a Canon 30D - with the
result that the Adobe converter botched the conversion badly.
But even with fully backed up meta data, can you reverse the process and
recreate the original file if you didn't embed it fully? If you can't
recreate the original file then at the very least you lost the ability
to use the specific processes for that file format.
Those two problems together mean that DNG only is a stopgap for cases
where a DNG converter for the RAW file in question exists but no proper
file specific RAW converter is available - and any conversion, no matter
how bad it may be, is preferable to having no conversion available at all.
In the context of the availability of open source programs like digikam,
ufraw, rawtherapee (to name but a few) that can all deal with RAW files
of any kind the need to go through that process of converting to DNG is
not needed any more, it only is needed in the closed source world of
Adobe...
regards
Karl Günter
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Gilles Caulier-4]

2011/9/28 Karl Günter Wünsch <[hidden email]>:
> On 09/28/11 09:10, Gilles Caulier wrote:
>> The problem is raw processor (libraw), which do not use this backup
>> properly for the moment.
> That backup is worthless in this context as the DNG specification
> mandates that the RAW processor is supposed to work solely on the
> transcribed subset of maker notes.

yes, but it's impossible to translate all" undocumented" makernotes
from all camera makers to the uncomplete list of DNG makernotes
transcryption...

So a copy of standard Exif makernotes byte-array from RAW to the
standard Exif makernotes byte-arry to DNG is the safe solution for the
moment.

>
>> At least NO data are lost in DNG
> Maybe in the version you programmed behaves like that but what about the
> Adobe one? I have tried DNG back when I still had a Canon 30D - with the
> result that the Adobe converter botched the conversion badly.

Agree. I tried too, and Adobe try to re-invent the broken wheel. Adobe
cannot reverse-coding all makernotes from all camera makers and models
to convert to the dedicated transcryted tags. It's a way to update DNG
converter for a life. Really a bad solution.

> But even with fully backed up meta data, can you reverse the process and
> recreate the original file if you didn't embed it fully?

makernote is saved as well. So it can be restored. But you cannot
recreated full RAW file of course.

The goal to backup makernotes is for the future improvements of Raw
processor, which will use these info as well in demosaicing.

Gilles Caulier
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Karl Günter Wünsch]

On 09/28/11 10:06, Gilles Caulier wrote:
> The goal to backup makernotes is for the future improvements of Raw
> processor, which will use these info as well in demosaicing.
Why would you invest effort into this lost cause (by your own admission
that the whole idea behind DNG is broken by design)? Why support Adobe
in their effort to dominate the RAW conversion world?
Show a big warning to the extent that DNG is a broken file format when
converting to DNG and keep the DNG development to the necessary minimum
(i.e. the cameras that produce DNG natively and only them).
Sometimes you have to play benevolent dictator in software development
and put failed formats out of their misery, DNG is such a failed format
for which IMHO open source like digikam should only provide the bare
minimum in terms of support to steer the users away from it -
practically for every RAW file format there is a superior native
conversion available to the users of digikam, so they should be guided
to use that instead of the crutch that is called DNG.
regards
Karl Günter
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users