using extracted jpeg as "sidecar" for raw files

[Elle Stone]

Hello digikam users,

I don't want to use dng. But I would like a way to tag, caption, etc my raw files. I was thinking of extracting the embedded jpegs and using them as a handy "side-cars" for the raw files for tagging, rating, etc. Has anyone done so? How did it work for you? Any reasons why it is perhaps not such a good idea?

If the jpeg "side-car"carries all the metadata produced by digikam, then if desired I could use exiftool to write selected jpeg metadata back to the corresponding raw file. And the jpeg offers visual confirmation that the jpeg actually belongs to its corresponding raw file.

I've separated all my in-camera-produced jpegs into a separate album from my raw files. I don't shoot simultaneous raw and jpeg (slow, uses a lot of extra space on the card), so there is no problem of confusing an extracted jpeg from a simultaneously-produced jpeg.

dcraw can extract the embedded jped: "dcraw -e *.cr2" (my camera is Canon 400d/xti). Not all the metadata gets carried across from raw to jpeg, but enough does, I think, for my purposes.

exiv2 also can extract an embedded jpeg and keeps some? all? the metadata, but I don't know how to use exiv2 - does anyone have any hints?

Thanks for advice,
Elle Stone

[Robert Zeller]

Hello Elle,

I am shooting both, jpg and raw (in my case Nikon .nef) files. Like you
I am storing the raw and the jpg files in different sub directories of
every album, sub dirs typically called jgp and raw. Since mostly the
jpg's are ok I do all the tagging on the jpgs and store the meta data in
the jpgs. After that is done, I extract the meta data from the jpgs
using exiv2: Typical command:
exiv2 ex *.jpg
; this creates sidecars with the jpgs basename and the extension .exv.
Then I move the .exv sidecar files to the raw subdirectory and write the
metadata to the raw files; using e.g. the following command sequence:

for F in `ls *.nef` ; do
exiv2 -iixe $F
done

the .exv files can be removed after that

you   can obtain all the exiv2 command options by:

man exiv2

Robert Zeller

On 01/06/2011 06:50 PM, Elle Stone wrote:

> Hello digikam users,
>
> I don't want to use dng. But I would like a way to tag, caption, etc my raw
> files. I was thinking of extracting the embedded jpegs and using them as a
> handy "side-cars" for the raw files for tagging, rating, etc. Has anyone
> done so? How did it work for you? Any reasons why it is perhaps not such a
> good idea?
>
> If the jpeg "side-car"carries all the metadata produced by digikam, then if
> desired I could use exiftool to write selected jpeg metadata back to the
> corresponding raw file. And the jpeg offers visual confirmation that the
> jpeg actually belongs to its corresponding raw file.
>
> I've separated all my in-camera-produced jpegs into a separate album from my
> raw files. I don't shoot simultaneous raw and jpeg (slow, uses a lot of
> extra space on the card), so there is no problem of confusing an extracted
> jpeg from a simultaneously-produced jpeg.
>
> dcraw can extract the embedded jped: "dcraw -e *.cr2" (my camera is Canon
> 400d/xti). Not all the metadata gets carried across from raw to jpeg, but
> enough does, I think, for my purposes.
>
> exiv2 also can extract an embedded jpeg and keeps some? all? the metadata,
> but I don't know how to use exiv2 - does anyone have any hints?
>
> Thanks for advice,
> Elle Stone
>  

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Elle Stone]

Robert,

Thank you - sounds like what you are doing is close enough to what I'm
envisioning that using extracted "thumb" jpegs as side-cars will work.
And thanks for the exiv2 tip.

Elle
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Milan Knížek]

Elle Stone píše v Čt 06. 01. 2011 v 09:50 -0800:
> I don't want to use dng. But I would like a way to tag, caption, etc my raw
> files. I was thinking of extracting the embedded jpegs and using them as a
> handy "side-cars" for the raw files for tagging, rating, etc. Has anyone
> done so? How did it work for you? Any reasons why it is perhaps not such a
> good idea?
>
I am the less lucky man (like you) - cr2 are not supported (yet) for
metadata writing.

I keep JPEG and raw files in separate directories and add metadata only
to JPEGs. I am not sure if exporting the metadata to XMP as a sidecar
for the corresponding raw file would provide extra benefit - I have the
JPEG already and intend to keep it. Also updating the sidecar for any
changes done in XX thousand image collections might prove difficult.

With the advent of non-destructive editing and xmp sidecars (digikam2,
darktable, possibly others), it is a question whether to keep the JPEGs
or not. Since I edit metadata for even old images now and then, the
major advantage would be time saving when rsyncing the album with the
external backup (xmp is smaller than raw/JPEG).

Yet, I find it better to continue to use JPEGs with raws - at the end, I
would like to have some final "photos" of all of my "negatives", whose
quality (colour, whatever else) is not dependent on the version of raw
convertor used (who would like to bet that non-destructive editing
recipe will work the same after another five years of raw convertor
development?).

My outstanding problems are:

x keep JPEGs and raws in sync (when I delete JPEG as an unwanted image,
I would like to get rid of the raw, too)

x copying metadata from one JPEG to another version of the same picture
re-developed from raw or processed in external editor, which damages
metadata (now the digiKam's GUI is a bit clumsy - hidden in the menu and
hence slow to use)

Both can be solved by some bash scripts, the latter one by assigning it
to "open with" right-click action. (Though, I have not yet written
them...)

So, that's my workflow.

Milan
--
http://www.milan-knizek.net



_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[ozzyprv]

Milan, I like this comment a lot:

".. at the end, I would like to have some final "photos" of all of my "negatives", whose
quality (colour, whatever else) is not dependent on the version of raw
convertor used "

Never thought of the jpg's that way.

Elle, just out of curiosity. What do you have against DNG? The way I see DNG's is as a secure way to store your negatives. I use Canon, so if one day Canon decides to stop supporting cr2 format I already have my negatives in an open-source digital negative format.

Good conversation!

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Elle Stone]

Milan, thank you, you've provided food for thought. I've used software
that generates xmp files, but they are very faceless little files and
I end up deleting them. I like what you said about the jpeg
side-car/negative being a record of the actual image.

Oz, in theory dng sounds great. But in practice, I'm not confident
that I can get my original Canon raw file out of the dng in one piece.

Also, I tried the digikam dng conversion on a sample raw file and the
resulting "dng-raw" colors came out drastically altered. I use "uniwb"
(http://www.guillermoluijk.com/tutorial/uniwb/index_en.htm;
http://www.malch.com/nikon/UniWB.html) custom color setting in my
camera, so all my raw file thumbnails look (and should look) green.
The digikam dng conversion turned everything magenta and my efforts to
rebalance the dng to realistic colors didn't work so well, whereas
balancing from green to normal colors with dcraw or ufraw is
straightforward.

Also, I use a custom camera profile generated with argyllcms, and dng
is oriented toward using adobe-supplied camera matrices/profiles.

Also, I've read the dng specs (tried to) and I just don't see how dng
makes a closed-source raw file any more open source than dcraw already
does.

On the other hand, if/when I ever purchase a new camera,
in-camera-produced dngs would be a selling point, because then the dng
would be the whole thing, not a funny wrapper around a proprietary raw
file.

Elle


On 1/8/11, Ozzy <[hidden email]> wrote:

> Milan, I like this comment a lot:
>
> ".. at the end, I would like to have some final "photos" of all of my
> "negatives", whose
> quality (colour, whatever else) is not dependent on the version of raw
> convertor used "
>
> Never thought of the jpg's that way.
>
> Elle, just out of curiosity. What do you have against DNG? The way I see
> DNG's is as a secure way to store your negatives. I use Canon, so if one day
> Canon decides to stop supporting cr2 format I already have my negatives in
> an open-source digital negative format.
>
> Good conversation!
>

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[ozzyprv]

Elle,

Thanks for this great information on DNG files/format. Definitely it is food for thoughts!

You are right about digiKam's DNG converter, I experienced the same problem. I added an extra step to my workflow by using Adobe DNG converter (runs fine under WINE).

Cheers.

Oz.

On Sat, Jan 8, 2011 at 7:18 PM, Elle Stone <[hidden email]> wrote:

Milan, thank you, you've provided food for thought. I've used software
that generates xmp files, but they are very faceless little files and
I end up deleting them. I like what you said about the jpeg
side-car/negative being a record of the actual image.

Oz, in theory dng sounds great. But in practice, I'm not confident
that I can get my original Canon raw file out of the dng in one piece.

Also, I tried the digikam dng conversion on a sample raw file and the
resulting "dng-raw" colors came out drastically altered. I use "uniwb"
(http://www.guillermoluijk.com/tutorial/uniwb/index_en.htm;
http://www.malch.com/nikon/UniWB.html) custom color setting in my
camera, so all my raw file thumbnails look (and should look) green.
The digikam dng conversion turned everything magenta and my efforts to
rebalance the dng to realistic colors didn't work so well, whereas
balancing from green to normal colors with dcraw or ufraw is
straightforward.

Also, I use a custom camera profile generated with argyllcms, and dng
is oriented toward using adobe-supplied camera matrices/profiles.

Also, I've read the dng specs (tried to) and I just don't see how dng
makes a closed-source raw file any more open source than dcraw already
does.

On the other hand, if/when I ever purchase a new camera,
in-camera-produced dngs would be a selling point, because then the dng
would be the whole thing, not a funny wrapper around a proprietary raw
file.

Elle

On 1/8/11, Ozzy <[hidden email]> wrote:
> Milan, I like this comment a lot:
>
> ".. at the end, I would like to have some final "photos" of all of my
> "negatives", whose
> quality (colour, whatever else) is not dependent on the version of raw
> convertor used "
>
> Never thought of the jpg's that way.
>
> Elle, just out of curiosity. What do you have against DNG? The way I see
> DNG's is as a secure way to store your negatives. I use Canon, so if one day
> Canon decides to stop supporting cr2 format I already have my negatives in
> an open-source digital negative format.
>
> Good conversation!
>

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Elle Stone]

Oz,

Thanks! for the tip about Adobe's dng converter - I didn't realize it
would run under wine. I might just give it a try.

Elle

On 1/8/11, Ozzy <[hidden email]> wrote:

> Elle,
>
> Thanks for this great information on DNG files/format. Definitely it is food
> for thoughts!
>
> You are right about digiKam's DNG converter, I experienced the same problem.
> I added an extra step to my workflow by using Adobe DNG converter (runs fine
> under WINE).
>
> Cheers.
>
> Oz.
>
>
> On Sat, Jan 8, 2011 at 7:18 PM, Elle Stone <[hidden email]> wrote:
>
>> Milan, thank you, you've provided food for thought. I've used software
>> that generates xmp files, but they are very faceless little files and
>> I end up deleting them. I like what you said about the jpeg
>> side-car/negative being a record of the actual image.
>>
>> Oz, in theory dng sounds great. But in practice, I'm not confident
>> that I can get my original Canon raw file out of the dng in one piece.
>>
>> Also, I tried the digikam dng conversion on a sample raw file and the
>> resulting "dng-raw" colors came out drastically altered. I use "uniwb"
>> (http://www.guillermoluijk.com/tutorial/uniwb/index_en.htm;
>> http://www.malch.com/nikon/UniWB.html) custom color setting in my
>> camera, so all my raw file thumbnails look (and should look) green.
>> The digikam dng conversion turned everything magenta and my efforts to
>> rebalance the dng to realistic colors didn't work so well, whereas
>> balancing from green to normal colors with dcraw or ufraw is
>> straightforward.
>>
>> Also, I use a custom camera profile generated with argyllcms, and dng
>> is oriented toward using adobe-supplied camera matrices/profiles.
>>
>> Also, I've read the dng specs (tried to) and I just don't see how dng
>> makes a closed-source raw file any more open source than dcraw already
>> does.
>>
>> On the other hand, if/when I ever purchase a new camera,
>> in-camera-produced dngs would be a selling point, because then the dng
>> would be the whole thing, not a funny wrapper around a proprietary raw
>> file.
>>
>> Elle
>>
>>
>> On 1/8/11, Ozzy <[hidden email]> wrote:
>> > Milan, I like this comment a lot:
>> >
>> > ".. at the end, I would like to have some final "photos" of all of my
>> > "negatives", whose
>> > quality (colour, whatever else) is not dependent on the version of raw
>> > convertor used "
>> >
>> > Never thought of the jpg's that way.
>> >
>> > Elle, just out of curiosity. What do you have against DNG? The way I see
>> > DNG's is as a secure way to store your negatives. I use Canon, so if one
>> day
>> > Canon decides to stop supporting cr2 format I already have my negatives
>> in
>> > an open-source digital negative format.
>> >
>> > Good conversation!
>> >
>> _______________________________________________
>> Digikam-users mailing list
>> [hidden email]
>> https://mail.kde.org/mailman/listinfo/digikam-users
>>
>

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users