[digiKam-users] managing old documents

Hello :-)

What I will ask is not about digikam, but showfoto or may be GIMP.

work in progress:

http://dodin.org/luciendodin/index.php?n=Biographie.LucienDodinPereCahier

(sorry, only in french)

It's a document from 1883 (yes, no error :-) by my gran father.

Given the nature of the document: very fragile old paper, too large for
a scanner (I couldn't find an A3 scanner at reasonable price), I use
photos. Now from a smartphone moto G4+, very good image wise.

But. I get results like this one:

https://www.cjoint.com/doc/20_05/JEgivUDTH5k_IMG-20200505-144724.jpg

pretty hard to read :-(

On a photo forum, a friend said he could have a pretty good result with
an application I don't know "color fx pro", obviously not on Linux :-(,
that is already much better than the original (for my purpose: better
readability)

https://www.cjoint.com/doc/20_05/JEgiMLo0vzP_Untitled.JPG

the setup is:

https://www.cjoint.com/doc/20_05/JEgiMk8lNnP_Capture-d%E2%80%99e%CC%81cran-2020-05-06-a%CC%80-10.37.48.png

obviously it's a tone control that makes clear tones lighters and medium
one (text) darker

is there a way to have a similar (or better result with showfoto?
eventually with GIMP (I'm much more familiar with showfoto than from gimp)?

thanks
jdd
--
http://dodin.org

[Gilles Caulier-4]

Hi,

With this kind of documents, i use :

-Restoration to remove artifact and increase sharpness.
-Sharp again with Refocus or Unsharp Mask
-Change the Gamma or the Contrast.
-Reduce colors with White Balance : it's better for text
-And save in lossless format of course for archiving.

All can be done in BQM in batch. Just found the right settings, save
in a workflow and voilà... take a coffee and process your items in
batch.

Note: in recent release, i integrated the GMicQt tool as digiKam and
showfoto plugin. This tool has plenty of advanced and intelligent
filters (too much in fact). Experiment, test, and report here of
course.

https://gmic.eu/

As French is your first language, take a look here and enjoy :

https://linuxfr.org/news/g-mic-2-7-0-une-rentree-pleine-de-style-pour-le-traitement-d-images

GMic is a monster...

Best regards

Gilles Caulier

Le mer. 6 mai 2020 à 11:17, [hidden email] <[hidden email]> a écrit :

>
> Hello :-)
>
> What I will ask is not about digikam, but showfoto or may be GIMP.
>
> work in progress:
>
> http://dodin.org/luciendodin/index.php?n=Biographie.LucienDodinPereCahier
>
> (sorry, only in french)
>
> It's a document from 1883 (yes, no error :-) by my gran father.
>
> Given the nature of the document: very fragile old paper, too large for
> a scanner (I couldn't find an A3 scanner at reasonable price), I use
> photos. Now from a smartphone moto G4+, very good image wise.
>
> But. I get results like this one:
>
> https://www.cjoint.com/doc/20_05/JEgivUDTH5k_IMG-20200505-144724.jpg
>
> pretty hard to read :-(
>
> On a photo forum, a friend said he could have a pretty good result with
> an application I don't know "color fx pro", obviously not on Linux :-(,
> that is already much better than the original (for my purpose: better
> readability)
>
> https://www.cjoint.com/doc/20_05/JEgiMLo0vzP_Untitled.JPG
>
> the setup is:
>
> https://www.cjoint.com/doc/20_05/JEgiMk8lNnP_Capture-d%E2%80%99e%CC%81cran-2020-05-06-a%CC%80-10.37.48.png
>
> obviously it's a tone control that makes clear tones lighters and medium
> one (text) darker
>
> is there a way to have a similar (or better result with showfoto?
> eventually with GIMP (I'm much more familiar with showfoto than from gimp)?
>
> thanks
> jdd
> --
> http://dodin.org

[Remco Viëtor]

On mercredi 6 mai 2020 11:30:19 CEST Gilles Caulier wrote:

> Hi,
>
> With this kind of documents, i use :
>
> -Restoration to remove artifact and increase sharpness.
> -Sharp again with Refocus or Unsharp Mask
> -Change the Gamma or the Contrast.
> -Reduce colors with White Balance : it's better for text
> -And save in lossless format of course for archiving.
>
> All can be done in BQM in batch. Just found the right settings, save
> in a workflow and voilà... take a coffee and process your items in
> batch.
>
> Note: in recent release, i integrated the GMicQt tool as digiKam and
> showfoto plugin. This tool has plenty of advanced and intelligent
> filters (too much in fact). Experiment, test, and report here of
> course.
>
> https://gmic.eu/
>
> As French is your first language, take a look here and enjoy :
>
> https://linuxfr.org/news/g-mic-2-7-0-une-rentree-pleine-de-style-pour-le-tra
> itement-d-images
>
> GMic is a monster...
>
> Best regards
>
> Gilles Caulier
>
> Le mer. 6 mai 2020 à 11:17, [hidden email] <[hidden email]> a écrit :
> > Hello :-)
> >
> > What I will ask is not about digikam, but showfoto or may be GIMP.
> >
> > work in progress:
> >
> > http://dodin.org/luciendodin/index.php?n=Biographie.LucienDodinPereCahier
> >
> > (sorry, only in french)
> >
> > It's a document from 1883 (yes, no error :-) by my gran father.
> >
> > Given the nature of the document: very fragile old paper, too large for
> > a scanner (I couldn't find an A3 scanner at reasonable price), I use
> > photos. Now from a smartphone moto G4+, very good image wise.
> >
> > But. I get results like this one:
> >
> > https://www.cjoint.com/doc/20_05/JEgivUDTH5k_IMG-20200505-144724.jpg
> >
> > pretty hard to read :-(
> >
> > On a photo forum, a friend said he could have a pretty good result with
> > an application I don't know "color fx pro", obviously not on Linux :-(,
> > that is already much better than the original (for my purpose: better
> > readability)
> >
> > https://www.cjoint.com/doc/20_05/JEgiMLo0vzP_Untitled.JPG
> >
> > the setup is:
> >
> > https://www.cjoint.com/doc/20_05/JEgiMk8lNnP_Capture-d%E2%80%99e%CC%81cran
> > -2020-05-06-a%CC%80-10.37.48.png
> >
> > obviously it's a tone control that makes clear tones lighters and medium
> > one (text) darker
> >
> > is there a way to have a similar (or better result with showfoto?
> > eventually with GIMP (I'm much more familiar with showfoto than from
> > gimp)?
> >
> > thanks
> > jdd
> > --
> > http://dodin.org

In this particular case, what worked rather well for me was channel mixer to
grayscale with 100% red, followed by using levels (or curve) to improve
contrast, and a dose of local contrast.

But as usual, there are several ways to get an improved readability of the
text.

[Sveinn í Felli-2]

Salut,

One first thing; improve your original image as much as possible.
There's far too much difference of light between the upper and lower
parts of the photo [1], and there's not enough light. Smartphones suffer
in low lighting conditions. Try to illuminate the manuscript with at
least two steady light sources, ideally ones where you can control the
frequency and/or strength. Camera and lights on tripods, please.

You could then experiment with what Gilles suggested, personally I'd
though start by fiddling with "Levels" either in Digikam or Gimp. Once
you have a good recipe, you could proceed to batch-processing (but
according my experience of such old tanned paper, manual tuning will
always be necessary).

Here are some screenshots:

Levels in Gimp:
<http://sv1.fellsnet.is/myndvinnsla/dodin/Skj%C3%A1mynd%20fr%C3%A1%202020-05-06%2009-31-52.png>

Levels in Digikam:
<http://sv1.fellsnet.is/myndvinnsla/dodin/Skj%C3%A1mynd%20fr%C3%A1%202020-05-06%2009-38-27.png>

[1]:
<http://sv1.fellsnet.is/myndvinnsla/dodin/JEgivUDTH5k_IMG-20200505-144724_v1.jpg>

Just thoughts,
Sveinn í Felli


Þann 6.5.2020 09:17, skrifaði [hidden email]:

> Hello :-)
>
> What I will ask is not about digikam, but showfoto or may be GIMP.
>
> work in progress:
>
> http://dodin.org/luciendodin/index.php?n=Biographie.LucienDodinPereCahier
>
> (sorry, only in french)
>
> It's a document from 1883 (yes, no error :-) by my gran father.
>
> Given the nature of the document: very fragile old paper, too large for
> a scanner (I couldn't find an A3 scanner at reasonable price), I use
> photos. Now from a smartphone moto G4+, very good image wise.
>
> But. I get results like this one:
>
> https://www.cjoint.com/doc/20_05/JEgivUDTH5k_IMG-20200505-144724.jpg
>
> pretty hard to read :-(
>
> On a photo forum, a friend said he could have a pretty good result with
> an application I don't know "color fx pro", obviously not on Linux :-(,
> that is already much better than the original (for my purpose: better
> readability)
>
> https://www.cjoint.com/doc/20_05/JEgiMLo0vzP_Untitled.JPG
>
> the setup is:
>
> https://www.cjoint.com/doc/20_05/JEgiMk8lNnP_Capture-d%E2%80%99e%CC%81cran-2020-05-06-a%CC%80-10.37.48.png 
>
>
> obviously it's a tone control that makes clear tones lighters and medium
> one (text) darker
>
> is there a way to have a similar (or better result with showfoto?
> eventually with GIMP (I'm much more familiar with showfoto than from gimp)?
>
> thanks
> jdd

[leoutation]

You'll get much time  and better result with taking an other photo with
better light conditions. Your photo has contrast distribution problem.
It's darker at the bottom of the picture, maybe because you shot indoor
near a window. So, it's not easy to increase its contrast...
Try to shot again, *outdoor*, when sky is cloudy, with no shadow.

On 5/6/20 11:17 AM, [hidden email] wrote:

> Hello :-)
>
> What I will ask is not about digikam, but showfoto or may be GIMP.
>
> work in progress:
>
> http://dodin.org/luciendodin/index.php?n=Biographie.LucienDodinPereCahier
>
> (sorry, only in french)
>
> It's a document from 1883 (yes, no error :-) by my gran father.
>
> Given the nature of the document: very fragile old paper, too large for
> a scanner (I couldn't find an A3 scanner at reasonable price), I use
> photos. Now from a smartphone moto G4+, very good image wise.
>
> But. I get results like this one:
>
> https://www.cjoint.com/doc/20_05/JEgivUDTH5k_IMG-20200505-144724.jpg
>
> pretty hard to read :-(
>
> On a photo forum, a friend said he could have a pretty good result with
> an application I don't know "color fx pro", obviously not on Linux :-(,
> that is already much better than the original (for my purpose: better
> readability)
>
> https://www.cjoint.com/doc/20_05/JEgiMLo0vzP_Untitled.JPG
>
> the setup is:
>
> https://www.cjoint.com/doc/20_05/JEgiMk8lNnP_Capture-d%E2%80%99e%CC%81cran-2020-05-06-a%CC%80-10.37.48.png
>
>
> obviously it's a tone control that makes clear tones lighters and medium
> one (text) darker
>
> is there a way to have a similar (or better result with showfoto?
> eventually with GIMP (I'm much more familiar with showfoto than from gimp)?
>
> thanks
> jdd

--
Maderios

Le 06/05/2020 à 14:31, [hidden email] a écrit :

> Try to shot again, *outdoor*, when sky is cloudy, with no shadow.
>

god, please, give me clouds, no wind and no rain :-)

sure it would be easier

I'll have to build a better light system, for sure

thanks
jdd

--
http://dodin.org

Le 06/05/2020 à 12:41, Remco Viëtor a écrit :

> In this particular case, what worked rather well for me was channel mixer to
> grayscale with 100% red, followed by using levels (or curve) to improve
> contrast, and a dose of local contrast.
>
> But as usual, there are several ways to get an improved readability of the
> text.
>
>
>
I noticed that I can shoot B&W on the phone and this is already better

thanks
jdd

--
http://dodin.org

[Sveinn í Felli-2]

Þann 6.5.2020 13:12, skrifaði [hidden email]:
> Le 06/05/2020 à 14:31, [hidden email] a écrit :
>
> I'll have to build a better light system, for sure
>

And pressing the documents flat with a glass plate enhances it even
further, beware of reflections though...

Good luck,
Sveinn í Felli

Le 06/05/2020 à 15:24, Sveinn í Felli a écrit :
> Þann 6.5.2020 13:12, skrifaði [hidden email]:
>> Le 06/05/2020 à 14:31, [hidden email] a écrit :
>>
>> I'll have to build a better light system, for sure
>>
>
> And pressing the documents flat with a glass plate enhances it even
> further, beware of reflections though...
>

yes, very difficult to prevent reflexions :-(

jdd



--
http://dodin.org

Le 06/05/2020 à 11:30, Gilles Caulier a écrit :
> Hi,
>
> With this kind of documents, i use :
>
> -Restoration to remove artifact and increase sharpness.
> -Sharp again with Refocus or Unsharp Mask
> -Change the Gamma or the Contrast.
> -Reduce colors with White Balance : it's better for text
> -And save in lossless format of course for archiving.

did not give really good results (until now)

>
> All can be done in BQM in batch. Just found the right settings, save
> in a workflow and voilà... take a coffee and process your items in
> batch.

I use BQM, but there are too many difference between documents to use it
here

>
> Note: in recent release, i integrated the GMicQt tool as digiKam and
> showfoto plugin. This tool has plenty of advanced and intelligent
> filters (too much in fact). Experiment, test, and report here of
> course.

In didn't find all in shofoto, but anyway it's a bit long to learn just
now. May be in the future

> GMic is a monster...

that's a problem as well as the solution :-(

thanks
jdd
--
http://dodin.org

[leoutation]

On 5/6/20 3:12 PM, [hidden email] wrote:
> Le 06/05/2020 à 14:31, [hidden email] a écrit :
>
>> Try to shot again, *outdoor*, when sky is cloudy, with no shadow.
>>
>
> god, please, give me clouds, no wind and no rain :-)

You don't need clouds to get diffused light, with no shadows.
Shot outdoor, under a tree, under anything to protect from direct sun
spot light. Best result when shooting under a white sheet in a sunny day
>
> sure it would be easier
>
> I'll have to build a better light system, for sure
>
> thanks
> jdd
>

--
Maderios

[Gilles Caulier-4]

Gmic plugin is only integrated officially in all bundles (Windows,
MacOS and AppImage).

For a native package, you need to install GMicQt as well. It's a stand
alone plugin which much be installed outside digiKam.

Best

Gilles


Le mer. 6 mai 2020 à 15:56, [hidden email] <[hidden email]> a écrit :

>
> Le 06/05/2020 à 11:30, Gilles Caulier a écrit :
> > Hi,
> >
> > With this kind of documents, i use :
> >
> > -Restoration to remove artifact and increase sharpness.
> > -Sharp again with Refocus or Unsharp Mask
> > -Change the Gamma or the Contrast.
> > -Reduce colors with White Balance : it's better for text
> > -And save in lossless format of course for archiving.
>
> did not give really good results (until now)
>
> >
> > All can be done in BQM in batch. Just found the right settings, save
> > in a workflow and voilà... take a coffee and process your items in
> > batch.
>
> I use BQM, but there are too many difference between documents to use it
> here
>
> >
> > Note: in recent release, i integrated the GMicQt tool as digiKam and
> > showfoto plugin. This tool has plenty of advanced and intelligent
> > filters (too much in fact). Experiment, test, and report here of
> > course.
>
> In didn't find all in shofoto, but anyway it's a bit long to learn just
> now. May be in the future
>
> > GMic is a monster...
>
> that's a problem as well as the solution :-(
>
> thanks
> jdd
> --
> http://dodin.org

Le 06/05/2020 à 16:23, [hidden email] a écrit :

> You don't need clouds to get diffused light, with no shadows.
> Shot outdoor, under a tree,

under a tree is the worst, ray of shadows all the time :-(

  under anything to protect from direct sun
> spot light. Best result when shooting under a white sheet in a sunny day

yep, but difficult to manage the shadows, there a always some light rays

I even have some glasses pretending being anti-glare but not good :-(

thanks
jdd


--
http://dodin.org

Le 06/05/2020 à 16:32, Gilles Caulier a écrit :
> Gmic plugin is only integrated officially in all bundles (Windows,
> MacOS and AppImage).
>

I mostly use appimage, but probably too old (6.3?) I will update asap

thanks
jdd


--
http://dodin.org

[Remco Viëtor]

On mercredi 6 mai 2020 17:15:25 CEST [hidden email] wrote:

> Le 06/05/2020 à 16:23, [hidden email] a écrit :
> > You don't need clouds to get diffused light, with no shadows.
> > Shot outdoor, under a tree,
>
> under a tree is the worst, ray of shadows all the time :-(
>
>   under anything to protect from direct sun
>
> > spot light. Best result when shooting under a white sheet in a sunny day
>
> yep, but difficult to manage the shadows, there a always some light rays
>
> I even have some glasses pretending being anti-glare but not good :-(
>
> thanks
> jdd

Classical setup I seem to remember is indoors, 2 lamps at 45° to the flat
subject (perhaps under glass) and camera on a tripod.

If the lamps are far enough away, that makes any specular reflections on the
flat subject or the glass plate impossible. Using 2 on opposite sides of the
subject helps getting a homogenous lighting.

Remco

[patdavid]

jdd,

You might also get some advice on the pixls.us forums - there was a post a while back to clean up images of books: https://discuss.pixls.us/t/working-with-afre-cleantext-filter-and-gmic-plugin

One of the people in that thread even provided some g'mic code (filter) to clean things up a bit (afre).  It might be possible that you can call that filter from G'MIC?

On Wed, May 6, 2020 at 10:16 AM [hidden email] <[hidden email]> wrote:

Le 06/05/2020 à 16:32, Gilles Caulier a écrit :
> Gmic plugin is only integrated officially in all bundles (Windows,
> MacOS and AppImage).
>

I mostly use appimage, but probably too old (6.3?) I will update asap

thanks
jdd


--
http://dodin.org

--

[James Orr-3]

When other attempts fail, one can often get good results finding diffuse light in the north-side shade of a structure. A taller one is generally better. The shade does nothing at all for the wind.

On 5/6/2020 8:15 AM, [hidden email] wrote:

Le 06/05/2020 à 16:23, [hidden email] a écrit :

You don't need clouds to get diffused light, with no shadows.
Shot outdoor, under a tree,

under a tree is the worst, ray of shadows all the time :-(

 under anything to protect from direct sun

spot light. Best result when shooting under a white sheet in a sunny day

yep, but difficult to manage the shadows, there a always some light rays

I even have some glasses pretending being anti-glare but not good :-(

thanks
jdd

[Sveinn í Felli-2]

Not sure it's always allowed to take such old documents outside, plus
you have much *less control* over the light, compared to an indoor setup.

Agree with Remco, a couple of similar strength lights at 45° could be
enough.

But controlling the frequency of the lights can be the key to
eliminating the brownish hue of the paper and at the same time enhance
the contrast of the text.

Be creative and test; UV-lights, optical filters, long exposures,
multilayer HDR, etc.

Don't recommend wetting the manuscripts in gasoline as people apparently
did before the 1920s...

Sveinn í Felli


Þann 7.5.2020 21:45, skrifaði James Orr:

> When other attempts fail, one can often get good results finding diffuse light in the north-side shade of a structure. A taller one is generally better. The shade does nothing at all for the wind.
> James M. Orr
> ________________________________
>
> 2705 E Sussex Way
>
> Fresno CA 93726
>
> I sincerely wish each of you, your family, friends, contacts, community the best of health in these times of pestilence and disease. There is no foreseeable end to proximity-distancing, and with that a concomitant need for social glue.
>
> ________________________________
> ________________________________
> On 5/6/2020 8:15 AM, [hidden email]<mailto:[hidden email]> wrote:
> Le 06/05/2020 à 16:23, [hidden email]<mailto:[hidden email]> a écrit :
>
> You don't need clouds to get diffused light, with no shadows.
> Shot outdoor, under a tree,
>
> under a tree is the worst, ray of shadows all the time :-(
>
>   under anything to protect from direct sun
> spot light. Best result when shooting under a white sheet in a sunny day
>
> yep, but difficult to manage the shadows, there a always some light rays
>
> I even have some glasses pretending being anti-glare but not good :-(
>
> thanks
> jdd
>
>
>

Le 07/05/2020 à 23:45, James Orr a écrit :
> When other attempts fail, one can often get good results finding diffuse
> light in the north-side shade of a structure. A taller one is generally
> better. The shade does nothing at all for the wind.

thanks
jdd


--
http://dodin.org