Adjusting filters to be deterministic
Locked
Adjusting filters to be deterministic
 
|
Hi,
JulienN raised a good idea in comments under my blogpost. Rewrite the filters to be deterministic and thus fully reproducible. This would help a lot for non-destruct. editng. I would go with leaving the random init part, but alter the filters in such way, that we can store the output values and reuse it later with those exact same values to get an exact same output.
The benefits would be quite big for NDE as now the concept is such, that with using any non-reproducible filter, new file must be created to preserve this exact output and also, any edit you did prior that will be unremovable without losing this output. Example: you use brightness, contrast and raindrops and then some other filters. Now you see that the initial brightness is not very good so you want to change/remove it. But if you do so, you must reapply all the filters in the stack which are after brightness. But since you can't reapply the raindrops, you need to lose this and that means you will lose any other edits you did after raindrops too.
What do you think? Would it be possible to alter all the filters so they could be reproduced to get the exact same output? I don't have very good code knowledge of them. Marty
_______________________________________________ Digikam-devel mailing list [hidden email] https://mail.kde.org/mailman/listinfo/digikam-devel |
Re: Adjusting filters to be deterministic
|
|
> JulienN raised a good idea in comments under my blogpost. Rewrite the > filters to be deterministic and thus fully reproducible. This would help a > lot for non-destruct. editng. I would go with leaving the random init part, > but alter the filters in such way, that we can store the output values and > reuse it later with those exact same values to get an exact same output. > > The benefits would be quite big for NDE as now the concept is such, that > with using any non-reproducible filter, new file must be created to > preserve this exact output and also, any edit you did prior that will be > unremovable without losing this output. Example: you use brightness, > contrast and raindrops and then some other filters. Now you see that the > initial brightness is not very good so you want to change/remove it. But > if you do so, you must reapply all the filters in the stack which are > after > brightness. But since you can't reapply the raindrops, you need to lose > this and that means you will lose any other edits you did after raindrops > too. > > What do you think? Would it be possible to alter all the filters so they > could be reproduced to get the exact same output? I don't have very good > code knowledge of them. Yes, it's a very interesting point. 1) Which filters are affected? A search for rand reveals: - BlurFX - DistortionFX - Filmgrain - Raindrop - localcontrast - in the CImg library I would separate out one group of filters: Those depending on very complex, possibly externally develop algorithms like CImg. For these, the algorithm can change with library updates, we dont have full control anyway. I am not sure which other plugins fall into this group; localcontrast? Gilles knows more about that. So lets talk about the four of BlurFX, DistortionFX, Filmgrain, Raindrop: 2) Possible approaches to make the random values deterministic a) store every single randomly generated value. This breaks when a large number of values is generated, like one value per pixel. It works if only a few dozen values are involved. b) When now a random value is generated per pixel, devise a solution to reduce this randomness to fewer calls to rand(). Problem: This changes the core of the algorithm, feasibility unknown for each single case. c) Store the random seed of the pseudo-random generator. With a real pseudo- random generator, the random values would then be determined. Problem: Depends on the underlying rand() implementation provided, and differing, by the platform. d) Use a defined pseudo-random generator, and store the seed, or the random number generator seed (available in binary form). The GSL provides a number of such algorithms: http://www.gnu.org/software/gsl/manual/html_node/Random-Number-Generation.html Now using the GSL directly is not a good choice, it is GPLv3 only (sigh...) and would be a big dependency. Ripping out one selected algorithm, be it from glibc, BSD, or GSL 1.9 (GPLv2), should be feasible, if someone sits down with patience and finds the right source files. More ideas are welcome ;-) _______________________________________________ Digikam-devel mailing list [hidden email] https://mail.kde.org/mailman/listinfo/digikam-devel |
Re: Adjusting filters to be deterministic
|
|
2010/6/17 Marcel Wiesweg <[hidden email]>:
> >> JulienN raised a good idea in comments under my blogpost. Rewrite the >> filters to be deterministic and thus fully reproducible. This would help a >> lot for non-destruct. editng. I would go with leaving the random init part, >> but alter the filters in such way, that we can store the output values and >> reuse it later with those exact same values to get an exact same output. >> >> The benefits would be quite big for NDE as now the concept is such, that >> with using any non-reproducible filter, new file must be created to >> preserve this exact output and also, any edit you did prior that will be >> unremovable without losing this output. Example: you use brightness, >> contrast and raindrops and then some other filters. Now you see that the >> initial brightness is not very good so you want to change/remove it. But >> if you do so, you must reapply all the filters in the stack which are >> after >> brightness. But since you can't reapply the raindrops, you need to lose >> this and that means you will lose any other edits you did after raindrops >> too. >> >> What do you think? Would it be possible to alter all the filters so they >> could be reproduced to get the exact same output? I don't have very good >> code knowledge of them. > > Yes, it's a very interesting point. > > 1) Which filters are affected? A search for rand reveals: > - BlurFX > - DistortionFX > - Filmgrain > - Raindrop > - localcontrast > - in the CImg library > > I would separate out one group of filters: Those depending on very complex, > possibly externally develop algorithms like CImg. For these, the algorithm can > change with library updates, we dont have full control anyway. CImg is included in digiKam. So the problem can be wrap around. But... last CImg do not include GreyStoration algoritm now. a new library have been developped and is named GMic... I already take a look into this lib, and i lost. It's difficult to use with digiKam > I am not sure which other plugins fall into this group; localcontrast? Gilles > knows more about that. I'm not sure to understand. Why localcontrast fall into this group. it just a color correction tool as BCG, or HSL... Gilles _______________________________________________ Digikam-devel mailing list [hidden email] https://mail.kde.org/mailman/listinfo/digikam-devel |
Re: Adjusting filters to be deterministic
|
|
ALGLIB (GPLv2) might be worth a look...
On Thu, Jun 17, 2010 at 8:08 PM, Gilles Caulier <[hidden email]> wrote: 2010/6/17 Marcel Wiesweg <[hidden email]>: -- Aditya Bhatt Blog : http://adityabhatt.wordpress.com Bitbucket: http://bitbucket.org/aditya_bhatt Face Recognition Library : http://libface.sourceforge.net _______________________________________________ Digikam-devel mailing list [hidden email] https://mail.kde.org/mailman/listinfo/digikam-devel |
Re: Adjusting filters to be deterministic
|
|
In reply to this post by Marcel Wiesweg
Le 17/06/2010 16:34, Marcel Wiesweg a écrit :
> >> JulienN raised a good idea in comments under my blogpost. Rewrite the >> filters to be deterministic and thus fully reproducible. This would help a >> lot for non-destruct. editng. I would go with leaving the random init part, >> but alter the filters in such way, that we can store the output values and >> reuse it later with those exact same values to get an exact same output. >> >> The benefits would be quite big for NDE as now the concept is such, that >> with using any non-reproducible filter, new file must be created to >> preserve this exact output and also, any edit you did prior that will be >> unremovable without losing this output. Example: you use brightness, >> contrast and raindrops and then some other filters. Now you see that the >> initial brightness is not very good so you want to change/remove it. But >> if you do so, you must reapply all the filters in the stack which are >> after >> brightness. But since you can't reapply the raindrops, you need to lose >> this and that means you will lose any other edits you did after raindrops >> too. >> >> What do you think? Would it be possible to alter all the filters so they >> could be reproduced to get the exact same output? I don't have very good >> code knowledge of them. >> > Yes, it's a very interesting point. > > 1) Which filters are affected? A search for rand reveals: > - BlurFX > - DistortionFX > - Filmgrain > > - Raindrop > - localcontrast > - in the CImg library > > I would separate out one group of filters: Those depending on very complex, > possibly externally develop algorithms like CImg. For these, the algorithm can > change with library updates, we dont have full control anyway. > I am not sure which other plugins fall into this group; localcontrast? Gilles > knows more about that. > > So lets talk about the four of BlurFX, DistortionFX, Filmgrain, Raindrop: > > > 2) Possible approaches to make the random values deterministic > a) store every single randomly generated value. This breaks when a large > number of values is generated, like one value per pixel. It works if only a > few dozen values are involved. > b) When now a random value is generated per pixel, devise a solution to reduce > this randomness to fewer calls to rand(). Problem: This changes the core of > the algorithm, feasibility unknown for each single case. > c) Store the random seed of the pseudo-random generator. With a real pseudo- > random generator, the random values would then be determined. > Problem: Depends on the underlying rand() implementation provided, and > differing, by the platform. > d) Use a defined pseudo-random generator, and store the seed, or the random > number generator seed (available in binary form). The GSL provides a number of > such algorithms: > http://www.gnu.org/software/gsl/manual/html_node/Random-Number-Generation.html > Now using the GSL directly is not a good choice, it is GPLv3 only (sigh...) > and would be a big dependency. Ripping out one selected algorithm, be it from > glibc, BSD, or GSL 1.9 (GPLv2), should be feasible, if someone sits down with > patience and finds the right source files. > > A sequence of random numbers is supposed to have two properties: -1- the next number should not be predictable -2- all numbers should appear with the same probability For Digikam we don't need 1 in fact on the contrary we want a deterministic sequence of random numbers. For this why not just use a constant as seed initialized at the start of the filter, for instance the constant 1 ? srand(1) By the way in digikam code there are macro to use different version of randr depending if we are on windows or not, we could use the rand fonctio provided by qt. Another point related to non destructive editing: I think a md5sum of the intermediate pictures should be stored in the database, when a picture is recomputed this would allows Digikam to detect if some algorithm has changed and warn the user that the result may be different from what he saw in the first place. Julien > More ideas are welcome ;-) > _______________________________________________ > Digikam-devel mailing list > [hidden email] > https://mail.kde.org/mailman/listinfo/digikam-devel > _______________________________________________ Digikam-devel mailing list [hidden email] https://mail.kde.org/mailman/listinfo/digikam-devel |
Re: Adjusting filters to be deterministic
|
|
> > 1) Which filters are affected? A search for rand reveals: > > - BlurFX > > - DistortionFX > > - Filmgrain > > Filmgrain is already deterministic I think. Yes, it is using a constant seed. Nonetheless, the algorithm is not defined. > For Digikam we don't need 1 in fact on the contrary we want a > deterministic sequence of random numbers. > > For this why not just use a constant as seed initialized at the start of > the filter, for instance the constant 1 ? > > srand(1) > > By the way in digikam code there are macro to use different version of > randr depending if we are on windows or not, we could use the rand > fonctio provided by qt. In the docs of Qt, there is no documentation that the function is fixed and not platform-dependent. > > Another point related to non destructive editing: I think a md5sum of > the intermediate pictures should be stored in the database, when a > picture is recomputed this would allows Digikam to detect if some > algorithm has changed and warn the user that the result may be different > from what he saw in the first place. The hash method works very well for example for text files, if you look at git. But it breaks for us: We modify image data, and we modify metadata. So we'd need a hash only over the image data. But this is not easily available at the moment. _______________________________________________ Digikam-devel mailing list [hidden email] https://mail.kde.org/mailman/listinfo/digikam-devel |
Re: Adjusting filters to be deterministic
|
|
In reply to this post by Gilles Caulier-4
> > I am not sure which other plugins fall into this group; localcontrast? > > Gilles knows more about that. > > I'm not sure to understand. Why localcontrast fall into this group. it > just a color correction tool as BCG, or HSL... Ok, then it's also in the first group. I thought localcontrast was also based on some black magic... _______________________________________________ Digikam-devel mailing list [hidden email] https://mail.kde.org/mailman/listinfo/digikam-devel |
«
Return to digikam-devel
|
1 view|%1 views
| Free forum by Nabble | Edit this page |