[digikam] [Bug 375560] New: interruptable BQM

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=375560

            Bug ID: 375560
           Summary: interruptable BQM
           Product: digikam
           Version: 5.4.0
          Platform: openSUSE RPMs
                OS: Linux
            Status: UNCONFIRMED
          Severity: normal
          Priority: NOR
         Component: general
          Assignee: [hidden email]
          Reporter: [hidden email]
  Target Milestone: ---

BQM is extremely slow; in my case, I have:
- Nikon 36,3 Mio pixel-pics.

I use

- metadata correction

PLUS

- Lens-auto- correction (which is available for my lenses!!)

PLUS  

- rezize                                                    

PLUS

- sharpen image (refocus)                                    

PLUS

- Add watermark (=Text)

??? With 4 quadcore AMD running @ 3300 MHz, some 40 pages need more than ten
hours processing times....

---


What is about the newly created batchfiles???
Can I delete these?
Or do they carry relevant information?

--
You are receiving this mail because:
You are the assignee for the bug.

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=375560

[hidden email] changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]
          Component|general                     |BatchQueueManager

--- Comment #1 from [hidden email] ---
Wow. Ten hours to process 40 36Mpx images... Incredible.

Do you process RAW files ?

How many RAM do you have ?

Do you process items in parallel with BQM ?

In all case, i have similar image size here (panorama), and more, and this do
not take this time to process. My computer : i7-3Ghz, 32Gb RAM.

If you share some items through the cloud, i can try to reproduce your workflow
to compare.

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=375560

--- Comment #2 from [hidden email] ---
Gilles:

I still run digiKam 5.4, you showed me download of:
digiKam-5.4.0-01-x86-64.appimage.

- Maybe resulting from this circumstances, I can _NOT_ start bug reports
from within the digiKam program automatically. So additional information
might by not processed.

Panorama still does not work but hangs with message
- [digikam] [Bug 374812] Panorama stops
- see attachment.

----------------
"Workslow" ( :-) :

- with digiKam, I only process jpg's; typical sites some 15 MB each in
average.

- my PC has 16 GB RAM.

- processing BQM in parallel with image editor- only a few times. Is
unpractical slow!

- looks like digiKam occupies alles resources whil only using single
CPU's. Don't know, how to eluciate this in detail, unfortunately.

- how can I send you some pics? My eMail ist too slow by far.

Am 25.01.2017 um 22:28 schrieb [hidden email]:

> https://bugs.kde.org/show_bug.cgi?id=375560
>
> [hidden email] changed:
>
>            What    |Removed                     |Added
> ----------------------------------------------------------------------------
>                  CC|                            |[hidden email]
>           Component|general                     |BatchQueueManager
>
> --- Comment #1 from [hidden email] ---
> Wow. Ten hours to process 40 36Mpx images... Incredible.
>
> Do you process RAW files ?
>
> How many RAM do you have ?
>
> Do you process items in parallel with BQM ?
>
> In all case, i have similar image size here (panorama), and more, and this do
> not take this time to process. My computer : i7-3Ghz, 32Gb RAM.
>
> If you share some items through the cloud, i can try to reproduce your workflow
> to compare.
>
> Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=375560

--- Comment #3 from [hidden email] ---
You can use GDrive, or another cloud storage service to share JPEG image to
test here.

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=375560

Maik Qualmann <[hidden email]> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |[hidden email]

--- Comment #4 from Maik Qualmann <[hidden email]> ---
With a Nikon D800 RAW (first tool converted to JPEG) test image I come with 4
pictures to about 4 minutes (2 cores used). With 40 pictures then probably
about 40 minutes. Lens correction takes the longest time. But I do not
understand 10 hours.

Maik

--
You are receiving this mail because:
You are the assignee for the bug.

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=375560

--- Comment #5 from [hidden email] ---
Same here.

I tested with My RAW file 24Mpx from Sony A77. My BQM workflow :

RAW demosaicing
Lens Correction.
Resize.
Sharpen.
Watermark.
export JPEG
remove metadata

I use a VM, with 4 cores and 8Gb and Mageia 6. This take 1m03 for one image. No
more, as expected... Can you imagine that we can hack, debug, test and fianlly
use digiKam BQM with this kind of performance without to be hurt... It's
impossible...

Notes:

- i use digiKam with debug symbols and no optimizations.

- in your workflow, remove metadata must be at end, else lens correction will
be lost to process

Conclusion : i suspect a dysfunction on your computer, or a wrong system
adjustment which slow down drastically all computation performances.

My tips : try with another computer to reproduce or not this problem.

--
You are receiving this mail because:
You are the assignee for the bug.

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=375560

--- Comment #6 from [hidden email] ---
Gilles:

I did not find a trustable service yet.
Private property isn't respected, unfortunately.
Keep on searching...


Axel


Am 26.01.2017 um 10:02 schrieb [hidden email]:
> https://bugs.kde.org/show_bug.cgi?id=375560
>
> --- Comment #3 from [hidden email] ---
> You can use GDrive, or another cloud storage service to share JPEG image to
> test here.
>
> Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=375560

--- Comment #7 from [hidden email] ---
Another question is : what do you see as backtrace when you run digiKam from a
console, while you process BQM ?

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=375560

--- Comment #8 from [hidden email] ---
Sorry, won't work.

As digiKam 5.4 is not installed regularly, I can not start it ina
terminal. It will not even found: "cnf digiKam-5.4.0-01-x86-64.appimage"
leads to nothing!

Q: is digiKam 5.4 available in the meanwhile? Maybe some of _my_
observations base on the "appimage-issue" (without beeing instlled in
the system)


Axel

Am 27.01.2017 um 06:04 schrieb [hidden email]:
> https://bugs.kde.org/show_bug.cgi?id=375560
>
> --- Comment #7 from [hidden email] ---
> Another question is : what do you see as backtrace when you run digiKam from a
> console, while you process BQM ?
>
> Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=375560

--- Comment #9 from [hidden email] ---
Of course, you can run it in terminal... even if you use AppImage bundle...

open terminal,
go to directory where AppImage bundle file is installed
start AppImage directly from command line. Exemple from my computer :

[gilles@localhost digikam5]$ pwd
/home/gilles/digikam5
[gilles@localhost digikam5]$ ls
digikam4.db  digikam-5.5.0-01-x86-64.appimage*  recognition.db
thumbnails-digikam.db
[gilles@localhost digikam5]$ ./digikam-5.5.0-01-x86-64.appimage
VideoDecoder::Register(..., CUDA)
Singleton 0x20462e0 created...
VideoRenderer::Register(..., OpenGLWindow)
Singleton 0x20468f0 created...
SubtitleProcessor::Register(..., FFmpeg)
Singleton 0x2041990 created...
AudioDecoder::Register(..., FFmpeg)
Singleton 0x2046f00 created...
AudioEncoder::Register(..., FFmpeg)
Singleton 0x2046ff0 created...
ImageConverter::Register(..., FFmpeg)
Singleton 0x2047540 created...
MediaIO::Register(..., QIODevice)
Singleton 0x2046b60 created...
MediaIO::Register(..., QFile)
AudioOutputBackend::Register(..., null)
Singleton 0x2046cf0 created...
VideoDecoder::Register(..., FFmpeg)
VideoEncoder::Register(..., FFmpeg)
Singleton 0x2047fa0 created...
registerRenderers...........
VideoRenderer::Register(..., OpenGLWidget)
VideoRenderer::Register(..., QGLWidget2)
VideoRenderer::Register(..., Widget)
VideoRenderer::Register(..., GraphicsItem)
digikam.widgets: Breeze icons ressource file found
digikam.general: AlbumWatch use QFileSystemWatcher
digikam.general: Database Parameters:
   Type:                     "QSQLITE"
   DB Core Name:             "/home/gilles/digikam5/digikam4.db"
   DB Thumbs Name:           "/home/gilles/digikam5/thumbnails-digikam.db"
   DB Face Name:             "/home/gilles/digikam5/recognition.db"
   Connect Options:          ""
   Host Name:                ""
   Host port:                -1
   Internal Server:          false
   Internal Server Path:     ""
   Internal Server Serv Cmd: ""
   Internal Server Init Cmd: ""
   Username:                 ""
   Password:                 ""

digikam.dbengine: Loading SQL code from config file
"/tmp/.mount_oL0Gjs/usr/share/digikam/database/dbconfig.xml"
digikam.dbengine: Checking XML version ID => expected:  3  found:  3
digikam.coredb: Core database: running schema update
digikam.coredb: Core database: have a structure version  8
digikam.coredb: Core database: makeUpdates  8  to  8
digikam.database: Creating new Location  "/photos/GILLES"  uuid
"volumeid:?uuid=9506d649-8dfd-4cf0-8098-0cff747c2cbf"
digikam.database: location for  "/mnt/data2/photos/GILLES"  is available  true
KMemoryInfo: Platform identified :  "LINUX"
KMemoryInfo: TotalRam:  33653878784
digikam.general: Allowing a cache size of 200 MB
digikam.thumbsdb: ThumbDB SelectThumbnailSetting val ret =  0
digikam.thumbsdb: ThumbDB SelectThumbnailSetting val ret =  0
digikam.thumbsdb: Thumbs database: have a structure version  "3"
digikam.general: Thumbnails database ready for use
digikam.dimg: ("/usr/share/color/icc", "/home/gilles/.local/share/icc")
digikam.dimg: No X.org XICC profile installed for screen  0
digikam.general: Camera XML data:
"/home/gilles/.local/share/digikam/cameras.xml"
digikam.facedb: FaceDB SelectFaceSetting val ret =  0
digikam.facedb: FaceDB SelectFaceSetting val ret =  0
digikam.facedb: Face database: have a structure version  "2"
digikam.facesengine: Face database ready for use
digikam.general: Face PipeLine: add database writer
digikam.general: Face PipeLine: add faces trainer
digikam.facesengine: Face database ready for use
digikam.general: Face PipeLine: add database writer
digikam.general: Face PipeLine: add faces trainer
...

Et voilà. Use digiKam as expected, and when dysfunction appear, catch the
terminal trace printed...

Nothing difficult to do...

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=375560

--- Comment #10 from [hidden email] ---
did so:

terminal as user and as superuser start
"digikam-5.4.0-01-x86-64.appimage" directly.

Will not be found- I get the known "cnf
digikam-5.4.0-01-x86-64.appimage" message indicating that
"digikam-5.4.0-01-x86-64.appimage" is _not_ available in the system.

Double-clicking works with normality.
(Rem.: This is my standard sourrounding since you offered me the
download of V 5.4)

BUT:

You write
"
...
 > open terminal,
 > go to directory where AppImage bundle file is installed
...
"

What do you mean with "installed"???

The program just lays in the path like any other file.
Activated "execute", so the rights are

Type:           Name:   r       w       x       Effective:
Qwner                   x       x       x       rwx
Qwning Group            x               x       r-x
Others                  x               x       r-x

This procedure seems to be correct, as some URLs indicate
<https://askubuntu.com/questions/774490/what-is-an-appimage-how-do-i-install-it>
.

Others tell me there is no installation necessary:
<https://musescore.org/en/handbook/install-linux>

Hmmm... ideas?




Am 28.01.2017 um 09:53 schrieb [hidden email]:

> https://bugs.kde.org/show_bug.cgi?id=375560
>
> --- Comment #9 from [hidden email] ---
> Of course, you can run it in terminal... even if you use AppImage bundle...
>
> open terminal,
> go to directory where AppImage bundle file is installed
> start AppImage directly from command line. Exemple from my computer :
>
> [gilles@localhost digikam5]$ pwd
> /home/gilles/digikam5
> [gilles@localhost digikam5]$ ls
> digikam4.db  digikam-5.5.0-01-x86-64.appimage*  recognition.db
> thumbnails-digikam.db
> [gilles@localhost digikam5]$ ./digikam-5.5.0-01-x86-64.appimage
> VideoDecoder::Register(..., CUDA)
> Singleton 0x20462e0 created...
> VideoRenderer::Register(..., OpenGLWindow)
> Singleton 0x20468f0 created...
> SubtitleProcessor::Register(..., FFmpeg)
> Singleton 0x2041990 created...
> AudioDecoder::Register(..., FFmpeg)
> Singleton 0x2046f00 created...
> AudioEncoder::Register(..., FFmpeg)
> Singleton 0x2046ff0 created...
> ImageConverter::Register(..., FFmpeg)
> Singleton 0x2047540 created...
> MediaIO::Register(..., QIODevice)
> Singleton 0x2046b60 created...
> MediaIO::Register(..., QFile)
> AudioOutputBackend::Register(..., null)
> Singleton 0x2046cf0 created...
> VideoDecoder::Register(..., FFmpeg)
> VideoEncoder::Register(..., FFmpeg)
> Singleton 0x2047fa0 created...
> registerRenderers...........
> VideoRenderer::Register(..., OpenGLWidget)
> VideoRenderer::Register(..., QGLWidget2)
> VideoRenderer::Register(..., Widget)
> VideoRenderer::Register(..., GraphicsItem)
> digikam.widgets: Breeze icons ressource file found
> digikam.general: AlbumWatch use QFileSystemWatcher
> digikam.general: Database Parameters:
>    Type:                     "QSQLITE"
>    DB Core Name:             "/home/gilles/digikam5/digikam4.db"
>    DB Thumbs Name:           "/home/gilles/digikam5/thumbnails-digikam.db"
>    DB Face Name:             "/home/gilles/digikam5/recognition.db"
>    Connect Options:          ""
>    Host Name:                ""
>    Host port:                -1
>    Internal Server:          false
>    Internal Server Path:     ""
>    Internal Server Serv Cmd: ""
>    Internal Server Init Cmd: ""
>    Username:                 ""
>    Password:                 ""
>
> digikam.dbengine: Loading SQL code from config file
> "/tmp/.mount_oL0Gjs/usr/share/digikam/database/dbconfig.xml"
> digikam.dbengine: Checking XML version ID => expected:  3  found:  3
> digikam.coredb: Core database: running schema update
> digikam.coredb: Core database: have a structure version  8
> digikam.coredb: Core database: makeUpdates  8  to  8
> digikam.database: Creating new Location  "/photos/GILLES"  uuid
> "volumeid:?uuid=9506d649-8dfd-4cf0-8098-0cff747c2cbf"
> digikam.database: location for  "/mnt/data2/photos/GILLES"  is available  true
> KMemoryInfo: Platform identified :  "LINUX"
> KMemoryInfo: TotalRam:  33653878784
> digikam.general: Allowing a cache size of 200 MB
> digikam.thumbsdb: ThumbDB SelectThumbnailSetting val ret =  0
> digikam.thumbsdb: ThumbDB SelectThumbnailSetting val ret =  0
> digikam.thumbsdb: Thumbs database: have a structure version  "3"
> digikam.general: Thumbnails database ready for use
> digikam.dimg: ("/usr/share/color/icc", "/home/gilles/.local/share/icc")
> digikam.dimg: No X.org XICC profile installed for screen  0
> digikam.general: Camera XML data:
> "/home/gilles/.local/share/digikam/cameras.xml"
> digikam.facedb: FaceDB SelectFaceSetting val ret =  0
> digikam.facedb: FaceDB SelectFaceSetting val ret =  0
> digikam.facedb: Face database: have a structure version  "2"
> digikam.facesengine: Face database ready for use
> digikam.general: Face PipeLine: add database writer
> digikam.general: Face PipeLine: add faces trainer
> digikam.facesengine: Face database ready for use
> digikam.general: Face PipeLine: add database writer
> digikam.general: Face PipeLine: add faces trainer
> ...
>
> Et voilà. Use digiKam as expected, and when dysfunction appear, catch the
> terminal trace printed...
>
> Nothing difficult to do...
>
> Gilles Caulier
>

--
You are receiving this mail because:
You are the assignee for the bug.

[bugzilla_noreply]

https://bugs.kde.org/show_bug.cgi?id=375560

--- Comment #11 from [hidden email] ---
AppImage bundle is a file do download from digiKam project page. Current one
for 5.5.0 pre release can found in this repository :

https://drive.google.com/drive/folders/0BzeiVr-byqt5Y0tIRWVWelRJenM

it will be stored in your computer after download.

it's an executable file embedding all that digiKam need. At run time it do not
install anything.

You run it into the console or not.

That all. What's complicated here ???

Gilles Caulier

--
You are receiving this mail because:
You are the assignee for the bug.