Batch converter: can it use more than 1 cpu?

[sean darcy]

On svn 1.1, I'm using the Raw Image batch converter on 35 images, on a
quad-core. top shows only 1 cpu being used.

Is there a setting someplace that would enable threading? Or, if not
technically threading, where the converter converts as many images at
once as no_cpus?

sean

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Gilles Caulier-4]

Batch Queue Manager already use multithreading, but not on multi core
computer...

But...

about RAW conversion, libraw included in libkdcraw support
parallelized demosaicing. You need to install OpenMp library and
recompile libkdcraw.

Check in digiKam Help/Components Info to see if parallelized
demosaicing is supported.

Gilles Caulier

2009/12/27 sean darcy <[hidden email]>:

> On svn 1.1, I'm using the Raw Image batch converter on 35 images, on a
> quad-core. top shows only 1 cpu being used.
>
> Is there a setting someplace that would enable threading? Or, if not
> technically threading, where the converter converts as many images at
> once as no_cpus?
>
> sean
>
> _______________________________________________
> Digikam-users mailing list
> [hidden email]
> https://mail.kde.org/mailman/listinfo/digikam-users
>

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[sean darcy]

Gilles Caulier wrote:

> Batch Queue Manager already use multithreading, but not on multi core
> computer...
>
> But...
>
> about RAW conversion, libraw included in libkdcraw support
> parallelized demosaicing. You need to install OpenMp library and
> recompile libkdcraw.
>
> Check in digiKam Help/Components Info to see if parallelized
> demosaicing is supported.
>
> Gilles Caulier
>
> 2009/12/27 sean darcy <[hidden email]>:
>> On svn 1.1, I'm using the Raw Image batch converter on 35 images, on a
>> quad-core. top shows only 1 cpu being used.
>>
>> Is there a setting someplace that would enable threading? Or, if not
>> technically threading, where the converter converts as many images at
>> once as no_cpus?
>>
>> sean
>>

Well, I do have libgomp - OpenMP installed, and kdegraphics finds it:

CMake version (cleaned): cmake version 2.6.4
-- Try OpenMP C flag = [-fopenmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Success
-- Try OpenMP CXX flag = [-fopenmp]
-- Performing Test OpenMP_FLAG_DETECTED
-- Performing Test OpenMP_FLAG_DETECTED - Success
-- Found OpenMP: -fopenmp
-- OpenMP detected: LibRaw will be compiled with parallelized code.

But it looks like libRaw is NOT built with -fgomp:

[  7%] Building CXX object
libs/libkdcraw/test/CMakeFiles/mem_image.dir/__/libraw/src/libraw_cxx.o
cd
/home/photo/rpmbuild/BUILD/kdegraphics-4.3.3/x86_64-redhat-linux-gnu/libs/libkdcraw/test
&& /usr/lib64/ccache/c++   -D_BSD_SOURCE -D_XOPEN_SOURCE=500
-D_BSD_SOURCE -DQT_NO_STL -DQT_NO_CAST_TO_ASCII -D_REENTRANT
-DKDE_DEPRECATED_WARNINGS -DUSE_LCMS -DDCRAW_VERBOSE -O3 -march=native
-mtune=native -msse4.1 -ftree-vectorize -floop-interchange
-floop-strip-mine -floop-block -mfpmath=sse  -Wnon-virtual-dtor
-Wno-long-long -ansi -Wundef -Wcast-align -Wchar-subscripts -Wall -W
-Wpointer-arith -Wformat-security -fno-exceptions -fno-check-new
-fno-common -Woverloaded-virtual -fno-threadsafe-statics
-fvisibility=hidden -fvisibility-inlines-hidden -O2 -DNDEBUG
-DQT_NO_DEBUG
-I/home/photo/rpmbuild/BUILD/kdegraphics-4.3.3/x86_64-redhat-linux-gnu/libs/libkdcraw/test
-I/home/photo/rpmbuild/BUILD/kdegraphics-4.3.3/libs/libkdcraw/test
-I/home/photo/rpmbuild/BUILD/kdegraphics-4.3.3
-I/home/photo/rpmbuild/BUILD/kdegraphics-4.3.3/x86_64-redhat-linux-gnu
-I/home/photo/rpmbuild/BUILD/kdegraphics-4.3.3/libs/libkdcraw/libraw
-I/home/photo/rpmbuild/BUILD/kdegraphics-4.3.3/libs/libkdcraw/test/../libkdcraw
-I/usr/include/kde4 -I/usr/include/kde4/KDE -I/usr/include/KDE
-I/usr/include/QtXmlPatterns -I/usr/include/QtWebKit
-I/usr/include/QtHelp -I/usr/include/QtAssistant -I/usr/include/QtDBus
-I/usr/include/QtTest -I/usr/include/QtUiTools -I/usr/include/QtScript
-I/usr/include/QtSvg -I/usr/include/QtXml -I/usr/include/QtSql
-I/usr/include/QtOpenGL -I/usr/include/QtNetwork
-I/usr/include/QtDesigner -I/usr/include/Qt3Support -I/usr/include/QtGui
-I/usr/include/QtCore -I/usr/include/Qt -I/usr/lib64/qt4/mkspecs/default
   -D_GNU_SOURCE -D_LARGEFILE64_SOURCE -fexceptions -DLIBRAW_BUILDLIB -w
-o CMakeFiles/mem_image.dir/__/libraw/src/libraw_cxx.o -c
/home/photo/rpmbuild/BUILD/kdegraphics-4.3.3/libs/libkdcraw/libraw/src/libraw_cxx.cpp

even though:

x86_64-redhat-linux-gnu/CMakeFiles/CMakeOutput.log:/usr/lib64/ccache/c++
    -O3 -march=native -mtune=native -msse4.1 -ftree-vectorize
-floop-interchange -floop-strip-mine -floop-block -mfpmath=sse
-Wnon-virtual-dtor -Wno-long-long -ansi -Wundef -Wcast-align
-Wchar-subscripts -Wall -W -Wpointer-arith -Wformat-security
-fno-exceptions -fno-check-new -fno-common -Woverloaded-virtual
-fno-threadsafe-statics -fvisibility=hidden -fvisibility-inlines-hidden
  -DOpenMP_FLAG_DETECTED -fopenmp  -fPIC
CMakeFiles/cmTryCompileExec.dir/src.cxx.o  -o cmTryCompileExec -rdynamic

sean

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[Gilles Caulier-4]

Yes i know this problem, and it's already fixed in libkdcraw from trunk.

Gilles Caulier

2009/12/27 sean darcy <[hidden email]>:

> Gilles Caulier wrote:
>> Batch Queue Manager already use multithreading, but not on multi core
>> computer...
>>
>> But...
>>
>> about RAW conversion, libraw included in libkdcraw support
>> parallelized demosaicing. You need to install OpenMp library and
>> recompile libkdcraw.
>>
>> Check in digiKam Help/Components Info to see if parallelized
>> demosaicing is supported.
>>
>> Gilles Caulier
>>
>> 2009/12/27 sean darcy <[hidden email]>:
>>> On svn 1.1, I'm using the Raw Image batch converter on 35 images, on a
>>> quad-core. top shows only 1 cpu being used.
>>>
>>> Is there a setting someplace that would enable threading? Or, if not
>>> technically threading, where the converter converts as many images at
>>> once as no_cpus?
>>>
>>> sean
>>>
>
> Well, I do have libgomp - OpenMP installed, and kdegraphics finds it:
>
> CMake version (cleaned): cmake version 2.6.4
> -- Try OpenMP C flag = [-fopenmp]
> -- Performing Test OpenMP_FLAG_DETECTED
> -- Performing Test OpenMP_FLAG_DETECTED - Success
> -- Try OpenMP CXX flag = [-fopenmp]
> -- Performing Test OpenMP_FLAG_DETECTED
> -- Performing Test OpenMP_FLAG_DETECTED - Success
> -- Found OpenMP: -fopenmp
> -- OpenMP detected: LibRaw will be compiled with parallelized code.
>
> But it looks like libRaw is NOT built with -fgomp:
>
> [  7%] Building CXX object
> libs/libkdcraw/test/CMakeFiles/mem_image.dir/__/libraw/src/libraw_cxx.o
> cd
> /home/photo/rpmbuild/BUILD/kdegraphics-4.3.3/x86_64-redhat-linux-gnu/libs/libkdcraw/test
> && /usr/lib64/ccache/c++   -D_BSD_SOURCE -D_XOPEN_SOURCE=500
> -D_BSD_SOURCE -DQT_NO_STL -DQT_NO_CAST_TO_ASCII -D_REENTRANT
> -DKDE_DEPRECATED_WARNINGS -DUSE_LCMS -DDCRAW_VERBOSE -O3 -march=native
> -mtune=native -msse4.1 -ftree-vectorize -floop-interchange
> -floop-strip-mine -floop-block -mfpmath=sse  -Wnon-virtual-dtor
> -Wno-long-long -ansi -Wundef -Wcast-align -Wchar-subscripts -Wall -W
> -Wpointer-arith -Wformat-security -fno-exceptions -fno-check-new
> -fno-common -Woverloaded-virtual -fno-threadsafe-statics
> -fvisibility=hidden -fvisibility-inlines-hidden -O2 -DNDEBUG
> -DQT_NO_DEBUG
> -I/home/photo/rpmbuild/BUILD/kdegraphics-4.3.3/x86_64-redhat-linux-gnu/libs/libkdcraw/test
> -I/home/photo/rpmbuild/BUILD/kdegraphics-4.3.3/libs/libkdcraw/test
> -I/home/photo/rpmbuild/BUILD/kdegraphics-4.3.3
> -I/home/photo/rpmbuild/BUILD/kdegraphics-4.3.3/x86_64-redhat-linux-gnu
> -I/home/photo/rpmbuild/BUILD/kdegraphics-4.3.3/libs/libkdcraw/libraw
> -I/home/photo/rpmbuild/BUILD/kdegraphics-4.3.3/libs/libkdcraw/test/../libkdcraw
> -I/usr/include/kde4 -I/usr/include/kde4/KDE -I/usr/include/KDE
> -I/usr/include/QtXmlPatterns -I/usr/include/QtWebKit
> -I/usr/include/QtHelp -I/usr/include/QtAssistant -I/usr/include/QtDBus
> -I/usr/include/QtTest -I/usr/include/QtUiTools -I/usr/include/QtScript
> -I/usr/include/QtSvg -I/usr/include/QtXml -I/usr/include/QtSql
> -I/usr/include/QtOpenGL -I/usr/include/QtNetwork
> -I/usr/include/QtDesigner -I/usr/include/Qt3Support -I/usr/include/QtGui
> -I/usr/include/QtCore -I/usr/include/Qt -I/usr/lib64/qt4/mkspecs/default
>   -D_GNU_SOURCE -D_LARGEFILE64_SOURCE -fexceptions -DLIBRAW_BUILDLIB -w
> -o CMakeFiles/mem_image.dir/__/libraw/src/libraw_cxx.o -c
> /home/photo/rpmbuild/BUILD/kdegraphics-4.3.3/libs/libkdcraw/libraw/src/libraw_cxx.cpp
>
> even though:
>
> x86_64-redhat-linux-gnu/CMakeFiles/CMakeOutput.log:/usr/lib64/ccache/c++
>    -O3 -march=native -mtune=native -msse4.1 -ftree-vectorize
> -floop-interchange -floop-strip-mine -floop-block -mfpmath=sse
> -Wnon-virtual-dtor -Wno-long-long -ansi -Wundef -Wcast-align
> -Wchar-subscripts -Wall -W -Wpointer-arith -Wformat-security
> -fno-exceptions -fno-check-new -fno-common -Woverloaded-virtual
> -fno-threadsafe-statics -fvisibility=hidden -fvisibility-inlines-hidden
>  -DOpenMP_FLAG_DETECTED -fopenmp  -fPIC
> CMakeFiles/cmTryCompileExec.dir/src.cxx.o  -o cmTryCompileExec -rdynamic
>
> sean
>
> _______________________________________________
> Digikam-users mailing list
> [hidden email]
> https://mail.kde.org/mailman/listinfo/digikam-users
>

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[sean darcy]

Great. I'll try that.

Thanks for the help.

sean

Gilles Caulier wrote:

> Yes i know this problem, and it's already fixed in libkdcraw from trunk.
>
> Gilles Caulier
>
> 2009/12/27 sean darcy <[hidden email]>:
>> Gilles Caulier wrote:
>>> Batch Queue Manager already use multithreading, but not on multi core
>>> computer...
>>>
>>> But...
>>>
>>> about RAW conversion, libraw included in libkdcraw support
>>> parallelized demosaicing. You need to install OpenMp library and
>>> recompile libkdcraw.
>>>
>>> Check in digiKam Help/Components Info to see if parallelized
>>> demosaicing is supported.
>>>
>>> Gilles Caulier
>>>
>>> 2009/12/27 sean darcy <[hidden email]>:
>>>> On svn 1.1, I'm using the Raw Image batch converter on 35 images, on a
>>>> quad-core. top shows only 1 cpu being used.
>>>>
>>>> Is there a setting someplace that would enable threading? Or, if not
>>>> technically threading, where the converter converts as many images at
>>>> once as no_cpus?
>>>>
>>>> sean
>>>>
>> Well, I do have libgomp - OpenMP installed, and kdegraphics finds it:
>>
>> CMake version (cleaned): cmake version 2.6.4
>> -- Try OpenMP C flag = [-fopenmp]
>> -- Performing Test OpenMP_FLAG_DETECTED
>> -- Performing Test OpenMP_FLAG_DETECTED - Success
>> -- Try OpenMP CXX flag = [-fopenmp]
>> -- Performing Test OpenMP_FLAG_DETECTED
>> -- Performing Test OpenMP_FLAG_DETECTED - Success
>> -- Found OpenMP: -fopenmp
>> -- OpenMP detected: LibRaw will be compiled with parallelized code.
>>
>> But it looks like libRaw is NOT built with -fgomp:
>>
>> [  7%] Building CXX object
>> libs/libkdcraw/test/CMakeFiles/mem_image.dir/__/libraw/src/libraw_cxx.o
>> cd
>> /home/photo/rpmbuild/BUILD/kdegraphics-4.3.3/x86_64-redhat-linux-gnu/libs/libkdcraw/test
>> && /usr/lib64/ccache/c++   -D_BSD_SOURCE -D_XOPEN_SOURCE=500
>> -D_BSD_SOURCE -DQT_NO_STL -DQT_NO_CAST_TO_ASCII -D_REENTRANT
>> -DKDE_DEPRECATED_WARNINGS -DUSE_LCMS -DDCRAW_VERBOSE -O3 -march=native
>> -mtune=native -msse4.1 -ftree-vectorize -floop-interchange
>> -floop-strip-mine -floop-block -mfpmath=sse  -Wnon-virtual-dtor
>> -Wno-long-long -ansi -Wundef -Wcast-align -Wchar-subscripts -Wall -W
>> -Wpointer-arith -Wformat-security -fno-exceptions -fno-check-new
>> -fno-common -Woverloaded-virtual -fno-threadsafe-statics
>> -fvisibility=hidden -fvisibility-inlines-hidden -O2 -DNDEBUG
>> -DQT_NO_DEBUG
>> -I/home/photo/rpmbuild/BUILD/kdegraphics-4.3.3/x86_64-redhat-linux-gnu/libs/libkdcraw/test
>> -I/home/photo/rpmbuild/BUILD/kdegraphics-4.3.3/libs/libkdcraw/test
>> -I/home/photo/rpmbuild/BUILD/kdegraphics-4.3.3
>> -I/home/photo/rpmbuild/BUILD/kdegraphics-4.3.3/x86_64-redhat-linux-gnu
>> -I/home/photo/rpmbuild/BUILD/kdegraphics-4.3.3/libs/libkdcraw/libraw
>> -I/home/photo/rpmbuild/BUILD/kdegraphics-4.3.3/libs/libkdcraw/test/../libkdcraw
>> -I/usr/include/kde4 -I/usr/include/kde4/KDE -I/usr/include/KDE
>> -I/usr/include/QtXmlPatterns -I/usr/include/QtWebKit
>> -I/usr/include/QtHelp -I/usr/include/QtAssistant -I/usr/include/QtDBus
>> -I/usr/include/QtTest -I/usr/include/QtUiTools -I/usr/include/QtScript
>> -I/usr/include/QtSvg -I/usr/include/QtXml -I/usr/include/QtSql
>> -I/usr/include/QtOpenGL -I/usr/include/QtNetwork
>> -I/usr/include/QtDesigner -I/usr/include/Qt3Support -I/usr/include/QtGui
>> -I/usr/include/QtCore -I/usr/include/Qt -I/usr/lib64/qt4/mkspecs/default
>>   -D_GNU_SOURCE -D_LARGEFILE64_SOURCE -fexceptions -DLIBRAW_BUILDLIB -w
>> -o CMakeFiles/mem_image.dir/__/libraw/src/libraw_cxx.o -c
>> /home/photo/rpmbuild/BUILD/kdegraphics-4.3.3/libs/libkdcraw/libraw/src/libraw_cxx.cpp
>>
>> even though:
>>
>> x86_64-redhat-linux-gnu/CMakeFiles/CMakeOutput.log:/usr/lib64/ccache/c++
>>    -O3 -march=native -mtune=native -msse4.1 -ftree-vectorize
>> -floop-interchange -floop-strip-mine -floop-block -mfpmath=sse
>> -Wnon-virtual-dtor -Wno-long-long -ansi -Wundef -Wcast-align
>> -Wchar-subscripts -Wall -W -Wpointer-arith -Wformat-security
>> -fno-exceptions -fno-check-new -fno-common -Woverloaded-virtual
>> -fno-threadsafe-statics -fvisibility=hidden -fvisibility-inlines-hidden
>>  -DOpenMP_FLAG_DETECTED -fopenmp  -fPIC
>> CMakeFiles/cmTryCompileExec.dir/src.cxx.o  -o cmTryCompileExec -rdynamic
>>
>> sean
>>
>> _______________________________________________
>> Digikam-users mailing list
>> [hidden email]
>> https://mail.kde.org/mailman/listinfo/digikam-users
>>

_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users

[S. Burmeister]

Am Sonntag, 27. Dezember 2009 17:13:48 schrieb sean darcy:
> On svn 1.1, I'm using the Raw Image batch converter on 35 images, on a
> quad-core. top shows only 1 cpu being used.
>
> Is there a setting someplace that would enable threading? Or, if not
> technically threading, where the converter converts as many images at
> once as no_cpus?

https://bugs.kde.org/show_bug.cgi?id=195538 is about that.

Sven
_______________________________________________
Digikam-users mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-users