[Digikam-devel] [Bug 133026] New: crashes on systems using hyperthreading

[Gilles Caulier-2]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=133026         
caulier.gilles kdemail net changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|                            |FIXED



------- Additional Comments From caulier.gilles kdemail net  2006-10-30 11:23 -------
Marcel,

I have executed your KDebug test program under my 3 HT computers... and it crash everywhere in libc after to post some messages in the console.

Conclusion : we _NEVER_ use KDebug in QThread !!!

I think this is an hight limitation of KDELib ! All KDElib API witch are non relevant of GUI, especially to debug an application must be thread-safe.

I close this file about digiKam, but i recommend to report this information to KDELib developpers team.

Gilles Caulier
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-2]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=133026         




------- Additional Comments From caulier.gilles kdemail net  2006-10-30 11:28 -------
Marcel, I have another question for you :

We using a lot of KDebug statements in DImg/Dmetadata API witch are multithreaded by digiKam... What do you think about ? We need to use QDebug instead everywhere ?

Is Amarok use QDebug instead KDebug in multithreaded implementation ?

Gilles
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=133026         




------- Additional Comments From marcel.wiesweg gmx de  2006-10-30 23:22 -------
Taking a closer look at kdebug.cpp, the function kDebugBackend(): it uses a global, static variable.

Yes, I think we should not use kdDebug anywhere outside the main thread from now.
I see four possibilities, for the 0.9.0 release:
- remove all kdDebug in multithreaded code
- use qDebug
- write a thin wrapper around kdDebug using a mutex
- write a replacement for kdDebug which is reentrant. Copy over all the nice operator<< implementations.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-2]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=133026         




------- Additional Comments From caulier.gilles kdemail net  2006-10-31 08:55 -------
Marcel,

Under KDevelop, searching all KDdebug() statements in digikam/libs give around 400 hits.

Also, the camera interface is multithreaded. We need to fix this part too.

In fact to be sure, i would to fix all KDdebug() statements in digiKam _and_ DigikamImagePlugins _and_ Kipi-plugins (where there is also some multithreaded parts).

I vote to write a thin wrapper around kDdebug using a mutex. I think it's the more simple to do... To have look like QDebug() work, it's completly different and require a lot of work.

I'm not agree to remove all KDebug() statments because it's a good report to investiguate indeep problems.

I think to write a replacement for KDdebug which is reentrant must be done by kdelibs developpers, not digiKam team (:=)))...

To fix kipi-plugins, the wrapper around kDdebug() using a mutex must be backported to the kipi-plugins/common/libkipiplugins.

Gilles
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=133026         




------- Additional Comments From marcel.wiesweg gmx de  2006-11-03 22:56 -------
Created an attachment (id=18389)
 --> (http://bugs.kde.org/attachment.cgi?id=18389&action=view)
Test kdDebug wrapper

The attached file contains the same stress test, this time using a thin wrapper
around kdDebug.
The implementation should be tested on affected systems before we continue.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-2]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=133026         




------- Additional Comments From caulier.gilles kdemail net  2006-11-04 08:51 -------
Thanks Marcel. I will take a look be only Monday morning because my HT computers are in my office.

Gilles
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-2]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=133026         




------- Additional Comments From caulier.gilles kdemail net  2006-11-06 08:14 -------
Marcel,

I have tested your kdDebug() wrapper under my HT computer in my office and there is _NO_ crash (:=)))

I have stressed the program to start more than on instance at the same time and let's running all during one hours : no problem...

Note : with the previous test program, the crash appear just a few seconds after to start it.

So, your implementation can be include in digiKam core, into /libs (put it in a new sub folder ?). This implementation need to be include in the shared library libdigikam.so because i will use it in DigikamImagePlugins.

Gilles
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-2]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=133026         




------- Additional Comments From caulier.gilles kdemail net  2006-11-06 18:35 -------
Marcel,

Just a little report. I have run your test program during this day under my 3 HT computers using ssh, and it doesn't crash...

Gilles
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-2]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=133026         




------- Additional Comments From caulier.gilles kdemail net  2006-11-07 08:32 -------
Marcel,

After investiguations, the better digiKam/DigikamImagePlugins common place to put your implementation is in the "libs/dimg/" folder. I recommend you to create a new class name DDebug (like Digikam Debug (:=))). Thanks in advance...

Gilles
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=133026         




------- Additional Comments From marcel.wiesweg gmx de  2006-11-07 23:31 -------
Created an attachment (id=18463)
 --> (http://bugs.kde.org/attachment.cgi?id=18463&action=view)
DDebug implementation

This is the current patch.
The idea is that using some #define magic (which I usually don't like at all,
but I am pragmatic) and gcc's include option, the source does not need to be
touched at all.
Three Makefile.ams are changed so that the ddebug.h is included in all source
files. This would have to be done with all other Makefile.ams.

Alternatively, we can #include <ddebug.h> in every file, or even dont use the
#define and replace every occurence of kdDebug with DDebug.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-2]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=133026         




------- Additional Comments From caulier.gilles kdemail net  2006-11-08 08:00 -------
Thanks Marcel for this implementation.

> Alternatively, we can #include <ddebug.h> in every file, or even dont use the
> #define and replace every occurence of kdDebug with DDebug.

I'm favorable to use this way intead a gcc macro witch can be an opaque solution. If we changing kdDebug with DDebug in source code we will be sure than we use it properly (:=)))

I will add your source code in svn and looking changes requires in other place.

Question : If i remplace all kdDebug() occurences in digiKam/DigikamImagePlugins, we will use DDebug in main thread too. I hope than this will not  be a cause of any problems. Right ?

Gilles
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-2]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=133026         




------- Additional Comments From caulier.gilles kdemail net  2006-11-08 13:21 -------
Marcel,

I think that digiKam kio-slave implementation no need to be fixed using DDebug() because it running in separate process. Right for you ?

Gilles
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=133026         




------- Additional Comments From marcel.wiesweg gmx de  2006-11-09 16:35 -------
I already saw all your commits, so you did the tedious work ;-)
#48: Not only can we use it in the main thread, we must use it in the main thread as well! If we used only kdDebug in the main thread, there could still be a race condition. We must replace every occurrence of kdDebug.

#49: Yes, no need to use it in the ioslave.

This is comment #50 for this bug. Time to celebrate ;-)
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-2]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=133026         




------- Additional Comments From caulier.gilles kdemail net  2006-11-09 16:51 -------
>We must replace every occurrence of kdDebug.

Well it's done in digiKam and DigiKamImagePlugins (:=)

Now, the problem is with kipi-plugins witch have some QThread parts. I wiill dupplicate your DDebug class into the common plugins area, change the namespace, and the name of the static mutex.

I will use DDebug in all plugins witch use QThread, but i need to use it also in all others plugins ?

Gilles
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=133026         




------- Additional Comments From marcel.wiesweg gmx de  2006-11-09 18:31 -------
I think you do not need to use it in all (non-multithreaded) kipi-plugins.
First, I think most plugins provide a modal dialog, so the host application is blocked anyway.
If the host application is doing some multithreaded work while the plugin is running, that's a problem: using DDebug won't help, the mutex of the plugin will be different from the mutex of the host app, so you can with the same effect not use any mutex at all.
That's a limitation of wrapping a common library function with a application-private mutex, the real solution (for the future) is to make kdDebug thread-safe.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-2]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=133026         




------- Additional Comments From caulier.gilles kdemail net  2006-11-09 18:38 -------
> If the host application is doing some multithreaded work while the plugin is running, that's a problem:

And it's possible because for example RAW converter dialog is non-modal. If you start digiKam camera interface during the RAW conversion there is a risk.

A solution is to move DDBug into libkipi witch is common at all. right ?

Gilles
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Marcel Wiesweg]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=133026         




------- Additional Comments From marcel.wiesweg gmx de  2006-11-09 18:42 -------
Yes, that's a solution. Then the mutex is shared.
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-2]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=133026         
caulier.gilles kdemail net changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |anaselli linux it



------- Additional Comments From caulier.gilles kdemail net  2006-11-09 18:49 -------
Angelo,

Can you give us your viewpoint about DDebug integration in libkipi ?

Gilles
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=133026         




------- Additional Comments From anaselli linux it  2006-11-12 23:25 -------
Well, my point is that the fix should be done in kdDebug
as said in comment #52.
A temporary workaround is to add a common library somewhere with
a thread safe implementation of kdDebug. If you think
libkipi is the right place well, maybe it should be a "special"
kdDebug extension that could add some helpful features, even if i
don't know which ones :)

Angelo
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel

[Gilles Caulier-2]

------- You are receiving this mail because: -------
You are the assignee for the bug, or are watching the assignee.
         
http://bugs.kde.org/show_bug.cgi?id=133026         




------- Additional Comments From caulier.gilles kdemail net  2006-11-24 09:26 -------
Angelo,

Well after checking all kipi code, there is a few KDdebug() statements to fix in multithreaded implementation. I will use QDebug instead. like this libkipi is untouched. It's more simple until KdDebug() implementation support it.

Gilles
_______________________________________________
Digikam-devel mailing list
[hidden email]
https://mail.kde.org/mailman/listinfo/digikam-devel