[etherlab-users] R: System randomly freezes in multi-thread Qt application with a RT process

Viola Roberto roberto.viola at systemceramics.com
Mon May 27 10:57:13 CEST 2019 

Hi Simone,
  from the logs it seems a issue releated to your i915 (video card, i guess inside your CPU).
You should try to understand the cause of issue: i suggest to try without the ethercat and stressing the cpu and the video card in some other way. I guess it’s not related to ethercat.
BTW what videocard do you have?
Did you try to catch the logs on other systems (laptop for example)?
R.

Da: Simone Comari [mailto:simone.comari2 at unibo.it]
Inviato: giovedì 23 maggio 2019 12:51
A: Viola Roberto <roberto.viola at systemceramics.com>; etherlab-users at etherlab.org
Oggetto: R: System randomly freezes in multi-thread Qt application with a RT process

Hi Roberto,

First of all, thank you for your quick response.
Attached you can find the kernel.log and system.log of a single session, that is:

  1.  Boot up
  2.  Application launch (successful ethercat network setup) through Qt IDE
  3.  Successful enabling of a single motor (i.e. one of the ethercat slaves) through our GUI
  4.  Simple operation (e.g. manual velocity control) until problem occurs (it took a couple of minutes this time)
  5.  Hard shut-down of the "frozen" system
I hope these are the logs you were talking about, please let me know otherwise.

Maybe it's worth mentioning we followed these<https://github.com/UNIBO-GRABLab/cable_robot/wiki/Installation> instructions to install both the RT kernel and ethercat libs, just in case we misused patches or configurations.

Thanks again.

Best regards,
Simone

Roberto Viola
Technical Dept
+39 0536836680

[cid:image2dbc08.JPG at 01dba098.4381a376]

SYSTEM CERAMICS S.p.A.
Via Ghiarola Vecchia, 73
41042 Fiorano (Mo) ITALY
+39 0536 836111
info at system-electronics.it<mailto:info at system-electronics.it>
www.system-electronics.it<http://www.system-electronics.it>

[cid:imagefda576.JPG at 4575342a.4fae7fd3]


________________________________


Le informazioni contenute in questa email, inclusi i suoi allegati, sono riservate e ad uso esclusivo del destinatario. Qualora le fosse pervenuta per errore, lei non è autorizzato a copiare, inoltrare e/o rendere nota questa email e i suoi allegati, totalmente o parzialmente, e pertanto la preghiamo di cancellarla immediatamente senza visionarne il contenuto e gli allegati.

Avvertenza: la presente casella e-mail ed i messaggi da essa derivanti, sono di esclusivo utilizzo aziendale /lavorativo e mai personale.

Risposte al presente messaggio: si avvisa il destinatario che eventuali sue risposte, potranno essere lette dall’intera azienda /ufficio /reparto di appartenenza del mittente.

The information contained in this e-mail, including attachments, is confidential and exclusively for the use of the intended recipient. If you received this communication by mistake you are not authorized to copy, send and/or publish this message and its attachments, in whole or in part and therefore please delete this message.



____________________________________________________

SIMONE COMARI

Research Fellow
DIN – Dept. of Industrial Engineering
Alma Mater Studiorum –  University of Bologna
Via Umberto Terracini, 24, 40131 Bologna (BO), Italy

E-mail: simone.comari2 at unibo.it<mailto:simone.comari2 at unibo.it>
Websites:
https://www.unibo.it/sitoweb/simone.comari2
http://grab.diem.unibo.it<http://grab.diem.unibo.it/>

________________________________
Da: Viola Roberto <roberto.viola at systemceramics.com<mailto:roberto.viola at systemceramics.com>>
Inviato: giovedì 23 maggio 2019 08:00
A: Simone Comari; etherlab-users at etherlab.org<mailto:etherlab-users at etherlab.org>
Oggetto: R: System randomly freezes in multi-thread Qt application with a RT process


Hi Simone, just a quick hint in order to understand the freeze: try to run the setup inside a VM (kvm or virtualbox) in order to catch the serial log from the kernel or, if you have a UART avaiable on your system, directly from it.

In this way we should try to understand the issue better.

R.



Da: etherlab-users [mailto:etherlab-users-bounces at etherlab.org] Per conto di Simone Comari
Inviato: mercoledì 22 maggio 2019 18:52
A: etherlab-users at etherlab.org<mailto:etherlab-users at etherlab.org>
Oggetto: [etherlab-users] System randomly freezes in multi-thread Qt application with a RT process



Hi all,



I am a young research fellow at the university of Bologna and I just started working with EtherCAT technology and RT systems yet, so please forgive me if I misuse words or I'm not precise enough.

First, I'll try to describe my setup:

  *   Ubuntu 16.04.6 with patched fully preemptible RT kernel 4.13.13-rt5
  *   Qt 5.12.2
  *   PCI driver e1000e
  *   Ethercat master running on this Linux RT
  *   Elmo GOLD SOLO WHISTLE Drives (ethercat slaves)

Secondly, a brief outline of my software architecture:

  *   POSIX threads
  *   Qt-based GUI running on a non-RT thread
  *   Ethercat network setup (ethercat master and slaves init) done in the same non-RT thread
  *   If initialization is successful, start a new RT-thread in charge of handling all ethercat-related functionalities (read/write/status-check).
  *   Shared resources between RT and non-RT ones handled with pthread_mutex (even if I'm not 100% sure I'm using it correctly)
  *   Implementation of our generic ethercat master can be found here<https://github.com/UNIBO-GRABLab/grab_common/blob/e5278b6fe611654bfa84c951d8b77e56ebbc8fa9/libgrabec/src/ethercatmaster.cpp>

Problem description:

  *   Once the ethercat network is setup and the RT thread is started, quite randomly the system freezes without errors of any sorts. Sometimes it happens when motors are enabled and operational, sometimes when they are enabled and idle, sometimes even if they are disabled. It is not reproducible and I couldn't link it to any particular step in my application. Sometimes it happens even if I simply start it, but always after successful initialization.
  *   Even when I manage to close the application, next time I try to run it it tells me that master is busy and ec_e1000e is in use. Only solution is to manually hard-shut-down the PC.
  *   Other thing I noticed is that even if the main thread (the GUI one, so non-RT) is closed, the child RT-thread stays running with status D  (uninterruptible sleep) blocking a great deal of CPU (that is why probably the whole system freezes).
  *   We tried with different computers (both laptop and desktop) to exclude a platform's dependency, but the issue remains.

Please let me know if there is any missing important information that can help understanding the problem.

Thank you a lot for the support.



Best regards,

Simone
Roberto Viola

Technical Dept

+39 0536836680


[cid:image001.jpg at 01D5147A.F003E390]



SYSTEM CERAMICS S.p.A.

Via Ghiarola Vecchia, 73

41042 Fiorano (Mo) ITALY

+39 0536 836111

info at system-electronics.it<mailto:info at system-electronics.it>

www.system-electronics.it<http://www.system-electronics.it>


[cid:image002.jpg at 01D5147A.F003E390]


________________________________


Le informazioni contenute in questa email, inclusi i suoi allegati, sono riservate e ad uso esclusivo del destinatario. Qualora le fosse pervenuta per errore, lei non è autorizzato a copiare, inoltrare e/o rendere nota questa email e i suoi allegati, totalmente o parzialmente, e pertanto la preghiamo di cancellarla immediatamente senza visionarne il contenuto e gli allegati.

Avvertenza: la presente casella e-mail ed i messaggi da essa derivanti, sono di esclusivo utilizzo aziendale /lavorativo e mai personale.

Risposte al presente messaggio: si avvisa il destinatario che eventuali sue risposte, potranno essere lette dall’intera azienda /ufficio /reparto di appartenenza del mittente.

The information contained in this e-mail, including attachments, is confidential and exclusively for the use of the intended recipient. If you received this communication by mistake you are not authorized to copy, send and/or publish this message and its attachments, in whole or in part and therefore please delete this message.


____________________________________________________

SIMONE COMARI

Research Fellow
DIN – Dept. of Industrial Engineering
Alma Mater Studiorum –  University of Bologna
Via Umberto Terracini, 24, 40131 Bologna (BO), Italy

E-mail: simone.comari2 at unibo.it<mailto:simone.comari2 at unibo.it>
Websites:
https://www.unibo.it/sitoweb/simone.comari2
http://grab.diem.unibo.it<http://grab.diem.unibo.it/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.etherlab.org/pipermail/etherlab-users/attachments/20190527/bf5ef9ba/attachment-0003.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.jpg
Type: image/jpeg
Size: 15896 bytes
Desc: image001.jpg
URL: <http://lists.etherlab.org/pipermail/etherlab-users/attachments/20190527/bf5ef9ba/attachment-0008.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.jpg
Type: image/jpeg
Size: 7366 bytes
Desc: image002.jpg
URL: <http://lists.etherlab.org/pipermail/etherlab-users/attachments/20190527/bf5ef9ba/attachment-0009.jpg>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image2dbc08.JPG
Type: image/jpeg
Size: 15896 bytes
Desc: image2dbc08.JPG
URL: <http://lists.etherlab.org/pipermail/etherlab-users/attachments/20190527/bf5ef9ba/attachment-0008.jpe>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: imagefda576.JPG
Type: image/jpeg
Size: 7366 bytes
Desc: imagefda576.JPG
URL: <http://lists.etherlab.org/pipermail/etherlab-users/attachments/20190527/bf5ef9ba/attachment-0009.jpe>

More information about the Etherlab-users mailing list