[etherlab-users] Call to ecrt_domain_process takes sometimes very long (10ms!)

[Michael Ruder]

Hello again,

after sleeping over the issue, I did some more tests.

In fact, the function that takes the long time is ecrt_domain_process (and not ecrt_domain_queue). However, if I do no longer the call to ecrt_domain_queue, then the ecrt_domain_process will not take long (that is what got me on the wrong track yesterday).

So during startup (Link up, Master gets to OP etc.), one or two calls to ecrt_domain_process take about 10 ms, instead of virtually nothing (500 ns) later on. When I disconnect the network cable from the slave then again some calls take long and after reconnecting as well. So always during transitions of the state.

This only happens on my Yocto system but not on a Gentoo system with the identical kernel (4.14.28-rt23). On the Gentoo, the calls never take long, even during transitions.
 
Thanks a lot for any hint,
Michael

> I successfully got the IgH EtherCAT Master running on an IEI board using Gentoo and the 4.14.28-rt23 PREEMPT kernel.
>
> Now, I switched to a self-made, yocto-based distribution, using the same kernel (I tried both the kernel built by our yocto
> recipe and my own "hand-built" kernel with the exact same result) and now I have a very strange problem:
> 
> The call to ecrt_domain_queue takes sometimes very long (about 10 ms, I measured that call separately). During normal operation 
> it takes no considerable amount of time but while the system is coming up (switching to OP etc.) or if I disconnect a cable it 
> starts taking those large amounts of time. Then of course a 1 ms cycle time fails catastrophic.
 
> On my gentoo based system, this behaviour does not occur. It never takes too much time, even if I disconnect a cable or reboot a 
> slave. 
 
> As it does not seem to be kernel related, what else could it be? GCC versions differ of course, yocto using 7.2.0 while gentoo is on
> 6.4.0.
 
> I narrowed down the issue to the ec_user_example already.
 
> Cyclictest shows no abnormality[http://www.dict.cc/englisch-deutsch/abnormality.html], max latency is < 25 us on both systems. I run
> the system with isolcpus=3 (and smp_affinity=7 for irq) and the cyclictest/ec test program with taskset 8 on the isolated core. So I
> rule out a generic latency issue.
 
> I am really a bit puzzled where to start debugging this strange issue and are happy for any hint!