[etherlab-users] MultiTask/Process Application
tom at wongcornall.com
Sat Oct 19 08:58:43 CEST 2019
On Sat, Oct 19, 2019 at 02:58:06PM NZDT, Peter Dupej (student) <p.dupej.1 at research.gla.ac.uk> wrote:
>Is it possible with EtherLAB to write a RT Application with multiple tasks
>running on multiple cores accessing a single master device similarly as
>modern TwinCAT 3 allows?
Yes, with caveats; I imagine this is how most of us are using EtherLab (at
least those who use it in userspace).
>I understand that there can only be one master process reserving the network
>device, but would you be able to configure a set of 2-3 additional processes
>that would communicate with the main one to distribute the computation load
>onto multiple cores?
That would be a reasonably standard way to do it, either separate processes or
separate threads. We use it like this, and I would guess many others would
find similarities in the following (although the RTAI folks would have a few
- Application runs in userspace, communicating with EtherLab running in
kernelspace (PREEMPT-RT in our case)
- One thread gets given highest scheduling priority and dedicated CPU; this
thread alone talks with EtherLab; although it does do some calculations
(like generating servo trajectories etc.), mostly it's fairly dumb and just
copies data to/fro EtherLab
- Other threads in same process do some heavier calculations and
communication with outside devices (CANbus, old-fashioned Modbus/TCP, RS232
etc.), and communicate with this thread via standard process memory
protected by normal pthread mutexes
- These other threads run on separate cores, still with high priority (but
not as high as the EtherCAT thread). We use things like "try-lock" on the
mutexes to try and avoid priority inversion, as our EtherCAT setup uses
distributed clocks and is very picky about latency.
- These other threads also communicate with a number of other processes on
the same box via all sorts of IPC mechanisms like shared memory, named
pipes, local sockets etc. These other processes do things like show GUIs,
log to disk, load/save production files for the machine to process etc.
If you're using a standard kernel (or even one with PREEMPT-RT patch), and
running in userspace, then your `EtherCAT process' is really just an ordinary
process with all the normal capabilities built into it for communicating with
other processes or threads.
The only thing to be aware of is that many EtherCAT applications are quite
sensitive of scheduling latency, especially if you're running distributed
clocks, so the natural bias is to use fast low-latency IPC mechanisms that
won't cause your EtherCAT process to get `unscheduled'. Shared memory is good
for this, maybe serialising/deserialising XML down a local TCP/IP socket would
not be so wise...
More information about the etherlab-users