[etherlab-dev] Userspace fork of Etherlab

[Frank Heckenbach]

Hi everybody,

this is to announce my plan to port the Etherlab code to userspace.
I'll explain my reasons and roadmap below. If someone is interested
in this or has some comments, please let me know. Otherwise, I
expect to proceed on my own and publish the result on my web site
when finished.

Reasons:

- I had to make a number of changes to the Etherlab code to fix some
  bugs and make it useable to us. The Etherlab developers are
  obviously not interested in those changes, so I have to maintain
  them myself. As was discussed on this list some months ago,
  keeping up with newer Etherlab versions would cost me additional
  maintenance and testing effort (already now since my changes are
  based on 1.5.0, whereas 1.5.2 has some conflicting changes), so
  I'd use my own fork of 1.5.0 which is known to work for us rather
  than 1.5.2 anyway.

- Keeping up with new kernel versions is also not always easy
  (especially for the drivers which are patched files from the
  standard kernel, but also other kernel interfaces are known to
  change often), whereas userspace code is much easier to maintain
  (incompatible library changes are quite rare).

- So far we've been using RTAI for our realtime code. But that's
  also always been a bit troublesome (kernel version dependencies,
  high crash potential in case of problems, additional code with its
  own set of bugs, etc.), so we'd rather try to get rid of it
  anyway. Meanwhile the RT capabilities of the standard kernel have
  improved in recent years, and due to the wide availability of SMP,
  we can, if necessary, increase RT-ability by using CPU affinity
  (reserve one CPU for RT code, leave the other CPUs for the rest --
  of course, kernel code, esp. network drivers might need special
  consideration here).

- Our application code uses a lot of floating point which is
  supported in RTAI (though with some quirks), not in non-RTAI
  kernel mode, but of course easily in userspace.

- Userspace code is generally much easier to debug.

- If I had known and considered all this back then, I might have
  started with other code instead of Etherlab which is already
  userspace based, but has different interfaces (and possibly
  different bugs). But as things are now, since my code is tightly
  bound to the Etherlab interface, and well tested with (the patched
  version of) it, it seems easier to port this code to userspace
  than change my application code.

Roadmap:

- Start with 1.5.0 plus my patches.

- Replace kernel infrastructure by corresponding userspace
  functionality (e.g. kthread -> pthread, semaphore ->
  pthread_mutex, kmalloc -> malloc, kprintf -> fprintf); copy kernel
  utilities that have no actual kernel dependencies (in particular
  kernel lists).

- Turn the code into a library which the application can use, so the
  userspace application thread(s) will do what the (normal or RTAI)
  kernel application thread(s) did and, being part of the same
  process, they can access the master's data structures (e.g. PDO
  memory, datagram queues) without explicit shared memory or such.
  (Though the library will start some threads of its own, e.g.
  master, EoE and cdev (see below), just like the master now starts
  some kthreads.) This will not allow the master to stay resident
  while the application is restarted, so it will need to re-scan the
  bus every time. This is not a high-priority problem for me
  (because we rarely restart our application code, and when we do,
  the hardware reinitialization takes longer than the bus scan
  anyway). If needed, one could solve this problem by putting a
  layer in userspace in between which would then need to do IPC
  and/or use shared memory for communication.

- Use the generic backend driver which also shouldn't require too
  many changes since it already uses a raw ethernet socket which is
  also available from userspace).

- Implement EoE via a tap/tun device.

- Replace the cdev ioctl interface by a (normal TCP) socket and
  adapt the library to use it. Basically wrap each ioctl data
  structure as-is together with its cmd value in a data packet.

- Omit the mmap for the cdev because I don't need it. (If someone
  has a need for it, it could be supported via userspace shared
  memory.)

- Omit the tty because I don't need it. (If someone has a need for
  it, it could be supported via a pty.)

- Generally, make as few changes to the code as possible
  (e.g. #define stuff in a common header rather than making global
  replacements wherever possible).

Regards,
Frank