[etherlab-users] Data Transfer options in TestManager

[Dr.-Ing. Wilhelm Hagemeister]

Hello Alex,

a bit of background to the testmanager. Actually I am the programmer of
the testmanager and of course, in reference to your earlier email, I
agree that there is a lot of documentation missing and also some
features are not really clear or self explanatory to the user. Actually
we are working on an improved version of the documentation and I will
make sure you get the english version as one of the first persons. But I
can't promise when this will be ready.

The testmanager is written in Delphi 5 ! and parts of the software are
already 12 years old and if you ask me where nobody can hear us, I would
not code it this days the way I designed it years ago.

It is a bit like an old house with a lot of sheds build around it. As a
user you might see that also, when you find out that some features are
missing which you can expect from a modern piece of software. A couple
of years ago we started to write a portable version in C++ and the
QT-library which compiles under Linux and Windows but we just don't have
the resources right now to bring this job to and usable state.

Anyway the testmanager does his job reliably and still gets bug fixed...

Now to your questions... find my annotations below:

Am 17.12.2010 12:03, schrieb Alex C. Marton:
> Dear friends at EtherLab community,
> 
> Thank you for giving me another chance! This time I will not let you down!
> I have been using TestManager in a real industrial hydraulics application for a 
> while now. I generally had good results with it, but now I wish to change my 
> sampling rate and also the accuracy of the data captured (I was using 1000 Hz @ 
> 2 accuracy which is too high). I have tried doing so but I see the following 
> observations about the options that I can set/choose:
> a. the sampling rate cannot be any value I wish to choose for all channels. I 
> tried sampling @ 300, 450, 720 Hz and always TestManager rounds off my value to 
> a different one, a calculated one I see, so it must implement a formula I do not 
> know about. I would like to sample @ 10 Hz in my application which I see it can 
> handle,  but I do not know why an intermediate value for the sample rate is not 
> possible.

We decided to make the data transfer rate from the realtime process to
the testmanger a multiple of the base tick.
Actually the downsampling algorithm just takes the n-th value of the
channel data. So with 1kHz base tick you can realize 500Hz, 250Hz,
200Hz, 166.666 Hz, 125Hz and so on but not 300,450 or 720Hz because they
are not a divider of 1000Hz.


> b. The data accuracy I choose is  not "recorded" while using the Scope. I tried 
> inputting different values from 1 to 9 and then verifying the .txt files 
> generated by exporting the acquired data in a Matlab format. Most of the time I 
> get values with 16 digits after the "." in the number. Other times I get values 
> with only 2 decimal points. I see the files are a lot larger when having 16 
> digit accuracy than when I have 2 digit accuracy and I would like to have 4 
> accuracy, because space is an issue for me.

The accuracy only has an effect on the data stream from the process to
the testmanager. Because the transport of the process values to the
testmanager is in formated text - one can discuss on that choice but a
binary format is also available but not used in testmanger - it is
useful in two ways.
1. it reduces the data rate when set to low values.
As an example: 123.34456465665,123.4567678790854,123.56778989043213132
takes more bandwith to transport than 123.34,123.45,123.56.

2. it enhances accuracy when dealing with small values:
0.0000010123,0.0000010232,0.0000010345,0.0000010123 is relatively more
accurate than 0.000001,0.000001,0.000001,0.000001

Internally all channel data are represented in double precision so for
exporting from the scope the accuracy has no direct affect on the saved
file size.

For sampling large amount of data you better use the "dls" program which
also comes with the EtherLab bundle. It brings effective data
compression algorithms with it.  The scope in the testmanager was only
intended to record over a "short" period of time and I didn't optimize
the export for file size. Also the sampled data gets stored and hold in
RAM so if sampling over a long period of time with the testmanager you
might run out of memory.



> c. What is the effect of a high "Buffer Size/s" in the Data Transfer option 
> pane? I already know the effect of having it too small (the graphics of the 
> signals are limited), but I would like to know what is the drawback of having it 
> too large? One thing I notice is the Data Rate going very high. What is an 
> optimal value for that to input?

Recieved data gets stored in a ring buffer. And the size of the ring
buffer is what you adjust here. High values with high sample rates will
use a lot of ram. The ring buffer prevents loss of data, I thought years
ago, not taking into account that the TCP/IP transport mechanism has a
buffer as well which might be sufficient. The scrolling instruments
display data directly from the ring buffer so they don't bring their own
buffer. If you make the buffer shorter than the time axis of this
instruments the signals will be limited as you already worked out. The
buffer size should have no influence on the data rate!! A value of 1 sec
is no problem is you don't have scrolling instruments, otherwise you
have to set the buffer size at least to the time interval of these
instruments for the channels you display there.


> My Data Rate I read here right now is  70-80 Kb/s @ 300 Hz. Does this have an 
> effect on the topics I've written so far?

For just visualization I use 10Hz update rate, for analysis I transfer
as quick as is necessary; just keep an eye on how much the testmanager
and also the etherlab_buddy loads your machines. Also the max. bandwidth
of your network might be an issue then.

Have fun with the testmanager

which version are you using?

Regards Wilhelm.