[etherlab-users] etherlab dc sync check

[Jun Yuan]

Thanks Jeroen.
I see you have only posted the synchrony process for the second slave.
There're no "Checking for synchrony" for the other slaves? Is it the
only slave with DC that you are using in your application?

On Fri, Jan 3, 2014 at 11:45 PM, Jeroen Van den Keybus
<jeroen.vandenkeybus at gmail.com> wrote:
> As far as 'simple experiment' goes. Here is the requested log of a 17-slave
> configuration:
>
>
> [2020582.839193] EtherCAT DEBUG 0-0: Checking system time offset.
> [2020582.843201] EtherCAT DEBUG 0-0: DC 64 bit system time offset
> calculation: system_time=1388788619730212737 (corrected with 4000000),
> app_time=1388788619729210397, diff=-1002340
> [2020582.843204] EtherCAT DEBUG 0-0: Setting time offset to
> 1388787098688224529 (was 1388787098689226869)
> [2020582.847211] EtherCAT DEBUG 0-1: Checking system time offset.
> [2020582.851219] EtherCAT DEBUG 0-1: DC 64 bit system time offset
> calculation: system_time=1388788619738253673 (corrected with 4000000),
> app_time=1388788619737210345, diff=-1043328
> [2020582.851221] EtherCAT DEBUG 0-1: Setting time offset to
> 1388787098694672687 (was 1388787098695716015)
> [2020582.855227] EtherCAT DEBUG 0-2: Checking system time offset.
> [2020582.859235] EtherCAT DEBUG 0-2: DC 64 bit system time offset
> calculation: system_time=1388788619746127955 (corrected with 4000000),
> app_time=1388788619745210316, diff=-917639
> [2020582.859237] EtherCAT DEBUG 0-2: Not touching time offset.
> [2020582.863244] EtherCAT DEBUG 0-3: Checking system time offset.
> [2020582.867251] EtherCAT DEBUG 0-3: DC 64 bit system time offset
> calculation: system_time=1388788619754132695 (corrected with 4000000),
> app_time=1388788619753210302, diff=-922393
> [2020582.867253] EtherCAT DEBUG 0-3: Not touching time offset.
> [2020582.871260] EtherCAT DEBUG 0-4: Checking system time offset.
> [2020582.875267] EtherCAT DEBUG 0-4: DC 64 bit system time offset
> calculation: system_time=1388788619762180961 (corrected with 4000000),
> app_time=1388788619762210451, diff=29490
> [2020582.875269] EtherCAT DEBUG 0-4: Not touching time offset.
> [2020582.879278] EtherCAT DEBUG 0-5: Checking system time offset.
> [2020582.883286] EtherCAT DEBUG 0-5: DC 64 bit system time offset
> calculation: system_time=1388788619767191997 (corrected with 0),
> app_time=1388788619770210343, diff=3018346
> [2020582.883288] EtherCAT DEBUG 0-5: Setting time offset to
> 1388787098704211274 (was 1388787098701192928)
> [2020582.887294] EtherCAT DEBUG 0-6: Checking system time offset.
> [2020582.891302] EtherCAT DEBUG 0-6: DC 64 bit system time offset
> calculation: system_time=1388788619775150970 (corrected with 0),
> app_time=1388788619778210426, diff=3059456
> [2020582.891304] EtherCAT DEBUG 0-6: Setting time offset to
> 1388787098700760295 (was 1388787098697700839)
> [2020582.895311] EtherCAT DEBUG 0-7: Checking system time offset.
> [2020582.899318] EtherCAT DEBUG 0-7: DC 64 bit system time offset
> calculation: system_time=1388788619783154795 (corrected with 0),
> app_time=1388788619786210495, diff=3055700
> [2020582.899321] EtherCAT DEBUG 0-7: Setting time offset to
> 1388787098701077013 (was 1388787098698021313)
> [2020582.903327] EtherCAT DEBUG 0-8: Checking system time offset.
> [2020582.907334] EtherCAT DEBUG 0-8: DC 64 bit system time offset
> calculation: system_time=1388788619791155722 (corrected with 0),
> app_time=1388788619794210387, diff=3054665
> [2020582.907337] EtherCAT DEBUG 0-8: Setting time offset to
> 1388787098702059792 (was 1388787098699005127)
> [2020582.911343] EtherCAT DEBUG 0-9: Checking system time offset.
> [2020582.915351] EtherCAT DEBUG 0-9: DC 64 bit system time offset
> calculation: system_time=1388788619799144106 (corrected with 0),
> app_time=1388788619802210392, diff=3066286
> [2020582.915353] EtherCAT DEBUG 0-9: Setting time offset to
> 1388787098703046442 (was 1388787098699980156)
> [2020582.919360] EtherCAT DEBUG 0-10: Checking system time offset.
> [2020582.923367] EtherCAT DEBUG 0-10: DC 64 bit system time offset
> calculation: system_time=1388788619807217599 (corrected with 0),
> app_time=1388788619810210385, diff=2992786
> [2020582.923369] EtherCAT DEBUG 0-10: Setting time offset to
> 1388787098694401972 (was 1388787098691409186)
> [2020582.927376] EtherCAT DEBUG 0-11: Checking system time offset.
> [2020582.931383] EtherCAT DEBUG 0-11: DC 64 bit system time offset
> calculation: system_time=1388788619815217343 (corrected with 0),
> app_time=1388788619818210352, diff=2993009
> [2020582.931386] EtherCAT DEBUG 0-11: Setting time offset to
> 1388787098694270706 (was 1388787098691277697)
> [2020582.935392] EtherCAT DEBUG 0-12: Checking system time offset.
> [2020582.939400] EtherCAT DEBUG 0-12: DC 64 bit system time offset
> calculation: system_time=1388788619823158753 (corrected with 0),
> app_time=1388788619826210432, diff=3051679
> [2020582.939402] EtherCAT DEBUG 0-12: Setting time offset to
> 1388787098702271332 (was 1388787098699219653)
> [2020582.943409] EtherCAT DEBUG 0-13: Checking system time offset.
> [2020582.947416] EtherCAT DEBUG 0-13: DC 64 bit system time offset
> calculation: system_time=1388788619831157270 (corrected with 0),
> app_time=1388788619834210362, diff=3053092
> [2020582.947418] EtherCAT DEBUG 0-13: Setting time offset to
> 1388787098703045985 (was 1388787098699992893)
> [2020582.951425] EtherCAT DEBUG 0-14: Checking system time offset.
> [2020582.955432] EtherCAT DEBUG 0-14: DC 64 bit system time offset
> calculation: system_time=1388788619839165891 (corrected with 0),
> app_time=1388788619842210326, diff=3044435
> [2020582.955435] EtherCAT DEBUG 0-14: Setting time offset to
> 1388787098703499171 (was 1388787098700454736)
> [2020582.959441] EtherCAT DEBUG 0-15: Checking system time offset.
> [2020582.963449] EtherCAT DEBUG 0-15: DC 64 bit system time offset
> calculation: system_time=1388788619847182087 (corrected with 0),
> app_time=1388788619850210371, diff=3028284
> [2020582.963451] EtherCAT DEBUG 0-15: Setting time offset to
> 1388787098700973845 (was 1388787098697945561)
> [2020582.967458] EtherCAT DEBUG 0-16: Checking system time offset.
> [2020582.971465] EtherCAT DEBUG 0-16: DC 32 bit system time offset
> calculation: system_time=1116891966 (corrected with 0),
> app_time=1388788619858210417, diff=3048243
> [2020582.971467] EtherCAT DEBUG 0-16: Setting time offset to 614131959 (was
> 611083716)
>
> [2020583.075648] EtherCAT DEBUG 0-1: Checking for synchrony.
> [2020583.079655] EtherCAT DEBUG 0-1: 37 ns difference after 4 ms.
>
> Cycle time: 1ms
>
>
> J.
>
>
> 2014/1/3 Jun Yuan <j.yuan at rtleaders.com>
>>
>> Oh, I understand, you're talking about synchronizing the master clock
>> to the ref slave. This has been implemented in the etherlab master.
>> You need to use the function ecrt_master_reference_clock_time()
>> instead of ecrt_master_sync_reference_clock(). An example is given in
>> example/rtai_rtdm_dc.
>>
>> Well, I think most of us are still used to sync reference clock to
>> master. And yes, sync master to ref clock would make the situation a
>> little bit better, as the ref clock itself doesn't need any adjustment
>> any more, making the ref clock signal stable for the other slaves. But
>> since the other slave are still having a bad time offset from the
>> master to have a same timebase as the ref_clock, they still suffer
>> from a large diff at the beginning of drift compensation. The problem
>> is not still there.
>>
>> I need some volunteers to do a simple experiment for me. You'll just
>> need a normal master application based on the current etherlabmaster
>> and a ethercat bus network with at least one slave having DC enabled.
>> 1. running your master application for about 10 minutes. This would be
>> enough for all the DC sync on the bus get converged. So that all the
>> DCs are running synchronized. You can check that with „ethercat
>> reg_read -tsm32 0x92c“, it should  be something below 1000 ns.
>> 2. turn the debug level of master to 1 using „sudo ethercat debug 1“.
>> 3. stop the master application and restart it right away, while don't
>> shut down your slaves. Your slaves should still have a relative good
>> sync with each other.
>> 4. check you system log, look for the lines like
>>
>> [ 9766.885265] EtherCAT DEBUG 0-0: DC 64 bit system time offset
>> calculation: system_time=xxxxxxxx (corrected with xxxxxxxx),
>> app_time=xxxxxxx, diff=xxxxxxxx
>> [ 9766.885268] EtherCAT DEBUG 0-0: „Setting time offset to xxxxxxx
>> (was xxxxxxx)“ OR: „Not touching time offset.“
>> …
>> [ 9767.292758] EtherCAT DEBUG 0-0: Checking for synchrony.
>> [ 9767.296758] EtherCAT DEBUG 0-0: Sync after    4 ms:    xxxxxxx ns
>>
>> send these 4 lines of your system log to me, and tell me the cycle
>> time of your loop. That’s it. Thanks!
>>
>> On Fri, Jan 3, 2014 at 10:52 AM, Slutsker, Rasty
>> <rasty.slutsker at servotronix.com> wrote:
>> > Master clock is accurate (more or less), but delivery is not, since
>> > software is involved.
>> > I'm just asking why slave(s) shall synchronize to "jumpy" clock of
>> > master? Maybe be it would be more correct, if master could synchronize
>> > itself to the first slave and then will run pll adjusting it's clock to the
>> > slave's one?
>> > In that case we can completely eliminate this time-consuming
>> > synchronization phase.
>> >
>> > Say,
>> > a) master sends it's virtual clock to the first slave and that slave
>> > becomes a clock master
>> > b) master re-distributes fist slave's clock to other slaves
>> > c) master adjust it's clock permanently, relatively to the clock master
>> > (first slave)
>> > d) master periodically re-distributes clock from the first slave to
>> > other in order to eliminate time skew in slave.
>> >
>> >
>> > ________________________________________
>> > From: Jun Yuan [j.yuan at rtleaders.com]
>> > Sent: Friday, January 03, 2014 11:10 AM
>> > To: Slutsker, Rasty
>> > Cc: Jeroen Van den Keybus; Raz; etherlab-users at etherlab.org
>> > Subject: Re: [etherlab-users] etherlab dc sync check
>> >
>> > I'm not saying that the clock of the master is inaccurate. I'm saying
>> > that the master software does a bad job in the calculation of the time
>> > offset for each slave. It should not use jiffies in the calculation,
>> > It should not have any correction at all. The app_time when the fprd
>> > datagram to 0x0910 is sent, let's call it app_time_sent, is exactly
>> > the time which should be compared to the system time of the slave.
>> >
>> >> Why not just send once a current muster time to the first slave, then
>> >> propagate(copy)  it to other slaves, and run PLL in the master that adjust
>> >> itself to the fist slave (clock master)?
>> >
>> > I think we cannot do that, it's against the ethercat standard :)
>> > It actually works similiar like you suggest, just in another way. It
>> > is something like this:
>> > Ref clock -> Master clock
>> > 1. The master asks the first slave which time it has.
>> > 2. The master compares the timestamp with its app time, and sends a
>> > new time offset to the first slave.
>> > 3. The first slave adds the new time offset to its clock. Now he has
>> > the same time as the master. Drift is the next thing to worry about.
>> >
>> > Other slave clock -> Ref clock
>> > 1. The master asks the slave which is not a ref clock which time it has.
>> > 2. The master compares the timestamp with the time of the ref clock,
>> > and sends a new time offset to the slave.
>> > 3. The slave adds the new time offset to its clock. Now he shall have
>> > the same time as the ref clock.
>> >
>> > I believe the current etherlabmaster doesn't do well in step 2. it
>> > could be done better.
>>
>>
>>
>> --
>> Jun Yuan
>> [Aussprache: Djün Üän]
>>
>> Robotics Technology Leaders GmbH
>> Am Loferfeld 58, D-81249 München
>> Tel: +49 89 189 0465 24
>> Mobile: +49 176 2176 5238
>> Fax: +49 89 189 0465 11
>> mailto: j.yuan at rtleaders.com
>>
>> Umlautregel in der chinesischen Lautschrift Pinyin: Nach den Anlauten
>> y, j, q, und x wird u als ü ausgesprochen, z.B. yu => ü,  ju => dschü,
>>  qu => tschü,  xu => schü.
>
>



-- 
Jun Yuan
[Aussprache: Djün Üän]

Robotics Technology Leaders GmbH
Am Loferfeld 58, D-81249 München
Tel: +49 89 189 0465 24
Mobile: +49 176 2176 5238
Fax: +49 89 189 0465 11
mailto: j.yuan at rtleaders.com

Umlautregel in der chinesischen Lautschrift Pinyin: Nach den Anlauten
y, j, q, und x wird u als ü ausgesprochen, z.B. yu => ü,  ju => dschü,
 qu => tschü,  xu => schü.