<p dir="ltr">2) sync the master to ref slave<br>
sorry about the error message. It is because I let the ecrt_master_reference_clock_time return EAGAIN error when the system time offset has not been calculated yet. I tried to avoid the error message in lib/master.c, but apparently I didn't do it right. I'll fix it ASAP. </p>
<p dir="ltr">Graeme Foot did it with 32-bits. I don't think we can have it in 64-bits, but I will check it out. You'll have the explanation tomorrow when I get back to the office. <br>
</p>
<div class="gmail_quote">Am 08.04.2014 06:30 schrieb "Gavin Lambert" <<a href="mailto:gavinl@compacsort.com">gavinl@compacsort.com</a>>:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div lang="EN-NZ" link="blue" vlink="purple"><div><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">After changing my user code to work similarly to your RTAI example (matching the master local clock to the reference clock directly instead of letting the master update the reference clock) then the DC sync reaches 0ns variation almost instantly (since there’s only a single slave at the moment).<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">However I did have to remove the fprintf from lib/master.c’s ecrt_master_reference_clock_time, as otherwise the app gets spammed into oblivion by an I/O error (presumably because it has not yet chosen the slave to use as the DC ref clock; I want this to use the default auto-selection most of the time).<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">I’m not sure I like this approach, though, as unless I’ve missed something this only seems to be able to sync the low 32-bits of the clocks, and the slave is producing 64-bit timestamps as process data that need to be convertible to the master’s local clock.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p><div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt">
<div><div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0cm 0cm 0cm"><p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Gavin Lambert<br>
<b>Sent:</b> Tuesday, 8 April 2014 14:35<br><b>To:</b> 'Jun Yuan'<br><b>Cc:</b> <a href="mailto:etherlab-users@etherlab.org" target="_blank">etherlab-users@etherlab.org</a><br><b>Subject:</b> Re: [etherlab-users] DC-Synchronization - Sync signal generation<u></u><u></u></span></p>
</div></div><p class="MsoNormal"><u></u> <u></u></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Hi Jun,<u></u><u></u></span></p><p class="MsoNormal">
<span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">I tried rebuilding the master with the code from your bundle (merged with the latest 1.5.2 tip) but I was still getting the 5000ms sync timeouts. (Although it does seem a lot rarer for it to actually print that error, it does still seem to take a few seconds to get to OP, which is indicative of sync taking a while.)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">I haven’t tried modifying my user code yet, but the way that it works at the moment is:<u></u><u></u></span></p>
<p><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><span>-<span style="font:7.0pt "Times New Roman""> </span></span></span><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Outside the cyclic thread, it calls ecrt_master_set_send_interval(master, TIMESPEC2NS(cycletime)/1000), sets up the slaves, and starts the cyclic thread (no calls to set master time).<u></u><u></u></span></p>
<p><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><span>-<span style="font:7.0pt "Times New Roman""> </span></span></span><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">At the top of the cyclic thread, it activates the master and inits the first wakeup time (as in the original dc_user example).<u></u><u></u></span></p>
<p><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><span>-<span style="font:7.0pt "Times New Roman""> </span></span></span><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">In the cyclic loop, it sleeps until the wakeup time, then receives and processes, and:<u></u><u></u></span></p>
<p style="margin-left:72.0pt"><u></u><span style="font-size:11.0pt;font-family:"Courier New";color:#1f497d"><span>o<span style="font:7.0pt "Times New Roman""> </span></span></span><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Every loop: ecrt_master_application_time(master, TIMESPEC2NS(time));<u></u><u></u></span></p>
<p style="margin-left:72.0pt"><u></u><span style="font-size:11.0pt;font-family:"Courier New";color:#1f497d"><span>o<span style="font:7.0pt "Times New Roman""> </span></span></span><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Every 2nd loop: ecrt_master_sync_reference_clock(master);<u></u><u></u></span></p>
<p style="margin-left:72.0pt"><u></u><span style="font-size:11.0pt;font-family:"Courier New";color:#1f497d"><span>o<span style="font:7.0pt "Times New Roman""> </span></span></span><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Every loop: ecrt_master_sync_slave_clocks(master);<u></u><u></u></span></p>
<p><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><span>-<span style="font:7.0pt "Times New Roman""> </span></span></span><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Then queue & send, and repeat loop.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">So it’s essentially the same as the dc_user example. Currently I’m testing it on a network with only one DC-enabled slave.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Possibly of interest is that the error seems to be related to the slave boots – if I reboot the slave (resetting its internal clock) then the next start of the master application will produce the 5000ms timeout. Subsequent starts of the master app seem to start ok.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">If I set it to “debug 1” then on that first run it prints “Sync after 4996 ms: 4293798555 ns” (and the number was decreasing), which looks like something needs to be signed rather than unsigned (it’s about -1ms). On subsequent runs the synchrony seems to be typically around 600ns (sometimes up to about 2000ns), which is pretty good. (In rare cases it does the negative value thing again although with a much smaller magnitude, and it takes a third or fourth try to “really” lock it in.)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">I had a quick look at nearby code and it looks like the number is a formatting bug; this patch fixes it:<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1f497d">--- a/master/fsm_slave_config.c<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1f497d">+++ b/master/fsm_slave_config.c<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1f497d">@@ -1400,8 +1400,8 @@<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1f497d"> EC_SLAVE_WARN(slave, "Slave did not sync after %lu ms.\n",<u></u><u></u></span></p><p class="MsoNormal">
<span style="font-size:11.0pt;font-family:"Courier New";color:#1f497d"> diff_ms);<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1f497d"> } else {<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1f497d">- EC_SLAVE_DBG(slave, 1, "Sync after %4lu ms: %10u ns\n",<u></u><u></u></span></p><p class="MsoNormal">
<span style="font-size:11.0pt;font-family:"Courier New";color:#1f497d">- diff_ms, EC_READ_U32(datagram->data) & 0x80000000 ? –abs_sync_diff: abs_sync_diff);<u></u><u></u></span></p><p class="MsoNormal">
<span style="font-size:11.0pt;font-family:"Courier New";color:#1f497d">+ EC_SLAVE_DBG(slave, 1, "Sync after %4lu ms: %10d ns\n",<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1f497d">+ diff_ms, (EC_READ_U32(datagram->data) & 0x80000000) ? –abs_sync_diff: abs_sync_diff);<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1f497d"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1f497d"> // check synchrony again<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1f497d"> ec_datagram_fprd(datagram, slave->station_address, 0x092c, 4);<u></u><u></u></span></p><p class="MsoNormal">
<span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">However the underlying problem remains; this just makes it show the initial sync difference after 1388ms is -371806ns and it gets worse over time instead of better (finishing up after 5s at close to -1ms). Subsequent runs always seem to be better, although sometimes it takes 2-5 runs to get it “right”. <u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Regards,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Gavin Lambert<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt"><div><div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0cm 0cm 0cm"><p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Jun Yuan [<a href="mailto:j.yuan@rtleaders.com" target="_blank">mailto:j.yuan@rtleaders.com</a>] <br>
<b>Sent:</b> Friday, 4 April 2014 21:00<br><b>To:</b> Gavin Lambert<br><b>Cc:</b> <a href="mailto:etherlab-users@etherlab.org" target="_blank">etherlab-users@etherlab.org</a><br><b>Subject:</b> Re: [etherlab-users] DC-Synchronization - Sync signal generation<u></u><u></u></span></p>
</div></div><p class="MsoNormal"><u></u> <u></u></p><div><div><div><div><p class="MsoNormal" style="margin-bottom:12.0pt">Hi,<u></u><u></u></p></div><p class="MsoNormal" style="margin-bottom:12.0pt">that's all right. I'm using Xenomai. I just want to demonstrate the idea about how to synchronize the master clock to ref slave clock in an alternative way. I choose the RTAI example to have the comparison with the method of Graeme Foot. <u></u><u></u></p>
</div><p class="MsoNormal">You can test the rest nevertheless without that part. I didn't change the API, so you don't need to change anything in your code. But <br>1) If you call ecrt_master_application_time() outside of the loop, it is recommended that you remove it. <br>
2) And only if you use ecrt_master_reference_clock_time(), you need to notice that at the program start, before the calculation of dc system time offsets for each slave has been done, ecrt_master_reference_clock_time() would now have errno EAGAIN to notify the user that the ref clock is not ready yet. So it is worth to always check the return value of ecrt_master_reference_clock_time(), like I did in my rtai_rtdm_dc example.<u></u><u></u></p>
</div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal">That's it.<u></u><u></u></p></div><div><p class="MsoNormal"><u></u> <u></u></p></div><p class="MsoNormal">Regards,<br>Jun<u></u><u></u></p>
</div><div><p class="MsoNormal" style="margin-bottom:12.0pt"><u></u> <u></u></p><div><p class="MsoNormal">On Fri, Apr 4, 2014 at 3:17 AM, Gavin Lambert <<a href="mailto:gavinl@compacsort.com" target="_blank">gavinl@compacsort.com</a>> wrote:<u></u><u></u></p>
<div><div><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Hi Jun,</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Thanks; I’m having a look at it, but much of it is new to me. I’m using PREEMPT_RT so my code is based on the dc_user example, not the RTAI examples, and I’d probably have to try adapting it before I could test it.</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Regards,</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Gavin Lambert</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt"><div><div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0cm 0cm 0cm"><p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Jun Yuan [mailto:<a href="mailto:j.yuan@rtleaders.com" target="_blank">j.yuan@rtleaders.com</a>] <br>
<b>Sent:</b> Friday, 4 April 2014 01:46<br><b>To:</b> Gavin Lambert<br><b>Cc:</b> <a href="mailto:etherlab-users@etherlab.org" target="_blank">etherlab-users@etherlab.org</a><br><b>Subject:</b> Re: [etherlab-users] DC-Synchronization - Sync signal generation</span><u></u><u></u></p>
</div></div><div><p class="MsoNormal"> <u></u><u></u></p><div><div><div><div><p class="MsoNormal" style="margin-bottom:12.0pt">Hi Gavin,<u></u><u></u></p></div><p class="MsoNormal" style="margin-bottom:12.0pt">your interest is my motivation. I have attached the bundle file. <u></u><u></u></p>
</div><p class="MsoNormal" style="margin-bottom:12.0pt">My changes is base on the newest Version 1.5.2 in 'stable-1.5' branch. I added a new 'rtleaders' branch first and did all my changes on that. So after "$ hg unbundle etherlab_1.5.2_jyuan.hg", don't forget to switch to the 'rtleaders' branch using "$ hg update rtleaders".<u></u><u></u></p>
</div><div><p class="MsoNormal" style="margin-bottom:12.0pt">I found a better way of synchronizing the master clock to ref slave clock. It is much faster and more stable. I managed to port my C++ code into C code in the rtai_rtdm_dc example today, but I cannot test if the new code compiles right now. If you have a rtai environment, please test it for me if it compiles, and give me some feedback. <u></u><u></u></p>
</div><div><p class="MsoNormal" style="margin-bottom:12.0pt">Besides that, there is a more accurate DC time offset calculation. There should be no more errors like "Slave did not sync after 5000ms". The accurate time offset estimation saves much time for the DC Sync procedure. Slaves would have such a small dc diff (several hundred ns maybe) at the beginning of the dc sync check, that I even changed EC_SYSTEM_TIME_TOLERANCE_NS from 1000000ns to 1000ns.<br>
<br>The postponed check of master->has_app_time makes the error "No app_time received up to now, but master already active" away.<u></u><u></u></p></div><div><p class="MsoNormal">And there are the bugfix for ecrt_master_select_reference_clock() from Graeme Foot, and some other bug fixes from Jeroen Van den Keybus.<u></u><u></u></p>
</div><div><p class="MsoNormal"> <u></u><u></u></p></div><div><p class="MsoNormal">Any feedback is welcome. Have fun testing those changes!<u></u><u></u></p></div><div><p class="MsoNormal"> <u></u><u></u></p></div><div><p class="MsoNormal">
Jun<u></u><u></u></p></div></div><div><p class="MsoNormal" style="margin-bottom:12.0pt"> <u></u><u></u></p><div><p class="MsoNormal">On Thu, Apr 3, 2014 at 12:13 AM, Gavin Lambert <<a href="mailto:gavinl@compacsort.com" target="_blank">gavinl@compacsort.com</a>> wrote:<u></u><u></u></p>
<p class="MsoNormal">On 2 April 2014 22:40, quoth Jun Yuan:<u></u><u></u></p><div><p class="MsoNormal" style="margin-bottom:12.0pt">> But there is a reason why we all put the ecrt_master_application_time() outside<br>> the loop. Because we all got burned by the error "No app_time received up to<br>
> now, but master already active.", which is a timing bug in Etherlab. I've<br>> resolved the problem by change the code of Etherlabmaster, which get rid of<br>> the "No app_time" bug. Now I don't need to call ecrt_master_application_time()<br>
> outside the loop any more. I will publish the bundle to the mailing list when<br>> I have time.<u></u><u></u></p></div><p class="MsoNormal" style="margin-bottom:12.0pt;margin-left:5.25pt">I'd be very interested to see this. Slave sync timing, "no app time", and the 5000ms sync timeout have been a recurring bugbear for me.<u></u><u></u></p>
</div></div></div></div></div></div></div><p class="MsoNormal"><br><br clear="all"><br>-- <br>Jun Yuan<br>[Aussprache: Djün Üän]<br><br>Robotics Technology Leaders GmbH<br>Am Loferfeld 58, D-81249 München<br>Tel: <a href="tel:%2B49%2089%20189%200465%2024" value="+4989189046524" target="_blank">+49 89 189 0465 24</a><br>
Fax: <a href="tel:%2B49%2089%20189%200465%2011" value="+4989189046511" target="_blank">+49 89 189 0465 11</a><br>mailto: <a href="mailto:j.yuan@rtleaders.com" target="_blank">j.yuan@rtleaders.com</a><br><br>Umlautregel in der chinesischen Lautschrift Pinyin: Nach den Anlauten y, j, q, und x wird u als ü ausgesprochen, z.B. yu => ü, ju => dschü, qu => tschü, xu => schü. <u></u><u></u></p>
</div></div></div></div></div></blockquote></div>