<p dir="ltr">Thank you so much for the test. I am sorry for the formatting error. I should test it before send it out but I didn't have the chance. </p>
<p dir="ltr">I'm out of the office today. So I give you a quick answer without looking into the code first right now. </p>
<p dir="ltr">1) sync ref slave to master.<br>
>From your description, I didn't see any problem in your code. <br>
Apparently the slave needs more time to get synchronized at the first time after a reboot. I think it is because the drift compensation is reset after the reboot, and the slave losts the last drift estimation value to the master clock. <br>
If the slave do need such a long time to get DC sync converged after a reboot, I don't think I can do anything to make it better. I also doubt that the original Etherlab-Master can do it better. After all, the drift compensation algorithm is on the slave side, the master can only try to give the best transmission delay estimation and the system offset value. <br>
In the situation that the slave always gets reboot, the difference between my changes and the original code is not so apparently. You'll have the benefit if the master program always need to be rebooted, and the slave has been synchronized with the master once before. <br>
</p>
<div class="gmail_quot<blockquote class=" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div lang="EN-NZ" link="blue" vlink="purple"><div><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Hi Jun,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">I tried rebuilding the master with the code from your bundle (merged with the latest 1.5.2 tip) but I was still getting the 5000ms sync timeouts. (Although it does seem a lot rarer for it to actually print that error, it does still seem to take a few seconds to get to OP, which is indicative of sync taking a while.)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">I haven’t tried modifying my user code yet, but the way that it works at the moment is:<u></u><u></u></span></p>
<p><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><span>-<span style="font:7.0pt "Times New Roman""> </span></span></span><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Outside the cyclic thread, it calls ecrt_master_set_send_interval(master, TIMESPEC2NS(cycletime)/1000), sets up the slaves, and starts the cyclic thread (no calls to set master time).<u></u><u></u></span></p>
<p><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><span>-<span style="font:7.0pt "Times New Roman""> </span></span></span><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">At the top of the cyclic thread, it activates the master and inits the first wakeup time (as in the original dc_user example).<u></u><u></u></span></p>
<p><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><span>-<span style="font:7.0pt "Times New Roman""> </span></span></span><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">In the cyclic loop, it sleeps until the wakeup time, then receives and processes, and:<u></u><u></u></span></p>
<p style="margin-left:72.0pt"><u></u><span style="font-size:11.0pt;font-family:"Courier New";color:#1f497d"><span>o<span style="font:7.0pt "Times New Roman""> </span></span></span><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Every loop: ecrt_master_application_time(master, TIMESPEC2NS(time));<u></u><u></u></span></p>
<p style="margin-left:72.0pt"><u></u><span style="font-size:11.0pt;font-family:"Courier New";color:#1f497d"><span>o<span style="font:7.0pt "Times New Roman""> </span></span></span><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Every 2nd loop: ecrt_master_sync_reference_clock(master);<u></u><u></u></span></p>
<p style="margin-left:72.0pt"><u></u><span style="font-size:11.0pt;font-family:"Courier New";color:#1f497d"><span>o<span style="font:7.0pt "Times New Roman""> </span></span></span><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Every loop: ecrt_master_sync_slave_clocks(master);<u></u><u></u></span></p>
<p><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><span>-<span style="font:7.0pt "Times New Roman""> </span></span></span><u></u><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Then queue & send, and repeat loop.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">So it’s essentially the same as the dc_user example. Currently I’m testing it on a network with only one DC-enabled slave.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Possibly of interest is that the error seems to be related to the slave boots – if I reboot the slave (resetting its internal clock) then the next start of the master application will produce the 5000ms timeout. Subsequent starts of the master app seem to start ok.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">If I set it to “debug 1” then on that first run it prints “Sync after 4996 ms: 4293798555 ns” (and the number was decreasing), which looks like something needs to be signed rather than unsigned (it’s about -1ms). On subsequent runs the synchrony seems to be typically around 600ns (sometimes up to about 2000ns), which is pretty good. (In rare cases it does the negative value thing again although with a much smaller magnitude, and it takes a third or fourth try to “really” lock it in.)<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">I had a quick look at nearby code and it looks like the number is a formatting bug; this patch fixes it:<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1f497d">--- a/master/fsm_slave_config.c<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1f497d">+++ b/master/fsm_slave_config.c<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1f497d">@@ -1400,8 +1400,8 @@<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1f497d"> EC_SLAVE_WARN(slave, "Slave did not sync after %lu ms.\n",<u></u><u></u></span></p><p class="MsoNormal">
<span style="font-size:11.0pt;font-family:"Courier New";color:#1f497d"> diff_ms);<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1f497d"> } else {<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1f497d">- EC_SLAVE_DBG(slave, 1, "Sync after %4lu ms: %10u ns\n",<u></u><u></u></span></p><p class="MsoNormal">
<span style="font-size:11.0pt;font-family:"Courier New";color:#1f497d">- diff_ms, EC_READ_U32(datagram->data) & 0x80000000 ? –abs_sync_diff: abs_sync_diff);<u></u><u></u></span></p><p class="MsoNormal">
<span style="font-size:11.0pt;font-family:"Courier New";color:#1f497d">+ EC_SLAVE_DBG(slave, 1, "Sync after %4lu ms: %10d ns\n",<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1f497d">+ diff_ms, (EC_READ_U32(datagram->data) & 0x80000000) ? –abs_sync_diff: abs_sync_diff);<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1f497d"> <u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1f497d"> // check synchrony again<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Courier New";color:#1f497d"> ec_datagram_fprd(datagram, slave->station_address, 0x092c, 4);<u></u><u></u></span></p><p class="MsoNormal">
<span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">However the underlying problem remains; this just makes it show the initial sync difference after 1388ms is -371806ns and it gets worse over time instead of better (finishing up after 5s at close to -1ms). Subsequent runs always seem to be better, although sometimes it takes 2-5 runs to get it “right”. <u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Regards,<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Gavin Lambert<u></u><u></u></span></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt"><div><div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0cm 0cm 0cm"><p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Jun Yuan [mailto:<a href="mailto:j.yuan@rtleaders.com" target="_blank">j.yuan@rtleaders.com</a>] <br>
<b>Sent:</b> Friday, 4 April 2014 21:00<br><b>To:</b> Gavin Lambert<br><b>Cc:</b> <a href="mailto:etherlab-users@etherlab.org" target="_blank">etherlab-users@etherlab.org</a><br><b>Subject:</b> Re: [etherlab-users] DC-Synchronization - Sync signal generation<u></u><u></u></span></p>
</div></div><p class="MsoNormal"><u></u> <u></u></p><div><div><div><div><p class="MsoNormal" style="margin-bottom:12.0pt">Hi,<u></u><u></u></p></div><p class="MsoNormal" style="margin-bottom:12.0pt">that's all right. I'm using Xenomai. I just want to demonstrate the idea about how to synchronize the master clock to ref slave clock in an alternative way. I choose the RTAI example to have the comparison with the method of Graeme Foot. <u></u><u></u></p>
</div><p class="MsoNormal">You can test the rest nevertheless without that part. I didn't change the API, so you don't need to change anything in your code. But <br>1) If you call ecrt_master_application_time() outside of the loop, it is recommended that you remove it. <br>
2) And only if you use ecrt_master_reference_clock_time(), you need to notice that at the program start, before the calculation of dc system time offsets for each slave has been done, ecrt_master_reference_clock_time() would now have errno EAGAIN to notify the user that the ref clock is not ready yet. So it is worth to always check the return value of ecrt_master_reference_clock_time(), like I did in my rtai_rtdm_dc example.<u></u><u></u></p>
</div><div><p class="MsoNormal"><u></u> <u></u></p></div><div><p class="MsoNormal">That's it.<u></u><u></u></p></div><div><p class="MsoNormal"><u></u> <u></u></p></div><p class="MsoNormal">Regards,<br>Jun<u></u><u></u></p>
</div><div><p class="MsoNormal" style="margin-bottom:12.0pt"><u></u> <u></u></p><div><p class="MsoNormal">On Fri, Apr 4, 2014 at 3:17 AM, Gavin Lambert <<a href="mailto:gavinl@compacsort.com" target="_blank">gavinl@compacsort.com</a>> wrote:<u></u><u></u></p>
<div><div><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Hi Jun,</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Thanks; I’m having a look at it, but much of it is new to me. I’m using PREEMPT_RT so my code is based on the dc_user example, not the RTAI examples, and I’d probably have to try adapting it before I could test it.</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Regards,</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Gavin Lambert</span><u></u><u></u></p><p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0cm 0cm 0cm 4.0pt"><div><div style="border:none;border-top:solid #b5c4df 1.0pt;padding:3.0pt 0cm 0cm 0cm"><p class="MsoNormal"><b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span lang="EN-US" style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Jun Yuan [mailto:<a href="mailto:j.yuan@rtleaders.com" target="_blank">j.yuan@rtleaders.com</a>] <br>
<b>Sent:</b> Friday, 4 April 2014 01:46<br><b>To:</b> Gavin Lambert<br><b>Cc:</b> <a href="mailto:etherlab-users@etherlab.org" target="_blank">etherlab-users@etherlab.org</a><br><b>Subject:</b> Re: [etherlab-users] DC-Synchronization - Sync signal generation</span><u></u><u></u></p>
</div></div><div><p class="MsoNormal"> <u></u><u></u></p><div><div><div><div><p class="MsoNormal" style="margin-bottom:12.0pt">Hi Gavin,<u></u><u></u></p></div><p class="MsoNormal" style="margin-bottom:12.0pt">your interest is my motivation. I have attached the bundle file. <u></u><u></u></p>
</div><p class="MsoNormal" style="margin-bottom:12.0pt">My changes is base on the newest Version 1.5.2 in 'stable-1.5' branch. I added a new 'rtleaders' branch first and did all my changes on that. So after "$ hg unbundle etherlab_1.5.2_jyuan.hg", don't forget to switch to the 'rtleaders' branch using "$ hg update rtleaders".<u></u><u></u></p>
</div><div><p class="MsoNormal" style="margin-bottom:12.0pt">I found a better way of synchronizing the master clock to ref slave clock. It is much faster and more stable. I managed to port my C++ code into C code in the rtai_rtdm_dc example today, but I cannot test if the new code compiles right now. If you have a rtai environment, please test it for me if it compiles, and give me some feedback. <u></u><u></u></p>
</div><div><p class="MsoNormal" style="margin-bottom:12.0pt">Besides that, there is a more accurate DC time offset calculation. There should be no more errors like "Slave did not sync after 5000ms". The accurate time offset estimation saves much time for the DC Sync procedure. Slaves would have such a small dc diff (several hundred ns maybe) at the beginning of the dc sync check, that I even changed EC_SYSTEM_TIME_TOLERANCE_NS from 1000000ns to 1000ns.<br>
<br>The postponed check of master->has_app_time makes the error "No app_time received up to now, but master already active" away.<u></u><u></u></p></div><div><p class="MsoNormal">And there are the bugfix for ecrt_master_select_reference_clock() from Graeme Foot, and some other bug fixes from Jeroen Van den Keybus.<u></u><u></u></p>
</div><div><p class="MsoNormal"> <u></u><u></u></p></div><div><p class="MsoNormal">Any feedback is welcome. Have fun testing those changes!<u></u><u></u></p></div><div><p class="MsoNormal"> <u></u><u></u></p></div><div><p class="MsoNormal">
Jun<u></u><u></u></p></div></div><div><p class="MsoNormal" style="margin-bottom:12.0pt"> <u></u><u></u></p><div><p class="MsoNormal">On Thu, Apr 3, 2014 at 12:13 AM, Gavin Lambert <<a href="mailto:gavinl@compacsort.com" target="_blank">gavinl@compacsort.com</a>> wrote:<u></u><u></u></p>
<p class="MsoNormal">On 2 April 2014 22:40, quoth Jun Yuan:<u></u><u></u></p><div><p class="MsoNormal" style="margin-bottom:12.0pt">> But there is a reason why we all put the ecrt_master_application_time() outside<br>> the loop. Because we all got burned by the error "No app_time received up to<br>
> now, but master already active.", which is a timing bug in Etherlab. I've<br>> resolved the problem by change the code of Etherlabmaster, which get rid of<br>> the "No app_time" bug. Now I don't need to call ecrt_master_application_time()<br>
> outside the loop any more. I will publish the bundle to the mailing list when<br>> I have time.<u></u><u></u></p></div><p class="MsoNormal" style="margin-bottom:12.0pt;margin-left:5.25pt">I'd be very interested to see this. Slave sync timing, "no app time", and the 5000ms sync timeout have been a recurring bugbear for me.<u></u><u></u></p>
</div></div></div></div></div></div></div><p class="MsoNormal"><br><br clear="all"><br>-- <br>Jun Yuan<br>[Aussprache: Djün Üän]<br><br>Robotics Technology Leaders GmbH<br>Am Loferfeld 58, D-81249 München<br>Tel: <a href="tel:%2B49%2089%20189%200465%2024" value="+4989189046524" target="_blank">+49 89 189 0465 24</a><br>
Fax: <a href="tel:%2B49%2089%20189%200465%2011" value="+4989189046511" target="_blank">+49 89 189 0465 11</a><br>mailto: <a href="mailto:j.yuan@rtleaders.com" target="_blank">j.yuan@rtleaders.com</a><br><br>Umlautregel in der chinesischen Lautschrift Pinyin: Nach den Anlauten y, j, q, und x wird u als ü ausgesprochen, z.B. yu => ü, ju => dschü, qu => tschü, xu => schü. <u></u><u></u></p>
</div></div></div></div></div>