[etherlab-users] Kernel Panic in EoE context
Erwin Burgstaller
ethercat.berknapp at spamgourmet.com
Fri Feb 27 13:05:15 CET 2009
Hi,
we have some problems with kernel panics, in context with the EoE part
of the ethercat master.
We're running version 1.4 with kernel version 2.6.22 smp-enabled on a
dual core machine. We have e1000 nics and some EoE enabled slaves.
I'll attach the output from syslog, captured on a serial console, for
both types of panic.
One case, that's were the back traces of calls ends at e1000_intr,
occurs when the cable for the EtherCAT connection is pulled out of the
slave and is then connected again.
In the other case the realtime client module has been unloaded and then
loaded again.
Any ideas for what we could do, to track down the problem?
Any help would be appreciated,
Erwin
--
Erwin Burgstaller
-------------- next part --------------
[ 906.321421] KEC::kecrtai: Domain0: State 2.
[ 906.328251] EtherCAT: eoe0s0 stopped.
[ 906.386080] EtherCAT: Scanning bus.
[ 906.409061] EtherCAT WARNING: 20 datagrams UNMATCHED!
[ 906.453031] KEC::kecrtai: 1 slave(s) responding.
[ 906.457839] KEC::kecrtai: Link is up.
[ 906.461646] KEC::kecrtai: Slave: 000, State UNKNOWN.
[ 906.466797] KEC::kecrtai: Slave: 000, offline.
[ 906.471396] KEC::kecrtai: Slave: 000, Not operational.
[ 906.476729] KEC::serialDevHandler: 0, Doing reset of device data
[ 907.261407] EtherCAT: Bus scanning completed in 916 ms.
[ 907.266816] EtherCAT: Starting EoE processing.
[ 907.453162] KEC::kecrtai: Slave: 000, State OP.
[ 907.457889] KEC::kecrtai: Slave: 000, online.
[ 907.462442] KEC::kecrtai: Slave: 000, operational.
[ 910.472098] EtherCAT: eoe0s0 opened.
[ 921.088356] BUG: unable to handle kernel paging request
[ 921.088360] LXRT CHANGED MODE (TRAP), PID = 2547, VEC = 14, SIGNO = 11.
[ 921.088370] BUG: unable to handle kernel paging request at virtual address 000060c1
[ 921.088374] printing eip:
[ 921.088375] f4b27d82
[ 921.088377] *pde = 00000000
[ 921.088381] Oops: 0002 [#1]
[ 921.088382] PREEMPT SMP
[ 921.088385] Modules linked in: kecrtai_mod(P) ec_e1000 ec_master i915 start_rt_timer_mod(P) rtai_shm rtai_sem rtai_fifos rtai_lxrt rtai_hal coretemp w83627hf hwmon_vid pcnet32 ata_piix usbhid i2c_i801 libata
[ 921.088404] CPU: 0
[ 921.088405] EIP: 0060:[<f4b27d82>] Tainted: P VLI
[ 921.088407] EFLAGS: 00010297 (2.6.22-rtai-smp-1.11 #7)
[ 921.088426] EIP is at ec_master_send_datagrams+0x205/0x234 [ec_master]
[ 921.088430] eax: 000060c1 ebx: c1905e44 ecx: c1905e34 edx: c1905e44
[ 921.088433] esi: c1905e34 edi: 00024c87 ebp: ef807000 esp: ef0e5f34
[ 921.088436] ds: 007b es: 007b fs: 00d8 gs: 0033 ss: 0068
[ 921.088439] Process U:HARD:0:6 (pid: 2547, ti=ef0e4000 task=eff31a70 task.ti=ef0e4000)<0>
[ 921.088442] I-pipe domain Linux
[ 921.088443] Stack: 00000142 f08c1ea5 e76bf82a ef0e5f5c ef807498 ef807084 e76bf81e e76bf826
[ 921.088452] 00000000 efe916c0 efe916d0 efe916d0 00000000 00000000 f4b61000 ef0e5fc0
[ 921.088459] f4b6f6ab f4b7bb80 f4b7198b f41e37e1 cb42be58 00000001 eff31b7c c17fba00
[ 921.088468] Call Trace:
[ 921.088470] [<f08c1ea5>] e1000_intr+0x9e/0x14a [ec_e1000]
[ 921.088483] [<f4b6f6ab>] run+0x21b/0x6b0 [kecrtai_mod]
[ 921.088490] [<f41e37e1>] rt_schedule+0x6aa/0xc08 [rtai_lxrt]
[ 921.088506] [<f41e493a>] kthread_fun+0x123/0x193 [rtai_lxrt]
[ 921.088520] [<f41e4817>] kthread_fun+0x0/0x193 [rtai_lxrt]
[ 921.088532] [<c01034db>] kernel_thread_helper+0x7/0x10
[ 921.088540] =======================
[ 921.088541] Code: 00 44 3f c0 8b 4c 24 28 8d 44 24 28 83 e9 10 8b 71 10 89 44 24 0c 83 ee 10 eb 23 8b 51 10 8b 41 14 c7 41 34 02 00 00 00 89 79 38 <89> 10 89 42 04 89 59 10 89 59 14 89 f1 8b 76 10 83 ee 10 8d 59
[ 921.088581] EIP: [<f4b27d82>] ec_master_send_datagrams+0x205/0x234 [ec_master] SS:ESP 0068:ef0e5f34
[ 921.288950] at virtual address 5ff5a04a
[ 921.293017] printing eip:
[ 921.295815] c1905e47
[ 921.298084] *pde = 00000000
[ 921.300985] Oops: 0002 [#2]
[ 921.303927] PREEMPT SMP
[ 921.306619] Modules linked in: kecrtai_mod(P) ec_e1000 ec_master i915 start_rt_timer_mod(P) rtai_shm rtai_sem rtai_fifos rtai_lxrt rtai_hal coretemp w83627hf hwmon_vid pcnet32 ata_piix usbhid i2c_i801 libata
[ 921.326138] CPU: 1
[ 921.326139] EIP: 0060:[<c1905e47>] Tainted: P VLI
[ 921.326141] EFLAGS: 00010286 (2.6.22-rtai-smp-1.11 #7)
[ 921.339980] EIP is at 0xc1905e47
[ 921.343332] eax: 00000000 ebx: ef8074e4 ecx: f4b7bb80 edx: f4b7bb80
[ 921.350389] esi: 80f4b6f2 edi: c1905e44 ebp: c1905e44 esp: c1905e51
[ 921.357417] ds: 007b es: 007b fs: 00d8 gs: 0000 ss: 0068
[ 921.363469] Process swapper (pid: 0, ti=c1904000 task=c18f7030 task.ti=c1904000)<0>
[ 921.370959] I-pipe domain Linux
[ 921.374507] Stack: e4f4b7bb e4ef8074 48ef8074 00f4b283 00ef8070 02c190e0 87000000 6000024c
[ 921.383365] 00c011d9 30c1806a a8c18f70 33c1905e 90c01138 10c1905e 10c190ea a4c190e8
[ 921.392214] a4e778bf 40e778bf 21c043f8 08000000 0ac03f43 d9000000 00c011aa 00c043c1
[ 921.401071] Call Trace:
[ 921.403835] =======================
[ 921.407537] Code: ef 80 bb b7 f4 80 bb b7 f4 f2 b6 f4 80 44 5e 90 c1 44 5e 90 c1 00 00 00 00 7b 00 e7 00 7b 00 00 e4 d8 00 ef 55 ff ff ff ff 47 5e <90> c1 60 00 00 00 86 02 01 00 bb b7 f4 e4 74 80 ef e4 74 80 ef
[ 921.428611] EIP: [<c1905e47>] 0xc1905e47 SS:ESP 0068:c1905e51
[ 921.434658] Kernel panic - not syncing: Fatal exception in interrupt
-------------- next part --------------
[ 2197.503965] EtherCAT: Requesting master 0...
[ 2197.508404] EtherCAT: Successfully requested master 0.
[ 2197.513928] EtherCAT: Domain0: Logical address 0x00000000, 48 byte, expected working counter 3.
[ 2197.522923] EtherCAT: Datagram domain0-0: Logical offset 0x00000000, 48 byte, type LRW.
[ 2197.531404] EtherCAT: Stopping EoE processing.
[ 2197.536037] EtherCAT: Master thread exited.
[ 2197.540383] EtherCAT: Starting EtherCAT-OP thread.
[ 2197.545370] EtherCAT: Starting EoE processing.
[ 2197.549967] KEC::kecrtai: Initialized.
[ 2197.553854] KEC::kecrtai: 1 slave(s) responding.
[ 2197.558644] KEC::kecrtai: Applicaton-Layer states: PREOP.
[ 2197.564236] KEC::kecrtai: Link is up.
[ 2197.568045] KEC::kecrtai: Slave: 000, State PREOP.
[ 2197.573016] KEC::kecrtai: Slave: 000, online.
[ 2197.595935] EtherCAT: Domain 0: Working counter changed to 2/3.
[ 2197.602057] KEC::kecrtai: Domain0: WC 2.
[ 2197.606107] KEC::kecrtai: Domain0: State 1.
[ 2197.616367] EtherCAT: Slave states: OP.
[ 2197.641921] KEC::kecrtai: Domain0: WC 3.
[ 2197.646003] KEC::kecrtai: Slave: 000, State OP.
[ 2197.650678] KEC::kecrtai: Slave: 000, operational.
[ 2197.655622] KEC::kecrtai: Domain0: State 2.
[ 2197.680316] Clocksource tsc unstable (delta = 68037327 ns)
[ 2197.685972] Time: jiffies clocksource has been installed.
[ 2198.551195] KEC::kecrtai: Applicaton-Layer states: OP.
[ 2198.624136] EtherCAT: Domain 0: Working counter changed to 3/3.
[ 2263.938044] udevd version 125 started
[ 2272.364705] KEC::kecrtai: Stopping...
[ 2272.368606] EtherCAT: Releasing master 0...
[ 2272.373043] EtherCAT: Stopping EoE processing.
[ 2272.377758] EtherCAT: Master thread exited.
[ 2272.382199] EtherCAT: Starting EtherCAT-IDLE thread.
[ 2272.387418] EtherCAT: 0 slave(s) responding.
[ 2272.391921] EtherCAT: Starting EoE processing.
[ 2272.396528] EtherCAT WARNING: Datagram e772cbc0 (master-fsm) was SKIPPED 1 time.
[ 2272.404211] EtherCAT: 1 slave(s) responding.
[ 2272.405232] EtherCAT: Released master 0.
[ 2272.405239] KEC::kecrtai: Unloading.
[ 2272.416375] EtherCAT: Stopping EoE processing.
[ 2272.469193] EtherCAT: Scanning bus.
[ 2272.735659] EtherCAT: Bus scanning completed in 320 ms.
[ 2272.741090] EtherCAT: Starting EoE processing.
[ 2272.751745] EtherCAT: eoe0s0 opened.
[ 2272.756801] EtherCAT: Slave states: PREOP.
[ 2272.766083] EtherCAT ERROR: Failed to set SAFEOP state, slave 0 refused state change (PREOP + ERROR).
[ 2272.775682] EtherCAT ERROR: Unknown AL status code 0x0000.
[ 2272.781439] EtherCAT: Acknowledged state PREOP on slave 0.
[ 2273.574942] KEC::kecrtai: --------------------------------------------------------------------------------
[ 2273.584976] KEC::kecrtai: Starting rrrr
[ 2273.588972] EtherCAT: Requesting master 0...
[ 2273.593392] EtherCAT: Successfully requested master 0.
[ 2273.598911] EtherCAT: Domain0: Logical address 0x00000000, 48 byte, expected working counter 3.
[ 2273.607914] EtherCAT: Datagram domain0-0: Logical offset 0x00000000, 48 byte, type LRW.
[ 2273.616376] EtherCAT: Stopping EoE processing.
[ 2273.621002] EtherCAT: Master thread exited.
[ 2273.625347] EtherCAT: Starting EtherCAT-OP thread.
[ 2273.630349] EtherCAT: Starting EoE processing.
[ 2273.634949] KEC::kecrtai: Initialized.
[ 2273.635976] KEC::kecrtai: 1 slave(s) responding.
[ 2273.635978] KEC::kecrtai: Applicaton-Layer states: PREOP.
[ 2273.635980] KEC::kecrtai: Link is up.
[ 2273.635982] KEC::kecrtai: Slave: 000, State PREOP.
[ 2273.635984] KEC::kecrtai: Slave: 000, online.
[ 2273.636146] EtherCAT WARNING: 1 datagram UNMATCHED!
[ 2273.658937] EtherCAT: Domain 0: Working counter changed to 2/3.
[ 2273.666933] KEC::kecrtai: Domain0: WC 3.
[ 2273.666935] KEC::kecrtai: Slave: 000, State OP.
[ 2273.666937] KEC::kecrtai: Slave: 000, operational.
[ 2273.666939] KEC::kecrtai: Domain0: State 2.
[ 2273.668121] EtherCAT: Slave states: OP.
[ 2274.636179] KEC::kecrtai: Applicaton-Layer states: OP.
[ 2274.660158] EtherCAT: Domain 0: Working counter changed to 3/3.
[ 2276.703506] EtherCAT ERROR: Reception of CoE SDO description response failed on slave 0: No response.
[ 2285.991109] BUG: unable to handle kernel paging request at virtual address 4fda504a
[ 2285.999086] printing eip:
[ 2286.001891] 44c1905e
[ 2286.004161] *pde = 00000000
[ 2286.007052] Oops: 0002 [#1]
[ 2286.010083] PREEMPT SMP
[ 2286.012810] Modules linked in: kecrtai_mod(P) ec_e1000 ec_master i915 start_rt_timer_mod(P) rtai_shm rtai_sem rtai_fifos rtai_lxrt rtai_hal coretemp w83627hf hwmon_vid pcnet32 usbhid i2c_i801 ata_piix libata
[ 2286.033238] CPU: 1
[ 2286.033239] EIP: 905e:[<44c1905e>] Tainted: P VLI
[ 2286.033241] EFLAGS: 00010286 (2.6.22-rtai-smp-1.11 #7)
[ 2286.047072] EIP is at 0x44c1905e
[ 2286.050413] eax: 00000000 ebx: e772cce4 ecx: f4b7bb80 edx: f4b7bb80
[ 2286.057444] esi: 80f4b6f2 edi: c1905e44 ebp: c1905e44 esp: c1905e51
[ 2286.064463] ds: 007b es: 007b fs: 00d8 gs: 0000 ss: 0068
[ 2286.070516] Process swapper (pid: 0, ti=c1904000 task=c18f7030 task.ti=c1904000)<0>
[ 2286.078012] I-pipe domain Linux
[ 2286.081572] Stack: e4f4b7bb e4e772cc 48e772cc 00f4b283 00e772c8 02c190e0 a6000000 60000781
[ 2286.090438] 00c011d9 30c1806a a8c18f70 33c1905e 90c01138 10c1905e 10c190ea 90c190e8
[ 2286.099313] 90c1905e 40c1905e 21c043f8 08000000 0ac03f43 d9000000 00c011aa 00c043c1
[ 2286.108152] Call Trace:
[ 2286.110901] =======================
[ 2286.114602] Code: Bad EIP value.
[ 2286.118048] EIP: [<44c1905e>] 0x44c1905e SS:ESP 0068:c1905e51
[ 2286.124251] Kernel panic - not syncing: Fatal exception in interrupt
More information about the Etherlab-users
mailing list