月明:一次ASM存储高可用故障解决过程分析"/>
计算机中丢失oraocr,守得云开见月明:一次ASM存储高可用故障解决过程分析
2018-08-16T14:56:04.957199+08:00 -->差不多和DB同时检测到IO错误
WARNING: Write Failed. group:1 disk:1 AU:1 offset:4190208 size:4096
path:/dev/rdisk/MES1_p4_moddb_ocrvote
incarnation:0xf0996d00 asynchronous result:'I/O error'
subsys:System krq:0x9ffffffffd285710 bufp:0x9ffffffffd299000 osderr1:0x69b7 osderr2:0x0
IO elapsed time: 0 usec Time waited on I/O: 0 usec
……
2018-08-16T14:56:34.400892+08:00
WARNING: Disk 4 (MES1_REDO02) in group 2 mode 0x7f is now being offlined
……
ORA-27072: File I/O error
HPUX-ia64 Error: 11: Resource temporarily unavailable
Additional information: 4
Additional information: 8188
Additional information: 4294967295
GMON checking disk modes for group 2 at 2460 for pid 49, osid 13802
2018-08-16T14:56:35.056049+08:00
NOTE: checking PST for grp 2 done.
2018-08-16T14:56:35.056382+08:00
Errors in file /grid/crs_base/diag/asm/+asm/+ASM1/trace/+ASM1_gmon_18741.trc:
ORA-27072: File I/O error
HPUX-ia64 Error: 11: Resource temporarily unavailable
Additional information: 4
Additional information: 8188
Additional information: 4294967295
ORA-27072: File I/O error
HPUX-ia64 Error: 11: Resource temporarily unavailable
Additional information: 4
Additional information: 8188
Additional information: 4294967295
ORA-27072: File I/O error
HPUX-ia64 Error: 11: Resource temporarily unavailable
Additional information: 4
Additional information: 8188
Additional information: 4294967295
2018-08-16T14:56:35.056592+08:00
GMON checking disk modes for group 2 at 2461 for pid 19, osid 18728
NOTE: initiating PST update: grp 2 (DG_REDO_MOD), dsk = 4/0xf0996e2a, mask = 0x6a, op = clear mandatory
GMON checking disk modes for group 1 at 2462 for pid 57, osid 24043
2018-08-16T14:56:35.057277+08:00
NOTE: checking PST for grp 2 done.
NOTE: initiating PST update: grp 2 (DG_REDO_MOD), dsk = 4/0xf0996e2a, mask = 0x6a, op = clear mandatory
2018-08-16T14:56:35.059431+08:00
NOTE: checking PST for grp 1 done.
2018-08-16T14:56:35.059697+08:00
NOTE: sending set offline flag message (3413962691) to 1 disk(s) in group 1
2018-08-16T14:56:35.059796+08:00
GMON checking disk modes for group 4 at 2463 for pid 59, osid 24045
2018-08-16T14:56:36.205496+08:00
NOTE: Attempting voting file refresh on diskgroup DG_CRS_MOD
WARNING: Read Failed. group:1 disk:1 AU:0 offset:0 size:4096
path:/dev/rdisk/MES1_p4_moddb_ocrvote
incarnation:0xf0996d00 asynchronous result:'I/O error'
subsys:System krq:0x9ffffffffd4a4068 bufp:0x9ffffffffcf91000 osderr1:0x534b4950 osderr2:0x0
IO elapsed time: 0 usec Time waited on I/O: 0 usec
NOTE: Refresh completed on diskgroup DG_CRS_MOD. Found 3 voting file(s).
NOTE: Voting file relocation is required in diskgroup DG_CRS_MOD
NOTE: Attempting voting file relocation on diskgroup DG_CRS_MOD
WARNING: Read Failed. group:1 disk:1 AU:0 offset:0 size:4096
path:/dev/rdisk/MES1_p4_moddb_ocrvote
incarnation:0xf0996d00 asynchronous result:'I/O error'
subsys:System krq:0x9ffffffffd4a4068 bufp:0x9ffffffffcf84000 osderr1:0x534b4950 osderr2:0x0
IO elapsed time: 0 usec Time waited on I/O: 0 usec
NOTE: Successful voting file relocation on diskgroup DG_CRS_MOD
2018-08-16T14:56:52.061661+08:00
System state dump requested by (instance=1, osid=3906536), summary=[SYSTEMSTATE_GLOBAL: global system state dump request (kjdgdss_g)].
System State dumped to trace file /grid/crs_base/diag/asm/+asm/+ASM1/trace/+ASM1_diag_18708_20180816145652.trc
2018-08-16T14:56:55.250804+08:00
WARNING: Read Failed. group:2 disk:4 AU:0 offset:0 size:4096
2018-08-16T14:56:55.250889+08:00
WARNING: Read Failed. group:2 disk:4 AU:7 offset:245760 size:4096
2018-08-16T14:56:55.251279+08:00
path:/dev/rdisk/MES1_p4_moddb_redo02
2018-08-16T14:56:55.251590+08:00
incarnation:0xf0996e2a synchronous result:'I/O error'
subsys:System krq:0x9ffffffffd4b2f80 bufp:0x9ffffffffd407000 osderr1:0x69b7 osderr2:0x0
2018-08-16T14:56:55.251753+08:00
path:/dev/rdisk/MES1_p4_moddb_redo02
IO elapsed time: 0 usec Time waited on I/O: 0 usec
2018-08-16T14:56:55.252112+08:00
incarnation:0xf0996e2a synchronous result:'I/O error'
subsys:System krq:0x9ffffffffd225ec0 bufp:0x9ffffffffcc0f000 osderr1:0x69c0 osderr2:0x0
IO elapsed time: 0 usec Time waited on I/O: 0 usec
WARNING: cache failed reading from group=2(DG_REDO_MOD) fn=8 blk=60 count=1 from disk=4 (MES1_REDO02) mirror=0 kfkist=0x20 status=0x07 osderr=0x69c0 file=kfc.c line=12986
WARNING: cache succeeded reading from group=2(DG_REDO_MOD) fn=8 blk=60 count=1 from disk=3 (MES2_REDO01) mirror=1 kfkist=0x20 status=0x01 osderr=0x0 file=kfc.c line=13031
2018-08-16T14:56:55.274598+08:00
SUCCESS: extent 0 of file 8 group 2 - all online mirror sides found readable, no repair required ---> 所有online mirror side是完好的
更多推荐
计算机中丢失oraocr,守得云开见月明:一次ASM存储高可用故障解决过程分析
发布评论