Server Administration :: Setting Up ASM Using ASMLib On Native Linux Multipath Mapper?

I've been trying to setup ASM disks for two node cluster using ASMLib. It's a multipath environment

Quote:multipath -ll
mpath2 (36842b2b000528f8d0000029b4ccead44) dm-1 DELL,MD32xx
[size=280M][features=3 queue_if_no_path pg_init_retries 50][hwhandler=1 rdac][rw]
\_ round-robin 0 [prio=100][active]
\_ 1:0:0:1 sdc 8:32 [active][ready]
\_ round-robin 0 [prio=0][enabled]
\_ 1:0:1:1 sdl 8:176 [active][ghost]

[code]....

I've been following Metalink note 602952.1, 309815.1 and 605828.1

metalink note 602952.1 states Quote:Note: Keep in mind for ASMLIB disks you will need to use the /dev/dm-* disks instead of the /dev/mapper/mpath* devices since they are present in the partition table ==)> /proc/partitions and ASMLIB will look at that file during the disk discovering phase. If you do not use the disks located at /dev/dm-* then ASM will not be able to discover them. and to create ASM disks with Quote:/etc/init.d/oracleasm createdisk DISK1 /dev/dm-<disk><partition> Also further stating Quote:NOTE: When scanning, only the device names known by the kernel are scanned. With device-mapper, the kernel sees the devices as /dev/dm-XX. The /dev/mapper/XXX names are created by udev for human readability. Any configuration of ORACLEASM_SCANORDER or ORACLEASM_SCANEXCLUDE must use the dm prefix.My environment is RHEL5 and towards the end of the note it says Quote:Note: On release RH 5 the /dev/dm-N devices are not persistent, starting, these devices are no longer created by udev. So release RedHat 5 you can create the ASMLIB disks on the /dev/mpath/<partition> or /dev/mapper/<partition>

Example:

# /etc/init.d/oracleasm createdisk VOL1 /dev/mpath/sata01p1

Or

# /etc/init.d/oracleasm createdisk VOL1 /dev/mapper/mpath2p1

Therefore I created a ASM disk using /dev/mapper path and it is only visible on the node it was created. Even after scandisks on other node it is still not visible when listdisks/oracleasm-discover

I've also tried creating the asm disks with dm-12 option and the same, disk get created but not visible on the other node.

Moreover if I restart the oracleasm restart the disk disappears from the node that it was visible Quote:

[root@db2 ~]# /etc/init.d/oracleasm createdisk vol1 /dev/mapper/mpath6p1
Marking disk "vol1" as an ASM disk: [ OK ]
[root@db2 ~]# ls -l /dev/oracleasm/disks/*
brw-rw---- 1 oracle dba 253, 12 Nov 8 10:22 /dev/oracleasm/disks/VOL1
[root@db2 ~]# /etc/init.d/oracleasm scandisks
Scanning the system for Oracle ASMLib disks: [ OK ]

[code]....

But disk lable is there and could be seen from both nodes on the same partition Quote:

head /dev/mapper/mpath6p1
VOL1ORCLDISKVOL1

My suspicion is on multipath configuration. I've tried both (not at the same time) combinations given below.

Quote:ORACLEASM_SCANORDER="dm"
ORACLEASM_SCANEXCLUDE="mpath mapper sd sda sdb"

and

Quote:ORACLEASM_SCANORDER="mapper"
ORACLEASM_SCANEXCLUDE="mpath dm sd sda sdb"

But still the problem is there, (disks disappears when asm is restarted and never visible on the "other" node).