Dm-multipath-vscsi

From Linux on Power wiki

Jump to: navigation, search

Using dm-multipath (no success...)

  • 1. Check minor/major numbers:
op720-1-client1:~ # ls -al /dev/sdd
brw-rw----  1 root disk 8, 48 Jan 11 08:47 /dev/sdd
op720-1-client1:~ # ls -al /dev/sdc
brw-rw----  1 root disk 8, 32 Jan 11 08:47 /dev/sdc
  • 2. Check disk size
op720-1-client1:~ # dmesg |grep 512-byte
SCSI device sdc: 20971520 512-byte hdwr sectors (10737 MB)
SCSI device sdd: 20971520 512-byte hdwr sectors (10737 MB)


  • 3. Create response file for dmsetup
op720-1-client1:~ # cat dmtest.txt
0 20971520 multipath 0 0 2 1 round-robin 0 1 1 8:32 1000 round-robin 0 1 1 8:48 1000
  • 4. Setup dm-devices
op720-1-client1:~ # dmsetup create dmtest dmtest.txt
  • 5. Multipath
op720-1-client1:~ # multipath -ll
dm names   N
dm table dmtest  N
dm table dmtest  N
dm status dmtest  N
dm info dmtest  O
dmtest
[size=10 GB][features="0"][hwhandler="0"]
\_ round-robin 0 [active]
 \_ #:#:#:#     sdc 8:32 [active][faulty]
\_ round-robin 0 [enabled]
 \_ #:#:#:#     sdd 8:48 [active][faulty]

-> No idea how to get it ready instead of faulty

op720-1-client1:~ # dmsetup info
Name:              dmtest
State:             ACTIVE
Tables present:    LIVE
Open count:        0
Event number:      0
Major, minor:      253, 0
Number of targets: 1
op720-1-client1:~ # dmsetup table dmtest
0 20971520 multipath 0 0 2 1 round-robin 0 1 1 8:32 1000 round-robin 0 1 1 8:48 1000
  • 6. Trying to use dm-device
p720-1-client1:~ # fdisk -l /dev/dm-0 
Disk /dev/dm-0: 10.7 GB, 10737418240 bytes
255 heads, 63 sectors/track, 1305 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk /dev/dm-0 doesn't contain a valid partition table

Formatting and mounting it

op720-1-client1:~ # mkfs.ext3  /dev/dm-0
mke2fs 1.38 (30-Jun-2005)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
1310720 inodes, 2621440 blocks
131072 blocks (5.00%) reserved for the super user
First data block=0
80 block groups
32768 blocks per group, 32768 fragments per group
16384 inodes per group
Superblock backups stored on blocks:
       32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
This filesystem will be automatically checked every 35 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.
mount /dev/dm-0 /mnt
tar clvf /mnt/test.tar /


op720-1-client1:~ # df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda3              16G  1.5G   15G  10% /
tmpfs                 491M  8.0K  491M   1% /dev/shm
/dev/dm-0             9.9G  964M  8.5G  11% /mnt
  • 7. Failure test

If VIO1 is going down and /dev/sdc is not available: system crashes


  • XX. Remove dm-Device
op720-1-client1:~ # dmsetup remove dmtest

Using md-device (works)

  • 1. Create md-devise /dev/md0 with multipath level using /dev/sdd and /dev/sdc (given sdd and sdc are one and the same lun exported through two VIO servers)
op720-1-client1:~ # mdadm --create /dev/md0 -l multipath --raid-devices 2 /dev/sdd /dev/sdc
  • 2. Check status
op720-1-client1:~ # cat /proc/mdstat
Personalities : [multipath]
md0 : active multipath sdc[1] sdd[0]
     10485696 blocks [2/2] [UU]
unused devices: <none>

from /var/log/messages:

Jan 12 09:41:58 op720-1-client1 kernel: md: bind<sdd>
Jan 12 09:41:58 op720-1-client1 kernel: md: bind<sdc>
Jan 12 09:41:58 op720-1-client1 kernel: md: multipath personality registered as nr 7
Jan 12 09:41:58 op720-1-client1 kernel: multipath: array md0 active with 2 out of 2 IO paths


  • 3. Format and mount it
mkfs.ext3 /dev/md0
mount /dev/md0 /mnt



  • 4. Failover (shutting down VIO2 providing /dev/sdd)

op720-1-client1:~ # cat /proc/mdstat

Personalities : [multipath] 
md0 : active multipath sdc[1] sdd[2](F)
     10485696 blocks [2/1] [_U]      
unused devices: <none>
  • 5. Readding device
op720-1-client1:~ # mdadm -r /dev/md0 /dev/sdd
mdadm: hot removed /dev/sdd
op720-1-client1:~ # mdadm -a /dev/md0 /dev/sdd
mdadm: hot added /dev/sdd
op720-1-client1:~ # cat /proc/mdstat 
Personalities : [multipath] 
md0 : active multipath sdd[2] sdc[1]
     10485696 blocks [2/1] [_U]     
unused devices: <none>
Personal tools