From Linux on Power wiki

Contents 1 Configuring Software Raid and LVM on Power using SUSE SLES9 SP1 1.1 Summary 1.2 Prerequisites 1.3 Background 1.4 Partitioning Layout 1.5 Manual installation: step by step 1.5.1 Start installation 1.5.2 Partitioning 1.5.3 Create raid devices 1.5.4 Create LVM and logical volumes 1.5.5 Boot loader installation 1.5.6 Reboot the system in rescue mode 1.5.7 Postinstallation tasks 1.5.7.1 Fix LILO script 1.5.7.2 Mirror Boot partition 1.5.7.3 Set boot list 1.6 Using autoyast 1.6.1 Chroot-scripts 1.6.2 Partitioning through autoyast

Configuring Software Raid and LVM on Power using SUSE SLES9 SP1

Authors: Martin Welk, Tomas Baublys

Summary

This document describes how to implement software raid on SUSE SLES9 on Power. If your machine has a hardware raid controller or a hardware raid enablement card configured, you should better use hardware raid instead of md-Devices. You can configure hardware raid before installing the operating system using diag tool from the latest Diagnostics CD1. Still there are cases, where hardware raid is not applicable: for example for a scenario using two VIO servers serving each one disk to the client LPARs and using software mirroring on the client side. If you follow those instructions and see how long it takes to install a single system, you may decide to use autoyast next time. Using autoyast is our first recomendation, but if you want to know how it works, you may install it manualy first.

Prerequisites

We are using SUSE LINUX Enterprise Server (SLES) version 9 with Service Pack 1. Use a network installation source with SP1 added (you can use yast – installation server to add additional media and provide SP1 iso images) or boot from SP1 CD.2 We did not test the same installation using base level CDs, and we are sure it will not work the same way. These instructions may become obsolete with SUSE SLES9 SP2 (at least we hope...)

Background

Using software RAID on POWER is different from using it on Intel due to the different boot loader. Power uses the Yaboot3 boot loader which is loaded by Open Firmware4 after identifying the PReP boot partition on the disk. Only disks containing the PReP boot partition can be selected in SMS (user interface to the Open Firmware). Yaboot starts and scans all the primary disk partitions searching for its own configuration file (usually /etc/yaboot.conf) and the kernel image to load. Unfortunately Yaboot can not scan software mirrored md-devices or LVM logical volumes. It also contains a build in knowledge of only some file systems: ext2 , ext3, reiserfs and fat. The main trick we are using for software mirror is to make the PReP boot partition big enough to contain the configuration file and the kernel together with the yaboot boot loader. This partition will be formatted in FAT16 instead of leaving it unformatted. There is one more restriction: the root file system can not reside on the logical volume, to overcome this we will use two md-devices: one for the root filesystem and one for LVM and all other logical volumes.

Partitioning Layout

Manual installation: step by step

Start installation

Start your installation from CD or network, if you use remote connection or a machine without graphics adapter provide “vnc=1 vncpassword=abc123” in order to have a graphical interface to the yast during the installation or use “usessh=1 sshpassword=abc123” for remote text based installation. If your system is not HMC (Hardware Management Console) attatched you may need to add “console=hvsi0” as a boot argument. If you already have one partition installed with SUSE SLES9 you may also use mkzimage_cmdline tool to hardcode your boot arguments in the network boot image.

For manual partitioning we will need to do a lot of changes, YaST2 is hard to navigate in text mode using virtual terminal or serial console. Using VNC you will only need to make some basic language, network and installation settings in text mode and will connect to your installation through vncviewer from your workstation and enjoy Yast2 in graphical mode.

Partitioning

Start installation, select partioning, expert partitioner. Create following partitions on your first disk:

Name	Type	Format	Size	Purpose
/dev/sda1	PrePBoot	No	24 MB	Boot partition
/dev/sda2	LinuxRaid	No	512 MB	Raid device for / will be included in md0
/dev/sda3	LinuxRaid	No	15GB	Raid device for LVM will be included in md1

Create almost the same partitions on the second disk, the only difference is, that the second partition should not have a PrePBoot partition.

Name	Type	Format	Size	Purpose
/dev/sdb1	Linux	No	24 MB	Boot partition
/dev/sdb2	LinuxRaid	No	512 MB	Raid device for / will be included in md0
/dev/sdb3	LinuxRaid	No	15GB	Raid device for LVM will be included in md1

Create raid devices

After the base partitioning is done, click on RAID button and create raid devices, select RAID1 for mirroring. Create /dev/md0 using /dev/sda2 and /dev/sdb2 and afterwards create /dev/md1 using /dev/sda3 and /dev/sdb3.

Name	Include	Format	Mount point	Purpose
/dev/md0	/dev/sda2, /dev/sdb2	ext3 or reiserfs	/	Root file system
/dev/md1	/dev/sda3, /dev/sdb3	No	No	LVM

Create LVM and logical volumes

After creating raid devices, press the LVM button and create a volume group (default: system). Add /dev/md1 as a physical volume. Once the volume group is created, create logical volumes. You can follow the example below or create your own plan. Remember to create /usr large enough and also not to put /bin , /sbin, /etc on logical volumes as those need to be on the same file system with /.

Name	Format	Size	Mount point
userlv	ext3, jfs or reiserfs	1.5 GB	/usr
optlv	ext3, jfs or reiserfs	1.5 GB	/opt
varlv	ext3, jfs or reiserfs	512 MB	/var
homelv	ext3, jfs or reiserfs	512 MB	/home
tmplv	ext3, jfs or reiserfs	512 MB	/tmp
swaplv	Swap	512 MB

Boot loader installation

Depending on what file system you are using for /dev/sdb2 yast may be get confused and bootloader installation fails, saying you need to provide boot="" argument. Just press the button to edit the yaboot configuration and add the line:

boot="/dev/sda1"

to make sure the yaboot loader is going to be installed on the first disk.

Reboot the system in rescue mode

After the first stage of installation is finished system is going to reboot. At this moment the boot list is changed already and without any action it will try to boot first disk. This reboot will fail, producing an error

“boot          : unable to parse”

This means the installation recognized that root is on an md-device, lilo script changed the PrepBoot partition file system type to fat, but there is an incorrect line in the yaboot.cnf on the /dev/sda1. So now we need to boot from CD or network again and start a rescue system. After linux is loaded we mount the /dev/sda1 (mount /dev/sda1 /mnt) and correct the error in the configuration file. Once the line “partition=” is deleted you can reboot the system from the hard disk and finish the installation.

After the rescue system is booted, mount the /dev/sda1 partition and edit the /mnt/yaboot.cnf (remove the “partition=” line)

# mount /dev/sda1 /mnt
# vi /mnt/yaboot.cnf
delete the line "partition="

After reboot the system will be able to boot from disk and finish the second stage of the installation.

Postinstallation tasks

Fix LILO script

We needed to use a rescue boot in order to delete “partition=” line in the yaboot.cnf file. This line is produced by the /sbin/lilo script used on SLES9 GA and SP1. In order to be able to run lilo in the future we need to correct one line:

Change the line 397 in /sbin/lilo from

echo "partition = ${yaboot_part##*:}" 

to:

test -z "${yaboot_part##*:}" || echo "partition = ${yaboot_part##*:}" 

Mirror Boot partition

Mirror boot partition, fix partition type and activate boot partition on 2nd disk:

# dd if=/dev/sda1 of=/dev/sdb1
49120+0 records in
49120+0 records out

# Set partition 1 on disk sdb to FAT16
parted /dev/sdb set 1 type 6
parted /dev/sdb set 1 boot on

Set boot list

We will need to set the boot list pointing to the yaboot files in the /dev/sda1 and /dev/sb1 partitions. The normal setting would only point to the partition itself, not to the file. This would not work in our case, because the partitions /dev/sda1 and /dev/sdb1 have a type of FAT16 and not PReP boot.

First we need to get the Open Firmware path for our disks:

# show_of_path.sh /dev/sda
/vdevice/v-scsi@30000003/disk@8000000000000000
# show_of_path.sh /dev/sdb
/vdevice/v-scsi@30000004/disk@8000000000000000

In the following command we add :1,yaboot as a pointer to the file and set both devices to our boot list in the normal mode:

# bootlist -m normal /vdevice/v-scsi@30000003/disk@8000000000000000:1,yaboot
/vdevice/v-scsi@30000004/disk@8000000000000000:1,yaboot

Using autoyast

The process described above can be fully automated using network installation and autoyast. In this case we will not need to do any rescue boot to correct anything, because anything is done by the scripts included in the autoyast.xml file.

Chroot-scripts

Here is an example autoyast file automating the painful process described before. The first section is the script for all the postinstall tasks: removing the “partition=” line in yaboot.cnf, fixing lilo script, cloning the boot partition and setting the boot list. This script is running before the system is going for the first reboot.

  <scripts>
     <chroot-scripts config:type="list">
       <script>
         <chrooted config:type="boolean">true</chrooted>
         <filename>fix-yaboot-cnf.sh</filename>
         <interpreter>shell</interpreter>
         <source><![CDATA[#!/bin/sh
YABOOT_PART=/dev/sda1
YABOOT_CNF=yaboot.cnf
YABOOT_CNF_BAK=yaboot.cnf.1
MNT=/yaboot_fs
[ -e ${MNT} ] || mkdir ${MNT}
[ -d ${MNT} ] || exit 1
# Fix /etc/lilo.conf content
LILOBOOTLINE=`egrep 'boot = ""' /etc/lilo.conf`
if [ $? = 0 ]
then
 cp -p /etc/lilo.conf /etc/lilo.conf.old
 cat /etc/lilo.conf.old | \
   sed -e s#"$LILOBOOTLINE"#"boot = /dev/sda1"# > /etc/lilo.conf
fi
unset LILOBOOTLINE
# Fix Yaboot configuration
mount ${YABOOT_PART} ${MNT} || exit 1
mv ${MNT}/${YABOOT_CNF}  ${MNT}/${YABOOT_CNF_BAK} || exit 1
egrep -v "^partition.*" ${MNT}/${YABOOT_CNF_BAK} > ${MNT}/${YABOOT_CNF} || exit 1
sync
umount ${MNT} || exit 1
rmdir ${MNT}
# Mirror boot partition, fix partition type and activate boot partition on 2nd disk
dd if=/dev/sda1 of=/dev/sdb
# Set partition 1 on disk sdb to FAT16
parted /dev/sdb set 1 type 6
parted /dev/sdb set 1 boot on
# Set system's boot list to both disks
BOOTDISKS="/dev/sda /dev/sdb"
BOOTOFPATHS=""
for DISK in $BOOTDISKS
do
 DISKOFPATH=`/bin/show_of_path.sh $DISK`
   if [ $? != 0 -o -z "$DISKOFPATH" ]
   then
     continue
   fi
 BOOTOFPATH="$BOOTOFPATH $DISKOFPATH:1,yaboot"
done
if [ -n "$BOOTOFPATH" ]
then
 bootlist -m normal -o $BOOTOFPATH
fi
unset BOOTOFPATH DISKOFPATH
exit 0
]]></source>
       </script>
  <scripts>

Partitioning through autoyast

In this section we create dos partitions, raid devices, LVM on the second raid device and logical volumes in it:

<partitioning config:type="list">
     <drive>
       <device>/dev/sda</device>
       <partitions config:type="list">
         <partition>
           <format config:type="boolean">false</format>
           <partition_id config:type="integer">65</partition_id>
           <partition_nr config:type="integer">1</partition_nr>
           <partition_type>primary</partition_type>
           <size>24M</size>
         </partition>
         <partition>
           <format config:type="boolean">false</format>
           <partition_id config:type="integer">253</partition_id>
           <partition_nr config:type="integer">2</partition_nr>
           <raid_name>/dev/md0</raid_name>
           <raid_type>raid</raid_type>
           <size>512M</size>
         </partition>
         <partition>
           <format config:type="boolean">false</format>
           <partition_id config:type="integer">253</partition_id>
           <partition_nr config:type="integer">3</partition_nr>
           <raid_name>/dev/md1</raid_name>
           <raid_type>raid</raid_type>
           <size>max</size>
         </partition>
       </partitions>
       <use>all</use>
     </drive>
     <drive>
       <device>/dev/sdb</device>
       <partitions config:type="list">
         <partition>
           <format config:type="boolean">false</format>
           <partition_id config:type="integer">131</partition_id>
           <partition_nr config:type="integer">1</partition_nr>
           <partition_type>primary</partition_type>
           <size>24M</size>
         </partition>
         <partition>
           <format config:type="boolean">false</format>
           <partition_id config:type="integer">253</partition_id>
           <partition_nr config:type="integer">2</partition_nr>
           <raid_name>/dev/md0</raid_name>
           <raid_type>raid</raid_type>
           <size>512M</size>
         </partition>
         <partition>
           <format config:type="boolean">false</format>
           <partition_id config:type="integer">253</partition_id>
           <partition_nr config:type="integer">3</partition_nr>
           <raid_name>/dev/md1</raid_name>
           <raid_type>raid</raid_type>
           <size>max</size>
         </partition>
       </partitions>
       <use>all</use>
     </drive>
     <drive>
       <device>/dev/md</device>
       <partitions config:type="list">
         <partition>
           <filesystem config:type="symbol">jfs</filesystem>
           <format config:type="boolean">true</format>
           <mount>/</mount>
           <partition_id config:type="integer">131</partition_id>
           <partition_nr config:type="integer">0</partition_nr>
           <raid_options>
             <persistent_superblock config:type="boolean">true</persistent_superblock>
             <raid_type>raid1</raid_type>
           </raid_options>
           <region config:type="list">
             <region_entry config:type="integer">0</region_entry>
             <region_entry config:type="integer">517</region_entry>
           </region>
           <size>max</size>
         </partition>
         <partition>
           <lvm_group>systemvg</lvm_group>
           <partition_id config:type="integer">142</partition_id>
           <partition_nr config:type="integer">1</partition_nr>
           <raid_options>
             <persistent_superblock config:type="boolean">true</persistent_superblock>
             <raid_type>raid1</raid_type>
           </raid_options>
           <region config:type="list">
             <region_entry config:type="integer">0</region_entry>
             <region_entry config:type="integer">69444</region_entry>
           </region>
           <size>max</size>
         </partition>
       </partitions>
       <use>all</use>
     </drive>
     <drive>
       <device>/dev/systemvg</device>
       <is_lvm_vg config:type="boolean">true</is_lvm_vg>
       <lvm2 config:type="boolean">true</lvm2>
       <partitions config:type="list">
         <partition>
           <filesystem config:type="symbol">jfs</filesystem>
           <format config:type="boolean">true</format>
           <lv_name>usrlv</lv_name>
           <mount>/usr</mount>
           <partition_id config:type="integer">142</partition_id>
           <size>1536MB</size>
         </partition>
         <partition>
           <filesystem config:type="symbol">jfs</filesystem>
           <format config:type="boolean">true</format>
           <lv_name>optlv</lv_name>
           <mount>/opt</mount>
           <partition_id config:type="integer">142</partition_id>
           <size>1536MB</size>
         </partition>
         <partition>
           <filesystem config:type="symbol">jfs</filesystem>
           <format config:type="boolean">true</format>
           <lv_name>varlv</lv_name>
           <mount>/var</mount>
           <partition_id config:type="integer">142</partition_id>
           <size>512MB</size>
         </partition>
         <partition>
           <filesystem config:type="symbol">jfs</filesystem>
           <format config:type="boolean">true</format>
           <lv_name>tmplv</lv_name>
           <mount>/tmp</mount>
           <partition_id config:type="integer">142</partition_id>
           <size>512MB</size>
         </partition>
         <partition>
           <filesystem config:type="symbol">jfs</filesystem>
           <format config:type="boolean">true</format>
           <lv_name>homelv</lv_name>
           <mount>/home</mount>
           <partition_id config:type="integer">142</partition_id>
           <size>512MB</size>
         </partition>
         <partition>
           <filesystem config:type="symbol">swap</filesystem>
           <format config:type="boolean">true</format>
           <lv_name>swaplv</lv_name>
           <mount>swap</mount>
           <partition_id config:type="integer">142</partition_id>
           <size>512MB</size>
         </partition>
       </partitions>
       <pesize>32M</pesize>
       <use>all</use>
     </drive>
   </partitioning>