Be sure the following prerequisites are met and information gathered before proceeding. Values selected and used for this example are shown in brackets.
If you are adding an additional iSCSI LUN to an RHEL server already mounting data via iSCSI, skip directly to the "????" section.
The configuration for eth0 must be altered to ensure eth0 correctly binds to "Network Adapter 1":
# Gigabit Ethernet Controller (Copper) [root@kratos iscsi]# cat /etc/sysconfig/network-scripts/ifcfg-eth0 DEVICE="eth0" NM_CONTROLLED="yes" ONBOOT=yes TYPE=Ethernet BOOTPROTO=none IPADDR=142.103.236.15 PREFIX=24 GATEWAY=142.103.236.254 DNS1=142.103.236.1 DEFROUTE=yes IPV4_FAILURE_FATAL=yes IPV6INIT=no NAME="System eth0" UUID=5fb06bd0-0bb0-7ffb-45f1-d6edd65f3e03 HWADDR=E4:1F:13:B9:DD:C8
Eth1 is configured to communicate with the vFiler:
# Ethernet Controller [root@kratos iscsi]# cat /etc/sysconfig/network-scripts/ifcfg-eth1 DEVICE="eth1" NM_CONTROLLED="yes" ONBOOT=yes TYPE=Ethernet BOOTPROTO=none IPADDR=10.20.14.15 PREFIX=24 GATEWAY=10.20.14.254 DEFROUTE=yes IPV4_FAILURE_FATAL=yes IPV6INIT=no NAME="System eth1" UUID=9c92fad9-6ecb-3e6c-eb4d-8a47c6f50c04 HWADDR=E4:1F:13:B9:DD:CA
Add an entry for the vFiler to /etc/hosts:
[root@kratos iscsi]# cat /etc/hosts 127.0.0.1 localhost.localdomain localhost ::1 kratos.phas.ubc.ca kratos localhost6.localdomain6 localhost6 10.91.134.136 PHAS_NASpool
RHEL5 systems use the Open-iSCSI daemon - this section describes how to configure and manage it.
NOTE: The iSCSI drivers were installed on all RHEL 5 images on Nov 16th, 2009.
# yum install iscsi-initiator-utils
Update initiator name to "iqn.1994-05.com.redhat:<fqdn>" where <fqdn> is replaced with the fully qualified domain name of the host (dbprod.arts.ubc.ca in this example).
# cd /etc/iscsi # cp -p initiatorname.iscsi initiatorname.iscsi.ORIG # echo "InitiatorName=iqn.1994-05.com.redhat:kratos.phas.ubc.ca" > initiatorname.iscsi [root@kratos iscsi]# cat initiatorname.iscsi InitiatorName=iqn.1994-05.com.redhat:kratos.phas.ubc.ca
Configure CHAP authentication, CHAP Credentials, and login timeouts:
# cp -p iscsid.conf iscsid.conf.ORIG # vi iscsid.conf <-- Uncomment following lines and set to shown values node.session.auth.authmethod = CHAP node.session.auth.username = phasiscsi node.session.auth.password = pHas_iscsi01 discovery.sendtargets.auth.authmethod = CHAP discovery.sendtargets.auth.username = phasiscsi discovery.sendtargets.auth.password = pHas_iscsi01 node.session.timeo.replacement_timeout = 5
Enable the iSCSI and netfs services at boot. Start the iSCSI service.
# chkconfig iscsi on # chkconfig iscsid on # chkconfig netfs on # service iscsid start Turning off network shutdown. [ OK ] Starting iSCSI daemon: [ OK ]
Discover targets:
[root@kratos iscsi]# iscsiadm --mode discovery --type sendtargets --portal 10.91.134.136 10.91.134.136:3260,2019 iqn.1992-08.com.netapp:sn.151709189:vf.893008c2-8c80-11e0-b2b4-00a098113b7a
If more than one target is returned, you must delete the unwanted targets (all except the 10.91.134.136 interface in this example).
Use the "delete" operation to delete all the unwanted targets. Verify only the desired target(s) remain.
# iscsiadm -m node --portal 172.20.24.227 --op delete [root@kratos iscsi]# iscsiadm --m node 10.91.134.136:3260,2019 iqn.1992-08.com.netapp:sn.151709189:vf.893008c2-8c80-11e0-b2b4-00a098113b7a
Now log into the target:
[root@kratos iscsi]# iscsiadm --m node -l Logging in to [iface: default, target: iqn.1992-08.com.netapp:sn.151709189:vf.893008c2-8c80-11e0-b2b4-00a098113b7a, portal: 10.91.134.136,3260] iscsiadm: Could not login to [iface: default, target: iqn.1992-08.com.netapp:sn.151709189:vf.893008c2-8c80-11e0-b2b4-00a098113b7a, portal: 10.91.134.136,3260]. iscsiadm: initiator reported error (15 - session exists) iscsiadm: Could not log into all portals
Verify that the LUNs on the target were successfully connected.
[root@kratos iscsi]# iscsiadm -m session -P 3 iSCSI Transport Class version 2.0-870 version 2.0-872 Target: iqn.1992-08.com.netapp:sn.151709189:vf.893008c2-8c80-11e0-b2b4-00a098113b7a Current Portal: 10.91.134.136:3260,2019 Persistent Portal: 10.91.134.136:3260,2019 ********** Interface: ********** Iface Name: default Iface Transport: tcp Iface Initiatorname: iqn.1994-05.com.redhat:kratos.phas.ubc.ca Iface IPaddress: 10.20.14.15 Iface HWaddress:Iface Netdev: SID: 1 iSCSI Connection State: LOGGED IN iSCSI Session State: LOGGED_IN Internal iscsid Session State: NO CHANGE ************************ Negotiated iSCSI params: ************************ HeaderDigest: None DataDigest: None MaxRecvDataSegmentLength: 262144 MaxXmitDataSegmentLength: 65536 FirstBurstLength: 65536 MaxBurstLength: 65536 ImmediateData: Yes InitialR2T: No MaxOutstandingR2T: 1 ************************ Attached SCSI devices: ************************ Host Number: 11 State: running scsi11 Channel 00 Id 0 Lun: 0 Attached scsi disk sdb State: running scsi11 Channel 00 Id 0 Lun: 1 Attached scsi disk sdc State: running
The above output shows that the LUN mapped to ID 0 is connected as /dev/sdb, and the LUN mapped to ID 1 is connected as /dev/sdc
Configure device mapper so a lun is always given a consistent and meaningful device name based on it's wwid. Once this step is complete the lun will always be assigned a consistent name under /dev/mapper, regardless of which LUN ID it is mapped to or which "/dev/sd?" device it is assgigned to.
Ensure "device-mapper-multipath" package is installed:
On RHEL 5 - # yum install device-mapper-multipath ON RHEL 4 - # up2date --install device-mapper-multipath
Retrieve the "wwid" of each lun:
[root@kratos iscsi]# /lib/udev/scsi_id --page=0x83 --whitelisted --device=/dev/sdb 360a98000572d4534694a646149766c41 [root@kratos iscsi]# /lib/udev/scsi_id --page=0x83 --whitelisted --device=/dev/sdc 360a98000572d4534694a646149767970
Save a copy of /etc/multipath.conf, then update it and assign a device mapper alias to each of the above wwid's
# cd /etc
# cp -p multipath.conf multipath.conf.ORIG
# vi multipath.conf
# Replace the entire contents with the following configuration:
defaults {
user_friendly_names yes
}
blacklist {
devnode "^sda$"
devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
devnode "^hd[a-z]"
devnode "^dcssblk[0-9]*"
}
##
## name : devices, list of per storage controller settings
##
devices {
device {
vendor "NETAPP"
product "LUN.*"
getuid_callout "/lib/udev/scsi_id --whitelisted --device=/dev/%n"
features "1 queue_if_no_path"
hardware_handler "0"
path_selector "round-robin 0"
path_grouping_policy multibus
failback immediate
rr_weight uniform
rr_min_io 128
path_checker directio
prio ontap
}
}
multipaths {
#
# name : multipath
# scope : multipath & multipathd
# desc : container for settings that apply to one specific multipath
#
# map /vol/... to /dev/mapper/VG01-PV01
multipath {
wwid 360a98000572d4534694a646149766c41
alias VG01-PV01
}
# map /vol/... to /dev/mapper/VG01-PV02
multipath {
wwid 360a98000572d4534694a646149767970
alias VG01-PV02
}
}
Start and check multipathd
[root@kratos iscsi]# chkconfig multipathd on [root@kratos iscsi]# service multipathd start [root@kratos iscsi]# multipath -ll VG01-PV02 (360a98000572d4534694a646149767970) dm-3 NETAPP,LUN size=64G features='1 queue_if_no_path' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=2 status=active `- 11:0:0:1 sdc 8:32 active ready running VG01-PV01 (360a98000572d4534694a646149766c41) dm-4 NETAPP,LUN size=64G features='1 queue_if_no_path' hwhandler='0' wp=rw `-+- policy='round-robin 0' prio=2 status=active `- 11:0:0:0 sdb 8:16 active ready running
Initialize the LUNs as LVM physical volumes:
# pvcreate /dev/mapper/VG01-PV01 # pvcreate /dev/mapper/VG01-PV02
Create volume groups and add physical volumes
# vgcreate VG01 /dev/mapper/VG01-PV01
Create logical volumes that use all space in the volume group
# lvcreate -l 100%VG -n LVhome VG01 # /sbin/mkfs -t ext4 /dev/VG01/LVhome # tune2fs -c0 -i0 /dev/VG01/LVhome # vgextend VG01 /dev/mapper/VG01-PV02
Create and configure the filesystems to trust journaling:
# /sbin/mkfs -t ext4 /dev/VG01/LVhome # tune2fs -c0 -i0 /dev/VG01/LVhome
Create the Mountpoints:
# mkdir /homes
Add the following lines to /etc/fstab. Note that the _netdev mount option defers the mount of the filesystem until the network is up and running:
/dev/VG01/LVhome /homes ext4 _netdev 0 0
Mount and check filesystem(s):
[root@kratos ~]# mount /homes
[root@kratos ~]# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vg_kratos-lv_root
50G 3.0G 44G 7% /
tmpfs 12G 88K 12G 1% /dev/shm
/dev/sda3 485M 56M 405M 13% /boot
/dev/sda2 200M 256K 200M 1% /boot/efi
/dev/mapper/vg_kratos-lv_opt
59G 180M 56G 1% /opt
/dev/mapper/VG01-LVhome
126G 188M 120G 1% /homes
Reboot the server - be sure all filesystems are available after the reboot....