kratos.phas.ubc.ca

Machine Type: 7944-AC1, Serial No.:KQ55F8D Product name: IBM System x3550 M3 server
Warranty expires July 8 2014, 1-800-426-7378

UpdateXpress (http://www-933.ibm.com/support/fixcentral/systemx/downloadFixes)
/opt/ibm/uxspi440.rhel6

IMM Management (http://www-933.ibm.com/support/fixcentral/systemx/downloadFixes)
IMM Users Guide (pdf)

MegaRAID Applications -> System Tools -> MegaRAID Storage Manager StartUpGuiUsername: rootPwd:root passwd
/usr/local/MegaRAID Storage Manager/startupui.sh

MegaRaid Storage Manager - /usr/local/MegaRAID\ Storage\ Manager/startupui.sh
MegaRAID_monitorconfig.xml file
MegaRaid Guide MegaCLI Guide

12-03-05 nfs Info

/var/lib/nfs/rmtab: each line shows the client's name and the file system imported from this server;
/var/lib/nfs/etab: the /etc/exports file only contains a wish list. etab is created by exportfs. It contains on each line detailed information about the options used when exporting a file system towards a single client. It's the reference file used by rpc.mountd when started
/proc/fs/nfs/exports contains the clients list as known by the kernel
/var/lib/nfs/xtab: Use for accuracy when etab holds client names and machines groups with wildcards. This file only contains explicit machine names.

11-12-22 Reboot Problem - won't shut down properly with reboot command
kratos reboot needs to be done via "/sbin/reboot.orig --force"
Solution (which did not work for me even though appears to be identical problem):

An option in multipath.conf that instructs it to fail I/O when multipathd isn't running -

# See https://bugzilla.redhat.com/show_bug.cgi?id=662433
# System hang on shutdown when iSCSI devices in use. Comment 32 suggests this.
        queue_without_daemon no

It would appear the problem is that just before shutdown the kernel sends a sync to all block devices. Even though file systems (except /) are unmounted , multipathd and the network are down at this point, the iSCSI block devices still exist. So the kernel sends off a sync to the (disconnected) iSCSI devices. The kernel side of the multipath driver then blocks waiting for a path to the iSCSI devices to complete the sync operation. It will wait forever.

'queue_without_daemon no' makes it fail the I/O to the devices instead of blocking for a path.

11-06-29 Currently E5620 @ 2.40 GHz 2x(4 Core), 24GB RAM, mirrored 146GB disks with hot spare. 3 free memory slots. 146 GB, 10K disks are FRU 43W7538.

Initial Setup for networking and iSCSI

12-01-10 kerberos/NFSv4 setup
Basically followed method at Kerberos/LDAP/NFSv4 HOWTO) with further info at How to configure nfsv4 with kerberos in RHEL and Part II)

/usr/bin/system-config-date
yum install krb5-server
cd /etc
cp krb5.conf krb5.conf.orig
vim krb5.conf
cd /var/kerberos/krb5kdc/
cp kdc.conf kdc.conf.orig
vim kdc.conf
cp kadm5.acl kadm5.acl.orig
vim kadm5.acl
vim /etc/gssapi_mech.conf 
yum install krb5-libs.x86_64 krb5-workstation.x86_64 pam_krb5.x86_64 cyrus-sasl-gssapi.x86_64 openldap-clients.x86_64
vim kadm5.acl
vim /etc/gssapi_mech.conf 
kdb5_util create -s
vim /etc/krb5.conf
vim /var/kerberos/krb5kdc/kdc.conf
vim /var/kerberos/krb5kdc/kadm5.acl
/usr/bin/authconfig-tui
kadmin.local -q "addprinc root/admin"
chkconfig kadmin on
service kadmin start
chkconfig krb5kdc on
service krb5kdc start

[root@kratos krb5kdc]# kinit
kinit: Client not found in Kerberos database while getting initial credentials
[root@kratos krb5kdc]# kinit root/admin
Password for root/admin@PHAS.UBC.CA:
[root@kratos krb5kdc]# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: root/admin@PHAS.UBC.CA

Valid starting Expires Service principal
01/12/12 09:55:48 01/13/12 09:55:48 krbtgt/PHAS.UBC.CA@PHAS.UBC.CA
renew until 01/12/12 09:55:52
[root@kratos krb5kdc]# kadmin.local
Authenticating as principal root/admin@PHAS.UBC.CA with password.
kadmin.local: addprinc -randkey nfs/kratos.phas.ubc.ca
WARNING: no policy specified for nfs/kratos.phas.ubc.ca@PHAS.UBC.CA; defaulting to no policy
Principal "nfs/kratos.phas.ubc.ca@PHAS.UBC.CA" created.
kadmin.local: ktadd nfs/kratos.phas.ubc.ca
Entry for principal nfs/kratos.phas.ubc.ca with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/etc/krb5.keytab.
Entry for principal nfs/kratos.phas.ubc.ca with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/etc/krb5.keytab.
Entry for principal nfs/kratos.phas.ubc.ca with kvno 2, encryption type des3-cbc-sha1 added to keytab WRFILE:/etc/krb5.keytab.
Entry for principal nfs/kratos.phas.ubc.ca with kvno 2, encryption type arcfour-hmac added to keytab WRFILE:/etc/krb5.keytab.
Entry for principal nfs/kratos.phas.ubc.ca with kvno 2, encryption type des-hmac-sha1 added to keytab WRFILE:/etc/krb5.keytab.
Entry for principal nfs/kratos.phas.ubc.ca with kvno 2, encryption type des-cbc-md5 added to keytab WRFILE:/etc/krb5.keytab.
kadmin.local: q

Check principal exists:

[root@kratos etc]# klist -k
Keytab name: WRFILE:/etc/krb5.keytab
KVNO Principal
---- --------------------------------------------------------------------------
   2 nfs/kratos.phas.ubc.ca@PHAS.UBC.CA
   2 nfs/kratos.phas.ubc.ca@PHAS.UBC.CA
   2 nfs/kratos.phas.ubc.ca@PHAS.UBC.CA
   2 nfs/kratos.phas.ubc.ca@PHAS.UBC.CA
   2 nfs/kratos.phas.ubc.ca@PHAS.UBC.CA
   2 nfs/kratos.phas.ubc.ca@PHAS.UBC.CA

Create service keys for kadmin/admin and kadmin/changepw. Not sure if this step was really needed.

[root@kratos]# kadmin.local -q "ktadd -k /var/kerberos/krb5kdc/kadm5.keytab kadmin/admin kadmin/changepw"
Authenticating as principal root/admin@PHAS.UBC.CA with password.
Entry for principal kadmin/admin with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/var/kerberos/krb5kdc/kadm5.keytab.
Entry for principal kadmin/admin with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/var/kerberos/krb5kdc/kadm5.keytab.
Entry for principal kadmin/admin with kvno 3, encryption type des3-cbc-sha1 added to keytab WRFILE:/var/kerberos/krb5kdc/kadm5.keytab.
Entry for principal kadmin/admin with kvno 3, encryption type arcfour-hmac added to keytab WRFILE:/var/kerberos/krb5kdc/kadm5.keytab.
Entry for principal kadmin/admin with kvno 3, encryption type des-hmac-sha1 added to keytab WRFILE:/var/kerberos/krb5kdc/kadm5.keytab.
Entry for principal kadmin/admin with kvno 3, encryption type des-cbc-md5 added to keytab WRFILE:/var/kerberos/krb5kdc/kadm5.keytab.
Entry for principal kadmin/changepw with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/var/kerberos/krb5kdc/kadm5.keytab.
Entry for principal kadmin/changepw with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/var/kerberos/krb5kdc/kadm5.keytab.
Entry for principal kadmin/changepw with kvno 3, encryption type des3-cbc-sha1 added to keytab WRFILE:/var/kerberos/krb5kdc/kadm5.keytab.
Entry for principal kadmin/changepw with kvno 3, encryption type arcfour-hmac added to keytab WRFILE:/var/kerberos/krb5kdc/kadm5.keytab.
Entry for principal kadmin/changepw with kvno 3, encryption type des-hmac-sha1 added to keytab WRFILE:/var/kerberos/krb5kdc/kadm5.keytab.
Entry for principal kadmin/changepw with kvno 3, encryption type des-cbc-md5 added to keytab WRFILE:/var/kerberos/krb5kdc/kadm5.keytab.

Make sure rpcgssd and rpcsvcgssd are configured to start on reboot and then start them.

[root@kratos init.d]# chkconfig --list | grep gss
rpcgssd        	0:off	1:off	2:off	3:on	4:on	5:on	6:off
rpcsvcgssd     	0:off	1:off	2:on	3:on	4:on	5:on	6:off
[root@kratos init.d]# service rpcgssd start
Starting RPC gssd:                                         [  OK  ]
[root@kratos init.d]# service rpcsvcgssd start
Starting RPC svcgssd:                                      [  OK  ]

kerberos/NFSv4 setup - Client Configuration

phasadmin@javelin:~$ sudo apt-get install krb5-user
phasadmin@javelin:~$ sudo vim /etc/krb5.conf
[libdefaults]
	default_realm = PHAS.UBC.CA

# The following krb5.conf variables are only for MIT Kerberos.
	krb4_config = /etc/krb.conf
	krb4_realms = /etc/krb.realms
	kdc_timesync = 1
	ccache_type = 4
	forwardable = true
	proxiable = true

[realms]
	PHAS.UBC.CA = {
		kdc = kratos.phas.ubc.ca
		kdc = ldap.phas.ubc.ca
		admin_server = kratos.phas.ubc.ca
		default_domain = phas.ubc.ca
	}

[domain_realm]
	.phas.ubc.ca = PHAS.UBC.CA
	phas.ubc.ca = PHAS.UBC.CA
	.physics.ubc.ca = PHAS.UBC.CA
	physics.ubc.ca = PHAS.UBC.CA
	.astro.ubc.ca = PHAS.UBC.CA
	astro.ubc.ca = PHAS.UBC.CA

Encountered and error when doing client config on ubuntu:

root@javelin:/etc# kadmin
Authenticating as principal root/admin@PHAS.UBC.CA with password.
Password for root/admin@PHAS.UBC.CA: 
kadmin: GSS-API (or Kerberos) error while initializing kadmin interface

Solved this by getting clocks in sync.
Run kadmin on client:

khroot@javelin:/etc# kadmin
Authenticating as principal root/admin@PHAS.UBC.CA with password.
Password for root/admin@PHAS.UBC.CA: 
kadmin:  addprinc -randkey nfs/javelin.phas.ubc.ca
WARNING: no policy specified for nfs/javelin.phas.ubc.ca@PHAS.UBC.CA; defaulting to no policy
add_principal: Principal or policy already exists while creating "nfs/javelin.phas.ubc.ca@PHAS.UBC.CA".
kadmin:  ktadd -k /etc/krb5.keytab nfs/javelin.phas.ubc.ca
Entry for principal nfs/javelin.phas.ubc.ca with kvno 2, encryption type AES-256 CTS mode with 96-bit SHA-1 HMAC added to keytab WRFILE:/etc/krb5.keytab.
Entry for principal nfs/javelin.phas.ubc.ca with kvno 2, encryption type AES-128 CTS mode with 96-bit SHA-1 HMAC added to keytab WRFILE:/etc/krb5.keytab.
Entry for principal nfs/javelin.phas.ubc.ca with kvno 2, encryption type Triple DES cbc mode with HMAC/sha1 added to keytab WRFILE:/etc/krb5.keytab.
Entry for principal nfs/javelin.phas.ubc.ca with kvno 2, encryption type ArcFour with HMAC/md5 added to keytab WRFILE:/etc/krb5.keytab.
Entry for principal nfs/javelin.phas.ubc.ca with kvno 2, encryption type DES with HMAC/sha1 added to keytab WRFILE:/etc/krb5.keytab.
Entry for principal nfs/javelin.phas.ubc.ca with kvno 2, encryption type DES cbc mode with RSA-MD5 added to keytab WRFILE:/etc/krb5.keytab.
kadmin:  q
root@javelin:/etc# klist -k
Keytab name: WRFILE:/etc/krb5.keytab
KVNO Principal
---- --------------------------------------------------------------------------
   2 nfs/javelin.phas.ubc.ca@PHAS.UBC.CA
   2 nfs/javelin.phas.ubc.ca@PHAS.UBC.CA
   2 nfs/javelin.phas.ubc.ca@PHAS.UBC.CA
   2 nfs/javelin.phas.ubc.ca@PHAS.UBC.CA
   2 nfs/javelin.phas.ubc.ca@PHAS.UBC.CA
   2 nfs/javelin.phas.ubc.ca@PHAS.UBC.CA

Edit /etc/default/nfs-common and make sure IDMAPD and GSSD lines are as follows:

NEED_IDMAPD=yes
NEED_GSSD=yes

Added this on KDC (kratos):

[root@kratos init.d]# kadmin
Authenticating as principal root/admin@PHAS.UBC.CA with password.
Password for root/admin@PHAS.UBC.CA: 
kadmin:  addprinc -randkey nfs/javelin.phas.ubc.ca
WARNING: no policy specified for nfs/javelin.phas.ubc.ca@PHAS.UBC.CA; defaulting to no policy
add_principal: Principal or policy already exists while creating "nfs/javelin.phas.ubc.ca@PHAS.UBC.CA".

root@javelin:~# apt-get install libpam-krb5

MegaRaid Storage Manager - /usr/local/MegaRAID\ Storage\ Manager/startupui.sh
MegaRAID_monitorconfig.xml file
MegaRaid Guide MegaCLI Guide

Initial Setup for networking and iSCSI

11-09-22 Will not shutdown

11-09-22 Disk Failure

System Info

A Note About User Management

BIOS Settings - 14-07-11

MegaRaid Storage Manager - /usr/local/MegaRAID\ Storage\ Manager/startupui.sh MegaRAID_monitorconfig.xml file MegaRaid Guide MegaCLI Guide

Initial Setup for networking and iSCSI

11-09-22 Will not shutdown

11-09-22 Disk Failure

System Info

A Note About User Management

BIOS Settings - 14-07-11

MegaRaid Storage Manager - /usr/local/MegaRAID\ Storage\ Manager/startupui.sh
MegaRAID_monitorconfig.xml file
MegaRaid Guide MegaCLI Guide