=============================================================== 16-01-20 on a normal update, zfs failed to update - see below. Briefly, fix was as follows: reboot after original update and zfs was not loaded [root@borg ~]# zpool status The ZFS modules are not loaded. Try running '/sbin/modprobe zfs' as root to load them. Tried: [root@borg ~]# yum -y reinstall dkms zfs libzfs2 zfs-dkms and got: ... Loading new zfs-0.6.5.4 DKMS files... Building for 2.6.32-573.12.1.el6.x86_64 Building initial module for 2.6.32-573.12.1.el6.x86_64 configure: error: *** Please make sure the kmod spl devel package for your *** distribution is installed then try again. If that fails you *** can specify the location of the spl objects with the *** '--with-spl-obj=PATH' option. Error! Bad return status for module build on kernel: 2.6.32-573.12.1.el6.x86_64 (x86_64) ... so did: [root@borg ~]# yum -y reinstall spl-dkms depmod... DKMS: install completed. Verifying : spl-dkms-0.6.5.4-1.el6.noarch 1/1 Installed: spl-dkms.noarch 0:0.6.5.4-1.el6 Complete! [root@borg ~]# yum -y reinstall dkms zfs libzfs2 zfs-dkms depmod... DKMS: install completed. Installing : zfs-0.6.5.4-1.el6.x86_64 4/4 Verifying : zfs-0.6.5.4-1.el6.x86_64 1/4 Verifying : zfs-dkms-0.6.5.4-1.el6.noarch 2/4 Verifying : dkms-2.2.0.3-30.git.7c3e7c5.el6.noarch 3/4 Verifying : libzfs2-0.6.5.4-1.el6.x86_64 4/4 Installed: dkms.noarch 0:2.2.0.3-30.git.7c3e7c5.el6 libzfs2.x86_64 0:0.6.5.4-1.el6 zfs.x86_64 0:0.6.5.4-1.el6 zfs-dkms.noarch 0:0.6.5.4-1.el6 Complete! After reboot, all was well ================================================================== [root@borg ~]# yum -y reinstall spl-dkms Loaded plugins: priorities, product-id, refresh-packagekit, replace, rhnplugin, security, subscription-manager This system is receiving updates from RHN Classic or RHN Satellite. Setting up Reinstall Process Resolving Dependencies --> Running transaction check ---> Package spl-dkms.noarch 0:0.6.5.4-1.el6 will be reinstalled --> Finished Dependency Resolution Dependencies Resolved ================================================================================================================================== Package Arch Version Repository Size ================================================================================================================================== Reinstalling: spl-dkms noarch 0.6.5.4-1.el6 zfs 449 k Transaction Summary ================================================================================================================================== Reinstall 1 Package(s) Total download size: 449 k Installed size: 2.4 M Downloading Packages: spl-dkms-0.6.5.4-1.el6.noarch.rpm | 449 kB 00:00 Running rpm_check_debug Running Transaction Test Transaction Test Succeeded Running Transaction Installing : spl-dkms-0.6.5.4-1.el6.noarch 1/1 Removing old spl-0.6.5.4 DKMS files... -------- Uninstall Beginning -------- Module: spl Version: 0.6.5.4 Kernel: 2.6.32-573.7.1.el6.x86_64 (x86_64) ------------------------------------- Status: Before uninstall, this module version was ACTIVE on this kernel. Removing any linked weak-modules WARNING: /lib/modules/2.6.32-573.12.1.el6.x86_64/weak-updates/zfs.ko needs unknown symbol __cv_wait_interruptible WARNING: /lib/modules/2.6.32-573.12.1.el6.x86_64/weak-updates/zfs.ko needs unknown symbol __cv_timedwait_interruptible WARNING: /lib/modules/2.6.32-573.12.1.el6.x86_64/weak-updates/splat.ko needs unknown symbol taskq_wait_all WARNING: Can't read module /lib/modules/2.6.32-573.7.1.el6.x86_64/weak-updates/spl.ko: No such file or directory WARNING: Can't read module /lib/modules/2.6.32-573.7.1.el6.x86_64/weak-updates/zavl.ko: No such file or directory WARNING: Can't read module /lib/modules/2.6.32-573.7.1.el6.x86_64/weak-updates/splat.ko: No such file or directory WARNING: Can't read module /lib/modules/2.6.32-573.7.1.el6.x86_64/weak-updates/zpios.ko: No such file or directory WARNING: /lib/modules/2.6.32-573.7.1.el6.x86_64/weak-updates/zfs.ko needs unknown symbol __cv_wait_interruptible WARNING: /lib/modules/2.6.32-573.7.1.el6.x86_64/weak-updates/zfs.ko needs unknown symbol __cv_timedwait_interruptible WARNING: /lib/modules/2.6.32-573.8.1.el6.x86_64/weak-updates/zfs.ko needs unknown symbol __cv_wait_interruptible WARNING: /lib/modules/2.6.32-573.8.1.el6.x86_64/weak-updates/zfs.ko needs unknown symbol __cv_timedwait_interruptible WARNING: /lib/modules/2.6.32-573.8.1.el6.x86_64/weak-updates/splat.ko needs unknown symbol taskq_wait_all egrep: /lib/modules/2.6.32-573.7.1.el6.x86_64//weak-updates/spl.ko: No such file or directory egrep: /lib/modules/2.6.32-573.7.1.el6.x86_64//weak-updates/zavl.ko: No such file or directory egrep: /lib/modules/2.6.32-573.7.1.el6.x86_64//weak-updates/splat.ko: No such file or directory egrep: /lib/modules/2.6.32-573.7.1.el6.x86_64//weak-updates/zpios.ko: No such file or directory spl.ko: - Uninstallation - Deleting from: /lib/modules/2.6.32-573.7.1.el6.x86_64/extra/ - Original module - No original module was found for this module on this kernel. - Use the dkms install command to reinstall any previous module version. splat.ko: - Uninstallation - Deleting from: /lib/modules/2.6.32-573.7.1.el6.x86_64/ rmdir: failed to remove `': No such file or directory - Original module - No original module was found for this module on this kernel. - Use the dkms install command to reinstall any previous module version. depmod... DKMS: uninstall completed. ------------------------------ Deleting module version: 0.6.5.4 completely from the DKMS tree. ------------------------------ Done. Loading new spl-0.6.5.4 DKMS files... Building for 2.6.32-573.12.1.el6.x86_64 Building initial module for 2.6.32-573.12.1.el6.x86_64 Done. spl: Running module version sanity check. ERROR: modinfo: could not open /lib/modules/2.6.32-573.12.1.el6.x86_64/weak-updates/spl.ko: No such file or directory - Original module - No original module exists within this kernel - Installation - Installing to /lib/modules/2.6.32-573.12.1.el6.x86_64/extra/ splat.ko: Running module version sanity check. - Original module - No original module exists within this kernel - Installation - Installing to /lib/modules/2.6.32-573.12.1.el6.x86_64/extra/ Adding any weak-modules WARNING: Can't read module /lib/modules/2.6.32-573.12.1.el6.x86_64/weak-updates/spl.ko: No such file or directory WARNING: /lib/modules/2.6.32-573.12.1.el6.x86_64/weak-updates/zfs.ko needs unknown symbol __cv_wait_interruptible WARNING: /lib/modules/2.6.32-573.12.1.el6.x86_64/weak-updates/zfs.ko needs unknown symbol __cv_timedwait_interruptible WARNING: Can't read module /lib/modules/2.6.32-573.7.1.el6.x86_64/weak-updates/zavl.ko: No such file or directory WARNING: Can't read module /lib/modules/2.6.32-573.7.1.el6.x86_64/weak-updates/zpios.ko: No such file or directory WARNING: /lib/modules/2.6.32-573.7.1.el6.x86_64/weak-updates/zfs.ko needs unknown symbol __cv_wait_interruptible WARNING: /lib/modules/2.6.32-573.7.1.el6.x86_64/weak-updates/zfs.ko needs unknown symbol __cv_timedwait_interruptible WARNING: /lib/modules/2.6.32-573.8.1.el6.x86_64/weak-updates/zfs.ko needs unknown symbol __cv_wait_interruptible WARNING: /lib/modules/2.6.32-573.8.1.el6.x86_64/weak-updates/zfs.ko needs unknown symbol __cv_timedwait_interruptible egrep: /lib/modules/2.6.32-573.12.1.el6.x86_64//weak-updates/spl.ko: No such file or directory egrep: /lib/modules/2.6.32-573.7.1.el6.x86_64//weak-updates/zavl.ko: No such file or directory egrep: /lib/modules/2.6.32-573.7.1.el6.x86_64//weak-updates/zpios.ko: No such file or directory depmod... DKMS: install completed. Verifying : spl-dkms-0.6.5.4-1.el6.noarch 1/1 Installed: spl-dkms.noarch 0:0.6.5.4-1.el6 Complete! [root@borg ~]# yum -y reinstall dkms zfs libzfs2 zfs-dkms Loaded plugins: priorities, product-id, refresh-packagekit, replace, rhnplugin, security, subscription-manager This system is receiving updates from RHN Classic or RHN Satellite. Setting up Reinstall Process Resolving Dependencies --> Running transaction check ---> Package dkms.noarch 0:2.2.0.3-30.git.7c3e7c5.el6 will be reinstalled ---> Package libzfs2.x86_64 0:0.6.5.4-1.el6 will be reinstalled ---> Package zfs.x86_64 0:0.6.5.4-1.el6 will be reinstalled ---> Package zfs-dkms.noarch 0:0.6.5.4-1.el6 will be reinstalled --> Finished Dependency Resolution Dependencies Resolved ================================================================================================================================== Package Arch Version Repository Size ================================================================================================================================== Reinstalling: dkms noarch 2.2.0.3-30.git.7c3e7c5.el6 epel 77 k libzfs2 x86_64 0.6.5.4-1.el6 zfs 113 k zfs x86_64 0.6.5.4-1.el6 zfs 324 k zfs-dkms noarch 0.6.5.4-1.el6 zfs 1.9 M Transaction Summary ================================================================================================================================== Reinstall 4 Package(s) Total download size: 2.4 M Installed size: 13 M Downloading Packages: (1/4): dkms-2.2.0.3-30.git.7c3e7c5.el6.noarch.rpm | 77 kB 00:00 (2/4): libzfs2-0.6.5.4-1.el6.x86_64.rpm | 113 kB 00:00 (3/4): zfs-0.6.5.4-1.el6.x86_64.rpm | 324 kB 00:00 (4/4): zfs-dkms-0.6.5.4-1.el6.noarch.rpm | 1.9 MB 00:02 ---------------------------------------------------------------------------------------------------------------------------------- Total 653 kB/s | 2.4 MB 00:03 Running rpm_check_debug Running Transaction Test Transaction Test Succeeded Running Transaction Installing : libzfs2-0.6.5.4-1.el6.x86_64 1/4 Installing : dkms-2.2.0.3-30.git.7c3e7c5.el6.noarch 2/4 Installing : zfs-dkms-0.6.5.4-1.el6.noarch 3/4 Removing old zfs-0.6.5.4 DKMS files... ------------------------------ Deleting module version: 0.6.5.4 completely from the DKMS tree. ------------------------------ Done. Loading new zfs-0.6.5.4 DKMS files... Building for 2.6.32-573.12.1.el6.x86_64 Building initial module for 2.6.32-573.12.1.el6.x86_64 Done. zavl: Running module version sanity check. Good news! Module version 0.6.5.4-1 for zavl.ko exactly matches what is already found in kernel 2.6.32-573.12.1.el6.x86_64. DKMS will not replace this module. You may override by specifying --force. znvpair.ko: Running module version sanity check. Good news! Module version 0.6.5.4-1 for znvpair.ko exactly matches what is already found in kernel 2.6.32-573.12.1.el6.x86_64. DKMS will not replace this module. You may override by specifying --force. zunicode.ko: Running module version sanity check. Good news! Module version 0.6.5.4-1 for zunicode.ko exactly matches what is already found in kernel 2.6.32-573.12.1.el6.x86_64. DKMS will not replace this module. You may override by specifying --force. zcommon.ko: Running module version sanity check. - Original module - No original module exists within this kernel - Installation - Installing to /lib/modules/2.6.32-573.12.1.el6.x86_64/extra/ zfs.ko: Running module version sanity check. - Original module - No original module exists within this kernel - Installation - Installing to /lib/modules/2.6.32-573.12.1.el6.x86_64/extra/ zpios.ko: Running module version sanity check. - Original module - No original module exists within this kernel - Installation - Installing to /lib/modules/2.6.32-573.12.1.el6.x86_64/extra/ Adding any weak-modules ERROR: modinfo: could not open /lib/modules/2.6.32-573.7.1.el6.x86_64/weak-updates/: Is a directory ERROR: modinfo: could not open /lib/modules/2.6.32-573.12.1.el6.x86_64/zavl.ko: No such file or directory FATAL: /lib/modules/2.6.32-573.12.1.el6.x86_64/zavl.ko: No such file or directory Warning: Module zavl.ko from kernel has no modversions, so it cannot be reused for kernel 2.6.32-573.7.1.el6.x86_64 ERROR: modinfo: could not open /lib/modules/2.6.32-573.7.1.el6.x86_64/weak-updates/: Is a directory ERROR: modinfo: could not open /lib/modules/2.6.32-573.12.1.el6.x86_64/znvpair.ko: No such file or directory FATAL: /lib/modules/2.6.32-573.12.1.el6.x86_64/znvpair.ko: No such file or directory Warning: Module znvpair.ko from kernel has no modversions, so it cannot be reused for kernel 2.6.32-573.7.1.el6.x86_64 ERROR: modinfo: could not open /lib/modules/2.6.32-573.7.1.el6.x86_64/weak-updates/: Is a directory ERROR: modinfo: could not open /lib/modules/2.6.32-573.12.1.el6.x86_64/zunicode.ko: No such file or directory FATAL: /lib/modules/2.6.32-573.12.1.el6.x86_64/zunicode.ko: No such file or directory Warning: Module zunicode.ko from kernel has no modversions, so it cannot be reused for kernel 2.6.32-573.7.1.el6.x86_64 ERROR: modinfo: could not open /lib/modules/2.6.32-573.8.1.el6.x86_64/weak-updates/: Is a directory ERROR: modinfo: could not open /lib/modules/2.6.32-573.12.1.el6.x86_64/zavl.ko: No such file or directory FATAL: /lib/modules/2.6.32-573.12.1.el6.x86_64/zavl.ko: No such file or directory Warning: Module zavl.ko from kernel has no modversions, so it cannot be reused for kernel 2.6.32-573.8.1.el6.x86_64 ERROR: modinfo: could not open /lib/modules/2.6.32-573.8.1.el6.x86_64/weak-updates/: Is a directory ERROR: modinfo: could not open /lib/modules/2.6.32-573.12.1.el6.x86_64/znvpair.ko: No such file or directory FATAL: /lib/modules/2.6.32-573.12.1.el6.x86_64/znvpair.ko: No such file or directory Warning: Module znvpair.ko from kernel has no modversions, so it cannot be reused for kernel 2.6.32-573.8.1.el6.x86_64 ERROR: modinfo: could not open /lib/modules/2.6.32-573.8.1.el6.x86_64/weak-updates/: Is a directory ERROR: modinfo: could not open /lib/modules/2.6.32-573.12.1.el6.x86_64/zunicode.ko: No such file or directory FATAL: /lib/modules/2.6.32-573.12.1.el6.x86_64/zunicode.ko: No such file or directory Warning: Module zunicode.ko from kernel has no modversions, so it cannot be reused for kernel 2.6.32-573.8.1.el6.x86_64 WARNING: Can't read module /lib/modules/2.6.32-573.12.1.el6.x86_64/weak-updates/spl.ko: No such file or directory WARNING: Can't read module /lib/modules/2.6.32-573.7.1.el6.x86_64/weak-updates/zavl.ko: No such file or directory WARNING: /lib/modules/2.6.32-573.7.1.el6.x86_64/weak-updates/zfs.ko needs unknown symbol __cv_wait_interruptible WARNING: /lib/modules/2.6.32-573.7.1.el6.x86_64/weak-updates/zfs.ko needs unknown symbol __cv_timedwait_interruptible WARNING: /lib/modules/2.6.32-573.8.1.el6.x86_64/weak-updates/zfs.ko needs unknown symbol __cv_wait_interruptible WARNING: /lib/modules/2.6.32-573.8.1.el6.x86_64/weak-updates/zfs.ko needs unknown symbol __cv_timedwait_interruptible egrep: /lib/modules/2.6.32-573.12.1.el6.x86_64//weak-updates/spl.ko: No such file or directory egrep: /lib/modules/2.6.32-573.7.1.el6.x86_64//weak-updates/zavl.ko: No such file or directory depmod... DKMS: install completed. Installing : zfs-0.6.5.4-1.el6.x86_64 4/4 Verifying : zfs-0.6.5.4-1.el6.x86_64 1/4 Verifying : zfs-dkms-0.6.5.4-1.el6.noarch 2/4 Verifying : dkms-2.2.0.3-30.git.7c3e7c5.el6.noarch 3/4 Verifying : libzfs2-0.6.5.4-1.el6.x86_64 4/4 Installed: dkms.noarch 0:2.2.0.3-30.git.7c3e7c5.el6 libzfs2.x86_64 0:0.6.5.4-1.el6 zfs.x86_64 0:0.6.5.4-1.el6 zfs-dkms.noarch 0:0.6.5.4-1.el6 Complete!