Oracle ASM is not up post kernel upgrade.

Recently we upgraded RHEL kernel from 3.10.0-514.16.1.el7.x86_64  to 3.10.0-693.17.1.el7.x86_64.  After kernel upgrade, ASM didn't come up.

Tried to mount the /dev/oracleasm/disks and got the below error.

[root@ip-test.sangam.com ~]# oracleasm init
Loading module "oracleasm": failed
Unable to load module "oracleasm"
[root@ip-test.sangam.com ~]#

I could see below error on /var/log/messages and /dev/oracleasm/disks file system was not accessible.

Apr 19 09:15:17 ip-test.sangam.com pushy-client: INFO: [ip-test.sangam.com] Incarnation ID: 5ead5023-2bc7-4079-911a-de1689b56405
Apr 19 09:15:17 ip-test.sangam.com pushy-client: INFO: [ip-test.sangam.com] Allowing fallback to unencrypted connection: true
Apr 19 09:15:17 ip-test.sangam.com pushy-client: INFO: [ip-test.sangam.com] Starting client ...
Apr 19 09:15:17 ip-test.sangam.com pushy-client: INFO: [ip-test.sangam.com] Retrieving configuration from https://giscinchef.corporate.ge.com/organizations/power-water//pushy/config/ip-test.sangam.com: ...
Apr 19 09:15:17 ip-test.sangam.com pushy-client: ERROR: Error connecting to https://giscinchef.corporate.ge.com/organizations/power-water/pushy/config/ip-test.sangam.com?ccpk=GAFx%25WSpj%3D-%3DIA%3E83VOtoy%231u-I4B%5EVYLf31%26W%5Dn&version=2.0, retry 1/5
Apr 19 09:15:21 ip-test.sangam.com modprobe: FATAL: Module oracleasm not found.
Apr 19 09:15:22 ip-test.sangam.com pushy-client: ERROR: Error connecting to https://giscinchef.corporate.ge.com/organizations/power-water/pushy/config/ip-test.sangam.com?ccpk=GAFx%25WSpj%3D-%3DIA%3E83VOtoy%231u-I4B%5EVYLf31%26W%5Dn&version=2.0, retry 2/5




Reason: On system we had below oracleasm module or rpms.

[root@ip-test.sangam.com ~]# rpm -qa|grep oracleasm
kmod-oracleasm-2.0.8-17.el7.x86_64
oracleasm-support-2.1.8-3.el7.x86_64
oracleasmlib-2.0.12-1.el7.x86_64

[root@ip-test.sangam.com ~]# uname -r
3.10.0-693.17.1.el7.x86_64
[root@ip-test.sangam.com ~]# uname -a
Linux ip-test.sangam.com.ec2.internal 3.10.0-693.17.1.el7.x86_64 #1 SMP Sun Jan 14 10:36:03 EST 2018 x86_64 x86_64 x86_64 GNU/Linux
[root@ip-test.sangam.com ~]# cat /etc/release
NAME="Red Hat Enterprise Linux Server"
VERSION="7.3 (Maipo)"
ID="rhel"
ID_LIKE="fedora"
VERSION_ID="7.3"
PRETTY_NAME="Red Hat Enterprise Linux Server 7.3 (Maipo)"



However for 3.10.0-693.17.1.el7.x86_64, we need to have latest oracleasm kernel module kmod-oracleasm-2.0.8-19.el7.x86_64.

Solution: Install the kmod-oracleasm-2.0.8-19.el7.x86_64 and restart oracleasm module.

[root@ip-test.sangam.com ~]# yum install kmod-oracleasm-2.0.8-19.el7.x86_64
Loaded plugins: 010-prune-by-regex, amazon-id, fastestmirror, rhui-lb, search-disabled-repos
Loading mirror speeds from cached hostfile
Resolving Dependencies
There are unfinished transactions remaining. You might consider running yum-complete-transaction, or "yum-complete-transaction --cleanup-only" and "yum history redo last", first to finish them. If those don't work you'll have to try removing/installing packages by hand (maybe package-cleanup can help).
--> Running transaction check
---> Package kmod-oracleasm.x86_64 0:2.0.8-17.el7 will be updated
---> Package kmod-oracleasm.x86_64 0:2.0.8-19.el7 will be an update
--> Finished Dependency Resolution

Dependencies Resolved

=======================================================================================================================================================================
 Package                                   Arch                              Version                                   Repository                                 Size
=======================================================================================================================================================================
Updating:
 kmod-oracleasm                            x86_64                            2
 .....
 ....
 ....
 Running transaction
  Updating   : kmod-oracleasm-2.0.8-19.el7.x86_64                                                                                                                  1/2
  Cleanup    : kmod-oracleasm-2.0.8-17.el7.x86_64                                                                                                                  2/2


  Verifying  : kmod-oracleasm-2.0.8-19.el7.x86_64                                                                                                                  1/2
  Verifying  : kmod-oracleasm-2.0.8-17.el7.x86_64                                                                                                                  2/2

Updated:
  kmod-oracleasm.x86_64 0:2.0.8-19.el7

Complete!
[root@ip-test.sangam.com ~]#


[root@ip-test.sangam.com ~]# oracleasm init
Loading module "oracleasm": oracleasm
Configuring "oracleasm" to use device physical block size
Mounting ASMlib driver filesystem: /dev/oracleasm
[root@ip-test.sangam.com ~]#

[root@ip-test.sangam.com ~]# /etc/init.d/oracleasm stop
Stopping oracleasm (via systemctl):                        [  OK  ]
[root@ip-test.sangam.com ~]# /etc/init.d/oracleasm start
Starting oracleasm (via systemctl):                        [  OK  ]
[root@ip-test.sangam.com ~]#

[root@ip-test.sangam.com ~]# oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...
Instantiating disk "TEST09_T1_256_DATA003"
Instantiating disk "TEST10_T1_256_DATA004"

...