Technology Blogs by SAP
Learn how to extend and personalize SAP applications. Follow the SAP technology blog for insights into SAP BTP, ABAP, SAP Analytics Cloud, SAP HANA, and more.
cancel
Showing results for 
Search instead for 
Did you mean: 
malcolm_weaver
Discoverer
0 Kudos
Shutting down db2 luw fully when it is hung (has processes hung in unix / linux)

NOTE:  one should collect troubleshooting data (db2support.zip, db2fodc -hang full{db2fodc will try to collect all data it can collect while the database is hanging. one should use this command only in the case of a database hang. one should not use it on a normally running system}, etc) first before doing these steps, if one wants root cause analysis of abnormal db2 stoppage to be done if and when db2 is restarted after these steps.

*********************************************************************************
if the following fails to stop DB2 LUW on unix / linux . . .
*********************************************************************************
db2 list applications
db2 force applications "(<pid>)"
db2 force applications "(####)"
db2 list applications
db2stop
db2stop force
*********************************************************************************


steps (brute force method) to shutdown DB2 LUW on unix / linux
*********************************************************************************
using . . . x6d . . . as the <SID> in the following items . . .
*********************************************************************************
1. db2 list applications / db2 force applications . . . / db2stop / db2stop force
.......if any of these hang / if any of these fail to finish . . .
2. db2_kill . . . this may or may not succeed . . .
.......if db2_kill hangs also just continue with the following steps
3. ps -ef | grep -i x6d . . . check for db2 processes run by db2x6d or x6dadm
4. if there is(are) orphaned process(es) still
.......kill them using unix . . . kill -9 <process_id>
5. ipclean -a
6. ipcs -a | grep -i x6d . . . make sure ipclean succeeded . . .
.......this command should come back with empty output
7. if . . . ipcs -a | grep -i x6d . . . does come back with output
.......ipclean failed to clear up all the IPC resources
one will have to delete them manually as the root userid using ipcrm . . .
ipcrm -q #### (for message queues)
ipcrm -m #### (for shared memory)
ipcrm -s #### (for semaphores)
*********************************************************************************
*********************************************************************************
Below is an example of the above commands done on an in-house sandbox SAP system
with <SID>=X6D.


0. prepare situation for the seven steps listed above
***********************************************************************************


0.1. logged in with putty session as db2<sid> userid.
***********************************************************************************
telnet (slingaix)
AIX Version 6
Copyright IBM Corporation, 1982, 2015.
login: db2x6d
db2x6d's Password:
*******************************************************************************
* Welcome to AIX Version 6.1! *
*******************************************************************************
Last unsuccessful login: Mon May 16 20:26:18 2015 on
Last login: Fri May 20 02:18:56 2016 on


slingaix:db2x6d 1>
**********************************************************************************


0.2. verify csh shell is being used.
**********************************************************************************
slingaix:db2x6d 1> ps -p $$
PID TTY TIME CMD
22282816 pts/8 0:00 csh
slingaix:db2x6d 2>
**********************************************************************************


0.3. set prompt to expose shell:[userid]@[hostname]:[current_directory]>
**********************************************************************************
slingaix:db2x6d 2> setenv SHELL csh
slingaix:db2x6d 3> setenv xxshell
slingaix:db2x6d 3> set prompt="`echo $xxshell`:`id |cut -f2 -d'(' |cut -f1 -d')'`@`hostname | cut -f1 -d'.'`:`pwd`> "
slingaix:db2x6d 3> alias idmw "id |cut -f2 -d'(' |cut -f1 -d')'"
slingaix:db2x6d 4> alias cd 'chdir \!* && set prompt="`echo $xxshell`:`idmw`@`hostname | cut -f1 -d'.'`:`pwd`> "'
csh:db2x6d@slingaix:/db2/db2x6d>
**********************************************************************************


0.4. list processes associated with sap and db2 luw . . .
(the list below was edited to only show db2 processes)
**********************************************************************************
csh:db2x6d@slingaix:/> ps -ef | grep sap | grep -v grep; ps -ef | grep db2 | grep -v grep
. . .
db2x6d 4653258 1 0 02:17:52 - 0:00 /db2/db2x6d/sqllib/bin/db2bp 11862174A820 5 A
. . .
db2x6d 24576090 22282816 0 02:34:07 pts/8 0:00 db2
db2x6d 24772774 22872206 0 02:37:19 pts/10 0:00 db2pd
. . .
root 1507824 1 0 02:13:44 - 0:00 db2wdog 0
db2x6d 4260152 1 0 02:17:03 - 0:00 /db2/db2x6d/sqllib/bin/db2bp 24838148A820 5 A
root 5636466 23528312 0 02:13:44 - 0:00 db2ckpwd 0
. . .
root 10289526 23528312 0 02:13:44 - 0:00 db2ckpwd 0
db2x6d 14942710 23528312 0 02:13:45 - 0:00 db2vend (PD Vendor Process - 258)
. . .
root 19595734 23528312 0 02:13:44 - 0:00 db2ckpwd 0
db2x6d 4391668 1507824 0 02:13:46 - 0:03 db2acd 0
db2x6d 4784718 1 0 02:21:04 - 0:00 /db2/db2x6d/sqllib/bin/db2bp 5505540A820 5 A
db2x6d 6161042 1 0 02:22:01 - 0:00 db2
db2x6d 13501126 1 0 02:34:07 - 0:00 /db2/db2x6d/sqllib/bin/db2bp 22282816A820 5 A
db2x6d 15008262 1 0 02:28:44 - 0:00 /db2/db2x6d/sqllib/bin/db2bp 19333540A820 5 A
. . .
db2x6d 4391864 4784340 0 02:25:22 pts/11 0:02 topas
db2x6d 11273024 11862174 0 02:23:15 pts/6 0:00 db2
. . .
db2x6d 18678590 1 0 02:17:30 - 0:00 /db2/db2x6d/sqllib/bin/db2bp 9240584A820 5 A
db2x6d 22348740 24838148 0 02:17:03 pts/2 0:00 db2
db2x6d 22741888 9240584 0 02:17:30 pts/5 0:00 db2
db2x6d 23528312 1507824 134 02:13:44 - 16:08 db2sysc 0
csh:db2x6d@slingaix:/>


db2 list applications / db2 force applications . . . / db2stop / db2stop force
if any of these hang / if any of these fail to finish . . .
***************************************************************************************
atttempt to execute . . . db2 list applications
***************************************************************************************
csh:db2x6d@slingaix:/> db2 list applications
. . . this hangs . . .
***************************************************************************************


 

1. opened new putty session . . .
***************************************************************************************
since session in 1. above is hung, then opened new putty session . . .
then did steps 0.1. thru 0.3. above


 

2. db2_kill . . . this may or may not succeed . . .
if db2_kill hangs also just continue with the following steps.
**********************************************************************************
csh:db2x6d@slingaix:/> /db2/db2x6d/sqllib/bin/db2_kill
ipclean: Removing DB2 engine and client's IPC resources for db2x6d.
csh:db2x6d@slingaix:/>


 

3. ps -ef | grep -i x6d . . . check for db2 processes run by db2x6d or x6dadm
**********************************************************************************
csh:db2x6d@slingaix:/> ps -ef | grep -i x6d
. . .
db2x6d 24772774 22872206 0 02:37:19 pts/10 0:00 db2pd
. . .
db2x6d 5898870 1 0 02:44:11 - 0:00 /db2/db2x6d/sqllib/bin/db2bp 11862174A820 5 A
. . .
db2x6d 22348740 24838148 0 02:17:03 pts/2 0:00 db2
db2x6d 23528330 1 0 02:44:21 - 0:00 /db2/db2x6d/sqllib/bin/db2bp 19333540A820 5 A
csh:db2x6d@slingaix:/>


 

4. if there is(are) orphaned process(es) still, kill them using unix . . . kill -9 <process_id>
***************************************************************************************
csh:db2x6d@slingaix:/> kill -9 24772774 5898870 22348740 23528330
csh:db2x6d@slingaix:/>
************************************************************************************


4.1 verity the kill -9 succeeded
************************************************************************************
csh:db2x6d@slingaix:/> ps -ef | grep -i x6d
. . .
csh:db2x6d@slingaix:/>


 

 

5. ipclean -a
***************************************************************************************
csh:db2x6d@slingaix:/> which ipclean
/db2/db2x6d/sqllib/bin/ipclean
csh:db2x6d@slingaix:/> ipclean -a
ipclean: Removing all IPC resources for db2x6d.
csh:db2x6d@slingaix:/>


 

 

6. ipcs -a | grep -i x6d . . . make sure ipclean succeeded . . .
.......this should come back with no output
***************************************************************************************
csh:db2x6d@slingaix:/> ipcs -a | grep -i x6d
m 2097169 0x00002752 --rw-r----- x6dadm sapsys x6dadm sapsys 0 348685544 2950134 2950134 ...
m 1048693 0x0382be85 --rw-r----- x6dadm sapsys x6dadm sapsys 0 4096 4326170 4391668 ...
m 4194423 0x0000271a --rw-r----- x6dadm sapsys x6dadm sapsys 0 200000000 2950134 2950134 ...
s 133 0x002deeeb --ra-r----- x6dadm sapsys x6dadm sapsys 60 22:25:50 6:11:51
s 142 0x002deeea --ra-r----- x6dadm sapsys x6dadm sapsys 60 22:07:25 6:12:06


 

7. if . . . ipcs -a | grep -i x6d . . . does come back with output
ipclean failed to clear up all the IPC resources
one will have to delete them manually as the root userid using ipcrm . . .
ipcrm -q #### (for message queues)
ipcrm -m #### (for shared memory)
ipcrm -s #### (for semaphores)


one with the ability to login in as root must complete the last items . . .
***************************************************************************************
csh:root@slingaix:/> ipcrm -m 2097169
csh:root@slingaix:/> ipcrm -m 1048693
csh:root@slingaix:/> ipcrm -m 4194423
csh:root@slingaix:/> ipcrm -s 133
csh:root@slingaix:/> ipcrm -s 142