Skip to Content

Upgrading a kernel was an easy move. But the growing complexity of SAP-systems, makes an
upgrade of the kernel more difficult than it was in the past.

I was motivated to change the kernel by the PAM on the SAP service marketplace. I checked the “lifetime” of the kernel by:

http://service.sap.com/pam

/wp-content/uploads/2013/08/b1_268214.jpg

  PAM-page

/wp-content/uploads/2013/08/b2_268215.jpg

searching my sap product

/wp-content/uploads/2013/08/b3_268216.jpg

Hmm bad news 🙁 A System without supported kernel? No way !

I needed a new kernel and the newest was 7.21 EXT I looked at the PAM and found:

/wp-content/uploads/2013/08/b4_268217.jpg

In order to be able to do the kernel upgrade in a proper way, I searched for the necessary
notes. I found 3 essential notes for the kernel upgrade.

1716826  – Usage of the downward compatible kernel 721 (EXT)            

1728283  – SAP Kernel 721: General Information            

1713986  – Installation von Kernel 7.21 (EXT)            

Now I had all the information I needed. The next task was to download the new kernel.

http://service.sap.com/patches -> Support Packages and Patches -> A – Z Index -> ….

And now a little happy question time:

Question: When I search the kernel for a SAP PI 7.1 EHP 1 system, where do I have to search?

1. S for SAP?

2. P for PI?

3. Z for Zorro?

Answer: No,  it’s N for Netweaver -> 1 hour lost for the answer of this question 😡

http://service.sap.com/patches -> Support Packages and Patches -> A – Z Index -> SAP NETWEAVER PI 7.1X -> SAP EHP1 FOR SAP NW PI 7.1 -> Entry by Component -> Application Server ABAP

I downloaded the following archives:

SAPCAR_315-20010445.EXE                 SAPCAR
DBATL720O10_31-20006704.SAR             DBATOOLS Package for Oracle 10g and 11g
SAPEXE_100-10011322.SAR                 Kernel Part I (database independent)
SAPEXEDB_100-10011323.SAR               Kernel Part II (database specific)
igsexe_5-20007786.sar                   Internet Graphic Server (IGS)
igshelper_3-10010245.sar                IGS Helper
sapwebdisp_421-20008606.sar             SAP Webdispatcher
SAPCRYPTOLIB_34-10010842.SAR            SAP Kryptolib
SAPHOSTAGENT147_147-20005726.SAR        SAPhostagent 7.20

SAP_112035_201303_HPUXIA64.zip          SAP Bundle Patch

After analyzing the 3 notes, I identified the next tasks:

1. ) check Note 1610716 – Correcting runtime objects with incorrect alignment
2.) update SAPJVM 5.1.047 to 5.1.084
3.) implement the latest SBP (SAP bundle patch)
4.) Upgrade the kernel

ok 4 tasks to do. Let’s do it:

Task 1: check Note 1610716 – Correcting runtime objects with incorrect alignment

I had to implement Note 1610716 with the transaction SNOTE and I got the report RUT_NOTE_1610716. I started the report first with option check and after that with option repair. It ran several minutes. Task was done.

Task 2:  update SAPJVM 5.1.047 to 5.1.084

Ok, now I need to search all relevant notes for patching a SAPJVM. I found:

1683392 – SAP JVM 5.1 Patch-Collection 58 (Build 5.1.074)
1434916 – How to find out the SAP JVM build version
1367498 – Installationsvoraussetzungen für SAP JVM
1025085 – How to manually patch the SAPJVM
1133020 – Importieren eines SAP-JVM-Patchs in Application Server Java

With note 1434916 I was able to identify the SAPJVM-Version. I was not able to identify the version of sapjvm on the system information page:( In my opinion the system information page of NW 7.0 is much better than on NW 7.1! The system information page of NW 7.0 is more simple, but the information is well-arranged.

Note 1025085 states:
“Patching the SAP JVM on an SAP NetWeaver system is only supported using the Software Update Manager (SUM) or the Java Support Package Manager (JSPM).”
I chose the JSPM.

After patching sapjvm I started the system and… it started without any error and… no java page could be displayed 😕

I posted my problem and a guru (big thanx to Reagan Benjamin) found the solution see:
http://scn.sap.com/thread/3388835 (No http page available after upgrade of SAPJVM)
After implementing Note 1625051 – “Wily Introscope agent: IllegalAccessError” the system runs fine again 🙂

ok next step:

Task 3) implement the latest SBP (SAP bundle patch)

Where can I find the SBP?
http://service.sap.com/patches -> Database and Database Patches (from other vendors) -> Oracle -> Oracle Patches 11.2.0.2.0

a) Checks
orasid> cd /oracle/SID/11203; bdf .
orasid> cd /oracle/stage/11203/database/SAP; bdf .
both filesystems should be 1GB free

b) Patche Opatch and MOPatch
orasid> setenv IHRDBMS /oracle/SID/11203; setenv OHRDBMS /oracle/SID/112_64; setenv ORACLE_HOME /oracle/SID/11203
orasid> cd /oracle/stage/11203/database/SAP
orasid> cp -p /inst_cd_sap/IA64/Oracle110203/updates/SAP_112035_201303/SAP_112035_201303_HPUXIA64.zip .
orasid> unzip -qd $IHRDBMS/sapbundle SAP_112035_201303_HPUXIA64.zip ‘SBP_112035_201303/OPatch/*’
orasid> mv $IHRDBMS/OPatch $IHRDBMS/OPatch-pre-SBP_112035_201303
orasid> mv $IHRDBMS/sapbundle/SBP_112035_201303/OPatch $IHRDBMS/OPatch
orasid> unzip -qd $IHRDBMS/sapbundle SAP_112035_201303_HPUXIA64.zip ‘SBP_112035_201303/MOPatch/*’
orasid> test -d $IHRDBMS/MOPatch && mv $IHRDBMS/MOPatch $IHRDBMS/MOPatch-pre-SBP_112035_201303
orasid> mv $IHRDBMS/sapbundle/SBP_112035_201303/MOPatch $IHRDBMS/MOPatch

orasid> $ORACLE_HOME/OPatch/opatch version

/wp-content/uploads/2013/08/b5_268218.jpg

  orasid> $ORACLE_HOME/MOPatch/mopatch.sh -h

/wp-content/uploads/2013/08/b6_268219.jpg

The following versinos should be displayed:
OPatch version 11.2.0.3.1
MOPatch version 2.1.13

c) stop the system
sidadm> stopsap
orasid> lsnrctl stop
orasid> ps -efax | grep ora
no ora process should run!

d) install SBP

orasid> setenv ORACLE_HOME /oracle/SID/11203
orasid> cd /oracle/stage/11203/database/SAP
orasid> /bin/sh $ORACLE_HOME/MOPatch/mopatch.sh -v -s SAP_112035_201303_HPUXIA64.zip

/wp-content/uploads/2013/08/b7_268226.jpg

orasid> lsnrctl start

<open a new session>
oraSID> sqlplus “/as sysdba”
SQL> startup

<old session>
oraSID> cd $OHRDBMS/rdbms/admin
oraSID> env ORACLE_HOME=$OHRDBMS $OHRDBMS/bin/sqlplus “/as sysdba”

SQL> @?/sapbundle/SBP_112035_201303/catsbp.sql

/wp-content/uploads/2013/08/b8_268227.jpg

Hmmmm, INCOMPLETE. This is a word sap administrators don’t like! After reading Note 1509324 I did as it was recommended in the note.

SQL> select action_time from registry$history group by action_time having count(*) > 1;

If rows appear then read OSS-Note: 1508602.

/wp-content/uploads/2013/08/b9_268228.jpg

no rows, great.

SQL> @?/rdbms/admin/utlrp.sql;

/wp-content/uploads/2013/08/b10_268230.jpg

SQL> shutdown

SQL> startup

SQL> @?/sapbundle/SBP_112035_201303/catsbp.sql

/wp-content/uploads/2013/08/b11_268231.jpg

No errors and two COMPLETEs. That was good.

I had to set two parameters ->

SQL> ALTER SYSTEM SET “_FIX_CONTROL”=
     ‘5099019:ON’,’5705630:ON’,’6055658:OFF’,’6399597:ON’,’6430500:ON’,
     ‘6440977:ON’,’6626018:ON’,’6972291:ON’,’7168184:OFF’,’8937971:ON’,
     ‘9196440:ON’,’9495669:ON’,’13077335:ON’,’13627489:ON’,’14255600:OFF’,
     ‘14595273:ON’ COMMENT=’SAP_112035_201302 RECOMMENDED SETTINGS’ SCOPE=SPFILE;

SQL> ALTER SYSTEM SET EVENT=
     ‘10027’,
     ‘10028’,
     ‘10142’,
     ‘10183’,
     ‘10191’,
     ‘10995 level 2’,
     ‘31991’,
     ‘38068 level 100’,
     ‘38085’,
     ‘38087’,
     ‘44951 level 1024’
     COMMENT=’SAP_112035_201302 RECOMMENDED SETTINGS’ SCOPE=SPFILE;

SQL> shutdown immediate
SQL> startup
SQL> shutdown immediate
SBP is installed, Task done!

Task 4: Upgrade the kernel

This is not as easy as it was in the good old days! Now you have to do some more steps. In the good old  days (when SAP ERP was R/3) you patched the system between 12:00 and 12:30 when users were at lunch! (Is it lunch or dinner? Hmm I don’t know. Please excuse my bad English!) But today you have to do it on weekend.

So let’s start.

Step 1: saphostagent upgrade

root> cd /tmp

root> mkdir saphostagent; cd saphostagent

root> /sapmnt/SID/SAPCAR -xvf /inst_cd_sap/IA64/kernel/PI_721_EXT_100/SAPHOSTAGENT147_147-20005726.SAR

root> ./saphostexec -upgrade

/wp-content/uploads/2013/08/b12_268235.jpg

Step 2: stop the system

Stopping the saphostagent:
root> /usr/sap/hostctrl/exe/saphostexec -stop
root> /usr/sap/hostctrl/exe/saposcol -k

/wp-content/uploads/2013/08/b13_268236.jpg

Stopping the sapwebdisp:

<user of sapwebdisp> stopsap all W90

/wp-content/uploads/2013/08/b14_268237.jpg

Stopping sapstartsrv:

sidadm> sapcontrol -nr <Systemnummer Instanz z.B. 00> -prot NI_HTTP -function StopService;

sidadm> sapcontrol -nr <Systemnummer ASCS> -prot NI_HTTP -function StopService;

sidadm> sapcontrol -nr <Systemnummer SCS> -prot NI_HTTP -function StopService;

sidadm> sapcontrol -nr <Systemnummer ERS> -prot NI_HTTP -function StopService;

/wp-content/uploads/2013/08/b15_268238.jpg

Deregister and stop CCMS-Agenten:

sidadm> cd /usr/sap/SID/SYS/exe/run/
sidadm> ./sapccm4x -u pf=/usr/sap/SID/SYS/profile/SID_DVEBMGS<Sy.Nr.>_<host>
sidadm> ./ccmsping -u pf=/usr/sap/SID/SYS/profile/SID_DVEBMGS<Sy.Nr.>_<host> -push -n<Sy.Nr>

stop Diagnosticagent
<D-Agentuser> stopsap SMDA90

/wp-content/uploads/2013/08/b16_268240.jpg

Step 3: Clear shared memory

sidadm> /usr/sap/SID/SYS/exe/run/showipc all
Shows all shared memorys of all systems.

sidadm> /usr/sap/SID/SYS/exe/run/cleanipc <SystemNrIPC> remove
Now check for open shared memory segments. There should be no segment
sidadm> /usr/sap/SID/SYS/exe/run/showipc all

/wp-content/uploads/2013/08/b17_268244.jpg

Step 4: get SAPCAR

sidadm> mkdir /tmp/sapcar
sidadm> cd /tmp/sapcar
sidadm> SAPCAR -xfv /<directory of sar-files>/SAPEXE_100-10011322.SAR
sidadm> cp SAPCAR /sapmnt/SID/

Step 5: removing the old kernel

root> cd /usr/sap/SID/SYS/exe/run/
root> rm -rf *

Step 6: implement the new kernel

sidadm> cd /usr/sap/SID/SYS/exe/run/
sidadm> /sapmnt/SID/SAPCAR -xfv /<directory of sar-files>/SAPEXE_100-10011322.SAR
sidadm> /sapmnt/SID/SAPCAR -xfv /<directory of sar-files>/SAPEXEDB_100-10011323.SAR
sidadm> /sapmnt/SID/SAPCAR -xfv /<directory of sar-files>/DBATL720O10_31-20006704.SAR
sidadm> /sapmnt/SID/SAPCAR -xfv /<directory of sar-files>/igsexe_5-20007786.sar

/wp-content/uploads/2013/08/b18_268245.jpg

Step 7: implement sapcryprtolib

sidadm> mkdir /tmp/sapcrypto
sidadm> cd /tmp/sapcrypto
sidadm> /sapmnt/SID/SAPCAR -xfv /<directory of sar-files>/SAPCRYPTOLIB_34-10010842.SAR
sidadm> cp -p hpia64-11.31-64/* /usr/sap/SID/SYS/exe/run/
sidadm> cp -p hpia64-11.31-64/ticket /usr/sap/SID/DVEBMGS<Sy.Nr>/sec

Step 8: start saproot

root> cd /usr/sap/SID/SYS/exe/run/
root> ./saproot.sh SID

/wp-content/uploads/2013/08/b19_268246.jpg

Step 9: install igshelper

sidadm> cd /usr/sap/SID/DVEBMGS<Sy.Nr.>; mv igs igs_old;
sidadm> /sapmnt/SID/SAPCAR -xfv /<directory of sar-files>/igshelper_3-10010245.sar

Step 10: new kernel for sapwebdisp

<user of sapwebdisp> cd /usr/sap/SWP_SID/SYS/exe/run
<user of sapwebdisp> rm -rf *
<user of sapwebdisp> /sapmnt/SID/SAPCAR -xfv /<directory of sar-files>/sapwebdisp_421-20008606.sar

Step 11: delete the exe-directories of all instances

root> cd /usr/sap/SID/DVEBMGS<Sy.Nr. of instance>/exe
root> rm -rf *

root> cd /usr/sap/SID/ASCS<Sy.Nr. of ASCS>/exe
root> rm -rf *

root> cd /usr/sap/SID/SCS<Sy.Nr. of SCS/exe
root> rm -rf *

root> cd /usr/sap/SID/ERS<Sy.Nr.of ERS>/exe
root> rm -rf *

Step 12: execute SAPCPE

It is recommended by sap to start sapcpe for every instance. Be aware to use the scs.lst file for the central services and the ERS!

For the central instance:

sidadm> cd /usr/sap/SID/DVEBMGS<Sy.Nr. instance>/work
sidadm> sapcpe pf=/usr/sap/SID/SYS/profile/SID_DVEBMGS<Sy.Nr. instance>_<host>

For the central services:

sidadm> cd /usr/sap/SID/SCS<Sy.Nr. SCS>/work
sidadm> sapcpe pf=/usr/sap/SID/SYS/profile/SID_SCS<Sy.Nr. SCS>_<host> list:/usr/sap/SID/SYS/exe/run/scs.lst

sidadm> cd /usr/sap/SID/ASCS<Sy.Nr. ASCS>/work
sidadm> sapcpe pf=/usr/sap/SID/SYS/profile/SID_ASCS<Sy.Nr. ASCS>_<host> list:/usr/sap/SID/SYS/exe/run/scs.lst

And for the Enhanced Replication Services (I only show it for one ERS because I’m too lazy;))

sidadm> cd /usr/sap/SID/ERS<Sy.Nr. ERS>/work
sidadm> sapcpe pf=/usr/sap/SID/SYS/profile/SID_ERS<Sy.Nr. ERS>_<host> list:/usr/sap/SID/SYS/exe/run/scs.lst

Last but not least the jvm needs it’s binaries too:

sidadm> sapcpe pf=/usr/sap/SID/SYS/profile/SID_DVEBMGS<Sy.Nr. instance>_<host> source:/sapmnt/SID/exe/jvm/hpia64/sapjvm_5.1.084 list:/sapmnt/SID/exe/jvm/hpia64/sapjvm_5.1.084/sapjvm_5.lst

Step 12:  post activities

Enter line 
rslg/new_layout = 9
in file /usr/sap/SIS/SYS/profile/DEFAULT.PFL
Then you have to delete the files of the syslog.

Final Step (13):  startsap

Puhhh 13 steps…I entered

sidam> startsap

/wp-content/uploads/2013/08/b20_268253.jpg

Sometimes startsap is a liar, so verified it:

sidadm> ps -efax | grep sap

/wp-content/uploads/2013/08/b21_268254.jpg

ok. ABAP was running, but J2EE is a diva. I took a look at the dev_server0 and…

/wp-content/uploads/2013/08/b22_268255.jpg

Thanx god java (the beast) is up and running

That was my adventure kernel patching a PI system. Finally I leaved the office and tried to relax at the rest of Sunday 😉 .

Best regards

Willi Eimler

To report this post you need to login first.

16 Comments

You must be Logged on to comment or reply to a post.

  1. Doug Turley

    Willi,

    A most excellent piece of documentation.  I wish I had seen this prior to just completing my landscape’s 7.21 kernel upgrade, as you would have saved me many hours of work.  The good news is that my process closely followed yours, so it is good to feel validated in that respect.

    Question:

    – What are your thoughts about also applying the latest dw_xxx-xxxxxxxx.sar archive file?

    I interpreted ‘Note 19466 – Downloading SAP kernel patches’ to say that it was okay to apply the latest patch level to the kernel as long as you use the dw archive (the programs bundled in this archive need to always be synched to the same level).

    – Do you think it is worthwhile to recommend to run SGEN, post upgrade, or is it just too obvious?

    One warning for other readers of this great doc:  This is an upgrade from 7.1x to 7.21.  If you are upgrading from a lower version read the notes carefully, as there are additional steps.  Example – Adding additional parameters for issues with dynamic processes.

    Once again, great work!

    Cheers,

    -Doug

    (0) 
    1. Reagan Benjamin

      Doug,

      – What are your thoughts about also applying the latest dw_xxx-xxxxxxxx.sar archive file?

      I normally use only the SAPEXE.SAR and SAPEXEDB.SAR files for the kernel update.

      I believe the dw_xxx.sar archive file contains most of the files present in the SAPEXE.SAR with an updated patch level. (I believe)

      – Do you think it is worthwhile to recommend to run SGEN, post upgrade, or is it just too obvious?

      When it comes to kernel update SGEN is ONLY required to be executed if there is a change in the release of the kernel.

      Eg: if the present release is 7.00 and the target is 7.20

      Regards

      RB

      (0) 
      1. Doug Turley

        Reagan,

        Correct on both points.  The latter tells me that SGEN should probably be in this doc since it is an upgrade from 7.1x to 7.21.  No biggie, as waiting to log in and all those ‘Load format not valid’ messages, once you do get in, will trigger about any basis person to run SGEN.  As for the dw package, I’m curious who does and who doesn’t use it.  The current dw archive jumps the SAPEXE kernel level from patch 100 to patch 130.  A fairly large jump. 

        Cheers.

        (0) 
    2. Willi Eimler Post author

      Hi Doug,

      first of all sorry for the late reply. I was on vacation.

      I’m the conservative type of administrator. I don’t use single patches. I always try to use Kernelstacks (SAPEXE.SAR, SAPEXEDB.SAR) like Reagan do!

      Normally I use SGEN after an upgrade, but it was only a PI system. There are no users that can blame me;)

      Best regards

      Willi Eimler

      (0) 
  2. Yves KERVADEC

    Great post

    Thanks for sharing you kernel journey !

    For the DW patch there is an answer in the guide

    “Deployment Strategies for the Kernel of the SAP Application Server ABAP”

    I thought I did get it from a note, not able to remember where I get it from…

    You usually need to apply such a kernel patch only in case you encounter a

    problem with the kernel for which no workaround solution exists, and the corresponding correction is so new that it is not yet contained in the latest available SP stack kernel.

    Almost a kind of “nightly build” at least an “early patch”  😉

    Cheers

    (0) 
    1. Willi Eimler Post author

      Hi Yves,

      very good document:

      “Deployment Strategies for the Kernel of the SAP Application Server ABAP”

      How did you find it. You got help from the oracle of delphi;)

      The 3 rules of the document are a big contrast to the recommendation of the EarlyWatchAlert (EWA always wants a new kernel).

      Best regards

      Willi Eimler

      (0) 
  3. Ulrich Windl

    In step 12 I wonder about “Be aware to use the scs.lst file for the central services and the ERS!”: Is this a SAP recommendation, or is it just for personal laziness (as there exists no ers.lst)? I built an ers.lst based on a slighly obsolete “Replikationsserver einrichten” that has just 12 lines/files instead of  54 in scs.lst. For disk space this means 76MB vs. 746MB (linux on x86_64).

    I think ASCS could also be scaled down significantly (e.g. dos it need disp+work?).

    (0) 
    1. Willi Eimler Post author

      Hi Ulrich,

      it is a SAP recommendation. There exists no ers.lst file. The command is:

      sidadm> sapcpe pf=/usr/sap/SID/SYS/profile/SID_ERS<Sy.Nr. ERS>_<host> list:/usr/sap/SID/SYS/exe/run/scs.lst


      Take a look at   1713986 – Installation of kernel 721 (EXT)

      there you find:

      For additional instances such as ASCS, SCS and ERS (exist only in a cluster):

      cd /usr/sap/<SAPSID>/<INSTANCE>/work

      sapcpe pf=/usr/sap/<SAPSID>/SYS/profile/<instance profile> list:/usr/sap/<SAPSID>/SYS/exe/run/scs.lst

      (0) 

Leave a Reply