Upgrade of Kernel from 7.11 to 7.21 EXT on a PI 7.1 EHP1
Upgrading a kernel was an easy move. But the growing complexity of SAP-systems, makes an
upgrade of the kernel more difficult than it was in the past.
I was motivated to change the kernel by the PAM on the SAP service marketplace. I checked the “lifetime” of the kernel by:
PAM-page
searching my sap product
Hmm bad news 🙁 A System without supported kernel? No way !
I needed a new kernel and the newest was 7.21 EXT I looked at the PAM and found:
In order to be able to do the kernel upgrade in a proper way, I searched for the necessary
notes. I found 3 essential notes for the kernel upgrade.
1716826 – Usage of the downward compatible kernel 721 (EXT)
1728283 – SAP Kernel 721: General Information
1713986 – Installation von Kernel 7.21 (EXT)
Now I had all the information I needed. The next task was to download the new kernel.
http://service.sap.com/patches -> Support Packages and Patches -> A – Z Index -> ….
And now a little happy question time:
Question: When I search the kernel for a SAP PI 7.1 EHP 1 system, where do I have to search?
1. S for SAP?
2. P for PI?
3. Z for Zorro?
Answer: No, it’s N for Netweaver -> 1 hour lost for the answer of this question 😡
http://service.sap.com/patches -> Support Packages and Patches -> A – Z Index -> SAP NETWEAVER PI 7.1X -> SAP EHP1 FOR SAP NW PI 7.1 -> Entry by Component -> Application Server ABAP
I downloaded the following archives:
SAPCAR_315-20010445.EXE SAPCAR
DBATL720O10_31-20006704.SAR DBATOOLS Package for Oracle 10g and 11g
SAPEXE_100-10011322.SAR Kernel Part I (database independent)
SAPEXEDB_100-10011323.SAR Kernel Part II (database specific)
igsexe_5-20007786.sar Internet Graphic Server (IGS)
igshelper_3-10010245.sar IGS Helper
sapwebdisp_421-20008606.sar SAP Webdispatcher
SAPCRYPTOLIB_34-10010842.SAR SAP Kryptolib
SAPHOSTAGENT147_147-20005726.SAR SAPhostagent 7.20
SAP_112035_201303_HPUXIA64.zip SAP Bundle Patch
After analyzing the 3 notes, I identified the next tasks:
1. ) check Note 1610716 – Correcting runtime objects with incorrect alignment
2.) update SAPJVM 5.1.047 to 5.1.084
3.) implement the latest SBP (SAP bundle patch)
4.) Upgrade the kernel
ok 4 tasks to do. Let’s do it:
Task 1: check Note 1610716 – Correcting runtime objects with incorrect alignment
I had to implement Note 1610716 with the transaction SNOTE and I got the report RUT_NOTE_1610716. I started the report first with option check and after that with option repair. It ran several minutes. Task was done.
Task 2: update SAPJVM 5.1.047 to 5.1.084
Ok, now I need to search all relevant notes for patching a SAPJVM. I found:
1683392 – SAP JVM 5.1 Patch-Collection 58 (Build 5.1.074)
1434916 – How to find out the SAP JVM build version
1367498 – Installationsvoraussetzungen für SAP JVM
1025085 – How to manually patch the SAPJVM
1133020 – Importieren eines SAP-JVM-Patchs in Application Server Java
With note 1434916 I was able to identify the SAPJVM-Version. I was not able to identify the version of sapjvm on the system information page:( In my opinion the system information page of NW 7.0 is much better than on NW 7.1! The system information page of NW 7.0 is more simple, but the information is well-arranged.
Note 1025085 states:
“Patching the SAP JVM on an SAP NetWeaver system is only supported using the Software Update Manager (SUM) or the Java Support Package Manager (JSPM).”
I chose the JSPM.
After patching sapjvm I started the system and… it started without any error and… no java page could be displayed 😕
I posted my problem and a guru (big thanx to Reagan Benjamin) found the solution see:
http://scn.sap.com/thread/3388835 (No http page available after upgrade of SAPJVM)
After implementing Note 1625051 – “Wily Introscope agent: IllegalAccessError” the system runs fine again 🙂
ok next step:
Task 3) implement the latest SBP (SAP bundle patch)
Where can I find the SBP?
http://service.sap.com/patches -> Database and Database Patches (from other vendors) -> Oracle -> Oracle Patches 11.2.0.2.0
a) Checks
orasid> cd /oracle/SID/11203; bdf .
orasid> cd /oracle/stage/11203/database/SAP; bdf .
both filesystems should be 1GB free
b) Patche Opatch and MOPatch
orasid> setenv IHRDBMS /oracle/SID/11203; setenv OHRDBMS /oracle/SID/112_64; setenv ORACLE_HOME /oracle/SID/11203
orasid> cd /oracle/stage/11203/database/SAP
orasid> cp -p /inst_cd_sap/IA64/Oracle110203/updates/SAP_112035_201303/SAP_112035_201303_HPUXIA64.zip .
orasid> unzip -qd $IHRDBMS/sapbundle SAP_112035_201303_HPUXIA64.zip ‘SBP_112035_201303/OPatch/*’
orasid> mv $IHRDBMS/OPatch $IHRDBMS/OPatch-pre-SBP_112035_201303
orasid> mv $IHRDBMS/sapbundle/SBP_112035_201303/OPatch $IHRDBMS/OPatch
orasid> unzip -qd $IHRDBMS/sapbundle SAP_112035_201303_HPUXIA64.zip ‘SBP_112035_201303/MOPatch/*’
orasid> test -d $IHRDBMS/MOPatch && mv $IHRDBMS/MOPatch $IHRDBMS/MOPatch-pre-SBP_112035_201303
orasid> mv $IHRDBMS/sapbundle/SBP_112035_201303/MOPatch $IHRDBMS/MOPatch
orasid> $ORACLE_HOME/OPatch/opatch version
orasid> $ORACLE_HOME/MOPatch/mopatch.sh -h
The following versinos should be displayed:
OPatch version 11.2.0.3.1
MOPatch version 2.1.13
c) stop the system
sidadm> stopsap
orasid> lsnrctl stop
orasid> ps -efax | grep ora
no ora process should run!
d) install SBP
orasid> setenv ORACLE_HOME /oracle/SID/11203
orasid> cd /oracle/stage/11203/database/SAP
orasid> /bin/sh $ORACLE_HOME/MOPatch/mopatch.sh -v -s SAP_112035_201303_HPUXIA64.zip
orasid> lsnrctl start
<open a new session>
oraSID> sqlplus “/as sysdba”
SQL> startup
<old session>
oraSID> cd $OHRDBMS/rdbms/admin
oraSID> env ORACLE_HOME=$OHRDBMS $OHRDBMS/bin/sqlplus “/as sysdba”
SQL> @?/sapbundle/SBP_112035_201303/catsbp.sql
Hmmmm, INCOMPLETE. This is a word sap administrators don’t like! After reading Note 1509324 I did as it was recommended in the note.
SQL> select action_time from registry$history group by action_time having count(*) > 1;
If rows appear then read OSS-Note: 1508602.
no rows, great.
SQL> @?/rdbms/admin/utlrp.sql;
SQL> shutdown
SQL> startup
SQL> @?/sapbundle/SBP_112035_201303/catsbp.sql
No errors and two COMPLETEs. That was good.
I had to set two parameters ->
SQL> ALTER SYSTEM SET “_FIX_CONTROL”=
‘5099019:ON’,’5705630:ON’,’6055658:OFF’,’6399597:ON’,’6430500:ON’,
‘6440977:ON’,’6626018:ON’,’6972291:ON’,’7168184:OFF’,’8937971:ON’,
‘9196440:ON’,’9495669:ON’,’13077335:ON’,’13627489:ON’,’14255600:OFF’,
‘14595273:ON’ COMMENT=’SAP_112035_201302 RECOMMENDED SETTINGS’ SCOPE=SPFILE;
SQL> ALTER SYSTEM SET EVENT=
‘10027’,
‘10028’,
‘10142’,
‘10183’,
‘10191’,
‘10995 level 2’,
‘31991’,
‘38068 level 100’,
‘38085’,
‘38087’,
‘44951 level 1024’
COMMENT=’SAP_112035_201302 RECOMMENDED SETTINGS’ SCOPE=SPFILE;
SQL> shutdown immediate
SQL> startup
SQL> shutdown immediate
SBP is installed, Task done!
Task 4: Upgrade the kernel
This is not as easy as it was in the good old days! Now you have to do some more steps. In the good old days (when SAP ERP was R/3) you patched the system between 12:00 and 12:30 when users were at lunch! (Is it lunch or dinner? Hmm I don’t know. Please excuse my bad English!) But today you have to do it on weekend.
So let’s start.
Step 1: saphostagent upgrade
root> cd /tmp
root> mkdir saphostagent; cd saphostagent
root> /sapmnt/SID/SAPCAR -xvf /inst_cd_sap/IA64/kernel/PI_721_EXT_100/SAPHOSTAGENT147_147-20005726.SAR
root> ./saphostexec -upgrade
Step 2: stop the system
Stopping the saphostagent:
root> /usr/sap/hostctrl/exe/saphostexec -stop
root> /usr/sap/hostctrl/exe/saposcol -k
Stopping the sapwebdisp:
<user of sapwebdisp> stopsap all W90
Stopping sapstartsrv:
sidadm> sapcontrol -nr <Systemnummer Instanz z.B. 00> -prot NI_HTTP -function StopService;
sidadm> sapcontrol -nr <Systemnummer ASCS> -prot NI_HTTP -function StopService;
sidadm> sapcontrol -nr <Systemnummer SCS> -prot NI_HTTP -function StopService;
sidadm> sapcontrol -nr <Systemnummer ERS> -prot NI_HTTP -function StopService;
Deregister and stop CCMS-Agenten:
sidadm> cd /usr/sap/SID/SYS/exe/run/
sidadm> ./sapccm4x -u pf=/usr/sap/SID/SYS/profile/SID_DVEBMGS<Sy.Nr.>_<host>
sidadm> ./ccmsping -u pf=/usr/sap/SID/SYS/profile/SID_DVEBMGS<Sy.Nr.>_<host> -push -n<Sy.Nr>
stop Diagnosticagent
<D-Agentuser> stopsap SMDA90
Step 3: Clear shared memory
sidadm> /usr/sap/SID/SYS/exe/run/showipc all
Shows all shared memorys of all systems.
sidadm> /usr/sap/SID/SYS/exe/run/cleanipc <SystemNrIPC> remove
Now check for open shared memory segments. There should be no segment
sidadm> /usr/sap/SID/SYS/exe/run/showipc all
Step 4: get SAPCAR
sidadm> mkdir /tmp/sapcar
sidadm> cd /tmp/sapcar
sidadm> SAPCAR -xfv /<directory of sar-files>/SAPEXE_100-10011322.SAR
sidadm> cp SAPCAR /sapmnt/SID/
Step 5: removing the old kernel
root> cd /usr/sap/SID/SYS/exe/run/
root> rm -rf *
Step 6: implement the new kernel
sidadm> cd /usr/sap/SID/SYS/exe/run/
sidadm> /sapmnt/SID/SAPCAR -xfv /<directory of sar-files>/SAPEXE_100-10011322.SAR
sidadm> /sapmnt/SID/SAPCAR -xfv /<directory of sar-files>/SAPEXEDB_100-10011323.SAR
sidadm> /sapmnt/SID/SAPCAR -xfv /<directory of sar-files>/DBATL720O10_31-20006704.SAR
sidadm> /sapmnt/SID/SAPCAR -xfv /<directory of sar-files>/igsexe_5-20007786.sar
Step 7: implement sapcryprtolib
sidadm> mkdir /tmp/sapcrypto
sidadm> cd /tmp/sapcrypto
sidadm> /sapmnt/SID/SAPCAR -xfv /<directory of sar-files>/SAPCRYPTOLIB_34-10010842.SAR
sidadm> cp -p hpia64-11.31-64/* /usr/sap/SID/SYS/exe/run/
sidadm> cp -p hpia64-11.31-64/ticket /usr/sap/SID/DVEBMGS<Sy.Nr>/sec
Step 8: start saproot
root> cd /usr/sap/SID/SYS/exe/run/
root> ./saproot.sh SID
Step 9: install igshelper
sidadm> cd /usr/sap/SID/DVEBMGS<Sy.Nr.>; mv igs igs_old;
sidadm> /sapmnt/SID/SAPCAR -xfv /<directory of sar-files>/igshelper_3-10010245.sar
Step 10: new kernel for sapwebdisp
<user of sapwebdisp> cd /usr/sap/SWP_SID/SYS/exe/run
<user of sapwebdisp> rm -rf *
<user of sapwebdisp> /sapmnt/SID/SAPCAR -xfv /<directory of sar-files>/sapwebdisp_421-20008606.sar
Step 11: delete the exe-directories of all instances
root> cd /usr/sap/SID/DVEBMGS<Sy.Nr. of instance>/exe
root> rm -rf *
root> cd /usr/sap/SID/ASCS<Sy.Nr. of ASCS>/exe
root> rm -rf *
root> cd /usr/sap/SID/SCS<Sy.Nr. of SCS/exe
root> rm -rf *
root> cd /usr/sap/SID/ERS<Sy.Nr.of ERS>/exe
root> rm -rf *
Step 12: execute SAPCPE
It is recommended by sap to start sapcpe for every instance. Be aware to use the scs.lst file for the central services and the ERS!
For the central instance:
sidadm> cd /usr/sap/SID/DVEBMGS<Sy.Nr. instance>/work
sidadm> sapcpe pf=/usr/sap/SID/SYS/profile/SID_DVEBMGS<Sy.Nr. instance>_<host>
For the central services:
sidadm> cd /usr/sap/SID/SCS<Sy.Nr. SCS>/work
sidadm> sapcpe pf=/usr/sap/SID/SYS/profile/SID_SCS<Sy.Nr. SCS>_<host> list:/usr/sap/SID/SYS/exe/run/scs.lst
sidadm> cd /usr/sap/SID/ASCS<Sy.Nr. ASCS>/work
sidadm> sapcpe pf=/usr/sap/SID/SYS/profile/SID_ASCS<Sy.Nr. ASCS>_<host> list:/usr/sap/SID/SYS/exe/run/scs.lst
And for the Enhanced Replication Services (I only show it for one ERS because I’m too lazy;))
sidadm> cd /usr/sap/SID/ERS<Sy.Nr. ERS>/work
sidadm> sapcpe pf=/usr/sap/SID/SYS/profile/SID_ERS<Sy.Nr. ERS>_<host> list:/usr/sap/SID/SYS/exe/run/scs.lst
Last but not least the jvm needs it’s binaries too:
sidadm> sapcpe pf=/usr/sap/SID/SYS/profile/SID_DVEBMGS<Sy.Nr. instance>_<host> source:/sapmnt/SID/exe/jvm/hpia64/sapjvm_5.1.084 list:/sapmnt/SID/exe/jvm/hpia64/sapjvm_5.1.084/sapjvm_5.lst
Step 12: post activities
Enter line
rslg/new_layout = 9
in file /usr/sap/SIS/SYS/profile/DEFAULT.PFL
Then you have to delete the files of the syslog.
Final Step (13): startsap
Puhhh 13 steps…I entered
sidam> startsap
Sometimes startsap is a liar, so verified it:
sidadm> ps -efax | grep sap
ok. ABAP was running, but J2EE is a diva. I took a look at the dev_server0 and…
Thanx god java (the beast) is up and running
That was my adventure kernel patching a PI system. Finally I leaved the office and tried to relax at the rest of Sunday 😉 .
Best regards
Willi Eimler
Cracker.
That goes into my bookmarks
Regards
RB
Hi Benjamin,
Thanx!
Willi,
A most excellent piece of documentation. I wish I had seen this prior to just completing my landscape's 7.21 kernel upgrade, as you would have saved me many hours of work. The good news is that my process closely followed yours, so it is good to feel validated in that respect.
Question:
- What are your thoughts about also applying the latest dw_xxx-xxxxxxxx.sar archive file?
I interpreted 'Note 19466 - Downloading SAP kernel patches' to say that it was okay to apply the latest patch level to the kernel as long as you use the dw archive (the programs bundled in this archive need to always be synched to the same level).
- Do you think it is worthwhile to recommend to run SGEN, post upgrade, or is it just too obvious?
One warning for other readers of this great doc: This is an upgrade from 7.1x to 7.21. If you are upgrading from a lower version read the notes carefully, as there are additional steps. Example - Adding additional parameters for issues with dynamic processes.
Once again, great work!
Cheers,
-Doug
Doug,
I normally use only the SAPEXE.SAR and SAPEXEDB.SAR files for the kernel update.
I believe the dw_xxx.sar archive file contains most of the files present in the SAPEXE.SAR with an updated patch level. (I believe)
When it comes to kernel update SGEN is ONLY required to be executed if there is a change in the release of the kernel.
Eg: if the present release is 7.00 and the target is 7.20
Regards
RB
Reagan,
Correct on both points. The latter tells me that SGEN should probably be in this doc since it is an upgrade from 7.1x to 7.21. No biggie, as waiting to log in and all those 'Load format not valid' messages, once you do get in, will trigger about any basis person to run SGEN. As for the dw package, I'm curious who does and who doesn't use it. The current dw archive jumps the SAPEXE kernel level from patch 100 to patch 130. A fairly large jump.
Cheers.
Hi Doug,
first of all sorry for the late reply. I was on vacation.
I'm the conservative type of administrator. I don't use single patches. I always try to use Kernelstacks (SAPEXE.SAR, SAPEXEDB.SAR) like Reagan do!
Normally I use SGEN after an upgrade, but it was only a PI system. There are no users that can blame me;)
Best regards
Willi Eimler
Great post
Thanks for sharing you kernel journey !
For the DW patch there is an answer in the guide
"Deployment Strategies for the Kernel of the SAP Application Server ABAP"
I thought I did get it from a note, not able to remember where I get it from...
You usually need to apply such a kernel patch only in case you encounter a
problem with the kernel for which no workaround solution exists, and the corresponding correction is so new that it is not yet contained in the latest available SP stack kernel.
Almost a kind of "nightly build" at least an "early patch" 😉
Cheers
Yves,
Thanks for the clarification.
Rule #2 spells it out very clearly.
Cheers!
Hi Yves,
very good document:
"Deployment Strategies for the Kernel of the SAP Application Server ABAP"
How did you find it. You got help from the oracle of delphi;)
The 3 rules of the document are a big contrast to the recommendation of the EarlyWatchAlert (EWA always wants a new kernel).
Best regards
Willi Eimler
Hi Willi
I did finally remember where I get that document from... it's on SMP in the download area for kernel patch...
Nice docs are sometimes very well hidden in SMP 😉
https://service.sap.com/~sapidb/011000358700000911102011E
By the way have you noticed the new way SAP used to communicate on Kernel bugs ?
Sounds like Agile development impact !
Best regards
Thanks for the well structured document.
regards
kartik
Thank you for your sharing !!! 🙂
regards
Trojan
Thanx for your comment!
Best regards
Willi Eimjler
Very good document, added it to bookmarks 🙂
In step 12 I wonder about "Be aware to use the scs.lst file for the central services and the ERS!": Is this a SAP recommendation, or is it just for personal laziness (as there exists no ers.lst)? I built an ers.lst based on a slighly obsolete "Replikationsserver einrichten" that has just 12 lines/files instead of 54 in scs.lst. For disk space this means 76MB vs. 746MB (linux on x86_64).
I think ASCS could also be scaled down significantly (e.g. dos it need disp+work?).
Hi Ulrich,
it is a SAP recommendation. There exists no ers.lst file. The command is:
sidadm> sapcpe pf=/usr/sap/SID/SYS/profile/SID_ERS<Sy.Nr. ERS>_<host> list:/usr/sap/SID/SYS/exe/run/scs.lst
Take a look at 1713986 - Installation of kernel 721 (EXT)
there you find: