Previously I have shared How to install the XSA WebIDE, the upgrade process is similar.
First backup all your existing XSA projects to Git and/or export the mta archives.
It's easy to forget exactly what's been installed, so I have captured here the pieces I needed.
If you extract the XSA Server (EXTAPPSER*.SAR) component to a sub-folder within the same parent directory as the install media (SAP_HANA_DATABASE) then the SAP_HANA_DATABASE\hdblcm will find EXTAPPSER.
media
|
|-- SAP_HANA_DATABASE
| |- hdblcm
| |- hdbinst
| |- hdbsetup
| |- ...
|
|-- XSA_SERVER (put EXTAPPSER*.SAR in here, and extract it)
| |- hdbinst
| |- ...
Running hdblcm from SAP_HANA_DATABASE directory and it will automagically find the XSA_SERVER and other packages within the directory structure.
Below I had already installed the XSA update using option 3.
With the install completed I then used the xs client commands to update the WebIDE and associated components.
First log into XSA and specifying the appropriate space.
I can now go ahead an install the download zip files.
Before actually using the updated WebIDE, it is required update the di-builder via the space enablement your non SAP space. To find out out the URL for this you can run this xs command.
Going to that URL,shows the space needs updating with the latest goodies.
For me to use external sites such as github.com or deploy to cloudfoundary required a proxy. This can done by specifying the JBP_CONFIG_JAVA_OPTS for the DI_CORE module and redeploying it. I created a .mtaext file with the following contents.
It's best to validate your MTA file with a YAML checker such as https://codebeautify.org/yaml-validator
After creating the .mtaext file I then needed to re-deploy the WebIDE to pickup the proxy settings.
Npm (node package manager) also requires the proxy to be set, I did this as below.
If you receive the following error
SAP Note 2654543 would have the full details.
To correct this, you need to execute the following from the SIDADM user
Previously I had not configured a trusted SSL certificate for XSA, but this can lead to unexpected errors with the different xsa services. Each untrusted URL needs to been granted browser exceptions.
There's a SAP Note 2243019 that describes what is required to configure SSL
ffad982802c345bdb6e7377a267f27ce wrote a great blog on his XSA configuration, the steps I used are similar but not the same.
I had already setup SSL for XS, so thought it would be easy to re-use some of that configuration (it's not that easy).
Within the newly generated export.pem there are multiple certificates and keys. The Private Key is the required part which starts with.
-----BEGIN PRIVATE KEY-----
XSA also requires a full certificate chain. This means you need to supply the host, intermediate and root certificates all combined into a single x.509 PEM encoded file.
I was provisioning and internal SAP host using our internal Web Enrolment Service.
If you are an SAP employee or have access to the corporate network this can be found here https://sapcerts.wdf.global.corp.sap/WebEnrollment.aspx
This provides the certificate for the XSA host, but it needs to converted and combined with the full certificate chain. I download the appropriate root (SAP Global Root CA) and intermediate (SAPNetCA_G2), which come in .crt format.
After successfully installing the certificate you need to restart each of the XSA apps individually or the whole of XSA (which is easier). This can be dome with the hdbadm user.
After which I have XSA & WebIDE running with a signed SSL certificate.
Including all the new graphical Calculation View operators like graph, anonymization, union, minus, intersect, hierarchy function, etc.
There's a great Troubleshooting guide in the Help section on the WebIDE.
SAP WEB IDE for SAP HANA Installation Troubleshooting Guide
If you do experience issues the following commands are useful for identifying, potential issues.
If the certificates get messed up you can reset those using the hdbadm user
I have experienced some errors when the configuration was not correct. Often the exposed error is only the latest error and not the root cause of the problem for me I was initially seeing this generic error
I also found these errors, showing that a process has not started.
Here di-core was not starting, due to a lack of allocated memory.
Rescaling the application fixed this.
The root cause for this was hidden deeper in the di-core logs, with an in-correct proxy configuration. Initially in the .mtaext I had ";" separators between my nonProxyHosts, switching to "|" fixed this.
Upon launching the WebIDE I have seen the response
Here resetting the certificate resolved the issue. Beware this does also stop and restart XSA without any confirmation.
Backup
First backup all your existing XSA projects to Git and/or export the mta archives.
Obtaining the Software
It's easy to forget exactly what's been installed, so I have captured here the pieces I needed.
| Search Words | Description | Filename (as at 11-Sept-2018) |
|---|---|---|
| EXTAPPSER | Extended App Services - XSA server binaries | EXTAPPSER00P_88-70001316.SAR |
| HRTT | HANA Run Time Tools, required for WebIDE | XSACHRTT06_62-70002322.ZIP |
| XSACMONITOR | XSA Monitoring, required for WebIDE | XSACMONITORING06_5-70002324.ZIP |
| WEBIDE | SAP WEBIDE 2 | XSACSAPWEBIDE03_6-80002257.ZIP |
Extracting the SAR
If you extract the XSA Server (EXTAPPSER*.SAR) component to a sub-folder within the same parent directory as the install media (SAP_HANA_DATABASE) then the SAP_HANA_DATABASE\hdblcm will find EXTAPPSER.
media
|
|-- SAP_HANA_DATABASE
| |- hdblcm
| |- hdbinst
| |- hdbsetup
| |- ...
|
|-- XSA_SERVER (put EXTAPPSER*.SAR in here, and extract it)
| |- hdbinst
| |- ...
Installing EXTAPPSER
Running hdblcm from SAP_HANA_DATABASE directory and it will automagically find the XSA_SERVER and other packages within the directory structure.
Below I had already installed the XSA update using option 3.
Updating the XSA Apps
With the install completed I then used the xs client commands to update the WebIDE and associated components.
First log into XSA and specifying the appropriate space.
mo-3fda111e5.mo:/usr/sap/IH2/HDB00> xs login -u XSA_ADMIN -p TopSecretPassword -a https://mo-3fda111e5.mo.sap.corp:30030 -s SAP
API_URL: https://mo-3fda111e5.mo.sap.corp:30030
USERNAME: XSA_ADMIN
Authenticating...
ORG: orgname
SPACE: SAP
API endpoint: https://mo-3fda111e5.mo.sap.corp:30030 (API version: 1)
User: XSA_ADMIN
Org: orgname
Space: SAPI can now go ahead an install the download zip files.
xs install XSACHRTT06_62-70002322.ZIP
xs install XSACSAPWEBIDE03_6-80002257.ZIP
xs install XSACMONITORING06_5-70002324.ZIP Space Enablement
Before actually using the updated WebIDE, it is required update the di-builder via the space enablement your non SAP space. To find out out the URL for this you can run this xs command.
mo-3fda111e5.mo:/hana/media/XSA-88> xs app di-space-enablement-ui --urls
https://mo-3fda111e5.mo.sap.corp:51022Going to that URL,shows the space needs updating with the latest goodies.
Proxy Configuration
For me to use external sites such as github.com or deploy to cloudfoundary required a proxy. This can done by specifying the JBP_CONFIG_JAVA_OPTS for the DI_CORE module and redeploying it. I created a .mtaext file with the following contents.
_schema-version: "3.1.0"
ID: com.sap.devx.xs2-config1
extends: com.sap.devx.webide
modules:
- name: di-core
parameters:
port: 53030
memory: 2G
properties:
JBP_CONFIG_JAVA_OPTS: '[java_opts: " -Dhttps.proxyHost=proxy.wdf.sap.corp -Dhttps.proxyPort=8080 -Dhttp.proxyHost=proxy.wdf.sap.corp -Dhttp.proxyPort=8080 -Dhttp.nonProxyHosts=\\"*.sap.corp|*.corp.sap|mo-3fda111e5|mo-3fda111e5.mo.sap.corp|localhost|127.0.0.1|10.*\\" -Dhttps.nonProxyHosts=\\"*.sap.corp|*.corp.sap|mo-3fda111e5|mo-3fda111e5.mo.sap.corp|localhost|127.0.0.1|10.*\\""]'It's best to validate your MTA file with a YAML checker such as https://codebeautify.org/yaml-validator
After creating the .mtaext file I then needed to re-deploy the WebIDE to pickup the proxy settings.
xs install XSACSAPWEBIDE03_6-80002257.ZIP -e My-WebIDE.mtaext -o ALLOW_SC_SAME_VERSION --delete-servicesNpm (node package manager) also requires the proxy to be set, I did this as below.
xs set-env di-local-npm-registry HTTPS_PROXY http://proxy.wdf.sap.corp:8080
xs set-env di-local-npm-registry HTTP_PROXY http://proxy.wdf.sap.corp:8080
xs restage di-local-npm-registry
xs restart di-local-npm-registryRenew trust relationship between XSA and HANA Database
If you receive the following error
error setting XS_APPLICATIONUSER failed due to invalid or expired tokenSAP Note 2654543 would have the full details.
To correct this, you need to execute the following from the SIDADM user
XSA renew-db-trust -u XSA_ADMINSSL - Private Key
Previously I had not configured a trusted SSL certificate for XSA, but this can lead to unexpected errors with the different xsa services. Each untrusted URL needs to been granted browser exceptions.
There's a SAP Note 2243019 that describes what is required to configure SSL
ffad982802c345bdb6e7377a267f27ce wrote a great blog on his XSA configuration, the steps I used are similar but not the same.
I had already setup SSL for XS, so thought it would be easy to re-use some of that configuration (it's not that easy).
- Export the private key from the in-built SAP keystore. A detailed blog on sapgenpse can be found here
sapgenpse export_p12 -p /usr/sap/IH2/HDB00/mo-3fda111e5.mo.sap.corp/sec/SAPSSLS.pse /hana/media/ssl/Export.p122. Convert the .p12 format to pem.
openssl pkcs12 -in Export.p12 -out Export.pem -nodesWithin the newly generated export.pem there are multiple certificates and keys. The Private Key is the required part which starts with.
-----BEGIN PRIVATE KEY-----
3. copy/paste just the private key that into a new file Private.key
4. Convert to pkcs8 (that XSA requires). This can be converted with this command.
openssl pkcs8 -topk8 -in Private.key -out Private_pkcs8.key -nocryptSSL Root & Intermediate Certificates
XSA also requires a full certificate chain. This means you need to supply the host, intermediate and root certificates all combined into a single x.509 PEM encoded file.
I was provisioning and internal SAP host using our internal Web Enrolment Service.
If you are an SAP employee or have access to the corporate network this can be found here https://sapcerts.wdf.global.corp.sap/WebEnrollment.aspx
This provides the certificate for the XSA host, but it needs to converted and combined with the full certificate chain. I download the appropriate root (SAP Global Root CA) and intermediate (SAPNetCA_G2), which come in .crt format.
1. Download the correct root & intermediate Certificates
2. Convert to x509 PEM format
openssl x509 -in SAPGlobalRootCA.crt -outform PEM -out SAPGlobalRootCA.pem
openssl x509 -in SAPNetCA_G2.crt -outform PEM -out SAPNetCA_G2.pem
openssl x509 -in mo-3f.txt -outform PEM -out mo-3f.pem3. Combine certificates in correct order (host, intermediate, root)
cat mo-3f.pem >> combined.pem
cat SAPNetCA_G2.pem >> combined.pem
cat SAPGlobalRootCA.pem >> combined.pem Upload to XSA
xs set-certificate mo-3fda111e5.mo.sap.corp -c combined.pem -k Private_pkcs8.keyAfter successfully installing the certificate you need to restart each of the XSA apps individually or the whole of XSA (which is easier). This can be dome with the hdbadm user.
mo-3fda111e5.mo:/usr/sap/IH2/HDB00> XSA restartAfter which I have XSA & WebIDE running with a signed SSL certificate.
Including all the new graphical Calculation View operators like graph, anonymization, union, minus, intersect, hierarchy function, etc.
Troubleshooting
There's a great Troubleshooting guide in the Help section on the WebIDE.
SAP WEB IDE for SAP HANA Installation Troubleshooting Guide
If you do experience issues the following commands are useful for identifying, potential issues.
xs logs product-installer --last 1000
xs logs webide --recent
xs logs di-core --recent
xs display-installation-logs <PID>If the certificates get messed up you can reset those using the hdbadm user
mo-3fda111e5.mo:/usr/sap/IH2/HDB00> XSA reset-certificateErrors
I have experienced some errors when the configuration was not correct. Often the exposed error is only the latest error and not the root cause of the problem for me I was initially seeing this generic error
xs display-installation-logs 99232 -scv
ERR com.sap.cloud.lm.sl.slp.client.communication.NotFoundException:I also found these errors, showing that a process has not started.
#2.0#2018 12 12 12:09:31.605#Z#INFO#com.sap.lm.sl.alm.26989.MAIN_LOG#
######com.sap.lm.sl.alm.prod.installer.process.steps.StepsUtil########pool-7-thread-2###
[PollStateStep] Unexpected error: Execution of step RestartAppStep has timed out#
#2.0#2018 12 12 12:09:31.626#Z#ERROR#com.sap.lm.sl.alm.26989.MAIN_LOG#
######com.sap.lm.sl.alm.prod.installer.process.steps.StepsUtil########pool-7-thread-2###
[PollStateStep] Unexpected error: Execution of step RestartAppStep has timed out#
#2.0#2018 12 12 12:09:31.663#Z#ERROR#com.sap.lm.sl.alm.26989.MAIN_LOG#
######com.sap.lm.sl.alm.prod.installer.process.steps.StepsUtil########pool-7-thread-2###
[PollStateStep] Installation of the component XSAC_SAP_WEB_IDE (sap.com) 4.3.60 failed during deployment.#
#2.0#2018 12 12 12:09:31.687#Z#ERROR#com.sap.lm.sl.alm.26989.MAIN_LOG#
######com.sap.lm.sl.alm.prod.installer.process.steps.StepsUtil########pool-7-thread-2###
[PollStateStep] Deployment of the MTA com.sap.devx.webide 4.3.61 failed. Check SAP release information note 2510063 and analyze the issue. In case you need help, contact SAP support using the SAP component HAN-WDE-PREF#
#2.0#2018 12 12 12:09:31.704#Z#INFO#com.sap.lm.sl.alm.26989.MAIN_LOG#
######com.sap.cloud.lm.sl.slp.steps.SLProcessStepHelper########pool-7-thread-2###
Deploy Service process instance [99232] will be aborted.#
#2.0#2018 12 12 12:09:33.837#Z#INFO#com.sap.lm.sl.alm.26989.MAIN_LOG#
######com.sap.cloud.lm.sl.slp.steps.SLProcessStepHelper########pool-7-thread-2###
Deploy Service process instance [99232] is aborted.#
#2.0#2018 12 12 12:09:33.837#Z#ERROR#com.sap.lm.sl.alm.26989.MAIN_LOG#
######com.sap.cloud.lm.sl.slp.steps.SLProcessStepHelper########pool-7-thread-2###
Exception caught#
java.lang.Exception: Unexpected error: Execution of step RestartAppStep has timed out
at com.sap.lm.sl.alm.prod.installer.process.steps.PollStateStep.pollStatus(PollStateStep.java:187)
at com.sap.activiti.common.impl.AbstractActivitiStepWithBridge.executeStep(AbstractActivitiStepWithBridge.java:59)
at com.sap.activiti.common.impl.AbstractActivitiStep.execute(AbstractActivitiStep.java:56)
at com.sap.cloud.lm.sl.slp.steps.AbstractSLProcessStepWithBridge.execute(AbstractSLProcessStepWithBridge.java:38)
at org.activiti.engine.impl.delegate.JavaDelegateInvocation.invoke(JavaDelegateInvocation.java:34)
at org.activiti.engine.impl.delegate.DelegateInvocation.proceed(DelegateInvocation.java:37)
at org.activiti.engine.impl.delegate.DefaultDelegateInterceptor.handleInvocation(DefaultDelegateInterceptor.java:25)
at org.activiti.engine.impl.bpmn.behavior.ServiceTaskJavaDelegateActivityBehavior.execute(ServiceTaskJavaDelegateActivityBehavior.java:49)
at org.activiti.engine.impl.bpmn.behavior.ServiceTaskJavaDelegateActivityBehavior.execute(ServiceTaskJavaDelegateActivityBehavior.java:40)
at org.activiti.engine.impl.bpmn.helper.ClassDelegate.execute(ClassDelegate.java:158)
at org.activiti.engine.impl.pvm.runtime.AtomicOperationActivityExecute.execute(AtomicOperationActivityExecute.java:60)
at org.activiti.engine.impl.interceptor.CommandContext.performOperation(CommandContext.java:97)
at org.activiti.engine.impl.persistence.entity.ExecutionEntity.performOperationSync(ExecutionEntity.java:648)
at org.activiti.engine.impl.persistence.entity.ExecutionEntity.performOperation(ExecutionEntity.java:643)
at org.activiti.engine.impl.pvm.runtime.AtomicOperationTransitionNotifyListenerStart.eventNotificationsCompleted(AtomicOperationTransitionNotifyListenerStart.java:52)
at org.activiti.engine.impl.pvm.runtime.AbstractEventAtomicOperation.execute(AbstractEventAtomicOperation.java:56)
at org.activiti.engine.impl.interceptor.CommandContext.performOperation(CommandContext.java:97)
at org.activiti.engine.impl.persistence.entity.ExecutionEntity.performOperationSync(ExecutionEntity.java:648)
at org.activiti.engine.impl.persistence.entity.ExecutionEntity.performOperation(ExecutionEntity.java:643)
at org.activiti.engine.impl.pvm.runtime.AbstractEventAtomicOperation.execute(AbstractEventAtomicOperation.java:49)
at org.activiti.engine.impl.interceptor.CommandContext.performOperation(CommandContext.java:97)
at org.activiti.engine.impl.persistence.entity.ExecutionEntity.performOperationSync(ExecutionEntity.java:648)
at org.activiti.engine.impl.persistence.entity.ExecutionEntity.performOperation(ExecutionEntity.java:643)
at org.activiti.engine.impl.pvm.runtime.AtomicOperationTransitionCreateScope.execute(AtomicOperationTransitionCreateScope.java:49)
at org.activiti.engine.impl.interceptor.CommandContext.performOperation(CommandContext.java:97)
at org.activiti.engine.impl.jobexecutor.AsyncContinuationJobHandler.execute(AsyncContinuationJobHandler.java:35)
at org.activiti.engine.impl.persistence.entity.JobEntity.execute(JobEntity.java:85)
at org.activiti.engine.impl.persistence.entity.MessageEntity.execute(MessageEntity.java:34)
at org.activiti.engine.impl.cmd.ExecuteJobsCmd.execute(ExecuteJobsCmd.java:88)
at org.activiti.engine.impl.interceptor.CommandInvoker.execute(CommandInvoker.java:24)
at org.activiti.engine.impl.interceptor.CommandContextInterceptor.execute(CommandContextInterceptor.java:57)
at org.activiti.spring.SpringTransactionInterceptor$1.doInTransaction(SpringTransactionInterceptor.java:47)
at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:133)
at org.activiti.spring.SpringTransactionInterceptor.execute(SpringTransactionInterceptor.java:45)
at org.activiti.engine.impl.interceptor.LogInterceptor.execute(LogInterceptor.java:31)
at org.activiti.engine.impl.cfg.CommandExecutorImpl.execute(CommandExecutorImpl.java:40)
at org.activiti.engine.impl.cfg.CommandExecutorImpl.execute(CommandExecutorImpl.java:35)
at org.activiti.engine.impl.jobexecutor.ExecuteJobsRunnable.handleMultipleJobs(ExecuteJobsRunnable.java:94)
at org.activiti.engine.impl.jobexecutor.ExecuteJobsRunnable.run(ExecuteJobsRunnable.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:836)
#2.0#2018 12 12 12:09:33.844#Z#DEBUG#com.sap.lm.sl.alm.26989.MAIN_LOG#
######com.sap.cloud.lm.sl.slp.steps.SLProcessStepHelper########pool-7-thread-2###
Step "Poll Deploy State" finished#Here di-core was not starting, due to a lack of allocated memory.
xs logs di-core -last 500
12/12/18 11:09:37.796 [APP/1-0] ERR #2.0#2018 12 12 11:09:37.796#+00:00#ERROR#com.sap.di.api.infrastructure.auth.HiabBuilderToCoreAuthenticationManager#
12/12/18 11:09:37.796 [APP/1-0] ERR ###4213DE1E966647AEA4B57BBB4EE6DB54###com.sap.di.api.infrastructure.auth.HiabBuilderToCoreAuthenticationManager########localhost-startStop-1#PLAIN##
12/12/18 11:09:37.796 [APP/1-0] ERR Received servlet context: org.apache.catalina.core.ApplicationContextFacade@153175d8
12/12/18 11:11:10.893 [API] ERR Please check state of app 'di-core' [Org 'orgname', Space 'SAP'] (droplet with id 1 of app 'di-core' [Org 'orgname', Space 'SAP'] created by buildpack 'sap_java_buildpack') as the portion of uptime 0.41 is below critical threshold.
12/12/18 11:11:40.878 [API] ERR Please check state of app 'di-core' [Org 'orgname', Space 'SAP'] (droplet with id 1 of app 'di-core' [Org 'orgname', Space 'SAP'] created by buildpack 'sap_java_buildpack') as the portion of uptime 0.33 is below critical threshold.
12/12/18 11:11:57.000 [RTR] OUT 10.44.103.208 - - to mo-3fda111e5.mo.sap.corp:53030 "POST /admin/builder/server/register HTTP/1.1" 503 sent 9709 in 208 by -
12/12/18 11:11:58.000 [RTR] OUT 10.44.103.208 - - to mo-3fda111e5.mo.sap.corp:53030 "POST /admin/builder/server/register HTTP/1.1" 503 sent 9709 in 207 by -
12/12/18 11:12:10.889 [API] ERR Please check state of app 'di-core' [Org 'orgname', Space 'SAP'] (droplet with id 1 of app 'di-core' [Org 'orgname', Space 'SAP'] created by buildpack 'sap_java_buildpack') as the portion of uptime 0.26 is below critical threshold.
12/12/18 11:12:40.888 [API] ERR Please check state of app 'di-core' [Org 'orgname', Space 'SAP'] (droplet with id 1 of app 'di-core' [Org 'orgname', Space 'SAP'] created by buildpack 'sap_java_buildpack') as the portion of uptime 0.21 is below critical threshold.Rescaling the application fixed this.
xs scale di-core -m 2G -fThe root cause for this was hidden deeper in the di-core logs, with an in-correct proxy configuration. Initially in the .mtaext I had ";" separators between my nonProxyHosts, switching to "|" fixed this.
Caused by: java.io.IOException: Unable to tunnel through proxy. Proxy returns "HTTP/1.1 403 Forbidden"
13/12/18 08:45:06.106 [APP/5-0] SYS at sun.net.www.protocol.http.HttpURLConnection.doTunneling(HttpURLConnection.java:2152) ~[na:1.8.0_181]
13/12/18 08:45:06.106 [APP/5-0] SYS at sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:183) ~[na:1.8.0_181]
13/12/18 08:45:06.106 [APP/5-0] SYS at sun.net.www.protocol.https.HttpsURLConnectionImpl.connect(HttpsURLConnectionImpl.java:162) ~[na:1.8.0_181]
13/12/18 08:45:06.106 [APP/5-0] SYS at org.springframework.http.client.SimpleBufferingClientHttpRequest.executeInternal(SimpleBufferingClientHttpRequest.java:75) ~[spring-web-4.0.9.RELEASE.jar:4.0.9.RELEASE]
13/12/18 08:45:06.106 [APP/5-0] SYS at org.springframework.http.client.AbstractBufferingClientHttpRequest.executeInternal(AbstractBufferingClientHttpRequest.java:48) ~[spring-web-4.0.9.RELEASE.jar:4.0.9.RELEASE]
13/12/18 08:45:06.106 [APP/5-0] SYS at org.springframework.http.client.AbstractClientHttpRequest.execute(AbstractClientHttpRequest.java:53) ~[spring-web-4.0.9.RELEASE.jar:4.0.9.RELEASE]Upon launching the WebIDE I have seen the response
Internal Server ErrorHere resetting the certificate resolved the issue. Beware this does also stop and restart XSA without any confirmation.
mo-3fda111e5.mo:/usr/sap/IH2/HDB00> XSA reset-certificate- SAP Managed Tags:
3 Comments
