No Java workload Analysis – example of a troubleshooting session + troubleshooting script (bonus)
The files on yellow were empty (0 bytes). These files log all Introscope Agent activity. If they are empty, it means that no activity was being done by the Agent. The ISAGENT collects Java data.
The Byte Code Adapter was OK ( with green status ) and the j2ee node details were showing correct ISAGENT parameters
The system was restarted, but the agent didn’t work.
I re-executed the setup, which pushes the parameters to the server nodes once more, and I restarted the system once more. This solved the issue.
The issue was that the setup was done but the Java stack was never restarted. The parameters were in place, but a restart is needed to make the ISAGENT effective. Currently, there’s no automated way to detect whether the system was restarted, but the managed system configuration has a mandatory step telling you to restart the system.
Troubleshooting missing workload analysis is easy!
I have elaborated a troubleshoot script which is 99% infallible: it will either solve your issue or it will lead you to the root cause. The items are ordered by more to least common issues, based on my experience with past and recent incidents.
1- Check the Managed System Configuration, all steps must be green. Many issues in the setup engine are either corrected by the latest LM-SERVICE (SOLMANDIAG) patch level or by the latest Basic Configuration note.
2- Check if your Introscope Agent should NOT be on version 22.214.171.124 patch level 0. How to check this? Go to the Component Info page (http://solman:port:8000/monitoring/ComponentInfo) and check whether your ISAGENT is 8 SP24 (126.96.36.199.20120127XXXXXX) where XXX does’t mind, what is important here is the DATE. 20120127, that is, 01/17/2012. This is the buggy version as documented in SAP KBA 1793553. If you have the faulty version, update it and follow the KBA’s instructions.
3. There are common issues related to missing FQDN. For that, you must follow the SAP note 1611483 – SolutionManager7.1 – Outside Discovery common error messages. You must download the PDF inside this note (SM71SP5OutsideDiscMissingFQDN.pdf) , which guides you to put the E2EDCC_HOST file on debug mode so you can check if the outside discovery is finding your FQDN. If there are issues, they are treated in the PDF, which contains all solutions. In some cases, the issue is caused by wrong HOSTS format. For some reason, some vendors recommended a non-RFC-compliant HOSTS format, and that needs to be corrected. I’ve written a KBA on the subject -> 1938013 – Troubleshooting Missing FQDN Issue – FQDN issues due to /etc/hosts file
4. Check if the Introscope Agent is running OK. You can use the file system browser, as I included in my example, or navigate directly to the filesystem. The file is located at /usr/sap/DAA/SMDA97/SMDAgent/temp (example directory) and the filename starts with IntroscopeAgent. Open the file and read what it says. Check if there are recent messages. The file may be populated, but the latest information may be from days or weeks ago, which means that for some reason the ISAGENT is no longer working. The ISAGENT loads with the Java stack, and it doesn’t unload itself, it keeps running. If there’s no activity in the logs, then the ISAGENT didn’t load with the startup.
5. Check your Wily EM. You should see the server node listed and you should see the SAP Netweaver node, JMX, J2EE metrics, etc.
6. Check if your Wily EM is green in the solman_setup -> basic configuration
7. Check the system’s extractors and look for errors
8. Check the extraction jobs running. Go to SM37, insert EFWK RESOURCE MANAGER in the job name and * in the user name field. Double click the entries, check the job log and, ideally, you should check the job spool (if available)
9. Check SM50 to evaluate whether the extraction jobs are running correctly. A performance problem can lead to delayed data extraction, leading to missing workload analysis data. Check my SAP KBA Note 1835721 to troubleshooting your performance problem.