Hello All,

Today I would like to talk about the Introscope Host Adapter (WilyHost application), an application which runs inside the SMDAgent to collect data from the managed systems. I already did a post showing how to check this application, but now I would like to provide some details about how this application works.

As I already mentioned, the Introscope Host Adapter is an application which runs inside the SMDagent, it is also known as SMDAgent agelet or WilyHost agent. The main goal of the WilyHost agent is to collect data from different configurable destinations and push them to the Introscope Enterprise Manager. The following picture provide a big picture of the architecture of the wilyhost agent.

/wp-content/uploads/2014/05/architecture_449951.png

All applications running inside the SMDAgent can be found win the Agent Administration UI. The WilyHost Agent can be found as shown in the below picture.

  1. Agent Administration UI
  2. Application Configuration
  3. com.sap.smd.agent.application.wilyhost

/wp-content/uploads/2014/05/app_config_449952.pngAll resources used by the WilyHost application can be found under the ‘Application Resources’ node:

/wp-content/uploads/2014/05/app_config_resource_449953.png

N.B.: When using the application configuration take care when performing changes. Do not perform changes in the scope GLOBAL if you do not know exactly what you are doing. The changes in scope global affect ALL SMDAgents connected to the Solution Manager system.

In Solution Manager 7.10, the WilyHost Agent is configured during the activity ‘Introscope Host Adapter’ of the Managed System Configuration for the target system. The following picture shows the exact location:

/wp-content/uploads/2014/05/msc_config_449954.png

When the activity Introscope Host Adapter is performed, the wilyhost agent application is configured to collect data from the managed system that will be monitored. The main files used by the agent are the following:

  • Files create/modified by setup:
    • SAPAgentConfig.xml
      • File created during the Wily host agent setup step. It contains the destinations configuration and the link to actions to be performed for each destination
    • IntroscopeSAPAgent.profile
      • File is created from the IntroscopeSapAgent.profile.template. The host and port of the Enterprise Manager are configured in this file.
  • Files used by application:
    • WilyHostAgentTemplate.xml
      • The is a global template file that defines all the possible actions possible for collecting all the Wily Host Agent data.
    • WilyHost.xsd
      • The is the XML schema file that defines the rules of validation for the WilyHostAgentTemplate.xml file. If you modify the action template it needs to conform to the schema.

N.B.: No manual configuration must be performed to the above files. The activity Introscope Host Adapter will configure the WilyHost application automatically.

The file SAPAgentConfig.xml holds the configuration of the Destinations and the Actions that will be used/performed to collect data from the managed system.

  • Destinations: The destinations will point out where the information will be collected. The destinations are configurable, for example a destination might be a Jco connection to the managed system or the location of a specific file.

For a typical double stack system you will have configurations generated for the following destinations:

    • ABAP instance
    • JAVA instance
    • SAPStartSrv
    • SAPHOSTAGENT
    • File (for GC)

The following picture shows an example of a Jco destination. As you can see, the Jco destination has the connection information to the managed system, client, user, password, system number and hostname of the application server.

/wp-content/uploads/2014/05/destination_example_449955.png

  • Actions: The actions indicates to the WilyHost Agent what are the metrics to be collected from the managed system. For all destinations in the SAPAgentConfig.xml a pool of actions are configured. The actions point to an action templates that are defined in the WilyHostAgentTemplate.xml file.

The following picture shows an example of some actions defined in a SAPAgentConfig.xml.

/wp-content/uploads/2014/05/action_example_449956.pngA SAPAgentConfig.xml fully configured will look as follows:

/wp-content/uploads/2014/05/sapagentconfig_449957.png

The following picture describes how the destinations and actions work in the managed system. As you can see, the destinations are the location where the data will be collected and the actions are the collections of data.

/wp-content/uploads/2014/05/architecture_actions_destinations_449958.png

The file IntroscopeSAPAgent.profile holds the connection configuration to the Introscope Enterprise Manager. This file will tell the WilyHost application what is the Enterprise Manager that the wilyhost application must connect. The activity Introscope Host Adapter in the Managed System Configuration uses the IntroscopeSapAgent.profile.template file and then creates the IntroscopeSapAgent.profile. The following picture shows an example of IntroscopeSapAgent.profile.

IntroscopeSapAgent.png

Troubleshooting


If you find out that there is some issue with the wilyhost application, for example: agent is not connected to the Enterprise Manager or managed system metrics are not being collected. Then the SMDAgent logs must be checked to find out what is the root cause of the issue. The wilyhost application may not work correctly due to distinct issues and the SMDAgent logs will show the root cause.

The SMDAgent logs can be downloaded using the Agent Administration UI:

/wp-content/uploads/2014/05/agent_admin_449960.png

The logs related to the wilyhost application are the following:

Logging information Log File

SMD Agent System log file

SMDSystem.X.log

SMD Agent Application log file. All of the interesting logging for the Wily Host Agent will be found here

SMDAgentApplication.X.log

Host Agent Information when trying to connect to the Enterprise Manager

/usr/sap/SMD/SMDA<xx>/work/jvm_SMDAgent.out
RFC Connections /usr/sap/SMD/SMDA<xx>/SMDAgent/dev_jrfc.trc

1) How to check if the WilyHost agent is connected to the EM: Check the jmv_smdagent.log, this file shows the information about the connection between the wilyhost application and the Enterprise Manager. If the WilyHost application cannot connect to the EM the jvm_smdagent.log will show the root cause. The following picture shows an example of log entry for a successful connection:

/wp-content/uploads/2014/05/jvm_smdagent_449961.pngIt is also possible to check if the wily host application is connected to the EM directly in the Introscope Investigator. By default the wilyhost application connects to the EM using the process name = ‘SAP HostAgent Process’ and agent name ‘SAP HostAgent <SMDAgent_instancename>’. The following picture shows the same agent of the above log entry connected to the EM.

/wp-content/uploads/2014/05/investigator_449962.png2) How to check if actions are being collected: Compare actions defined in the SAPAgentConfig.xml with information available in the Wily Investigator. You should see nodes in the investigator with the same prefix of the actions.

actionsXinvestigator.png

If you find out that some actions are not started, e.g. workload analysis is missing, nodes in the introscope investigator are not available. Then the SMDAgentApplication.log must be checked. Usually, the actions are started when the SMDAgent process is started. So, to check issues with the actions it is good to restart the SMDAgent. It can be done in the Agent Administration UI as follows:

/wp-content/uploads/2014/05/reboot_agent_449964.png3) Example ABAP Instance action cannot be started: You see that you have no ABAP data in Introscope or in workload. You download the SMD log files and you see that you have a JCO issue. The SMDAgentApplication.log shows that the user is locked:

/wp-content/uploads/2014/05/abap_jco_issue_449965.png

Solution: The WilyHost application uses the SMDAGENT_<Solman_SID> to connect to the managed system. In this case, it is necessary to check the password of user SMDAGENT_<Solman_SID> and unlock the user. This can be performed in step Create Users of the Managed System Configuration for the affected system.

4) Example SAP GC action cannot be started: You see that you have no SAP GC instance data in Introscope or in workload. The SMD Agent logs shows the following errors:

/wp-content/uploads/2014/05/gc_action_1_449966.png

/wp-content/uploads/2014/05/gc_action_2_449967.png

Solution: The SAP GC is collected from file std_server* which is located in the managed system work folder. In this case, it is required to ensure that the path to the files is correct. It is possible to compare the file shown in the log error or in the action definition in the SAPAgentConfig.xml. If the path is wrong, check the paths provided in step Enter Landscape Parameters in the Managed System Configuration for the affected system.

I hope you enjoy this post.

You can leave questions below, I will try to answer them as soon as possible.

Thanks / Obrigado

Allam Drebes

To report this post you need to login first.

11 Comments

You must be Logged on to comment or reply to a post.

  1. Tom Cenens

    Hi Allam

    Nice blog post.

    I would recommend you to move it to the Solution Manager space though on SCN as this will be of interest to a good amount of community members but it’s rather hard to find as it resides in your personal blog space at the moment. In edit mode, you find an option in the right menu to move the blog post.

    Best regards

    Tom

    (0) 
  2. Paulene Siew

    Hi Allam,

    Thank you for such awesome blog.  Do you know if it is possible to fine tune the time interval (default from your diagram is 15s) of collections from wily host agent to the managed system? My customer feedback to me that too many smdagent_<sid> processes and it goes give a bit of load to the system.

    sometimes, when the system hang despite it is due to the db issue, most of the time they see smdagent_<sid> processes on the SM50 and thus presume the issue caused by smdagent.

    If you know how we can fine tune this would really help the customer.

    Thanks

    Paulene

    (0) 
    1. Tom Cenens

      Hi Paulene

      To troubleshoot further, I would recommend to enable debug mode on certain log files. Then you’ll see all the nitty gritty details which can help you.

      Best regards

      Tom

      (0) 
    2. Allam Drebes Post author

      Hello Paulene,

      Sorry for the delay, I didn’t have time to answer you before.

      Firstly, the RCA applications should not cause a high load in the managed systems. Only in rare cases it is required to modify the metric collection configuration. Also, there are some corrections available that decreases the impact, for example note 1956818.

      1956818 – Increasing ABAP memory usage by /SDF/IS_PROXY

      Regarding the metric collection:

      Not all metrics are collected every15s, the fact is that the Introscope EM is updated every 15s by the agents. For the SMDAgent, the metrics collection is defined in the WilyHostAgentTemplate.xml, each metric group defined in this file have a specific period to run.

      Anyway, it is not recommended to perform manual changes in this file.

      I hope this helps you.

      BR,

      Allam

      (0) 
  3. Robin Singh

    Hi Allam,

    Once again a masterpiece, seriously!! Very insightful blog. Thanks for your efforts and time.

    I would be grateful if you can help me out with one last piece of the monitoring puzzle. What is the role of SAP hostagent. How are the various processes within it namely : saposcol, saphostexec and saphostctrl functioning with respect to each other?

    Thanks for your help.

    Robin Singh

    (0) 
    1. Allam Drebes Post author

      Hi Robin,

      The SAP Host Agent provides several functions in distinct web services that are used by the SMDAgent.

      For example, the host information CPU, Memory, Disks is provided in the SAPOsCol web services. The SMDAgent gets the host information consulting this web services provided by the SAP Host Agent. The Web Service SAPHostControl provided by the SAPHostAgent provides information about the instances and DB status.

      There are other several services provided by the SAP Host Agent that are used by the SMDAgent. Almost or maybe all host related information is provided by the SAP Host Agent.

      I hope this clarify the SAPHostAgent role in the Diagnostics and Monitoring applications.

      Kind Regards,

      Allam Drebes

      (0) 
  4. Aniket Singh

    Hi Allam,

    This blog was indeed helpful in understanding the methodlogy for deployment of Bytecode agent adapter.

    I had one query regarding the integration of EM and Solman in case our EM host is running on a different server than a Solman server. What is the way to achieve it.

    Do we need to deploy SMD agent on EM host as well to achieve this communication between EM and Solman. If yes, what is the steps that we need to follow.

    Regrads,

    Aniket

    (0) 
    1. Allam Drebes Post author

      Hi Aniket,

      The Enterprise Manager can be installed on any server you want, the solution manager supports it. It is not required to install the EM in the Solman server only.

      To perform the administrative tasks, the solution manager requires a SMDAgent installed on the Enterprise Manager host. This is described in the Introscope Release notes:

      1565954 – Introscope 9 Release Notes

      2071100 – Introscope 9.5 Release Notes

      2138309 – Introscope 9.7 Release Notes for changes and open issues

      So basically you need to:

      1) Install the EM on the target host

      2) Install a SMDAgent in the EM host and connect it to the Solman as usual

      3) Discover/Register the EM in the Basic Configuration

      Regards,

      Allam

      (0) 

Leave a Reply