Skip to Content

In previous versions of the BI Platform (Seagate Info, Crystal Enterprise, BusinessObjects Enterprise) one of biggest issues for administrators, consultants, and support engineers has been the lack of tools needed to see “under the hood”.  With such a complex, heterogeneous, and varied landscape the task of finding performance bottlenecks amongst many critical components can be a time consuming and often times frustrating mission.  It is with great pleasure that I can now say the process of diagnosing performance issues has been greatly simplified in the SAP Business Intelligence Platform 4.0.

Since the merger of SAP and BusinessObjects, the BI Platform has been updated to be supported by the amazing SAP Solution Manager product.  SAP Solution Manager is an application lifecycle management platform that provides many different capabilities from implementation blueprints and solution documentation to change management, business process monitoring, and much much more.  One of the key tools that Solution Manager integrates into it’s monitoring and diagnostics suite is Computer Associates Wily Introscope.  From this partnership with CA, SAP is able to provide customers (with a standard or enterprise support contract) the ability to download and implement a read-only version of Wily Introscope in their landscape.  This version is known as the “right to view” version and is free of charge.

While I highly recommend implementing SAP Solution Manager in your landscape, it may not make sense to implement the entire Solution Manager suite if you do not yet have other SAP products such as (BW, ECC, etc).  The great news is that you can now implement Wily Introscope for BI Platform 4.0 in a standalone configuration and still take advantage of the framework that the Solution Manager Diagnostic Agent provides.

In terms of product supportability, Wily Introscope will help you:

  • Diagnose tricky performance problems
  • Spot bottlenecks that may lead to system downtime
  • Resolve issues that result in poor end user experience
  • Improve supportability (close messages and solve bugs faster)
  • Ensure proper scalability of your BI landscape
  • Predict future growth through trend analysis
  • Measure the results of load testing prior to product launch

Ok, you might be thinking “Hey, BI 4.0 already comes with monitoring… why do I need or want Wily Introscope?”.  This is certainly a valid concern however it just so happens that Wily Introscope is a perfect complement to the monitoring and probe framework that ships with SAP Business Intelligence Platform 4.0.

Consider the following diagram which shows the overlap of features from both monitoring solutions.

/wp-content/uploads/2012/08/compare_125044.png

The features shown in green are those provided by BI Platform 4.0 and those in pink are features provided by Wily Introscope.  The features shown in the intersecting section are those included with both monitoring solutions.  Generally, the tools included with the BI 4.0 monitoring suite are more geared for day to day administrative activities whereas the tools included with Wily Introscope are more suited for in-depth diagnostics and root cause analysis.  This is of course a generalization and both tools can most certainly be used for both purposes.

In the table below I review the unique features from both solutions and discuss the criteria and delta that makes it unique.

/wp-content/uploads/2012/08/sapsmall_126800.png

Feature Description of BI 4.0 Feature Introscope RTV Solution Delta
Probes

Probes allow the BI administrator to programatically simulate workflows that are critical to the proper functioning of the system.

Not available in Wily Introscope
Alerts

Alerts can be created based on the status of watches (thresholds) that are defined by the administrator.  Alerts can be sent to SMTP or BI Alert Inbox destinations

Alerts cannot be created in the “right to view” version of Wily Introscope.  You must upgrade to the SED package (SAP Extended Diagnostics) or install SAP Solution Manager to get access to alerts based on Wily data.
Trend Universe In BI4.0 monitoring you can create reports based on historical data.  A universe is included that simplifies this process. No universe exists for Wily SmartStor database.  Direct data access is possible via Web Services SDK and JDBC driver.
Derived Metrics Custom metrics can be created by the administrator Custom metrics can only be created in Introscope by upgrading to the SED package (SAP Extended Diagnostics)
Topology Metrics & UI Topology metrics in BI.40 can quickly show the administrator the overall health of Service Categories (e.g. Crystal Reports Services) or Server Groups N/A

/wp-content/uploads/2012/08/wilya_126803.png

Feature Description of Introscope Feature BI4.0 Monitoring Solution Delta
Granular code level monitoring Wily Introscope exposes performance data at the function/method call level via instrumentation introduced in XI 3.1 and BI 4.0. BI4.0 exposes pre-defined server metrics, not actual code level function calls.
E2E Tracing With Wily Introscope it is possible to monitor a user workflow across all components in the BI landscape to quickly diagnose performance bottlenecks Metrics in BI4.0 are specific to one component.
Application Server monitoring Introscope excels at monitoring java applications including application servers such as Tomcat, Websphere, Weblogic, Netweaver, etc Not possible in BI4.0 monitoring solution.
Rich user interface Introscope provides a very mature and user friendly GUI for analyzing performance problems and mining for data Interface is not as friendly or feature rich
Live Error Viewer Introscope provides a client that detects exceptions as they occur across the entire managed BI landscape Doesn’t exist in BI4.0 monitoring solution

Conclusion

As can be seen from the feature delta, each monitoring solution excels in certain areas and compliments the other in other areas.  Implementing the two solutions in tandem will yield the most flexible and feature rich solution without introducing additional cost factors.  For an excellent look at the BOE dashboards, check out Tim Coffman’s blog Diagnostics in SAP BI 4.0 – Pillars of Monitoring – Metrics.

In addition to little or no cost, the time factor to deploy Introscope in your BI landscape is also much less than you would expect.  With a two system sandbox environment I was able to fully install and deploy Introscope in about three hours of working time.  The key here is to know where to find the right documentation and follow it verbatim.

Here are the resources you will need to get started:

Planning your Wily Introscope Enterprise Manager Installation

First, you will want to locate a server that you can use as your dedicated Wily Introscope Enterprise Manager system.  It is recommended that this system have at least 2-4GB of available ram and plenty of hard disk space.  The longer you wish to store historical data the more hard disk space you will need.  By default, Wily will store data in the SmartStor database for a default of 30 days.  Check out this knowledge base for instructions to extend the storage of data for 1 year.  Additionally, for optimal performance and responsiveness, Computer Associates recommends that you have a dedicated hard disk partition for your SmartStor database.  For estimating disk space, a sizing calculator is also included in the /docs directory of your Wily Introscope Enterprise Manager installation.  For more information on this topic, refer to the Introscope Configuration and Administration Guide.

Downloading and Installing Wily Introscope

The SAP note Note 797147 – Wily Introscope Installation for SAP Customers will guide you through the download and installation of Wily Introscope Enterprise Manager.   If you are unable to find Wily Introscope on the software download centre, try going to support packages and patches and then use the search term “Introscope”.  Download the version listed as Introscope Enterprise Manager 9.

Note that you will also need to download the eula.txt (replace ‘reject’ with ‘accept’ to accept the license agreement) and OSGI zip package for your version of Wily Introscope.  These files must be placed in the same directory as your Wily Introscope installer otherwise the install will fail.

Deploying the Management Modules (BOE Dashboards)

To prepare Wily Introscope to consume metrics from BI Platform 4.0 you must download and deploy the management modules that were designed for this product version.  Complete instructions are covered in the note Note 1540591 – Wily Introscope Setup for SAP BOE 4.0

Configure your BI Platform 4.0 landscape

The last step to implement Wily Introscope monitoring is to configure each node in your BI Platform 4.0 landscape.  For each node (including your Web Application Tier) you will need to install and configure the Solution Manager Diagnostic Agent in standalone mode.  The SMD Agent provides operating system level metrics via the SAP Host Agent and SAPOSCOL application which are included with the SMD Agent installation.  Furthermore, by installing the SMD Agent, you will also be able to configure Host Metrics inside your BI Platform 4.0 monitoring application.  Follow the instructions in the document Standalone_SMD_Agent_setup.pdf to install and configure the SMD Agent.

Next, configure your Tomcat application servers for monitoring per SAP note 1438005 Wily Introscope Agent for Apache Tomcat Server and then add the BOE 4 instrumentation per the document ISAgentBOE40WebApp.pdf which is attached at the end of note 1540591. If you are using SAP BusinessObjects Explorer, you may also implement additional monitoring and dashboards per the instructions in the document MM_for_BOE40_BOExplorer40_DS40.pdf.

Lastly, you need to finish configuring the Introscope placeholders for your SIA nodes in the Central Management Console.  Refer to section 25.5.2.1 in the BI 4.x Administrator’s Guide for instructions on configuring the Introscope placeholder values.

After successfully accomplishing these tasks your Wily Introscope setup for BI Platform 4.0 is complete!

I hope you found this blog helpful.  Keep an eye on the Remote Supportability SCN space for more information on remote supportability and diagnostics for SAP Business Intelligence products.

Happy monitoring!

To report this post you need to login first.

36 Comments

You must be Logged on to comment or reply to a post.

  1. David Gilbertson

    Hi Toby,

    Nice post. Has much changed in BI4 SP4? I know there were some decent changes in the monitoring area, but is it still a very little brother to Wily?

    (0) 
    1. Toby Johnston Post author

      Hi David,

      Thanks for the feedback.  Yes, in Support Pack 4 there were some very good improvements in the native monitoring application.  Check out the SP04 What’s New Document for a complete breakdown.

      The delta’s described in this article are based on the SP4 monitoring application.  So, I’ll leave it up to you to decide whether it’s the little brother to Wily.  In my opinion, the two play quite nicely together.

      Cheers

      Toby

      (0) 
      1. Michael Moloney

        Thanks Toby, I have had many questions from customers about the inbuilt tool, and how it compares to Solution Manager. This is very useful.

        Regards,

        Michael.

        (0) 
  2. Bo Xu

    Hi Toby,

    great post, thumbs up and book marked. 😎

    amazing to see what a complete set of info it contains.

    Cheers

    Angela

    (0) 
  3. Shawn Gagnon

    Nice blog Toby! This will help a lot with the BI 4.0 customers that are searching for a diagnostic solution and aren’t ready for Solution Manager.

    (0) 
  4. Greg Myers

    Followed these instructions over the last few days and was successfully able to stand up Wily 8 for BI4! It would have taken much longer if I didn’t have all of the needed info curated here. Thanks Toby!

    (0) 
  5. Vlad Mily

    Hi Toby,

    Would you know the difference between Remote Support Component 1.0 (RSC) and Wily Introscope for BI 4.0 installation described in your article? Does RSC provide any additional functionality? Thank you. Vlad

    (0) 
    1. Toby Johnston Post author

      Hi Vlad,

      Thanks for the comment.  Remote Support Component is not yet available for BI4.0 but is planned for sometime next year.  In terms of XI3.1, RSC provides a vehicle for autodeploying Wily Introscope (features are very similar with exception to BOE InfoStore metrics), plus it also provides automated EarlyWatch Alert Reporting.  In BI4.0, I believe the plan is for RSC to provide many more diagnostic features that you find today with Solution Manager although it is still too early to say which features will be included at release.

      Thanks,

      Toby

      (0) 
  6. David Duan

    Hi Toby,

    I spend some time and translated into Chinese and posted in SCN Chinese community with keeping your original link. Hopefully this helps customers start using it.

    It is a such good document.

    (0) 
    1. Toby Johnston Post author

      Thanks David!  Very soon I will move this information to a pattern book format, I think this should make things even easier in the future.

      (0) 
  7. Samik Sarkar

    Hi,

    Nice post. I have a couple of questions on this. I have setup Wily for BI platform as well as Tomcat and everything looks good in Investigator mode. I could find BI dashboards for webview (after deploying the management modules). However, the webview for Tomcat Apache Server not working. Do we need separate management modules for this ?

    Also,about EWA report. the reports are generated, but it does not have any data. Only 1 graph of java Heap Usage is displaying. Can you post any sample EWA for a BI 4 system ?

    Do we need to separately include the Web Application server in a Logical Component ?

    Thanks,

    Samik

    (0) 
    1. Toby Johnston Post author

      Hi Samik,

      You need to change your Tomcat to use the IntroscopeAgent_tomcat.profile and in the profile you should have the following directives.

      introscope.autoprobe.directivesFile=sap_typical.

      jar,sap_tomcat_typical.jar,sap_tomcat_full.jar,sap_boe_webapp.jar

      Solution Manager 7.1 SP3 or higher is required for BI4 EWA.  See note 1646341

      I’ll see if I can find a sample EWA and post

      (0) 
      1. Samik Sarkar

        Thanks Toby, appreciate your response.

        I remember changing that value, but for the different IntroscopeAgent.profile. I have changed the value in Tomcat profile now ( I found this profile in 2 places, /usr/sap/ccms/wily and also E:\SAP_BusinessObjects\SAP BusinessObjects Enterprise XI 4.0\java\wily\webapp).

        I changed in both the profiles. Waiting to restart the apps.

        introscope.autoprobe.directivesFile=../sap_typical.jar,../sap_tomcat_typical.jar,..sap_tomcat_full.jar,../sap_boe_webapp.jar

        We have Solman 7.1, SP7. So should be good for EWA. All we are getting in the EWA is a list of components with patch details and 2 graphs showing Java Heap usage. We configured yesterday, does it take some time to build the reports with all graphs. The extractors are running OK.

        Thanks in advance.

        (0) 
      2. Samik Sarkar

        One more thing I observing, I have data in my E2E Workload Analysis, I can see graphs. But no data for Apache Tomcat in my EWA for BOE Solution. No mention of it at all. Do I need to include Tomcat into a logical component, it only gives option to add the BI Platform system SID_BOE.

        (0) 
      3. Samik Sarkar

        Hello Toby,

        I got my EWA working now. Can see all graphs. Can you tell me if the generated EWA report will have any rating, currently they font have any rating (Gray).

        Thanks.

        (0) 
        1. Toby Johnston Post author

          Hi Samik,

          It sounds like there is an issue with the configuration.  Please open a message under component SV-SMG-SER-EWA and this team will help you correct the problem.

          Cheers,

          Toby

          (0) 
  8. David Duan

    Hi Toby and all,

    I have only set up this for two customer’s actual production system. One customer is just okay to use it but we observed system performance became slow down. Another customer was using BI4+HANA and after we activated everything the system hang very frequently. Do you have any experience on it?

    (0) 
    1. Toby Johnston Post author

      Hi David,

      I’ve configured Wily 8 for XI 3.1 and BI4 on literally hundreds of different landscapes and never noticed a performance hit higher than 4% or 5%.  Usually it’s around 1% or 2% at the most.  Also I’ve never had a system intermittently hang from Introscope. 

      Are you sure the performance and/or hanging issues are related to Introscope?  Please open a support message for each issue if the problem persists.

      (0) 
  9. Samik Sarkar

    Hello Toby,

    Thanks for such a nice blog. However, encountering an issue with my bobj setup where need some help.

    The entire managed config/Wily/EWA set up is fine for our BOBJ BI4 system. Suddenly the SMD Agent started to eat up the entire CPU of my BOBJ server, I have stopped it, but whenever I start the java.exe process (running under SAPServiceSMD) and saspstartsrv.exe running under sapadm consuming all CPU causing 100%.

    How to troubleshoot ?

    Thanks

    (0) 
    1. Toby Johnston Post author

      Hi Netweaver Consultant,

      Which of the two processes is consuming the CPU?  I’d have a look inside the SAP Management Console, there will be helpful diagnostic information and logs in there about the state of the agent’s processes.  Is the problem specific to one node in your landscape or multiple nodes?  If the problem persists I would open a support message on this, it should be investigated.

      Thanks for the comment.

      (0) 
  10. Vikram Veeranagari

    Hi Toby,

    Thanks for this really great post. 

    Reading through the post I could find instructions to install the Introscope agent for Web App Server however if a BI4 environment is distributed with Tomcat on one host and BOBJ services (Server Intelligence Agent) on another host, what is the procedure to install/configure Introscope agent on the server hosting SIA.

    The BI4 installation guide reads the following:

    6.5.3 To configure CA Wily Introscope Agent post installation

    If you did not configure CA Wily Introscope during the installation process, you can configure it later in the Central Management Console (CMC).

    Note:
    The Introscope Agent must be installed and running before configuring it in the CMC.

    1. Open the Central Management Console.
    2. Select the Servers tab.
    3. Expand the SIA Nodes folder in server list, and right-click the SIA to be updated.
    4. Select “Placeholders” from the context menu.
    5. Ensure that the placeholders related to Introscope are set correctly:
    a. Change the %IntroscopeAgentEnableInstrumentation% placeholder from false to true.
    b. Update the %IntroscopeAgentManagerHost% placeholder with the Introscope Agent hostname.
    c. Update the %IntroscopeAgenEnterpriseManagerPort% placeholder with the Introscope Agent port number.
    d. Review %IntroscopeAgenEnterpriseManagerTransport% to ensure that the correct network transport is selected (for example, TCP).
    6. Save and close the “Placeholders” screen.
    7. Restart the SIA.
    8. When more than one SIA is present, repeat steps 3-7 for each SIA in the Nodes folder.
     

    I am looking for instructions to achieve this. Is the agent for this (SIA) different from the one for Web App Server? Is there a windows executable to install this agent? If yes, Can you guide me to its download?  

    We are using Wily Introscope Enterprise Manager 9.1.0 and our BI4 SP4 is installed on Windows 2008.

    Thanks in advance!

    -Vikram.V

    (0) 
    1. Toby Johnston Post author

      Hi Vikram,

      Here is the answer to your question

      “What is the procedure to install/configure Introscope agent on the server hosting SIA?”

      The Introscope Java Agent is included with BI4 under <BOBJ_HOME>/java/Wily.  For Java based BOBJ servers (APS, AJS, etc) they are already configured to be monitored.  You need to just edit the placeholders in the CMC and the IntroscopeAgent.profile is automatically maintained for you.  For c++ based BOBJ server (CMS, CRProcServer, etc) and OS metrics, you must install the SMD Agent per the instructions referred to in my blog.

      Hope it helps.

      Thanks

      Toby

      (0) 
      1. Vikram Veeranagari

        Thanks a lot Toby for responding to my post.

        However I’m left with one last query. The instructions to edit the SIA placeholders are as follows:

        a. Change the %IntroscopeAgentEnableInstrumentation% placeholder from false to true.

        b. Update the %IntroscopeAgentManagerHost% placeholder with the Introscope Agent hostname.

        c. Update the %IntroscopeAgenEnterpriseManagerPort% placeholder with the Introscope Agent port number.

        What would be the Introscope Agent port number? Are they referring to the Introscope Enterprise Manager port and host?

        -Vikram.V


        (0) 
        1. Toby Johnston Post author

          Hi Vikram,

          Introscope Agent port number refers to the Wily Introscope Enterprise Manager agent listening port.  So by default this value is 6001

          Thanks

          Toby

          (0) 
  11. Toby Johnston Post author

    For those interested, I’ve created a note which includes a shell script for executing a silent install of the SMD Agent in stand-alone mode

    1825993 – How to perform a silent install of the SMD Agent for BI Platform 4.0 in stand-alone mode

    (0) 
  12. Vikram Veeranagari

    Hi Toby,

    Thanks for the note. Following it I was successfully able to install SMD agent in stand-alone mode. Now I’m able to see all the BI4 server metrics in the Introscope workstation.

    However I’m having a tough time understanding these metrics and also in configuring alerts. Is there any documentation that can help me on this?

    We are using a licensed version of Wily EM.

    Also the transaction tracer doesn’t show any events. It always reads “Traced 0 transactions”. For the transaction trace session I chose the tracing agent as “SAP HostAgent SMDA98”. I performed refresh of a couple of WEBI reports after starting this session but no events being shown in that session.

    And in one of my other landscape I installed SMD agent with SLD registration and connection to SOLMAN. However the workstation doesn’t show the node for the SAP HostAgent Process i.e., the Metrics for c++ based servers are not showing up. Only the node for Java based servers is showing up? Is this by design? Will it show up only after performing the Managed System Setup?

    Appreciate your help on this.

    -Vikram.V

    (0) 
    1. Toby Johnston Post author

      Sorry I missed your comment and thanks for posting.

      What filter did you use in the Transcation Tracer?  Also, for Webi you must also have the Java Agents instrumented as per this blog.  You will get trace from BILaunchpad webapp (cadenza) and also traces from Webi/Connection Server API calls (Webi Processing Server). 

      For Solution Manager, yes the metrics will not show up until you run the Managed System Setup.

      Regards,

      Toby

      (0) 
  13. Walter Tran

    Toby,

    Great article. You’re still helpful as ever, I remember when I had the great opportunity to work with you during one of our issues and you had worked with me to implement RSC for our XI 3.1 environment.

    And I’ve followed this to implement SMD/Introscope agent in a 4.0 environment, but I’m still facing the same problem with RSC, and that is this information is too granular and requires in-depth knowledge of the code that goes into BOE.

    The idea of this is great, but seems like you need prior knowledge of BOE SDK… Given that the non-boe specific metrics are easier to consume, like host info, etc.

    What would be the best way to even go about understanding the more BOE specific metrics and to be able to properly analyze them? Is the only way to just get familiar with SDK?

    Thanks again for all your sharing and hard work! Keep it up!

    (0) 
  14. Thomas Neel

    Hi,

    We have configured wily introscope web view for bi 4.1 sp 5.

    Can we be able to measure the network performance , latency, throughput, stress testing?

    Thanks

    (0) 

Leave a Reply