Technology Blogs by SAP
Learn how to extend and personalize SAP applications. Follow the SAP technology blog for insights into SAP BTP, ABAP, SAP Analytics Cloud, SAP HANA, and more.
cancel
Showing results for 
Search instead for 
Did you mean: 
Raghav_S
Employee
Employee
With 7.1, the capabilities of central monitoring has immensely improved in Solution Manager. For setting up of monitoring of systems in the BI landscape and objects in these systems, we have certain options available now.
1. Via Technical Monitoring - BI Monitoring
2. Via Business Process Monitoring (BP MON) - BW process chain monitoring.
3. Via Unified Job Monitoring

I would like to highlight what is a good option to do and what are the pros and cons with each approach.

What is BI Monitoring In technical Monitoring

To provide central monitoring and alerting capabilities. Integrated with Guided Procedure for alert resolution path. This is to cater to the need of an administrator to get an overview of the health of the systems participating in the BI landscape in addition to the objects specific to the data flows within the landscape.
Target audience:
BI administrator
BOBJ administrator
Application Support
BI operations Team

Runtime:

1. Overview Monitor

A single screen overview of the
Health of all systems participating in the BI landscape
  • Availability
  • Performance
  • Exception
View of the health of data flow entities in the landscape
o BW Process chains (ETL)
o BOBJ jobs (Reporting)
o DS Jobs (Replication)
o Bex Queries & Templates (Ad-hoc Reporting)

2. Detail Monitors

Provides specific information on the health of these monitored objects by monitoring certain metrics per instance of these recurring jobs which are representative of the health of the data flows across the systems.
Overall health of the job
o Status
o Error logs * (managed system login maybe required)
Scheduling metrics
o Start delay
o Not Started on Time
Runtime metrics
o Duration
o End delay
o Out of time Window
Data integrity metrics
o Records processed
o Data packages processed
o Rows_read
o Rows_written * Data services

Supported system types in BI Monitoring
System Monitoring Metrics are integrated in the overview monitor of BI for following system types

  • BW JAVA
  • SAP HANA Database
  • SAP SLT
  • BWA(TREX system)
  • ABAP Source system
  • BOE WAS (TOMCAT, WebSPhere, SAP_J2EE)
In addition to the system monitoring metrics, we can configure jobs/reports on top of these systems to be monitored.
  1. SBOP  (SAP Business objects platform jobs in CMC)
    1. 3.x
    2. 4.x
  2. SAP DATA SERVICES (Jobs)
    1. 4.1
    2. 4.2
  3. BW ABAP server (Process chains, BeX Queries, Templates)


As you can see, these screens have a designated flow for navigation. A single overview screen to show the health of all participating systems in the landscape and subsequently drill down capability by system-type and then to the monitored objects per system and then to its instance details!

Some key features in BI Monitoring (Configuration time in SOLMAN_SETUP) which caters to handle mass objects in configuration UI

  1. Mass maintenance of Thresholds
  2. Take from managed system to assist in configuring thresholds
  3. Excel Upload & Download
  4. Provides the managed object details in the configuration to assist in providing thresholds value for metrics like  (Duration, records processed etc.)
Nevertheless, this application has certain limitations.
1. Runtime (Monitoring UI) is not always colured!

The Monitoring UI could report grey rating for certain monitored objects which are not frequently executed in the managed system. Like a chain that is executing only once a week in the managed system. The collection frequency is set to 5 minutes in solution manager. So the status of these chains in the monitoring UI will turn grey,10 minutes after the chain ends today. And kicks in only for the next execution, that is in the next week! However the alert (if any) would remain in the alert inbox with thehistory of measurements.
{Increase the collection frequency. Instead of (once every) 5 mins, make it once every hour. Esp for longer running chains. This would mean the monitoring stays colourful for twice the collection frequency. so instead of 10 minutes, its available for 2 hours.  This has a flip side, "delay in alerting" bad news that of failure.}

2. Alert inbox has multiple alert groups open, for instance, for the same BW process chain LOG_ID due to
    1. Grey metrics (collector issue, MAI extractor issue, Engine design
    2. If an open alert of a chain failure is confirmed in the alert inbox, the next collection will again report this error and an alert is opened up again! (owing to a fixed look back time of 36 hours for the ST-A/PI collector)

=> SERIOUS CONSEQUENCE : Multiple (duplicated) email automatic notifications

Workaround for reducing the number of duplicate emails. (this does NOT eliminate the duplicate alerts. This would only reduce the duplicate emails)

1. Reduce the collection interval to a small and relevant window (to reduce the occurrence of grey metric from collector)

    1. However, in the advanced tab in scheduling of data collection, Managed system time zone is  not handled in ST< SP10 (UTC is considered)
    2. If the chain is  executed  over the midnight, it is not possible to configure this restriction in the design time :sad:

2. Increase the collection frequency. Instead of (once every) 5 mins, make it once every hour. Esp for longer running chains. This reduces the probability of a grey alert!  This has a flip side, "delay in alerting" bad news that of failure.


3.  Starting SP10, 2118848 can be implemented and the report program AC_BIMON_MASS_AUTOCONFIG to be executed with 'set retention time' to cicumvent this problem.



3. No analytics capabilities of the collected metrics. vis-a-vis Interactive reporting. No BW reporting!


What is it to monitor BW process chains Via BPMon - BW process chain monitoring?


    In order to support the end to end monitoring of Business processes, that could span across several systems and can internally comprise of different entities like interfaces, jobs, process chains, in Solution Manager, we have the possibility to orchestrate a Business Process and setup monitoring for the participating entities. In such a context, Process chains can be setup for monitoring.
    Advantages:

    => A clear Business process driven approach for monitoring
    => Support of extended schedule and multiple not started on time etc,.
    Limitations:
    -  has no overview of the underlying technical systems health
    -  has no contextual navigation to the underlying system monitoring.
    -  Always require a BP solution to be orchestrated in Solution Manager to setup monitoring!


    What is Unified Job Monitoring?


    Over the last one year, we have been pondering on means to fix these known issues and develop an application that serves the customer requirements by closing the existing gaps. Starting SP10, we unveiled the new work center -> Unified Job Monitoring.
    • There is a consistent approach to monitor all type of jobs (BW process chains, ABAP jobs, SBOP jobs, SAP Data Services jobs). Also monitoring jobs scheduled from EXTERNAL scheduler that use the SMSE interface. for instance, SAP CPS (REDWOOD) scheduled jobs.
    • Reporting of background jobs without requiring direct access to production systems using the collected metrics (BW analytics)
    • Powerful monitoring capabilities with factory calendar awareness, job log content, and Business process context so on


    We have developed a brand new monitoring UI keeping in the interest to transition to the new html5 technology (SAPUI5). Find below a glimpse of this monitoring UI.
    Key features
    In order to remove redundant collection in the managed system and to provide a persona-specific runtime view, we have ensured to unify the configuration, data persistency, collection and monitoring UI.


    Design Time:

    1. Reuse of Monitoring Objects from the three entry points - BP Monitoring solution, Technical Monitoring scenarios, Job Documentation.
    2. Pattern-based MO is supported for ABAP, BO, and DS. However, BW Process Chain has to be fully qualified name

    Runtime:
    1. Intermittent grey alerts are avoided
    2. Multiple email notifications are overcome
    3. Support of BW reporting.

    When to use what?
    1. If a need for Overview Monitor exist:     Technical Monitoring - BI Monitoring. With certain workarounds for the known limitations.
    2. For a pure Business process context:   BP MON - BW Process chain monitoring
    3. For a harmonized approach towards monitoring, Starting SP12, please migrate to BPMON and MAI integrated Unified Job Monitoring.
    SAP would continue to invest only in this option. Overall, Unified Job Monitoring addresses the known limitations of BI Monitoring and integrates
    BP monitoring based process chain monitoring.
    However, there are still gaps owing to the time required to develop. We intend to bring the best of both worlds together in this Unified job monitoring.
    Starting SP12, you could migrate existing BPMON Solution to MAI based solution. This would ensure automatic usage of Unified job monitoring if there are jobs, Process chains available in the classical solutions.
    Details: Execute the migration report R_AGS_BPM_MIGRATE_SOLU_TO_MAI via SE38. Use the F4 help to identify your solution.

    Similarly, there exist also a means to migrate relevant objects of existing Business Intelligence Monitoring scenarios to Job Monitoring Scenario.
    Details: In transaction SE38: AC_JOBMON_MIGRATION. Migrate or copy from existing BI Monitoring scenarios, job type objects (BW Process Chains , SBOP jobs, DS jobs) to a new Job Monitoring scenario to utilize the new collection framework and monitoring UI
    Comparison
    In 7.1 SP12 , all the three monitoring work centers co-exist. A comparison chart of features between Technical Monitoring - BI Monitoring versus Unified Job Monitoring is as below.

    FeatureBI MonitoringUnified Job Monitoring
    ABAP Jobs and stepsNAX
    BW Process chains and stepsXX
    SAP Business Objects Jobs (SBOP)XX
    SAP Data services jobsXX
    External Scheduler (SAP CPS REDWOOD)NAX
    BW Bex Reports & TemplatesXNA
    MAI features
    • Notifications
    • Incidents
    • Alert Inbox
    • Third party
    XX
    Integration to MAI System Monitoring and contextual navigation to System MonitoringXPlanned
    Contextual Navigation to Managed system analysis tools from Monitoring UIXPlanned
    Overview monitor For viewing the overall scenario health of all landscape entities & JobsXPlanned
    (MAI) Work mode awareness
    NAPlanned
    Mass handling of monitored objects & thresholdsXPlanned
    Integration to Job DocumentationNAX
    Guided Procedure for Alert ResolutionXX
    Please write to me regarding:
    1. How is job monitoring done today?
    2. Which job scheduling tools are used in the landscape (embedded schedulers from managed systems, CPS, UC4,CA WLA, AUTOSYS Solution Manager JSM)?
    3. What are the relevant/important job types?
    4. Is SAP Solution Manager-based Job/BI monitoring used?What is the feedback? Which functionality is missing?

    Regards, Raghav, S

    Development Manager, SAP Solution Manager


    8 Comments