With 7.1, the capabilities of central monitoring has immensely improved in Solution Manager. For setting up of monitoring of systems in the BI landscape and objects in these systems, we have certain options available now.
1. Via Technical Monitoring – BI Monitoring
2. Via Business Process Monitoring (BP MON) – BW process chain monitoring.
3. Via Unified Job Monitoring
I would like to highlight what is a good option to do and what are the pros and cons with each approach.

What is BI Monitoring In technical Monitoring

To provide central monitoring and alerting capabilities. Integrated with Guided Procedure for alert resolution path. This is to cater to the need of an administrator to get an overview of the health of the systems participating in the BI landscape in addition to the objects specific to the data flows within the landscape.
Target audience:
BI administrator
BOBJ administrator
Application Support
BI operations Team
centralmontiroing_BIMON.PNG

Runtime:

1. Overview Monitor

A single screen overview of the
Health of all systems participating in the BI landscape
  • Availability
  • Performance
  • Exception
View of the health of data flow entities in the landscape
o BW Process chains (ETL)
o BOBJ jobs (Reporting)
o DS Jobs (Replication)
o Bex Queries & Templates (Ad-hoc Reporting)

2. Detail Monitors

Provides specific information on the health of these monitored objects by monitoring certain metrics per instance of these recurring jobs which are representative of the health of the data flows across the systems.
Overall health of the job
o Status
o Error logs * (managed system login maybe required)
Scheduling metrics
o Start delay
o Not Started on Time
Runtime metrics
o Duration
o End delay
o Out of time Window
Data integrity metrics
o Records processed
o Data packages processed
o Rows_read
o Rows_written * Data services
DETAIL_BIMON.PNG

Supported system types in BI Monitoring

System Monitoring Metrics are integrated in the overview monitor of BI for following system types

  • BW JAVA
  • SAP HANA Database
  • SAP SLT
  • BWA(TREX system)
  • ABAP Source system
  • BOE WAS (TOMCAT, WebSPhere, SAP_J2EE)
In addition to the system monitoring metrics, we can configure jobs/reports on top of these systems to be monitored.
  1. SBOP  (SAP Business objects platform jobs in CMC)
    1. 3.x
    2. 4.x
  2. SAP DATA SERVICES (Jobs)
    1. 4.1
    2. 4.2
  3. BW ABAP server (Process chains, BeX Queries, Templates)

MONITORS_BI_MON.PNG

As you can see, these screens have a designated flow for navigation. A single overview screen to show the health of all participating systems in the landscape and subsequently drill down capability by system-type and then to the monitored objects per system and then to its instance details!

Some key features in BI Monitoring (Configuration time in SOLMAN_SETUP) which caters to handle mass objects in configuration UI

  1. Mass maintenance of Thresholds
  2. Take from managed system to assist in configuring thresholds
  3. Excel Upload & Download
  4. Provides the managed object details in the configuration to assist in providing thresholds value for metrics like  (Duration, records processed etc.)
Nevertheless, this application has certain limitations.
1. Runtime (Monitoring UI) is not always colured!

The Monitoring UI could report grey rating for certain monitored objects which are not frequently executed in the managed system. Like a chain that is executing only once a week in the managed system. The collection frequency is set to 5 minutes in solution manager. So the status of these chains in the monitoring UI will turn grey,10 minutes after the chain ends today. And kicks in only for the next execution, that is in the next week! However the alert (if any) would remain in the alert inbox with thehistory of measurements.
{Increase the collection frequency. Instead of (once every) 5 mins, make it once every hour. Esp for longer running chains. This would mean the monitoring stays colourful for twice the collection frequency. so instead of 10 minutes, its available for 2 hours.  This has a flip side, “delay in alerting” bad news that of failure.}

2. Alert inbox has multiple alert groups open, for instance, for the same BW process chain LOG_ID due to
    1. Grey metrics (collector issue, MAI extractor issue, Engine design
    2. If an open alert of a chain failure is confirmed in the alert inbox, the next collection will again report this error and an alert is opened up again! (owing to a fixed look back time of 36 hours for the ST-A/PI collector)

=> SERIOUS CONSEQUENCE : Multiple (duplicated) email automatic notifications

Workaround for reducing the number of duplicate emails. (this does NOT eliminate the duplicate alerts. This would only reduce the duplicate emails)

1. Reduce the collection interval to a small and relevant window (to reduce the occurrence of grey metric from collector)

    1. However, in the advanced tab in scheduling of data collection, Managed system time zone is  not handled in ST< SP10 (UTC is considered)
    2. If the chain is  executed  over the midnight, it is not possible to configure this restriction in the design time 🙁

2. Increase the collection frequency. Instead of (once every) 5 mins, make it once every hour. Esp for longer running chains. This reduces the probability of a grey alert!  This has a flip side, “delay in alerting” bad news that of failure.


3.  Starting SP10, 2118848 can be implemented and the report program AC_BIMON_MASS_AUTOCONFIG to be executed with ‘set retention time’ to cicumvent this problem.


3. No analytics capabilities of the collected metrics. vis-a-vis Interactive reporting. No BW reporting!


What is it to monitor BW process chains Via BPMon – BW process chain monitoring?


    In order to support the end to end monitoring of Business processes, that could span across several systems and can internally comprise of different entities like interfaces, jobs, process chains, in Solution Manager, we have the possibility to orchestrate a Business Process and setup monitoring for the participating entities. In such a context, Process chains can be setup for monitoring.
    Advantages:

    => A clear Business process driven approach for monitoring
    => Support of extended schedule and multiple not started on time etc,.
    Limitations:
    –  has no overview of the underlying technical systems health
    –  has no contextual navigation to the underlying system monitoring.
    –  Always require a BP solution to be orchestrated in Solution Manager to setup monitoring!


    What is Unified Job Monitoring?


    Over the last one year, we have been pondering on means to fix these known issues and develop an application that serves the customer requirements by closing the existing gaps. Starting SP10, we unveiled the new work center -> Unified Job Monitoring.
    • There is a consistent approach to monitor all type of jobs (BW process chains, ABAP jobs, SBOP jobs, SAP Data Services jobs). Also monitoring jobs scheduled from EXTERNAL scheduler that use the SMSE interface. for instance, SAP CPS (REDWOOD) scheduled jobs.
    • Reporting of background jobs without requiring direct access to production systems using the collected metrics (BW analytics)
    • Powerful monitoring capabilities with factory calendar awareness, job log content, and Business process context so on

    MOTIVATION_JOB.PNG

    We have developed a brand new monitoring UI keeping in the interest to transition to the new html5 technology (SAPUI5). Find below a glimpse of this monitoring UI.
    jobMON-MOnUI_Sp12.PNG
    Key features
    In order to remove redundant collection in the managed system and to provide a persona-specific runtime view, we have ensured to unify the configuration, data persistency, collection and monitoring UI.

    Design Time:

    1. Reuse of Monitoring Objects from the three entry points – BP Monitoring solution, Technical Monitoring scenarios, Job Documentation.
    2. Pattern-based MO is supported for ABAP, BO, and DS. However, BW Process Chain has to be fully qualified name

    entry_points.PNG
    Runtime:
    1. Intermittent grey alerts are avoided
    2. Multiple email notifications are overcome
    3. Support of BW reporting.
    reporting_JOB_MON_SP12.PNG

    When to use what?
    1. If a need for Overview Monitor exist:     Technical Monitoring – BI Monitoring. With certain workarounds for the known limitations.
    2. For a pure Business process context:   BP MON – BW Process chain monitoring
    3. For a harmonized approach towards monitoring, Starting SP12, please migrate to BPMON and MAI integrated Unified Job Monitoring.
    SAP would continue to invest only in this option. Overall, Unified Job Monitoring addresses the known limitations of BI Monitoring and integrates
    BP monitoring based process chain monitoring.
    However, there are still gaps owing to the time required to develop. We intend to bring the best of both worlds together in this Unified job monitoring.
    Starting SP12, you could migrate existing BPMON Solution to MAI based solution. This would ensure automatic usage of Unified job monitoring if there are jobs, Process chains available in the classical solutions.
    Details: Execute the migration report R_AGS_BPM_MIGRATE_SOLU_TO_MAI via SE38. Use the F4 help to identify your solution.
    Similarly, there exist also a means to migrate relevant objects of existing Business Intelligence Monitoring scenarios to Job Monitoring Scenario.
    Details: In transaction SE38: AC_JOBMON_MIGRATION. Migrate or copy from existing BI Monitoring scenarios, job type objects (BW Process Chains , SBOP jobs, DS jobs) to a new Job Monitoring scenario to utilize the new collection framework and monitoring UI
    Comparison
    In 7.1 SP12 , all the three monitoring work centers co-exist. A comparison chart of features between Technical Monitoring – BI Monitoring versus Unified Job Monitoring is as below.

    Feature BI Monitoring Unified Job Monitoring
    ABAP Jobs and steps NA X
    BW Process chains and steps X X
    SAP Business Objects Jobs (SBOP) X X
    SAP Data services jobs X X
    External Scheduler (SAP CPS REDWOOD) NA X
    BW Bex Reports & Templates X NA
    MAI features
    • Notifications
    • Incidents
    • Alert Inbox
    • Third party
    X X
    Integration to MAI System Monitoring and contextual navigation to System Monitoring X Planned
    Contextual Navigation to Managed system analysis tools from Monitoring UI X Planned
    Overview monitor For viewing the overall scenario health of all landscape entities & Jobs X Planned
    (MAI) Work mode awareness
    NA Planned
    Mass handling of monitored objects & thresholds X Planned
    Integration to Job Documentation NA X
    Guided Procedure for Alert Resolution X X
    Please write to me regarding:
    1. How is job monitoring done today?
    2. Which job scheduling tools are used in the landscape (embedded schedulers from managed systems, CPS, UC4,CA WLA, AUTOSYS Solution Manager JSM)?
    3. What are the relevant/important job types?
    4. Is SAP Solution Manager-based Job/BI monitoring used?What is the feedback? Which functionality is missing?

    Regards, Raghav, S

    Development Manager, SAP Solution Manager


    To report this post you need to login first.

    8 Comments

    You must be Logged on to comment or reply to a post.

      1. Raghav S Post author

        Yes Mike. It is solved with Job monitoring. Because we have a collector in the managed system in ST_PI which understands the final state.

        (0) 
    1. Mike Zhang

      Hi Raghav,


      Is this statement true, should the status keep for 36 hours?


      “Like a chain that is executing only once a week in the managed system. The collection frequency is set to 5 minutes in solution manager. So the status of these chains in the monitoring UI will turn grey 10 minutes after the chain ends today.”


      Best Regards,

      Mike

      (0) 
      1. Raghav S Post author

        In BI monitoring in the BW PC detail monitor. the colour will expire. However the alerts(if any) would continue to remain until someone acts on it.

        (0) 
    2. akshara akshara

      Hi Raghav,

      We are currently using BW 7.3 system and use process chains for all dataload activities. We have been suggested to use solution manager for monitoring our process chain activities.

      I have searched the forum but didn’t find any article about pros and cons of using solution manager for monitoring Process chains.  I am looking for information where using solution manager would be advantageous for monitoring PCs and also looking for downfalls if any.

      Are there any activities which we cannot do using sol manager which is possible only in BW? Are they any new facets which we observe in sol manager over BW? Can we eliminate the BW lookup for monitoring process chains when we us sol manager?

      Can you please help in the above queries?

      Thanks,
      Akshara

      (0) 
      1. Raghav S Post author

        Hello Akshara,

        Thank you for reaching out.

        What is the solution manager Support pack stack 7.1 SP?

        How many chains do you want to monitor? Do you in general have a dedicated monitoring team for the BW PC or is there a central monitoring team for all your application/systems?

        – Central monitoring via solution manager is to setup proactive monitoring combined with alerting and monitoring usecases.

        – When you have a business process driven approach or a system monitoring driven approach or a data flow driven approach (across BI landscape) , it is easier to set this up in solution manager.

        – Local monitoring is system specific. Nevertheless also detailed and line-item wise.

        – Central monitoring can monitor – status, duration, start delay, end delay, records processed and so on.

        Regards, Raghav

        (0) 

    Leave a Reply