SAP Analytics Cloud – Monitoring All Model Import/Export Jobs
If you have a huge number of import models on your SAP Analytics Cloud tenant and wondering how you can monitor all of their import jobs from one place, you came to the right blog post.
In this blog post I will be explaining how you can create a one stop shop story, from which you check:
- When the job was executed.
- By which user it was executed.
- Whether it failed or succeeded.
- Which queries are used in which models (By following a certain logic).
Before we start, please note that there is already a notification feature available per each model, it can be switched on from the “Data Management” tab, through the toggle button labeled “Notify me of refresh failures by Email” at the bottom of the page, by activating this, you will be recieving an email for each import job that failed.
There is also the “Schedule Status” page that shows the status of all the import jobs, it can be accessed through from “Connections” then “Schedule Status” tab.
Step by Step Guide:
Activate SAP Analytics Cloud Usage Tracking Content
Ignore this step if it is already active.
The SAP Analytics Cloud Usage Tracking Content provides useful insights about the activities, objects, files, and objects on your tenant. In our scenario we will be using the Activities and Objects data models.
The content can be activated by navigating to Content Network > SAP Analytics Cloud Usage Tracking Content > Adjust the import settings > Import.
This content is very useful in general, I have already used it before in a scenario where the customer wanted to check which models are used in which stories.
Create a new Story
Choose between Responsive and Canvas based on your Preference.
Insert a table, in the Buildler Panel, click add Linked Models, navigate to the same path and choose the “SAP__SAC_USAGE_OTHER_OBJECTS” model, create the link based on Object ID, make sure you have selected description.
The story design is entirely based on what you need, I will be referencing the SAP Analytics Cloud objects and features I used in my scenario in the upcoming steps.
I used a responsive page, created 3 lanes, 1 top horizontal lane where a chart is placed, 2 vertical lanes with 1 table inside of each.
If you need the story to fetch data based on the current date, simply create a story filter based on “Timestamp”, filter by range, set the granularity for Day.
In case you want to do a comparison between different time periods, do not set this as a story filter.
Set its value to “Model”.
Set its value to “Import”, you can also select “Export” in case monitoring export jobs was desired.
Select “All Members”, the usage of this filter can also extend to searching for the used queries as well, currently there is no way to retrieve the query related to a model, however I had the inspiration for this idea from a very clever customer I had, simply suffix or prefix the model description by the query name that is used, by doing this you can easily search for the query name in the description filter, and isolate all the models that use it.
Comparison – Stacked Bar/Column
Placed in the top horizontal lane, based only on model “SAP__SAC_USAGE_ACTIVITIES”, having in measures “Count of Activities” and “Status” in the Color, using a new pallet consisting of 2 Swatches Red, and Green.
Activated Linked Analysis based with Interactions apply to “All widgets on the Page”, the purpose of this is to filter all other widgets based on the status in a visual manner.
The Models by Status Table
This table shows the Timestamp, Model description, User ID, Status, and the calculated measure flag.
- Only the Model Description is coming from the “SAP__SAC_USAGE_OTHER_OBJECTS” Model.
- Flag is a calculated measure, having the formula”IF([d/”SAP__SAC_USAGE_ACTIVITIES”:STATUS_TYPE]=”Success”,
- From the “Styling Panel” you can choose how the thresholds can appear, in my case I left it as symbol, then set the font to transparent in order to hide the values.
- The Count of Objects measure from the other model is hidden by using the “Hide Column” feature.
- Linked Analysis has been applied to only affect the Navigate to Model table.
Navigate to Model Table
This model contains only the Description and Name ID, it was created to be able to directly jump to the relevant model using the hyperlinking feature, the reason why it needed a separate table is a current limitation on linking external content on a blended table.
This feature is very interesting, it can be used to jump to go to any link that has a static format until a certain value that you have in your report, lets have the current scenario as an example.
You can directly navigate to any SAP Analytics Cloud model by using this format https://<TENANT LINK>/sap/fpa/ui/app.html#/modeler&/m/model/<NAME ID> to achieve this, right click any Dimension in the relevant table> Table Functions > Add > Hyperlink
Place the static part of the link, then in the relevant position, in our case the last part of the URL, use the drop down list to select the dimension supplying the values, in our case it’s available in IN USE, choose Name ID.
The Label indicates the text that will appear when you choose the relevant row and click on “Navigate to External URL”.
Navigating to the URL can either be done through selecting a row, clicking Navigate to External URL, then selecting the Description that appears, or simply adding the relevant Dimension, in this case “Name ID” to the main table, then “Show Hyperlinks in Table”, then it will turn into what is exactly shown in the screenshot above, and can be directly consumed.
The story design is entirely based on your needs, it can be simpler or more complicated than this.
I hope this was informative, it was also a good chance to emphasize and introduce some features that exist and can be used in different ways on SAP Analytics Cloud.
Thank you for your time, please feel free to add any suggestions or feedback in the comments.
Two questions arise by myself:
a) Why is nothing similiar available out-of-the box?
b) Is there any possibility to get actively informed in case an import job fails?
Thank you for your feedback!
Currently what is available out of the box is the "Schedule Status" available in the "Connections".
As for getting actively informed when an import job failes, toggeling the "Notify me on refresh failures by email" will send you an email like the one in the screenshot below:
Refresh failure in model XYZ
This will only be recieved if a specific job failes, and you can directly go to the model using the "View Model" button.
Where can I find the option "Notify me on refresh failures by email"?
Is it just available for new models only or also for classic model (where I´ve currently imports running)? Further I also couldn´t find the toggle for dimension imports.
Hi Thomas Karner
you can find that setting at the bottom of the 'data management' screen (where the import jobs are scheduled). if memory serves me right, it exists for both old and new models.
Thanks! How could I have missed that 🙂
can you tell me WHO is going to receive the notification?
My import failed, but although toggled the button, there was no email.
Now, it is poosible, that we didnt't get one or we just looked at the wrong mailbox.
Thanks in advance:)
It will go to the email assigned to the user you are using, you can see it by navigating to Security > Users.
Hope this helps.
Thanks Mohamed. Nice Job!!
I have a question: What do you mean with "Refresh Now" button in Action col? is this to relaunch the import job again manually?
I'm so interested in being able to launch a specific import job from a button manually in case it fails..
Thanks for your feedback! Yes, this is possible from the "Schedule Status" window, in the Action Column you will find a button labeled "Refresh Now", it will manually trigger the import job once it is clicked.
After creating all the steps we cannot able to see the Model ID's in the instead we can see Description, even turning Display options to ID and Description both are same no ID is showing as C********. Where as when we open the model we can see in the URL Model is called by C********.
Appreciate your suggestions here.
Please make sure you are using the "Name ID" Dimension from the model
Hope this helps.
Thank you for your suggestions.
I have taken the model SAP__SAC_USAGE_OTHER_OBJECTS and didn't applied any link as you shown in screenshot, even though we are not able to see the ID's exactly starting with C*******.
We are facing a strange issue the models we created ourselves are showing as a members in NameID, but when we try to access log is there is null records we tried with linking both models too.
Appreciate your suggestions in advance.
Apologies for the late reply, I am not sure I understand your issue as you are covering the model IDs, can you please shed more light on the issue you are having?
Is it possible to set up a Generic email id which will get a notification when the schedule job fails. A user may end up being on holiday sometimes and miss the job fail notification.
There is no standard way to do this, however one workaround can be that you create a dummy user with a generic email address, use it to logon to SAP Analytics Cloud and enable the notification setting for the desired model.
Another solution is to check wheather you can use a distribution list email to create a user, use it to logon to SAP Analytics Cloud then enable the notification setting for the desired model. I have not checked it before, I believe that would be an alternative solution as you can add/remove whoever needs to be notified fromt he distribtution list itself.
Hope this helps!
That is what I thought myself. Thank you for confirming Ezzat
do you know, if in the meantime there is some standard way how to send an automatic notification to several recipients in case an import job fails?
Alternatives you have already described to Tanisha Gupta...
Thanks & BR
Hi Aleksej Busch,
Sorry for the delyaed reply, unfortunately so far I think not, I will reply back to your comment in case anything comes up.
Hello Mohammed Izzat ,
May i know why you have written the IF statement as
Coz this formula throwing an error .
What is the significance of 99 here ? We could write as simple
IF([d/"SAP__SAC_USAGE_ACTIVITIES":STATUS_TYPE]="Success", 1, 0 )
and then while creating threshold , will use greater than equal to 1 means success and <1 means fail .
Could you please help us to understand ? Thanks
Sorry for the delayed reply, if i remember correctly there were other statuses that were irrelevant to either success of fail, 99 was to represent a unique value to exclude what I do not need.
This is a very interesting and inspiring article. What I would like to ask, is if it is possible to retrieve information in a similar dashboard as the one you have described for all the information that are appeared under Connections -> Schedule Status.
For example, I want in a story to retrieve, not only all the Jobs that have an active/enabled schedule but also the Import/Export Job Name that has either Failed or Completed successfully, plus the Schedule timing per Import/Export job (which is shown under Data Management in each Model). The last one I understand that may need to be retrieved in another story. For example, we need to retrieve in a story all the Scheduled timings from all Models per Import/Export Job grouped by Model.
Thanks for your feedback. Right now this whole idea can be replaced by the "Admin Cockpit". What you are looking for is located on the "DM Job Statistics and Analysis Tab", navigate to "Statistics" you will find the time (this will be the last loading time) and load frequency.
This is really really helpful. But while looking at it, I noticed that column "Frequency" does not include information about the timing. It only includes info on how often the job is executed (e.g. Daily) and I would expect to say: Daily - Every day at 07:00:00 (as it is shown on my second screenshot of my initial question). This info that seems to be missing from this table, is something that can be retrieved from the Cockpit or somewhere else?
Thanks a lot
Additionally, to the above, I just noticed that the values of the Frequency column, seem not be aligned with what is shown in the Schedule of a Job, but seem to be produced automatically.
E.g. if a Job is executed either manually either on schedule every week, then this column shows Weekly, if it s executed either manually either on schedule every day, then this column shows Daily.
What I actually need, is a list of all the Jobs per model that are about to Run / or have already Run ONLY on Schedule (including timing) excluding those that run manually. Is that possible?
Check the datasource itself, I would say check the schedule related fields first, unfortunately I am using a test tenant so I do not have any scheduled jobs, let me know if it works.