Hello All, so here is some cool stuff for all those who try to ensure quality of your Dashboard.

Testing has always been my favorite area and here I would like to share few knowledge on this. Let me start with a question – Testing your Dashboard is easy? Its after all a Single Screen application , should be easy right? Well, answer depends on various factors but irrespective of whether its easy or difficult , remember it’s important.

A single screen visualization which pulls data from various sources, extracts data from multiple sources,calculates / consolidates numbers  and provides decision making information has to run into thorough QA right? Let’s discuss the  various testing types and testing approaches with respect to testing a Dashboard created using SAP Businessobjects Design Studio.I have not listed all the test scenarios but the key things to be taken care of in each type of testing.

Types of Testing:

Functional Testing

Functional Testing ensures the core functionality of the Dashboard as specified in the Requirements which includes but not limited to

• Data Validation

Numbers are the heart of the Dashboard. If this goes wrong, then there is big trouble.

• Validate the Data is extracted correctly at different levels and visualized properly.
• Validate all the calculation and aggregation logics performed at different levels.
• Check all the alerting rules , conditional formatting rules work fine based on the data in various scenarios.Boundary value analysis approach can be used here.
• Always do not stick to testing your dashboard statically .Data is going to grow and Dimension members , key figure Values are going to change.Simulate the change of data like addition of new rows, deletion of rows, during runtime and check in those scenarios.
• Ensure the working of your chart/ component for different kinds of data like decimal values, very large values,negative values,null values, zero values.
• Ensure the same for Dimension Members. Few cases are members having space character in them, comma,semicolon,period within them. The reason is few special characters are used as separators while parsing data. If your data contains such characters then there could be a issue.

• Searching,Sorting,Filtering,Paging
  • Most of the Dashboard have these features which play a major role in user   interactivity.
  • I have seen many Dashboards with searching working fine , but searching after a filtering may break. Sorting on a Search Result, etc are bug prone scenarios which may need attention.

• Triggers/Events
  • Ensure your scripts work fine and run at configured events.
  • To Debug,At regular intervals alert and check if the values and type of values have passed correctly.Ensure the default values have set up on startup event.

• Navigation

            A complex Dashboard used to have multiple views either by using tabstrip or by         switching visibility of the containers based on selection.

• Ensure you are able to navigate to all the views in all possible combinations. By this I mean ,things may be fine if you navigate in order View 1,View2 ,View3 but not in the other way.
• Scalability

            Test if your dashboard will be able to handle different scenarios due to changes in future.As a simple example you would have split a string by a seperator “/” and manipulated something. What if in future there is a data contains the separator within it.

Integration Testing

  Components individually may work fine delivering their functionality.But as a Dashboard the component is placed in containers in a nested manner and also placed with other components. There are few scenarios one component will affect the rendering of other component. Also few charts do not behave as usual when placed in container Components .

Non Functional Testing

• UI Testing

          We need to ensure that the  dashboard meets the UI standards.Few Scenarios include.

• Check if the alignment of all the components is proper relative to other components.
• Dynamic resizing during runtime scenarios
• Browser resizing scenarios
• Ensure no Overlap between components and no overlap between the elements in the component.
• Watch the boundaries. As an example there may be a dropdown at the bottom whose list may exceed the scope of the screen when expanded.

• Usability Testing

            This area needs much focus as the end users should see and understand the dashboard in few seconds . The Dashboard should be intuitive enough so that the user needn’t undergo any documentation to use the Dashboard. Few scenarios include

• Check if Tooltips are present wherever necessary.
• Title ,Subtitle and other information should convey clearly what data is the chart showing.
• Scroll Bars are necessary when the list exceeds the scope.
• Warning,Warming messages at necessary actions.
• Validations – As an example  if a user has selected an invalid filter , there should be a nice message instead of an error.
• Clear buttons for all input and Search Boxes would be good.
• All components should go well with the theme.

• Compatibility Testing

This is most Challenging area. The environment , platform may branch out into a number of combinations and it is impossible to test in all environments that your Dashboard works fine.

• Platform Compatibility:
  • BI/HANA/NW
    • versions
• Datasource Compatibility:
  • HANA
  • Universe
  • Bex Query/BW Cube
  • Excel /CSV Datasources.
• Browser Compatibility:
  • Chrome
    • versions
  • IE
    • versions
  • Firefox
    • versions
  • Safari
    • Versions
• Device Compatibility:
  • Mobile
    • IOS
    • Android
  • Ipad
  • Desktop

• Migration Testing
  • Tool Migration – Design Studio x version to latest version
  • Platform Migration – BI/HANA/NW x version to latest version
  • Environment Migration – Dev  to QA to Staging to  Production
  • Data(Backend) Migration – BW/HANA version migration
  • Extension  Migration – Your Dashboard may contain custom SDK extensions and when there is new version of those  , a migration testing is required.
• Performance testing

Dashboard’s performance plays a major role in user experience.Initial load time of your dashboard should not exceed 5-10 seconds.

• Analyse the cost of each datasource,Component rendering for every action like filtering etc and optimize accordingly.
• Measure the performance for different volumes of Data.
• Perform a Stress Testing and check what is threshold limit of Data volume at which your Dashboard crashes.
  
• Consider Parallel groups wherever possible .

• Security/ Access restriction Testing

Regression Testing

  This area is a major challenge. Regressing your Dashboard after every fix is mandatory as that fix may bring a bunch of other issues.

How to handle this?

My approach would be forming a Impact Distribution Matrix for your Dashboard which gives you information about the dependency of a component on a Datasource / Script/Variable etc.

With this information , if you change something somewhere, you know what are all the areas that needs to be tested.

Hope this provides a bird eye view on Dashboard Testing. Happy Testing!!