Skip to Content

Introduction

Panaya offers several SAAS solutions for ERP and one of them is SAP upgrade automation which can be seen as competition for the Solution Manager Change Impact Analysis.

How do you perform an analysis using Panaya?

You extract data from your ERP system (using a delivered report) and upload the data to the SAAS solution provided by Panaya. There the data is analyzed and within 24 to 48 hours you can view the results online.

Advantage(s):
  • It’s really simple, run a report, upload a file and voilà the results are there to be viewed online 24 to 48 hours later.
Disadvantage(s):
  • Can you trust the analysis result completely? From what I have heard, not yet, there where still a few minor impacts that were not present in the analysis result. I have also heard Panaya is working hard to further improve their service and get rid of those few minor impact cases that are not yet caught by the analysis.
  • There could be data governance issues here, what can they actually do with the extracted data (does the tool extract Z program source code?). To what extent are they allowed to use it?

How do you perform an analysis with Solution Manager?

You have to create an implementation project in Solution Manager and define your business scenarios with their respective business processes that are used in your ERP system. Once that is done you create TBOM’s (Technical Bill of Material) of each business process by tracing them through manual execution. Next you download the upgrade or support package stack files and upload them to the EPS/in of your transport directory. You create an analysis project to analyze the impact on the business processes caused by the upgrade or support package stack.

Advantage(s):
  • You can use The BPCA (Business Process Change Analyzer) of Solution Manager for more than just ERP systems.
  • The data stays available in your Solution Manager system and as such you don’t encounter data governance issues.
Disadvantage(s):
  • Your Business Processes have to be created in a correct fashion and have to be maintained and even then your impact analysis isn’t 100% reliable. Maintaining the business processes correct isn’t such an easy task in large enterprises where business processes tend to flow through multiple products and technologies. So here we again see the trust issue that the results might not capture everything. It relies strongly on the correct maintenance of your business processes.
  • The tracing itself can also be a problem. If you have products like CRM 7 with very new end-user interfaces and you cannot trace through the product properly (lack of functionality available to do so) you end up with no result or bad results and your impact analysis will be rendered almost useless.
  • The process from start to end to perform an analysis takes a lot of time. The steps that have to be performed in combination with the different interfaces, the necessary know-how and the technical prerequisite to create TBOM’s make it way to complicated compared to the very simple process Panaya offers. To use the service from Panaya the end-user doesn’t need any training. For Solution Manager your end-user would need training so that sums it up nicely.

Product pricing

Here it depends a bit how far you want to go. Panaya also offers a solution for automated testing where Solution Manager also offers the possibility to do automated testing and both can provide the necessary feedback which tests should run according to the analysis run.

From what I have heard from customers the Panaya Upgrade Impact Analysis on-demand isn’t exactly a cheap product. The price is based on the size of your ERP system according to what I was told and Panaya knows SAP customers are looking into alternatives to provide them these kind of services. Why? Because it can or should save you money and effort as you should have to do less testing and have less incidents afterwards.

SAP Solution Manager on itself is a free product, of course you have your hardware cost and you should have enterprise support which isn’t actually free of course since you pay a maintenance fee. If you want to go full blown and you decide to start using automated testing, most customers go for a combination of Solution Manager and other products like HP Quality Center and SAP TAO and those aren’t exactly free of charge either.

You can’t really find reference prices for either solutions and more often than not it seems like a product or solution has a price tag that was based on which direction the wind was blowing that day. It comes down to negotiating a better price but in the end the products that come with license costs are not cheap.

Is it worth it?

According to the customer I spoke with, Panaya is useful to see where you can expect impact but since they noticed the results so far weren’t 100% reliable they test everything anyway. They did notice a decrease in incidents compared to previous upgrade runs but since the pricing isn’t cheap either the question is do you reduce your total cost of ownership?

On Solution Manager side I don’t have a reference of a customer that really actively uses Change Impact Analysis. I did attend a webinar from SAP which included a customer testimonial on the business process documentation and automated testing part. According to that customer it does save them money in the end but they also have someone that does maintenance of the business processes and updates the necessary related content a half day a week (note that this was used throughout their SAP system landscapes). The thing is they didn’t use impact analysis. If it was really just another click away they would of course. It doesn’t seem to be a feature that is used often and it must have it’s proper reasons (see Solution Manager disadvantages).

The message is clear: don’t just buy into the marketing lingo for these products. Lines like you can save up to 80% costs should be corrected into you might save up to 80% costs which can also mean you save 5% for that matter or you don’t save any costs at all. No matter which product you choose, it’s not guaranteed full proof. One can be cheaper license cost wise than the other but then it also requires more effort to maintain.

If you want to check out the Solution Manager functionality start by documenting and setting up a small business process and see how everything works before making a decision to go all the way.

Solution Manager on-demand?

So how about SAP creating an impact analysis on-demand service that can analyze the necessary data and give back the result in way less than 24 hours? Surely it should be possible if it would be powered by SAP HANA. As SAAS solution it could also be affordable or even free of charge (those maintenance fees should create value right).

A few weeks ago I saw some comments on “Thought leadership” that there is too much thinking going on these days and not enough action. “Action leadership wanted” was the message there.

I have to agree to a large extent so I also created an idea on idea place to give the thinking more meaning and possibly transform it into action if enough community members decide it should and of course given SAP picks up the idea. SAP Ideaplace is a nice initiative and we should all profit from having it around and we can do that by creating ideas and voting ideas up that we want to see incorporated.

 The idea

The process to go through to check out the impact of a transport request, support package stack or upgrade on a managed SAP system is too complex and takes too much time.

I would love to see SAP come up with an on-demand solution that can offer impact analysis on a relative short timeframe. This should be possible since there are already existing products from external parties (one of them being Panaya but more exist) that offer similar functionality.

It looks like SAP tried to incorporate too much into the Business Process Change Analyzer. The idea is good but it doesn’t really feel right and it doesn’t deliver what it should deliver. A more simple process that gives you all the impact (even on unused Businesss Processes for that matter) can be more valuable. A resultset based on used transaction and another based on unused transactions can already offer additional value by taking into account used transaction of your productive SAP system.

By leveraging the cloud and powering the impact analysis by HANA such impact analysis should be possible on relative short timeframes and can provide added value for the customers.

You can vote up (or down) the idea here: https://cw.sdn.sap.com/cw/ideas/6470.

To report this post you need to login first.

31 Comments

You must be Logged on to comment or reply to a post.

  1. Martin English
    Hi Tom,
    thanks for doing this research; its a good summary of an alternative to Impact Analysis.  It’s interesting that Panaya still need to work on their coverage
    (0) 
    1. Tom Cenens Post author
      Hello Martin

      Thanks for your comment.

      The major impacts where detected by Panaya but a few (three ~ four) smaller impacts were not the last time it was used at one of my customers.

      So they still have work to do which is why the prize is too high for sure. If you can rely on it to be 100% correct the prize might be justifiable.

      There are other parties that offer similar “impact analysis” tools and services but Panaya is definitely one of the more interesting ones.

      Kind regards

      Tom

      (0) 
    2. Tom Cenens Post author
      Hello Martin

      Thanks for your comment.

      The last upgrade run there were about three to four minor impacts that were not detected.

      All major and most minor impacts were detected but it wasn’t 100% fullproof. Since the prize isn’t cheap one would expect it to be 100% fullproof and of course then it would also have much more value.

      Kind regards

      Tom

      (0) 
    3. Bjoern Panter

      Custom Code and usage coverage is always an important question. Thats the reason why SAP is shipping the CCLM Custom Code Lifecycle management in Solman 7.1 supporting the kernel based logging feature UPL /Usage & procedure Logging) unveiled at SAP teched 2011. Now and only SAP is now able to track usage of ABAP procedures down to subroutine levels.
      We find everything thats ABAP code based. User Exits, used modification, smartzforms, function modules and much more.

      • Tracking of successfull custom code projects 
      • Tracking of unsuccessfull custom code
      • Support of Audits, Security-Audits
      • Improve services on Custom Code Management
      • Improve system quality  
      • Efficient Upgrades (UPL in a preanalysis to determine impact on used custom code. Controlled generation of mostly used objects to minimize uptime after upgrade) 
      • Focus on critical code objects
      • Efficient test coverage (UPL a a validation to crosscheck test cases with reality usage) 
      • Test quality Check of saved test management cases in Solution Manager: UPL <> T-BOM’s

      Looks like SAP is also working some nice tools. Everthing to manage custom code will be made in CCLM. A central single source of truth to have always transparency about all custom code objects including usage, quality etc…..

      Last week I found some implemented custom code in a productive environment of a customer. CCLM automatically detected the quality level and it was an amazing result

      PROG ZPANA*_RECORD_S*   336 Very High 22 Medium 2 Low priority erros
      PROG ZPANA*_EXTRACT   210 Very High 45 Medium 0 Low priority erros
      PROG ZPANA*_IMPORT_ACC* 2 Very High 1 Medium 0 Low errors

      We recommended to remove the 3 programs from the systems cause they will have a negative impact on the system performance and there is no need to use them.

      All you need is available in Solution Manager 7.1 with

      CCLM = Custom Code Lifecycle Management
      Clonefinder 3.0 = with new enhanced use cases including impact analysis of support packages
      UPL = Track usage with only a 1-click activation
      and some more new things we will release soon.

      Be part of teh SAP community and trust the creators of ABAP.
      Only we know what you should change and test.

      regards

      Björn

      (0) 
      1. Jansi Rani Murugesan

        Hi Bjoern,

        You have given the clear usage and purpose of CCLM and CDMC, I really liked the way you explained.

        Now I am with tricky situvation, where my client already using Panayya for testing (as already concluded each has its own purpose, no longer comparison required).

        They too using BPCA for specific core business process impact analysis,

        Now they would like to use CDMC to analyse the impact on their custom developments after the upgrade. (they are still with SM 7.0 Ehp1)

        I was almost gone through all the documet, I found that there are missing input from SAP, regarding performance and memory related issue on CDMC.

        if can, can you give your own experience view on the discussion http://scn.sap.com/thread/3200907

        Thanks,

        Jansi

        (0) 
  2. Michelle Crapo
    Long time – I’m not real sure what year it was.  I met some of the Panaya people at an SAP Teched conference.  They were impressive.  They were developers and not marketing people.

    It seemed like an amazing tool.  Like all tools it had some glitches.  At the time we were not doing an upgrade.  When we did do our upgrade we didn’t use anything that wasn’t already available.  IE SPAU.  We had a list of obsolete or updated FM and fixed our custom code.

    We have a lot of custom code.  That made it harder.  BUT we have to retest everything as we are an FDA firm.  It just means there cannot be mistakes / or the mistakes are in non-GMP code.  If you don’t know what I’m talking about – you are in a great place.  We document more than we write code sometimes.

    And so…  We did not buy anything extra to help with the upgrade.  It’s interesting to hear about Panaya again.  We have a tool to help with testing.  Of course we have never taken the time to learn how to use it.  I get conference information from them once in awhile.

    Maybe someday we will have the time.  And then maybe it will be time to look at the different tools.  Of course, I am not the one who decides on any of this, but I can point people back to this blog that I am bookmarking.  Also I think we are starting a new “Testing” team.  I’m sure they will learn to use the tool that we have. 

    Thank you for writing about Panaya vs. Solution Manager!  This is a hard blog to write.  I’m looking forward to seeing the comments from Panaya and SAP type people.

    Michelle

    (0) 
    1. Tom Cenens Post author
      Hello Michelle

      We have such customers with very strict rules so yes I have an idea what you mean 🙂

      Panaya nowadays also offers testing possibilities. I haven’t yet seen it live or used it myself so I can’t comment much on that except for the fact that they offer such a solution for SAP ERP.

      It’s a blog that I read over a couple of times. I don’t have any share in either products or services so I based this on what I have seen / used and by talking with a customer who actively uses Panaya.

      Kind regards

      Tom

      (0) 
    2. Tom Cenens Post author
      Hello Michelle

      We have such customers with very strict rules so yes I have an idea what you mean 🙂

      Panaya nowadays also offers testing possibilities. I haven’t yet seen it live or used it myself so I can’t comment much on that except for the fact that they offer such a solution for SAP ERP.

      It’s a blog that I read over a couple of times. I don’t have any share in either products or services so I based this on what I have seen / used and by talking with a customer who actively uses Panaya.

      Kind regards

      Tom

      (0) 
  3. Amit Bendov
    @Tom,
    Thanks for taking the time to discuss the topic!
    I’d like to answer a couple of the question you raised:
    >> Can you trust the analysis result completely?
    I don’t know about you, but personally I wouldn’t trust anything in life completely 🙂
    Now seriously, while naturally,  some of our customers report a few undetected issues, what we offer is most likely the most accurate and comprehensive alternative out there, with most customers reporting over 95% accuracy. I believe this is in line with what you reported? Plus, because we are a SaaS with thousands of users, the cumulative knowledge of the community wrt to such issues is immediately shared with the user community. So the accuracy gets better every day.

    While the customer you spoke to elected to test everything, Panaya’s accuracy is light years ahead of what our users have been using before (mostly manual guess work) and is more than adequate to shave off major portions of their testing effort. And this works well in practice – you can find many case studies in here http://www.panayainc.com/Case-Studies.html

    Finally, I’d like to point out that Panaya isn’t only about testing LESS- it’s about smarter testing, and minimizing risk with finite resources. Here’s the issue:
    Consider that the average Support Pack Stack impacts a mere 3% of the ERP system. Not a lot. But which 3%? What should you test? Everything? That’s not a terrible option, if you can afford it financially. Most companies can’t, especially these days. How about testing only the critical business processes? This is a more affordable option, but you will almost certainly be testing some critical processes that were not impacted, and will not be testing non-critical – yet important – processes that were impacted.

    Our goal at Panaya is to help you test less, while being safer.

    >> To what extent are they allowed to use it?
    We are not allowed. Period. And certainly no more than, say, a consulting company you’d hire to work on you system. In fact, we are much more secure, since there are no people working on your system- the process is fully automated and our employees do not have access to the customer’s system, unless the customer explicitly grants us such access (e.g., if needed for technical support). I’d also mention here that we use the best security technology money can buy, in many cases much safer than what our customers use for their on-premise installations. As you can see on our customers web page, we have customers in some of the most security-sensitive industries such as Aerospace, Defense, and Banking.

    I hope this answers your questions

    Anyway- would be happy to chat 🙂

    (0) 
    1. Michelle Crapo
      Amit,

      I believe you are one of the people I met.  I also met a developer.  Have you thought about doing a blog on Panaya – I know you could point to your site.  It couldn’t be marketting, but it could be customer information / a case study from a customer.

      Or it would be great if one of your customers wrote a blog.  🙂  Even better would be if one of your developers wrote a blog!  It could be about things you can do to customize Panaya.  The hooks to change code.  (If there are any.  That type of thing.)

      FYI – for the curious like I am:  The customer case studies are here:

      http://www.panayainc.com/Case-Studies.html

      Keep in mind – that I have never used this product.  But as always I’m curious to learn more.  I haven’t had the time to look at the customer case studies.

      BR,

      Michelle

      (0) 
      1. Amit Bendov
        Hi Michelle,

        Of course I remember you- we met in Phoenix. Great to hear for you!

        You mentioned “impressive” engineering folks so I figured you must be alluding to someone else LOL.

        Anyway, great suggestions! We do have a customer blog in the working, and as for developers- I know our developers love to write code- I’ll see if the love posting just as well 🙂

        Best,

        Amit

        (0) 
    2. Tom Cenens Post author
      Hello Amit

      Nice to see someone from Panaya commenting on this blog.

      Yes Panaya is pretty accurate (not 100% but still very good) is what I have heard from my customer.

      If the results get improved even further perhaps they can stop doing all tests alltogether and focus on the area’s Panaya highlights.

      I don’t have much reference to competitor products who offer the same functionality (I know some exist, perhaps that also indicates they aren’t as good yet) except for what is available within Solution Manager so I can’t make much of a statement on that.

      Perhaps a good question which could be of interest to possible customers, do customers have a possibility to try out Panaya (without cost or at very low cost)?

      Companies with low budgets (small-medium sized enterprises) often don’t use all the functionality in their ERP system and when they only use a very small percentage I don’t think it really makes sense to do an upgrade impact analysis. Of course it is always interesting but the question then becomes can you justify the prize for them?

      Thanks for your comment which features nice additional information for the blog readers.

      You are welcome to contact me so we can chat. I’m always to connect and find out more about interesting SAP related products. (You can find several contact details on my SCN business card).

      Will you be at SAP TechED or SAPPHIRENOW in Madrid by any chance?

      Kind regards

      Tom

      (0) 
      1. Amit Bendov
        >> do customers have a possibility to try out Panaya (without cost or at very low cost)?

        You bet. Our belief is that you have to SEE the value before you buy. All customers are offered a free trial, with no fine print and no strings attached.

        Still undecided wrt Madrid, but I’ll let you know if I’m going.

        (0) 
  4. Bjoern Panter
    Dear Tom,

    It makes no sense to compare the Solution Manager and Panaya, cause SAP follows a different strategic goal.

    With the ALM process of SAP we have a new topic called Custom Code Management and inside we provide many tools and services to support customers with the given challenge.

    See details under http://service.sap.com/alm-processes in detail under Custom Code Management

    What is our offering portfolio compared to Panaya?

    Custom Development Management Cockpit (CDMC) to support a clearing of unsued custom code. Combined with a change impact analysis to find out which custom code will fail after a SAP code change.  This is always a pessimistic results based on fact we want to ensure that you see every potential error situation. Our goal is to ensure the productive environment is really running, because every second of productive down situation also in case of a root cause in custom code will increase the disappointment of the SAP product at all.

    Furthermore the most results are based on ST03N workload data. We all know that there is a gap inside and this logging will not be able to find all executed programs. Also here only SAP created a solution you can see on the TechEd2011 (ALM218, ALM278). We unveiled the SAP Usage & Procedure Logging. A brand new capability to track all used ABAP procedure not only from a transaction code or program level, but down to function modules, class-methods and also subroutines. Now we are able to find out if a User Exit or modification was really executed and how often.  At the moment re integrate the new features in all of our tools.

    SAP UPL: Helps you to improve the quality of custom code. Track the real execution of custom code element in test system and observe it in productive environments. All comined in a hiuge data source within SAP solution Manager 7.1 with capability of BW reporting. Execution of Custom Code, always full transparency without effort.

    SAP Clonefinding: Yes we can and much better. We release a cool feature to detect real clones of SAP programs in your custom code. Our goal is to help you to go back to SAP standard. It makes no sense to change again and again the clones. One big reasons is that shipped innovation controlled by Business Functions will not be able to be reactivated in a cloned program. So if you adapt your clone, you close the door for innovations. SAPs approach is removing clones. We show you also a similarity degree between SAP and custom code. And we hope that using the updated, fixed and innovated Sap original program will give you the opportunity to stay on track for the future.

    SAP Dynamic references of custom code: Look in the small text areas of 3rd party offering. Dynamic programming within SAP play no role? That¡¦s not true. Think about ALV or the whole CRM logic. Or banking industry solution. Also here SAP supports the customers to find out where custom code was linked dynamically to customizing tables, enhancement techniques and much more.

    SAP Modification Overview: Who creates how much modification? Yes we can support you.

    SAP Transaction code Similarity: user and Roles concept to support authorization requirements leads to a huge number of unnecessary cloned TA codes. We help you to reduce the overall number of custom TAs and show you where SAP standard makes more sense. SAP is providing always the latest legal requirements up to date. Take this advantage!

    SAP Code Remote Comparison: Find out where are real code differences in a multi system landscape. But not with a  simple lines of code comparison. State of the art algorithms to detect similar code objects to support any kind of differences.

    SAP Custom Lifecycle Management: Comprehensive Stock overview of all custom code objects combined with technical metadata and assignment to responsibles and owners. No more project based analysis necessary. On-demand. Direct available overview about all your custom code objects, including status, quality level, impact and usage. No need to implement any Z-report and execute an analysis. The results are always there available in the SAP CCLM of Sap Solution Manager.

    So you can see the wheel is still turning and SAP will help the customers to drive the strategies like
    –     Clean System
    –     Back to SAP standard
    –     Activate Innovation

    If anybody is interested to cooperate with us, please contact me.

    Outlook: We are increasing our portfolio to support the customers with tailored made solution released for every customer. It is worth to have a look from time to time to see what we are doing .

    PS: You think our UIs are not very nice. I¡¦m sorry, but we follow the Sap UI guidelines in SAP GUI or Webdynpro. We do not have (now)  the flexibility to make-up results with HTML or flash animations. ƒº

    (0) 
    1. Tom Cenens Post author
      Hello Bjoern

      Thanks a lot for your comment.

      I have heard about some of those processes and there are definitely interesting new features among them. I already showed some new features to one of our customers.

      They do sound more like features a developer would use and not so much a SAP system administrator, would that be correct?

      “It makes no sense to compare the Solution Manager and Panaya, cause SAP follows a different strategic goal.”
      -> I can agree that it’s not completely a one on one comparison but still customers look at both solutions to provide them an impact analysis. Just yesterday I had a call with a customer who was looking into both Panaya and Solution Manager to provide such an analysis.

      It looks like SAP’s focus is very much on custom coding and not so much on standard coding.

      I find it strange that SAP doesn’t provide functionality that lies closer to what Panaya is offering.

      I do belief that it will become more and more important as a consultant to have Solution Manager skills.

      Thanks for listing the options and taking the time to comment. I would be happy to talk to you about this, are you going to SAP TechED Madrid by any chance?

      Kind regards

      Tom

      PS: concerning the UIs: It’s more about having a UI that is intuitive. BPCA doesn’t really feel intuitive. Certain UI’s look outdated in Solution Manager 7.1 because they were not changed compared to Solution Manager 7.0.

      (0) 
    2. Tom Cenens Post author
      Hello Bjoern

      Thanks a lot for your comment.

      I have heard about some of those processes and there are definitely interesting new features among them. I already showed some new features to one of our customers.

      They do sound more like features a developer would use and not so much a SAP system administrator, would that be correct?

      “It makes no sense to compare the Solution Manager and Panaya, cause SAP follows a different strategic goal.”
      -> I can agree that it’s not completely a one on one comparison but still customers look at both solutions to provide them an impact analysis. Just yesterday I had a call with a customer who was looking into both Panaya and Solution Manager to provide such an analysis.

      It looks like SAP’s focus is very much on custom coding and not so much on standard coding.

      I find it strange that SAP doesn’t provide functionality that lies closer to what Panaya is offering.

      I do belief that it will become more and more important as a consultant to have Solution Manager skills.

      Thanks for listing the options and taking the time to comment. I would be happy to talk to you about this, are you going to SAP TechED Madrid by any chance?

      Kind regards

      Tom

      PS: concerning the UIs: It’s more about having a UI that is intuitive. BPCA doesn’t really feel intuitive. Certain UI’s look outdated in Solution Manager 7.1 because they were not changed compared to Solution Manager 7.0.

      (0) 
  5. Kevin Ko

    Tom:

    Great read!

    Another alternative that I have seen on many Fortune 500 company’s radar lately is the LiveCompare product. It is just a simple application that performs sophisticated Impact Analysis of any SAP environment. Coca-Cola, Kimberly-Clark, and Hershey’s are already using LiveCompare to streamline change management to lower their time, risk, and effort when testing.

    Just my two cents.

    Regards,

    Kevin

    (0) 
    1. Tom Cenens Post author

      Hi Kevin

      Thanks for your comment.

      SAP has already done some efforts to improve existing functionality and deliver new functionality since this blog post was written/posted so there are improvements from SAP side as well.

      CBTA (Component Based Test Automation) for example.

      Thx for adding value to the post as well by mentioning LiveCompare!

      Best regards

      Tom

      (0) 
    1. Roberto Campo

      I used Panaya in several projects and assessments since 2009 for Customer in Italy and Europe and found it terrific. The results are quite reliable and the data and functionalities are available to consultants & users in a friendly interface that in my opinion is not comparable with Solman. I had access to some comparison studies especially for SPS implementation vs SAP BPCA and there was no story both in terms of effort (it takes a life to create a valid TBOM baseline) and reporting.

      I agree the price is the real key even if I know for a fact that Panaya is able to adapt to different Customer scenario.

      Regarding the real effort saving in my opinion this is more visibile in SPS project than in upgrade ones, but be aware that once you finish the upgrade you’ll be involved in the SPS story after 2 months!

      Testing automation too is very interesting (also here I have just a small experience of eCatt and even if the products have different target also here I think Panaya is far ahead).

      I’m very interested in any case to understand better how SAP is closing the gap and try to understand what SAP Clone Finder can do for free compared to the deep analysis that Panaya was already able to do 5 years ago.

      (0) 
        1. Roberto Campo

          If I undertood correctly also SEA has been built on top of TBOM that need to be created manually by the users…. So in order to assess the impact it takes a lot of time in navigating the actual processes! And hoping you don’t forget anything…

          (0) 
          1. Tom Cenens Post author

            Hi Roberto

            A couple of comments on the recent comments here 🙂 :

            The blog post is “old” by now in my opinion so it’s not a recent comparison.

            The recently introduced test option with CBTA (Component Based  Test Automation) is super interesting (ease of use etc is there) for automated testing for SAP UI technology. It’s free for Enterprise Support contract customers and it works well, we are using it and we like it.

            By executing CBTA test scripts, you can automate TBOM generation. So that’s one way you can automate the TBOM generation.

            The SEA scenario (as from 7.1 SP11) can automatically generate a business process documentation tree and you can also now generate semi-dynamic TBOM’s in background so it’s not a prerequisite to generate dynamic TBOM’s (which require either manual action or CBTA automated generation) and you can still get a decent result which effort is required.

            You can have a look at the slidedeck of the Solution Manager roadmap for more details on SEA on https://service.sap.com/roadmaps

            I haven’t followed up further on Panaya’s functionality since the customer I was working for, stopped using Panaya in favour of SAP Solution Manager functionality – Business Process Documentation – Solution Documentation Assistant – Business Process Change Analyzer – Test Scope Optimization & Component Based Test Automation

            From what I’ve seen in the past, Panaya was easy to use, pretty easy to understand, recording also looked easy but it costs money 🙂 .

            Best regards

            Tom

            (0) 
  6. Darryl Michael

    Tom / Et al,

    To add a 2015 twist to a thread that started back in 2011! The SolMan <> Panaya discussion and comparison is still alive and kicking…

    We’ve been implementing SolMan for Test Management with BPCA/UPL / Semi-dynamic TBOMs [SP11 and SP12] for Test Option 1 and Test Option 2 and are still working out some of the kinks. Two key challenges are (1) a lot of clients don’t have a BPH defined or maintained; (2) if SOLMAN-SETUP, including activating the UPL Framework, is not done methodically, a number of issues pop up at the execution level.

    Our SolMan mix now combines SoDocA [for BPH generation & usage], HP ALM using the SolMan Adapter, TAO for automation, BPCA/UPL , CDMC as well as ChaRM! A number of concurrent projects are in flight so empirical results are still awaited.

    That being said clients are looking at Panaya because of low resource impacts for getting up-and-running fast as well as comprehensive test data generation. Relatively high annual and service costs are still cited as well as some reservations with SaaS, especially with clients who are sensitive about cloud-based or off-premise access to their systems.

    Are members of this community today seeing clearer Pros & Cons for SolMan Vs Panaya, or vice versa? Am prepared to be agnostic so long as clear client-value is delivered.

    Cheers.

    Darryl

    (0) 
    1. Thomas Winkler

      Hallo Darryl!

      I see this in the same way – to mention there are 2 different approaches, which have to considered:

      1. Have the customer the solman (+tools, ..) in the customer enviroment established – or more likely [= solman fans]

      2. ALM is more or less a “green field” – with the minimum requirements (only MOPz, …)

      ad 1:

      Yes there are customers with a longterm strategy and they invest their money within SAP Tools, which where integrated, approved, recommended, …

      Here is Panaya not the right approach, because we can make much more with the complete ALM/ITSM-platform.

      • Coming from the SAP Bestpractices and recommendations, take services, …
      • Train the complete CoE organization (BASIS, APPL-Team, incl. Management Team (Reporting, ..)
      • establish new ITSM/ALM-serviceprocesses with this tool-sets

      – This would be my prefered recommendation – because it’s my consulting area

      – but it needs much time (more than 2 days) , resources and project budgets, …

      – It is hard to convince all stakeholders to take all the ALM-tools in the right manner

      – to make itself felt for a long time – it is sustainable ❗ and helps in the different phases of EACH project.

      – SAP Premiumsupportcustomers should consider to take solman here, because it makes no real sense to pay 2x times and get as a result: NO single source of truth!

      – ….

      – Solman is necessary for cCoE, needs longer time to establish, brings additional SAP Services to go very deep, the only tool which is really integrated ❗ , is in maintenance included AND for the whole SAP/IT-Stakeholders, takes other risks and differentiates the projects (Impl/Upgp. versus Maintenance projects), covers complete lifecycle, …

      ad 2:
      Otherwise – not many customers have Solman in reality in the usage with the mentioned tools and contents ready to run.

      The effort to do this, is much higher than a fast SaaS (ready to run) solution – therefore SAP can’t build a panaya substitute – for this group of customers:
      A example for the efforts for the Panaya implementation:
      Here I need 30 min for a report and a new RFC connection and then in 2 days we get  
      a complete working area, with some good reports in a very nice UI.

      So often I’ll need an update, I can get it – no developer can implement more actionpoints than before 😥 – the upgradeprojectmanager will see this in the moment.

      – here the customer will pay a fee – the investment goes externally

      – the longlasting approach is from this point very expensive

      – In this way you’ll have only a SINGLE project based approach -> after the project the chaos in the documentation is like before

      – BUT as a projectmanager you have a trusted tool.

      – Panaya is here a quick (not integrated ❗ ), deep going and expensive approach!

      The discussion was very academic, which tool is better than – because we have to look first from which situation and requirement, maturity of the customer landscape, organization, etc. .. it comes.

      I know both tools – each vendor targets different business views and needs and requirements of the customers, therefore it is hard to be only on one side.

      Servus from vienna

      Thomas

      (0) 
      1. Darryl Michael

        Thanks for the in-depth and thoughtful reply, Thomas!

        Yes, we have clients across all points of the spectrum: full-breadth ALM/SolMan users; some who are using only embedded tools [e.g. Test Option 1 including CBTA]; a number who have already invested in the HP suite [HP ALM +]/ TAO; others with Test Option 3; and still others investigating tools like Panaya.

        Your point about them choosing what works best in their context is well taken.

        The other considerations looked at include: (a) the security / privacy aspects – some clients are very sensitive to any offerings that they don’t have full control of or that sit in their controlled environments; (b) whether what’s needed is a full ALM suite or just test management.

        The discussion is actually very practical Vs theoretical, as our clients also want to know about leading practices and what others are doing, so we often point them to threads like this.

        Cheers.

        Darryl

        (0) 
        1. Tom Cenens Post author

          Hi Darryl

          Thomas is right, customers choose what fits depending on their context to the extent they are aware of things. CBTA, explained and mentioned below, is often not well known and yet undiscovered terrain for many customers.

          To revisit this blog post, I would have to review what Panaya (and others) can bring in the meantime in order to make a new comparison but what I’ve heard and seen is that some customers switch off / move away from Panaya after licensing discussions where the price of Panaya rises.

          Solution Manager has advanced in the meantime in terms of impact calculation based on UPL statistics (on customer side), shipping off statistics to SAP backend system (on SAP side) and providing results on how much impact will come by applying update / upgrade of their system for example. Something I hinted to SAP years ago when Panaya was already leveraging customer statistics and crunching numbers on their side.

          In terms of testing tools, CBTA (Component Based Test Automation) which is available in later version of SAP Solution Manager 7.1 is very promising. Testing a CRM UI based process for example (let’s say Change Request Management) is much easier with CBTA compared to leveraging HP QC/QTP. CBTA does that nearly out of the box (limited adjustment needed) where HP QC/QTP combination required lots of Visual Basic coding to be able to do the same thing.

          CBTA only covers SAP stuff though so it’s not company wide. SAP suggests combining CBTA and HP QTP to test non-SAP stuff.

          CBTA can also be used to automate the regeneration of TBOM’s (Technical Bills Of Material) which can be leveraged by BPCA (Business Process Change Analyzer) to give detailed information on the impact of changes (transport requests for example) on business processes.

          SAP keeps focusing on improving these test management scenario’s. A big plus for Enterprise Support contract customers (and higher contract types like MaxAttention etc) is that CBTA comes free of charge and is pretty interesting to test SAP UI based processes ~SAPgui, SAP Portal, SAP UI5 (haven’t tested this yet but supported), SAP CRM UI, …

          Kr

          Tom

          (0) 

Leave a Reply