There’s a very interesting thread going on in the SAP Mentor Forum on SCN. As it’s private, I’m not going to be quoting from it, nor revealing who is on which side. Comments posted the last several days have hit the mark as far as I am concerned, not to mention conversations with Jon Reed on exploring the topic. What I’m going to document below are comments I received on my Solution Manager presentation at SAP TechEd 2009.
It can be disheartening to see low or mixed scores, or to read negative feedback. I know the conventional wisdom is that constructive criticism is only meant to help. But gosh, what some people write can cause a newbie speaker, or someone with a thin skin, to give up trying. I’m neither, though, and will continue to present my story when I can.
I’ll split the comments into left and right – the left being “what was helpful” and right being “what could be improved” – and we all know these are thin disguises for “liked / disliked”.
|
|
[All comments transcribed verbatim. Whether the spelling errors are from the authors or the transcibers is impossible to know]
As you can see, there are more areas for improvement than for useful content. What’s funny, or perhaps sad, is that verbal comments to me were uniformly positive. I guess most people prefer to criticize anonymously, fearing a confrontation with a speaker.
Well, how much of the “could be improved” suggestions are due to my speaking skills (or perceived lack thereof), how much are due to my choice of doing a “consumers report” style analysis of what we like and don’t like about Solution Manager, and how much of the comments are completely divorced from the session I gave?
I’ll focus one that annoyed me, apologizing in advance to whoever wrote this comment, and to anyone who thought they were going to get a by-the-book lecture on how to drink the kool-aid:
- ‘Speaker did not seem to “own” content, and later said he didn’t write it. Seemed bored with topic.’
I’m not clear how this perception came across. I’ve spent time trying to get Solution Manager to deliver value back to my company. It isn’t happening. I wrote the material, I did an ASUG webcast on the same subject (there’s a podcast out there as well), and if someone heard me say I “didn’t write it”, I have no idea how that came across. “Seeming boredom”? Right; I’m no cheerleader. I’m an analyst, a critic, and a judge of how well software works. Sorry to burst your bubble.
Data Loss
The image below is from a Solution Manager report, where one week everything is fine, and the next week, no data. Turns out there’s a bug in some SAP systems, that was triggered on 22-Feb-2010. I noticed it the next Monday in Early Watch report, and it took time and effort to find the error, and make necessary corrections. As the repair depended on daily data being available, and we only keep about a week of that online, we now have holes in our data trending.
I have hardly ever heard a Teched presentation that sharply criticized an SAP product. Such feedback is more common in ASUG feedback sessions though. So the shock of hearing it was probably what caused the negative feedback to you. Halo effect kicked in, and since you were the one delivering the bad message – people will tend to identify you with the same attributes as the ones you credited Solman with 🙂
SAP deserves lot of credit for letting you present something less than flattering about their product in their premier event. (Unless some “generic” editor just spell checked your content and made sure you used the right fonts 🙂
That second question is harder. The third is the toughest in my opinion though: how do SAP and its customers get on the same page regarding Solution Manager? I’m still doing my research and due diligence, but from conversations such as the private one we have had in the Mentor forum, talks with SAP, and talks with users, I see a big gap in many cases between how SAP sees the value of SolMan and how its customers do.
As you say, “I’ve spent time trying to get Solution Manager to deliver value back to my company. It isn’t happening.” This is not the only perspective I hear out there on SolMan, but it’s certainly one I’ve heard more than a few times.
The issues with SolMan are complex. It’s not as simple as “the tool sucks” or “the tool is great but the users don’t know how to leverage it.” What I personally believe is that’s important to share honest user experiences on this topic and I’m glad you have done so. I hope that others will do so also, and that SAP takes a real concerted look at what is being said.
For my part, I plan to create more content that shares Solution Manager issues, successes, and, yes, frustrations. Perhaps it will be one more contribution to a more open dialogue on this topic.
-Jon
1) SolMan is not treated like a critical system in sites I am familiar with. The SolMan landscape doesn’t look similar to other landscapes.
2) Jim mentioned bug in “some SAP systems” generated no data. I assume something changed on 22-Feb-2010 “in some SAP systems”. Was this change implemented in other non-P systems? Was this tested in non-P systems using non-P SolMan?
3) It appears Jim’s team(+ probably SAP) took 3 weeks to identify the bug and make necessary corrections. Was this treated like a “production-down” situation and everyone worked 24×7 or treated like ‘nice-to-get-it-fixed’? If this resolution took 3 weeks in spite of right resources working , then yes, I agree SolMan has issues.
4) If 3 weeks is due to “nice-to-get-it-fixed” syndrome, then I can understand why the database restore option was not considered to make daily data available for the repair to work. I don’t have all details but this may have necessiated 3 restores. Also one lesson I would learn from this experience: next time make sure to retain all data until bug is fixed.
5) I agree Jim’s graph is not perfect. I am unhappy to see a hole for 3 weeks. However I don’t mind making an assumption that the response time for all 5 transactions remained almost same as for other weeks. Only one transaction (triangle) shows a bit wider variation in the response time for the weeks having data. In my opinion, that one graph demonstrates the ‘beauty'(half full?) of SolMan – an ability to check the health of other systems from SolMan.
Finally, I probably wouldn’t like the idea of attending a seminar/class to learn the “negatives”. As a consultant, my customers are looking for solutions for the issues they have. Negative stuff is something good to know but not critical.
Thanks,
Bala
I certianly would like to know negatives. However, do I want to pay and/or take extra efforts and time to learn primarily the negatives? Unfortunately the answer is no.
Thanks,
Bala
(1) Data silently stopped being collected. The fact that I noticed within a week is merely because I am looking closely at our systems after a hardware upgrade. A month could have easily gone by. I need monitoring systems that tell me when something is wrong with data collection.
(2) Having ZERO values inside of a table that produces averages and trends, is bogus. The chart should show “NO DATA” and not a transaction response time of 0.000.
(3) Data retention, and time-to-repair were not my point. For the former, it seems silly to expand each of my repository retentions “just-in-case”. For the latter, the underlying bug was SAP’s fault, not ours, so how is the repair time my issue? My point was adding complexity increases the likelihood of lost data, misinterpreted trending, and longer, more difficult root cause analysis.
Lastly, if you look at my TechEd presentation, and listen to the podcast, I doubt you will find it is full of negatives, despite the perceptions of some of the audience. It’s likely that those who “got it” didn’t need to write a negative review. I prefer to think of my content as a “reality check” on how well Solution Manager performs in an enterprise customer installation. If there are better, cheaper, and easier to use products to deliver the same service, how am I being negative by sharing that evaluation with my peers?
Jim
> I wanted to respond to your blog response after listening to your podcast. I have been searching for a long time and couldn’t find it. Can you provide URL to podcast?
See my blog posts entitled A Crowd of Solution Managers for a link to the audio of an ASUG webcast I did on this topic in August 2009. The MP3 file is around 30MB.
Jim
I listened to your podcast. I didn’t hear negative tone in your presentation. It sounded normal. (Note: I don’t have access to ASUG. Today I signed up for membership and would take 2-3 days before I have access to your slides).
Jim, I believe the issues I discuss below support what Jon called “the issues with SolMan are complex”.
In your cost-benefit analysis discussion, you correctly pointed out it is bit difficult to determine the benefits of maintaining SolMan system. I guess this is the primary reason why we continue(will probably continue) to have issues in SolMan. Unfortunately these days CFO is the one who approves IT requests for hardware, software etc. We in IT don’t purchase stuff as and when needed basis. When we perform capacity planning, we not only include today’s business requirements but also future growth. So what CFO knows is that he/she doesn’t need to worry about upgrading the infrastructure for next X number of years.
We (including CFO) know some level of monitoring and maintaining would be required.
I guess there are different schools of thought on quality and quantity of data needed to monitor the systems. To some (I included), the chart you provided is good enough considering the fact that we don’t invest a lot on SolMan.
Yes the chart is scientifically wrong as you pointed out: “no data” is different from value 0. However in business world, we don’t always do what is scientifically correct; we do what works.
The point I am trying to make is, Jim, why we expect SolMan should work first time and every time consistently without following the normal Software Development Life Cycle(R/3 landscape has D/Q/P (+ sandbox probably); the development team spends x amount of time in D, Dev + QA team spends y amount of time in Q and then the change is implemented in P.) And SolMan is more complex than R/3 system. The bug you reported occured in another SAP system and not in SolMan.
Another issue:
As a technical person, I understand what SAP meant by “SolMan is free”; however I guess the challenge is that there are others(a lot unfortunately) who probably thought they didn’t have to spend a dime to implement SolMan. May be when business/CFO came to know about the requirements for additional hardware, software, SAN, human resources etc. for SolMan, they became frustrated. As a result, they probably became poor listeners when we started discussing about the benefits of SolMan.
I also believe the customers rush to implement CPH. You may have already reviewed this document on CPH:
“http://www.sdn.sap.com/irj/scn/go/portal/prtroot/docs/library/uuid/90ab19e2-804a-2a10-4f86-eb41bab86636?QuickLink=index&overridelayout=true“.
I learnt two great tips from this document:
1) Don’t underestimate the complexity of CPH. It costs time to become an expert
2) Collect some data in CPH for testing only – one work week data offers all kind of challenges CPH can offer
You mentioned in your podcast that you had activated more monitoring elements than necessary. And you mentioned you didn’t activate what was needed.
I follow those tips and am having a reasonably good results with CPH.
Lastly, thanks for your book suggestion in another blog: Effective Writing for Engineers, Managers, Scientists” by H.J. Tichy. This is a great book and I wish I purchased this book several years ago.
Thanks for your time,
Bala
While I wasn’t able to attend TechED, your post just reminded me of what I heard from a few of the TechED attendees – they would prefer to attend ASUG sessions such as yours and not the “rah rah” salesy sessions from SAP. “Tell me like it is, and what I can learn from customer experiences”.
Thank you for reminding me of the value of ASUG. It was the ASUG Design team who selected your session, and as a member of that team, I am glad we did.
Tammy