The Frozen Hare and the Tortoise
One Sentence Summary
How companies can find their SAP systems frozen waist deep in the ice, unable to pursue the torrent of new business opportunities rushing by – though in fact that sort of change can make you stronger, not freeze you.
I Could Have Been Someone
I Could Have Been a Table of Contents
· Three Ways to Keep up with the Joneses
· Why companies are Scared of those 3 things
· How to come out stronger from these Scary Situations
This Frozen Parrot is Dead
As been pointed out time again the pace of change in society – both technological and otherwise – is getting faster and faster and when people ask “when is this going to stop” the answer is “it isn’t”.
There are two sides to this coin and it’s a “heads I win, tails you lose” situation. With all this change come new opportunities as well as dangers and if your IT system is “agile” enough then you can bend with the wind and surge ahead of the pack. If you can’t adapt then – as I believe Charles Darwin said in “Origin of the Species” – you will be an ex-parrot, you will cease to be.
So can we in SAP world use technology to rise to this challenge? In theory – yes. As was obvious at SAP TECHED this year, SAP are not getting left behind here, the pace of change in the ABAP language alone is breathtaking, and of course there is S/4 HANA whatever in the world that might be.
What’s the Problem then?
However often to take advantage of new technology often you have to upgrade your SAP system to the latest version. And many companies are just too scared to do this. Or even to keep up to date with support packs.
If you can’t upgrade another tactic is to make use of existing technology in your current system that you have not explored yet – like BOPF or BRF+. But many companies are too scared to do that also.
All right then, at the very least you can update your custom programs – change or add extra functionality – to adapt to the changed situation. But many companies are too sacred to do that either on the grounds it generally stuffs up existing functionality.
The end result is they stay frozen waist deep in the ice, like Satan in Dante’s “Inferno” groaning in despair as they see their competitors rushing headlong into the future, leaving them behind.
Satan frozen waist deep in the Ice: Not Happy Jan
Surely you can’t be serious?
I am serious – and stop calling me Shirley. To make matters worse either people don’t acknowledge their own aversion to change, or more often are all too aware of the need for change but cite the perceived difficulty/complexity of making any sort of change to a running SAP system.
I have two quotes as examples. The first comes from the CIO of a large bank here in Australia. I cannot remember her exact words but she noted she had two different major ERP systems running her business – one was the “spaceman” system which could spin round on a sixpence at the drop of a hat and the live system could be adapted to any new thing that came along in the blink of an eye. The other was a “caveman” system where you could make minor changes once a quarter if you were really lucky.
Spaceman vs Caveman
As I am sure you have guessed SAP ECC was the “caveman” system. Hopefully I don’t think things are really as bad as that, but it does not help that all the Solution Manager presentations coming out of SAP last year kept going on about “with our latest advances you can now make TWO releases of your custom code a year!” as if that was the best thing since sliced bread. Surely you would want to make assorted changes to production once a week and aim for as many major changes as you need as and when you need them?
The second quote comes from an SAP “poster child” company, by which I mean a company which takes all the latest technology from SAP the week it comes out, and their management get wheeled out at SAPPHIRE keynotes to say how great SAP is.
Here is a quote from an ASUG online article about “Florida Crystal’s Ground-Breaking SAP S/4 HANA Story”.
“Florida Crystals Vice President and CIO Don Whittington, a former ASUG Board Member, talked about what he hoped to realise with the move (to S/4 HANA): “But what we’re looking forward to most (with HANA in-memory computing) are benefits we haven’t even dreamed of yet, similar to what we experienced with cloud technology … I imagine SAP solutions running as fast as Excel, and being as easy to use, and as easy to implement”.
I have read that quote many times
(I checked a month later and it had vanished from the online article I was looking at, though you can still see it here http://www.asugnews.com/article/being-bold-the-benefits-of-pushing-the-edges and here http://www.asugnews.com/article/asug-in-review-2015-s-4hana-reimagines-saps-business-suite )
and as far as I can see what he is saying is it would be lovely if SAP was as fast and easy to use as Excel and he imagines that might possibly happen now he is on S/4 HANA, so obviously the prior ECC system was nowhere near as good as Excel.
That is a bit of back-handed compliment to SAP, and a whopping great compliment to Microsoft. And this is supposed to be one of SAP’s biggest fans – what in the world are their enemies saying? It’s not difficult to spot which is the caveman product and which is the spaceman product here either.
I can imagine all the people who have been saying “get away from spreadsheet hell” all these years hearing the Florida Crystals quote and banging their heads on the ground.
Mr Benn to the Rescue
One person who can turn from a caveman to a spaceman in an instant is Mr Benn. He goes into the hat shop and – as if by magic – a shopkeeper appears, and gives him a hat to wear and suddenly Mr Benn becomes whatever he wants to be.
Mr. Benn can change with the wind
So how can we be like Mr. Benn? Let us recap the three areas I highlighted earlier than companies are unwilling to change for fear of breaking something – the whole system, use of new SAP technology and custom programs.
Upgrades are Scary
The other day, whilst reading about S/4 HANA my former colleague (from the late 90’s) Mark Chalfen wrote that SAP customers had been “spoilt” by how easy upgrades had been until now. I replied that they weren’t as easy as all that, sometimes the whole project takes a year from get-go to completion. He in turn replied – quite correctly – that this is usually down to the extreme complexity of the current system.
That is the exact situation I am talking about – a really complex existing solution, and even though the technical bit of an upgrade can now be done on the long weekend you want to know what to test to see if it has stopped working. That is why companies like Panaya and Intellicorp sell tools to analyse the system and guide you as to what to test – although the answer is all too often “everything!”
As might be imagined “the business” don’t want you to stop developing for even one week, let alone a month, and suck people out of their days jobs to do regression testing when a simple way to avoid both is not to do an upgrade in the first place.
In the past often the only way you could get the upgrade project over the line – and this is a true story I have seen first-hand – is to tell the powers that be that the current version is going out of support and the maintenance fee will go up from 2% to 4% of whatever it is, a huge increase. Then I saw the light bulb go on over the manager’s head and he said “Oh I See! It’s a cost saving – that’s all right then!” After all the traditional view of IT is that is just a great big cost, necessary evil, an albatross round the neck, ALBATROSS! ALBATROSS!, and if you can reduce that cost then great.
So companies would upgrade every five years or so because they had no other choice. That was the only way I personally would ever get access (at work) to the latest SAP ABAP tools, things that I had been reading about for years. In Thomas Jung’s book “Next Generation ABAP Development” the lead character Russell finds himself in just such a situation, being on the receiving end of a long delayed upgrade, and jumps for joy.
Then of course SAP had to go and pull the rug from under my (and Russel’s) feet, and announce that ECC 6.0 was supported until 2025. That’s still almost ten years away – ten years until I can use ABAP in Eclipse in my day to day work. It’s heart breaking, especially as I spend so much time playing on the latest version of ABAP in my spare time.
2025 – We will all be living on Mars and I can upgrade!
Now you might say – “hang on just a minute you most Foolish of Fools, oh Clown Prince of Foolish Fools, how Foolish thou art, what about the enhancement packs that come out every year or so. You can just pop them into your system with no disruption at all”.
Now some have said that installing an Enhancement Pack is 95% of the effort of an upgrade, and though SAP swears blind if you put one in all the new stuff is dormant so it will have no effect at all, can you really believe this? I personally think it might just be true, but a lot of companies will not take this on blind faith, and so it is “test everything” time again just like with an upgrade. I stress again it is not the technical bit that takes the time, it is the regression testing.
So in some places not only is there no chance of an enhancement pack at all ever in ECC 6.0, as when the next deadline falls it will be time to either leave SAP (not very likely) or move to S/4 HANA.
Even worse the exact same argument can be used against support stacks – when presented with the testing effort the question come back “do you have a specific problem or problem(s) the support stack will solve?” The answer is usually you don’t, because if you did, you would have installed one or more OSS notes to solve the specific problem(s).
“Nonsense, nonsense!” I hear you cry “Check your facts, oh Stupid One, companies are upgrading all the time!”
I am sure this is true to an extent, but I did notice at TECHED in Las Vegas last year there was a great big crowd to hear Karl Kessler talk about the latest ABAP features in the 7.5 release. He asked for a show of hands to see who in the audience was currently on 7.4 and he seemed quite surprised when only about 5% of the audience put their hands up. Internally at SAP they must have been on 7.4 what seemed like forever, so it must be easy to forget that the customers are not there yet. I think that explains the debacle which was the release of the 7.40 GUI. Apparently on a 7.4 system the new GUI worked just fine, and all lower releases it had so many bugs it was Hell on Earth. I was one of the idiots who installed it on the first day.
To summarise : a lot of organisations will not upgrade until there is no other choice as they are pretty sure some things are going to break, they do not know what, and would have to expand a vast amount of time and resources to find out what had in fact broken and this all too real possibility is scary.
New SAP Technology is Scary
Come on Dad – it’s obvious how to use this!
I keep coming back to this like a broken record but Graham Robinson hit the nail on the head when he did his blog about how many programmers refuse to use new technology and like to stick to function modules and DYNPRO GUI programs and what have you. After all – it works!
Round about two years ago I was thinking I might want to use BRF+ to store the complicated rules in a major project we were doing. I had thought about it but since most people did not know BRF+ from a bar of soap that would cause a maintainability problem so I backed off in the end, but I can’t help but wonder what the end result would have been had I done so, as we ended up building a very similar custom framework ourselves.
In the same way I spent the whole of last Tuesday morning making sure all my ABAP unit tests were up to date in a gigantic application I had spent literally two years building. I can tell you now I do not have even 10% of the unit tests written I really wanted, I have been building them up when I had the odd spare second over the two years, and now I have at least the “happy path” fully covered and will know instantly if any future changes break the fundamentals.
Now I would have liked to spend far more time on this but I am all too aware that in many companies a lot of people – especially management – would consider every instant I spent on such activities a waste of time. Luckily not my CIO (because he’s read my book!) but many people would not have such understanding bosses.
The “sad” thing is that doing test driven development with ABAP unit does double the initial development time, and even if that results in 90%+ reduction over the life cycle of the application (because you can change things with impunity, instantly knowing if anything is broken) the first bit is the only thing people consider i.e. the project will not come in on time, the project manager will not get the bonus, and stuff the fact the end result is an unstable application that will haunt the company till the end of time.
The funny thing is that in a recent (2014) survey (Oxford Economics: The 2020 Workforce) employees were asked what they were most worried about in their job. You would think in the unstable world we live in that getting made redundant would be top of the list. However “layoffs” came in at number 5 with 18% (you could vote for more than one).
Join the Club!
So you end up with the ever popular “rock and a hard place” situation. If you are in an environment where no-one else knows how to use a certain new technology (or even that such a thing exists) then even if it is appropriate for a new business challenge it might not be a good idea to use it as you have a maintainability problem, so you would be hurting the organisation as a whole.
Conversely, if you don’t use it, then – as really the best way to learn something new is by applying it to real life problems – you are hurting yourself, as the Kodak Bear found out.
You could argue the way out of this impasse is to train the whole team in the new whatever it is, but then you run into the “that costs money/time and there is no need as everything can be done using the way we have always done it” which is of course true and not everyone has the time or inclination to learn about new technology in their spare time as a hobby, and if you don’t know what something is all about then it is scary and its use might breaksomething.
Changing Custom Programs is Scary
“How Now Brown Cow, oh Foolish One, that is just nonsense” I hear you cry “your Foolishness knows no bounds, everyone changes custom programs all the time!” How true that is, and the danger here is all too common scenario where an application gets bigger and bigger and more and more complicated over time until it gets to a stage where it is so complicated that no-one really understands any of it, and a one line change in any part of it will always have ripple effects throughout the whole application, breaking something else – seemingly totally unrelated – without fail. That’s scary.
Now here is a little story and you have to guess if I am making it up or not. Once upon a time there lived an application that was vitally important to the business. One routine inside it performed a fairly simple function.
Over the years assorted new business scenarios came along each requiring a little tweak to the logic. Since the routine was so small it was so easy just to add a new branch to an IF statement construct and just cut and paste the five lines of code into each branch, followed by some code to cater for the particular scenario.
Then some more requirements came along, this time applying to every scenario. So the new code was added to every branch of the IF statement, again just a few lines, what does it matter?
Over the years this pattern repeated, a new IF branch here, a new chunk of code to be applied to every branch there. After ten years there were multiple branches all with a very large chunk of code in each branch, with some minor differences between the branches.
A developer, let’s call him Mr. Banana, was doing a peer review and noticed the same change – for a new business requirement – getting added in assorted different places. He expressed his concern to the programmer who had done the change – let’s call him Mr. Pineapple – that this was not only a pain for Mr. Pineapple as he was doing six times the work he really needed to do, even if it was all “Control C / Control V” stuff, and more importantly what if one day one of the many places the change was needed was forgotten, or even worse if the change ended up getting made in a different way in a different place.
The answer was simple – from a technical point of view – abstract the identical code to its own FORM routine or (dare I say it) method. Mr. Pineapple agreed – he saw the problem and said that the routine had grown over time and each time he went to change it he considered doing such refactoring but it was always “rush, rush, rush”. Moreover the program worked perfectly and the new functionality was needed tomorrow morning, a whole bunch of end users were coming in – flying in – to do some acceptance testing.
The business analyst Mr. Grapefruit was called in and said “only make the change if the business benefit outweighs the risk”. The problem of course that the business benefit of such refactoring is on the surface non-existent (zero) and the risk of making any change to a program you know that works is greater than zero (it’s so easy to break something) so the non-zero change of breaking the program is obviously bigger than the supposedly zero benefit of making the code “better”.
Mr. Banana could see why it was probably not a good idea to jeopardise the testing scheduled for tomorrow but as a general principle quoted from the 1999 book “The Pragmatic Programmer…
You might want to explain this principle to the boss by using a medical analogy: think of the code that needs refactoring as a “growth.” Removing it requires invasive surgery. You can go in now, and take it out while it is still small. Or, you could wait while it
grows and spreads—but removing it then will be both more expensive and more dangerous. Wait even longer, and you may lose the patient entirely. – Pragmatic Programmer, The: From Journeyman to Master / Andrew Hunt & David Thomas
The analyst Mr. Grapefruit said that he could see the concept, but in this case would rather take the risk of dying from a tumour at an unspecified point in the future than face the certainty of being shot by his boss the very next day if the code that worked today broke tomorrow.
It’s the same deal with the extended syntax check and code inspector. The argument goes that there is no time to deal with sort of nonsense, and anyway the code works so why bother?
This is a point I have mentioned before, again and again like a broken record, but I just can’t stop myself fixing something sub-optimal (untyped parameter or variable with a meaningless or misleading name, duplicate code or some such ) inside a routine I am working on, following the good old “boy scout rule” as penned by “Uncle Bob”
That’s not a very common attitude as I understand it – as the slightest change can break something; the usual procedure is to only make a fix after a problem has actually occurred.
As mentioned above that is analogous to never applying Support Stacks to the system, just applying a single OSS note after you have encountered an actual problem in production.
Put yet another way – “cure is better than prevention”. You don’t often hear people say that in any other area of life; it’s always the other way around – so why should it be backwards for computer software?
Too Busy to fix Sub-Optimal Code
The obvious solution to this is to start using ABAP Unit – then you would know the instant your refactoring changes broke anything, but people are too busy for that as well.
So the duplicate code and unused variables and all their mates will most likely sit there for time immemorial, laughing and singing, due to the fear you will breaksomething by removing them. Therefore, changing such custom code is scary.
One line Summary of why all this is Scary
We don’t want to change anything in case something breaks. Plus we have always done things the current way, using existing technology; and it works fine.
We’ve always done it this way!
Breaking things is sometimes a GOOD Thing
A few years ago on the SCN I came across the following blog which I found very interesting indeed:-
This was all about a book by Nicolas Nassim Taleb called “Anti-Fragile : Things that Gain from Disorder”. The book was not about software but the writer of the blog (Vikas Singh) considered that software programming could benefit from the general concepts in the book and I think he is bang on.
I finally got round to starting reading the book itself the other day. The author of the book has a word for change which is “volatility” and this means the sort of change that breaks things as a matter of course. Interestingly one such cause of dangerous change is noted as time, which of course can knock down mountains eventually. I think the second law of Thermodynamics says “entropy increases” which means that as time goes on things fall apart faster and faster.
In IT world the term for violent change is “disruptive technology” and here “disrupt” also means “break” as in breaking the mould, or breaking the dominance of the technology. So you could say that digital cameras disrupted Kodak’s film technology, no matter how many Muppets they wheeled out to keep promoting it.
When confronted with new technology that threatened to break everything about his business Kodak should have said “Who loves ya Baby?” which I understand was his catchphrase. He could have then gone on to play the villain in a Bond movie.
In SAP world the “volatility” is the stream of constant change requests to change or add functionality, which is supposed to be addressed by the constant change in the technology available to solve these problems.
In the Anti-Fragile book there are three categories of how things respond to change:-
· They are fragile, and they break, like a toaster being hit with a sledgehammer.
· They are robust and stay unchanged in the face of stress, like those Tardigrade creatures that end up unchanged by the most violent types of change e.g. death. http://www.bbc.com/earth/story/20150313-the-toughest-animals-on-earth
· They are anti-fragile, and the more they get harmed by change, the stronger they bounce back, like a Hydra growing seven heads when you cut one off whilst doing your seven labours of Hercules, a common requirement in modern day internships, or maybe getting a horrible disease like Scarlet Fever which causes you a lot of pain when you are a child, and then being immune to the disease thereafter.
It is common in fiction for the story to start with something really bad happening to the key character, and they are forced to do something in response, and at the end of the story end up in a far better position than if the bad thing had never happened in the first place.
You may have seen the film “A Shock to the System” where Michael Caine faces the scary situation of being passed over for a promotion, and so starts murdering everybody in his way a la Richard III until he ends up running the company, and likewise “Wolf” where Jack Nicholson faces of scary situation of being demoted, and so turns into a werewolf and once he starts killing and eating people and generally acting like a savage animal his superiors realise that he is ideal for management and thus he ends up running the company.
Charming as these stories are, I don’t think we need to go quite as far in order to turn our three shocking situations – upgrades, new technology and changing custom programs – into events that work for us and not against us i.e. making our computer system “anti-fragile” to such situations.
Anti-Fragile on an Upgrade Level
On a personal level I love it when the SAP system where I work is upgraded. As I stay up to date via the SCN about all the lovely new goodies in each release, when my system is upgraded I am like a bull in a china shop trying out all the new toys. These days even support stacks add new functionality e.g. the BOPF, so even if there is no upgrade I like it when support stacks go in.
However a common perception is that installing a support stack is like hitting your PC with a sledgehammer and hoping it still works afterwards, and doing an upgrade is like hitting your computer with a sledgehammer, then bathing it in a vat of sulphuric acid, then fishing it out and coating it with plastic explosive and throwing it in a live volcano, then throwing a hydrogen bomb down the volcano after it, and hoping the PC still works afterwards.
I dropped my mobile phone about a foot the other day, and it is not in a good way now, so I can see how fragile some things are. However I personally believe an SAP upgrade is not quite as dangerous as that, having gone through a fair few.
I talked about the negatives earlier, they are so obvious they do not need dwelling on, but I wouldlike to counter that with a few positives.
Some would say that you get loads of new functionality, not only tools for us developer types, but for the primary consumers, which is the actual business, if you can remember them. Others might add that you are paying a trillion billion dollars a year on maintenance fees and what you are supposed to be getting out of that is not only the pretend support, but also the “free” upgrades with all the new functions.
It’s like paying for some cheese up front, and then never eating the cheese. I use cheese as an example because of that book “Who moved my cheese?” where one of the characters says “I’m not sure I’d want to eat new cheese!” i.e. the cheese he had before was the only sort of cheese he would ever want, even when it was there no longer (out of cheese support).
Amazing as it may seem those arguments just don’t seem to work. Here are some more I can think of, which most likely will not work either.
Firstly the more often you do something, the better you get at it. That is why when you are travelling to Mars or something for 18 months your muscles start to waste away in the zero gravity as they have no work to do. If you only do an upgrade every seven years you have more or less forgotten what it is like, but when one department I knew did the “annual BW upgrade” (the technology was changing that fast) it was “ho hum, business as usual”.
Also, if you leave it ten to twenty support stacks before you install them, the delta change will be a lot greater than if you installed them twice (or even once) a year. You’d still have to do just as much testing, but less things would break because less things would have changed.
The same principle applies with upgrades – I knew one petroleum company who were hanging on for dear life to SAP Version 3.0, and the longer they waited, the more new versions came out, and the more difficult it was going to be when the inevitable happened.
Anti-Fragile on a Personal Level
Once upon a time there was someone who liked to be told exactly what to do, and was very comfortable with what they knew, what consisted of what they had been taught when they were at a large consulting company.
Then, when they were an independent consultant, because all the big consulting companies had gone belly up and made all their SAP consultants redundant, they had to work for companies which wanted SAP experts but would not spend one red cent on training.
Mr. Banana told him that the next task was to develop some custom code inside SAP which would talk to some sort of SAP Enterprise Portal application being created by a web developer.
“I don’t know anything about the Enterprise Portal” he cried, dismayed, “I have never done anything like that before.”
“Oh, you will just have to work it out yourself” said Mr. Banana “I will help – I already managed to figure a lot of it out by trial and error, and once you get there in the end you will have another thing to add to your CV!”
“I don’t want it on my CV” said the consultant “I never want to use that new scary technology ever again!”
He’s not in IT now. Last I heard he was a newsreader, most likely a lot happier in life, radical new ways of reading the news don’t come along all that often. Reading the weather is different; there are always new ways of doing that, like bouncing on a trampoline or jumping from island to island on a scale model of the UK.
Anyway, the point is, if he had soldiered on with that Enterprise Portal business he would indeed have come out of it with an improved CV. The horrible stressful change of having to learn a brand new technology might have been really painful but he would have come out of it more employable.
Even older people can learn new technology
Anti-Fragile on a Technical Level
Traditionally when we have to go and change a program due to change, it usually falls into the “fragile” category and breaks. If it manages to survive the change without breaking anything then we deem the program “robust” and break out the champagne and balloons, throw a wild party and put “Conga-Longa-Max” by Max Bygraves on the office record player, and dance around the room with joy.
You just can’t go Wronga – Singalongaconga!
We have managed to make a change without breaking anything! Surely life does not get any better than that? This is the maximum we can expect from the software lifecycle process; this is self-evident –isn’t it?
As mentioned above good old “Uncle Bob” thinks not. His position would be that although you have gotten away with it this time (because there are no pesky kids around) all you have usually managed to do is make the program more fragile with extra conditional logic branches and global variables and the like; as Mr. Scott would say “Captain! It’s all held together with bogies and string – the engines, they canna take it no more!”
Instead in the “Boy Scout” quote I alluded to above he (Uncle Bob) suggests it is possible that every change can actually make the program less fragile, and it does not take a ten ton mega-genius with bronchial pneumonia to figure out how. You do not even have to wear a woggle.
As we have seen when confronted with a new disease, provided the end result is not actually death, once the immune system has fought the new disease off the immune system thinks to itself “I will be ready for that next time” and if the same disease comes knocking the next year it gets killed before it reaches Bombay i.e. it does not cause any harm at all.
“Adaptive (or acquired) immunity creates immunological memory after an initial response to a specific pathogen, leading to an enhanced response to subsequent encounters with that same pathogen. This process of acquired immunity is the basis of vaccination.” – Wikipedia
Hence the phrase popularised by The Joker “what does not kill me makes me stranger”.
So, pretend for a moment you are the immune system, and the change request you have just successfully completed was the disease. This is not that difficult an analogy as, after all many of the change requests you get from the users / business analysts probably make you physically sick.
There are several questions you have to ask your little lymphocyte self about what just happened:-
· Can it happen again ( answer = YES )
· What was difficult about how I worked out how to fix the problem?
· Can I strengthen the defences in some way to be ready for the next time this happens?
For the first question the answer is obvious – if it happened once it can happen again. You also need to ask yourself can something similar happen? Let us say your program presumed a one to one relationship between a sales organisation (VKORG) and a company code (BUKRS) and then the organisational structure changed and that assumption went out the window, and your program fell in a heap.
You can use table TVKO to get the company code for a sales organisation, rather than using the two elements interchangeably as you may have done before. With that problem out of the way you should then look for similar assumptions e.g. an assumption that there is a one to one relationship between a profit centre and a cost centre, or any other silliness which may work at the moment but could break at any time if the configuration or master data changes.
Another example – I have even seen a program which assumed a purchase order number was nine characters long and started with “45”. Most purchase orders in most companies do start with “45” but most are ten characters long and anyway, field EKKO-BSTYP is a better evaluation mechanism to see if something is a purchase order as opposed to a schedule agreement or something.
The next point is – of the problem happened in the program at hand, and is down to some sort of faulty assumption or whatever, is that same assumption or problem also in other existing programs but no-one has encountered it yet? Perhaps making a pre-emptive strike here is the go, rather than waiting for the users to find the problem?
Even if there is no actual problem but you have found an assumption which can break in the future e.g. one cost centre = one profit centre, would it not be a good idea to prevent that potential problem in every program it could occur? That could, admittedly, be a bit of a hard sell. Immune system cells are lucky – if they find a problem in one part of the body they can travel to every other part of the body ready to guard against that problem, without having to fill in a business case evaluation. This is partly because a single celled organism often cannot pick up a pen to sign the document.
Amazingly some people might say that is a silly analogy – they would spit in my face and say a company doing such pre-emptive work in advance of a problem actually occurring just sucks up time and money that could be spent elsewhere. Well, where is the immune system getting its energy from? The Moon? The non-conscious part of the human organism lets the immune system have all the resources it wants as it is wholly focussed on survival, unlike most companies, where politics is the primary driver and the manager who stops the immune system from functioning can jump out of the patient before it dies, and jump into another host, and get more money and prestige as a result (and in the UK a Knighthood).
How difficult was the fix?
As someone somewhere once said “Can we fix it? Yes we can! Can we fix it? Yes we can!” Generally it is not the end of the world to fix whatever the problem is. However I have found that a fairly large amount of the work involves replicating the problem, and the huge bulk involves locating exactly where things are going wrong, and once you have got to that point fixing matters up is a walk in the park.
As can be imagined if the problem is in the area of Scooby Doo it would be good if the routine / method was called “Scooby Doo” but it often seems that when developers name routines / methods they have a temporary attack of mild insanity and call the routine “DOG_DAY_IS_OVER” making the developer trying to fix the problem ask “Scooby Doo – Where are You?”.
SAP have a horrible habit of putting routines in includes with names like R54RZSD instead of “release credit block” and a lot of programmers have decided that is “best practice” and followed suit.
When I started learning German it made debugging standard SAP programs a lot easier but sad to say a lot of routine names make no sense in either German or English.
The point I am trying to make here is that if it takes you half a day to even find the code where the problem is occurring then next year when a similar problem occurs will it still take you half a day to find it again? Human nature is to forget how your own code works, let alone someone else’s.
The answer is so obvious I was not even going to say it, but just in case – “give routines/methods/variables a meaningful name”.
Stupid and/or joke names are even worse. A new programmer won’t be in on the joke and will have no clue what the silly name represents. This is why a lot of ABAP programmers are often scared of web development – they see a big list of tools you need, all with silly names like GIT, GRUNT, FOSSIL, JENKINS, ANT AND DEC, PINKY PONK, NINKY NONK, IGGLE PIGGLE and UPSY DAISY and cannot guess what the tool does from the name. Not that you could guess what ABAP means, or any of the ever changing forest of SAP acronyms. At least “Service Adaption Definition Language” sounds more grown up than “Ninky Nonk” even if you don’t know what either of them means.
In any event, moving to the opposite extreme, the latest SAP programming model – CDS views – is predicated upon being “as close to conceptual thinking as possible”. This is not a new idea, many academics (for example, Spiderman) have created whole new programming languages, to try and have the program read like plain English.
Furthermore when you have your code reading like English – again as mentioned in all those programming textbooks, such as “The Code Cleaner” and “Dougal and the Blue Cat” – it will be obvious when it is wrong.
IF x > y
IF number_of_hotel_floors > number_of_stars_in_the_sky ….
The syntax check will never work out this is an impossible situation, and whilst the variables names are meaningless neither will someone reading the code, but as soon as the variable names mean something any human can see what is going on.
The above blog was not about ABAP but the idea is the same – if you name everything in human terms it becomes obvious when the code is wrong in the same way a wrongly structured written or spoken sentence is clearly wrong.
Another quote from Uncle Bob is “we are authors”. Your code is going to get read by real people, and they view things very differently from a machine. Machines are quite happy with every variable being one letter long.
As a – very obscure – analogy, if a man goes into a clothes shop to buy a suit and the assistant says to him “X?” that would most likely puzzle the vast majority of customers.
Instead, in the real world, if the clothes shop happened to be located in the UK, the shopkeeper would say something like “I’m Free!” or “Oh! Oh! Suits You Sir! Suits You! Oh! Oh!” Some would say that’s not much better, but that’s what actually happens.
This might also explain why most men’s clothes in the UK are bought by women on the man’s behalf but anyway if you can stop messing about for just one second and get back to talking about programming the point is that if it has taken me half an hour to track down the routine in a custom program that changes everything and I could not find it because it was called ZCD4356 then after fixing the problem I am also going to change the name of the routine to “CHANGES_EVERYTHING” so next time I go looking for it I can find it.
This principle was invented by top programming duo Climie/Fisher (hence the name) and I was only seventeen when I started programming that way. It seems like yesterday.
How to strengthen the defences?
We just talked – at inordinate length – about how naming things sensibly can make life easier the next time we come back to the program i.e. because we did this renaming the very fact there was a problem with the program in the first place made the end result easier to read, understand and thus maintain.
Next we have the situation Mr. Banana and Mr. Grapefruit were arguing about earlier – if you had to make the same change in multiple chunks of identical code then clearly abstracting that code will speed up similar changes in the future, make sure there is no disparity between the way various sections of the program do the same thing, and make the code easier to read to boot.
Another fun thing about duplicate code is how it usually arrives – via cut and paste. You copy a section of code dealing with goats, and then change all the variables to do with goats so they now deal with sheep, but miss one, so the program ends up doing something crazy like reading the goat table and putting the value in the sheep variable. Having the duplicate code in its own method where the things that change are passed in and out as parameters goes a long way towards solving that problem.
Moving along, the warnings in the extended program check and code inspector are not just there for the fun of it – as just one example removing unused variables is good as it frees up just a tiny bit of memory, but more importantly makes the program easier to read. In the 740 version for ABAP and above those unused variables should be a lot rarer as often you only declare them inline the first time they are used.
Moreover ABAP in Eclipse has automated tools to help you with those two examples – abstracting out code and removing unused variables.
Another one is an assumption that there will only ever be one material type for widgets and suddenly there are two, so you replace a hard coded value with a range that is read from a customising table.
I could go on all day – the point is that every time you fix a hard coded problem, instead of making it worse – longer routines, more conditional logic – you can make it better.
No Risk = No Progress?
SAP sometimes gets the reputation of a “caveman” system, not able to respond to change in an “agile” manner. Where this occurs it cannot be laid at the door of the company SAP but rather the fear that some companies (people) have of encountering pain when something breaks as a result of changing anything other than the bare minimum in the system.
This takes (at least) three forms, and in each case you can make the pain work for you, thus making your system more “anti-fragile” i.e. something that benefits from the grief caused by the changes needed to keep up with a world that spins faster and faster each year.
One – changes don’t get more painful than an upgrade or even a support stack. However doing such an exercise makes you come out stronger as the more often you do them (a) the better at them you will become and (b) the smaller the delta change will be so the less will break.
Two – leaving your comfort zone and learning a new technology seems so painful it can make developers want to run and hide behind the sofa, but you come out of it stronger as you are more employable.
Three – constantly cleaning up your custom programs as you are forced to change them due to business requirements is a risk as if such a non-mandatory change breaks something your boss will shoot you with a machine gun and that is painful. However using this technique (cleaning your code, not shooting people with a machine gun) makes your custom programs stronger as they are easier to understand and thus maintain the next time a problem comes along, and moreover they are less likely to break and have that problem in the first place.
That’s All Folks
Most of the points in this blog I have covered before at some stage, though not combined in this fashion.
In my personal situation I am allowed to clean up the custom code to make it more robust as time goes by, but the prospect of an upgrade is many years away, and support stacks are not very likely on the application side.
I am would be interested to know – have you managed to persuade the higher ups that an SAP upgrade is a good thing, and as a result it actually happened? If so, what arguments did you find worked?
Paul, another insightful blog... perhaps inciteful too, as in you'd no doubt like to incite some change in the hearts of others with the 'caveman' approach to what should theoretically be 'spaceman' systems!
For much of it I felt like I was reading the story of my own shop. Are you sure you aren't actually one of my teammates in disguise, posing as Paul Hardy, famous book author?
In other words, yes, we avoid change like the plague here, much to my own chagrin. We have a history of applying Enhancement Packs in the past and then never activating any new functionality because it scared us too much, or didn't work in the old-fashioned way we wanted it to, so historically management has been resistant to doing it again, since we go through a lot of trouble (all that regression testing!), and things break, and we don't gain anything from it anyway. So, traditionally, we only upgrade when End of Life -- and further, End of Extended Maintenance -- forces us to. We just completed an upgrade away from SRM 5.0, which has been beyond Extended Maintenance lifetime for a while.
Support Packs are considered by the business as a necessary once-per-year evil, and then only because they are required in Payroll for year-end tax reporting legal changes that aren't delivered any other way. If we can apply just the HR legal change support packs without requiring a full stack update, then there is significant pressure to only do that. Of course, that meant this past season we had to do a full stack update, and it was quite a change. It also meant we had to wait for some of the nice Basis and ABAP updates that came only with the newer stack.
But... wonder of wonders... we are now going to embark on another Enhancement Pack upgrade of our core ERP system, because... miracle of miracles... management is now very enamored of some of the wonderful things we could do if only we were on the latest and greatest release. The business is dubious, of course, but it looks like it Might Actually Happen.
So .... putting in enhancement packs and never switching any business functions on .... only putting in support packs because of HR ... not upgrading till end of life.
You are right. We must be working in the same place.
The one time we did try to switch a business function on - multiple account assignment - we discovered far too late it does not work until you are on EHP6, and it's non-reversible. So development has been out of synch with test and production, GR/IR wise, for two years. During that period SAP have been throwing OSS notes at the problem, 57 thus far, most fixing a problem caused by the previous OSS note. The original problem was that the SAP programmer had mixed up BUKRS and WERKS and started using them interchangeable, so you had standard SAP programs saying "Company Code XYZ does not exist" when XYZ was actually the plant in the purchase order"
Based on that lovely experience I can see why no-one was in a mad rush to try and switch anything else on.
Paul, I'm afraid it's all too common a story.
Actually, one of our biggest holdups has been Concurrent Employment. Being a K-12 Education shop, we actually use this piece of obscure and relatively undeveloped functionality, and that means when new developments come out, CE-enablement of those new developments typically lags far behind, so we have to wait anyway. Then when a fancy new CE enhancement is actually released, it represents such a radical change to the master data structure that enabling it is something of a major project with obvious rewards but also significant risks. Benefits Administration is a perfect example of this. Currently, we store benefits information, such as dependents and so forth, on the Personnel Assignment level, as that is how the standard system worked when we implemented HR. So, if an employee terminated one assignment but kept a second (and/or third) one, all that dependent information had to be copied out of the terminated assignment and moved to one of the remaining active ones. Like many customers dealing with this, we developed procedures and reports and customizations to help us get around this trickiness, but it's still cumbersome. Lo and behold, SAP delivers an enhancement pack that allows the migration of benefits data to the Person ID level, so it will be common across all assignments. Wonderful! Just what we want, and how it always should have been! Except now all of our reports and customizations and so forth are instantly obsolete, and we'll have to develop new reports and processes and so forth. Ideally, it would be a no-brainer to do this, but the business says "We already have reports for this, we want your developers to work on this other functionality..." and so we don't get to revisit Benefits in this way. So, no activation of that function.
GREAT blog! Yes, I read the whole thing and didn't even nod off once! 😛
I would add another scenario to your lengthy list....what about the technologies SAP jumps on and/or develops that are "neat" but have no immediate real world use/proof/etc, yet SAP will tell you how much you need it. haha HANA is the obvious one....because you know....closing those monthly books at lightening speed is more than enough to justify the costs of a project to implement this new whiz bang tech!....and SAP isn't alone in this....just look at things like Oculus Rift...now everyone wants to tell you how great VR is and how you need everything VR-enabled. I am already seeing some fatigue on this one. Oh yeh...what about those nifty high price big,flat screen 3-D tvs? Every channel and station was going to jump all over 3-D programming....you better get on that ride quick!...football games in 3-D so you feel right in the action!!!!.....remember that? No....oh you blinked? Yep...it was here and gone that fast. haha Soooo there is that angle as well...customers have to be careful not to jump to the next new shiny tech that is being shovel fed to them....which often leads to the much dreaded "paralysis by analysis" syndrome. 😉
Matt Fraser ...it lives!!!!! *SQEEEEEE!!!!!! *** 😛 😆 🙂 😈
I'm guessing this is the comment that got blocked? lol.
Sitting in the mod queue waiting for a kind lovely mod to come along and set it free to run wild.
No wonder it got moderated - doubting HANA value, what blasphemy! 😀
Oh it's not that I doubt it...I think HANA is all amazing and a great platform....BUT it seems a bit overkill in a lot of cases/customers for day-to-day backoffice use/needs.....I mean, sure, you could put a state-of-the-art jet engine in my car and I am quite sure I would get where I am going faster....but what if I just wait a bit longer for flying cars or Star Trek-ish transporters. 😉 😛 😆
Chris, wouldn't a jet engine in your car simply result in your car smashing into things out of control due to being overpowered? Right tool for the job.... 😉
Whoa? What? My post was immediately hidden....one message said my post was under review from mods due to my lack of points. WHAT?!?!?! haha The others said "This comment has been hidden. This can happen if the comment has been hidden by a moderator, or has been reported as abusive"....man, that was quick! haha
Heh, now one has to wonder just what you said in your earlier post (do you mean comment?) that triggered the automatic moderation. 😉
As an FYI, the "due to lack of points" message is generic, and has nothing to do with any actual reason for auto-moderation (obviously you have plenty of points). What probably happened is you used one of the super-secret keywords that caused your comment to be auto-blocked.
That's the thing! I actually didn't say much except "great blog" and how new tech comes out without a clear "use case" while being shoveled down customer throats as "must have". Gave examples of a few (non-SAP even!) ....and boom....DENIED! haha
"Seven words you can't say on SCN", huh? 🙂
Great blog, Paul, as always (I tend to rate your blogs 5 stars before even reading, haven't had to change it so far 🙂 ).
Compared to the SAP customer base, our "shop" is probably rather medium-size with a very small SAP support team. For us the support pack "upgrade" is usually just a question of the resource availability and budget. If we are lucky, we might have a project that business needs and that requires certain SPS. Otherwise we tend to use "hey, it's been too long since last upgrade" argument. (If we are also lucky there might actually already be a precedent of an SAP note that can't be applied unless we upgrade.) And I agree with you that the more you upgrade the better you get at it. "Practice makes perfect".
On the naming subject - just the other day I was searching for a function I created myself about 5 years ago. This function was related to a project called, say, "Awesome Website". But searching by either z*awesome* or z*website* didn't bring any results. After some head scratching, I found that function was actually named something like ZRFC_CREATE_ORDER. Then I remembered that at the time there were some plans (never materialized) to use the same function for other interfaces. Therefore a more generic name was chosen. But now I'm not sure if I actually did a disservice to the future generations of ABAPers who no doubt would also look for z*awesome*. 🙂
You cannot have a blog entitled "Frozen" without someone telling you to "let it go".
Firstly, for those who don't know the expression, "Not Happy Jan", here's where the famous Australian saying came from:
[embed width="425" height="350"]https://www.youtube.com/embed/2akt3P8ltLM[/embed]
Secondly, I've been lucky and either had the right mind set in place already or been able to encourage upgrades/patching early enough so that they do occur (albeit this does mean I'm one of those who gets to put wild and wonderful new stuff on their CV all the time with the requirement to learn very fast) but with support packs, enhancement packs and most recently, even Kernel updates - the trust is being lost with functionality changes. e.g. A kernel update to support Personas recently blew up the document management component; and I found out after I said "You should update Kernel updates without fear, as they shouldn't break things provided you've done some level of testing outside of prod...".
Similarly, I cannot look at dodgy code and refactor if I need to touch the code where there's obvious:
a) Missing error handling;
b) Poor error messages;
c) Poorly named variables that make sense;
d) Incredibly long methods or functions (this one is obviously the most dangerous);
e) Opportunity for reuse rather than copy and paste;
Though I have to factor how much time I can spend on this without going overboard (though it was annoying recently when my naming standard didn't have the useless v in it for variable and had to use ADT to refactor lots of code).
Anyway, good luck in being upgraded - it will be definitely worth it even if it is just to keep you happy 🙂 .
Thank you very much for this blog post. It was pure joy to read it, both for the insights and for the writing. I just ordered your ABAP book, again for both. 🙂
Stellar stuff Paul!
The best customers I have seen have a regular upgrade schedule - typically quarterly & they usually upgrade to current SP-1. The -1 reduces risk as hopefully the worst of any issues have already been reported & have SAP Notes already released.
Coincidentally these are often the most innovative & early adopters. Regular updates mean the tech is not in their way when they want to experiment with something new. After all it usually takes a couple of months to wrap your head around new stuff, come up with a suitable use case & find a sponsor.
For the rest I add my favourite Strictly Ballroom quote...
"A life live in fear is a life half-lived"
...and watch that increasing count on the number of Fortune 500 companies who keep disappearing... Mostly by failing to adapt. Playing safe can seriously reduce your lifespan
10 years to extinction: S&P 500 companies
Jocelyn, I have to say I have never myself been a subscriber to the SP-1 philosophy. All those correction Notes are contained in the next support pack, usually, and so why not apply them the easy way, vs applying a great many correction notes separately (and then dealing with them all in SPAU with the next round of support packs)? I do think it makes sense to not apply a new SP the moment it is released -- give it time to percolate and let the obvious correction Notes accumulate.
Gotta say...I have been on many client sites that take the "SP minus one" approach. I too never "got it", but then, I don't have the "luxury" of enjoying "ease and zero disruption" (at least from all I have read from SAP 😛 ) of applying packs/notes/etc.
No doubt an excellent blog!
I liked every line of this document expressed very well the paradigms of the companies on the changes in their IT sector (not only in the SAP world).
Absolutely incredible blog, and it happens to relate to my daily situation right now.
Very insightful and a joy to read! Thanks!
I did not have the time to read it thouroughly,
but "cross reading" and "picture looking" caused * * * * *