Maniacal Monday – The Bugs Win
On Friday evening, after I was home and preparing for a busy weekend, not doing tech support this time other than Scouting data record keeping, I noticed a tweet from @CGibbo [AKA Chris Gibson]. It was already Saturday morning in Australia where Chris lives, so I could tell it might be important. Without hesitation, I sent the link below off to others in my organization, figuring we’d need to review it first thing Monday morning.
In case you are fortunate enough to live in a country where the time zone is fixed, and don’t need to deal with anyone in any other area where the time of say sometimes jumps forwards or backwards for one hour, you can disregard this message. For the rest of us, failures in system clocks can be a nightmare.
Possible Action Required: System time may not change properly at DST start/end dates on […]
I have not been posting much about SAP on this site lately, partly because I’ve been working on projects beyond the SAP atmosphere. Infrastructure, bolt-ons, and other applications only peripherally related (like internet and power availability) — critical stuff but nothing (literally) to write home about. One primary responsibility is with our enterprise scheduling system, or workload automation, or job management. Which brings me back to the topic of time, and system time management. A few seconds, a few minutes, or even an hour discrepancy between your wall clock and the time a job runs may not seem important, but if you miss the daily shipper pickup, you miss a day in delivery time. So keeping everything synched is important. If the rules change, as has happened in the U.S. about once every 10 or 20 years, and as happens in other countries whenever their legislators randomly decide to make up new rules, one must stay vigilant.
The bug referenced above seemed to affect new systems, not old systems, which is usually the opposite case with daylight savings time or other “rule changes.” Typically, the rules that were delivered several years ago by software and hardware vendors, before the rule changes were announced, become obsolete and must be patched. And, also typically, as legislation grinds slowly, the lead time before those new rules must be applied is months and months. Occasionally, I have seen some slip by the people who look out for this stuff, or, I’ve seen cases where the application or database rules and algorithms for time management don’t match what the hardware or operating system vendors provide. I can remember getting calls from one of our sites saying jobs were running at the wrong time of day, and finding out we missed a time rule change patch announcement completely.
Do I have to reboot?
This coming weekend, the U.S. “Springs Forward” and clocks jump from 2:00 AM to 3:00 AM. For our enterprise scheduling software, we’re well aware, and accustomed to, how job scheduling is affected. In the past, we would have to stop databases or entire systems as they were not written to accommodate time leaps. I recall having to deal with some SAP applications in that regard years ago as well, but whether that was vendor issues or self-inflicted I could not say. What’s funny is that I was just talking about not having to deal with time zone change a day or so before the vendor announcement.
To keep this more SAP focused than OS vendor, the bug that was discovered forced us to review every system in our environment to discover the risks. Unfortunately, many systems are affected, so our hardware team spent a lot of their Monday planning and communicating system outages. As one senior member said as I ruined his day, “this was not what I planned to do on Monday.” I felt bad being the bearer of bad news, but like a doctor who discovers an illness, the sooner treatment can begin the better the chances of recovery.
After I post this blog I’m going to drop a message into one of the SAP forums, for those who may have missed the announcements (that also came out on Monday). I have posted to the scheduling software forum, as the people that run your punch clocks need to know this stuff. What, you don’t know what a punch clock is?
this particular HANA instance has been up for 940366965 ms. how do i know? there seems to be a bug in the performance trace setup or i'm simply missing some privileges to stop it.
enjoying the cloud