How to do (or not) an application log audit. (Fun)
As an alternative to real-time reporting, I sometimes hear that SAP systems are checked for real incidents at regular intervals (e.g. monthly). For this purpose, the SAP Security Audit Log among many other logs are checked for violations of system integrity.
Based on this report, customers can then take appropriate measures to mitigate or report the violation.
A small to medium SAP Application landscape generates 5.000 to 20.000 log events per second.
IF we do a quick calculation:
20.000 events per sec. x 60 sec. x 60 min. x 24 hours x 30 days = 51.840.000.000 in words fifty-one billion eight hundred forty million log entries per month.
In my opinion: Nobody can seriously read and crosscheck this!
I imagine such an audit or check could look like this:
“Good Morning, Captain Kirk.”
“Good Morning, Spock, what’s up?”
“Today is audit day again.”
“Oh, no! Not again, Spock!”
The 1.970 metric ton heavy (4.350.000 pounds) and 50-kilometre-high logbook falls crashing onto the control panel of Captain Kirk.
Much fun checking the math – our assumptions are:
One SAP Application Event has ~ 300 – 400 characters.
Paper we use: DIN A4 sheets (8-1/4 x 11-3/4 inches).
With a character size of 6 and no margins we can assume to fit one event in two lines.
We assume we can write 115 lines per DIN A4 page (8-1/4 x 11-3/4 inches); double-sided we can write 230 lines = 115 events per sheet.
Weight of paper: 0,07 kg per square meter; with pretty exactly 16 sheets per square meter one sheet weighs about 0,004375 kg.
51.840.000.000 events / 115 events per sheet = 450.782.608 sheets.
450.782.608 sheets x 0,004375 KG = 1.972.173 KG. ~ 4.350.000 Pounds.
A normal 70g/m² paper has a thickness of 0,112mm.
450.782.608 pages x 0,112mm = 50.487.652mm = 50.487 meter ~ 50 km high.
“Okay, Spock, let’s do it the usual way. I’ll randomly select 1.000 pages and we’ll see what we can find.”
2 days later …
“Captain. Somebody made a five gigatons download from our top-secret map of the unexplored regions of the galaxy, we suppose that our greatest treasures are hidden there”.
“Very well done, Spock! What an excellent piece of good luck indeed! What are the standard security procedures?”
“We must report the incident within 72 hours and determine what data was downloaded.”
“Can we find out which data was downloaded and where was it downloaded to?”
“Yes, we just need to quickly analyze all the log data, Capitan.”
(just to remind you, we talk about 51.840.000.000 log events per month. A log entry has about 10 – 20 strings / IP addresses, time stamps etc. We calculated with 15 strings per event. Fast readers get up to 1000 strings per minute. 1.000 strings per minute are extremely fast. And you have to remember to crosscheck and correlate all these events in a useful manner.)
Spock sets the local Borg data security office team of 30 Borg to work, day and night, without break, weekend, or vacation.
These are exceedingly fast readers, managing 1,000 words per minute and are mentally interlinked to also do cross-checks and correlation of the different events on the fly.
A log entry has about 10-20 strings (IP address, time stamps, etc.) – let’s see them as one word each although maybe a slightly complex one.
Much fun again with math!
Seven hundred and seventy-seven million six hundred thousand minutes later / 30 Borg = 50 years.
“Captain, we have the result. The data has been downloaded from the Black Planet.”
“Were we able to find out what was downloaded, Spock?”
“Unfortunately, no, sir. Our read access logs go back ten days, in order to cut cost on data storage and maximize our profits.”
“Very well then, Spock. Please go ahead and check the security procedure.”
“According to our regulations, no data loss could be proven, the data is still there.”
– The bureaucratic mentality is the only one you can rely on in the universe. –
“Perfect, Spock. Then we can finally close the case. Please take a note:
‘Logbook entry Star Time 4-1-15-06-2070: Event analysis 4-1-15-06-2020 completed, no proof of lost data possible.’”
“Captain, do you have any further specific recommendations?”
“No, Spock, everything is fine. Carry on!”
– Fate protects little children, and ships named Enterprise. –
How do you review your application logs?
P.S.: Save trees: please do not print out your application logs.
Co-Author: Tobias Keller