The dictionary definition of a log is, “a part of the trunk or a large branch of a tree that has fallen or been cut off”. The log gives information about the tree it was cut off from, such as age of the tree, type of the tree and health of the tree. Later, the word was adopted in the marine industry to describe the route of the ship – log of a ship. IT industry adopted the term log to describe the chronology of events or changes. The formal definition of logs in IT is “timestamped history of events or changes, generated automatically, about the applications or systems”. To simplify, any information that answers the question - What happened and When? – is the log. For information to be considered as a log, it should include source, time and event (or change).
Logs are classified based on source, based on usage or based on functionality it provides. Based on the source, logs can be divided into two types – Application logs and Infrastructure logs. Application logs provides history of the changes made to the application, such as failed login attempts or who accessed the sensitive data, while infrastructure logs provide changes to the hardware, network etc. Based on the usage, logs can be categorized into five categories – informational, warning, debug, error or alert. Lastly, Audit logs, Security logs, Operation logs etc. are examples of the logs classified based on the functionality.. This category is somewhat gray and other types can be added or can be customized based on organization’s requirements.
What can logs do for me? I hear you ask. Logging and the usefulness of it, is under appreciated. Logs are generated automatically as a by-product of using an application or system and provide wealth of information at no additional cost to the organization. Some examples include, but not limited to, the following-
Statistics: At the least, logs provide statistics about your application, system or infrastructure. Logs can provide statistics like which employee is spending how much time on shopping using company’s network or which application is rarely used and organization can save money by not renewing it. Infrastructure logs can be used to provide system utilization information, such as CPU, RAM or Storage utilization. The cloud management is fully automated so users do not realize but the system utilization plays vital role in load management.
Audit Trail: As mentioned above, logs answer the questions – What happened and When? Audit trail adds a third dimension to it - Who. Audit trail answers, “Who did, what and when?” Important part of audit trail is Who. It links back to the person or system who executed the event or the change. Audit trail is used to identify and investigate issues like multiple failed login attempts by a user or who copied a file on Sunday at 6pm. Audit trail, also known as Audit logs, can be used to help in criminal investigation.
Security: Logging provides tons of information related to security. To start with, audit trail aids in security by answering one aspect of security (Who), but there are logs generated from various applications and tools used exclusively for security, such as, Firewalls, Intrusion Detection/Prevention System, Vulnerability scanners, Anti-Virus scanners. These logs can be classified as Security logs. As the name indicates, Security logs can provide insight into potential security issues. Firewall logs, for example, can show if there are any attempt to access from the part of the world where you do not have any users or IPS logs can highlight the intrusion events that have been prevented. Security logs can also be configured to use with incident response to generate alerts for ops team for the suspicious activities.
Compliance: Logs are very instrumental in maintaining compliance. Out of the 12 core requirements of PCI-DSS, logs can be helpful tool in meeting several requirements, however, requirement #10, “Track and monitor all access to network resources and cardholder data” specifically calls for tracking and monitoring logs. Health Insurance Portability and Accountability Act (HIPAA) specifies the need to track and review the information system activities – in the form of audit logs, activity reports etc. - to protect health information. NIST Special Publication 800-53 also describes the requirements of the complete Log management lifecycle – collection, analysis, protection and destruction.
Once the logs are generated, the next step is Log management. Log management in a nutshell, is the process of analyzing, storing, protecting, retaining and destroying logs. Managing logs manually is a nightmare, but thankfully automation comes to our rescue with slew of tools to manage the process efficiently without much manual intervention. It is very important to understand who manages the logs, especially when the Shared Responsibility model is used in the cloud environment. Usually, the Cloud Service Provider (CSP) manages the Infrastructure and Security logs while the Cloud user manages application and audit logs. Log management in itself is a subject of another blog.
