Skip to Content
Author's profile photo Serena PONTA

Configuration Validation and Assessment

The importance of security is nowadays well recognized and mechanisms to enforce it are being developed and adopted within enterprises. However, this is not sufficient to ensure that security requirements are met as such mechanisms have to be correctly configured and maintained at operations time. A significant share of vulnerabilities results from security misconfiguration, as shown by data breach reports such as the Uk security breach investigations report and projects such as the OWASP Top 10. The reason is that activities targeting the creation and maintenance of a secure setup, such as patch or configuration management, are labor-intense and error-prone. Software vendors, for instance, issue an increasing number of security advisories, while users, on the other hand, struggle to understand if a given vulnerability is exploitable under their particular conditions and requires immediate patching. As another example, configuration best-practice provided as prose documentation and supposingly supporting system administrators, is often very broad and ambiguous.

Due to such difficulties, configuration validation is needed to gain assurance about system security, but again, often requires manual intervention, and thus is time-consuming and limited to samples. New trends focus on providing standards for security automation, e.g., the Security Content Automation Protocol (SCAP), provided by the National Institute of Standard and Technology (NIST), whose specifications receive a lot of attention in the scope of the configuration baseline for IT products, used in US federal agencies ( SCAP comprises a language that allows the specification of machine-readable security checks to facilitate the detection of vulnerabilities caused by misconfiguration. While this represents an important step towards the standardization and exchange of security knowledge, SCAP focus on the granularity of hosts and operating systems, and as such cannot be easily applied to fine-granular and distributed systems independent from their environment, e.g., a Java Web Application. Furthermore, SCAP does not leverage standards and technologies in the area of system and configuration management, in order to, for instance, separate check logic and information about configuration retrieval.

COAS (COnfiguration Assessment as a Service) is a prototype developed within the EU project PoSecCo for the automated validation and assessment of configuration settings over distributed environments. It assesses if a discrepancy between expected and actual configuration values exist. Given a discrepancy it analyses the semantic difference in order to establish if the discrepancy is indeed a problem. COAS uses checks and checklists specified by using an extended version of the OVAL and XCCDF standards, respectively (provided within the SCAP family). OVAL checks define the logic for assessing whether a configuration setting in place is

  • Compliant to best practices and application-specific requirement
  • Subject to known vulnerabilities.

XCCDF checklists (or benchmark) define a structured collection of OVAL checks. The actual configuration values to be assessed can be provided in input to COAS together with the OVAL checks or the checklist.

An overview of the architecture of the COAS prototype is shown in the figure below.


The key features of COAS are:

  • Decoupling of check definition and collection mechanism to support the use of multiple sources, e.g., actual systems/management interfaces;
  • Standard-based language easing the re-use of existing content and enabling external authoring;
  • SOAP interface(s) to ease the integration within existing SAP products;

The tool can be used with two flavors

  • Target discovery and configuration retrieval done by consumer (easily accessible)
  • Discovery and retrieval done by service (requires CMDB and collector callback)

The result documents are provided in XML and can be useful for reporting and as audit evidence. The service provides a graphical overview of the validation result. The figure below provides the results for the HANA security checklist defined based on the SAP HANASecurity Guide.


COAS provides additional information about the discrepancy. Details for one of the failed check on system ‘vehxs002’ are shown in the figure below.

08-01-2014 12-20-08.png

Other than showing details about the evaluation that failed, specific modules are offered for the assessment of discrepancies related to access control configurations for J2EE web application, SQL-based applications. The modules provide detailed information about the differences in the assignment of permissions to users and roles between the desired and actual configurations. Finally the tool allows to manually override the result.


By assessing the compliance of information systems’ configuration settings, COAS supports and automates auditing activities and provides greater assurance to various stakeholders about the effectiveness of security controls in the operational landscape. More details about the PoSecCo project at!

Assigned Tags

      Be the first to leave a comment
      You must be Logged on to comment or reply to a post.