Intro
Adoption of API-led and event-driven architectures, combined with the encouragement of an API-as-a-Product model and advancement of the mindset that gravitates towards product team-entered, decentralised ownership of a fleet of APIs in an enterprise IT landscape, taken all together, emphasise the necessity and criticality of API governance. One of the key aspects and integral parts of that is adherence to established organisation-wide best practices and style guides that aim to standardise and consistently design, develop, and publish APIs.
Usually, an API style guide includes naming conventions and recommendations about design patterns and design decisions. While a lot of key principles and patterns originate from well-established, time-proven, and battle-tested constraints (for example, constraints for RESTful APIs, common best practices about what qualities a well-designed RESTful API or event-driven API shall demonstrate), many companies choose to tailor and extend these principles to fit the organisation in the best way.
For RESTful APIs, there is a plethora of examples of corresponding style guides that have been published by various companies. To get inspired by some examples, you may familiarise yourself with a collection of API style guides published here.
This practice is less common and yet emerging for event-driven APIs as corresponding concepts and standards have been introduced more recently, but we can evidence a growing interest in event-driven architecture and shall expect to see more style guides that apply to event-driven APIs in course of time.
API style guide enforcement problems
While the value of API style guides is no doubt, a format that is traditionally chosen - a comprehensive document that is published and maintained in an enterprise content collaboration platform - implies certain challenges and risks that get amplified when trying to enforce such guidelines across many teams, naturally and effectively embed them into processes of designing and developing APIs at scale:
- Considerable time and effort are required for newly onboarded API architects and developers to familiarise themselves with and memorise all rules contained in an API style guide,
- Interested parties may occasionally miss updates to an API style guide,
- A non-trivial and time-consuming process of verifying APIs’ compliance and adherence to an API style guide, especially when this process is not automated and requires a substantial manual effort (for example, when it is a part of API design review sessions).
The above-mentioned challenges lead to higher risks of inconsistencies among developed APIs, longer API design and development period, and endanger the evolution and advancement of an enterprise API initiative/program. One way to address these hurdles and mitigate associated risks is to leverage techniques and tools that help automate style guide compliance checks for artifacts that form a basis of an API design.
While different competing standards may be used to produce API design artifacts and to compose or generate an API contract, nowadays the highest level of recognition and adoption belongs to:
Despite perceived and apparent differences in integration styles, use cases, scenarios, and the entire underlying mechanics that are associated with these two API styles, they have a substantial commonality in standards that are widely used to define them: an AsyncAPI standard has been significantly inspired by an OpenAPI standard and both stipulate the use of YAML and JSON formats to represent an API contract definition.
Linter to the rescue!
A well-written API style guide is a set of rules that an API contract must comply with. A RESTful/event-driven API contract is a well-formed YAML or JSON document that complies with the corresponding API definition standard. What are we missing to join the dots and automate the validation of an API contract against rules contained in an API style guide? A structured, strictly formalised representation of validation rules and a capable JSON/YAML linter!
Same as for other flavours of linters, a JSON/YAML linter must inspect a validated document (here, an API contract defined using an OpenAPI/AsyncAPI standard) against a provided ruleset (here, an API style guide ruleset), and produce an output that lists detected discrepancies, non-adherence to rules (e.g., standards and conventions).
The linter should also support:
- Integration into API contract design tools and editors. This facilitates rapid checks and feedback to API designers even before an API contract is committed,
- Standalone execution. This enables integration into a CI/CD pipeline as one of the mandatory quality assurance steps.
We can find a lot of linters that satisfy these requirements either in part or in full - both web-based online versions, standalone applications, and plugins/extensions for code editors. Here are just a few feature-rich linters that are worth considering and trying:
There are also many frameworks and libraries available for different programming languages that can be embedded in developed applications and that can parse and lint JSON/YAML documents either directly, or with the help of generated neutral document model representations - for example, an AML Modeling Framework.
These tools vary in maturity level and features, all of them are equipped with capabilities to perform OpenAPI linting, and some of the mentioned tools can perform AsyncAPI linting as well.
Spectral overview
Amongst a variety of available tools, I would like to draw attention and give prominence to the Spectral tool. Spectral can be used both as an OpenAPI and AsyncAPI linter. Moreover, the tool is a general-purpose JSON/YAML linter, so areas of its use go far beyond the linting of OpenAPI and AsyncAPI documents.
The core tool is written in TypeScript and supports different distribution and installation methods (a Node.js module, an executable, a Docker image) as well as has extensions/plugins for popular code editors (such as Microsoft Visual Studio Code) and IDEs (such as IntelliJ IDEA and other IDEs from JetBrains). In my opinion, the tool provides splendid functionality and handy extensibility options.
Spectral includes built-in rulesets for the validation of OpenAPI and AsyncAPI documents – corresponding rulesets are shipped as a part of the Spectral distribution and are documented here for OpenAPI rules and here for AsyncAPI rules. More recently, an additional ruleset that implements validations for OWASP API Security risks, has been released and is available here.
In addition to that, several companies published their rulesets. These rulesets are a good source of inspiration and provide a solid foundation for adopting and creating custom-tailored rulesets. Relevant rulesets can be found here.
Moreover, Spectral provides convenient ways for the customisation of provided rulesets and the development of custom rules. In more complex scenarios, custom functions can be developed to implement advanced validation logic and to use those functions in rules.
Rulesets can be shared within the organisation and outside of it, if necessary – they can be hosted on a file system, served from a content management server / a repository using an HTTP server, or distributed using Node Package Manager (npm) by bundling rulesets into npm packages. In this way, a tailored and customised ruleset and accompanying custom functions can be consolidated and maintained centrally and consistently using a version control system (for example, Git-based), with the proper workflow and change management in place, and represent a single source of truth when it comes to an enterprise API style guide.
Spectral in action: basic examples
To begin with, let’s have a look at how Spectral can be embedded into the used OpenAPI/AsyncAPI document editor. There are sophisticated visual and code editors - Stoplight Studio, Hackolade, Apicurio Studio, SwaggerHub / Swagger Editor, and AsyncAPI Studio, to name a few. Rather than using a special-purpose OpenAPI/AsyncAPI editor, here I will use an alternative approach and will make use of one of the popular multi-purpose code editors - Microsoft Visual Studio Code. Its core functionality can be enriched with the help of various extensions, including those turning it into a proper OpenAPI/AsyncAPI editor (some good examples of such extensions are an OpenAPI/Swagger Editor extension for OpenAPI and an AsyncAPI Preview extension for AsyncAPI). One of the available extensions is a Spectral extension, and this is what I’m going to use in my first example.
The Spectral extension can be configured in VS Code settings - for example, we can use a custom ruleset as opposed to built-in rulesets. Below are illustrations of how the Spectral extension produces an output of linting using built-in rulesets when being applied to the above-mentioned API contracts:
Spectral is also distributed as a command-line interface tool that is distributed in several different guises. To proceed with my next demonstration and given I already have Node.js (including npm) installed on my machine, I installed Spectral as a Node.js module in a global mode using npm:
npm install --global @stoplight/spectral-cliWith the help of different output formats, we can make a linting output format compatible with such tools as JUnit and TeamCity, or produce an output in generic plain text, beautified text, JSON, or HTML to allow its further parsing and analysis using other relevant tools.
Customisation: brief notes on rulesets and custom functions development
Spectral’s true power and potential come from the capability to customise and extend existing rulesets and develop custom rulesets. Rulesets are collections of rules developed using YAML, JSON, or JavaScript/TypeScript. Rules are either inherited from another ruleset (when a ruleset is an extension of another) or custom-built rules. Every rule is described in a declarative way and must adhere to a defined structure. While the overall structure of a rule is more complex and includes additional properties, a few of the most important components of the rule are:
- Selector. A JSONPath Plus-based expression is used to select the part of the validated OpenAPI/AsyncAPI document, to which a validation function must be applied.
- Function. A reference to a function that implements validation logic to be applied to the selection.
- Message. A meaningful message is produced in case the validated assertion fails.
- Severity level. A severity level (error, warn, info, hint) of the produced message.
Spectral already includes a rich selection of built-in functions that are a part of the core of the tool and that can be used in rules, but if some advanced and highly specific logic is required in a validation/assertion, a custom function can be developed and used in a rule. Custom functions are developed using JavaScript/TypeScript. A function must accept an input value (a value that has to be validated), may also accept options (in case a function allows customisation – if that’s the case, function parameters shall be specified in a rule configuration), and must return a result message (or multiple messages) in case the validation fails.
Further reading
Spectral is accompanied by comprehensive documentation that contains a lot of details about the tool, its installation and usage, as well as rulesets customisation, development of rules and custom functions. The documentation contains a lot of examples that are worth getting acquainted with. Particular attention should be given to built-in rulesets that provide a solid foundation and a high-quality basis.
Besides the official documentation, I strongly encourage you to have a look at API style guides and rulesets that have been published by some other organisations (I provided links to some of those resources throughout this blog post), they exemplify additional real-life recommendations and concerns that originate from API style guides, illustrate customisation of rulesets, development of custom functions and assertions.
Many more examples of Spectral rulesets and custom functions can be found in public code repositories such as GitHub and an npm registry – search for the spectral ruleset.
- SAP Managed Tags:
2 Comments
