Alternatives to Postman for Exploratory and Automated API Testing

Postman is a great tool for exploratory API Testing. However, it is not the best tool for automated API system testing and there will be a change in September which can result in higher costs for using Postman. In this blog post, I will explain why from my perspective code based open source tools are often better for automated API system testing than GUI based tools like Postman or Insomnia. I will also provide some recommendations which tool to use for which technology stack.

System Testing against the API layer is a vital part of software development, as it ensures that the APIs are functioning correctly and meeting the expected requirements. However, not all API testing tools are created equal. In this blog post, I will compare and contrast two types of API testing tools: code based open source tools and GUI based tools like Postman or Insomnia. While Postman and Insomnia have both value in the context of exploratory API Testing – I will argue that code based open source tools are better for automated API system testing, especially in terms of maintainability and integration into existing development infrastructures.

Postman change on the 15th of September 2023

On September 15th, Postman will remove the legacy Scratch Pad component from its Postman Desktop App, but you can still continue using the same API testing, documentation and design functionality in the same app for free, in combination with a SaaS account that provides additional free functionality like monitors, mocking, real time collaboration and developer portal integration. Pricing is not changed, unless you have not paid for Postman before and need to adapt an Enterprise plan in order to use Postman’s SaaS offering in a company specific compliance manner. Still compliance requirements make it necessary for many companies to switch to an Enterprise plan.

Newman is OpenSource

Newman is an official open-source project associated with Postman. Its purpose is to enable the execution of any Postman collection test suite within a Continuous Integration / Continuous Delivery (CI/CD) pipeline. It can also be integrated natively into a JavaScript-based execution environment using an npm module. Moreover, Newman can successfully function with collections located within Postman’s SaaS platform, files on the local file system, and files within a version control system that follows the official collection format. In essence, collections can be run using this open-source tool. However, for convenient maintenance of collections and other artefacts, Postman is required.

What are code based open source tools for API system testing?

Code based open source tools are API testing tools that allow the user to write and execute test scripts using a programming language, such as Java, Python, JavaScript, etc. Some examples of code based open source tools are REST-Assured, Karate, supertest and PyTest.

What are GUI based tools like Postman or Insomnia for exploratory API Testing?

GUI based tools like Postman or Insomnia are API testing tools that provide a graphical user interface (GUI) for the user to create and run test requests using a point-and-click approach. These tools are popular among developers and testers, as they offer a user-friendly and intuitive interface. I also use them heavily for exploratory API testing. Some features of these tools include:

• Support for various authentication methods
• Ability to create and manage collections of requests
• Built-in support for API documentation
• Wide range of integrations and plugins

Postman versus Insomnia Licensing Model

Based on my research, Insomnia and Postman have different licensing models. Insomnia is an open-source, cross-platform API client that is free to use and modify. However, it also offers a paid plan that provides more features and support. Postman is a commercial API platform that offers a free plan for individual users and small teams, and paid plans for larger teams and enterprises. Besides, there are many other alternatives like Paw, plugins for VSCode or IntelliJ.

Both tools have their pros and cons, depending on your needs and preferences. Some factors to consider when choosing an API testing tool are:

• The features and functionalities that you require for your API development and testing
• The level of collaboration and integration that you need with other colleagues, tools and systems
• The budget and resources that you have available for the tool
• The user interface and experience that you prefer

Why are code based open source tools better for automated API system testing?

While GUI based tools like Postman or Insomnia have their advantages, they also have some limitations when it comes to automated API system testing. Automated API system testing is the process of running a set of test cases repeatedly and automatically to verify the functionality, performance, and security of the APIs. This process is essential for ensuring the quality and reliability of the software.

Code based open source tools are better for automated API system testing for the following reasons:

• Maintainability: Code based open source tools allow the user to write test scripts using a programming language, which gives them more control and flexibility over the test logic and flow. This makes it easier to maintain and update the test scripts as the APIs change or evolve. On the other hand, GUI based tools like Postman or Insomnia rely on a graphical interface, which can be cumbersome and prone to errors when dealing with complex or dynamic scenarios. Moreover, code based open source tools can leverage existing frameworks and libraries that provide reusable and modular code components, which can improve the readability and maintainability of the test scripts.
• Integration: Code based open source tools can be easily integrated with other tools and systems, such as continuous integration (CI), continuous delivery (CD), version control, reporting, etc. This can enhance the automation and collaboration capabilities of the testing process. GUI based tools like Postman or Insomnia have some extensibility features, but they are not as comprehensive or customizable as code based open source tools.
• Cost-effectiveness: Code based open source tools are free and open source, meaning that anyone can use them without paying any fees or licenses. This can save a lot of money and time for the users, as they do not have to worry about purchasing or renewing subscriptions or licenses. GUI based tools like Postman or Insomnia have free versions, but they also have paid plans that offer more features and functionalities. Sometimes these paid plans can be quite expensive or the license come with changes of terms and conditions that can be problematic for the users. Test code can be as important as productive code. You need to consider that for some products the code will need to be maintained for many decades. Sometimes the test code even outlives the productive code, since it contains the domain knowledge in the format of tests. This can remain relevant even if you rewrite the productive code. With 3rd party tools, there comes a risk and a dependency to the supplier including licensing issues.

Therefore, for test code you should prefer open source tools over 3rd party tools.

Further benefits mentioned by teams after migrating to code-based tools for automation

The following further disadvantages of a Postman setup were mentioned by teams after migrating to code-based tools for automation:

• High complexity of test sequences and scripts in Postman / Newman.
• Poor maintainability, unstable execution
• Good Integration of Postman requires higher tier: Using newman and postman CLI, you can run Postman API tests from any Git based pipeline, also on the free plan. The higher tiers of Postman also provide a built-in GitHub, GitLab, ADO and BitBucket integration, including support for Pull Requests.
• Code Review with pull requests is not as straight forward, since collections are harder to review. The higher tiers of Postman provide better support.
• The common approach to store collections externally in the cloud can be incompliant with company policies. On the other hand, sharing code instead in a repository is transparent and compliant.

Recommendations: Alternative for Postman and Insomnia for Automated System Testing

Recommendations: Alternative for Postman and Insomnia for Automated System Testing

For Automated System Testing against the API layer, you can use the tool, which is fitting best for your technology stack. Below some code based Open Source recommendations for the respective technology stacks – the list is not exclusive and there are further similar Open Source alternatives.

• Java
  • Karate
  • RestAssured
• JavaScript
  • Mocha or Jest NodeJS test frameworks
  • Supertest HTTP client library
• Python
  • Pytest Test framework
  • Requests HTTP client library
• ABAP
  • Either leveraging JavaScript based tools or writing system test with ABAP unit by leveraging OData Proxy

Migration from Postman to Karate

For Karate test tool there is also a migration support for Postman available. While Karate itself is purely code based and OpenSource, the functionality for migration is only included in their IDE support that according to the website of the company behind Karate is only included in their proprietary Pro plan, for USD 52 monthly.

Conclusion

In conclusion, code based open source tools are better for automated API system testing than GUI based tools like Postman or Insomnia, especially in terms of maintainability. Code based open source tools offer more control, maintainability, flexibility, extensibility, and cost-effectiveness than GUI based tools. Therefore, I recommend using code based open source tools for your automated API system testing needs. And tools like Postman and Insomnia only for exploratory API testing. If Postman is becoming too expensive due to the changes in September, you can check out Insomnia for the exploratory API system testing case.

Alternatives to Postman for Exploratory and Automated API Testing

Resources

• Testing Terminology: System Testing
• Visual Studio Code plus REST Client
• JetBrains PyCharm plus HTTP client
• Plugin for Postman to Karate migration

Related Blogs on Test Automation

You can find more information about test automation in the following blogs:

• Get Higher Quality and Productivity by tackling the Broken Window Effect
• The hidden cost of flaky tests
• Why a maintainable and readable test suite is important to stay agile