Stay up-to-date on the latest developments and product news about intelligent customer experience and CRM technologies through blog posts from SAP experts.
While we just released the Tech Beacon which reaches way into the future and defines 5 trends for the next 5 years, a new trending topic called No Code/Low Code (short: NCLC) is keeping my coworker Valentin Vieriu and me busy. In this blog post I’ll first outline what NCLC is all about. Next, I’ll then try to bring this concept down to a more technical level – in my opinion NCLC is a lot about flow-based programming. For this style of programming, we’ve looked at a few open-source options out there and we’ve chosen to research one specific one in combination with Kyma, a platform for extending applications with serverless functions and microservices. For this – early – integration, I’ll be sharing the relevant Kubernetes/Kyma configurations which will allow you to install this in you own Kyma cluster.
Please don’t be shy and provide feedback! What you find in this blog post is work in progress and we’re happy to get feedback and discuss with you. You can easily contact me (https://people.sap.com/svenhaiges) or via the comment field below.
Also, before we start, one tiny reminder that this topic is not quite new for the SAP Customer Experience Labs team. A search for the popular flow-based programming tool Node-RED reveals that our first experiments date back to 2015. Our main focus at that time was on IoT and we’ve used Node-RED to reconfigure our Moto showcase.
No Code / Low Code 101
Let’s first deconstruct and define NCLC. First of all, it’s two things that for whatever reason are currently often discussed altogether: there is “No code” and there is “Low code”. It’s not one thing and while the lines might be blurry between the two, there are important differences. And knowing what you’re really speaking about helps! So let’s start with the most extreme part of NCLC:
No Code
What we really mean when we say No Code is a “No Code Development Platform”. Such a platform enables programmers and non-programmers alike to create applications via a graphical user interface. It would strongly suggest that a No Code Development is targeting the non-programmers first while of course also programmers will be able to use such a platform.
What is No Code Development?
I think where things get really confusing, is when it comes to the definition of an “application”. According to Wikipedia and a few other places, such an application has an UI and it’s typically a mobile web-driven one. The target group for such a mobile web app is often a mobile workforce. Using a No Code platform, a business-user without programming skills can now create web apps for it’s mobile workforce. The UI is made up of preset components which explains while customer end users will typically not be the target audience for such an app – it’s often not that beautiful and cannot be styled without a programmer or designer.
Low Code
In comparison, Low Code Development Platforms are typically used by developers. Low Code programming reduces the amount of “hand coding” and also opens up the benefits of programming to a “wider range of people” – to me it means that also tech-savvy business users can make use of these platforms. For simple apps, a (technical) business user might create an application all alone. Some more complex applications might require a developer to write part of the code or might require designers to step in to optimise the UI.
What is Low Code Development?
At least Wikipedia also mentions mobile UIs when it comes to Low Code development platforms. I am somewhat questioning the importance of a UI for these Low Code applications, as I can also see a lot of use cases for non-UI style applications. For example, kicking off a workflow based on an incoming order for a commerce application and finally sending out an email with a special voucher code, is – for me – a totally viable application without any UI involved (I already see comments telling me that an E-Mail has a certain UI…).
Before we move on to Flow-Based Programming, I want to note that both No Code and Low Code development is linked to the discussion of fourth-generation programming languages and tools for rapid application development. These are not new topics at all – but maybe the time has finally come for these concepts to gain broader adoption.
Flow-Based Programming
Not necessarily No Code Development, but at least Low-Code Development is strongly linked to Flow-Based Programming. In essence, a flow-based program is typically authored in a visual flow editor and a flow consists of reusable components that can be connected to achieve the desired functionality. Data can be passed from one component to the next with the help of message passing. The connections between the components are defined outside of the components and in the configuration of the flow, which is why the components can be combined in many ways. Once a flow has been authored in the editor, the flow runtime is then responsible for executing it whenever it needs to run.
What is Flow-Based Programming?
And I hate to say it, but flow-based programming is a pretty old concept. It dates back to the 1970’s and was invented by J. Paul Morrison. But of course the tools and possibilities have changed dramatically.
Have you noticed that we’ve lost some discussion about UIs? While the editors of flow-based programming tools are of course highly visual and allow the developers to connect and configure their components, the applications created by these flows do not need to have a visual components. I would even argue that flow-based programming is – currently – not very often related to the creation of “visible” applications. This is just an observation – and definitely an interesting area to research.
Let’s take a look at two open source flow-based programming tools.
Node-RED
A very well-known low-code, flow-based programming tools is the node.js-based Node-RED. I’ve quickly started the Node-RED docker image and stitched together a small flow to create the screenshot below. The Node-RED Flow Editor
To create a flow, a developer pulls out some of the “nodes” on the left and then creates connections between them. Each flow has to start with some sort of trigger, which can be a HTTP Request or some incoming MQTT message. In my case, the trigger is an interval which fires every second. The message passed between the trigger and the first nodes is in this case a JSON object having the timestamp as the payload. While sending this timestamp out to the Internet with a HTTP Request code, I’ve also attached a small custom function to the interval node. This function uses a few lines of JavaScript code to determine if the timestamp is an even number or not – depending on the outcome, the first or second output node is executed next.
Node-RED has its origins in the visualisation and manipulation of MQTT streams and was therefore originally focused on the IoT application space. But it has grown out of this space and while many of the default components are still related to IoT (such as the MQTT nodes) in can as well be used for more generic event-driven applications. From an extensibility point of few, it is quite easy to write new nodes which encapsulate your custom logic. This way, it is also possible to create higher level or more abstract nodes which could be more easily be used by non-developers.
Just recently, Node-RED 1.0 was released (September 30, 2019) which marks an important milestone in the development of this tool. By now, Node-RED has enable pluggable Storage (e.g. how and where flows are stored), pluggable Context and there is also a clean separation between the UI editor and the Admin API that the editor interfaces with.
Flogo
While it would be enough for a short introduction to flow-based programming to just highlight the most known tool – Node-RED – we’ve also quickly evaluated Flogo, which in my opinion provides a really interesting and fresh perspective in some areas of flow-based programming.
Just like Node-RED, Flogo is event-driven. A chain of activities can be linked together in a visual editor and the resulting flow needs to have a trigger to be kicked off. Just like in Node-RED, the result of the editor is also a JSON file which describes the flows, the activities and how these are connected. The Flogo Web UI – see flogo.io for more
But as Flogo is written in Golang, the flogo-cli now takes this JSON representation of a flow and compiles it into a binary. The size depends of course, but as just those parts that are required by the flow need to be compiled, we’re talking of binaries as small as 10MB. Compare this to Node-RED, which requires a runtime container with all node-dependencies installed and can easily be a few hundred MB. If you extend a small Alpine Linux base image and add such a Flogo binary on top, you get a very small, efficient and scalable flow runtime. I’d say this is cool.
Node-RED on Kyma
I would like to finish this blog post with giving you a glimpse of what our research on flow-based programming on Kyma and NCLC on Kyma has yielded so far. I just cannot stress this enough, it’s work in progress and we might change our direction or even abandon this at any time. For now we’ve been concentrating our efforts on Kyma and Node-RED. As previously explained, Node-RED is a node.js based and pretty mature event-based flow-based programming environment. The fact that it is using node.js makes it somewhat more accessible to us, which is one of the reasons we initially went with Node-RED. And then there is Kyma – a platform for extending applications with serverless functions and microservices. It has the concept of Application Connectors that connect to other enterprise systems and these Application Connectors can emit events, which makes it a nice fit for an event-based and flow-based runtime such as Node-RED. On top of that, Kyma is based on Kubernetes and we can easily expose a flow-based UI if we wanted to with the help of Istio’s VirtualServices / APIs (these resources essentially configure Istio in a way that services running in the cluster can be exposed to the outside world).
This is what I would like to share with you:
The Kubernetes Resources such as Deployments, Pods and Services to run a single-instance Editor/Runtime of Node-RED
How we extended the Kyma Console UI to include our Node-RED based editor within the Kyma Console
How to create Subscriptions that result in event-delivery to our Node-RED instance to kick off flows.
Some thoughts on extending Node-RED – for example for picking up the current list of available events and ways to store the flow data and other Node-RED configuration in Kubernetes directly, e.g. via ConfigMaps (and not in the file system in the container).
Ready? Let’s go.
Running Node-RED inside of Kyma
Due to the fact that Node-RED provides ready-made Docker images, this task boils down to creating the K8S artifacts such as a Deployment, a Pod Spec, the service to expose it internally and an Istio VirtualService or API resource to expose the Node-RED Editor UI externally. It shall be noted that as this point, the web resources are exposed without any security. We cover one way of making sure that only authorized users which are logged in to the Kyma Console have access in the next step.
As you can see, our Deployment uses a custom Node-Red image. This is because we integrated this with the UI Framework Luigi which is used by Kyma Console UIs. If you just want to bring up Node-RED in your own Kyma Cluster, just replace the image with the default nodered/node-red image from Docker Hub and you should be able to directly access the exposed web editor UI. Please make also sure to adapt the namespace and external host name of your cluster – in our case both the namespace and cluster is called flumen, which is our internal project name.
Extending the Kyma Console UI
To integrate any UI into the navigation structure of Kyma, a so-called ClusterMicroFrontend Resource has to be created in the Kubernetes cluster. Below you’ll find this resource in YAML format:
At this point the Kyma Console will now include the Visual Editor ClusterMicroFrontend which points to the web UI of our Node-RED instance running in the Kyma cluster.
And this is how it looks with the UI of the Kyma Console around it:
To secure this UI, we’ve decided to integrate a bit deeper with the Luigi Framework. This explains also why we needed to create a custom Docker image. Below you can see the the contents of low-code.js which my co-worker Valentin added to the custom Node-RED project to pick up a context object which is passed to our UI only if we’re running as part of a logged in Kyma Console UI session. This blocks the initialisation and usage of this web UI if accessed standalone.
//this needs to be placed in //packages/node_modules/@node-red/editor-client/src/js/low-code.js //and needs to be made part of the Node-RED build process
Subscribing to Kyma Events and Triggering a Node-RED Flow
The natural way to trigger our Node-RED flows is via the event system of Kyma. Kyma events are passed on to subscribing containers via HTTP Post requests to endpoints that they expose. To tell Kyma that our Node-RED Deployment is interested in receiving some events, we need to create a subscription such as this one:
This will activate event delivery immediately. To test this, you have to create:
A Node-RED flow in your container that listens to an incoming HTTP Request at /trigger
Emit an order.created.v1 Event via a “real” Application Connector such as the SAP Cloud Commerce Connector or via a mock connector.
Creating a simple Node-RED HTTP Request Trigger
All that is needed is an HTTP IN Node that is configured to react to POST requests to /trigger. In the following screenshot I wired this up to a simple HTTP Response node which by default will reply with a HTTP 200 OK status. This is very important for the event system of Kyma, as no response to the event would otherwise not remove the event from the queue. On top, I’ve connected this trigger to a simple debug node to see output in the debug log of the Node-RED editor UI.
Installing the Kyma Mocks
Installing the Kyma Application Connector Mocks for Commerce, Marketing and C4C (Cloud for Customer) can be done in 10 minutes, but it’s a bit too much for this blog post. I’ve therefore quickly recorded a screencast which shows the steps. To get started visit this page for the documentation and if you need help watch the screencast below.
Extending Node-RED
This last part will cover two outcomes of our research. First, we found a really nice way for Kyma Console UIs to query the Kyma system for available events. Second, we’re exploring ways to store data outside of the Node-RED containers. This way one container could be the pure editor and others could offer the runtimes – if possible in an elastic scalability way.
Getting a list of all available events
As it turns out, Kyma has a built-in GraphQL server with fixed / compiled functionality that offers its services around querying and changing the Kubernetes system it’s running on. We’ve also rcently explored how integrate Kyma and GraphQL, so click here if this is interesting! It’s used by other Console UI applications, too, and we’ve started exploring it for getting a list of events. As a reminder: this list will vary based on how many Application Connectors are plugged into your Kyma. If you have Kyma running, the console-backend pod/service/VirtualService will give you access to a GraphQL endpoint as well as the GraphiQL UI. To figure out your specific URL to this backend, you can run this kubectl command:
kubectl get virtualservice -n kyma-system | grep console-backend
Part of the output will be the URL that the UI is accessible under. Now you’re just one more hack away from accessing this UI: you need to send a Authorization Header to pass the security. For this, you’ll need to extract the idToken which is part of the Luigi context that is passed to any Kyma Console UI – so open developer tools of Chrome and look into the network traffic. Then use a Chrome extension such as Modify Headers to add this request header to your requests: No worries, this IdToken has long expired…
You should now see the GraphiQL frontend:
For your reference, here’s the GraphQL query to get all events across all installed Kyma Application Connectors:
Ways to store the Node-RED data via the Storage API
We’re currently also exploring ways to store the Node-RED flow configuration, context and other data such as the configuration of the editor itself outside of the container image. This is of course a requirement to seperate the Node-RED editor from the runtime. The idea is to use the Node-RED Storage API but it’s a bit early to show code here. Once we have some progress I am sure this would be an interesting contribution to Node-RED.
Over and Out
This is a long post and looking back, I should have probably split it up into multiple blog posts. That’s what happens when you are super focused sitting in your home office environment and begin to write!
I hope you find post relevant when it comes to exploring No Code/Low Code, Flow-Based Programming or even some specific tool such as Node-RED or Flogo on Kubernetes and Kyma.
