Background
In our previous blogs we have been able to successfully create a custom connector and able to use hooks to format the response from Trello dashboard. I felt something was still missing. Each time i have to run my resources to fetch the data. What if we want to get real or near real time data from Trello. I could see two option one to pull the data from Trello or second is pushing the data from Trello to our Open Connector. Pulling data from Trello at regular interval is known as polling and when Trello pushes data to our connector that is done via webhooks. So lets discuss about how we got near real time data using Polling method.
Demystifying Polling
At high level Polling is like a scheduled job in background which fetches data at regular interval for the given URL. It is like cron which we use in our web developments for scheduling certain scripts, read here for more details regarding Polling.
The first and the basic requirement to use polling is that the response shall have created or update date and time stamp which in our case we do have. We have added date and time also in the formatted response which was missing in last blog.
Next step was to open our custom connector->Go to Setup tab->Switch on Events-> Choose Polling.
Next we need to specify the conditions on what criteria the data needs to be pulled and at what interval. Since our Trello API provider does not provide query filter via where clause we will leave created and updated field as blank. In case it was supported all you need to do was to map the field name here and in where clause of the URL. Since we do not have any filter option we will do it via custom javascript code in next few steps.
We also have a javascript console at polling level to enrich our received data. This is similar to what we have seen in about hooks previous blog. We have event object available here as can be seen below where we have added event type and object type detail and called the done method.
done({
"events": [{
"event_type" : "UPDATED",
"event_object_type" : "dashboard",
}]
});Now since each time we are getting all the boards without any filter. We will do it by enhancing our Hook code where we will check if anything is changed in last 5 minutes we need information of that Trello board only. So the response now is not only being formatted but also being filtered.
var result = response_body.map(resp => {
var date = new Date();
var currentISODate = new Date(date.toISOString());
var recordISODate = new Date(resp.dateLastActivity);
var diffInMinutes = ( currentISODate - recordISODate ) / 60000;
var processedresult = {};
if(diffInMinutes<5){
processedresult.name = resp.name;
processedresult.url = resp.url;
processedresult.dateLastActivity = resp.dateLastActivity;
processedresult.minute = diffInMinutes;
processedresult.currentdate = currentISODate;
return processedresult;
}
});
done({
'response_body': result
});Final Output
Basic testing can be done by first creating an instance of custom connector then manually executing our endpoint as can be seen below. I have changed a repository few minutes back so that is only returned.
But since we have already implemented it via polling lets create a new instance with polling option active. This will be triggered every 5 minutes. I will be making a change to a repository it shall fetch the record in first run and after it should be blank in further runs. The execution logs of all API’s calls etc. can be found in activities tab( thanks to Polling discovered it,will discuss in detail some other times) as can be see below.
What is next?
Since now we have covered polling next step is to explore webhooks how they will fit in our storyline of custom connector. The main use of fetching this automatically will be discussed in detail in coming part of the series. Feel free to provide your feedback, open to all ears.
- SAP Managed Tags:
