Real time twitter Tracking with Design Studio

What does the application do?

Recently I had some discussions where we thought about tracking twitter messages. Retrieving Twitter messages and being able to put them into an analytic framework. There are scenario’s where this kind of live tracking can be an added value. For example in communications where you want to have a feel for the opinions about the organization at a glance. Or at a helpdesk. For example an internet provider could track how people are tweeting about the service level or how they feel they have been treated during an outage.

image: live tracking of twitter messages

I made a bare-bone example with a twitter component and a real time barchart.

The barchart tracks the results of the number of tweets during a period of time. The component has a property “numberoftweets” that you can use and pass to the bar chart. Each time a value is passed to the bar chart a new bar will appear and the older bars will move to the left and fade to white.

images : Progression of the values

In the future 1.3 release you could also pass the values to a planning model and store these results.

How does it work?

The Tweet component

First of all I searched the internet. I looked for a javascript solution that didn’t have to do all the authorization and needed server side scripting. I found something at http://jasonmayes.com/projects/twitterApi/#sthash.V0yIsiJI.dpbs. I used it and pressed the donate button for his caffeine / beer beverage to code even more.

This is a version where you use your personal twitter account (or organization) to set up widgets. The javascript code uses these widgets to collect the data.

To be able to pass the number of tweets to another component I added the property number of tweets. In the afterupdate event I have 2 lines to update the value :

numberTweets = tweetMessages.length;

this.firePropertiesChanged(["tweetnumber"]);

for the example I just counted the tweets although it is fixed in a parameter. You can also do a substring and search for “seconds ago” and “minutes ago” in the lines. This would give you the number of tweets in the last minute or hour.

to build the html table I relied heavily on the simple table example. As the retrieved tweets are already html it is possible to just append them to the component :

var html = '<ul>';
           while(n < x) {
             html += '<li class=\"" + "tweetline" + >' + tweetMessages[n] + '</li>';
             n++;
           }
           html += '</ul>';
           this.jqTable.append(html);

the javascript itself was documented on the site and looks like this :

twitterFetcher.fetch('449308105306693633', 'example4', 7, true, true, true, '', false, handleTweets, false);

note that the long number is the widget number and the “handletweets” is a function that translates this to a variable that we work with.

The Bar Chart

I will not go too much into the barchart itself but instead focus only on the part that makes it a realtime chart. The intention is that at the start no bars are visible, but that values are added to the chart and represented in the bars.

In the init stage I set up the dataset for the chart. This is an array. The length depends on a property settings (how much history you want to see).

In the getter/setter function for the history length the array is reset when the property is changed :

this.history = function(value) {
             if (value === undefined) {
                    return historyBars;
             } else {
                    historyBars = value;
                    dataset.length = 0;
                    for (var i=0,historyBars; i<historyBars; i++) {
                           dataset[i] = {};
                           dataset[i].value = 0;
                           dataset[i].index = i;
                    };
                   
                    return this;
             }

Each time a new number is added the number will be set on the right hand side and the value to the left will be removed. This is a getter/setter that is attached to the method

this.measurenumber = function(value) {
             if (value === undefined) {
                    return measureNumber;
             } else {
                    measureNumber = value;
                    for (var i = 0; i < dataset.length; i++) {
                             (function  {
                                    dataset[i].index = i - 1;
                                    })(i);
                           }
                    dataset.shift();
                   
                    dataset[dataset.length] = {"index": dataset.length, "value": newNumber};
                   
                    return this;
             }

The method is in the ztl file and looks like this :

void setNewNumber( int NewNumber)
{* this.newnumber = NewNumber;*}

Note that the source of the number also can be a datasource or any other component that provides an integer number. For a timer you can use the SDK component that SAP already provided as a standard example.

Finally for the fading color I use a a d3 scale function. With D3 you can set a domain and a range. The domain is the high and low value of your data. The range is the high and low value of the way you want to represent it. In a bar chart it translates as a height, but you can do any translation. In this instance we use the index value. The higher the value the closer the color gets to midnight blue. The lower, it gets closer to white :

                     var color = d3.scale.linear()
                    .domain([0, 25])
                    .range(["#ffffff", "#191970"]);

Summary

Using these two techniques I was able to build the realtime scenario. Adding things will make it more useful for all kinds of scenario’s. planning functions in 1.3 will make storage available. For the future and for bigger scenario’s you probably want to implement the more official version with the twitter 1.1 api. There is  an application authorization available that is enough for any search query. But with looking for javascript on the internet you can go a long way.