Anything of slightest importance that happens in this event filled world of ours, people generate huge data by posting their status updates on minor events like meeting a friend over lunch, or taking a personal trip around their town.
One can only imagine – what happens during bigger events.
Yes. Millions of people generating terabytes of data that still does not fill up all the social platform tools like Facebook, Twitter, Google+ and many other social chat platforms like WhatsApp, Instagram, SnapChat etc.,
Data gets generated in such quick pace every millisecond, – of course, there are applications built and available to analyze or predict where we are talking ourselves into. The time from data getting generated and someone or some application analyzing such generated data has reduced to what we now know as Real Time Analytics. The process of data generation, data collection for processing, processing before analysis and the final analysis (or at least the availability of data for analytics) are separated only by microseconds. This used to be more than few hours few years back.
This is relatively an aged process (in technical terms) that is already happening in many of the companies where data research, generation and computations are the main saleable product (like maintaining your social status / twitter or companies that track your online shopping habits) and with a big customer base of the indirectly revenue generating public who do so using online data or few of the many social platforms that are available to them.
In the other world of companies that hold a relatively limited customer base – they are not lagging and fast adapting to this. For example, when a customer enters into the store – they want to know from their mobile app (assuming they have it installed in their mobile device and working in background) how it is possible to make the customer spend bit more than what he or she had originally planned. This is assuming that they did not bring the potential customer (literally) by holding their hand (using the mobile QR code on the advertisements using applications like Hybris) from the time the customer had seen an advertisement (may be) on a bus stop poster, directing the customer to a nearby store, just to make them spend that extra money. This is one of the few ways to analyze where the revenue can be generated for the company in real-time.
For this need of continuously expanding data few enterprise resource planning applications businesses are providing in-memory computing solutions. They have developed solutions to meet the demand of faster analytics using in-memory computing. In applications such as, SAP HANA, columnar based in-memory computing, there are tools for real time and predictive analytics (as well as operational data management thus removing the barrier between Transaction and Analytics systems).
I wanted to put this to test on a real time data feed during one such major political event. With the US looking to hire their new President, there are few debates happening and one such debate was held by news channel CNN for the Democratic contenders on October 13th, 2015. I was interested in analyzing the data in a simple way how the people are responding in any particular social platform using SAP HANA. I decided to use the Streaming API to feed the data to my SAP HANA instance on the cloud, using the openly available JavaScript node.js (Light weight event-driven JavaScript based on Chrome JavaScript engine – more info on nodejs.org). After feeding the data (based on specific hashtag and limited by some interruptions) into SAP HANA, I text-indexed the data using a specific configuration for Voice of Customer to analyze the sentiments of the tweeter.
Here are some of my analysis based on the data feed. Take it with a grain of “tweet” that this is very crude data analytics only with data from the democratic contenders. This information is leading me to think – May be the Whitehouse will have another democrat as tenant starting 2016 also.
The overall positive sentiment is 29591 vs. 12192.
Who will that be? The below results might indicate better.
Reference - SAP HANA Academy.
https://www.youtube.com/watch?v=la9V2hAeuK8&list=PLkzo92owKnVw1SONF9PQy2S0IgX8ICL9G
- SAP Managed Tags:
