SNC: Social Network data Collection – Blue Skies analytics Opening your windows
For far too long we have been analyzing and understanding all that goes on within the walls of our enterprise and partners. As global corporations we have possibly become myopic. However as the world flattens, and becomes more horizontal, and as our customers are not just neighbors in our block, city or country we need to seriously consider opening our windows and getting some fresh air in. SCN,m or Social Network data Collection is possibly the fastest growing field in 360 degree BI solutions. Social networks are not only changing how we communicate and share our deepest thoughts but also changing business decison processes.
Business: There are $40 billion worth of text messages sent every year in the US alone, some of these are about your product or services. raw unadultrated emotions. There are millions of customers out there exchanging notes, comments and opinions about your company, your product or services. J&J just bought two social network business for SNC work. Southwest and other airlines provide special prices only on social networks like tweets (as an competitor you need to map these special offers that could be filling their empty seats and bleeding you to death).
PR: There are many people out there just starting to discuss a PR fiasco on the blogosphere and if caught in time you can prevent it hitting the fan.
Tourism: for tourist countries there are thousands of people communicating from your locations and the last thing you can afford in an epidemic. There are people tweeting their illnesses and if you see a spike of such tweets then you can react way before and possibly contain your epidemic from crashing your tourism dependent economy.
Homeland Security: In march 2009 Australia stopped 4 terrorists from blowing up an army location (wall street journal). They were caught red-handed and with all the explosives still in the car.The examples are endless you just have to know who to find it and how to apply it. I like to call this the 'Blue Sky' analytics. Its out there floating all around all you have to do is catch it.
What is a "Blue Sky company"?
Blue Sky is a company that is not only looking at the enterprise from within the walls of plants and processes, they have opened their windows and can see the sky outside. Imagine two companies both with world class, high business value analytics that meet business needs.
Company A knows all things about their stocks, processes, customers, materials, shipments, partners, distributor stocks and the rest there is to know. Company A knows all there is to know internally. Company B has all A has plus knows all that is happening on the outside. Company A firefights PR and customer isse=ues after the event. company B seem to routinely catch them before they hit the fan, they seek it, identify it and apologize and fix the issues as their highest pririty. So it is of no surprise that Company A works hard and Company B seems to work hardly at all. Company A grown with peaks and troughs, Company B has a stellar growth with only satisfied customers all across the planet. Who would you like to be. Company A seems to be always left catching the dirty end of the stick with PR issues that they come to know only when it hits the fan, the media and the stockholders.
The four methods of Social Network value attainment
Method 1 = Push method: A lot of companies have bought social network companies to push their opinions and goodwill to select member groups. For example one can buy a blog to push one's image. Over time I have a feeling that the audience will know when push is being done, just as I know this network has been bought by Company X and then it looses all its prime purpose.
Method 2 = Pull method: A few governments and companies are leveraging the pull method. In this method one scrubs the internet and tries to find 'key-words' of things that concerns your company. This has so far been a manual and complex task as when you come from an ERP world we suddenly find that we have been living in a micro environment. In the pull method the final aim is to collect relevant data, then load that into a BI environment and then contextually map it with existing reports and analytics specially designed for merging the internal air with the air from outside.
Method 3= Discover method: In this method manual operators go looking for specific conversations, the spooks departments across all nations look for terrorists and such. A corn products manufacturer could look for overall satisfactions of their feeder farmers and their schools like a mother hen. The corn belt evangelists look for what is worrying the corn growers and inform then on things they should know. The city police is scrubbing to see if any student is planning to do something that is not desirable ( June 2009 Finland stopped a second school shootout after their first ever in early 2009 – they reacted to the principle of ‘tipping point’ and started scrubbing the internet and it paid itself in gold and more).
Method 4= Short dump: In this method you open 5 or 200 twitter accounts and then you broadcast yourself to get followers. You provides information and/or ask questions and basically analyzes the data that you receive in your accounts. I have not seen anyone use this but this would ultimately be like viewing an hundred squarefoot piece of land on planet earth to decide what all species exist on this 3rd rock from the sun.
Reality check: Size matters when planning SNC
I don’t think I will ever forget my first meeting with my SNC group and the mental gymnastics I underwent. It was like a roller-coaster ride internally. The last time I had felt like that was in 1994 when I first learnt SAP. My first training session was like a roller-coaster ride in reverse as the trainer went from one screen to another and we are talking version 2.1. My first meeting with the SNC group was like two people for different planets meeting for the first time.
I was asked by my current colleague what the average size of the SAP DW was. I thought a little and decided to impress them. ‘around 25 terabytes”. "No" he immediately said sipping his cold coffee "I mean your large ones". I was taken aback and decided to thrown all reality out of the door and really size this statement “ around 50 terabytes” I was sure he would say wow!. His response was “ …I mean the big ones, the big fortune 100 companies on SAP’. I soon found out that companies like face book work with 3 Exabyte’s of data and it is growing and growing very fast. That is 3, 000 terabyte of data.
Think Google. If 512 people sign up for Gmail that is a terabyte gone. 1 million and this goes to 2 Exabyte’s. Google gets about 250,000 new users a month. When we speak Google we are possibly talking of an Exabyte a month if not faster. Take their whole services and we now get to understand the word Google. So the recommendation is to start one step at a time and then work from that point forward.
If volumes are so large then how the…. can we manage
Thought for the day: So as we discussed the ‘Sweet Spot’ for SNC is the Pull method buteven if you pull only tweets, which is a great place to start and blogosphere we are possibly talking of 5 to 7 Exabyte’s of data to be scrubbed. If you were thinking BW Accelerator your would possibly need to add a hundered blades a day to keep pace with that volume of data coming.
Value review: Do not go for short dump solutions where you only collect what people tell you in a few accounts. Go for the whole environment scan. Reality check: So what is the solution? Right now there are three solutions.
Solution 2- Manual: Followed by close to 90% of the network scrubbing articles and reports that one has read starting from the Finnish government, to the Australian Army and onwards closer home to airlines and corporations. This method uses manual scrubbing either in the form of sub-contracting or hiring resources to scrub the net and collect data for analysis.
Solution 3 – Automation: In this method companies provide automation tools that will scrub the whole desired ecosystem, provide data centers that will collect specific and focused the data in accordance to plans, then scrub the data for required reports and analytics. The big issue is the storage and currently IData3 is providing free data center offerings for collection of data as these volumes are extremely large even for the initial scrubbing. Our recommendation is to start with Twitter and the blogosphere and learn lessons there and then move onwards to more exotic areas of SNC. The secret is One-Step- at-a-Time.
Key success factors
1. Planning: Fail to plan = Plan to fail is true here as in all other BI tasks. Firstly define Business Value Attainment and exactly what one is looking for. The data volume is extremely large and one cannot afford to follow the traditional ERP ‘data modeling’ techniques of ‘ lets collect all the data and what we need will then be found’ . we need to work in the domain of efficiency and Information modeling.
2. Key Word definition: The second component is that once we have found the business value drivers we now need to define the ‘key word search criteria’s’ and their combination with other parameters. For example if you are Nike you want strong key words to be driven by Nike, then lets say you want to review your shoes, so the next level will be LeBron, LeBron Air max, Air max+, air H+, Jordan, Air Jordan 12, and so on and so forth. Then you want to catch the emotion so your need to define the positives excellent, wow, cool, vasool, paisa valoos, wonderful comfortable, unbelievable, etc. hen you want to capture the negative, hurts, lousy, useless, a waste, etc. You get th drift. Then the spiders go out an collect all conversations with a relative data elements and a time stamp for proactive analytics.
3. Type of collection: SNC collection can be done by governments and government spooks (keep off area), the army, i.e after the fort hood we hope the army there is undertaking serious SNC scrubbing in order to prevent the next happening, as based on the book by Gladwell ‘Tipping point’ the probability of such events escalates right after the event for a year or two (people on the fence suddenly drop to one of the sides – theory). Then there are police and schools that need to monitor activities that may not be good for the community from community security of many types. Then there are the corporate PR folks that need to catch a negative comment or happening before it hits the fan. As we move ahead there are the marketing folks that need to capture customer emotions on products and services, and shipping companies that may be able to catch chatter on the somalian coast and avert a piracy disaster. The possibilities are endless.
4.Data collection: We are talking very large databases. Type of collection most ERP vendors and customers are unused to. As select companies have collected such data for many years and have developed specialized algorithms they become the clear choice for discussions. Service providers need to throw in data centers and servrs for free for the first six to twelve months of trials and honing on to what works with what type of audiences.
5. Collaboration: there has to be a constant collaboration and networking during the development phases. Definitions and key words are unique to each type of search and new ones need to be added and old ones that do not work deleted. The work is constant and needs commitment, scrubbing cleansing and re-developments till one gets it just right. After that it just seems to hum along.
6. IDCM: Once all this is done there is the original goal and the final deliverable. Information Delivery Chain Management. Analytics are demand driven thus the mode is delivery. Things like how and who can access the data. How to package it or where to place it . Does the customer need it in an atomic form or summarized. Should it go the the BW or the Accelerator. All these are operational decisions that also need to be worked out
In a flat world we cannot afford to look vertically anymore
As the supply chain gets more and more stretched and the customers more scattered we need to open our windows, doors and attitudes of how we want to analyze our markets. Simply having a myopic view or our company and its internal partners can make us just that – myopic. We need to now get 20:20 vision of our products and services and cross all boarders of environments and view the whole enterprise framework from within and without. As a SAP customer you are in a unique position the 2008 BusinessWeek/Interbrand Best Global Brands ranking - engaged in different online social media channels ranked SAP in the top 10 most enguaged companies in Social Media developments.It’s not a simple world in the Social media network.
Just to site an example, and my apologies for this one. But on 9.11 the tracing started with first identifying the responsible persons by probabilities. All passengers phones and emails were scrubbed. Soon a pattern of emails and frequency was determined. For example if 10 offenders were isolated all all of them emailed to #6 and 7 and # 6 & 7 always communicate with #4. # 4 also routinely communicated with all 10. So we establish a pattern and identify #4 as a leader. That is how Mr Mohamed Attah was found to be the leader. Try doing that without SNC.
A penny for your thoughts?
We are all interested in what you think. Is the success of proactive companies to look only inwards or to open their doors? How do you plan to handle these very large containers of data that may exceed to total capacity of your current data centers?Business Case
A leading bank tried SNC and found a lot of stuff they already knew, but then found a diamond in the rough. they found that their broker, clients, were sharing complaints about the disappointment with the bank. Here is what was happening. The broker needed to buy stocks, the bank would ask them to fax their authorization and the fax would get mixed up. When the broker called after 30 minutes the bank staff stated they had not received the fax. For the bank it was just another fax but for the broker it was a possible identity threat with all back and account details missing too. The bank management changes their fax to secure fax machines andbroker traffic increased by 18% in the next 6 months.
today the Bank regularly tracks SNC BI feeds and their partner looks for new patterns of customer value.
Think SNC, Think 360, think what you can glean from customer emotions and merge to your internal information and you have 360 degree BI
