Using Data Services Text Data Processing with SAP Lumira to Analyze Tweets
Time for some self-introspection – what have I been tweeting about the last 3+ years? At first I suspected that the ASUG hashtag would be the one that I tweeted most. I was wrong.
I first downloaded an archive of my tweets. Then I used the Data Services to read the 140 characters of unstructured data – what hashtags have I tweeted the most about?
You can see on the left I am reading in my tweets, and using the Base Entity Transform from Data Services.
There are several options in the Text Data Processing transform – people, product, person, organization, etc. For now I am just selecting social media.
After running the Data Services batch job, it outputs it to a file which I take to SAP Lumira.
Above is the initial heat map. It’s hard to read so next I filter down to the highest number of tweets.
So you can see that ASUG is not my most tweeted hashtag, as I thought, but it’s SCN, then SAP, then ASUG.
When I look at the grid view, you can see case matters with the hashtags – note #SAPTechEd SAPTechED (wrong) and sapteched.
Finally, what’s text data processing without a tag cloud?
Related links:
Get Your Data Geek Badges Today! by Anita Yuen
Sentiment Analysis using SAP BusinessObjects Data Services by Louis de Gouveia
SAP TechEd Las Vegas | October 21–25, 2013 | ASUG Pre-Conference Seminars – BI4.1 hands-on includes SAP BusinessObjects Lumira
Text Data Processing Transform – Going from Unstructured to Structured
Nice post Tammy, interesting analysis 🙂
So much of different kind of analysis are possible in these new BO tools. I mean this is great for analyzers and for designers too. Thanks for this Tammy.
I like it! Can we get dump of other's tweets as well based on hashtag or security issue?
Thank you for reading and your comments.
I am not familiar with how to get other people's tweets
Twitter allows you to do download your own tweets.
Thanks for sharing your analysis, Tammy. I'm really happy to see that #SCN was your most frequent tweet! Love the tag cloud. 🙂
You'll need to post a picture in your DG t-shirt!
Jeanne - thank you!
I will post a picture as soon as I receive the DG shirt.
Regards,
Tammy
Text Data processing combined with rich visualization! Very cool. The challenge looks very interesting. Can't wait to post my own!
Thank you for commenting, Runali Ghosh
It was very easy to do this in Data Services, to go from unstructured to structured - I surprised myself at how easy and interesting this was.
Regards,
Tammy
Tammy, I loved this! It is fun but as Manna Das says illustrates the analysis potential of the tools.
Awesome work analysis.Is there a trail version available for SAP BODS ?.
I am not aware of a trial version for Data Services - I recommend asking this in the Data Services space - perhaps there is one.
okay thanks for your response. 🙂
nice article for a beginner on text data analysis 🙂
HI Tammy
This blog is very interesting!