Skip to Content
Author's profile photo Tammy Powlas

Using Data Services Text Data Processing with SAP Lumira to Analyze Tweets

Time for some self-introspection – what have I been tweeting about the last 3+ years?  At first I suspected that the ASUG hashtag would be the one that I tweeted most.  I was wrong.

I first downloaded an archive of my tweets.  Then I used the Data Services to read the 140 characters of unstructured data – what hashtags have I tweeted the most about?

/wp-content/uploads/2013/08/1textdataprocessing_256725.jpg

You can see on the left I am reading in my tweets, and using the Base Entity Transform from Data Services.

/wp-content/uploads/2013/08/2filterbysocialmedia_256726.jpg

There are several options in the Text Data Processing transform – people, product, person, organization, etc.  For now I am just selecting social media.

After running the Data Services batch job, it outputs it to a file which I take to SAP Lumira.

/wp-content/uploads/2013/08/3heatmap_256727.jpg

Above is the initial heat map.  It’s hard to read so next I filter down to the highest number of tweets.

/wp-content/uploads/2013/08/4filteringheatmap_256728.jpg

So you can see that ASUG is not my most tweeted hashtag, as I thought, but it’s SCN, then SAP, then ASUG.

/wp-content/uploads/2013/08/5casematters_256729.jpg

When I look at the grid view, you can see case matters with the hashtags – note #SAPTechEd SAPTechED (wrong) and sapteched.

/wp-content/uploads/2013/08/6tagcloud_256733.jpg

Finally, what’s text data processing without a tag cloud?

Related links:

Get Your Data Geek Badges Today! by Anita Yuen

Sentiment Analysis using SAP BusinessObjects Data Services by Louis de Gouveia

SAP TechEd Las Vegas | October 21–25, 2013 | ASUG Pre-Conference Seminars – BI4.1 hands-on includes SAP BusinessObjects Lumira

Text Data Processing Transform – Going from Unstructured to Structured


Assigned Tags

      14 Comments
      You must be Logged on to comment or reply to a post.
      Author's profile photo Louis de Gouveia
      Louis de Gouveia

      Nice post Tammy, interesting analysis   🙂

      Author's profile photo Former Member
      Former Member

      So much of different kind of analysis are possible in these new BO tools. I mean this is great for analyzers and for designers too. Thanks for this Tammy.

      Author's profile photo Former Member
      Former Member

      I like it! Can we get dump of other's tweets as well based on hashtag or security issue?

      Author's profile photo Tammy Powlas
      Tammy Powlas
      Blog Post Author

      Thank you for reading and your comments.

      I am not familiar with how to get other people's tweets

      Twitter allows you to do download your own tweets.

      Author's profile photo Jeanne Carboni
      Jeanne Carboni

      Thanks for sharing your analysis, Tammy. I'm really happy to see that #SCN was your most frequent tweet! Love the tag cloud. 🙂

      You'll need to post a picture in your DG t-shirt!

      Author's profile photo Tammy Powlas
      Tammy Powlas
      Blog Post Author

      Jeanne - thank you!

      I will post a picture as soon as I receive the DG shirt.

      Regards,

      Tammy

      Author's profile photo Former Member
      Former Member

      Text Data processing combined with rich visualization! Very cool. The challenge looks very interesting. Can't wait to post my own!

      Author's profile photo Tammy Powlas
      Tammy Powlas
      Blog Post Author

      Thank you for commenting, Runali Ghosh

      It was very easy to do this in Data Services, to go from unstructured to structured - I surprised myself at how easy and interesting this was.

      Regards,

      Tammy

      Author's profile photo Former Member
      Former Member

      Tammy, I loved this!  It is fun but as Manna Das says illustrates the analysis potential of the tools.

      Author's profile photo Former Member
      Former Member

      Awesome work analysis.Is there a trail version available for SAP BODS ?.

      Author's profile photo Tammy Powlas
      Tammy Powlas
      Blog Post Author

      I am not aware of a trial version for Data Services - I recommend asking this in the Data Services space - perhaps there is one.

      Author's profile photo Former Member
      Former Member

      okay thanks for your response. 🙂

      Author's profile photo Swetha N
      Swetha N

      nice article for a beginner on text data analysis 🙂

      Author's profile photo Lizhu Zhao
      Lizhu Zhao

      HI Tammy

      This blog is very interesting!