Hey Everyone,

After much intrest in Predictive Custom R Scripts and Lumira Data Access Extension to acquire Google Spreadsheets, i tought my next project would be connecting to “Twitter”


It’s again a great information source and completes my strategy that i should demonstrate how it is different methods and technics to utilize cloud data. So let’s create our “Twitter Extractor”

1) First Steps – Create A Twitter APP                                                                     

Twitter requires you to create an “APP” so it will give you some keys and will recognize you when you ask for data programmatically.

login to twitter and visit https://apps.twitter.com/ and create your new app from there.

Afterwards, check your keys tab to note down you api key and secret :

ScreenHunter_695 Sep. 11 23.43.jpg     ScreenHunter_697 Sep. 11 23.51.jpg

2) Hardest or Easiest Part : Creating the “Extension”                                             


Language : Different from the visualization extensions, this time Lumira is looking for an independent executable file to speak to. This gives you unlimited freedom in options. I really had no idea how to create an “.exe” file, but i’ve been hearing Python from the communities. After hearing there are options to pack Python programs into exe files, i decided to give it a go. Ok it’s not so easy to figure out programming at first 🙂 But don’t give up. Theres nothing you can’t do after few sleepless nights and lots of caffeine.

ScreenHunter_700 Sep. 12 00.08.jpgScreenHunter_699 Sep. 12 00.07.jpg       ScreenHunter_663 Sep. 10 14.53.jpg

Extension guide : As always, help.sap.com is our home. I found “SAP Lumira Data Access Extensions Developer Guide”. You’ll figure it out very easily. You just need to implement Preview, Edit and Refresh modes, and handle parameters and send the data in a specific format.

      ScreenHunter_662 Sep. 10 14.48.jpg

ScreenHunter_701 Sep. 12 00.12.jpg

3) Most Of It’s Already Done By Someone Else                                                      


What seemed impossible at the first place, really looked doable after a few days. Python is famous with lots of packages for almost anything you want to do. I used tweepy for twitter connections, dpapi for encryption, easygui for dialogs etc. Tkinter seemed uber-complex for a first timer. Special thanks to everyone who are kind enough to contribute to open source software and make those packages available for everyone.

ScreenHunter_703 Sep. 12 00.20.jpg

ScreenHunter_702 Sep. 12 00.19.jpg

You can see the complete code and download the executable in my github repository

4) Use Your New Extension And Get Insights                                                        


Now lets see there’s any insight to be found in Twitter… Hint : Of course there is!

First step would be to use our extension as “external data source” to download twitter data. After that we can even “refresh” anytime we want.

ScreenHunter_658 Sep. 10 14.23.jpgScreenHunter_704 Sep. 12 00.30.jpg

And now… We’re ready to get insights :

ScreenHunter_705 Sep. 12 00.32.jpg

Tweets sent from an iphone are getting more retweets compared to others.

5) Final Comments, Code, Executable And Call To Action                                     

An extensibility is really worth a thousand features. Because the big SAP Community can contribute and adapt our software to virtually any need. I also want to contribute and make my codes and executables available here on github. Did i tell you that this is an “extension” and the extension code is not supported by SAP. It is not supported by me either, use at your own risk and feel free to enhance it even further.

Do you know a specific data source that Lumira currently does not support and your customer needs? Please share your toughts and comments below here and let’s discuss.

To report this post you need to login first.

24 Comments

You must be Logged on to comment or reply to a post.

    1. Alper Derici Post author

      Hi Daniel,

      There are a few things you need to do to active Data Source Extensions in Lumira :

      1) In C:\Program Files\SAP Lumira\Desktop, open SAPLumira.ini file.

      2) Add the following entries to the SAPLumira.ini file, and save the file.
             -Dhilo.externalds.folder=C:\Program Files\SAP Lumira\Desktop\daextensions
             -Dactivate.externaldatasource.ds=true
      3) Create the folder “C:\Program Files\SAP Lumira\Desktop\daextensions”

      We are ready to go!

      (0) 
      1. Daniel Davis

        Cool only I get an error…

        I edit the INI

        I create the folder

        I download the EXE

        I move the EXE

        I start Lumira (1.19)

        I see the new data type

        I select the datasource and…

        /wp-content/uploads/2014/10/e1_560378.jpg

        on NEXT I get..

        /wp-content/uploads/2014/10/e2_560379.jpg

        Any ideas?

        (0) 
        1. M. van Foeken

          Hi Daniel,

          Try to restart Lumira or reboot your system. I have seen this before and think it’s related to the ‘in-build’ Sybase instance in Lumira. Haven’t found another workaround yet. I have seen this also not related to data extensions but using regular data sources.

          With kind regards,

          Martijn van Foeken | Intenzz

          (0) 
  1. Daniel Davis

    Martijn,

    Actually after a reboot, a reinstall, a reboot and another re-install is started working… sometimes… not all the time but more than not 🙂

    Thanks and regards

    Daniel

    (0) 
  2. Paul McDonnell

    I’ve just set this up and its a great addition to Lumira analysis, thankyou for developing and sharing.

    Is there any connectors or methods to connect to SuccessFactors Cloud data from Lumira/Unx that you know of today?

    (0) 
    1. Daniel Davis

      Twarit,

      I got the same error a bunch of times… in the end I uninstalled, rebooted and went straight to the latest version of Lumira. a quick re-install later, edit the .ini file and away is went.

      Regards

      Daniel

      (0) 
        1. Daniel Davis

          Twarit,

          I wish I could offer more – I know that a number of people have had the same issues – I certainly did but (perhaps with luck) I managed to sort it. I would still remove, reboot, re-install Lumira, reboot, edit the ini file and try once again… also I would ask the spec of the PC / Laptop? I have a T530 with 16GB ram – I don’t know if this helps with it working for me? How much memory? what else was running ? were you on the VPN (don’t be).

          Hope something helps… and remember, it’s only a demo someone built and therefore i’m assuming not really supported etc.

          Hope this helps.

          Daniel

          (0) 
  3. Daniel Davis

    Hi,

    Having recently upgraded to 1.25 the extension does not want to work for me anymore. Might there be a new version for the latest Lumira ??

    Thanks and Regards

    Daniel

    (0) 
    1. Jan-Gerrit Groeneveld

      Hi Daniel, I did install the extension again today, and in principle it works.  Even on the latest Latest release 😉

      Only problem is that it does not find many tweets – searching, e.g., for Lumira, only gives me a few hundred from the past couple of weeks.

      (0) 
      1. Alper Derici Post author

        I am not limiting search results other than the limit user enters on the selection screen.

        But could it be something about the API? If you think a correction is needed could you provide an example hashtag and provide lumira result number vs twitter web access results if you can count?

        Best regards,

        (0) 
        1. Alexey Zimin

          Yes, it looks like API is limiting number of records. I try to set limit on input screen between 10000 and 500000 and in all cases received no more then 500 records ((

          Regards,Alex

          (0) 
  4. Alper Derici Post author

    Hi,

    for the “preview” pane tweets are limited to 5. but when you decide to acquire you’ll see all tweets inside lumira,

    Best regards,

    (0) 
  5. Charles Valentine

    This extension is really good !

    We are just required to put in those 2 lines in the SAPLumira.ini file again after an upgrade to a newer version, may be because the INI file is overwritten during upgrade.

    I was required to do this when I upgraded from 1.25 to 1.27.

    (0) 
  6. Mohanapriya Muthulingam

    Hi,

    I am using SAP Lumira 1.25 32 bit version and followed this blog downloading the TwitterExtractor.exe and adding the twitter information. I am able to view the TwitterExtractor as external datasource and add the API key & Secret Key. After this, clicking “ok”, i get the below error. Could anyone please help me on what went wrong?

    Thanks.

    LumiraDAE_Error.png

    (0) 

Leave a Reply