Skip to Content
Author's profile photo Uwe Fetzer

A story about Twitter, XML and WD4A

Prolog

Like many of us, I’m a kind of addicted to Twitter. But a few weeks ago, the admins of my client cuts the connection to Twitter and all of the known anonymanizers like “agentanon“. My hands began to tremble, my work became poorer and poorer (just a joke!).

Two lucky circumstances:

  • first free weekend since many month
  • my SAP PRD server is already up and connected to the internet, because I have a presentation on Monday

Why don’t turn a problem into a challenge and develop \ my own “ABAP twitter client” ?

Okay, with the help of the twitter API wiki I’ve started with a tiny program like this to see the data format of the twitter response:

DATA: gr_client TYPE REF TO if_http_client
    , gv_content TYPE string
    .

cl_http_client=>create(
  EXPORTING
    host = 'twitter.com'
  IMPORTING
    client = gr_client
    ).

gr_client->request->set_header_field(
  name = '~request_uri'
  value = '/statuses/public_timeline.xml'
  ).

gr_client->authenticate(
  username = 'se38'
  password = 'not_my_password'
  ).

gr_client->send( ).
gr_client->receive( ).
gv_content = gr_client->response->get_cdata( ).
gr_client->close( ).

The response looked like this:

/wp-content/uploads/2009/02/twitter_response_107715.jpg

Not really complicated I thought and I coded a \ corresponding data structure:

TYPES: BEGIN OF ts_user
     ,   id TYPE string
     ,   name TYPE string
     ,   screen_name TYPE string
     ,   description TYPE string
     ,   location TYPE string
     ,   profile_image_url TYPE string
     ,   url TYPE string
     ,   protected TYPE string
     ,   followers_count TYPE i
     , END OF ts_user

     , BEGIN OF ts_status
     ,   created_at TYPE string
     ,   id TYPE string
     ,   text TYPE string
     ,   source TYPE string
     ,   truncated TYPE string
     ,   in_reply_to_status_id TYPE string
     ,   in_reply_to_user_id TYPE string
     ,   favorited TYPE string
     ,   user TYPE ts_user
     , END OF ts_status
     .

DATA:  gt_statuses TYPE TABLE OF ts_status.

Because of the simple structure I have decided not to use simple transformation but the class CL_XML_DOCUMENT, that I have used before for creating an XML document out of a DDIC structure.

After hours of trying to parse the twitter response into my structure without success, I tried the vice versa way: filling my structure with test data and creating an XML document. The result:

/wp-content/uploads/2009/02/class_export_107716.jpg

Do you see the difference?

The “status”-node became “item” and all other tags are in upper case.

Using REGEX within ABAP and much “Hirnschmalz” I transformed the twitter response into the CL_XML_DOCUMENT conform input.

SPLIT gv_content AT cl_abap_char_utilities=>newline INTO TABLE gt_data.

LOOP AT gt_data
  ASSIGNING <gv_data>.

  IF sy-tabix > 1.

    FIND ALL OCCURRENCES OF REGEX '<[^>]*>' IN <gv_data> RESULTS gt_results.

    LOOP AT gt_results
      ASSIGNING <gv_result>.

      ASSIGN <gv_data>+<gv_result>-offset(<gv_result>-length) TO <gv_tag>.
      TRANSLATE <gv_tag> TO UPPER CASE.

    ENDLOOP.

  ENDIF.

ENDLOOP.

REPLACE ALL OCCURRENCES OF '<STATUS>' IN TABLE gt_data WITH '<item>'.
REPLACE ALL OCCURRENCES OF '</STATUS>' IN TABLE gt_data WITH '</item>'.

With this input the parsing works as expected:

DATA: gr_xml_doc TYPE REF TO cl_xml_document
       , gv_rc      TYPE sysubrc
       .

CREATE OBJECT gr_xml_doc.
CHECK gr_xml_doc IS BOUND.

gv_rc = gr_xml_doc->parse_table( gt_data ).
CHECK gv_rc IS INITIAL.

gr_xml_doc->get_data(
  IMPORTING
    retcode = gv_rc
  CHANGING
    dataobject = gt_statuses
    ).

Be careful here: for GT_DATA don’t use a “TYPE TABLE OF STRING”, else:

  1. For the tag replacements you cannot use offsets
  2. The CL_XMS_DOCUMENT produces a dump 😉

Most of the work done (I thought!), now the WD4A GUI

First build the context like the above defined \ structure, bind the table to the context, embed a table into the view, bind the \ context to this table, etc. etc. etc.

But no:

lo_nd_statuses->bind_table( new_items = gt_statuses ).

The status fields were filled, but not the user substructure!

And again, the world would be so boring without these little hurdles 😉

New internal table without substructure, mapping the fields to the new table, creating new context:

/wp-content/uploads/2009/02/context_107834.jpg

The rest is some kind of finger exercise and not part of this blog.

Conclusion and questions:

  • It works:-)
  • Why we have to use the “item” tag ?
  • Why we have to use upper case tags ?
  • Why we cannot bind a table with substructures into a context with substructures ?
  • Was a nice “project”, had a lot of fun!

If someone has a better/smarter/quicker way to develop such kind of XML-WD4A-Bindings, or has the answers for the mentioned questions, please don’t hesitate to comment this blog.

By the way: here is the result:

/wp-content/uploads/2009/02/client_107835.jpg

G+

Assigned Tags

      29 Comments
      You must be Logged on to comment or reply to a post.
      Author's profile photo Thorsten Franz
      Thorsten Franz
      Hi Uwe,
      Thanks for the interesting blog. I like reading about those kind of hurdles and how they are overcome. 🙂
      One observation about your ABAP data structures: Now you have typed GT_STATUSES to TABLE OF TS_STATUS. If instead you give it one named element STATUS which is typed to TABLE OF TS_STATUS, your XML document would probably come out with the desired structure, thus saving you the nasty search and replace business.
      Cheers & I look forward to your next blog,
      Thorsten
      Author's profile photo Uwe Fetzer
      Uwe Fetzer
      Blog Post Author
      Hi Thorsten,
      thanks.
      If I remember correctly, the named structure "status" was one of the tasks, which I have already tried yesterday. Unfortunately this doesn't work neither.
      Uwe
      Author's profile photo Jim Spath
      Jim Spath
      To follow Uwe on twitter, go to twitter.com/se38 (as long as your security allows it).  Jim
      Author's profile photo Former Member
      Former Member
      although I stopped twitter use due to high time consumption of "twitter practice" I will test your application, great job. Thanks for sharing.
      Ignacio.
      Author's profile photo Steve Oldner
      Steve Oldner
      Very cool!  I have not done any type of ABAP like this before.  May I see the full program?
      Author's profile photo Uwe Fetzer
      Uwe Fetzer
      Blog Post Author
      Hi Steven,
      the only missing part in the blog is the "send" routine. Will post the code later (only have a Webgui connection to my server in the moment).
      Distribute the code snippets to the appropriate WD4A methods and you have a working application 😉
      But like I mentioned in the blog: to create the web application was not the problem (and not part of this blog, there are much better blogs about WD4A development).
      Regards, Uwe
      Author's profile photo Uwe Fetzer
      Uwe Fetzer
      Blog Post Author
      And here's the missing part:


      *--- send message ---*
        cl_http_client=>create(
          EXPORTING
            host = 'twitter.com'
          IMPORTING
            client = lr_client
            ).

        lr_client->request->set_header_field(
          name = '~request_uri'
          value = '/statuses/update.xml'
          ).

        lr_client->request->set_header_field(
          name  = '~request_method'
          value = 'POST'
          ).

        lr_client->request->set_form_field(
          name  = 'status'
          value = lv_twitt
          ).

        lr_client->authenticate(
          username = lv_name
          password = lv_password
          ).

        lr_client->send( ).
        lr_client->receive( ).
        lv_content = lr_client->response->get_cdata( ).
        lr_client->close( ).

      Author's profile photo Former Member
      Former Member
      Already told you on Twitter...This is brilliant! I really love how you manage to bring Twitter to the SAP world...Great job! You have already made a place into my "Blag's best blog picks for 2009"!

      Greetings,
      Blag.

      Author's profile photo Moya Watson
      Moya Watson
      believe it or not, you can work at SAP without knowing how to code ABAP -- but this sort of thing makes me want to learn! ABAP!

      thanks for this post,
      -m

      Author's profile photo Moya Watson
      Moya Watson
      ... but uwe -- i can't find you on twitter! what's your twitter username?

      thanks,
      @moyalynne

      Author's profile photo Somnath Manna
      Somnath Manna
      @moyalynne follow uwe on twitter @se38 what else can you expect from him 🙂
      @som_nath
      Author's profile photo Uwe Fetzer
      Uwe Fetzer
      Blog Post Author
      As promised: now available for download http://www.se38.de/twitter.htm
      Author's profile photo Former Member
      Former Member
      Hi, very nice blog! Im new on this so I was wondering since you mentioned that gt_data shouldn't be a type table of string, what should it be then? Pardon the ignorance. Thanks.
      Author's profile photo Uwe Fetzer
      Uwe Fetzer
      Blog Post Author
      Hi,
      if I remember correctly, I have defined a local type "tv_data TYPE C LENGTH 5000".
      Hope this helps, Uwe
      Author's profile photo Former Member
      Former Member
      And gt_data is supposed to be gt_data type table of tv_data? was trying to download the nugget, but i can't seem to make it work. Please be kind enough to post the data declaration of gt_data. Thanks.:)
      Author's profile photo Uwe Fetzer
      Uwe Fetzer
      Blog Post Author
      You are right with "gt_data type table of tv_data".
      What is the problem with the nugget? The download or the installation of the nugget?
      You can contact me via twitter (@se38) 😉
      Regards, Uwe
      Author's profile photo Former Member
      Former Member
      Hi,

      I'm trying to rebuild your project and I get the tweets from the twitter server. That works really perfectly.

      But I have an other simple problem:

      - How can I get line breaks in the result table playced in the webDynpro view???

      Greets and merry xmas

      Dirk Voigt

      Author's profile photo Uwe Fetzer
      Uwe Fetzer
      Blog Post Author
      Hi Dirk,
      nice you like it 🙂
      Try to search for (or add) cl_abap_char_utilities=>cr_lf.
      Season greetings, Uwe
      Author's profile photo Former Member
      Former Member
      Hi Uwe,

      thanks for reply, but this is not what it solves.

      My problem is that every line of a table in WebDynpro have the same text and if the text is longer than the cell no line breaks will be inserted.

      Now I have included a new column with an image field containing a large sap icon. The height of the line won't adapt but the icon will a truncated. 

      In the following link on twitpic you see my probs:

      http://twitpic.com/u0gqn

      Author's profile photo Uwe Fetzer
      Uwe Fetzer
      Blog Post Author
      Hi Dirk,
      I see. Look at http://twitpic.com/u0lrp
      On the property page of the ALV-field you have to mark the checkbox "wrapping".
      Uwe
      Author's profile photo Former Member
      Former Member
      Hi Uwe,

      I know this checkbox but it doesn't work ... until I have unchecked the "fixedTableLayout" checkbox in the table preferences ... Homer Simpson would say D'oh! 😀

      But now it looks much better. Thanks for help

      http://twitpic.com/u0p2l

      Author's profile photo Tobias Hofmann
      Tobias Hofmann

      That's ... frustrating? Got a similar problem here. 75% are just gone, and the blog was featured on the Custom Development -> Java landing page of the old SCN.

      Lucky me: I write my blogs 1st as a DOC and then copy & paste the blog-worthy content.

      Author's profile photo Uwe Fetzer
      Uwe Fetzer
      Blog Post Author

      I'm writing blogs also in Word first. But after the blogs where published, I always delete them on my PC. Why hold a copy if we have them on SCN 😉

      (already in contact with Oliver Kohl to rescue my blogs)

      Author's profile photo Steve Rumsby
      Steve Rumsby

      I always keep copies of blog posts, whether here on SCN or elsewhere, separately. I save them into Evernote where I have both a cloud-based copy and a local copy on my laptop. I'm afraid I just don't trust any service, whether SAP, WordPress or Google (my current personal blog is on Blogger) to not trash my content.

      One of my Golden rules - you can never have too many backups.

      Author's profile photo Uwe Fetzer
      Uwe Fetzer
      Blog Post Author

      I just don't trust any service

      I'm still learning (every day)

      Author's profile photo Thorsten Franz
      Thorsten Franz

      Just in case rescuing doesn't work, you might be able to find a copy on one of the notorious rip-off sites that steal blog content from SCN to attract page views and make money from showing Google Ads. Remember #blogtheft?

      Cheers,

      Thorsten

      Author's profile photo Tobias Hofmann
      Tobias Hofmann

      Most of these site integrate by iFrame, so that content is gone there too. And in my case: nobody really copied my content 🙁

      Think that is a sign I should finally switch to ABAP ...

      Author's profile photo Uwe Fetzer
      Uwe Fetzer
      Blog Post Author

      "Unfortunately" such an old blog cannot be found on the #blogtheft sites. You can't trust the thiefs anymore 😉

      Also the scn blogs are not crawled by the way back machine http://web.archive.org due to the strict robots.txt.

      Author's profile photo Uwe Fetzer
      Uwe Fetzer
      Blog Post Author

      Blog post is repaired and back online 🙂