A story about Twitter, XML and WD4A

[UweFetzer_se38]

Prolog

Like many of us, I'm a kind of addicted to Twitter. But a few weeks ago, the admins of my client cuts the connection to Twitter and all of the known anonymanizers like "agentanon". My hands began to tremble, my work became poorer and poorer (just a joke!).

Two lucky circumstances:

• first free weekend since many month
• my SAP PRD server is already up and connected to the internet, because I have a presentation on Monday

Why don't turn a problem into a challenge and develop \ my own "ABAP twitter client" ?

Okay, with the help of the twitter API wiki I've started with a tiny program like this to see the data format of the twitter response:

DATA: gr_client TYPE REF TO if_http_client
    , gv_content TYPE string
    .

cl_http_client=>create(
  EXPORTING
    host = 'twitter.com'
  IMPORTING
    client = gr_client
    ).

gr_client->request->set_header_field(
  name = '~request_uri'
  value = '/statuses/public_timeline.xml'
  ).

gr_client->authenticate(
  username = 'se38'
  password = 'not_my_password'
  ).

gr_client->send( ).
gr_client->receive( ).
gv_content = gr_client->response->get_cdata( ).
gr_client->close( ).

The response looked like this:

Not really complicated I thought and I coded a \ corresponding data structure:

TYPES: BEGIN OF ts_user
     ,   id TYPE string
     ,   name TYPE string
     ,   screen_name TYPE string
     ,   description TYPE string
     ,   location TYPE string
     ,   profile_image_url TYPE string
     ,   url TYPE string
     ,   protected TYPE string
     ,   followers_count TYPE i
     , END OF ts_user

     , BEGIN OF ts_status
     ,   created_at TYPE string
     ,   id TYPE string
     ,   text TYPE string
     ,   source TYPE string
     ,   truncated TYPE string
     ,   in_reply_to_status_id TYPE string
     ,   in_reply_to_user_id TYPE string
     ,   favorited TYPE string
     ,   user TYPE ts_user
     , END OF ts_status
     .

DATA:  gt_statuses TYPE TABLE OF ts_status.

Because of the simple structure I have decided not to use simple transformation but the class CL_XML_DOCUMENT, that I have used before for creating an XML document out of a DDIC structure.

After hours of trying to parse the twitter response into my structure without success, I tried the vice versa way: filling my structure with test data and creating an XML document. The result:

Do you see the difference?

The "status"-node became "item" and all other tags are in upper case.

Using REGEX within ABAP and much "Hirnschmalz" I transformed the twitter response into the CL_XML_DOCUMENT conform input.

SPLIT gv_content AT cl_abap_char_utilities=>newline INTO TABLE gt_data.

LOOP AT gt_data
  ASSIGNING <gv_data>.

  IF sy-tabix > 1.

    FIND ALL OCCURRENCES OF REGEX '<[^>]*>' IN <gv_data> RESULTS gt_results.

    LOOP AT gt_results
      ASSIGNING <gv_result>.

      ASSIGN <gv_data>+<gv_result>-offset(<gv_result>-length) TO <gv_tag>.
      TRANSLATE <gv_tag> TO UPPER CASE.

    ENDLOOP.

  ENDIF.

ENDLOOP.

REPLACE ALL OCCURRENCES OF '<STATUS>' IN TABLE gt_data WITH '<item>'.
REPLACE ALL OCCURRENCES OF '</STATUS>' IN TABLE gt_data WITH '</item>'.

With this input the parsing works as expected:

DATA: gr_xml_doc TYPE REF TO cl_xml_document
       , gv_rc      TYPE sysubrc
       .
CREATE OBJECT gr_xml_doc.
CHECK gr_xml_doc IS BOUND.

gv_rc = gr_xml_doc->parse_table( gt_data ).
CHECK gv_rc IS INITIAL.

gr_xml_doc->get_data(
  IMPORTING
    retcode = gv_rc
  CHANGING
    dataobject = gt_statuses
    ).

Be careful here: for GT_DATA don't use a "TYPE TABLE OF STRING", else:

1. For the tag replacements you cannot use offsets
2. The CL_XMS_DOCUMENT produces a dump 😉

Most of the work done (I thought!), now the WD4A GUI

First build the context like the above defined \ structure, bind the table to the context, embed a table into the view, bind the \ context to this table, etc. etc. etc.

But no:

lo_nd_statuses->bind_table( new_items = gt_statuses ).

The status fields were filled, but not the user substructure!

And again, the world would be so boring without these little hurdles 😉

New internal table without substructure, mapping the fields to the new table, creating new context:

The rest is some kind of finger exercise and not part of this blog.

Conclusion and questions:

• It works:-)
• Why we have to use the "item" tag ?
• Why we have to use upper case tags ?
• Why we cannot bind a table with substructures into a context with substructures ?
• Was a nice "project", had a lot of fun!

If someone has a better/smarter/quicker way to develop such kind of XML-WD4A-Bindings, or has the answers for the mentioned questions, please don't hesitate to comment this blog.

By the way: here is the result:

G+