Twibap: the ABAP Twitter API
You probably know my last year’s SCN Blog post A story about Twitter, XML and WD4A 😉 In April this year I’ve received a short DM from Mark F. (http://www.sdn.sap.com/irj/scn/bc?u=g9g0pzynhca%3d) “getting #SAP #ABAP on this list would be wholesome” (he referted to the site: http://dev.twitter.com/pages/libraries (http://dev.twitter.com/pages/libraries)).
Nice idea, I thought, give me time until the summer break. End of July I wanted to refresh my Twitter API knowledge by reading the docs and I saw this message on the dev site: “The @twitterapi team will be shutting off basic authentication on the Twitter API. All applications, by this date (http://countdowntooauth.com/), need to switch to using OAuth.” No problem, with OAuth I’ve battled already while developing Wave and Streamwork apps. (haha, more later)
Chapter 1 – The Twitter API
Chapter 2 – The JSON Parser
In my Twitter WDA client I’ve used the XML response. Since I felt in love with JSON (http://json.org/) while working with Python, I’ve decided to use JSON this time. First prob: how to parse JSON ABAP? My search on SCN found the nice JSON function group from Quentin Dubois (http://www.sdn.sap.com/irj/scn/bc?u=z5uinayahfg%3d) (Wiki page (http://wiki.sdn.sap.com/wiki/pages/viewpage.action?pageId=163840922))
Because the Twitter response contains not only flat data but also embedded objects (a status object always contains a user object) and some responses are arrays, the mentioned function group is not really the solution I needed, so I’ve decided to write my own parser (but with parts of the code of the module, hope Quentin doesn’t kill me now). The result of a parsed JSON object is now a hashed key/value table, where we can read each element by simply call e.g. “text = simplejson->get_value( ‘text’ ).“.
Is the result of the read element again an object, you have just to parse it again:
user = simplejson->get_value( 'user' ). "returns another object user_data = simplejson->parse_object( user ). "parse object simplejson->set_data( user_data ). "set new data in parser screen_name = simplejson->get_value( 'screen_name' ). "get element
The result of a parsed JSON array is a standard table of the hashed key/value table. With this tiny simplejson helper class I wrote my first twitter API classes, and since the basic authentification is not cut off yet, all test went well until here.
Chapter 3 – OAuth
The next step of the journey was the implementation of OAuth. With a look at my Python sources (Streamwork OAuth implementation) and the first chapters of the Twitter OAuth docs (http://dev.twitter.com/pages/auth) it seemed very familiar and I began with the realization.
An OAuth request contains, amongst others, two fields called “oauth_nonce”, a string with random characters, and “oauth_timestamp”, the seconds counted from Jan. 1st 1970. Because there are no standard functions (I think so), I’ve developed two small helper methods:
method CREATE_NONCE. CONSTANTS: chars TYPE string VALUE '0123456789ABCDEFGHIJKLMNOPQRSTUVWXTZabcdefghiklmnopqrstuvwxyz'. DATA: rnd TYPE i. DO length TIMES. "input parameter (twitter needs 8 chars) CALL FUNCTION 'QF05_RANDOM_INTEGER' EXPORTING ran_int_max = 60 ran_int_min = 0 IMPORTING ran_int = rnd. CONCATENATE nonce chars+rnd(1) INTO nonce. ENDDO. endmethod. method CREATE_TIMESTAMP. CONSTANTS: unix TYPE d VALUE '19700101'. "Unix' birthday DATA: timestamp_i TYPE i , diff TYPE i . GET TIME. diff = sy-tzone. "diff to UTC in secs IF sy-dayst = 'X'. "daylight saving active? ADD 3600 TO diff. ENDIF. timestamp_i = ( sy-datum - unix ) * 86400 "days in secs + sy-uzeit(2) * 3600 "hours in secs + sy-uzeit+2(2) * 60 "mins in secs + sy-uzeit+4(2) "secs - diff "diff to UTC in secs . timestamp = timestamp_i. timestamp = timestamp(10). "w/o sign (trailing space) endmethod.
Hint under friends: if the timestamp is not correct, Twitter will refuse your request, believe me 😉
-> set your system time correctly!
-> read more: OAuth at Twitter (http://dev.twitter.com/pages/auth)
Chapter 4 – HMAC-SHA1
But the first test results brought me back down to earth: I’ve overseen the tiny remark “Twitter requires that all OAuth requests be signed using the HMAC-SHA1 algorithm.” WTF? Streamwork uses PLAINTEXT authentification, but what is HMAC-SHA1? Googlegooglegoogle The search brought me two results (ok, much more than two, but these two are the most relevant ones):
- a simple note in an , means there is a function module called “CALCULATE_HASH_FOR_CHAR”
SE24, “CL_*JAVA*”, points me to this SAP help site (http://help.sap.com/saphelp_nw70ehp1/helpdata/en/49/a8e3c8d59811d4b2e90050dadfb92b/frameset.htm)
Again WTF…. But thanks god (and SAP!), we still have the old docus available: here you can find the relevant part from NW7.0 (http://help.sap.com/saphelp_nw70/helpdata/en/86/8676416e805958e10000000a1550b0/frameset.htm)
abap.oauth_signature = b64_hmac_sha1(abap.oauth_secret, abap.basestring) + '=';
and with this code we finally can sign the message:
*--- compile source and bind variables ---* js_processor = cl_java_script=>create( ). js_processor->bind( EXPORTING name_obj = 'abap' name_prop = 'oauth_secret' CHANGING data = me->oauth_secret ). basestring = create_basestring( method ). js_processor->bind( EXPORTING name_obj = 'abap' name_prop = 'basestring' CHANGING data = basestring ). js_processor->bind( EXPORTING name_obj = 'abap' name_prop = 'oauth_signature' CHANGING data = me->oauth_signature ). return_value = js_processor->evaluate( js_source ).
In addition I only had to develop my own encoding method called “percent_encode”, because the “cl_http_utility=>escape_url()” method doesn’t fit to the OAuth dictate, where the only characters you can ignore are “- _ . ~” (and some other abnormalities).
The whole Twitter workflow works nice now, but I was not very satisfied with this JS solution. Therefore back to SAP notes and google for a deeper search for more information about the function module “CALCULATE_HASH_FOR_CHAR“.
Chapter 6 – The SecureStore
In Note 1416202 I finally found the answer. The function modules are NOT “secret”, but “The raw documentation was not activated.” With NW 7.01 SP7 the documentation will be delivered (I was so close to install SP7 on my system…, but luckily I found the docu in the infinite vastness of the internet).
In the documentation of the function group “SECH” and its function modules we can read, that we can use these function modules for our own purposes. So did I:
"*--- set secret to SecureStorage ---* CALL FUNCTION 'SET_HMAC_KEY' EXPORTING keycstr = me->oauth_secret client_independent = space. "*--- calculate base64 signature ---* CALL FUNCTION 'CALCULATE_HMAC_FOR_CHAR' EXPORTING alg = 'SHA1' data = basestring key_must_exist = 'X' IMPORTING hmacbase64 = signature.
Boom, Dump, failed again. What happened? In the first steps of the OAuth authorization process (request_token etc.) the oauth_secret contains only the consumer_secret (42 characters + “&”). The function module ‘SET_HMAC_KEY’ works brilliant until that point, where I want to sign a user action (e.g. sending a tweet). In this case the secret combines the consumer secret and the token secret (of the user). The function module responses with an “parameter_length” exception.
With some experiments I found out, that the FM only accepts 81 characters as maximum.
Hey, why? I only want to SET the key, no process at this moment. And in addition: nowhere in the HMAC-SHA1 OAuth key definition is a length maximum mentioned. In my despair I opened a SCN forum thread (HMAC (SHA1) key longer than 81 characters not possible?). And what a surprise (or not): no 24 hours later I’ve got the solution 🙂 SCN members rock!
The solution: if the key is longer than 81 characters, we have to store the hash of the key, not the key itself (still don’t know why).
The code snippet:
"*--- FM 'SET_HMAC_KEY' doesn't accept keys > 81 chars ---* IF STRLEN( me->oauth_secret ) < 82. "*--- set secret to SecureStorage ---* CALL FUNCTION 'SET_HMAC_KEY' EXPORTING keycstr = me->oauth_secret client_independent = space. ELSE. "*--- hash the secret ---* CALL FUNCTION 'CALCULATE_HASH_FOR_CHAR' EXPORTING data = me->oauth_secret IMPORTING hashx = hashx. secret_hashed = hashx. "*--- set hashed secret to SecureStorage ---* CALL FUNCTION 'SET_HMAC_KEY' EXPORTING keyxstr = secret_hashed client_independent = space. ENDIF.
You: “And for what is it good for?”
Me: “No idea”
You: “But why did you make it”
Me: “You could also ask: Why are you running Android 2.2 (Froyo) on a WindowsMobile phone. The answer would be the same:
“Because it works, and it makes so much fun 😉 “