Skip to Content
Author's profile photo Former Member

OpenSearch on SCN

Overview

I’m part of an SAP team specialized in search applications. This team is working among other things on the engine that powers the SCN search and its interfaces. One of these interfaces, is the well known search UI: http://search.sap.com , which you see when you execute a search on SCN. Another, less publicized interface, is the Opensearch: http://search.sap.com/opensearch . This search API is available alongside the search UI and enables client applications to access and integrate the SCN search functionality in their own environments or browsers. As a result, I thought it would be a good idea to spend some words on the subject and hopefully open up new consumption scenarios to the SCN search.

Some words about Opensearch

Opensearch can be seen as a simple API standard for search results. By implementing it, any content provider, like SCN, can export the functionality of its internal search engine to be used by external parties. The argument behind it was that traditional search engines often cannot do a good job of crawling and indexing content at specialized web sites and that local search engines may understand local content far better, so why not call them directly.

With Opensearch you can execute searches, determine the size of the results, page through them and, depending on the extensions implemented, even filter and sort them. It’s like taking search results and cutting out the UI. Interesting for developers wanting to access and aggregate results from different search engines, but also to end users wanting to integrate a search provider in their browser, as we’ll see. 

In Opensearch, Search Requests are made via HTTP Requests with information sent as URL parameters and Results received as HTTP Responses in XML (RSS and ATOM) or HTML format.

A Simple Search

Let’s take a simple example and search for “hana”. Just call the Opensearch base URL and append the “query” parameter at the end. Don’t forget to URL encode the search term.

http://search.sap.com/opensearch?query=hana

Results in HTML, ATOM and RSS

Results appear by default in a rudimentary HTML format that can be integrated in your browser’s search box. However, to access the full functionality provided, you’ll need to use one of the Feed formats available RSS or ATOM. This can be achieved by introducing the “format” parameter. Then you can also visualize the results with your favorite Feed Reader. 

http://search.sap.com/opensearch?query=hana&format=rss

http://search.sap.com/opensearch?query=hana&format=atom 

The standard feed formats have been extended by the OpenSearch standard and SAP with some extra elements in order to accommodate data related to the search results, as you will see in the next sections.

Here’s an example, with the RSS format:

resultsInRss.png

Paging through results

Paging can be achieved via the parameters “itemsperpage” and “startindex”. The first one indicates the number of results per page and the second the index of the first result desired.

For example, you can define pages of 10 results and retrieve the first and second page via the following URLs:

         -> first page (first 10 results)

         -> second page (next 10 results)

Extracting Metadata

Metadata associated with each result can be retrieved  and inspected by using the parameter “extended”. Three values are possible: “id”, “label” or “both”, depending on whether you are interested in getting only the IDs, labels or both of the attributes and corresponding values associated with each result. 

http://search.sap.com/opensearch?query=hana&format=rss&extended=both

Here’s an example, with the “both” value:

resultsWithMetadata.png

Sorting Results

After having inspected the metadata associated with each result you might want to sort your results by one of the attributes available. This can be achieved via the parameter “sort” and the following syntax:

          sort=attributeName:sortDirection

where:

  • attributeName = name of the attribute (ID)
  • sortDirection = ascending or descending

Don’t forget to URL encode ”attributeName:sortDirection”.

As an example, to sort by descending order of modification date you would URL encode the value “scm_a_moddate:descending” and add it to the URL as indicated below:

http://search.sap.com/opensearch?query=hana&sort=scm_a_moddate%3Adescending 

Filtering Results

In a similar way, after having inspected the metadata associated with each result you might want to do some filtering based on that. For example, get only the results of Asset Type “Wiki Page” and Solution Suite “SAP NetWeaver”.

This can be accomplished by using the “filter” parameter and the following syntax:

         filter=attributeName(value=attraibuteValue/operator=attributeOperator)&filter=… 

where:

  • attributeName = name of the attribute (ID)
  • attributeValue = value of the attribute
  • attributeOperator = operator linking attribute and value (EQ, LT or GT)

Don’t forget to URL encode ”attributeName(value=attraibuteValue/operator=attributeOperator)” and note the following:

  • following operators are supported for properties of data type INTEGER, DATE and FLOAT: EQ (equals), GT (greater than) and LT (less than)
  • following operators are supported for properties of data type STRING and BOOLEAN: EQ (equals)
  • date values need to be provided in the following format: “yyyy-mm-dd”
  • several “filter” URL parameters are linked together with AND if they contain different properties or with OR if they refer to the same property (i.e. multi-value properties)

So, let’s try that out and filter only results of type “Wiki Page” from site “SCN” and author “Rafael Rhoden”. The corresponding filters are:

  • filter=scm_a_resourceType(value=scm_v_resType288/operator=EQ)
  • filter=scm_a_site (value=scm_v_Site11/operator=EQ)
  • filter=scm_a_author(value=Rafael Rhoden/operator=EQ)

which after encoding look like:

  • filter=scm_a_resourceType%28value%3Dscm_v_resType288%2Foperator%3DEQ%29
  • filter=scm_a_site+%28value%3Dscm_v_Site11%2Foperator%3DEQ%29
  • filter=scm_a_author%28value%3DRafael+Rhoden%2Foperator%3DEQ%29

Resulting in the following URL for the search term “srm”:

http://search.sap.com/opensearch?query=srm&

filter=scm_a_resourceType%28value%3Dscm_v_resType288%2Foperator%3DEQ%29&

filter=scm_a_site+%28value%3Dscm_v_Site11%2Foperator%3DEQ%29&

filter=scm_a_author%28value%3DRafael+Rhoden%2Foperator%3DEQ%29

You may find it interesting to notice, that when you use the search UI and go about filtering your results by clicking on the available filters on the left side of the screen, the corresponding URL changes to reflect the attributes and values you select (as filters) as you go along. These (URL) filters, although with a different syntax, contain the same attributes and attribute values that you use when filtering with OpenSearch and, in this sense, can be used as a helping hand to find out which attributes and attribute values you want to use in your OpenSearch filter. For example, the corresponding UI filter for the above OpenSearch one would be:

http://search.sap.com/ui/#

query=srm&startindex=1&

filter=scm_a_site(scm_v_Site11)&

filter=scm_a_resourceType(scm_v_resType288)&

filter=scm_a_author(Rafael+Rhoden)

A closer look at Results

As you probably noticed, the search results in XML format don’t exactly match the standard RSS and ATOM feeds. They contain new tags, nonexistent in the definition of these standards. These are extensions. Some from the Opensearch standard (tags qualified with namespace “opensearch”) and others from SAP (tags qualified with namespace “sap_it”).

Let’s have a look at some of them:   

Feed Extensions

  • itemsPerPage = number of results displayed per page
    • Example: <opensearch:itemsPerPage>10</opensearch:itemsPerPage>
  • totalResults = total number of results found
    • Example: <opensearch:totalResults>190</opensearch:totalResults>
  • startIndex = index of the first result displayed
    • Example: <opensearch:startIndex>1</opensearch:startIndex>
  • status = status of the search. “success” if there were no errors and “error” otherwise.
    • Example: <sap_it:status>error</sap_it:status>
  • message = corresponding error message, if “status”=”error”
    • Example: <sap_it:message> Please enter a correct Filter Expression: attributeName(value=attributeValue/operator=operatorValue)</sap_it:message>

Item Extensions

  • snippet = content snippet highlighting the search term.
    • Example: <sap_it:snippet><![CDATA[[<b>HANA</b> Overview…]]></sap_it:snippet>
  • metadataAttribute = metadata associated with each result. Depending on the URL parameter “extended” includes technical ID and label of each attribute and its values.
    • Example: extended =both

               <sap_it:metadataAttribute>

                   <sap_it:id>scm_a_country</sap_it:id>

                   <sap_it:label language=”en“>Countries</sap_it:label>

                   <sap_it:valueId>scm_v_country237</sap_it:valueId>

                   <sap_it:valueLabel language=”en“>United States</sap_it:valueLabel>

               </sap_it:metadataAttribute>

Description Document

This XML document specified by the OpenSearch standard, is the place where we publish the web interface of our search service. It’s like a formal description of the OpenSearch signature provided, so that clients know which URL templates are available and how to call them.

You can read more about the different elements in the standard OpenSearch Description Document Documentation and you can access ours under  http://search.sap.com/opensearch/description

Online Documentation

If you want more information on available features and usage, you can check out the delivered online documentation by clicking on “SAP IT Opensearch Results…”  (title above search results) or going directly to http://search.sap.com/opensearch/description?type=extensions.

Browser Integration and Auto-Discovery

One of the nice features about OpenSearch is its browser integration capabilities. You can add SCN as a search provider to your browser’s search box and launch the SCN search from within the browser without having to go to SCN.

browserIntegration1.png

Most modern browsers support this functionality by allowing its search box to be customized. They support auto-discovery of OpenSearch description documents and integration of search providers in the search box.

Here’s how you can do it in Firefox:

First, you need to call a page containing a so-called auto-discovery link to our search provider in its HTML header. You can enable one of your own pages by including the link below in the header or you can just call our documentation page http://search.sap.com/opensearch/description?type=extensions, which already includes one. This link basically says to the browser that this page contains a search provider that can be added to the browser.

<head>

     <link rel=”search”

              type=”application/opensearchdescription+xml”

              href=”http://search.sap.com/opensearch/description

              title=”SAP IT OpenSearch”

     />

</head

You can see this, as the browser immediately displays a different color on the dropdown arrow of its search box. If you click on it, you see the already installed search providers and the possibility to add a new one: “SAP IT Opensearch”. Add it and you’re good to go.

browserIntegration2.png

Now, you can select “SAP IT Opensearch” from the dropdown anytime and launch an SCN search from within the browser.  

browserIntegration3.png

Note: In reality, and to be 100% correct, the description above integrates in your browser not only the SCN search, but rather the more general SAP External search, which includes SCN.

Accessing Private Content

Our OpenSearch Interface is intended to be used primarily by Anonymous clients to access Public content and therefore does not require authentication.

However, if you need to access restricted content and have the corresponding credentials, you can do it. A Basic Authentication mechanism is provided, which allows you to preemptively send the user credentials in the HTTP header of the request and the server will check for this. If credentials are present, it will try to authenticate the user and the results returned will be according to his authorizations. Otherwise, only pubic content will be returned. 

The credentials should be send using the HTTP Authorization header, as a “userName:password” Base64 encoded pair in the form indicated by the example below.

We recommend to use this scenario only via HTTPS, so that credentials are sent via a secure connection.

For example:

If you want to authenticate using the following user:

username = user123

password = open123

You need to first base64 encode the string: “user123:open123”.

The result is: “dXNlcjEyMzpvcGVuMTIz”.

Finally, you need to include the header indicated below in the HTTP Request:

Authorization: Basic dXNlcjEyMzpvcGVuMTIz

Assigned Tags

      31 Comments
      You must be Logged on to comment or reply to a post.
      Author's profile photo Former Member
      Former Member

      Hi Vasco,

      great blog, thanks for sharing this. Browser integration of the OpenSearch is cool!

      Thanks, Martin

      Author's profile photo Alon Aizenberg
      Alon Aizenberg

      Vasco, good work!

      Thanks

      Author's profile photo Paul Aschmann
      Paul Aschmann

      Nice and detailed blog, thanks for sharing Vasco. Looking forward to using OpenSearch in a new app I have on the way, and this will make it much easier.

      Author's profile photo Former Member
      Former Member
      Blog Post Author

      Thanks everyone!

      Author's profile photo Former Member
      Former Member

      Hi Vasco,

      Nice blog,

      I'm using NWBC, and I wish to add SAP open Search as Search provider, I use search URL (http://search.sap.com/opensearch?query={0}&format=html) , but I dont obtain the result on the preview as if I use Open Search description file like this URL http://en.wikipedia.org/w/opensearch_desc.php

      Did SAP search has an URL for a description file?

      Author's profile photo Former Member
      Former Member
      Blog Post Author

      Hi Jose,

      Thanks! The link to our description document is indicated above in the blog and is: http://search.sap.com/opensearch/description

      Kind Regards,

      Vasco

      Author's profile photo Former Member
      Former Member

      Hi Vasco,

      I put the URL that you said on the NWBC a Search provider application and it's not working as other search provider (like the wikipedia one).

      Can you explain how to create a Search provider on NWBC with SAP OpenSearch?

      Regards

      Jose

      Author's profile photo Former Member
      Former Member
      Blog Post Author

      Hi José,

      Unfortunately, I'm not familiar with NWBC. You may need to contact someone who knows how to integrate a search provider on NWBC and if there are any restrictions there.

      Sorry,

      Vasco

      Author's profile photo Martin Schygulla
      Martin Schygulla

      Great post. Thanks for sharing.

      Author's profile photo Martin English
      Martin English

      If you want to base64 encode anything as a one off for authorisation as described above, you can use this link I found at thecoderstoolboxnet

      http://coderstoolbox.net/string/#!encoding=base64&action=encode&charset=us_ascii

      There's a few of these about, but this one was quick, ads are unobtrusive and it also has links to other useful tools,such as Time Conversion, Number Conversion and network and bandwidth calcultaors.

      hth

      Author's profile photo Former Member
      Former Member

      Thanks a lot. very interesting.

      Author's profile photo shravan kumar
      shravan kumar

      Great post ....

      Author's profile photo Amarpreet Singh
      Amarpreet Singh

      Wow makes me respect SCN Search a lot more, never knew there was so much we could do with it .

      Awesome work . 🙂

      Author's profile photo Zsuzsanna Hartyani
      Zsuzsanna Hartyani

      great:-)

      Author's profile photo Dimitris Karagiannis
      Dimitris Karagiannis

      Nice post.. Thanks

      Author's profile photo Robert Russell
      Robert Russell

      Hi Vasco,

      Many thanks for the blog, great information and it's inspired me to try the Opensearch tool and I ended up trying a few SCN search related experiments. Hopefully I will post a blog on this soon.

      Cheers

      Robert

      Author's profile photo Kjetil Kilhavn
      Kjetil Kilhavn

      This was great news (new to me) - you (SCN as search provider) has been added!

      You aren't the default search provider (yet) since KDE Activities don’t support having a special default search provider for my "Work" activity.

      Author's profile photo Former Member
      Former Member

      Great information, thank you for sharing it.

      Thanks,

      Author's profile photo Santhosh Kumar A
      Santhosh Kumar A

      Nice blog, Thanks for sharing..

      Regards,

      Santhosh

      Author's profile photo Former Member
      Former Member

      Very useful information.

      Thanks

      Ralf

      Author's profile photo Mahesh Bramharoutu
      Mahesh Bramharoutu

      Hi

      Very Nice post Thanks

      Thanking you

      Regards

      Mahesh

      Author's profile photo Louis Ojuwu
      Louis Ojuwu

      Thnak you. will be sharing this link.

      Author's profile photo Former Member
      Former Member

      Very helpful document for everyone.Thanks!! a lot.

      Please keep rocking.

      Best,

      SAtya.

      Author's profile photo Former Member
      Former Member

      Thanks

      Author's profile photo MARC FILLION
      MARC FILLION

      Hi Vasco,

      Thanks for this really interesting blog post.

      Marc

      Author's profile photo Former Member
      Former Member

      Great Article Vasco!!

      Author's profile photo Peter Wiech
      Peter Wiech

      Good and helpful. Thanx. 🙂

      Author's profile photo Former Member
      Former Member

      Good one

      Author's profile photo Nizar Fanany
      Nizar Fanany

      Thank you, very helpful

      Author's profile photo Former Member
      Former Member

      Pls mind that the OpenSearch interface has been finally retired together with search on Trex.

      Author's profile photo Osvaldo Lopez
      Osvaldo Lopez

      I think we must report this doc. It's outdated