Skip to Content
Author's profile photo Eddy De Clercq

From the Grumpy Old Man: Searching for the needle in the SDN haystack

Isn’t it about time that SAP reviewed the SDN search engine? It is obvious that there are some (serious) shortcomings, which transform the search functionality into a burden instead of a helpful working instrument. I would appear that I’m not the only one that has this feeling about the search engine.
So what exactly are the painful areas within the search engine?

  • First of all, it’s rather slow. In fact, I’ve not seen a single search engine (within a site) that performs as badly as the SDN search engine. I know what I’m talking about since I’ve been monitoring things for the last 6 months now in a controlled way and have an average response time of 19 seconds. On only 3 occasions did I get a result in less than 15 seconds and I have even had peaks of 29 seconds a couple of times. I guess this won’t improve with the growing amount of content.
  • The performance would certainly be improved if it stopped indexing and showing results based on URLs instead of content. What do I mean? Well, try searching on BSP e-mail. The first page will show links to The problem is that only two results are valid within those first ten links. All the rest refer to the same content, but because it has different URLs, it is shown multiple times in the result list. The fun thing is that it is only a subtle difference in the URLs that engenders this clutter. Compare e.g. with
    Yes indeed, only one digit causes this to happen. In the first page of 10 results we get in total 6 variations of this without there being any difference content wise, in which case only 1 link is really useful. Clustering results would be more than advisable.
  • Expanding on the same theme, the search engine is even able to index non existing content. I don’t know how it is able to do this, but apparently the engine discovered the words BSP e-mail in Yet another search result that shouldn’t be there. I know by now how to avoid clicking on them (there is no size indication for this document), but it is very annoying when you click on something and get a 404 in return.
  • The search engine should only index the actual forum thread content and not the surrounding links. Let’s use the same BSP e-mail example again. You get forum threads like BSP & HTMLB tutorials back. You will discover that this thread has nothing to do with sending e-mail within BSP. Something stupid is causing this problem: in each forum post there is a link called E-mail this post. That means that the search engine will find 6900 results where only a few are effectively relevant. No logic operator will help you to avoid this.
  • Speaking of logical operators. It is far from logical (sorry for the pun, I couldn’t resist it) which operator’s do apply. I regularly see remarks on the AND operator and whether it works or not. But what if you want to search on AND itself. Should one use + instead? And what about the rest of the logical operators for e.g. excluding things? It also seems that the general search works differently from the forum search.
  • Why are there 2 different engines anyway? It makes things even more complicated and painful to search. Bhanu Gupta asked me something the other day via a forum thread. I didn’t have the time to reply immediately and when I did have time to compose an answer I couldn’t find the thread back again in the forums, since I didn’t set a watch on the thread. “No problem,” – I thought – “I’ll just search via user id in the forum search”. Well that seems to fail and the search engine gives me “More than one user found. Please refine your search.” as reason. To my knowledge, there is only one Bhanu and if not, why doesn’t it give me the threads of all those people with the same user? So I tried the general search engine indicating that I only wanted forums results. Surprisingly, it doesn’t grumble over the user, but it also retrieved the things that Bhanu isn’t even the author of.

I guess that our lives would be much easier without the above problems. Being a full Grumpy however I won’t be satisfied until the following features are included too:

  • Full MyCroft standard compatible in order to show the results in the Mozilla/SeaMonkey search bar. This is something I’ve been Foxy ISO SDN 4 LTR > 1.5 years for.
  • Speaking of which, Opensearch compliance is needed in order to be MIE 7 search bar compliant
  • Search suggestion compliancy via JSON is needed for Firefox 2. Wouldn’t it be nice if one could see what the most used search terms are?

It might not be the core business of SAP, but an industry leading company can’t afford to lag behind on such matters.

Assigned Tags

      You must be Logged on to comment or reply to a post.
      Author's profile photo Bhavesh Kantilal
      Bhavesh Kantilal
      I got to agree with you. Hopefully Craig and team are listening too.


      Author's profile photo Former Member
      Former Member
      Yes even I agree with the last point.. even I have faced the same problem...

      I would also like to suggest that in our Business Card under contribution tab the post posted by us and the post in which we took part we should be differentiate.

      It will defiantly of great help for me and I think for other also..


      Author's profile photo Former Member
      Former Member
      You are absolutely on the point. Search on SDN is extremely inefficient.

      After trying to figure it out (as to how it must be working internally) and failing (repeatedly), I am just resigned to not using it except in extreme situations. Which is a pity, for search is first tool you use to find something at any other site (even OSS which is way better/predictable than the one on SDN).

      Most apt weblog I should say.


      Author's profile photo Former Member
      Former Member
      Hi Eddy,

      Thx for the valuable feedback!
      It's passionate user like you and constructive feedback that help us make SDN and the SAP Enterprise Portal and Knowledge Management better.

      I'd like to share with you our short and longer term plans for improving the search:

      - Short Term: In around November we plan to upgrade SDN to NW04 (currently we are on EP 6.0 SP2). With this upgrade we will be releasing some backend improvements to the search that will include:
      1. Improved Performance.
         - No HTMLB
         - Reduced page size
         - Permission checks are not done when rendering the search results but at another point in time which makes the rendering faster
         - Search queries and results are cached on a user basis
         - The search configuration can be accessed in a faster manner.
      2. Improved Indexing.
         - The reason you get 404 errors on some of the links is that we are not able to index our content (such as as much as we need to. with the new release that should be improved allowing us to index more often and therefore have better updated links in the search.
      3. Use of Boolean operators.
      In the new search you'll be able to use the Boolean operators AND, OR and NOT. If you wanted to search for the word "and" you'd have to enclose it in double quotes to make it clear that this is the term you want to search for and not an operator.
      4. Search over forums. There are currently 2 different searches over the forums. one via the SDN left side navigation or advanced search. This uses the SAP KM search product. The other search is in the discussion groups themselves which uses the third part (Jive) discussion groups search facilities. With the NW04 upgrade we will also be upgrading from Jive 3.x to Jive 4.x. we will then monitor to see if there are any improvements in the jive search and if not we will follow up with jive to see what can further be done.
      5. Regarding your note on searching and getting content twice: We agree, it is annoying to get seemingly the same content twice. However this is the way content on is structured. They have a new section for each product release. As you pointed out the difference between the two links you brought up as an example is just one digit. However it is a very important one. It signifies two different product releases, namely 6.20 and 6.30. has duplicated its content for each release. In many cases the content hasn't changed and is the same for each release (like in the example he brought up). However there are also pages that have changed with the releases.
      * If you search only in the category (by choosing it in the drop down box) and open its advanced search you can choose to search only in the content of a specific release. Then you won't get duplicates.
      * In the new search we specify the product release a document refers to in the meta property section of the search result. This makes it easier to distinguish between seemingly duplicate entries that refer to different releases.

      - Longer terms plans:
      We are currently defining the road map of our site for 2007. I know that our SDN solution office has further improvements to search (search 2.0)  on the list. Once the road map and planning is completed the assignment of resources and budget can be done to further enhance this very important part of the site.

      Again thx for the valuable feedback and keep up the good work!
      Kind Regards,

      Ran Cliff
      Senior Director SDN Platform.

      Author's profile photo Eddy De Clercq
      Eddy De Clercq
      Blog Post Author
      Hi Ran,

      Many thanks for the clear roadmap and tips.
      Looking forward to it.