What is the optimal configuration for Platform Search in BI4.x?
In previous versions of SAP BusinessObjects the search functionality was basic, with BI4.0 a higher level of search has been introduced, an application in itself with the name of Platform Search.
I am often asked what is the best configuration for Platform Search, the answer is not as simple as it seems, because this feature is extremely powerful and flexible, that needs to be adjusted according to each installation’s business needs.
With this page I will try to provide some personal insights that can give a better understanding of the capabilities of this powerful feature.
Within Platform Search is possible to leverage the levels, type of content and the modality of indexing.
Levels of Indexing:
as from BI4.0 there are three levels of content indexing for the search functionality of BI Platform
They are configured in the CMC > Applications > Platform Search Application > properties
– Platform Metadata: only titles, keywords and descriptions of the documents
– Platform and Document Metadata : indexes Platform Metadata plus creation/modification dates, name of the author
– Full Content (default): same as Platform and Document Metadata plus all the content in the document, including prompts and LOVs, as well as Charts, Graphs, Labels.
Types of Objects:
the BI Administrator can also set the types of objects that can be indexed, both BI Content and agnostic documents (any non-SAP BI Platform object, including MS Office documents of any kind, PDF, text, etc.)
When the Full Content is selected, the Platform Search service, that is hosted in the Adaptive Processing Server, will index every object that that is published to the BI Platform. This indexing process is called crawling.
If the universe indexing is selected, when a universe is published the objects it contains are parsed (tested against the database), when this happens you may see activity in the reporting database, SQL statements are being run apparently by no apparent user, they are actually being sent by the BI Platform. (see KBA 1625939 – Ghost queries are run to the reporting database from BO server)
As a consequence, when a consistent number of content is published to the BI Platform, one may experience a high load on the reporting database, to not mention network traffic between the BI servers and the reporting database and the Platform Search database, which can result in performance and/or availability issues.
(You may also experience high memory and/or cpu usage by the processing servers that are used by Platform Search to validate the objects)
The solution in that case would be to schedule the crawling so that indexing can be done when times when there is a lower usage of the resources, taking into account also availability for scheduled jobs.
To configure the scheduling there are two steps:
1. In the CMC > Applications > Platform Search Application properties page, select Scheduled Crawling
then Save and Close
2. Go to the CMC > Folders > Platform Search Scheduling Object
in the properties page it is possible to specify the desired settings
In conclusion, the Platform Search application is extremely useful, and should be configured according to the business needs, after analysis and planning.
If the BI Platform has a lot of content and there is the need to have a high level of indexing, the suggestion is to use scheduled crawling.
On the other hand, if there is no need of searching throughout all content, it is preferable to tune the settings, so that the indexing affects only the level and type of content that is an acceptable compromise between performance and search capabilities.
Is there an interface with security for what content is searched? According to your blog, a user will be able to search anything that they have View permission for. For our installation, we don't want the users to be able to see content from other user's Inbox. The problem with this is that the user must have View Objects permission on the Inboxes in order to send reports to other users.
Do you know of a solution for this?
The result of the search should be subject to the security criteria, if you notice anything odd please open an incident with support providing details and reproducing steps
Thanks
Simone
Interesting that for the BI Launchpad, there doesn't appear to be an option to search folder names.
I have no issues searching folder names in the CMC, but logged in as an Administrator in BI Launchpad, there seems to be no option to search folder names.
Hey,
the indexing takes hours for my system. I want the opportunity that the users can search for documents. So I just need to index the title of a document and the path. Is that possible? With the index level ‘Platform Metadata’ I get heaps of information which I don’t need. Is it possible to index just the title and the path of a document?
Did you create a separate APS for platform search? Is your platform search location local to server or on SAN?
Is your platform settings is crawling or schedule?
Hey Jawahar,
i created a separate APS for the platform search and the location for the search is local.
The platform settings are scheduled.
Why don't you schedule during off peak hours? Do you have a lot of content in your system?
And also you can exclude Universe from the list of indexes? That will improve little bit.
Apparently there isn't an option just to search for document title and path. The Platform Metadata is the option that returns the least amount of information.
You may need to run the Platform Search in continuous mode until the base index is built. You can then schedule it to do updates. The collection process has been running in our Prod system for over 30 hours so far and it is still processing.
Simone,
We are a Very Large installation with over 140,000 objects for Platform search. The first time I built the Index, it took 14 days on BI 4.1 SP1 Patch 3. We are now on Patch 8. We do have a separate APS for it on servers with 64GB of memory and more CPU cores than we need. The only content I have selected is WebI and Workspaces. Other tools only take hours to catalog more content than just WebI. So, is there a known issue at our patch level and if so, what SP should we traget for the fix? Are there any known settings that may improve the speed of indexing on our current patch level?
Our surrent issue is that on Continuous Crawling, it still shows reports that were deleted weeks ago and the same for one that were moved to another folder (still shows old folder).
Jon
Deere&Co
Jon,
According to the Support Person that I have been working with on Platform Search, there is a bug with a query that selects the items that need to be indexed. The bug causes the query to time out. In our case, our Performance Manager didn't seem to index anything. It did eventually index about 20 documents per day.
"Platform Search sends a query to the CMS which fails due to the length of time it takes to run. There is a 9 minute timeout on the CMS and the query exceeds this timeout. This is mainly due to the amount of rows the query is attempting to return."
The Development Team is working on a fix for this but they don't know when it will be ready.
We are currently testing a work around that the Support Person recompiled and our index has been running since July 7th and as of this morning was up to 180623 objects.
We are on BI4.1 SP2 Patch 4.
End result, yes there is a bug with the process, they are working on a fix for it but they don't know yet when it will be ready.
It is good to know. Thanks for sharing.
Thanks. Tell us more about the recomplied work-around. Is that a new JAR file? Would it help to increase the 9 min CMS timeout to longer?
I don't think it is a totally new JAR file. The Support Person made some changes to it and recompiled specifically for our version of Business Objects. I did replace the file on our system with the one that he recompiled. After the initial collection is completed we will be switching back to the original JAR file. I know this is not intended to be a permanent fix.
It might help if that timeout could be increased in the CMS but I would hope the Support Person would have taken that route if it was feasible. Our timeout for the CMS is set at 20 minutes yet the query that Platform Search was running was timing out at 9 minutes.
John C.
If I recall, the 9 min timeout is set on the CMS command line as a RequestTimeout. It may also be a Registry Entry on the Server. Has nothing to do with the 20 min timeout setting in the CMC. We had to set this to run the UMT for migration from XI 3.1
Jon
Just wanted to know which server (service) is used when indexing mode is selected as 'Crawling' and which server (Service) is used when its 'Scheduling' mode?
Where is the index saved? In standalone and in clustered landscape.
What are recommendations for failover in a clustered landscape? Can we have AJSs and APSs running on multiple nodes with PS services?
Does indexing based recommendation fall under the same rule of having a common (master) index, similar to explorer indexing where there should be only one Master indexing server running in cluster? (both are different features and app components, but yet indexing as a concept is common)
Kind Regards,
Prakhar T
Hi Prakhar,
good questions indeed...
The only difference between crawling and scheduling is that the first is continuous, the second is scheduled at determined times. In crawling mode, whenever a change is detected, the Platform Search service will index what has been added or changed.
The APSs that are in charge of the indexing first save temp data locally and then those data about the index are consolidated in the location specified in the Platform Search application page properties, index location section.
Considering that the Platform Search service is not so vital, I don't think there is a need for failover, unless you need an extremely high accurate indexing...
However this one is my own personal take on this.
Thanks
Simone
what is the freqney we can set it for scheduling object to run?
Can we run it daily on off paek hours?
or once in week?
Hi Jagadish,
If you have set your Search Engine to run in a Scheduled Mode, you can control the schedule using the "Platform Search Scheduling Object" available in the platform.
This can be found in > Top Level Folder > Platform Search Scheduling
As this is using the "default" Job Engines from the Platform, you can control the start time of the job as well as it's frequency.. Under the "Platform Schedule Duration" Settings of the "Platform Search Scheduling Object" you can also control how long a schedule is allows to run
HI,
I have scheduled it to tun once in a week and allowed job to run for 10hrs over the offpeak hours(weekend)..It has ran and index got updated. Also found that we are not getting erros with DFS after set it to scheduled crawling.....
We also discovered that the default time it will try to run is 20 minutes. Then it will quit, claim it was successful, but produce no real results. For our environment, it takes about 14 days for this schedule to really complete. So, that setting depends on the size of your repository and search settings. Stick with Platform Metadata only and exclude Universes. You may also want to exclude PDF/Excel etc. I assume those are both instances and agnostic files uploaded to folders.
All,
The default setting for the Search Engine have been changed within BI4.1 SP05.. previously the default setting was the full / crawling mode. This has now been switched to Platform Metadata (the lowest level)
Hi Merlijn,
This is great new. Something which is easily missed when performing a migration which can but quite some stress on your source systems ;-).
With kind regards,
Martijn van Foeken
Hi Simone,
Nice write up.
Thanks,
BW
Curious if anyone knows what controls the performance of the search box itself in bi Launchpad? We have search on minimal settings, skipping instances, so the index is only 282mb. Yet the search box is still slow. For example, the predictive drop down often does not appear unless you wait a while for it to pop up. Any way to increase the threads/caching/connections? We have enough memory to cache the whole index in memory if that was an option.
Hi,
In our SAP BO system we would like to change the Platform Search configuration from Continuos Crawling to Scheduled Crawling.
We would like to know if this task can be invoked from another tool outside the CMC, that is, if it can be invoked from an application like control-m.
Thank you very much in advance
Greetings