Converting AdminTools Data into a Reportable Format
Background and Purpose
If you are using Business Objects 4.2 SP3 or higher, you have the option of using the CMS data access driver to query your metadata. If however you are running an older version of the platform, your best bet for easily querying metadata are the AdminTools.
Unfortunately, the output of AdminTools does not lend itself well to scale-able reporting, so without taking additional steps the utility of that application is limited to looking only at a few records at a time. While that metadata is also exposed through the Business Objects SDK, the downside there in the 3.X environment is that you would additionally need to configure Eclipse, Tomcat and roll some of your own Java code to get to that same information, all of which can be fairly involved.
Rather than using the SDK, this guide will show you how you can use the output of the AdminTools data and leverage Python to convert it to a more digestible reporting format.
Pre-Requisites and Installations
Some background knowledge of Python would be beneficial, although it should not be required for this guide. To make things as simple as possible, you should download Python 3.4 or higher as it will contain the pip tool to help install and maintain the required package. You may be able to use older versions of Python and install pip separately, but walking you through that process is out of the scope of this document.
Once you have Python installed, you’ll want to download the BOATParser package available on GitHub. If you have Git installed, you can clone the repository via “git clone firstname.lastname@example.org:WillAyd/BOATParser.git” – otherwise simply download the package and unzip.
Using the command line, navigate to the directory where those files are located. You first need to build a source distribution, by running the following command.
python setup.py sdist
This will create a subfolder “dist” containing the installable file. Next you will use pip to install the package as follows, replacing VERSION_NUM with whichever version of the parser you have downloaded. Pip will take care of installing all of the dependent packages for you.
pip install dist/BOATParser-<VERSION_NUM>.tar.gz
Using the BOATParser
With everything installed, the first step is to run the appropriate AdminTools query in your Business Objects environment (for a good guide on how to do this, please refer to the Business Objects Query Builder – Basics blog entry on SDN).
Once you’ve executed the desired query, the output should look similar to the following screenshot:
At this point you need to save the output locally to your file system. This will depend on the browser you are using, but typically you should be able to do a “File” > “Save As” and save only the page source / HTML to your computer.
Using the command line, navigate to the directory where you saved the output of the AdminTools query. Open up an interactive Python shell (typically you can do this by just typing “python” on the command line, assuming Python is installed on your PATH). Once the interactive session has started, import the BOAdminToolsParser class from the BOATParser package you installed earlier, and instantiate an instance of the class.
from BOATParser import BOAdminToolsParser bp = BOAdminToolsParser()
Optionally you can call “help(bp)” to see the methods and docstrings available to you with the BOAdminToolsParser.
In the screenshot above you’ll notice the BOAdminToolsParser class provides a “frame_from_file” method. This is the first method you should call after instantiating the object, providing the location of your file as an argument. In the below screenshot, we are reading in the “test.html” file and assigning it to the “df” variable – be sure to replace “test.html” with the name or location of your file.
df = bp.frame_from_file("test.html")
The “frame_from_file” method returns a DataFrame – a tabular structure containing the output of your AdminTools query. To see this, go ahead and run the “head” method of your data frame.
Each report from the AdminTools output gets mapped to a unique row and every property gets mapped to a column and value. For properties that contain subtables (like SI_PATH from our example) the value returned will be a dictionary containing key/value pairs for every entry within that subtable.
The other method called out in the help screenshot is “expand_paths.” Provided a DataFrame with SI_PATH and SI_NAME columns, this will return a Series of normalized folder paths that can help you more easily identify where content is stored within your repository. Below we’ll map this to a column called “expanded_path” within your DataFrame, although you can choose whichever column name you would like.
df['expanded_path'] = bp.expand_paths(df)
The last step here will be to export your data to CSV.
Open your CSV file and you’ll see all of the information you pulled out of AdminTools, but in a nicer format that can be shared and more easily analyzed by others within your organization.
Suggested Further Reading
While this document has shown you the basics of converting your AdminTools output into a more reportable CSV format, you may find value in diving deeper into Python and especially the pandas package. Pandas provides plenty of utilities to analyze and transform your data, by notably allowing for SQL-like joins. If you run multiple queries against AdminTools to inspect various types of objects (ex: one query to pull universes and another to pull reports) you could use the BOATParser to create two distinct DataFrames and merge them together directly in a Python script, rather than having to export to other tools like Excel and perform the lookups from there. Pandas also supports other output formats besides csv (excel, html, text, etc…).
I hope you have found this blog post to be of value. If you run into any bugs while using the BOATParser, please be sure to open it as an issue on GitHub so that it may be addressed. If you are a Python developer, merge requests are welcome and much appreciated.