Online RowStore reorganization cannot be executed when metadata store is not separated

This issue has popped on a full tiered environment that was upgraded revision 85.03 to revision 102.04.

The fix will be implemented in revision 102.06 and SP11.xx, however that doesn’t help anyone who isn’t planning on yet another upgrade.

The issue is in regards to two statements, which each a separate route cause in the same space.

Issue occurs when executing the following statements;

– ALTER SYSTEM RECLAIM DATAVOLUME 105 DEFRAGMENT;

ALTER SYSTEM RECLAIM DATA SPACE;

Running these statements could come up with the following error message:

online RowStore reorganization cannot be executed when metadata store is not separated”

CHECK_CATALOG Reports the Error ‘invalid address detected : not occupied slot’ classid : 39

First you run the following command to check the Referential Integrity

CALL CHECK_CATALOG (‘CHECK_OBJECT_REFERENTIAL_INTEGRITY’,,,)

This will in our case result in the following output:

/wp-content/uploads/2016/03/39_915155.png

The owner – OID cannot be traced back from the indexes due to the fact it is linked to the SAML certificates within the system. The missing metadata cannot be added and thus the SAML providers need to be dropped.


SAML Certificates

All SAML certificates that are placed in the SAML Identity Providers Tab within the Security Console prior to the upgrade are impacted.

/wp-content/uploads/2016/03/saml_915157.png

Solution – classid : 39

Unfortunately the solution is a nasty workaround for Production Systems as it means you need to break the SAML authentication and thus leaving your business temporarily without SSO via SAML authentication. Luckily the solution can be done very quickly if you are aligned with either the right authorizations or ensure you are working with a team that has all required authorizations to provide the fix. Execution of this is quick, but you need to ensure that on a Productive environment it is planned in an outage window not to upset the business.

  1. Make sure you have all SAML certificates from the environment at hand before starting to implement the fix!! You will need to implement the SAML certificates at the end to ensure a minimal downtime on SAML SSO.
  2. Prior to looking at dropping the SAML Certificates you will an updated list on the environment of all the users per SAML Provider (to be placed back after). List can be obtained via: select * from “SYS”.“SAML_USER_MAPPINGS”
  3. Drop the SAML Provider assignment from the users via SQL script.
  4. Only after all assignments have been dropped ensure to check if no user mapping is left before proceeding to the next step.
  5. Manually delete SAML providers manually from the security tab and make sure to execute after deletion.
  6. Now you can directly add back the SAML Providers and hit execute again.
  7. Set the user assignment back for SAML via SQL script.
  8. Now SAML authentication is back in place and users can continue to work.
  9. Run the CHECK_CATALOG statement again to ensure you no longer have classid: 39 present in the system.

    CALL CHECK_CATALOG (‘CHECK_OBJECT_REFERENTIAL_INTEGRITY’,,,)

   

CHECK_CATALOG Reports the Error ‘invalid address detected : not occupied slot’ classid : 31

First you run the following command to check the Referential Integrity

CALL CHECK_CATALOG (‘CHECK_OBJECT_REFERENTIAL_INTEGRITY’,,,)

This will in our case result in the following output:

/wp-content/uploads/2016/03/31_915183.png

Trace back the index to the source

Execute the following command based on the owner ID that can be found in the error message when doing the CHECK_CATALOG.


Example in this case:

select * from indexes

  where index_OID = 452341 or index_OID = 452343 or Index_OID = 452345

/wp-content/uploads/2016/03/index_915185.png

Now you see the issue lies with the FULLTEXT indexes on certain schema’s with their associated table names.

Solution – classid : 31

You need to ensure you find the right table name and schema name and double check the current index in place to ensure you recreate this in the same way it is set now! This means that all parameter should be present again. *TIP*, you can find the full statement from looking at the SQL of the particular table, which ensures you don’t leave any relevant information out.

Once you identify the full name of the FULLTEXT INDEX you can create the drop statements. Example below:

DROP FULLTEXT INDEX “_SYS_REPO”.“FTI_ACTIVE_OBJECT_TEXT_CONTENT_CONTENT”;

DROP FULLTEXT INDEX “_SYS_REPO”.“FTI_ACTIVE_OBJECT_CDATA”;

DROP FULLTEXT INDEX “_SYS_REPO”.“FTI_ACTIVE_CONTENT_TEXT_CONTENT_CONTENT”;


After the drop has been successfully executed you can directly run the create statements to place the indexes back into their former glory, example below:


CREATE FULLTEXT INDEX “FTI_ACTIVE_CONTENT_TEXT_CONTENT_CONTENT” ON “_SYS_REPO”.“ACTIVE_CONTENT_TEXT_CONTENT” (“CONTENT”) LANGUAGE COLUMN “LANG” ASYNC LANGUAGE DETECTION (‘EN’) PHRASE INDEX RATIO 0.000000 FUZZY SEARCH INDEX ON SEARCH ONLY OFF FAST PREPROCESS OFF TEXT MINING OFF TEXT ANALYSIS OFF TOKEN SEPARATORS ‘\/;,.:-_()[]<>!?*@+{}=”&’;

CREATE FULLTEXT INDEX “FTI_ACTIVE_OBJECT_CDATA” ON “_SYS_REPO”.“ACTIVE_OBJECT” (“CDATA”) ASYNC LANGUAGE DETECTION (‘EN’) PHRASE INDEX RATIO 0.000000 FUZZY SEARCH INDEX ON SEARCH ONLY OFF FAST PREPROCESS OFF TEXT MINING OFF TEXT ANALYSIS OFF TOKEN SEPARATORS ‘\/;,.:-_()[]<>!?*@+{}=”&’;

CREATE FULLTEXT INDEX “FTI_ACTIVE_OBJECT_TEXT_CONTENT_CONTENT” ON “_SYS_REPO”.“ACTIVE_OBJECT_TEXT_CONTENT” (“CONTENT”) LANGUAGE COLUMN “LANG” ASYNC LANGUAGE DETECTION (‘EN’) PHRASE INDEX RATIO 0.000000 FUZZY SEARCH INDEX ON SEARCH ONLY OFF FAST PREPROCESS OFF TEXT MINING OFF TEXT ANALYSIS OFF TOKEN SEPARATORS ‘\/;,.:-_()[]<>!?*@+{}=”&’;


After the drop and create have been successfully executed you can rerun the CHECK_CATALOG statement and ensure that you have no pending items for classid: 31 left in the system.

CALL CHECK_CATALOG (‘CHECK_OBJECT_REFERENTIAL_INTEGRITY’,,,);

If all classid 31 and 39 errors have been cleared you can now run the rowstore reorg and reclaim data space statements again as expected.

Reference Notes

2290385 – CHECK_CATALOG Reports the Error ‘invalid address detected : not occupied slot’ classid: 31

2281574 – CHECK_CATALOG Reports the Error ‘invalid address detected : not occupied slot’ classid: 39

Occurs when?

CALL CHECK_CATALOG (‘CHECK_OBJECT_REFERENTIAL_INTEGRITY’,,,)

Known impact

ALTER SYSTEM RECLAIM DATAVOLUME 105 DEFRAGMENT;

ALTER SYSTEM RECLAIM DATA SPACE;


Known errors

online RowStore reorganization cannot be executed when metadata store is not separated”


Error Code

2576 Invalid address detected: not occupied slot


Known issue

Metadata issues in the database that can occur after an upgrade of your HANA environment. This example is based on a revision upgrade from revision 85.03 to revision 102.04



To report this post you need to login first.

Be the first to leave a comment

You must be Logged on to comment or reply to a post.

Leave a Reply