Recover a Multiplex with DAS Dbspaces when you have an Out of Space Condition in IQ_SYSTEM_MAIN

If you encounter an out of space error in IQ_SYSTEM_MAIN which brings down your coordinator node, the normal recovery procedure can appear blocked if your multiplex has a DAS dbspace. This blog shows you how to work around the issue, and bring your multiplex back online.

Issue:

You have a multiplex with at least one DAS dbspace. The coordinator node fails with an error indicating an out of space condition in IQ_SYSTEM_MAIN.

Solution:

Run sp_iqcheckdb in dropleaks mode twice: first against IQ_SYSTEM_MAIN, and then against your DAS dbspace:

1. Shut down all multiplex nodes
2. Start the coordinator node with these three start_iq parameters:

   ◦Single-node (-iqmpx_sn)

   ◦Override (-iqmpx_ov)

   ◦Forced recovery (-iqfrec)

   Example:

   start_iq -n mpx_abc123_m -x tcpip'{port=9612}’ -iqmpx_sn 1  -iqmpx_ov 1 -iqfrec mpx_abc123.db /machinedqp- c2/users/your_name/sp12/abc/mpx/m/mpx_abc123.db

3. Connect to the coordinator and run sp_iqcheckdb in dropleaks mode with IQ_SYSTEM_MAIN as the dbspace target:

   sp_iqcheckdb (‘dropleaks dbspace IQ_SYSTEM_MAIN’); checkpoint;
   

4. Shut down the coordinator.
5. Restart the coordinator normally without the -iqmpx_sn, -iqmpx_ov, and -iqfrec parameters.
6. Connect to the coordinator and run sp_iqcheckdb in dropleaks mode with your DAS Dbspace as the dbspace target.

   Example:

   sp_iqcheckdb (‘dropleaks dbspace ‘); checkpoint;

7. Run sp_iqcheckdb in dropleaks mode for any other user dbspaces.
8. Bring your DAS dbspace back online.
9. Synchronize all multiplex secondary nodes.