So what’s this blog all about?
An (really) important part of IDM is the provisioning to other systems. And it’s a part that loves to keep us on our toes, as the amount of threads and blogs about this topic show. Through my time working with IDM the provisioning stopped working quite a few times due to different reasons. And every step of the way I learned a new reason and a new solution to get it going again. To keep up with all of that and to help me solve it faster when it decides to get stuck yet again I started to write down a checklist.
Some time ago I posted the better part of my little checklist in a thread and through some encouragement by Matt I decided to create a blog post out of it to share the whole thing to a wider audience and explain the steps a bit more. This is my first technical blog here on SCN so I’m a little nervous and excited at the same time. ^^
ℹ Just to be clear: Not all of the points of the checklist might work for you, since we’re on a Windows server with the IDM management console and use an Oracle database.
- Access to the Management Console (MMC) of the IDM
- Access to the Monitoring-tab via http://<portalurl:port>/idm/admin
- SQL developer with database connection to IDM database
- Permission to access the Windows services of the IDM-server and to start/stop the services
- Permission to reboot the IDM-server
- Really good connections to the database administrators
How do I know it’s that time again?
There are three signs that I check if the provisioning is really stuck again:
- I look at the “Job log” in the MMC to see if the last entry for the provisioning-dispatchers is from more that 15 – 20 minutes ago (even through it was triggered in the last minutes).
- The provisioning queue on http://<portalurl:port>/idm/admin is only growing.
- The dispatchers, that are assigned to do the provisioning, are shown as running in the MMC under “Dispatchers > Status” and the timestamp for “Last check” is updated when I click on refresh.
If all those steps come back with a “yes”, I’ll get…
- Check the “Status”-overview in the MMC to see, if a job is showing the state “Error”.
- Restart the provisioning dispatchers in the “Services”-menu of the server.
- Check for waiting tasks via the SQL developer.
- Check the “Windows RT conn. string” on the Database-tab of the Identity Center configuration in the MMC.
- Reboot the server the MMC and dispatchers are installed on.
- Restart the IDM database.
That’s the checklist in short, if you just need a little reminder or an idea for what to look at next. I’ll explain the points a bit more in detail now.
1. Check the “Status”-overview in the MMC to see, if a job is showing the state “Error”
In the MMC you’ll find the “Status”-overview as the first entry under “Management”.
It shows all the jobs for that Identity Center connection (in this case it’s named IMP). To check for jobs that have the current state “Error”, just click on the column header “State” and it will be sorted by content. If you have checked the box “Show only enabled jobs” at the bottom of the page, the jobs with error-state should be shown in red font at the top or end of that list.
If you find a job that is associated with provisioning and it’s on “Error”, right-click on it and start it with “Run now”.
2. Restart the provisioning dispatchers in the “Services”-menu of the server.
Go to “Start > Administrative Tools > Services” on the IDM server and look for your dispatchers, that are assigned to the provisioning, in that list. They should be shown as started. Right-click on them and choose “Restart”.
ℹ This is what gets our provisioning going most of the time.
3. Check for waiting tasks via the SQL developer.
Open the SQL developer and work your way through the following SQL statements:
select * from mxp_provision where msg like 'Wait for%'
This checks for tasks, that wait for the successful state of other tasks. The MSG-column gives you an auditid for the next SQL-statement. It’s the ID of the task, that is blocking the execution of the first task.
select * from mxp_audit where auditid=<auditid>
The MSG-column now shows information about the state (e.g. failed) of the blocking task, the reason and for which user and which assigment. With these information you can decide to leave it alone to handle itself (because it’s got nothing to do with the blockage) or you can use the next SQL-statement.
update mxp_audit set provstatus=1100 where auditid = <auditid>;
This last statement sets the blocking task to the state “OK” (= 1100) and therefor the waiting task (the one you found with the first statement) can finally be executed.
4. Check the “Windows RT conn. string” on the Database-tab of the Identity Center configuration in the MMC.
When you click on the IC configuration (the “IMP” in the first screenshot), the “Database”-tab will be displayed. At the end of it you’ll find the string under “Runtime engine”.
Open it and test the connection on the “Connection”-tab. If it comes back as failed, correct the name of the data source and/or the logon information. Then test it again to see if it’s successful now.
5. Reboot the server the MMC and dispatchers are installed on.
Well, that’s pretty self-explanatory. ^^
If you don’t have permission to do this yourself, have the contact data of the administrator(s) at hand, who can reboot the server for you (just like with the restart of the services in #2).
6. Restart the IDM database.
This was only necessary once (until now), but to complete my checklist I’ll include it here, too. Since I don’t have direct access to our oracle database, I let our database administrators handle this one for me.
What’s more to say?
Well, that’s it! I hope the list can help you when your provisioning decides to take a break again. This is – of course – nowhere near a complete list, but a result of my experience with the issue.
If you have some tips of your own to add, I absolutly welcome it! As you know “Sharing is caring”. 😉 So leave a comment when you have your own little checklist for this issue or if you want to give some feedback for my blog (which I’m really looking forward to, because I am a big fan of constructive criticism and it is my first technical blog).
Thank you for your time and attention. I hope it was not wasted! *g*