In our previous two installments:
- FIM 2010- Build 4.0.3573.2 Performance Improvements, part 1 – I documented the base configuration of the Hyper-V environment
- FIM 2010- Build 4.0.3573.2 Performance Improvements, part 2 – I documented the configuration of the VM's, the two disk configurations and baseline results in build 4.0.3531.2 (RTM w/Update 1) to use as our comparison for the new hotfix build 4.0.3573.2
In this installment I'll show you how new features can be enabled in this build to improve your initial load times by a factor of 3.
Installation
To download the new hotfix, follow the link and file the request:
http://support.microsoft.com/kb/2417774
All of the features get a boost here and FIM CM gets a bunch of fixes that you can read about in the hotfix article (sorry Brian). The two we will focus on here are:
- FIMService_x64_KB2417774
- FIMSyncService_x64_KB2417774
Don't forget that there are updates to PCNS and the FIM Add-ins to deploy as well! As always, remember to take a good backup of your databases in case you need to roll these back. I didn't encounter any issues during the installation other than the fact you must manually stop the service before the installer can continue.
Case Bug Fixed
One of the significant, yet minor fixes resolves an issue where updates to string attributes would be marked as completed but fail to update. This happened whenever you were just changing the case of the string, and not the full value itself:
Case-only changes that are made to existing attributes are not applied to the FIM service database even though the Requests are marked as Completed.
I've tested this and it works, no more exported-change-not-reimported errors due to case updates!
Initial Load Performance Improvements
The new hotfix provides an asynchronous export mode from the FIM MA to the FIM Service. While in some configurations this could lead to a slow down in portal usability, if you have deployed your Service Partitions correctly then you should be pointing your FIM Sync to either a dedicated Admin Portal instance of the FIM Service, or a dedicated instance for FIM Sync as I do. However, I should note, that even in these situations, if you completely load up the SQL Server that is hosting the FIMService database, you will experience a slow down on all service partitions regardless.
There are two additions you can make to the configuration files in order to enable the new 'async' mode:
Microsoft.ResourceManagement.Service.exe.config
The resourceManagementService section has three new settings:
- synchronizationExportThrottle="Single" – Default mode, do nothing and you have the existing behavior of ~0.6 Exports/sec
- synchronizationExportThrottle="Unlimited" – New mode where each export is confirmed immediately allowing the next export to export right away. The exports to the FIM Service are cached and evaluated asynchronously
- synchronizationExportThrottle="Limited" requestRecoveryMaxPerMinute="60" – New mode allowing you to use the capabilities of the async mode but in a responsible manner so as not to degrade portal performance; use the second parameter to help gate this
There is an excellent note here I'll reiterate:
We expect that customers will have to optimize this setting to for their environment, and to accommodate their hardware capabilities and portal load. To tune this setting, monitor the FIM database SQL CPU usage and the Windows Workflow Foundation Workflows In Memory performance counters. Adjust the throttle up or down until you obtain a maximum throughput state. Example target metrics include SQL CPU usage of about 70% and Windows Workflow Foundation not building up a large queue in the Workflows in Memory performance counter.
This setting can be changed dynamically You do not have to re-start the FIM service.
It's nice that you don't have to bounce the service to get this change to go into effect, but I agree with Craig's Connect suggestion that these should have been Run Profile options which would allow us to call a predetermined configuration whenever we needed without resorting to changing the configuration file between runs.
miiserver.exe.config
In the FIM Sync configuration file we have two new lines you need to insert, but only if you wish to change the default behavior once either the "Unlimited" or "Limited" modes are enabled.
I didn't notice this line initially and I ended up with a nasty BAIL error when running any of the FIM MA run profiles, so make sure you insert this under the
In the main body of the configuration file you can insert a resourceSynchronizationClient tag now, the default of which is:
This allows you to tune the three parameters if you have a high-performance disk array:
| Property Name | Default Value | Description |
|---|---|---|
| exportFetchResultsPollingTimerInSeconds | 5 | When the Synchronization service is exporting objects in asynchronous mode, this property controls the frequency of polling results that are returned from the FIM service by the FIM MA. Changing this value may give a higher processing rate, depending on your system configuration. |
| exportRequestsInProcessMaximum | 50 | When the Synchronization service is exporting objects in asynchronous mode, this property controls how many requests can be queued up in the FIM service for processing. If this limit is met, FIM MA will wait until asynchronous results are sent back before resuming additional exports. Setting this value higher may provide additional processing throughput during export. However, during system failures, these objects may have to be re-exported from the synchronization engine when the FIM-Export process restarts. |
| exportWaitingForRequestsToProcessTimeoutInSeconds | 600 | This is the time-out value that FIM MA will use to wait for the FIM service to process a request. If no response is received from the FIM service within this time, FIM MA will end the export with a “cd-error” error. |
Now, with these settings you must restart the FIM Sync service in order to get them to apply.
Performance Tuning Results
I ran two tests, both with the new "Unlimited" switch enabled – the first was with the default Sync settings and the second I increased the values to see if I could eek out any additional performance; here are the results:
| 4.0.3573.2 Disk Configuration 2 synchronizationExportThrottle=" Unlimited" | 4.03573.2 Disk Configuration 2 exportFetchResultsPollingTimerInSeconds="15" exportRequestsInProcessMaximum="100" exportWaitingForRequestsToProcessTimeoutInSeconds="600" | |
| Records (8 attributes/record) | 11,251 | 11,251 |
| FIM MA Export Only Elapsed Time (mins) | 92 | 103 |
| FIM MA Objects Exported/sec | 2.031 | 1.818 |
| Processor Time - miiserver | 2.071 | 1.901 |
| Processor Time - fimservice | 2.13 | 1.89 |
| Processor Time – SQL | 66.231 | 60.545 |
| Logical Disk (SQL) - Average Disk Queue Length | 0.986 | 1.594 |
| Logical Disk (SQL) - Average Disk sec/Transfer (ms) | 7 | 11 |
| Objects Exported/sec Improvement Factor over Previous configuration | 2.97 | 0.90 |
| Elapsed Time improvement over previous configuration (mins) | 122 | -11 |
In the first test we can see a huge improvement – 3x over our best run with the previous build breaking 2 Objects Exported/sec and we came in 122 minutes under our prior time! Note that our disk latency and queue are now beginning to show the signs of another bottleneck. In both tests the CPU on the SQL Server was above 60% indicating there was still room to push the system but disk got in the way.
In the second test, I increased the default settings and we ran 11 minutes over reducing our Objects Exported/sec to 1.818 exposing disk as our bottleneck again with an Average Disk Queue Length of 1.594 and latency up to 11ms. Again, this is a home Hyper-V setting with desktop components, so a good Enterprise class deployment should be able to exceed these numbers. It's encouraging that the new asynchronous mode will let us stress the disk a bit more which seems to indicate that the caching can further expand performance on well tuned systems.
I would encourage everyone to start with the defaults and get a good grasp on what your overall disk performance is like so that you know when to back off of some of these settings. If you can keep your queue lengths to 1 or below then you should be at the right mark. In future tests, I hope to move some of the VHD's onto the SSD and see if I can eek out any more performance on this system.
0 comments:
Post a Comment