Installing the WSS 3.0 Beta in a VPC after NewSID

Bill Sheldon — Mon, 22 May 2006 16:32:00 GMT

So you are trying to install the Beta Windows Sharepoint Services 2007 in a Virtual PC. The install seemed to run fine, and you’ve started the configuration process and you have just gotten the error: Exception: System.InvalidOperationException: This access control list is not in canonical form and therefore cannot be modified

There isn’t much documentation on this but from what I can tell unless you follow the unofficial solution below it is a show stopper. The cause, well if you tried to save time by having a VPC image preset with Windows Server 2003 and just copied that image and then ran something like NewSID on that image you got this error. Something in the SID reset logic isn’t clean enough and the Sharepoint installation picks up on the SID issue and doesn’t actually create the correct security settings and corrupts your registry in a minor way. Since I hate the thought of needing to forever wait through the Windows Server installation process, I wanted a solution to the problem.

I found a few hints that talked about things like making sure you have the latest version of NewSID and or running the Sysprep.exe utility that ships as an optional tool on the Windows Server DVD. The fact is I attempted several different combinations of these tools in an attempt to resolve the configuration errors and they don’t clean up the issue, which brought me to digging through the actual details of the error.

The good news is that the configuration wizard for WSS can be run repeatedly so I could debug each of the issues that cause this error in Step 4 of the configuration process.

The first issue is fairly simple. It occurs in your systems Users and Groups. WSS creates three new security groups: WSS_ADMIN_WPG, WSS_RESTRICTED_WPG, and WSS_WPG. Only the third of these is corrupt.

The WSS_WPG group in my experience with this error is missing it’s members the LOCAL SERVICE and NETWORK SERVICE accounts need to be part of this security group.

The next two issues both involve registry keys and require you to use the Run command to open “Regedit” and hack on your registry.

The first key at issue is located in the HKEY_CLASSES_ROOT hive. (Note this isn’t the order the errors appear but my choice for editing.)

From within this hive go to the AppID key. Within the AppID key find the key {6002D29F-1366-4523-88C1-56D59BFEF8CB}.

Right click on the key (starting with 6002) and select “Permissions…” from the context menu. At this point you should get a warning dialog indicating that this key’s permissions are corrupt. Say OK to the error message and go into the permissions dialog.

That dialog will look similar to this one:

Notice the “Advanced” button. Select it to get to the dialog you actually need:

On this dialog you’ll note that my example shows a set of non-inherited permissions, it’s this list that has some problems. You basically need to Add LOCAL SERVICE, NETWORK SERVICE and WSS_ADMIN_WPG as having full control on this key and sub keys. Additionally the SYSTEM and Administrators permissions should be for this key and all sub keys and any references to any User or Power User specific permissions that are not inherited should be deleted.

Once you have updated the first key it’s time to move to HKEY_LOCAL_MACHINE to get to the second set of keys. These keys are under Software\Microsoft\Shared Tools\Web Server Extensions\12.0.

The WSS Key under the above path needs it’s permissions corrected, and when you right click it and select “Permissions…” you will again be warned that it’s permission hierarchy is corrupt. Say ‘OK’ again and head in as above. In this case on the second screen take a moment and select “Replace permission entries on all child objects with entries shown here that apply to child objects”.

Next delete any pre-existing permissions which are not inherited and then add a new permission for WSS_WPG. In this case when adding the permission you do not use full control instead use the settings shown in the screen shot below. Once this has been added “Apply” these changes and return to Regedit.

Next go to the key “ComponentsToRegister” located below the WSS key you were just in. Again right click and go to the advanced screen. This time you need to add WSS_ADMIN_WPG to the list of those granted permission and then set it’s special permissions with the screen shown below.

At this point you can close regedit and restart the Sharepoint configuration wizard. It should run past steps 1-3 very quickly and then successfully get through step 4. Once step 4 is complete the remaining 6 steps should complete without issue.

For me this is a quicker alternative to building a new image for each time I reinstall WSS. Now on to my next task, adding the Office Sharepoint Server (OSS) to the mix. I’ll post if it has any issues of its own.

SharePoint 3.0

Installing the WSS 3.0 Beta in a VPC after NewSID