Don’t worry, this story has a happy ending…finally.
I have been working with one of our customers to get them migrated out of our old Unidesk environment, into the new Horizon 6 environment. We provided them a template machine cloned off of our Gold Master. This is one of two pools that will not be managed with App Volumes and Mirage simply because of the issues we’ve had with this users pool in the past. I am not going to get into all that at this time.
They completed their app configuration, and tested it. They certified the template. I then went in and completed the final steps we normally go through for the template. I verified VMtools and the Horizon agent, cleaned up some temp files, then ran the ClonePrep tool from Symantec to clear all the SID’s (this allows for a clean Symantec SID so that you don’t have issues joining the master server. More can be found on that here. I then shut the machine down and created the snapshot. Easy right?
Once that was all set, I setup the pool and pushed the provisioning. Everything was going great. Composer was doing its thing. Everything was great until we got to the Sysprep customization. It just sat there. And sat there. And sat there. Checked the console, and they still had the template name. Never rebooted.
At this point I assumed something was broken. I started doing some digging. Normally, I don’t disjoin the template from the domain and it doesn’t give issues. I also normally don’t release the IP on the template. Also, normally no issues. I tried all of the following:
1. Disjoin from domain
2. Release the IP
3. Apply MS hotfix KB2550978 for VMXNET3
4. Created a new Sysprep config in vSphere
None of these fixes worked at all. I tried to create a clone of the VM straight from vSphere and that didn’t work. This lead to thinking it was a Microsoft issue. It was time to look at Sysprep itself. The log files can be found:
The errors in the Sysprep logs stated:
failed to query pending cbs operations
A quick search on this lead me to a registry key fix:
Value: 0xffffff (4294967295) 2) Reboot
3) As a Administrator open a Command Prompt and run “SFC /SCANNOW”. The command should complete successfully and if any errors were found, they should be corrected.
After the reboot, I tried the vSphere clone again. Still failing. Though, this time, I had a much different set of errors.
SYSPRP RunExternalDlls:Not running DLLs; either the machine is in an invalid state or we couldn’t update the recorded state
SYSPRP WinMain:Hit failure while processing sysprep cleanup external providers
I did some searching on this, and found some terrible news. All of the “accepted answers” from MS or from others, was that the image was toast and needed to be rebuilt. This was just simply not acceptable. I could not have this user build their whole image again. I read just about every post on every search result I could find. I finally found someone saying that if the machine you were trying to sysprep, was in fact sysprepped itself to be created, that could cause an issue.
Bingo. We clone all our templates off the gold master, and sysprep them to give them the DNS name, etc. The following steps cleared out the Sysprep files and set the re-arm skip to 1, allowing Sysprep to once again be run off the machine:
1. Navigate to %WINDIR%\System32\Sysprep\Panther
2. Delete or Move / Rename all of the files in the Panther directory
3. Open RegEdit
4. Navigate to HKEY_LOCAL_MACHINE\SYSTEM\Setup\Status\SysprepStatus\
5. Change the GeneralizationState key to 7
6. Navigate to HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion\SoftwareProtectionPlatform\
7. Change the SkipRearm key to 1
8. Complete template finalization steps
9. Shutdown / Snapshot
Voila! Sysprep ran like a dream. First on the vSphere clone, then on the recompose of the busted pool. That was a rough one.
Hopefully this doesn’t happen to you, but if it does, never fear.