You know your SBS 2008 server is having a bad day when…


So, a couple of days ago I’m in the office when a client calls. Nothing works they say, and when they checked their server they couldn’t log into it.

Small office, and everything runs on a single SBS box. “OK”, I ask. “What happens when you try to log on?” “We can’t,” they say, “the screen looks like the Matrix.”

Well, that’s new. I obviously can’t get anywhere further with this conversation, and no remote tools are connecting to the server, so I jump in the car and head to the customer’s office.

When I get there, I am greeted with this screen:

Hm. Well.

Keyboard entries and mouse movements had no effect, so I power cycled the server. Upon reboot, the server demonstrated the same behavior. I swapped out the monitor to verify that this was a server issue, and not a monitor or video connection issue. There was no change. So as not to have to start testing theories in front of the customer, I returned to the office with the server and the (hopefully unnecessary) restoration media.

Upon return to the office, the server was able to power on and provide meaningful information through the monitor. Perhaps letting the capacitors discharge during the drive fixed that problem.

However, now while the server would post, it would not boot to the operating system. The error changed nearly every time, but the common denominator was an inaccessable boot device, due to a missing or currupt Silicon Image 3124 SATA driver – error 0xc000221. (Although occasionally it reported missing winload.exe and error 0xc0000225). Apparently Windows Updates had been applied the night before, and this would have been the first attempted restart after the installation.

The restore media would only take us back to 6 days prior (what luck) so I made the decision to attempt to repair the existing install as opposed to restoring from backup – even though this would likely take longer.

A repair using the install media did not work, nor did startup repair.

I tried to copy in a SATA driver into the OS by using the command line after booting to the install media DVD, and that did not work. A colleague was able to find a different version of the SATA driver, and when I repeated the attempt with that driver, I was able to get the server to boot into the OS.

Yay! I’m done! Boot, log in, test, and then return to customer, right? Of course nothing is that easy…

Once the OS boot cycle started, it attempted to apply the updates from the previous reboot cycle. The server would get to the third phase, then hang, as seen here:

More problems? Really? 45 minutes later I gave up on this and power cycled again. Again it booted, and again it hung in the same spot. Time to break out the Google-fu…

Apparently others have seen this issue before.

I performed the following process as per http://social.technet.microsoft.com/Forums/en-US/winserverhyperv/thread/106471aa-cc0e-40c8-b5c1-d863bb8e2aed :

1.  Insert the Windows Server installation disc in the disc drive, and then restart the computer.

2.  When you are prompted to restart from the disc, press any key.

3.  When you are prompted, configure the Language to install, Time and currency format , and Keyboard or input method options that you want, and then click Next .

4.  On the Install Windows page, click Repair your computer .

5.  On the System Recovery Options page, click the version of the Windows Server operating system that you want to repair, and then click Next .

Use the System Recovery options to rename the Pending.xml file and edit the registry : 1.  On the System Recovery Options page, click Command Prompt .

2.  Type cd C:\windows\winsxs , and then press ENTER.

3.  Type ren pending.xml pending.old , and then press ENTER.

4.  Type regedit , and then press ENTER.

5.  Select HKEY_LOCAL_MACHINE .

6.  On the File menu, click Load Hive .

7.  Locate the following folder: C:\windows\system32\config\components

8.  When you are prompted for a name, type Offline_Components .

9.  In Registry Editor, locate and then delete the following registry subkey:

HLKM\Offline Components\AdvancedInstallersNeedResolving

HKLM\Offline Components\PendingXmlIdentifier

Exit Registry Editor :

9.1.  At the command prompt, type exit to exit Registry Editor.

9.2.  Press ENTER.

9.3.  Click Restart .

That had the server get a little further, but then it would hang again.

After reading other posts about the issue, I found a mention that SBS may hang when installing updates if there was no NIC connected. Oops. I knew that. Although that was not an original issue, there was no NIC connected now that I had the server set up in the office. I connected a NIC, and power cycled yet again. After about 30 minutes of applying updates it finally gave me a logon screen.

I booted again to verify that it actually would boot again, and then returned it to the customer. While on site I tested with all users, and also made some changes to the SSL configuration of the server to overcome an issue plaguing their lone Mac user.

When I left everything was tested and functioning normally. My lesson for the day was a reminder to always have the NIC plugged in on a Windows server. You never know what dependencies you may break if no network connectivity is present – even if only from the server back to itself.

This outage lasted about 5 hours from the customer’s perspective. I’m a big fan of SBS. Just like the Mac Mini Server, I think it provides a price appropriate, all-in-one solution for small offices. However, this episode highlights the issue of having all services on one box.

In a perfect world, I would have had a slightly beefier physical server, and then run several Windows servers in VMware. If the customer had 2 domain controllers, and Exchange server, and a MSSQL / SharePoint server on separate virtual machines, any one could have had a failure after applying updates (or for any other reason), and while they would have been impacted, they would not have been completely down. Also, VMs would allow snapshot backups, so if necessary we could roll any server back to the prior day. One final benefit – you would no longer need special installer tools to apply service packs to allow for the idiosyncracies created by running so many server apps on one Windows instance.

I’d love to see Microsoft price SBS as a handful of VMs on a single piece of hardware as opposed to running all in one Windows instance. I believe their customer base would be better served.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s