Friday, July 5, 2013

Be Careful upgrading Hyper-V Integration Services. Can Break Exchange 2007

Scenario:
We are running a hosting environment of Windows 2012 Hyper-V, with Virtual Servers running Windows 2008 R2.  In this environment, we also are running Exchange 2007.

If you examine the status of a VM in the Hyper-V manager, you may notice that the Integration Services is running in a "degraded" state and is not fully functional.  Example: it does not show the IP address of the VM.
Hyper-V Manager on a Windows 2012 showing the Networking status of a VM which has the original version of Integration Services in a Windows 2008 R2 Server, out of the box.

Note that the version of Integration Services that comes in Windows 2008 R1 is reported as version 6.1.  Hyper-V 2012 would have you install version 6.2.

Well, I elected to install the newer version by simply clicking the menu to select to install the new version from the mounted disk.


The Problem:
A reboot was required to complete the Integration Services upgrade.
After the reboot, Exchange would not mount one of the databases.

The error was, in part, in the event log said "Error -546, unable to read the header of log file E01.log".
Also, when a repair is attempted, "JET_errorLogSectorSizeMismatch, -546".


An examination of the database that would not mount showed that it was in a Dirty-Shutdown status.

All attempts to run eseutil to repair or cleanup the problem resulted in the above errors.  Error -546.

The Solution:
We theorized that the Integration Services upgrade had caused the problem, but how to confirm this? 
Easy:  we shut down the VM, mounted the disks temporarily on a VM of the same OS but with the original Integration Services, and low and behold!  We were able to easily run the eseutil commands to clean up the database and return it to a "Clean Shutdown" state.

Root Cause:
See this article regarding disk geometry, and you will quickly see that there is an opertunity to create a disk on new OS's, that is not compatible with some older applications, such as dare I say, Exchange 2007.
Now this is not a problem unless you have a database that is in a "Dirty-Shutdown" state and it needs files that were saved using the "old" disk geometry, but reside on the new Geometry. 

See http://support.microsoft.com/kb/2510009

It appears that Exchange (at least version 2007 ) is one of the "incompatible applications" mentioned in the above link.

My advice?  Do not upgrade the Integration Services unless you have tested EVERYTHING.  Don't assume that it will all be ok.

Additional References:
http://blogs.technet.com/b/exchange/archive/2013/04/24/exchange-2010-database-availability-groups-and-disk-sector-sizes.aspx

1 comment:

comnam90 said...

I've just run into this.
Were you able to resolve things by using ESEUTIL to repair the DBs on an external server and then re-mount them on the the original Exchange 2007 server with integration services updated?