Wednesday, October 16, 2013

vCenter 5.5 upgrade or fresh install failure

I ran into an issue today when trying to upgrade vCenter 5.1 to 5.5 where every time the SSO MSI installer tried to perform the upgrade it would  choke, fail, and then rollback. After searching the logs, looking at the MSI output and removing all of the vCenter services I was still stumped.

Typically with errors like this I am able to figure the problem out, or if I spend enough time asking Doctor Google he provides me some good leads which help push me in the direction of a solution. However in this specific situation no amount of googling could save me and I was on my own.

I decided to first clean up the previous installation by removing all of the directories, registry values, and hidden files in TMP and %PROGRAMDATA% and then rebooted the server.

With the reboot complete and my Windows 2008 R2 server looking so "fresh and so clean clean" I decided to give the vCenter installation another try.

Once again, I was met with failure and no amount of reviewing logs was going to help this time around. DRAT!

Feeling dismayed as I had now been "punked" multiple times with this installation I decided it was time to reach out and use my lifeline to call some help. A few calls later and I had my A-Team ready to take a look so I fired up a WEBEX and we all joined in. I admit, I was feeling pretty good at this point and was ready to "slay the dragon" aka get vCenter 5.5 installation completed.

In the end, even the best and brightest VMware VCAP and VCDX engineers were all beaten and I accepted my fate and and called in support from VMware on a SEV1 as they surely could get it fixed.

The engineer and I looked at everything over as period of 3 hours. Reviewing logs, the installers, trying to ascertain some reason as to why the installation was failing but we were all dumbfounded. VMware support grabbed a few other engineers and together we step by step walked through the installation installing all of the prerequisite components individually in their respective order but when it came time to get the SSO MSI to knock out the instal lit failed yet again.

Finally, one very smart engineer decided to look take a peek at DNS records. We did a forward and reverse lookup of the server via nslookup and everything looked fine, but then he noticed there was a stale and incorrectly formatted hostfile record that was pointing to the WRONG ip address.

A quick comment of the hostfile record, and everything installed without any issues! THANK GOD!

In the end, it just goes to show you, no matter how many times you have installed vCenter or the vSphere suite, there is always some little gotcha that can sneak up on you and ruin your day. SIGH!

Big thanks to VMware support for talking me off the cliff when I couldn't get this installation completed and for thinking outside of the box and forcing us all to take things step by step and validate all prerequisites. Goes to show you, no matter how many VMware VCDX and VCAP engineer you know, sometimes you really need a fresh set of eyes and an open mind.

No comments:

Post a Comment