70GB Snapshot, YIKES!

Thursday, May 15, 2008

This week I ran into a 70 GB snapshot head on while upgrading an ESX host. The 70 GB snapshot was the result of a snap of a 100 GB SQL server VM that had been left unchecked for about a year. Surprisingly, I run into this situation a lot although, not usually so massive. People still seem to be mystified by the correct usage of snapshots and seem to use them as one time backups to be saved indefinitely. This inevitably becomes a huge untamable problem. This particular situation had numerous Virtual Machines with several GB snapshots. The 70 GB one was just the worst of the bunch. I had tried to commit several smaller ones but soon became nervous about data corruption of the VMDK files. VMware Converter to the rescue!

I decided that rather than deal with the delicate nature of committing these huge snapshots with just my fingers crossed, I would scrap them all together.

I created an ISO of VMware Converter and uploaded it to my ESX Hosts and booted each of the affected Virtual Machines into the WinPE environment as if they were physical machines. From there, I just 'converted' them into new VMs in VirtualCenter. The process was very fast – a 20 GB VM took about 15 minutes and the result was an identical Virtual Machine without any troublesome snapshots associated with it. I did have to remove the USB drive virtual hardware that was added to the newly created VM. (I wish there was a way to select the virtual hardware for a P2V destination during the process to avoid these extra steps)

A couple of fast conversions later and the systems were in a stable state again and all the Production Virtual Machines were snapshot free!

Moral of the story: Keep your snapshots for a week or two and then Commit or Discard them!

Next Post »