VMware

Datastore Filled Up from Old Snapshot, VM unresponsive

In ESXI 6.5, I made a huge noob mistake and left an old snapshot that grew in size to the capacity of the datastore of 9TB in size. The datastore was full as a result of this. This prevented me from running the virtual machine and causing it to power off automatically. The datastore does not have any other VMs or files associated with it except for the culprit VM. I was able to increase the capacity of the datastore by about 5TB and am trying to delete the base level snapshot, but googling has stated that deleting snapshots will require free space equal to the size of the snapshot being deleted due to the data from the delta disks being written to the parent disks because of how snapshots work. The deletion process is currently hanging at 0% right now, so my hopes for this to work out is low.

Is there any workaround to free up the space without installing a set of drives to create a larger datastore and migrating the VM to that larger datastore and deleting the snapshots? I currently do not have the ability to do that right now. There are multiple other datastores with data on them that add up to about 4-5TB of space. I don’t have too much knowledge of Vsphere/ESXI. TIA


View Reddit by sunderblundersView Source

Related Articles

10 Comments

  1. Clone it.

    FYI don’t t power it off if you remove the snapshot. If you start removing the snap with the VM powered off you can’t turn it back on until it’s finished.

    Power it down and clone it. Just need the same space the original VM consumed.

    Other option is to back it up with vsphere replication appliance. This can be done while it’s powered on, then cut it over to the replica. It will power the VM off and copy the deltas. Should be low downtime.

    I think vsphere replication appliance is free.

    Edit: just read you already started the removal. Once it’s complete check you have no disconnected snapshots.

  2. There’s no need for the same space again in free space for the snapshot to commit – but it will take quite some time.
    There’s a VMware KB article which explains how to check (on the command line) how a snapshot delete is proceeding – that will give you some idea if something is actually going on.

    Edited to add – On second thoughts, the above was assuming your VM has thick provisioned disks. To give a better idea, we need to know the following:
    – Are the VM disks thick or thin provisioned?
    – How large are the VMs disks?

  3. Do you have the means to add more space to the ESX?
    I got out off a no space jam like this by adding local disk to a host and created a datastore on it. Yeah it was redududant, but it wasn’t there for long.

  4. Delete the VM and restore it from your latest backup.

    Alternatively, clone it, if that’s workable.

    Or since you say it’s just a file server – stand up a new file server VM and dump the content over and delete the 9TB bloatmonster.

  5. Just a heads up… if you recently updated vmtools from the HTML web client the default that’s hidden away is to take a snapshot before the install and keep it. I don’t recall at the moment where but there is a locations to change that default behavior. The flash web client does not have that same default. Source: I got burnt on this one.

  6. I believe you will need nearly the same space as the snapshot, but I cannot find supporting evidence.

    I will tell you that the snapshot will take many hours to delete if it is 9TB in size. I’d highly recommend that VM remain shut down while you wait for that snapshot to be deleted. Is this VM running SQL? Are VMware Tools installed on the VM?

Leave a Reply

Your email address will not be published. Required fields are marked *

Close