How to unlock a vmdk file without a host reboot

I ran into a situation recently with a client that had an Exchange server VM that would not start due to a locked VMDK file on the ESX host. I am writing this post to remind myself and others how to unlock a VMDK (or any open file) on an ESX (with Service Console) host without rebooting the host. Why couldn’t I reboot the host? Here’s the story.

I was called in to help a client with an ESX 3.5 host failure in their environment. HA had kicked in, but they only had one host available (with multiple VMs already running on it) and the Exchange server VM would not boot. I got the Exchange server to boot, but that is a story for a different post . When the Exchange VM booted, I discovered that the system volume (C:) was nearly out of space. So, the Exchange Transport service would not start. So, I used VMware Converter to convert the existing Exchange VM to a new VM on the same ESX host with a larger system volume (C:).

However, there was a catch. I couldn’t convert the system volume (C:) and the data drive for the Exchange VM. There was not enough space on the datastore. So, I just converted the system volume (C:) and re-sized it accordingly. Then, the plan was to move the data drive into the newly converted VM’s folder and attach it to the VM. It would have worked too, if it wasn’t for that meddling open file lock. I got the old “Cannot start <VMName> because there is a lock on a file” error when I tried to start the new VM.

I verified that the original Exchange VM was powered off, so I figured it must be a hung process that had the VMDK file still open. Here is how you can find and kill the process.

• Log on to the Service Console of your ESX host.
• Use the Linux lsof (list open files) command along with grep to find the VM process that is responsible for locking your VMDK file.
  • lsof |grep vmname (Where vmname is the name of the VM in question.)
• Then, kill any processes that have your VMDK file open with the kill -9 pid command (Where pid is the process ID of the offending process).

That’s it! Next time you find yourself needing to kill a process on your ESX host instead of rebooting, you can just track it down.