VMware

The effect of vMotion on guests

I am trying to understand what my service provider is telling me about their vmware setup and how vMotion works.

There are 2 hosts and 1 of my guests on each host. Eg. Guest A on Host A, Guest B on Host B.

I ping Guest B from my client machine whilst the service provider uses vMotion to transfer Guest A to Host B.

During the vMotion the pings to Guest B rise dramatically to over 1000ms even though Guest A is the one being transferred.

The service provider states that vMotion shares the same network as the guests.

They say that even if vMotion had it’s own network the guests would be affected in the same way.

Can anyone explain what is going on with vMotion please?



View Reddit by 12treesView Source

 

To see the full content, share this page by clicking one of the buttons below

Related Articles

10 Comments

  1. They set it up wrong. vMotion should be on its own non routable (usually) network. If it’s 10gb, the vlans can share physical NICs, but your LAN shouldn’t be co-hosting vMotion traffic.

  2. Quick answer with broken sentences because I’m drinking

    they probably share network interfaces

    Vmotion is very efficient and I would bet anything these are only 1g links. Because very document published by vmw since 3.0 or whenever the fuck vmotion waa moved from experimental to production-ready has said to isolate it logically and physically from your production lan. ~~There is *no bandwidth throttle for vmotion* **at fucking all**~~ (post-rage edit. As pointed out this is controllable with NIOC. ) it will move as fast as possible across the network. The hosting provider is for sure thrashing their physical interfaces.

    So vtm triggers and bandwidth on the nic goes to 99.999+% used causing fat latency to anything else

    “Even if they had it’s own network guest would act the same” total fucking dogshit. I ran a hosting company with 5000+ managed vms built correctly and literally never-fucking-ever experienced this kind of shit. We ran N+2.5 and I could evac multiple hosts anytime desired with DRS cranked to max. Literally hundreds of thousands of DRS intimated vmotion actions over 10 years.

    Fuck you hosting provider up their stupid asses

    Your hosting company is a bag of dicks

  3. Pings rising or dropping during a vmotion are not uncommon.

    Their setup seems odd

    They could mean that their hosts use the same interfaces for vmotion and guest traffic, but at least they should be different port groups/vlans.

    This sounds more of a os dealing with the micro stun effect of the vmotion and less a network problem. But thats my .02

  4. Agree with the comments already posted regarding their V-Motion setup.

    Are they doing vMotion or vMotion with storage migration?

    Regardless, at around the 21% mark (if memory serves), you’ll see some ping spikes, but die down relatively quick. What they are doing is either lazy, lack-of-knowledge, or poor design implementation/planning.

  5. My guess would be that the vmotion traffic is swamping the network. It sounds like they’ve made a few design mistakes with this environment.

    “vMotion shares the same network as the guests” – If this means sharing the same subnet, then that’s a bad idea as vMotion traffic is unencrypted by default. If this just means the same physical network and/or the same physical NICs, that’s a pretty standard approach and shouldn’t be an issue unless the network is way underspecced.

    “even if vMotion had it’s own network the guests would be affected in the same way.” – This shouldn’t be the case. If the network is underspecced they should be using network I/O control on the distributed switches to prioritise VM traffic. If the network isn’t underspecced then vMotions should be transparent to other VMs (and largely transparent to the VM being vMotioned as well.

  6. You can isolate vMotion to another VLAN, or a separate switch.

    They may already have another VLAN, but their underlying switch infrastructure is just garbage then if it’s still impacting another VLAN’s performance.

    The most you should see during vMotion is 1 ping loss, this is during the suspend/resume event between the source host, and destination host.

    I also like this line from the documentation:

    Configure each host with at least one network interface for vMotion traffic. To ensure secure data transfer, the vMotion network must be a secure network, accessible only to trusted parties. Additional bandwidth significantly improves vMotion performance. Consider that when you migrate a virtual machine with vMotion without using shared storage, the contents of the virtual disk is transferred over the network as well.

    If vMotion is truly on the same network as your guests you have a security issue and you’re SP is non compliant. So you’ll fail any audit.

  7. These guys sound like clowns. Vmwares own best practice docs clearly call out that vnotion interfaces should be seperate from guest interfaces. The incredibly high latency you’re seeing is a direct result of them misconfiguring the hosts.

    What’s worse though is that they are lying about the fact they have things setup correctly. I would definitely shop for another provider.

  8. Generally yes vmotion should be on it’s own network. They should set it up as it’s on VLAN if they have the equipment to do so and it “should” have at least one adapter according to the best practice guideline.

    Do you have more details on the networking setup?

  9. Love this thread. The only other thing I would request and verify from your “Service provider” (purposely in quotes) is that your guest vm’s are also not swapping memory. If your guest VM has swapped memory it could result in blips of connectivity as the delta of your disk has to be copied and handed off to the next host thats running as swap.

    EDIT:

    Also, putting multiple guests from different customers on a shared network is a complete violation of pretty much every security policy in use today. Might even be good enough to get out of any contracts you’re currently in since thats a breach of security policy. I’m not sure if you mean shared network, or just shared interface for the vmotion however.

Leave a Reply