Virtual Processor Scheduling – How VMware and Microsoft Hypervisors Work at the CPU Level

Have you ever wondered how virtual machines get access to the physical processor cores in the server?  Ever wonder why one or two VMs run slow sporadically?  The topic today is Virtual Processor Scheduling.  There are no good books to read on this subject, trust me I’ve looked.  Maybe in 10 years as a look back at the history of virtual computing, but not today.  So let’s try to shed some light on the subject for today’s article in our VMware or Microsoft? – The Complete Series.  

First things first.  It is fairly simple to schedule against physical processors when the virtual machines each have a single virtual processor and there are plenty of physical cores to go around.  We (VMware and Microsoft) can send the workload to the first physical core that is available, easy.  There are algorithms for figuring out the best processor core to send the workload too, but I will not go into advanced crazy algorithm mathematics today.

Now the trick/challenge/opportunity/problem (hey whatever you want to call it) that really comes into play when you have multiple virtual CPUs inside the virtual machines.  The original architecture of operating systems made a perfectly good assumption that basically says “Ok I see 4 CPUs, I own all 4, period.  I mean who else is close by to use them?  No one. I’m lonely, but all powerful.”  Isn’t that nearly always the case?  Take Lord Voldemort for example, sure he had resources, but no one really wanted to be near him.  <voice from overhead> Virtual processing is the subject Tommy. </voice from overhead> Oh right, sorry folks, continue on.

So when VMware re-introduced virtualization into the market, handling the single CPU wasn’t a big issue. However, bigger workloads call for more processors and therefore in order to scale, a new way to schedule CPU cycles against any given processor core was necessary.  This is where Gang Scheduling comes into play.  VMware, drawing upon the methods generated by the older Unix technology, uses a Gang Scheduler approach.  What this means in basic terms is this:  When a multi-vCPU machine requires processor time, all of the vCPUs are “ganged” together and scheduled to perform work against the physical cores. This is done in order to streamline the process, and attempt to keep the processing synchronized.  In other words, like networking, we don’t like a lot of packets arriving out of turn on a load balanced network, the same stance is assumed in the VMware CPU scheduling process. 

Hyper-V does things a bit differently.  Virtual processors Read more