VMware ESX/VC 3.5 Update 2 - Bug or Feature within EVC / CPU Masking?

A customer of mine had issues with CPU masking after setting up a new ESX host. They specific ordered the hardware to be alike with the other 3 hosts in that cluster. So it was the same model both Server and CPU wise.

VMotion worked just fine between all hosts as long as the VM was powered on, on one of the three oldest hosts.

But when they powered on a VM on the new host - this VM could not be migrated with VMotion to the other 3 hosts. It came up with the usual Host CPU is incompatible error.

I analyzed the /proc/cpuinfo file in the service console to determine differences between the hosts. The new host was CPU stepping 11 compared to the older which was 6. And the new host had the NX flag enabled. The results on this was:

/proc/cpuinfo file	Older hosts	New hosts
Processor	0	0
Vendor_id	GenuineIntel	GenuineIntel
CPU Family	6	6
Model	15	15
Model Name	Intel(R) Xeon(R) CPU 5150  @ 2.66GHz	Intel(R) Xeon(R) CPU 5150  @ 2.66GHz
Stepping	6	11
CPU MHZ	2660.059	2660.063
Cache Size	4096 KB	4096 KB
Fdiv_bug	No	No
Hlt_bug	No	No
F00f_bug	No	No
Coma_bug	No	No
Fpu	Yes	Yes
Fpu_exeption	Yes	Yes
Cupid level	10	10
Wp	Yes	Yes
Flags	fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss tm lm	fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss tm nx lm
Bogomips	5308.41	5308.41

Then I checked to see if the new Enhanced VMotion Compatibility (EVC) was enabled. But this option was grayed out (disabled) – because the three oldest hosts were not compatible with EVC.

Before Update 2 – and when you had VMotion CPU compatibility issues, VMotions didn’t work either way. So something happened with this in Update 2. In the past you manually had to enter the correct CPU Bits, into each VMs configuration file for VMotion to work.

But in this case I could VMotion all VMs between all hosts forth and back - as long as the VM was started on one of the three older hosts.

I noticed that on the VMs powered on, on the oldest hosts - that CPU masking bits were entered into the vmx file. This was done automatically by the system!

This was added to the vmx file of VMs powered on, on the older hosts.

cpuid.1.eax = “xxxx————xx————–” cpuid.1.ecx = “R—-R–R-RRRR-0———–H-R–” cpuid.1.edx = “—————————T—-” cpuid.80000001.eax.amd = “xxxx————xx————–” cpuid.80000001.ecx.amd = “——————RR-RR-RRR-0—” cpuid.80000001.edx = “—-R—————H———–” cpuid.80000001.edx.amd = “—–R————–H——T—-”   cpuid.1.ecx.amd = “R——-R——————-R—”   hostCPUID.0 = “0000000a756e65476c65746e49656e69″ guestCPUID.0 = “0000000a756e65476c65746e49656e69″ userCPUID.0 = “0000000a756e65476c65746e49656e69″ hostCPUID.1 = “000006f6000208000004e3bdbfebfbff” guestCPUID.1 = “000006f800010800000022110febbbff” userCPUID.1 = “000006f6000208000004e3bdbfebfbff” hostCPUID.80000001 = “00000000000000000000000120000000″ guestCPUID.80000001 = “00000000000000000000000120000000″ userCPUID.80000001 = “00000000000000000000000120000000″ evcCompatibilityMode = “FALSE”

 

 

So - I’m just wondering if this is a new feature in Update 2 VI 3.5. It is not described anywhere (at least I have not found any documentation on it).

After some thoughts I summed up - that indeed this is a nice feature to have (If you dont have the possibility to turn on EVC)… Just make sure to power on your VMs on the old servers and it automatically entered the CPU bits needed to do VMotions against the new host. And still a little weird since EVC was disabled. But it removed all the work on manually entering the CPU bits on every VM.

My concern though is that manually editing the CPU bits hasn’t been supported by VMware… so is this supported?

So is this by design or is it a bug (hence that EVC was disabled)?