Updated: VMware ESX/VC 3.5 Update 2 - Bug or Feature within EVC / CPU Masking

I wrote last week on this bug or new feature observation after Update 2. To read my first post about this click here.

I have been in touch with VMware to find out if this is a bug or a new feature. They confirmed there has been a change to VMotion CPU constraints.

Based on my experience, VMware is a little more flexible on CPU constraints than earlier. To sum this up:

If you have two almost identical CPUs, then if you power up VMs on the “oldest” CPUs these VMs would be able to migrate to the new CPU and back. However if you power up your VM on the newer CPU (enabling the new features) then this VM would not be able to VMotion to the older hosts.

However - one thing to keep in mind though.. even though you can avoid the manual work on configuring CPU masking on every VM… DRS and/or VMware HA would over time power on VMs on all hosts - and those powered up - on the new hosts would then have problems doing VMotion.

The solution to this is to use the new feature on EVC (Enhanced VMotion Compatibility). This knowledge base article lists the CPU which is supported for EVC usage: Enhanced VMotion Compatibility (EVC) processor support

www.it-experts.dk relaunched - a danish IT community site.

We have been working on redesigning the site to provide new features for the community and about one month ago we transitioned to the new site. We already have 500 users on the site - but of course we would welcome more from the Danish community to join us.

One of the important features that we introduced was blogging facilities for our users in addition with forum features. The old site was also missing key features like RSS-feeds, which also now is available.

So to all you Danish IT professionals out there - come and join us at www.it-experts.dk

 

 

Free Solarwinds ESX monitoring utility…

Solarwinds has released a little cool - Vista-gadget alike monitoring utility for ESX server.

Download it here:  http://www.solarwinds.com/products/freetools/vm_monitor.aspx

VMware ESX/VC 3.5 Update 2 - Bug or Feature within EVC / CPU Masking?

A customer of mine had issues with CPU masking after setting up a new ESX host. They specific ordered the hardware to be alike with the other 3 hosts in that cluster. So it was the same model both Server and CPU wise.

VMotion worked just fine between all hosts as long as the VM was powered on, on one of the three oldest hosts.

But when they powered on a VM on the new host - this VM could not be migrated with VMotion to the other 3 hosts. It came up with the usual Host CPU is incompatible error.

I analyzed the /proc/cpuinfo file in the service console to determine differences between the hosts. The new host was CPU stepping 11 compared to the older which was 6. And the new host had the NX flag enabled. The results on this was:

/proc/cpuinfo file	Older hosts	New hosts
Processor	0	0
Vendor_id	GenuineIntel	GenuineIntel
CPU Family	6	6
Model	15	15
Model Name	Intel(R) Xeon(R) CPU 5150  @ 2.66GHz	Intel(R) Xeon(R) CPU 5150  @ 2.66GHz
Stepping	6	11
CPU MHZ	2660.059	2660.063
Cache Size	4096 KB	4096 KB
Fdiv_bug	No	No
Hlt_bug	No	No
F00f_bug	No	No
Coma_bug	No	No
Fpu	Yes	Yes
Fpu_exeption	Yes	Yes
Cupid level	10	10
Wp	Yes	Yes
Flags	fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss tm lm	fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss tm nx lm
Bogomips	5308.41	5308.41

Then I checked to see if the new Enhanced VMotion Compatibility (EVC) was enabled. But this option was grayed out (disabled) – because the three oldest hosts were not compatible with EVC.

Before Update 2 – and when you had VMotion CPU compatibility issues, VMotions didn’t work either way. So something happened with this in Update 2. In the past you manually had to enter the correct CPU Bits, into each VMs configuration file for VMotion to work.

But in this case I could VMotion all VMs between all hosts forth and back - as long as the VM was started on one of the three older hosts.

I noticed that on the VMs powered on, on the oldest hosts - that CPU masking bits were entered into the vmx file. This was done automatically by the system!

This was added to the vmx file of VMs powered on, on the older hosts.

cpuid.1.eax = “xxxx————xx————–” cpuid.1.ecx = “R—-R–R-RRRR-0———–H-R–” cpuid.1.edx = “—————————T—-” cpuid.80000001.eax.amd = “xxxx————xx————–” cpuid.80000001.ecx.amd = “——————RR-RR-RRR-0—” cpuid.80000001.edx = “—-R—————H———–” cpuid.80000001.edx.amd = “—–R————–H——T—-”   cpuid.1.ecx.amd = “R——-R——————-R—”   hostCPUID.0 = “0000000a756e65476c65746e49656e69″ guestCPUID.0 = “0000000a756e65476c65746e49656e69″ userCPUID.0 = “0000000a756e65476c65746e49656e69″ hostCPUID.1 = “000006f6000208000004e3bdbfebfbff” guestCPUID.1 = “000006f800010800000022110febbbff” userCPUID.1 = “000006f6000208000004e3bdbfebfbff” hostCPUID.80000001 = “00000000000000000000000120000000″ guestCPUID.80000001 = “00000000000000000000000120000000″ userCPUID.80000001 = “00000000000000000000000120000000″ evcCompatibilityMode = “FALSE”

 

 

So - I’m just wondering if this is a new feature in Update 2 VI 3.5. It is not described anywhere (at least I have not found any documentation on it).

After some thoughts I summed up - that indeed this is a nice feature to have (If you dont have the possibility to turn on EVC)… Just make sure to power on your VMs on the old servers and it automatically entered the CPU bits needed to do VMotions against the new host. And still a little weird since EVC was disabled. But it removed all the work on manually entering the CPU bits on every VM.

My concern though is that manually editing the CPU bits hasn’t been supported by VMware… so is this supported?

So is this by design or is it a bug (hence that EVC was disabled)?

Compellent to Offer Automated Business Continuity with Live Volume

EDEN PRAIRIE, Minn ., Oct. 13, 2008 – Compellent Technologies, Inc. (NYSE Arca: CML) today announced it will provide continuous, coordinated access to all data stored on Compellent SANs between remote locations with a new automated business continuity feature, Live Volume.

Live Volume will allow enterprises to manage their disparate storage sites around the world as one virtual data center. As customers move applications from one physical or virtual server to another for maintenance, local site issues or disaster recovery, Live Volume will enable IT managers to proactively and automatically migrate the associated storage volumes. By not requiring investments in third-party tools or mirroring of storage resources, Compellent’s live transfer of data will make it easier and cost-effective to manage all resources regardless of the physical location.

“Compellent is applying the same storage innovation that made automated tiered storage more affordable to next-generation technology like Live Volume and SSD that will not only deliver the highest levels of data performance and availability, but also minimize total cost of ownership,” explained Bruce Kornfeld, vice president of marketing for Compellent. “We are enhancing our automated data movement engine to help ensure enterprises can achieve true business continuity and instant access to their data whether their IT infrastructure is on-premises or delivered by a cloud computing service.”

Automating Business Continuity for Enterprises

Based on Compellent’s unique patented Dynamic Block Architecture, Live Volume is designed to eliminate downtime for enterprises that need to migrate servers and storage for disaster recovery or onsite maintenance. Each data volume will be present and automatically available on Compellent SANs at either site at all times.

“The integration of a feature that could take my storage network from manual disaster recovery to automated business continuity without lifting a finger is a very exciting prospect,” explained Mark Lynch, senior data consultant of Synetrix, based outside London. “As a managed service provider replicating across nine Compellent SANs, our clients’ assets must be protected and online at all times – an outage in our headquarters cannot impact our clients throughout the UK. Live Volume promises to keep all of our data online – whenever, wherever – and ensures downtime is a thing of the past.”

Compellent’s Live Volume seamlessly integrates with the industry’s leading virtual server platforms, enabling servers to share centralized storage resources regardless of the actual location or OS virtualization platform. The new virtual storage technology will complement server virtualization to help further reduce costs and simplify data management.

“The introduction of Live Volume will enable customers to take a step up the ladder from reactive disaster recovery to proactive business continuity in the most cost-effective way,” said Mark Gluckman, managing director of Regal IT, a Compellent channel partner based in Sydney. “We believe Live Volume will eliminate the delays associated with data replication, and provide the ideal architecture for virtualization by migrating virtual machines between sites without any downtime. That’s a valuable commodity to our customers implementing server and storage virtualization in the data center.”

Delivering Performance of SSD for Virtual Data Centers

For enterprises requiring the fastest storage performance for data management, migration and processing, Compellent’s integrated suite of virtualized storage applications will also support solid state drives (SSDs). The Compellent SAN will reserve frequently accessed, active blocks of data for “tier 0” storage for applications like transactional databases that can take advantage of the significant performance gains of SSDs, and dynamically move inactive data blocks to lower storage tiers. Because the Compellent SAN is expected to be the first to automate tiered storage for SSD, customers will be able to accurately plan SSD purchases along with other drive technologies to significantly reduce total costs while maintaining optimal storage utilization and performance.

Availability

Compellent Storage Center with SSD is currently being evaluated by select beta customers. Compellent plans to offer SSD to all markets in Q1 and Live Volume in Q2 of 2009 exclusively through the company’s growing international network of channel partners. Additional product information and pricing details will be announced at launch. More information about Compellent’s Dynamic Block Architecture is available at www.compellent.com/products.

Update 3 for VirtualCenter 2.5 released.

This release doesnt contain any new features. It’s only hotfixes in this release, but some very important ones especially in regard of the VMware HA feature (high availability). Also the license server gets an upgrade.

Also this update contains a whole lot of fixes for VMware Update Manager and VMware Converter as well. So make so to upgrade your VI plugins. During the upgrade of VMware Update Manager it will also contain a database upgrade.

Make sure to read the release notes before upgrading: Release notes

To download the VirtualCenter update 3, click here

Remember to do a backup of the SQL database before upgrading. If VirtualCenter run as a virtual environment take a snapshot of it and if possible use VCB (VMware Consolidated Backup) to do an image-level backup as well. That way you can get back to the old one.

Also VirtualCenter needs to be upgraded before VMware Update Manager and Converter.

An ext3 filesystem is remounted read-only by the kernel - problem with certain linux distributions in VMware

Today a collegue of mine experienced problems with a SLES10 after P2V process was done. The VM after a perioed became unstable and the kernel reporting that it was mounted in read-only mode. Hmrf….

Well even he had several Linux guys telling him that it was a VMware issue and after some researching and use of google he found several document about this. In fact this problem is not only related to SLES10 but several linux distributions.

VMware has also published a kb article on this, which can be read here

Another useful link from Novell: Click here