Question

Q: migration queued and ends up showing error the operation didn’t complete on resource virtual machine

Dear All,

Windows 2012 R2 Server with Hyper-V role VM live migration failing after windows patches update. Status is migration queued and ends up showing error the operation didn’t complete on resource virtual machine. Event viewer shows error as follows.

Faulting application name: vmwp.exe, version: 6.3.9600.18936, time stamp: 0x5a7b4367

Faulting module name: KERNELBASE.dll, version: 6.3.9600.18938, time stamp: 0x5a7ddf0a

Exception code: 0xe06d7363

Fault offset: 0x0000000000008eac

Faulting process id: 0x180c

Faulting application start time: 0x01d3d3f48c045791

Faulting application path: C:\Windows\System32\vmwp.exe

Faulting module path: C:\Windows\system32\KERNELBASE.dll

Report Id: ca9f5bf3-3fe7-11e8-80df-a0d3c1fcc2bd

Faulting package full name:

Faulting package-relative application ID:

Please help.

Regards,

Shibu



Hi Partner,

 

Thanks for posting in our community. This is Cici and I am glad to assist you with the post.

 

From your description, I understand that Windows 2012 R2 Server with Hyper-V role VM live migration fails after windows patches update. If anything is misunderstood, please feel free to let me know.

 

Before moving on, to narrow down this issue, would you please help us verify the following information?

1. I noticed that live migration fails after windows patches update, which updates have you installed?

2. Could you please provide the screenshot of the error message about “the operation didn’t complete on resource virtual machine”?

 

Based on my research, this issue seems to happen due to speculative execution side-channel vulnerabilities. Live migration of a virtual machine with the new processor capabilities will fail when moving to Hyper-V hosts without updated firmware. To enable live migration to a host without updated firmware, stop exposing the updated processor capabilities within that guest virtual machine. The easiest way to do this is to modify MinVmVersionForCpuBasedMitigations to a VM version above that of the virtual machine that needs to migrate, and perform a cold boot of that virtual machine. To get more information, we can refer to the following article:

 

Windows Server guidance to protect against speculative execution side-channel vulnerabilities

https://support.microsoft.com/en-us/help/4072698/windows-server-guidance-to-protect-against-the-speculative-execution

 

Also, migration of virtual machines without the new processor capabilities will succeed when moving to Hyper-V hosts with updated firmware. However, a cold boot is required for these guests to see the updated firmware capabilities. To get more information, we can refer to the following article:

 

Protecting guest virtual machines from CVE-2017-5715 (branch target injection)

https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/CVE-2017-5715-and-hyper-v-vms

 

If anything is unclear, please feel free to let me know.

 

Best regards,

 

Cici Wu

Microsoft Partner Support Community Technical Support Engineer

Microsoft Global Partner Services

------------------------------------------------------------------

This posting is provided "AS IS" with no warranties, and confers no rights

Please Mark As Answer after signing in if this post is helpful to you. Thanks for your cooperation.

 

Did this solve your problem?

Sorry this didn't help.


Hi Cici,

Thanks for the update.Following are the latest patches installed.

I have the screen shot of the patches installed. Tried to paste it in this screen. But not sure it will display to you or not. Just shows as a small icon.

Did this solve your problem?

Sorry this didn't help.


Hi,

Server model - HP DL380 GEN8

Regards,

Shibu

Did this solve your problem?

Sorry this didn't help.


Hello Cici,

VM configuration version is 5.0. Do you think this version using new processor capabilities.As per guidance VMs using new processor capabilities will fail if Server firmware is not updated.

Are you suggesting to upgrade firmware of the Server?Please advise.

Regards,

Shibu

Did this solve your problem?

Sorry this didn't help.


Hi Partner,

 

Thank you for your reply.

 

For your first question, if there are any virtual machines with VM version below 8.0, you must set a specific registry value on the host operating system. This will configure Hyper-V to expose the new processor capabilities into guest virtual machines with lower VM versions.

 

That registry value is MinVmVersionForCpuBasedMitigations under HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Virtualization. The value should be set to the minimum VM version that needs access to the updated firmware capabilities, in the format “Major.Minor”. To expose the firmware to all virtual machines on the host (i.e. version 1.0 and above), run the following command on the host:

reg add "HKLM\SOFTWARE\Microsoft\Windows NT\CurrentVersion\Virtualization" /v MinVmVersionForCpuBasedMitigations /t REG_SZ /d "1.0" /f

Note: VM Version 8.0 was introduced with Windows Server 2016 and the Windows 10 Anniversary Update. Hosts running Windows Server 2012 R2 and below must set the registry value

Warning: live migration will fail between hosts with the updated firmware and hosts without the updated firmware. For more details, see the FAQ at the bottom of this document.

 

For your second question, Yes, you need to update the host operating system and firmware before cold booting your virtual machines.

 

Still, to get more information, we can refer to the following article:

 

Protecting guest virtual machines from CVE-2017-5715 (branch target injection)

https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/CVE-2017-5715-and-hyper-v-vms

 

If anything is unclear, please feel free to let me know.

 

Best regards,

 

Cici Wu

Microsoft Partner Support Community Technical Support Engineer

Microsoft Global Partner Services

---------------------------------------------------------------------------------------

This posting is provided "AS IS" with no warranties, and confers no rights.

Please Mark As Answer after signing in if this post is helpful to you. Thanks for your cooperation.

 

Did this solve your problem?

Sorry this didn't help.


Hi Cici,

Thanks for your clarification.

I have suggested the following action plan to customer.

Suggested  the following action plan.

 

1) Update the host operating system to same latest patch levels in all Hosts

2) Ensure the virtualization hosts  has to be  updated to latest firmware. There is latest firmware available on  HP site.

3) Ensure Hyper-V is configured to expose new processor capabilities ( There are three registry keys to be enabled on the Hosts and observed  it is already enabled)

4) Guest virtual machines by shutting down the VMs and starting again (to enable the firmware related mitigation for VMs)

5) Update the patches on guest operating system as required and reboot

6) There are few VMs whose integration service to be upgraded.

Please let me know if anything missing or to be deleted.

Regards,

Shibu

Did this solve your problem?

Sorry this didn't help.


Hi Cici,

Regarding my suggestion customer advised as follows.

Dear Shibu,

 

You can apply the below action plan first on Host 1(10.163.1.37) which is up to date with windows patch updates and when the issue is completely resolved then we can move VM’s from Host 2 to Host 1 and then update host 2 and same with Host 3, so there will be no down time as we have some critical server there.

 

Please advise for both posts for today.

Regards,

Shibu

Did this solve your problem?

Sorry this didn't help.


Hi Partner,

 

Thank you for your reply.

 

Please kindly understand our forum is focus on one issue per post. If the original issue about Windows 2012 R2 Server with Hyper-V role VM live migration fails after windows patches update has been resolved. For this new issue with best practice about updates in Windows Server 2012 R2,we highly suggest you post new case in our forum, and we will continue to help you.

 

Thanks for your understanding and cooperation.

Best regards,

Cici

 

Microsoft Partner Support Community Technical Support Engineer

Microsoft Global Partner Services

------------------------------------------------------------------

This posting is provided "AS IS" with no warranties, and confers no rights.

Please Mark As Answer after signing in if this post is helpful to you. Thanks for your cooperation.

 

Did this solve your problem?

Sorry this didn't help.


Hi Cici,

My colleague have visited site and done a firmware update  for the affected HP Servers and installed the remaining latest Windows patches. But issue of VM migration still persisting. Then uninstalled the patches one by one.once KB4093114 and KB4093121 patches uninstalled the issue resolved. Conclusion is  these two patches creating the issue on VM migration.  

 

Do you have any idea as these patches making the issue and if yes please share some articles.

Regards,

Shibu

Did this solve your problem?

Sorry this didn't help.


Hi Partner,

 

Thanks for your reply.

 

For such kind of query, Ive done a lot of research, however, the official article is limited. At such situation, we will try the best to deliver the information to the product team to see if anyone can confirm this, but not guaranteed. Once there is any update, we will get back to you at earliest time. Thanks for your kind understanding.

 

Best regards,

 

Cici Wu

Microsoft Partner Support Community Technical Support Engineer

Microsoft Global Partner Services

------------------------------------------------------------------

This posting is provided "AS IS" with no warranties, and confers no rights

Please Mark As Answer if this post is helpful to you. Thanks for your cooperation.

Did this solve your problem?

Sorry this didn't help.


 
Question Info

Views: 222 Last updated: April 26, 2018 Applies to: