artrix
(RAJOD)
August 20, 2021, 3:34am
1
I have a Dell 12 core host running Server 2012 HV. Its fine no issues but MS stopping updates in 2023. I started doing some testing and find that 2019 VMs are 2x slower on passmark testing.
I’ve not even done share and disk testing but this is not a good sign.
2012 Host with 2012 VM (6 core 6gig ram) - PM 9 cpu mark - 5444
2019 Host with 2012 VM (6 core 6gig ram) - PM 9 cpu mark - 2305
2019 Host with 2019 VM (6 core 6 gig ram)- PM 9 cpu mark - 2329
Over 2x cpu performance hit in the VMs.
I’m thinking of 2016 instead even though its EOF is not as long as 2019. I don’t have the numbers on 2016 yet but I tested it a long time ago and found it to be close to 2012 in performance.
The only thing bad i’ve read about 2016 is the Slow windows update. Not sure if that issue has been fixed.
Anyone regret updating a HV host to 2019 with the slow VM issues?
10 Spice ups
The hardware used, the storage disk configuration and overall age of the equipment should be a bigger concern than the version of the OS itself, since that is what greatly affects the performance of the virtual machines running on the host.
I’m currently using Server 2019 and it is running fine for me, even though the hardware is somewhat older and not really “Enterprise build quality”.
DoctorDNS
(DoctorDNS)
August 20, 2021, 9:24am
3
It is hard to say without knowing more about the detailed hardware configuration of each VM host, and the VM configuration.
You should start by looking at perfmon - what processes/services/etc are running that are consuming differently?
One obvious question - how much vRAM have you given each VM?
1 Spice up
Robert5205
(Robert5205)
August 20, 2021, 10:07am
4
Are these test numbers on the same physical host?
2 Spice ups
artrix
(RAJOD)
August 20, 2021, 11:37am
5
hardware Dell R610 64gigs twin socket xeon hex cores w HT.
The hardware is not changing the only thing I’m chaning is the OS. The only reason to change the OS is windows updates will end on 2012.
You would not know 2019 is as fast as 2012 or 2016 if you did not test it.
artrix
(RAJOD)
August 20, 2021, 11:38am
6
Yes the Exact same hardware. wiped clean and installed 2012, 2019 and now 2016 for testing.
artrix
(RAJOD)
August 20, 2021, 12:51pm
7
Wiped and put 2016 on. 2016 is a little faster than 2012 and much faster than 2019 HV. At least on this Dell Server. I may try newer hardware to see if 2019 can catch up, 2x slower is too much to ignore.
Unless I can find a tweak for 2019, 2016 seems to be a better fit. Not sure why 2019 is such a dog.
2012 Host with 2012 VM (6 core 6gig ram) - PM 9 cpu mark - 5444
2019 Host with 2012 VM (6 core 6gig ram) - PM 9 cpu mark - 2305
2019 Host with 2019 VM (6 core 6 gig ram)- PM 9 cpu mark - 2329
2016 (1607) Host with 2019 VM (6 core 6 gig ram)- PM 9 cpu mark - 5579
1 Spice up
Hyper-V on server 2019 uses the core scheduler by default, which may have an impact on VM CPU performance. Perhaps try switching to the classic scheduler to see if that alleviates your issue.
2 Spice ups
da-schmoo
(Da_Schmoo)
August 20, 2021, 1:48pm
9
I did not know this. Interesting.
da-schmoo
(Da_Schmoo)
August 20, 2021, 3:51pm
10
RAJOD:
I have a Dell 12 core host running Server 2012 HV. Its fine no issues but MS stopping updates in 2023. I started doing some testing and find that 2019 VMs are 2x slower on passmark testing.
I’ve not even done share and disk testing but this is not a good sign.
2012 Host with 2012 VM (6 core 6gig ram) - PM 9 cpu mark - 5444
2019 Host with 2012 VM (6 core 6gig ram) - PM 9 cpu mark - 2305
2019 Host with 2019 VM (6 core 6 gig ram)- PM 9 cpu mark - 2329
Over 2x cpu performance hit in the VMs.
I’m thinking of 2016 instead even though its EOF is not as long as 2019. I don’t have the numbers on 2016 yet but I tested it a long time ago and found it to be close to 2012 in performance.
The only thing bad i’ve read about 2016 is the Slow windows update. Not sure if that issue has been fixed.
Anyone regret updating a HV host to 2019 with the slow VM issues?
I ran a test before and after switching the core scheduler. After the switch, the Passmark CPU score increased by 70% in the guest. (Passmark CPU only test - not the full suite) Negligible change testing the host.
I made no changes to the VM itself.
dreniarb
(dreniarb)
August 20, 2021, 5:03pm
11
So the Passmark score increased but as far as you could tell the VM itself wasn’t necessarily running any different? Clicking around and doing things inside the VM felt the same to you?
da-schmoo
(Da_Schmoo)
August 20, 2021, 5:17pm
12
It was the only host I had available that I could reboot. Only workload is a RDS server which was sitting idle at the moment and some Hyper-V replicas. Workload really shouldn’t be different before and after.
artrix
(RAJOD)
August 20, 2021, 8:06pm
13
Not sure if you meant me? Yes the VMs did feel a tiny bit laggy on 2019 vs 2012. Its subtle as computers are so fast that even if you slow them down by 1/2 the windows experience from a user perspective is still usable. It when you get into a situation where a application that use to take 1 hour to complete now takes 2 hours is where you notice it.
artrix
(RAJOD)
August 20, 2021, 8:31pm
14
Nice catch there. That sounds like it might be the issue. Oh glad I imaged with Macrium Reflect can revert back pretty fast to verify.
What you just linked is pretty big I think. I’ve read quite a few posts on 2019 and many users are saying they have to toss 2x the virtual cpus and ram to get them to be like 2016 but they never really said why. I’m wondering if this is not the reason for that. People just threw more Vcpus at the issue. Not sure about the ram issue though.
From what I have read 2012 and 2016 are both fast no issue or tweaks needed. People complain that 2016 had slow updates. Not sure if they mean just the initial installation or ALL updates even the ones ones in 2021.
2019 issues seem to be many. And people have actually given up and downgraded back to 2016. All of this sort of scares me off from 2019 as it looks like it might create too much WORK tracking down all the issue. People that start off with 2019 might never know because how many are going to downgrade to 2012 to see if that performs better? not too many. They just plug and play.
“From another thread”
We’ve seen ~ -50% performance degradation when moving workloads from W2012DC to W2019DC. The W2019 latencies and 3x9’s latencies are off the charts poor.
We’ve seen ~ -(8-9%) performance degradation W2012 and W2016 with W2012 being the best performing of the 3.
artrix
(RAJOD)
August 20, 2021, 8:48pm
15
Da_Schmoo:
I ran a test before and after switching the core scheduler. After the switch, the Passmark CPU score increased by 70% in the guest. (Passmark CPU only test - not the full suite) Negligible change testing the host.
I made no changes to the VM itself.
Thanks for doing that!
I’m thinking there has to be many people using 2019 that are running 2x slower VMs and not realizing it.
da-schmoo
(Da_Schmoo)
August 20, 2021, 8:53pm
16
This needs to be tested in a real life scenario, not just relying on a benchmarking app though.
artrix
(RAJOD)
August 20, 2021, 10:16pm
17
Oh I will be doing that too, this is the road to that. Thanks for helping.
1 Spice up
tim7139
(Tim7139)
August 20, 2021, 11:22pm
18
From a workload point of view I have not noticed a decrease in performance moving from 2016 to 2019. I did find there seemed to be more adjustments needed to drivers and settings, and I observed benchmarks that didn’t match real world testing as closely (specifically disk and network).
artrix
(RAJOD)
August 21, 2021, 2:12am
19
Da_Schmoo:
I ran a test before and after switching the core scheduler. After the switch, the Passmark CPU score increased by 70% in the guest. (Passmark CPU only test - not the full suite) Negligible change testing the host.
I made no changes to the VM itself.
Ok I wiped the system and put 2019 back on.
bcdedit /set hypervisorschedulertype Classic (reboot)
Yes much better now. Funny neither 2016 or 2019 have near the single thread performance of 2012
2012 - Single thread - 1340
2016 - 932
2019 - (HV type 3 core) 655
2019 - (HV type 2 classic) 995
Anyway it puts 2019 more in line with 2016 in terms of VM cpu performance.
The over all cpu mark is actually a little better than 2016 after the HV change.
2012 - 5444
2016 - 5579
2019 - 2329 (default)
2019 - 5627 (after change to HV type 2 scheduler or classic)
3 Spice ups
Is the VM you’re using to test a freshly created VM on 2019, or was it restored/migrated from an earlier version Hyper-V configuration? If you’re seeing the performance corrected by using the classic scheduler, and if the latter is true, check to see if the VM configuration is using the updated settings for the core scheduler.
The value of the following powershell command should be “0” for the VM:
Get-VM | Get-VMProcessor | FT VMName,HwThreadCountPerCore
If it’s “1”, that might also reduce the available processing grunt the VM has when using the core scheduler, since it would ignore hyperthreaded cores assigned to the VM.
Source : Windows Server 2019 Hyper-V might only use half the available CPU’s? – Kevin Holman's Blog
*Edit: Original Microsoft Docs article linked in source article: About Hyper-V hypervisor scheduler type selection | Microsoft Learn *
Strangely the core scheduler change wasn’t included in either of the 2016 or 2019 “what’s new in hyper-v” article on Microsoft docs, I just remember seeing it in a compatibility matrix on some other Microsoft article somewhere, which I seem to have lost the link to