1

Hello,

I have a Dell T430 server with 2 CPUs with 20 cores and 40 logical processors and 48GB of ram, Windows Server 2019 Standard installed. There are 6 disks total, 2 are mirrored and have the OS installed. 4 are setup in RAID 5 for VMs to be located. I have the host OS installed and it seems to be running normally.

I installed guest VMs and I am seeing very slow performance. Not just network. Everything is slower. Delays clicking on the start button… delays opening windows explorer. I started with just 2 virtual processors and 8GB of RAM. After boot up, CPU usage on VMs is not maxed and memory drops to just over 2GB. Increasing resources to the machine helps slightly, but not significantly.

Have had tech calls with Dell and the hardware seems to be functioning normally. Did some restores of data and it takes over twice as long to restore data to new VM than old VMs running on T710 purchased in 2010 with one processor and Windows 2012 R2.

I have disabled the VMQ on the NICs and RSC on the virtual switch and didn’t see any improvement. I also tested with a Windows Server 2016 VM and have the same result.

Ideas?

Thanks.

8 Spice ups
2

One, RAID 5 is bad unless it’s SSD, that wasn’t mentioned. Two, what RAID adapter? There are some really poor PERC options, and really good ones.

3

Bryan,

It has a PERC H330. They are not SSDs. I know RAID 5 is not ideal, but the VM isn’t even under a load yet and running slow and we’re not seeing a lot of disk activity.

Thanks.

4

Is that the full edition of Server with the Hyper-V role installed on the host? It should be Hyper-V, not the full install.

If it were me, I’d change to the H730 and RAID 10. What speed are the disks? Have you tested the disk speed in the VM?

Also, check firmware. I had similar issues in the past with ESXi, related to Dell firmware.

5

It is the desktop experience. Yes. My understanding is it adds some additional overhead but overall not have a significant impact on performance. It is more for security reasons to install core to reduce the attack surface of the OS.

It has 6 disks, the hardware is almost 3 years old. The disks are 2TB 7.5K SAS drives. What do you recommend to test disk speed? I could test on the host and vm and compare.

I talked with Dell support and the drivers and firmware are all up to date.

Thank you!

6

Welcome to the community!

That controller has zero cache, so parity RAID will always have poor performance.

Those drives are poorly performing to begin with, and are too large to be safely used with RAID 5.

Who did the purchasing of this server? It has the number of cores to run maybe 50 VMs. It has the RAM to run maybe 4-8 VMs. It has disk configuration to poorly run maybe 4 VMs with low I/O requirements.

What is the goal for this server?

I would look at replacing with SSD first. Two SSD in RAID 1 would be fine. You could do 3+ SSD in RAID 5 if you replaced the controller with a H730.

2 Spice ups
7

Seconding the above suggestions, I would clarify that frankly speaking, you are running multiple virtual machines on top of a single HDD. That is exactly what happens with WRITE workload in case of parity RAID is being used. Multiple simultaneous write requests from multiple virtual machines are causing a so-called IO blender effect What is I/O (input/output)? | Definition from TechTarget , dramatically decreasing overall performance. If using parity RAID is a must, I would agree with Kevin above and replace the HDD disks with SSDs https://www.starwindsoftware.com/blog/raid-5-was-great-until-high-capacity-hdds-came-into-play-but-ssds-restored-its-former-glory-2 . Otherwise, an OBR10 (one big RAID10 array) is a much better option.

8

To test storage performance, use Diskspd and fio. In your case, you should benchmark local RAID storage to storage inside the VM performance and see the overhead.
https://gallery.technet.microsoft.com/DiskSpd-a-robust-storage-6cd2f223

You might also test various VM configurations, e.g., Generation 1 and 2, to add more CPU and RAM to how the performance changes. https://www.hyper-v.io/several-tips-hints-full-throttle-hyper-v-performance/

Research of “PERC H330 boost performance” returns an assumption that it might also be the RAID controller since mostly you see “RAID 5 on these controllers is horrible”. You could configure it to HBA mode and set up a software RAID using ZFS on Linux or mdadm by passthrough storage directly to a VM.

1 Spice up
9

Appreciate the input. Unfortunately, given the state of things, we have to work with what we have. At this moment, multiple VMs are not running at this time and performance still stinks. I have a similar setup from 2010 on an old server and it is running just fine. In fact, this one was purchased to replace it.

So if it’s the parity that is the issue, I could setup as RAID 10 instead of RAID5. The OS is on a mirrored pair, I will create a VM on that set of drives and see how it performs.

10

Since all the drives you currently have are the same, make sure you remove the RAID1 assigned for the hypervisor and create a RAID10 using all the six disks present. This way, you will squeeze all the possible performance out of your current hardware.

11

The disks are not all the same.

12

Changed from RAID 5 to RAID 10 and saw a significant improvement. Thanks!

1 Spice up