1

I was hesitant to even post this as I know there has been much discussion on this topic already. Most of what I found wasn’t very recent though, so I thought I’d bring this up to see what I get. I’ll just point out my current experience and why we’re even considering leaving Veeam and feel free to comment at will.

We are a small environment with the following setup (as it relates to backups) at present:

  • 2 VMware ESXi 6.x hosts using Essentials license and direct attached storage (SSD for database drives and HDD for some larger file storage)
  • One is the Primary Host with two sockets with both occupied and runs all production server VM’s
  • One is Backup Host with two sockets and only one occupied and runs: vCenter Server appliance, Veeam Backup proxy, and Secondary AD/DC. Also receives and hosts Veeam Replication jobs so we can quickly spin up recent (within an hour) server images if primary hardware fails.- On-site NAS device for backup storage of VM’s and Endpoints
  • Also backs up to external USB disk each night that has no write permissions from anywhere but local NAS to prevent ransomware from getting all backup copies- Off-site NAS device that receives backup copies each night

Total backup size is a little less than 2 TB when considering all endpoints and historical points in time, but not including replicas as they are on the backup host and not external storage.

So, I feel I have multiple copies of the data, at multiple points in time, and stored in multiple locations. I feel pretty good about coverage. I have been able to restore data from Veeam Backups and Replicas with no issues over the years during testing or actual data loss situations, so I trust their product.

The problem is, we’ve been having some datastore issues where the latency spikes to several thousands of ms during Veeam’s replication process. I have tried many things to correct this (controller firmware, VMware drivers for controller, settings changes, proxy changes, etc.) and haven’t gotten very far. Veeam support was not up to the quality it used to be through this issue either. Often, they seemed to be looking at things in the log files to blame the issue on rather than resolving it. For instance, they noticed I was backing up our Veeam server during a backup job, so they said that might be the issue. Well, if they’d read my problem ticket, I’m not having trouble with that backup job; I am having trouble with a Replication job that has never included the Veeam server.

So, I just don’t trust their support anymore and can’t live with the impact on the production environment (20 second delays every hour are murder on databases and users). I’ve been considering alternatives and Unitrends has popped up as a front runner. I know Unitrends offers most of the same features and seems to have a good reputation. It would also be nice to “recover” some of the Windows Server licenses for other projects that are currently occupied just running Veeam processes (Veeam Server, Proxy, and Gateway as suggested by Veeam support).

What I’m wondering about is what direction should I go with it if I go with Unitrends? Should I go with their Software Appliance and pay for endpoint agent licenses as well? Should I look at their Physical Appliance and start breaking up our workload from one host onto two? What about replication? Unitrends does not do this, so what are some alternative products for just replication? Should I just have the Backup Host in “cold” mode and only power it up to restore from Unitrends backup files in the event of primary host hardware failure and forget about replication? Should I give Veeam another chance somehow? I’ve been using their products for at least 6 years, so I hate to let them go, but I can’t keep them if I can’t rely on them.

Thanks for any insights you may have as a Veeam or Unitrends user, but I’d really like to avoid vendors chiming in unless it is with solution ideas and not just sales pitches (I realize those are similar, just don’t take it too far is my point).

EDIT: See here for more information on things I have tested and hardware specs…

8 Spice ups
2

Hello,

My only advice with Unitrends is to get their hardware appliance. It’s probably cheaper than what you can homebrew and their support will be much better. I tried to deploy Unitrends software on some older hardware a few years ago and to be frank, it demo’d fine, but once under the load of our entire enterprise backups, it choked. Support was attentive but unable to resolve the issues due to how we deployed it. I heard they abandoned it completely only a few months after deployment and I am not sure if they were refunded any of the substantial cost of deployment.

2 Spice ups
3

Keep in mind that if you use VMware Snapshots all you’re really doing is telling VMware to take a snapshot then delete that snapshot.

What you’ve described sounds like a VMware issue.

5 Spice ups
4

Please keep in mind that this latency issue is not Veeam, but VMware. Veeam uses the VMware snapshot process to prepare the replication and what you are experiencing is VM stun. I’m not sure moving to another product that also uses snapshots is going to fix your problem.

7 Spice ups
5

And these folks are correct. I don’t see how switching to Unitrends fixes this issue you are having.

2 Spice ups
6

Also, to give you some insight, some applications hate latency. For example, I tried replication during business hours, and as soon as the snap happened on my server, my AutoCAD users would have their drawings go into read-only and they had to save to a temp location and copy back. I’d love to use it, but for my filer, I can’t.

3 Spice ups
7

I got agree what has been said it the core issue is your VMware setup. A Unitrends appliance is still going to use snapshots but it is separate hardware which might help. I don’t have any recent experience with other backup vendor but I have and still get great support from Unitrends.

1 Spice up
8

Just thought I’d shoot this down (or attempt to) up front. I have tested snapshots. No issues when triggering them and removing them from VMware. The issue is only happening when Veeam Replication is taking place.

So, while I agree VMware is where the problem appears to be, it never has happened with Unitrends or VMware snapshots or releases, but does with Veeam. I’ve been battling this for months, so I have tested this quite a bit.

1 Spice up
9

Thanks for considering Unitrends Luke.

For an environment like yours, you’ll usually find that one of our physical appliances is a better fit for a few reasons, but mostly, the ability to diversify your environment without worrying about licensing. Our physical appliances are licensed by the capacity of the unit, while our software is licensed by protected resource. This means if you add another VMware host, you’ll need another software license, but so long as you’re getting the retention you need on a physical appliance, and you’re comfortable setting the backups up for a new client, there’s no reason to even call us.

Also, I want to make sure that you were aware that replication is included in our standard, enterprise, and enterprise plus software editions. Is there something unique in your replication setup that makes you think that our replication won’t work for you?

1 Spice up
10

Are you using a controller with BBC?

11

When you say replication, are you referring to the Backup Copy? I’m talking about storing the physical backup files as a VM on another host so that they can be turned on at a moments notice and be running in a matter of minutes. Veeam allows this as well as failing back over to the original host with all new data. I have not found anything in Unitrends that leads me to believe it has this capability. Thanks!

12

Also ESXi 6 has improved the snapshot stun time:

1 Spice up
13

Have you looked at the output from something like esxtop during a replication cycle to see what’s going on?

I can see how it could be something like Veeam trying to pull too much simultaneous IO but I’m still struggling with the fact Veeam is just using VMware snapshot/backup API’s to do replication far as I know.

2 Spice ups
14

Maybe this will help…

15

Have you tried using the storage latency control settings in the Veeam options?

1 Spice up
16

Thanks for these details. It sounds like either our Instant Recovery , which allows you to failover any VMware, Windows, or Hyper-V client in the event of a failure within your local environment, or our Disaster Recovery as a Service , which allows you to spin your clients up in our cloud. Do you want to do this failover on or off premise?

17

We started using Veeam a couple of years ago when we did a hardware refresh for our software conversion. We had these exact same issues, our local partner who helps manage our backups worked on the issue with Veeam support for a year and never was able to to get them resolved. We switched back to using ShadowProtect and have not had a issue since.

1 Spice up
18

LOL I saw that earlier and was wondering how John would comment.

19

Hi Logan

VMware storage snapshots can add an additional layer of I/O to source server virtual disk resulting in performance issues on your applications , I wrote a article comparing agentless vs agent backup that has a short section on this.

https://community.spiceworks.com/topic/1990236-agents-wont-die-agent-vs-agentless-backup

I would recommend you look at Arcserve , for VM replication to the second host we have a few options.

Firstly can buckup source servers agentless via VMware software snapshots and create virtual instances out of the backup points rather than an additional software snapshot you could increase backup intervals to bring down RPO data loss.

Secondly , we can do the same as the first option but rather use agents inside the guest operating system so that we no longer use I/O intensive VMware software snapshots , we will still create virtual instances on second host.

Thirdly , we can use Full system High Availability functionality to continuosly replicate I/O every second to standby VM on the second host . I/O is captured in the guest operating system via an agent and replicated realtime. This would give you ZERO RPO on data loss.

Both options Two and Three can be monitored through heartbeats and allow for automatic failover to standby VMs on second host.

In all the above scenarios we only license the source data , removing additional licesing requirement for standby host.

Arcserve is sold in both software or Appliance.

I believe this is a suitable fit and recommend you look at the below links.

Download UDP: http://okt.to/eXmslE

Live on-line events for Arcserve UDP: All timezones: http://okt.to/RruMRZ

Arcserve UDP Live Demo: Every Friday, 10:00 GMT: Register: http://okt.to/fALlGx

Arcserve High Availability Live Webcast: Every Tuesday, 10:00 BST: Register: http://okt.to/mR4fEN

Unified Data Protection for Virtual & Physical Servers: Every Thursday, 12:00 CEST: Register: http://okt.to/bhHIT5

20

I believe what you are saying, but I cannot wrap my head around why this would be the case. Snapshot processing is done from VMware in any case, you can watch the commits happen in Vcenter when the backup software sends the commands over. Could it just be that the data is larger when Veeam is doing it? As you know, making a snapshot of a guest, going to lunch, coming back and deleting it will take almost no effort from your storage. It has to be that there is just a long sustained write happening, I would suspect. This is certainly strange.

2 Spice ups