Evaluating Futuremark's Servermark VDI on the Supermicro SYS-5028D-TN4T
by Ganesh T S on September 1, 2016 8:00 AM EST- Posted in
- Benchmarks
- Futuremark
- Supermicro
- Servers
- Xeon-D
Futuremark Servermark VDI
The previous section briefly mentioned that VDI is quite popular with business / enterprise use-cases, particularly for office applications. Futuremark has kept this in mind while formulating the benchmarking strategy for virtualization capabilities. On the desktop side, Futuremark already has the well-respected PCMark 8 system benchmark. It considers various use-cases and can report scores for 'Work', 'Creative' and 'Home' workloads. Servermark VDI is based on PCMark.
Multiple PCMark instances are executed on the VDI server simultaneously (one instance per VM) in order to determine the server performance. Based on the scores from each instance, it is possible to determine the maximum number of virtual desktops that the server can support with acceptable performance. Correspondingly, it can also report the performance of the server with a particular number of clients.
The various PCMark instances are controlled by a Servermark Control Console that runs on a separate machine that is on the same network as that of the virtual machines. It is possible, but, not advisable, to run the Control Console on one of the VMs under test. The Control Console also takes the server configuration as input (through a JSON file) and uses it for generating the final report.
Servermark Control Console takes a number of parameters. The most important amongst those include the IP addresses / ports of the VMs (in either IPv4 or IPv6 format), the number of times that PCMark needs to be run on each VM, the duty cycle (fraction of time, as percentage, that should run the benchmark simultaneously on the VMs), the stagger time for the start of PCMark on each of the VMs and a timeout for processing each PCMark run.
By default, the Control Console runs PCMark three times on each VM. The duty cycle is set at 30%, stagger time at 10s and processing timeout of 60 minutes. The PCMark workload utilized is the 'Work' preset - this goes well with primary use-case for VDIs being in the office / business space. For VMs with OpenCL capabilities, the PCMark 8 benchmark can be run with accelerated compute turned on.
Servermark VDI suggests that each VM score at least 2000 on the 'Work' preset. If the minimum score is not achieved, the PCMark 8 scores of each VDI client are reported along with various statistical parameters such as the minimum, median, maximum, average and standard deviation. If the minimum acceptable score is met, the benchmark also creates a PDF file with all the hardware and score details. This serves as a certification from Futuremark that the server configuration is able to run the specified number of concurrent VDI clients with an acceptable level of performance.
Facebook
Twitter
RSS
18 Comments
View All Comments
Ryan Smith - Thursday, September 1, 2016 - link
Ultimately this is an article we worked on because Futuremark's software looked neat, and we wanted to see if it could tell us anything useful.We're nerds at heart. We like looking at new things.=)
aaronb1138 - Friday, September 2, 2016 - link
Although FM puts "2000" as their recommended benchmark level, I think you will find for VDI that the bar is much, much lower. The main gist of VDI is the fact that most users need very little computational power continuously. They need the boot and load software bumps and then most productivity software sips at a few percent of CPU. I've noticed web browsers, especially Chrome, have become the biggest consumers of CPU on workstations, and it seems almost vindictive the way certain browsers will hose a system when resource starved. Mozilla/Firefox is especially designed to make virtualization and RDC scenarios bad (the thousands of cache folders is intentionally shit design for roaming profiles and causes huge login hangs).I noticed a NUC gets around 5000 in the benchmark. From my experience in VDI as well as XenApp and RDS architectures, a NUC could support around 10 users with sufficient RAM (16GB would do it - 32GB and we're talking 20+ users, maybe 40 in XenApp/RDS instead of VDI).
Also, for the love of all that is holy, do not benchmark against RAID 0, it invalidates much of the testing. Test against both RAID 10 and 5 as those are industry standards for such infrastructures.
If you really want to deep dive, check out differences in memory consumption in HyperV vs VMware with VDI. While both support memory deduplication, experience shows that HyperV is much faster and more effective at VDI memory dedupe, while VMware seems to rely a tad heavy on memory ballooning. In server farm / VDI situations, RAM and storage I/O are ALWAYS more limiting than CPU throughput. You're not putting Pro-E and Solidworks into a VDI farm most of the time, just MS Office, a line of business application, and a browser.
madisson - Thursday, September 1, 2016 - link
maybe i am approaching this too focused on a consumer's mind, but all of that hardware needs to be shipped, unpacked, installed, and plugged in. For me, I prefer to log-in to a cloud PC for gaming and virtual PC. There are several services similar to what I'm talking about, www.paperspace.com being the one I happened to pick.I'm sure FM goes for more enterprise-level agreements, but what about the small teams and individuals?
name99 - Thursday, September 1, 2016 - link
It seems to me this is completely a WINTEL benchmark.That's fine for a certain class of users, I guess, but is, I think, a lot less interesting for many readers. It does nothing to clarify whether either
- a Linux PC would be a better server solution OR
- whether an ARM server could handle a particular load.
Obviously AnandTech can do what they want in their reviews, but I, for one, am a lot less interested in comparisons that are focussed on the few percent differences between Wintel box A and Wintel box B than in comparisons between very different systems.
powerarmour - Thursday, September 1, 2016 - link
Exactly.aaronb1138 - Friday, September 2, 2016 - link
Well, they were looking at VDI. I don't know what kind of monster would try VDI+Linux+end user workloads in the workplace. But yeah, keep saying dumbass shit about Linux and ARM.gabemcg - Tuesday, September 6, 2016 - link
Lots of hostility unfairly being directed at the authors/editors here. I thought using the benchmark preview as a way to couch a hardware review was genuinely interesting both as a nerd with a home lab, and a professional wanting to keep abreast of how the enterprise class gear is being evaluated. Keep up the good work, haters gonna hate!simran sidhki - Thursday, October 18, 2018 - link
hello