Meltdown & Spectre: Analyzing Performance Impacts on Intel's NUC7i7BNH
by Ganesh T S on March 23, 2018 4:15 PM EST- Posted in
- Systems
- Spectre
- Benchmarks
- Meltdown
BAPCo and Futuremark Benchmarks
BAPCo's SYSmark 2014 SE is an application-based benchmark that uses real-world applications to replay usage patterns of business users in the areas of office productivity, media creation and data/financial analysis. In addition, it also addresses the responsiveness aspect which deals with user experience as related to application and file launches, multi-tasking etc. Scores are meant to be compared against a reference desktop (the SYSmark 2014 SE calibration system in the graphs below). While the SYSmark 2014 benchmark used a Haswell-based desktop configuration, the SYSmark 2014 SE makes the move to a Lenovo ThinkCenter M800 (Intel Core i3-6100, 4GB RAM and a 256GB SATA SSD). The calibration system scores 1000 in each of the scenarios. A score of, say, 2000, would imply that the system under test is twice as fast as the reference system.
We see that the patching does have an impact on the performance. It ranges from around 5% for the Office Productivity scenario to around 10% for the Responsiveness metric.
SYSmark 2014 SE also adds energy measurement to the mix. We see that the patched system, despite being slower, consumes lower energy for the completion of the tasks.
Futuremark PCMark 10
UL's PCMark 10 evaluates computing systems for various usage scenarios (generic / essential tasks such as web browsing and starting up applications, productivity tasks such as editing spreadsheets and documents, gaming, and digital content creation). We benchmarked the configurations with the PCMark 10 Extended profile and recorded the scores for various scenarios. These scores are heavily influenced by the CPU and GPU in the system, though the RAM and storage device also play a part. The power plan was set to Balanced for all the PCs while processing the PCMark 10 benchmark.
Similar to SYSmark 2014 SE, we see performance impacts in all the PCMark 10 scenarios also. For example, the Essentials workload score for the fully patched system is around 6.5% lower than the unpatched system. The gaming workload doesn't seem to be affected much by the patching, though. The overall score is around 2.3% lower.
Futuremark PCMark 8
We continue to present PCMark 8 benchmark results (as those have more comparison points) while our PCMark 10 scores database for systems grows in size. PCMark 8 provides various usage scenarios (home, creative and work) and offers ways to benchmark both baseline (CPU-only) as well as OpenCL accelerated (CPU + GPU) performance. We benchmarked select PCs for the OpenCL accelerated performance in all three usage scenarios. Since these scores are heavily influenced by the CPU in the system, it is not surprising to see a performance impact. The patched configuration, on average, scores around 2% lower than the unpatched one.
Miscellaneous Futuremark Benchmarks
PCMark 7's PCMark Suite shows a 4% performance loss.
3DMark 11's entry level score and 3DMark 2013's Cloud Gate score actually end up better after the patch, but, 3DMark 2013's Ice Storm score shows a performance loss of more than 13%.
Futuremark PCMark 8 Storage Bench
PCMark 8 has a storage bench where certain common workloads such as loading games and document processing are replayed on the target drive. Results are presented in two forms, one being a benchmark number and the other, a bandwidth figure. We ran the PCMark 8 storage bench on the ADATA NVMe SSD in all the configurations.
While the absolute score is just around 1.3% lower for the patched configuration, the storage bandwidth is around 29% worse. This significant performance loss is partly due to the NVMe drive performance now being CPU bound. It is highly likely that SATA drives will not experience such a drastic performance loss due to the patching.
Facebook
Twitter
RSS
83 Comments
View All Comments
iter - Monday, March 26, 2018 - link
According to whom? You, the workstation all-seer? Or perhaps some statistics done over the internet?iter - Monday, March 26, 2018 - link
Also, if a "standalone system" is for you the opposite of "connected to the internet" that is quite indicative... You know there exists this thing called a network, on top of which the internet runs. You can have a load of workstations and servers in a network that is not connected to the outside world.Most places that do important work do it this way. Eliminates 99.99% of threats from the outside and the from the inside. Just one of many other common sense things, such as disabled usb storage devices, unauthorized network clients and whatnot. Machines that do connect to the internet are physically isolated from the secure network. They use secure proprietary interfaces for explicit data transfer between the two networks under tight scrutiny.
rhoades-brown - Tuesday, March 27, 2018 - link
Eh? So, your saying that you would put your workstations unpatched and completely unprotected on a network where other devices can connect to it?Did you hear about WannaCrypt? Your network connected workstation would have been easy prey.
Would you allow these unprotected workstations to share files with other workstations and what about the cheaper machines? I assume that you are either creating or processing content/data of some description. Have a look at MS16-120 - 'The most serious of these vulnerabilities could allow remote code execution if a user either visits a specially crafted website or opens a specially crafted document.'
What about USB sticks? Something on one machine could easily be spread to another, and people are stupid enough to plug in a USB stick that they found in a car park, etc.
There are exceptions- air-gaped networks to make things highly secure, but that seems unlikely, and if your workstations are in that rare scenario, have a look at xLED which uses a compromised switch to flash it's status LEDs to share data- crazy, I know; scary, absolutely.
Gasaraki88 - Monday, March 26, 2018 - link
Wow, that's a big exaggeration...Bulat Ziganshin - Friday, March 23, 2018 - link
>Though there is a certain irony to the fact that taken to its logical conclusion, patching a CPU instead renders storage performance slower, with the most impacted systems having the fastest storage.It looks ironic because it was incorrectly attributed as CPU bug. But the point is that it allows to discover information when OS allows it, and thus it's an OS bug of not preventing it. As far as you run pure CPU computations, it doesn't need any mitigations.
The only thing that need to be patched is communication between OS and application, and therefore you got larger hit when these communications are more intensive - on higher-IOPS operations. So f.e. I/O in large blocks (1 MB or so) is unaffected, but 4K I/O is affected, especially with higher-performance drives and higher QD scenarios.
jordanclock - Friday, March 23, 2018 - link
It is a CPU bug. The speculative execution is faulty and that is a CPU feature. The OS patches are simply workarounds to prevent certain kinds of speculative execution.Reflex - Friday, March 23, 2018 - link
It is not a bug at all at either level. It is a feature that was found to be able to be abused. That happens all the time. Once found, it was mitigated, in this case by disabling the feature (Meltdown) or mitigating the impact (Spectre). In future designs it will be mitigated or eliminated.There are all sorts of features your CPU is capable of utilizing that can compromise your data or stability (hey, you can still run in unprotected mode for memory!), when it is found to be a problem it is typically disabled at the appropriate level (microcode/firmware/OS).
bji - Friday, March 23, 2018 - link
Uh, no. It's a feature that comes with an unintended side effect of allowing data reads that should be disallowed. That part of it is a bug, plain and simple. I guess you are the kind of person that would call a bug that crashes the computer a "feature" because "it saves you power when your PC is off because it crashed".PixyMisa - Friday, March 23, 2018 - link
So it's a bug.yeeeeman - Saturday, March 24, 2018 - link
Bug is something that doesn't work as designed. I am pretty sure that they designed and verified it this way. These vulnerabilities are not bugs, they are just security loopholes.