System Performance: UL and BAPCo Benchmarks

Our 2022 Q4 update to the test suite for Windows 11-based systems carries over some of the standard benchmarks we have been using over the last several years, including UL's PCMark and BAPCo's SYSmark. New additions include BAPCo's CrossMark multi-platform benchmarking tool, as well as UL's Procyon benchmark suite.

UL PCMark 10

UL's PCMark 10 evaluates computing systems for various usage scenarios (generic / essential tasks such as web browsing and starting up applications, productivity tasks such as editing spreadsheets and documents, gaming, and digital content creation). We benchmarked select PCs with the PCMark 10 Extended profile and recorded the scores for various scenarios. These scores are heavily influenced by the CPU and GPU in the system, though the RAM and storage device also play a part. The power plan was set to Balanced for all the PCs while processing the PCMark 10 benchmark. The scores for each contributing component / use-case environment are also graphed below.

UL PCMark 10 - Performance Scores

The Phoenix-based systems are on top across every segment, except for the 'Essentials' workload. On account of their multi-core / multi-threaded performance advantage and the iGPU capabilities, that is no surprise. The single-threaded performance is a little bit behind Intel's Raptor Lake, even when the power limit is bumped up to 65W, and that is evident in the Essentials score.

UL Procyon v2.1.544

PCMark 10 utilizes open-source software such as Libre Office and GIMP to evaluate system performance. However, many of their professional benchmark customers have been requesting evaluation with commonly-used commercial software such as Microsoft Office and Adobe applications. In order to serve their needs, UL introduced the Procyon benchmark in late 2020. There are five benchmark categories currently - Office Productivity, AI Inference, Battery Life, Photo Editing, and Video Editing. AI Inference benchmarks are available only for Android devices, while the battery life benchmark is applicable to Windows devices such as notebooks and tablets. We presents results from our processing of the other three benchmarks.

UL Procyon - Office Productivity Scores

The workloads involved in the 'Office Productivity' segment all benefit from single-threaded performance, and that is evident as the Raptor Lake-based systems all have a good lead over the Phoenix-based ones. At 65W, the GTR7 manages to inch closer to the 35W Core i9-13900H in the GEEKOM Mini IT13, but the Phoenix systems are a step behind irrespective of the power limits.

From an energy consumption viewpoint, the 4nm process for Phoenix seems to be working wonders, as the 4X4 BOX-7840U in its 28W avatar manages to complete the workload with minimal energy consumption. It might not be the fastest machine for the workload, but it is quite power efficient in getting the job done.

Moving on to the evaluation of Adobe Photoshop and Adobe Lightroom, we find that the multi-core performance and iGPU advantage helps the Phoenix systems lock up the top spots. Interestingly, there is not much difference between the 40W and 65W versions, and this points to the overdrive to 65W being of not much benefit.

UL Procyon - Photo Editing

In fact, the energy consumption for the GTR7 is discernably more than the one for the 40W version of the 4X4 BOX-7840U, pointing to a sweet spot of around 35W for the processor to deliver a good performance / power efficiency balance.

UL Procyon evaluates performance for video editing using Adobe Premier Pro.

UL Procyon - Photo Editing

This workload appears to benefit from the extra power budget, and the 40W+ configurations of both Raptor Lake-P and Phoenix provide similar performance numbers.

However, the 4X4 BOX-7840U configurations are the most energy efficient of the lot, with the 40W mode presenting a good balance between performance and energy consumption with the second place in both graphs.

BAPCo CrossMark 1.0.1.86

BAPCo's CrossMark aims to simplify benchmark processing while still delivering scores that roughly tally with SYSmark. The main advantage is the cross-platform nature of the tool - allowing it to be run on smartphones and tablets as well.

BAPCo CrossMark 1.0.1.86 - Sub-Category Scores

The use of idle time compression favors Intel-based systems, but that is not indicative of real-world performance. Based on these results, it would appear that scripting and automating workloads to keep the system busy all the time would probably be better on the Intel-based PCs. Since CrossMark attempts to consolidate different workloads together without idle time intervals and play it back in a non-real-time environment, it is not entirely representative of real-world performance like PCMark 10 and UL Procyon. Therefore, tasks requiring frequent user interaction are better represented by those other benchmarks.

Setup Notes and Platform Analysis System Performance: Application-Specific Workloads
Comments Locked

13 Comments

View All Comments

  • ingwe - Thursday, December 28, 2023 - link

    Can the system be powered via one of the USB4 ports? I didn't see that noted but would much prefer that option. Particularly with the size of the power brick.
  • meacupla - Thursday, December 28, 2023 - link

    I doubt it. They have "USB/DP" labelled, but the lack of "/PD" is a sign that it most likely does not.
  • TheinsanegamerN - Tuesday, January 2, 2024 - link

    No, its wired to use the power brick for power input.
  • kenyee - Thursday, December 28, 2023 - link

    They should have made both Ethernet ports 2.5GB.
    Nice design otherwise....
  • meacupla - Thursday, December 28, 2023 - link

    I'm kind of curious to know why they include a 120W power adapter when it only consumes 74W at full load.
    Do the 5x USB ports support 10W output each or something?
  • PeachNCream - Friday, December 29, 2023 - link

    In this case 120W as a worst-case scenario with a bit of wiggle room seems reasonable especially when compared to nerds that routinely vastly overestimate their power supply needs and stuff a irrationally overspec PSU into the dinosaur-obsolete desktop form factor gaming/streaming/"esports" case.
  • TheinsanegamerN - Tuesday, January 2, 2024 - link

    USB 3 supports .9 amp per port, so 2.7 total, plus 1 for the 2 on the back. 3.7x5 is 18.5. 92.5 overall.

    Most off the shelf supplies are either 90, 120, or 135. So a 120 it is then. This also gives you headroom for capacitor aging and heat related power draw.
  • mode_13h - Thursday, December 28, 2023 - link

    I don't see how they can pitch it as an industrial PC without ECC support (which I assume it lacks, since it wasn't mentioned).

    As a generic mini-PC, it does look like a good option, both in terms of multithreaded performance and efficiency. Too bad they didn't manage to close the gap with Intel's Raptor Lake-P NUCs, on idle power.
  • PeachNCream - Friday, December 29, 2023 - link

    They're probably just using that in marketing materials so that retail/home buyers feel like they're getting something more reliable because its supposedly designed for industry usage. It's similar to how companies proclaim something is "off-road" to sell something to someone that might drive through their neighbor's yard or hit a curb at the grocery store. Also, if everyone does it, you can't be the vendor left out or people will ask, "Why does such-and-such not have a Sport Utility Vehicle Super Sport model? The word sport should be in the name or its not as good!"
  • charlesg - Friday, December 29, 2023 - link

    I agree with Peaches.

    It's basically marketing, which is mostly, um, using appealing terms, true or not.

Log in

Don't have an account? Sign up now