The Black Shark 2 Review: A Gaming Phone's Existential Crisis
by Andrei Frumusanu on September 25, 2019 8:00 AM EST- Posted in
- Mobile
- Smartphones
- Xiaomi
- Snapdragon 855
- Black Shark 2
Machine Learning Inference Performance
AIMark 3
AIMark makes use of various vendor SDKs to implement the benchmarks. This means that the end-results really aren’t a proper apples-to-apples comparison, however it represents an approach that actually will be used by some vendors in their in-house applications or even some rare third-party app.
Unfortunately for the Black Shark 2, the devices lacked the proper drivers to properly run AIMark, and the benchmark repeatedly crashed upon starting the benchmarks. We had the same issue on the OnePlus 7 Pro, pointing out to some software incompatibility.
AIBenchmark 3
AIBenchmark takes a different approach to benchmarking. Here the test uses the hardware agnostic NNAPI in order to accelerate inferencing, meaning it doesn’t use any proprietary aspects of a given hardware except for the drivers that actually enable the abstraction between software and hardware. This approach is more apples-to-apples, but also means that we can’t do cross-platform comparisons, like testing iPhones.
We’re publishing one-shot inference times. The difference here to sustained performance inference times is that these figures have more timing overhead on the part of the software stack from initialising the test to actually executing the computation.
AIBenchmark 3 - NNAPI CPU
We’re segregating the AIBenchmark scores by execution block, starting off with the regular CPU workloads that simply use TensorFlow libraries and do not attempt to run on specialized hardware blocks.
In AI Benchmark’s CPU workloads, the Black Shark 2 ends up with a bit of a odd spread of scores. In the shorter running benchmarks the phone is getting relatively average inference times, while on the longer running tests for some reason the BS2 falls behind other S855 devices. In fact it looks like the BS2 is landing as amongst the worse off S855 devices in the latter listed tests.
AIBenchmark 3 - NNAPI INT8
AIBenchmark 3 - NNAPI FP16
AIBenchmark 3 - NNAPI FP32
In the INT8, FP16 and FP32 accelerated tests which make use of acceleration blocks such as the Hexagon DSP and the GPU, we see the Black Shark 2 perform very well and in line with other Snapdragon 855 devices.
Overall, the Black Shark 2 is a good performer in the machine learning inferencing benchmarks, but like other devices, it’s not quite the very best in every regard, pointing out that the vendor could have improved upon its performance by keeping the software stack more up to date with what Qualcomm is offering, a widespread issue that I expect to persist over the next years as the ecosystem quickly evolves.
Facebook
Twitter
RSS
63 Comments
View All Comments
s.yu - Thursday, September 26, 2019 - link
I'm quite aware of people who never go for the achievements. When my cousin plays Kingdom Rush he DOESN'T EVEN GO FOR ALL 3 STARS.The horror! It's like eating an apple with two or three random bites then throwing away all the rest!
StormyParis - Wednesday, September 25, 2019 - link
Reviewing stuff as part of a for-pay arrangement must require very high-level mental, ethics, ... jiu-jitsu. I'm not sure how long the partnership w/Qualcomm will last, but, as a reader, thanks for the heads up.IUU - Wednesday, September 25, 2019 - link
They are trying to imitate Apple. Too much fluff, so people believe they buy a superior device(it is the price too).So they hope they can make some profit out of thin air.
I hope you should be critical of the iphones in this fashion as well.
They allocate a bigger amount of transistors to the faster cores , so they
can claim supremacy, no matter the fact that on general cpu performance
they are about the same.
No matter that their gpu is anemic and nowhere near snapdragons.
melgross - Thursday, September 26, 2019 - link
Exactly what fluff are you talking about?You really are deliberately ignorant.
edsib1 - Wednesday, September 25, 2019 - link
Another phone review where the benchmarks are all wrong. Put the phone in game mode - it is easily in the top 5 855 based phones in terms of gaming performance.cfenton - Wednesday, September 25, 2019 - link
But is that because it cheats and runs without any thermal limitations when it's in game mode? Personally, I don't want my phone getting hot enough to be uncomfortable to hold.edsib1 - Wednesday, September 25, 2019 - link
But it doesnt get hot. They run at full tilt and dont throttle.Read the other reviews on the internet about these phones. It is Anandtech that is out of step with the other sites.
The gaming phones are basically running in saver mode - unless you turn game mode on.
Exactly the same problem with the review of the Oppo Reno 10x zoom....
Andrei Frumusanu - Wednesday, September 25, 2019 - link
> But it doesnt get hot. They run at full tilt and dont throttle.I already demonstrated that's not what happens.
> Exactly the same problem with the review of the Oppo Reno 10x zoom....
What's the problem? That phone didn't throttle. It had a performance mode but that's essentially just a cheating button and running all frequencies pegged at max.
edsib1 - Thursday, September 26, 2019 - link
Game mode is not a cheating mode if the phone can maintain that speed. If you run CPU throttle on the Oppo for over 10mins the phone maintains a very high score with game mode engaged (around 15% higher than without).The Exynos 9820 on the other hard throttles hard after a couple of minutes losing about 25% performance.
So why would the game mode that doesnt throttle be more of a cheat mode than the 9820 which throttles hard?
edsib1 - Thursday, September 26, 2019 - link
Reviewing a gaming phone without using gaming mode is like testing an Audi RS6 without putting it into sport mode - pointless.