Galaxy S21 By the Benchmarks: The performance plateau…

Samsung was early to market with the S21 this year.

In a move to reduce prices over last year’s S20, the new S line from Samsung is a less feature rich offering than its predecessor. While those compromises are debatable, and are certainly more a conversation about personal preferences than objective improvements, the main touted improvement for the phone has been the top of the line Snapdragon 888.

This chip marks a change in SOC from Qualcomm, including one larger CPU performance core along side three more powerful CPU cores, in addition to a quartet of lower power CPU cores. The SD888 is also packing the latest and greatest GPU from Qualcomm, and as expected, our synthetic benchmarks show generational improvements.

That’s only the beginning of a conversation about performance though. How does the S21 stack up against last year’s handsets? How should we feel about a Galaxy S which is pre-throttled out of the box?

Performance testing is more involved than running a synthetic bench and comparing who’s number is bigger. Let’s dig a little deeper than that. Here’s the Galaxy S21: By The Benchmarks!

How are each of these tests performed? Here’s my process for benchmarking phones!

Synthetic Bench – Geek Bench 5

Immediately we run into a problem.

Samsung is forcing me to duplicate ALL of my work. The S21 is extremely finnicky. Out of the box, Samsung is tuning this phone for better battery life, and hidden in optional settings (first found three layers deep in the battery stats) is a toggle for “Enhanced Performance”. Like previous generations of Samsung (running screens at lower resolutions) this continues a tradition of Samsung making BIG claims, but offering  a “lesser” out of the box real-world experience.

I’ve decided to follow other tech reviewers in testing here. For less popular brands, it’s a common refrain that “Average Consumers” won’t go digging through settings to tweak their devices. I’ve performed my normal lineup under Samsung’s out-of-the-box settings.

To “be fair” to the world’s largest smartphone manufacturer (making big performance claims), I’m also testing under the most optimal performance settings with Enhanced mode active AND with the screen set to 60fps.

As expected, CPU scores climb for both Enhanced on and off. Out of the box, we see a roughly 9% improvement over phones like the Note 20 Ultra, and about a 15% improvement over most phones from last year on the SD865.

These numbers improve further with Enhanced mode, with about a 15% jump over the Note 20 Ultra, and around a 21% jump over last year’s phones.

Enhanced mode isn’t listed as changing graphics power, but over successive runs, I did see small but consistent differences in GPU scores. Enhanced mode seems to nudge the GPU scores just a little.

Comparing best runs from the Note 20 Ultra to the S21, GeekBench reports a nearly 30% improvement to graphics.

These synthetic scores are encouraging, but there are some concerns with the process of sampling a single phone. The performance swings on the S21 are more severe than on any phones I tested last year.

I always try to let a phone cool in between tests, and I typically perform each benchmark 2-3 times. Even for Geekbench, it wasn’t difficult to end up with an errant run on the S21 which stuck out from the pace of other runs. The S21 heats up faster, stays hotter longer, and has needed the most downtime of any premium phone I’ve reviewed lately.

I am handling the S21 with more consideration and care than phones previously reviewed using these apps. Philosophically, this is NOT as “real world” as previous benchmark runs I’ve performed in the past.

Video Render – PowerDirector

Even with powerful hardware, optimization still matters. The S21 suffers its first defeat at the hands of the LG V60. I’m not sure what special sauce LG includes in their imaging software, but PowerDirector REALLY likes it.

With Enhanced mode disabled, rendering time falls in line with SD865 phones, or SD855 outliers like the Pixel 4XL and LG V50. My worst run from the S21 was slower than my best run on the LG Wing. Optimization is critical.

Video Render – KineMaster

KineMaster uses a faster rendering engine, but it’s also not as well optimized for the architectural changes on this SOC. Where consistently, I could arrive at a tiny fraction faster render on the S21 with Enhanced disabled.

Factoring in the S21’s fastest render, it still loses to the Note 20 Ultra and RedMagic 5S.

Video Transcode – PowerDirector

A longer and simpler video project, the S21 scores fall right in line, just a touch slower than last year’s phones. There’s little difference between Enhanced modes on the S21.

Again, an interesting twist on optimization is seeing the Surface Duo continue to lead the pack on a simple project render.

Video Transcode – KineMaster

KineMaster results mirror PowerDirector, and again Enhanced brings little change in final output.

For a phone that boasts “Pro Grade Cameras”, and a gadget that is often held up as a crown jewel solution for mobile content creation, there’s little to suggest that Samsung’s offerings should be considered the de facto standard for Android creators.

The issue for video rendering, it’s a real-world use that drives a phone hard BEFORE you push render. These scores are all timed on pre-edited project files. If you were editing out in the field, the S21 would already be hot before pushing render. If you have to take the time to cool the phone after editing, it somewhat defeats the purpose of mobile video production. You lose the critical advantage of IMMEDIATE output.

Performance is inconsistent, thermals slow the phone down under more aggressive use, and the built in storage would absolutely be a concern when working out in the field.

Podcast MixDown – Audio Evolution

The audio test is the one critical victory I can point to for this phone. It’s the only test where the S21 claimed an outright victory above older devices. The S21 completes a podcast mixdown with a respectable lead over last year’s phones.

Given audio recording is less resource intense, audio files take up less space, and this final project performance, the S21 makes a terrific argument as a mobile audio production platform. Pairing up the S21 with a newer USB audio device or wireless lavs, one would easily be able to leave laptops at home for interviews and podcasts.

Stabilization – Google Photos

Video stabilization is still my voodoo test, and the S21 times further muddy the waters. There was a loose trend last year, phones with higher refresh rate displays generally performing better. The S21 completes this test “well”.

The scores are still hit or miss, and there’s not much I would point to here as a predictable trend. I’m gonna keep running this until I can figure out what actually influences performance.

Compression Test – RAR Lab

Phones are pocket computers, and Samsung devices (with Dex) make GREAT arguments for laptop replacements. Looking at file management and compression however, the S21 floats at nearly the same level as SD865 devices.

The S21, with Enhanced on, is handily outperformed by the Note 20 Ultra in RAR Lab’s synthetic test. The XPERIA 5ii also manages to barely squeak by. With Enhanced off, the S21 scores are more mid-pack against last year’s phones.

Of course, synthetic tests don’t always reflect real performance. With Enhanced on, the S21 loses pretty handily to most phones from last year. A longer single compression test helps highlight thermal throttling, where the S21 finishes closer to the Duo than phones like the OnePlus 8 or Sony XPERIAs.

Turning Enhanced off, and phones from 2019 are able to slip past the S21.

Photo Processing – PhotoMate R3

PhotoMate also has a synthetic test, and the S21 scores poorly opposite SD865 phones. With Enhanced on, the S21 is only slightly faster at completing the test than a OnePlus 7T.

Of course, synthetic tests aren’t “REAL”.

The S21 manages a respectable second place finish behind the RedMagic 5S in the most brutal test I perform. This is notable, as the RM5S has active cooling. Running a fan on a phone, we should expect a higher tier of performance from a “last gen” SOC.

The S21 does showcase heavier thermal throttling than many SD865 phones however. The second batch of 100 photos took roughly 6% longer than the first. Thermal throttling on the OnePlus 8T and XPERIA 1ii is more in the ballpark of 2%. The RedMagic is roughly 1% slower.

Turning Enhanced off however, the S21’s first 100 photos are right in line with the XPERIA 1ii, but the second batch of photos takes roughly 20% longer. That puts the second batch of photo processing near Surface Duo performance.

I ran these tests twice on different days, and found similar results. The tricky aspect again, if one were shooting some photos, then editing a handful, creating a template, and applying it to a batch of RAW photos, the phone would already be running at a higher temp than the controlled method I’m using to test here. The real “in-the-hand” experience would likely be degraded for a batch process before starting the batch.

Gaming – BrightRidge

In my short intro run on Brightridge, the S21 scores very well. Maxing out all the graphics settings, we can easily see the touted GPU upgrades.

Playing through the same opening section, the S21 is able to best the Note 20 Ultra by over 15% average frame rate, and even manages to beat the RedMagic 5S by a small margin.

If you’re into mobile gaming, I think I would recommend deactivating Enhanced performance for most titles. It’s at least worth a test drive. Enhanced is mostly a CPU throttling “feature”, and the S21 needs all the help it can get at managing internal temperatures.

Gaming – Undead Horde

A game I’m considering adding to this review lineup, Undead Horde is a fun twin-stick unit management game from one of my favorite developers. It’s a port from the Nintendo Switch, and represents an interesting torture test for phones. As the game progresses, you control more units on screen, which becomes more and more demanding on the phone.

While less visually impressive, it’s a more “computer-y” kind of game, and can hit phones as hard or harder than graphically lush titles, especially getting into the end game with larger maps and armies of units.

The S21’s performance is concerning. Immediately, the phone spikes to some impressively high frame rates. This game was challenging to run above 50fps on phones from 2020. Early game play with frame rates in the 90’s looks fantastic.

Unfortunately after about 20 minutes of gaming, and acquiring more minions, frame rate drops to an average somewhere around the mid 30’s.

By comparison, a OnePlus 8T never spikes as high for the maximum frame rate, but it also doesn’t drop as dramatically. Getting to the same point, nearly 20 minutes into the game, the 8T is maintaining a frame rate in the mid to high 40’s.

After half an hour of playing Undead Horde, the S21 is only managing about a 2-4 average FPS advantage over the Razer Phone 2 from 2018.

Concerning Thermals

It has been suggested that I might have a lemon of an S21, but no other testing seems out of line with outlets performing more granular performance measurements.

Running multiple synthetic benchmarks produces similar scores to what I’ve seen highlighted on other editorials. My graphics measurements seem in line with reviews from GizmoChina, Anandtech, and Golden Reviewer. Especially when compared to the Xiaomi Mi11 scores, where Samsung seems to have a tighter grip on thermals, and that’s coming at the expense of sustained performance.

I lack the equipment to properly measure, but looking at some of the early tests done by outlets like Anandtech and Golden Reviewer, it seems the GPU on the S21 is capable of some serious power draw. Some measurements on the SD888 as high as 8.5W.

If the phone COULD sustain its peak output, it would likely drain a full battery in under two hours.

The Galaxy S21 is a High-Performance Phone, but…

It’s a high-performance phone that largely just keeps pace with phones released last year.

This makes the S21 very difficult to position as a recommendation. WHO is the right consumer for this hardware?

The S21 breaks a consistent string of performance improvements. From the Galaxy S7 to the Galaxy S20, we could easily demonstrate performance improvements on the first day of testing a new phone.

It is fair to think that performance will improve as software is better optimized for this new SOC, and Samsung can tune that over time. However, last year we didn’t need to wait, or hope for updates, to show how the S20 could outpace the S10. Those improvements were immediately accessible.

With the exception of audio production (which you know I’M going to nerd out about), there’s precious little benefit in daily use real-world apps. If you’re buying a phone to show off the best synthetic benchmark scores, the S21 might be “worth it”, but it doesn’t seem to get much work done faster than the phone it replaces.

Worse, the phone it replaces had a nicer screen, with the option of higher resolution OR faster frame rate. The S20 had memory card expansion, which certainly helps for content creation. The S20 had a useful feature in supporting magnetic stripe payments in Samsung Pay. Handy here in the USA, where NFC adoption has lagged the rest of the world.

When you consider the used pricing on a Galaxy S20 is near the used pricing of an S21, it’s not difficult to understand why. The S20 was “more phone”. Pure and simple. The main highlight for the S21 is a newer SOC that outperforms the S20 on paper, but has yet to be properly optimized. “Average Consumers” can’t seem to utilize that “extra performance” on real-world apps. At least, not on any apps that complete tasks in longer stretches of activity. The S21 seems primed for shorter interactions.

Many people criticize my brand of performance testing, hiding behind “average consumer” daily driver use, but considering the claims made by Samsung, I think it’s fair to think some consumers might buy an S21 expecting a premier computing platform.

This is even more precious territory for Samsung, as Dex demonstrates one of the key critical advantages of a Galaxy over the competition.

Dex fulfills a competitive computing solution for folks who don’t want to carry multiple computers around. A stunningly high degree of work can be accomplished from a Dex setup. We’re overdue Samsung offering better power management and power pass-thru options for Dex, and currently, the S21 thermals would be concerning for more aggressive Dex use.

More testing needs to happen now.

Samsung was beaten to the SD888 punch by Xiaomi in international markets, but still managed a significantly earlier release than previous Galaxy phones. Next, we need to test additional devices to see if performance improves. Every manufacturer puts their own unique spin on phones, if it’s Sony with advanced power management, OnePlus with a fluid UI, or gaming phones with advanced cooling.

Are the issues on the S21 unique to Samsung, or is the SOC the culprit?

Stay tuned. 2021 is going to be an interesting year for phones…