I love GaryExplains. Easily one of my favorite tech channels on YouTube. He’s one of the only TechTubers I feel actually explains things worth explaining.
Gary recently put out a video (embedded at the top of this article) where he explains why he doesn’t test network performance in his smartphone commentary. His reasoning is absolutely sound.
If you use a network speedtest app, there are way too many variables to consider for that test to have a scientific consistency.
But, just because network speedtest apps can’t be counted on for consistency, doesn’t mean we can’t control for SOME variables on our own.
At a basic level, we can check how a phone reports signal and connection to a router or tower. This is not a comprehensive measure of performance, but when standing in the same spot, with similar network apps, we can use this as a loose guideline.
This is of course easier to do in AndroidLand where our phones all include a VERY basic LTE reception notifier in settings. It’s also easier to grant access to this information on 3rd party apps. It’s not suspicious at all how hard Apple works at keeping this information from users. Nope. Totally normal.
I know what you’re thinking though. A basic reporting of reception doesn’t really tell us how well the phone is utilizing that connection. To Gary’s point, I personally haven’t found a good way to stress test LTE. It’s way too complicated taking network congestion and interconnection issues into consideration for a fair comparison.
But what about WiFi?
I use a big NAS to store project files and media for a Plex server. The NAS app is kind of ugly, but it does report file transfer speeds.The app seems to have feature parity between iOS and Android, though the operation of the app is noticeably smoother on iOS.
I can control network activity. I can use the same source, while downloading the same file. I can stand in the same spot (and try to hold the phones in the same position). Then, I can time how long it takes to download the file. It’s all on my personal network, and A LOT of this is under my direct control.
As a little cross platform test, using my iPhone XS against a OnePlus 7T, let’s time how long it takes to download a 2.2GB movie.
Standing right over the main hub of my mesh router, the iPhone achieves a faster file transfer speed. Topping out at 46MB/s, it finished the file transfer in 52 seconds. The OnePlus 7T maxed out at a 41MB/s transfer speed, and finished the transfer in 57 seconds.
But who wants to stand over their router like some kind of plebe? The point of WiFi is to be able to move around, and not be directly tethered to your router. Moving to my garage, almost directly under my main mesh router, I reset each phone. Then, I cleared the NAS app cache, toggled WiFi on and off each phone to make sure they were on the same node of my network, and ran each test again.
The iPhone peaked at a max transfer rate of 21MB/s, and it took a LOT longer to build up to that speed. The full transfer took 1:41, or roughly 49 seconds longer than the “optimal” file transfer. The OnePlus 7T hit a max speed of 36MB/s, and completed the transfer in 64 seconds. The OnePlus only sacrificed about 10 seconds when it had to deal with the ceiling of my garage.
This jives with the more recent tests I’ve conducted in my LG and Sony reviews, and seems to be a fairly consistent measure of performance and reception. It’s still a bit limited in scope, only applies to WiFi, but we can glean some useful data from a real-world file transfer test. It’s not comprehensive, but is at least an indicator of antenna and modem performance.
Collecting this data, we can form a hypothesis on how different phones perform on WiFi. We can START to predict which phones will have connectivity issues, and which phones will hold onto signal longer in challenging conditions.
I am on the hunt for other methods to compare performance cross-platform and across multiple devices. I think connectivity is an important performance point when talking about EXPENSIVE connected devices. If anyone has thoughts for additional tests, or a way to consistently test LTE performance, please drop some comments on this post.
And Gary, give me a call sometime! I’d be curious to hear your thoughts on methods to compare these pocket computers.
I don’t know, I think using an app for that is a little iffy since it could rely on how optimized the app is for that OS and phone, not the phone’s qualities itself.
It has been a long time since I used Android and I never tried to use a file share on it, but I know on iOS now you can map an SMB share in Files. Could you not do the same in Android and try transfer tests that way? Then you’re testing the phone without having the variable of a third party app.
As far as network tests in general though, I agree that tests are very subjective. Even this test could be impacted by factors that can’t be accounted for. Subjectively though, I will say that iPhone’s having worse network performance than Android phones doesn’t surprise me at all. It was one of the first things I noticed when I switched from Android to iOS. And I’ve had tons of Wi-Fi issues with my XS Max despite tons of troubleshooting and multiple replacements.
If there’s any advantage for the app on one platform over the other, it definitely seems the iPhone would benefit over Android devices. As for how I’m collecting this data, I haven’t seen anything which contradicts the trends Ive been following, and my results have been fairly consistent over the last year of testing like this. It’s why I’d like more folks to try similar tests instead of giving up and shrugging off real-world performance testing.
Okay, what you have here is a problem with statistical QUALITY.
Taken in bite-sized scopes the results and conclusions can vary greatly. Obtaining statistics from other sources can be problematic as there is very little control or basis.
However, what both you guys need to realise is the Statistical QUALITY is not nothing. If you apply a reasonable control and measures, and collect a lot of data/quantity, then you will produce statistical sound data and form solid conclusions. Think about how scientific research handles this for things like pain management.
I suggest collecting the most popular phone variants, testing them all together at popular use times, recording the data, and repeating it (several times a day, everyday, for a month). Then you can plot variances, and actually conclude which device is better overall.
No-one is rich enough to own two dozen devices, and no-one has the patience to conduct something tedious, and no-one has the ethics to open up the data to scrutiny/display non-bias. That’s why we haven’t really seen a research into this matter. But maybe one of you gents might be upto the task?
Cheers!