The main takeaway, which became resoundingly clear over the last 48 hours?
Marketing works.
Tuesday we all sat through a round up of new product announcements from Apple, with the stock boiler plates of innovation and improvement. The most amazing products yet created. The most powerful products on the market. We were shown this year’s collection of bar graphs touting the huge lead Apple hardware enjoys over the competition.
Revealing “pro” versions of the iPhone, Apple leaned heavily into video creation to showcase the new cameras and the new A13 chipset.
The iPhone 11 Pro is so powerful you can make movies on it! Just don’t pay attention to the multi-million-dollar sound-stage, and the hundreds of thousands of dollars spent on equipment, lighting, and the talented team of filmmakers behind the phone, but I digress…
It’s those bar graphs though. Since the PowerMac days, Apple has had a love affair with simplistic, unlabeled bar graphs. We’ve seen them through so many keynotes over the last decade, that we just take them for granted. It’s just common knowledge that iPhones are more powerful than Androids.
But what happens when we test Apple’s performance claims against their video production claims?
iOS fans on Twitter get REAL flustered…
I’ve been running video rendering tests since my days at Pocketnow. Synthetic benchmarks are too easy to manipulate, and I’m kind of desperate for real world performance metrics beyond gaming. I talked about that shortly after the iPhone XS launch last year in this video:
Video production is a demanding use of any computer, so it’s one area that can help us visualize the impact on the CPU and GPU of a phone. My current test involves taking a one-minute video clip from my Panasonic G85 mirrorless camera. It’s a pretty basic h.264 clip, UHD resolution, 30fps, at a 100Mbps bitrate. It’s important to control for the video, using a separate source, so no one phone gains any kind of advantage.
The Twitter tirade started after we saw yet another “Apple Blue Line Bar Graph Better Than Android Gray Line Benchmark”. The A12 is more powerful than any Android, and the A13 will beat that!
But here’s the problem.
I truly believe Apple chips are silly powerful, but for the last four years, Apple really hasn’t let us touch that power. I shared my rendering experiences again, comparing the iPhone XS against the iPhone SE. In iMove, the iPhone SE continues to render video faster than the XS.
There’s no good explanation for this. If we’re buying new phones on the promise of better performance, then this kind of restriction (on a first party Apple app) is unforgivable. What good is “more power” if you can’t use it on something as resource intensive as creating video?
Because Apple claimed the A12 in the iPhone XS could outperform current Androids, I included some comparisons using my favorite video editing app on a OnePlus 7 Pro. iMovie isn’t on Android, and I genuinely enjoy working in PowerDirector Mobile.
Rendering the same video, the OnePlus is a LOT faster at the task than the more expensive XS. The OnePlus also delivers a final video at twice the bitrate of the iPhone (which does look better to my eye). Better quality, twice the size, in two thirds the time.
That’s when the drama REALLY started.
I wasn’t being “fair” to Apple in that comparison. According to angry people on Twitter, you’re never able to compare performance between two products unless you can test in EXACTLY the same program. Fake news!
Which to a degree, I can appreciate the idea that it’s not a perfect oranges-to-oranges test, but here’s what we have to admit when we see iMovie lose this bad.
Apple software developers, coding an app for Apple hardware, are SO bad at their jobs, that they can’t optimize iMovie to beat PowerDirector (a third party app) on an Android phone not made by PowerDirector.
As long as we’re willing to acknowledge that Apple’s hardware and software “synergy” affords NO benefit to consumers in this test, then I’m happy to concede that it’s not a “fair” test for the iPhone.
Thankfully there is a cross platform video editing app. KineMaster is available on both iOS and Android. And in a brief Twitter exchange, a former developer for KineMaster even repeated his personal experiences touting iPhones as superior to Androids for performance.
Here’s the problem though.
iPhone’s are still kind of terrible at file management. There’s no way to just add a video to the iPhone camera roll without jumping through some hoops. The iPhone XS says it can’t play the video files from my Panasonic, but you can sync them just fine to the Apple TV app. However, if a video is in Apple TV, it’s locked from other apps like video editing. In iMovie, I can import from cloud storage, but iOS prevented me from sharing my normal test video to KineMaster. I was locked out.
By comparison, when I work in Android, I pull the memory card out of my camera, plug it into my phone, and I can do whatever I want with the files I create. You know, how professionals might enjoy working out in the field.
After a couple of hours trying different syncing and sharing solutions, I gave up trying to use the control test file I’ve used for years. I couldn’t get a video ON TO the iPhone, but I can get video OFF OF the iPhone. I shot one minute of UHD/30p video and one minute of UHD/60p video and sent them to my OnePlus 7 Pro.
I need that to sink in for a second.
To compare performance, the iPhone will be rendering video FROM the iPhone. The OnePlus, which normally shoots MP4, will be rendering an Apple created MOV. In terms of testing, this is about as big an advantage as I could give to the iPhone. Same app. Same video file. Push render, and this happened:
It’s not even close.
Apple likes to make big claims. I’m actually thankful that Apple promotes mobile video production. I’ve been trying to get people to do more from their phones for years. I feel like folks have been overbuying on their pocket computers, and not really getting their money’s worth. This is one area where our phones can genuinely outperform most consumer laptops.
The reality of using an iPhone for this kind of work borders on misery though. The iPhone can only work for you if you use it in exactly the way that Apple thinks you should. From video capture to file management, and only on accessories and services that play nice with iOS. There’s not much flexibility. One tool off the reservation, like my Panasonic camera, and you’re instantly roadblocked. I’m not going to choose what camera I shoot on for work, based on what phone can use that footage. That’s backwards.
The most galling aspect of this “Pro” iPhone discussion, most of these workflow issues would be fixed if iOS could inherit the same updates Apple delivered in iPadOS. Namely better support for file management. According to Apple, you need to buy a completely different computer just to read a memory card, but I digress again…
When you jump through all those hoops, when you change your workflow to appease your phone, you’re still “rewarded” with a slower overall experience.
These tweets kinda blew up. More than a few folks were genuinely surprised. People aren’t used to Apple getting challenged like this, and not a lot of people knew that Apple has been struggling here for a couple years now.
The purpose of this kind of testing is NEVER to shame a consumer who genuinely enjoys the product they own. I’m never going to be the guy who tells you how you should spend your cash.
Instead, tech reviewers have neglected their responsibilities in holding companies accountable, especially judging popular products against the manufacturer’s marketing claims. All things being equal, video performance on an iPhone is still “very good”, but it absolutely doesn’t live up to the expectations set by Apple PR.
Of course, now a small group of Apple fanboys are requesting I move the goalposts again. I need to be MORE FAIR STILL to the iPhone. Comparing iMovie to PowerDirector? Not fair! Comparing KineMaster on Android vs iOS? Still not fair!
Now the “true” test should be using the absolute best app on each platform! I should be comparing Luma Touch on the iPhone against PowerDirector on the OnePlus!
I’m kinda done shooting videos of every single render test combination we can come up with, but I have run those comparisons over the last year. Luma on the iPhone XS, and PowerDirector on a premium Android.
The iPhone XS still lost. By quite a bit.
***UPDATE – 1***
What the hell. One more speed test for the road.
Here’s the iPhone XS running Luma Touch against the LG V30 running PowerDirector. “Standard quality” on Luma is pretty close to “High Quality” on PowerDirector. Let’s see if the iPhone XS can beat an LG from 2017…
***UPDATE – 2***
Big thanks to all the folks who weighed in on my file transfer issues. I still can’t find a direct connection method to import my Panasonic footage into the iPhone camera roll, but if I transfer it through my NAS, I can import that back to the phone. Whew. This just made my life a LOT easier! You tech nerds rock!
Once again showing us why you’re the best Tech Reviewer
Hey man, you made a typo, its than instead of then,
When you are using comparative, I’m sorry if I’m wrong English is not my first language, excellent post, gonna forward it to all my ifags.
please add your twitter post link also
The two main threads are linked.
thanks… great observation…. but I think this is their software department’s fault for not using the full power of the cpu…. if you ask them they will reply you that they intentionally did this to save the battery consumption…..
That’s very likely the case. And we know iPhones get real unstable driven to their battery limits, bit if that’s the case, then maybe they shouldn’t be bragging about chipset performance the user can’t really use?
That “update 2”, isn’t that kinda like Apple’s standard method to do these things? First upload your file to the cloud (it’s a 100GBs file and will cost you time and money? We don’t care) and then download from the cloud for your device?
It’s stupid IMO, there’s occasions when you just don’t have the connection, and it’s to slow to do the work.
It’s frustrating for sure, but he fact that i can get this done with my NAS means there has to be a way to do this in the field without the cloud. Baby steps!
As a multimedia SW engineer who programmed h/w video codecs for different platforms I have to say: author doesn’t understand technical details and thus blames CPU/GPU while he should blame VPU (Video Processing Unit). That’s why I think he spreads misunderstanding.
VPU is neither related to CPU or GPU. In modern SoCs it’s a separate unit which is responsible for video encoding/decoding and its performance at the moment stuck at 4K/60fps and it doesn’t grow for few last years like for CPU/GPU. VPUs used in recent SoCs are still designed to handle 4K/30-60fps, not faster. So how you can expect exporting video from your video editors faster than 4K/60fps?
Some VPUs could be a little bit more efficient than others and thus have better peak performance. For example, if some VPU’s peak encoding performance is 63fps for 4K, then you’ll win a few additional seconds while exporting. Nothing else. But such VPU still be advertised as 4K/60fps, because 60fps is one of standard video rates and 63fps is not.
That is the reason why exporting one minute of 4K video on latest or 3 year old smartphone takes about one minute of real time and they all comparable to each other: https://www.youtube.com/watch?v=YqIepaNY1m0
Once chip manufacturers will create 8K/60fps VPUs, then you’ll definitely get faster exports for your 4K/60fps videos. Until that day you’ll spend one minute of real time to export one minute of 4K video.
The issue at hand ISNT just the performance discrepancy, it’s also the claims made by Apple. When a consumer sees CPU/GPU charts in a keynote (where half the iPhone Pro discussion was about pro cinema) what conclusions are they apt to draw. If Apple’s video processing has remained the same since the iPhone 6S, and is out performed by a 2017 LG, then how can this company make claims about “power for pros”?
As to “win a few seconds” we’re talking export rates 20-50% faster on snapdragons. Go ask a video editor what they’d give for a 25% increase in rendering speed.
I think that are other people that don’t understand Juan’s point (that I agree).
Don’t matter what piece of silicon is being used. Each year Apple claims it got faster and faster and that no competitor comes close, so why the iPhones are losing against everything here? Even with your explanation about the VPU, how can a SOC from 2015 with a GPU from 2014 be losing against a SOC from 2018 with a GPU from the same year?
The A12 is much faster than the A9, why can’t we touch that performance difference?
Why inferior hardware on inferior software is faster?
I’m not claiming to be a pro. But.
You claimed that “VPU is neither related to CPU or GPU.” Yet, in the A12’s component break down, I’m seeing the Video processor listed, so how is it separate?
Either way, even if it’s separate, a chain is only as strong as it’s weakest link. A industry best CPU with a third of the ram can’t be a good thing. What about the small battery it’s paired with, you really can’t have the A12 flexing it’s full muscles to die even faster..
Apple users like to claim the A12 is years ahead, but the here is a test showing different.
“you really can’t have the A12 flexing it’s full muscles to die even faster”
What’s the point of all that power then?
About the battery, fab processes improves with time. The A9 is a dual core 14nm SOC, the A12 is a dual+quad core on 7nm!
Also, the XS battery is almost double the size.
I thought iOS 13 is alleviating most of the file handling part?! You plug in the card (with a dongle ofcourse) and you’re good to go right?
Didn’t work with my Panasonic 4k/60p files.
Do pros really transfer files from a video camera to edit on a phone!?
They do when working in a mixed camera situation and they need to meet a deadline. But for the purposes of benchmarking, apple fans seem to get real flustered when you normalize the video file so the iPhone doesn’t get an advantage. Not that it mattered. The OP7 still beat it when rendering a video from the iPhone.
I can easily read videos from my cameras, from SD card, with Apple lightning SD card reader.