Android 12 Broke ALL My Benchmarks: Scoped Storage Rant

This is a little ugly. For those who want a TLDR, the video above should help explain.

I’m trying to run some newer phones through my slate of tests to grade performance. The results don’t compare favorably to similar tier phones I’ve tested in the past.

Everything has gotten more complicated to test AND scores are lower. I think I’ve found the main culprit.

Scoped Storage.

Android used to have a fairly open file system. It was blissfully easy to access most files on your phone. It was a huge power user advantage over iOS.

The new Storage Access Framework is a miserable change to that access, and seems to degrade performance for any apps that need to access files on your phone.

So basically ALL apps I care about.

[What is Scoped Storage – Android Dev Blog]

To grossly over-simplify:

Android used to organize files in folders more like windows. We rarely had access to program files, but we could save anything we wanted anywhere we wanted. No artificial limitations on photos, videos, or music. It played well with memory cards and external storage. We had familiar folders for different kinds of files.

Apps like video editors could set up their own folder, and in that folder we could save templates, project files, saved projects, and we could easily copy and back that data up ANYWHERE we wanted ANY TIME we wanted. A video editing app could broadly access the Android file system after granting it permission.

Scoped Storage now forces developers to create tidy little folders in storage that users can no longer access directly. We can’t easily back up projects. Current solutions for saving or transferring projects are laughably bad. Instead of copying a save file and directly sharing it to another phone, I have to use THE CLOUD in a proprietary container, which takes up space on my GDrive account (most devs aren’t giving us options to save to other cloud services).

The process of granting apps access to storage is confusing. If I have a photo editing app, I can’t grant it top level access to all my user storage. I have to manually assign individual folders, and that’s only if the developer gives me the ability to add multiple folders access. This is clumsy and unnecessary.

Worse still, because this Scoped Storage is locked up in a different way from the open file system, it seems to significantly slow performance when an app needs to access files. Libraries are significantly slower to access, and files often need to be temporarily copied to scoped storage and then returned to user storage.

My photo editing benchmark is utterly destroyed. Applying changes to a batch of 200 photos used to take about two seconds before processing those changes. Now it takes several minutes.

Importing a project into my podcast editing app is now a painful exercise of hoop jumping. Putting a project in one folder, that the app can scan, and copy to another private folder. I used to be able to delete old project files in any file explorer of my choice. Now i need to dig through the app’s menus, and the developer hasn’t made this process easier yet.

I still don’t understand what’s supposed to be better here.

Users have less access. Users and developers have less control. App performance suffers. There are MORE opportunities for users to break expected workflow.

In the worst possible ways, this is starting to feel like the aspects of iOS that keep me from fully embracing iOS as a platform. When I complain about a lack of competition in the mobile space, this is one of the issues I’m concerned about.

Short story incredibly long…

I don’t know what to do about benchmark results. Basically every phone that properly jumps from Android 11 to Android 12 is now properly off the list of legacy support for the old file system access. If it’s just a software upgrade, your phone gets a performance downgrade.

Looking at new phones, and new SOCs, none of my old results are applicable. The batch photo test MIGHT perform similarly saving each file, but the entire process of creating that batch is SUBSTANTIALLY longer and slower. There’s no fair comparison to be made anymore. Literally a phone from 2019 can outperform a powerhouse in 2022 over the course of an entire task, even if the 2022 phone performs better in ONE aspect of processing that task.

It’s a miserable turn of events, and I might need to completely give up on benchmarking. At least until ALL new devices are in the same boat, and only then can we compare against similarly new phones. YEARS of my benchmark results are essentially defunct.

I’ve never been more desperate for Windows on ARM to appear on pocket computers. There’s got to be a better way…