Tech reviewers have a lot on their plates. A smartphone isn’t a “phone”. It’s a catch-all mobile computing platform. Tech reviewers need to be multi-disciplined in their commentary. Of course, it’s impossible for an individual to be an expert in all areas, so we all rely on assistance for the subjects we might be less well versed. Sometimes we’re guided by other reviewers. Sometimes we get good materials from manufacturers to guide our coverage. Regardless, no reviewer is an island.

In experiencing numerous gadgets throughout a career, it’s also easy to fall into patterns. We’ve encountered so much data, and observed so many trends over time, that we might not always be rationally dissecting a product, as much as we might be intuitively or emotionally arriving at our conclusions.

Smartphone audio is an excellent example of a topic which is often overlooked. Worse, when it is discussed, it’s frequently considered in a subjective fashion.

“I like the sound of phone A, but I don’t like the sound of phone B.”

It’s not uncommon for a reviewer’s opinion of the phone overall to influence that opinion on the audio produced.

“I like phone A, so I prefer the sound on phone A.” 

Like most aspects of technology, when viewing a product subjectively, we’re more apt to appreciate the familiar, and grade a product based on that familiarity. We strive for objectivity, but hearing or seeing something different than what we’re used to will feel foreign.

“I’m used to the sound on phone A, so phone B sounds wrong”.

The thing is, audio isn’t terrifically difficult to measure. Out of all the aspects we’re tasked with reviewing, audio should result in one of the better balances between subjective experience and reporting on objective data.

This issue has been highlighted over the last three phone generations with LG’s move to include higher quality signal processing in their V series phones. That audiophile hardware contrasting against an industry trend of removing 3.5mm phono jacks from competing handsets.

We see a lot of opinions.  We see a lot of assumptions. We don’t see a lot of people backing up their assertions with real evidence.

I’m a big fan of David Ruddock’s work over at Android Police, so to pick on him a bit, his review of the LG V30 is a perfect example of arriving at the wrong conclusion by following his gut. Audio is a major selling point for the V30, though some have questioned whether LG’s Quad DAC is a marketing gimmick. This aspect of the phone was omitted from his original article when first published. He later added the following paragraph.

The headphone jack brings me to LG’s vaunted Quad DAC (this section of the review was added post-publication). Many have been quick to point out that LG’s use of a different DAC and amplifier make the V30 the “audiophile” choice among smartphones, and for those driving high-impedance headphones, this may well be the case. However, as was clear to me on the V20 last year, the phone on which this Quad DAC debuted, LG’s use of digital filtering makes qualitatively evaluating the benefits of this hardware very difficult. The moment you “activate” the Quad DAC mode, the sound signature from the headphone jack changes extremely noticeably. The reason LG does this is so that you notice something when you press this magic “sound quality” button on your phone. The truth is that all this does is muddy the issue of whether or not the Quad DAC is doing anything to noticeably improve quality, as it has already warped the audio signature so much as to render listening comparisons pointless. I will say the headphone jack on the V30 can push a lot of power relative to other phones I’ve used, and it sounds great. Most people, however, will not notice any kind of improvement in listening experience (at least, no more than using an EQ preset would improve their experience). It’s nice to have, and for the audience that it serves, it’s there.

I very much value David’s commentary on smartphones, and I appreciate that he stops short of declaring the V30’s audio a simple trick of EQ manipulation. However, David is at a loss to explain the differences he hears between the DAC built into Qualcomm’s chipset and the ESS DAC LG supplies for the V30’s Hi-Fi mode. He can hear changes “extremely noticeably”, but offers no explanation for those changes.

The reason LG does this is so that you notice something when you press this magic “sound quality” button on your phone.

At this point in his review, one might wonder, what exactly is LG doing? However, including the word “magic” and putting “sound quality” in quotes is an easy way to dismiss any perceived differences as unwanted manipulation. It’s a trick, or a scheme. We don’t know what they’re doing, but it’s probably not worth investigating.

However, as was clear to me on the V20 last year, the phone on which this Quad DAC debuted, LG’s use of digital filtering makes qualitatively evaluating the benefits of this hardware very difficult.

David’s assertion here, that evaluating this hardware is “very difficult” simply isn’t true. We can connect cables to a headphone jack, record samples from it, measure the output, and compare it against other phones recorded under similar conditions. We should be able to see when a phone manufacturer is delivering higher quality output. By contrast, we should also be able to see when a manufacturer is altering the audio file the consumer is playing.

Above is an example of altering a file. This graph shows the EQ performance of the HTC 10 with Boomsound off (in green) and Boomsound on (in white). From left to right, this graph visualizes frequencies of sound from low to high. With Boomsound activated, we can see HTC radically alters the characteristics of the audio being played. HTC isn’t necessarily producing a higher quality output, as much as it’s “juicing” crowd pleasing bass frequencies to appeal to consumers. It’s like slathering extra saturation and sharpening on a photo. Boomsound is “ear candy”.

If LG were employing filtering which altered the music file, we should see evidence of that.

Nope. The graphs between the Qualcomm DAC (white) and the ESS DAC (green) are nearly identical.

It’s worth repeating that David specifically avoids saying that LG is manipulating EQ, but he does criticize “digital filtering”. The only problem here, eventually we all arrive at some kind of analog output, something our ears can actually interact with.  A claim is being made that there might be some kind of manipulation of the audio before it hits our ears. However, there simply isn’t any evidence to support that assertion.

There are differences we can observe however. The ESS DAC has a lower noise floor. The ESS DAC has a wider signal to noise ratio. The Hi-Fi amp can deliver a substantially louder output when used on higher impedance headphones.

HTC’s Boomsound operates like the “magic makes the music sound bassy good” button David alludes to. LG isn’t doing anything like that on the V30.

The moment you “activate” the Quad DAC mode, the sound signature from the headphone jack changes extremely noticeably.

David isn’t wrong. You can hear a difference between the Qualcomm and ESS DACs. It’s important here to acknowledge the claim LG makes when referencing the ESS Quad DAC in the V20 and V30. The signal processing chain is designed to eliminate noise generated by the phone to produce the cleanest possible representation of the audio you choose to play. It’s not designed to manipulate the content of your audio files. Android Authority has a great article on the benefits of this system.

All audio equipment makes its own noise. A clumsy metaphor, LG’s Quad DAC is akin to a camera taking multiple RAW exposures in a burst to better isolate and reduce grainy noise in a single photo. Each stage of the Quad DAC adds to the signal without adding more noise, thus reducing noise the phone might be contributing to your music.

We have a stimulus: “I hear a significant difference.”

We have a claim from the manufacture.

We can measure the output.

Our data supports the manufacturer’s claim.

It’s my hypothesis that David is hearing a cleaner representation of his test audio files. If he’s listening on higher impedance headphones (he never disclosed what he used to test the V30), then he’s likely hearing frequencies of sound missing from other phones that can’t properly drive those headphones. Given that I know he liked the Pixel last year (which had a ridiculously weak headphone amp) that could sound like EQ manipulation. Even if the LG amp isn’t being driven in “High Impedance” mode, it’s still louder than most of the phones we got last year. If you’ve not been hearing those frequencies of sound properly represented, the V30 could sound like Boomsound style trickery.

So why did I tag GSMArena in the title of this article?

I don’t expect David to be an audio expert. However, where might a reviewer gain insight into performance aspects they aren’t sure how to test? Personally, my main resource for confirming phone specs is GSMArena. I think many tech reviewers would join me in celebrating them as a resource. They do an absolutely brilliant job of collecting measurements and testing consistently. When it comes to audio however, we have a small problem.

The GSMArena audio benchmark is woefully out of date.

Their testing protocol hasn’t evolved beyond the minimum needed to analyze CD quality sound. When looking at their results, we might only see decimal point differences between phones. Yet looking at benchmarks like Frequency Response, we know many of these handsets are capable of generating frequencies of sound that dogs can’t hear, well outside the range GSMArena is measuring. It’s like using a high school microscope to try and get a look at the atomic structure of an object.

Looking at GSMArena’s review of the V30, they include this graph:

The above graph stops just shy of 20KHz. That means GSMArena is evaluating phones with a 16bit/44.1K test file. Repeating my V30 chart from earlier in this article…

If you click the above image to enlarge, you’ll see this graph stops well past 80KHz. The V30 is capable of faithfully reproducing 32bit /192K audio. It should be tested closer to that scale, even if it is gross overkill for almost everyone’s music collection. This is actually what you’re paying for when you buy this phone.

GSMArena is one of the only outlets that even bothers to apply a testing methodology to audio, but their testing peaked somewhere around the HTC One M7. Improvements to phone audio since then aren’t being properly acknowledged measuring on this scale.

Here’s a chart from GSMArena comparing various phones’ audio quality.

By comparison, I’ve also analyzed output from several of these devices. See if you can spot any differences.

This causes a problem.

If a reviewer like David makes an assumption based on his subjective experience, informed by a history of reviewing NUMEROUS devices, and compares that assumption against the audio charts at GSMArena, it’s not only possible, but extremely likely that the wrong result will be delivered in the review.

That’s a shame if it misrepresents features a consumer might appreciate. We take audio for granted, but listening to music, podcasts, watching videos, and gaming are mission critical applications for many consumers. Some people might care to know which phones offer truly top tier performance in this area, and those folks will appreciate the difference between speculation and fact.