Can you hear the differences between 128/192/VBR and 320kpbs?

[ScorLibran]

You'd be surprised at how little difference the quality of your audio equipment makes on ABX testing.

In my experience, lowpass is the biggest culprit. If the gear is so cheap that it can't reproduce frequencies meeting the high frequency threshold of my own hearing (i.e., AM radio-level of sound quality), then I can clearly ABX lowpasses on the good equipment that I'd miss on the cheap equipment, simply because the equipment would fail before the encoder would going up the frequency scale.

But once you're working with relatively low-priced equipment with high frequency reproduction capabilities above you're own perceptual threshold, high-end equipment really doesn't reveal artifacts beyond the lower-end stuff the way you think it would. Everything sounds better on better equipment up to a point, as you'd expect, but there's no significant difference in the ability to detect encoding artifacts.

I specifically ABXed audio samples on my lowest-end stuff (audio card/headphones/speakers), and then on my high-end stuff. I couldn't differentiate psychoacoustic encodings from lossless references on the high-end gear any more than I could on the low-end gear.

Don't think of referencing good gear against poor gear, think of referencing lossless audio sources on good gear against compressed audio on good gear, and then lossless audio on cheap gear against compressed audio on cheap gear.

And the only way to know is to test. You'd be surprised at how much of what seem like very obvious differences is placebo. I know I was. It gave me a significant respect for double-blind testing, and for the power of placebo.

(And saying all this isn't taking a snobby position. Quite the opposite... It's saying that the guy who spent $50K on his audio system can't differentiate any more encoding artifacts than he could on a $1000 system - or maybe even on a cheaper one than that.)

[discoleo]

After reading carefully (only) the title of this topic, I found myself with a dilemma. It reads clearly as:

Can you hear the differences between 128/192/VBR and 320kpbs?

So, are there any differences between these 3 numbers/words: 128, 192, 320kbps and VBR?

I could almost swear that there are some differences.

Lets start with 128: the spelling suggests that 128 sounds like one-hundred-and-twenty-eight, that is it starts with the sound 'o'.

On the other hand, you may wonder, but 'vbr' starts with a 'v', which is unmistakebly different from the 'o' in '128'.

Then I went to the more complex task of deciphering potential differences between '128' and '192' Complex phonetical studies, including double blind hearing experiments and very advanced statistics, let me conclude that there are various sounds inside '192' which were clearly different from those inside '128'.

Of course, there is always the potential for auditory hallucinations and various superimposed noises.

I would therefore welcome any independent validation. Considering the complexities of the human brain, this is surely warranted (for a more cross platform, ... brain..., portability ).

This could be indeed a significant breakthrough in sound research, so take a break and have some fun.

The spelling of '320kbps' is left as an exercise to the interested reader.

[gaekwad2]

Interesting, however the title doesn't say Can you hear the differences between 128/192/VBR and 320kpbs if read?

Without the additional, and error prone, transformation of reading 128/192/VBR and 320kpbs sound exactly the same, at least when somebody is listening.

[ScorLibran]

[mistermeow]

I think mp3 at 128k is poop. To me that is comparable to a cheap quality tape recording. I think mp3 at 160k is my minimum quality standard. Depending on the type of music, I encode mostly mp3 224-256 more or less.

I have found Fraunhofer IIs very good for low bit rate. For example, I put all 16 Black Sabbath albums on 1 CD with fraunhofer IIs mono at 96 to 112k.

Ogg has really good quality at low bit rates, but does not have universal support like mp3.

To me, none of the lossey codecs sounds as good as the original cd, even at the maximum quality.

[gaekwad2]

prove it (at least the 128k part)

[ScorLibran]

Originally posted by mistermeow
To me, none of the lossey codecs sounds as good as the original cd, even at the maximum quality.

To you?

To you...

Originally posted by ScorLibran
The only way to know is to use double-blind, hidden-reference differential analysis using an ABX comparator.

Or you can reference the results of listening tests other people have completed. A lot of thorough information can be found that way.

Anyone making their own claims of hearing differences between different types of audio encoding are full of placebo until proven otherwise. In other words, they're speculating.

[swingdjted]

Do you REALLY need a double blind test to tell the difference between 128 and 320?

I don't think so. Really. There is some risk of and validity to the 'placebo' effect, but when the encoding of well-recorded music is that much different, a good ear should be able to tell the difference regardless whether or not you're setting up the world's most error-free research method, so if saying this makes me 'foolish' in some people's eyes so be it.

One reason it's sometimes hard to tell the difference is because the original recording gear wasn't all that great, or it wasn't used right, and therefore the original recording had problems, making the reproduction/re-encoding problems much less noticeable, because it seems harder to tell the difference between an acccurate and inaccurate reproduction of poorly recorded music.

By the way, haven't we done this thread a few times before?

[ScorLibran]

Originally posted by swingdjted
Do you REALLY need a double blind test to tell the difference between 128 and 320?

You'd be greatly surprised.

I didn't think so, either, until I tried it.

In my testing experience, a good LAME reference encoder working @ 128kbps is over 95% perceptually transparent for me. (Meaning over 95% of samples encoded at - or averaging - 128kbps were indistinguishable from uncompressed sources.)

It's not a matter of "sounding foolish" or not. It's simply a matter of "how do you know?" I was sure I needed at least 256kbps (or high-bitrate VBR modes in this range) until I tried using ABX. I was very surprised at the results.

Since then, I've mostly used bitrates between 128kbps and 192 kbps, depending on the format, and that's even allowing for a little headroom, "just in case". By "headroom" I mean using a slightly higher setting (10-20kbps) than what I measure as perceptually transparent.

You certainly don't "need" to do any such testing for yourself. But people who know psychoacoustic audio encoding won't take claims of encoding sound quality issues seriously unless there has been some double-blind testing.

And no, it's not just for HA. It's because you come to know and respect the power of placebo in a general way, and because you've heard all these claims before from people who "know what they hear". It's like reading an ad for a $2000 headphone cable.

[swingdjted]

I'd be more convinced if someone were to tell me it's virtually impossible to hear the difference between 256 and 320, but if I rip the same CD at 128 and 320, and had someone listen to each recording in an otherwise quiet room, they'd most likely tell the difference correctly, even if they were 'blind' to the specs. The difference in sound is difficult to describe in laymans terms, but it is evident to me and other serious listeners that a 128 recording isn't as good as something better. I just don't think it's worth the space-saving.

Try listening to the warm wood sound quality of a cello, the amazing range and variety of timbres of a pipe organ, or the crystal clarity of a piccolo trumpet's metallic harmonics ripped at 128 and 320 and you'll see what I'm talking about. It's evident to a human ear, and you probably don't even have to work hard to notice it.

[gaekwad2]

Well then it should be easy for you to pick the 128k versions here.

[discoleo]

Well then it should be easy for you to pick the 128k versions here.

I began the test, but I noted first that even the reference sounds bad. Is it a true reference???

That is, is this really the original soundfile, before any compression? Than it must be of very low quality.

I found both tracks quite distorted, especially from 26-37 s (the instrumental part). The drum blends somehow with the other instrument and I found this annoying.

The intro (0 to 5 s) is quite simple, therefore 128 and 192 kbps shouldn't make any difference. This is definetely an example of how such a test SHOULD NOT be done. I recommend in the future to strip such portions from test files, because they do NOT bring any information.

From 4.8 - 7.3 s, both files sound again equally (low complexity), BUT the overall dynamic range of the output is somehow compromised. I tried various equalizer settings (external, NOT PC), but non could really compensate. Increasing the low frequency seemed to correct some of the problems, BUT it still had an artificial touch. (Though you can probably hear these differences only on a high end system; it happens that I have one.)

Another problem with this tune is, that one instrument is superposed on a backround hiss (you hear the drums more or less like a hiss, see 11-15.6 s, and up to 20+ s), so, even if there are differences, the brain is likely to process them as a hiss. Also, because of this hiss, the additional instrument (piano?) is overshadowed, so differences are again minimized.

21 to 26 s, again very LOW complexity, therefore NOT suitable for comparisons.

The real interesting part was that I consistently disliked the 27-36s sequence, because there were so many artifacts introduced.

I used an AA+ amplifier, and, for some of the tests, an external high-end equalizer.

Then I tried ABX sample 4 vs Original and the result were 5 of 5 (p=0.031). I used only the 28.60-33.28s interval for my guess. The test was difficult, but I was able to distinguish the 2 files. Though, this misses the point. As stated previosly, I disliked even the original in this sequence.

And by the way, this thread misses the point. YOU NEED HIGH QUALITY originals, to be able to tell the difference. However, what one can ALWAYS tell, is IF the file sounds good or not, and even a 320 kbps encoding might sound bad (because of poor original, or, on some difficult file, maybe due to the encoding itself). If 2 encodings sound equally bad, that doesn't make an 128 equivalent to 320 kbps. Basta.

[gaekwad2]

The reference is -V2 -vbr-new (aka alt-preset fast standard) which produces on average slightly more than 192kbps, since the question was whether the difference between 128 and 192 was audible (and also to keep the download size down, the lossless version of that sample alone is 4.65MB).
I deliberately didn't choose a high-end sample (or a known mp3 killer sample) either since this test was about performance with regular music.

Oh, and could you attach your zipped results (.erf) file? Otherwise, since the samples are randomised, there's no way of telling which one you managed to ABX.

[rockouthippie]

Originally posted by swingdjted
but it is evident to me and other serious listeners that a 128 recording isn't as good as something better. I just don't think it's worth the space-saving.

I'm not all that serious, but I have good equipment. I can definitely tell the difference. It becomes really apparent in high complexity music.

Listen to the CD, then listen to a 128k encode. You'll notice big time..... You didn't need the blind test and it's not the placebo effect.

If you're playing the music on a car stereo or PC speakers, you might not notice.

I think 160k VBR is pretty good, but a trained ear still might notice some artifacting.

MP3 of any kind isn't remarkable. Ogg is better. Flac is perfect.

I'd use 320k MP3 if I were using MP3. Mainly that's because you might want to transcode the MP3 into a lower bitrate for portables or MP3 disks.

I usually make 64 k MP3 disks for the car, because car stereos suck anyway.

[ScorLibran]

Originally posted by swingdjted
Try listening to the warm wood sound quality of a cello, the amazing range and variety of timbres of a pipe organ, or the crystal clarity of a piccolo trumpet's metallic harmonics ripped at 128 and 320 and you'll see what I'm talking about.

I have, through an ABX comparator, many times. The result is, indeed, very surprising. And, in most cases, quite clear.

Originally posted by rockouthippie
I'm not all that serious, but I have good equipment. I can definitely tell the difference. It becomes really apparent in high complexity music.

Listen to the CD, then listen to a 128k encode. You'll notice big time..... You didn't need the blind test and it's not the placebo effect.

Classic!

(Admit it, you got that from 1999, didn't you?)

Originally posted by rockouthippie
MP3 of any kind isn't remarkable. Ogg is better. Flac is perfect.

Originally posted by rockouthippie I usually make 64 k MP3 disks for the car, because car stereos suck anyway.

You need to come sit in my car for a moment.

[mistermeow]

If you encode your entire music collection at 128k mp3, you will be sorry when you realize how lacking it is.

When I first got into encoding my collection a decade or so ago, I thought 128k was good too. I put my whole collection in 128k and a little bit later I was very sorry when I started to notice how bad it was, and had to redo the entire thing.

Most of my music is encoded from 192 to 256, rarely 160 or 320k. Some albums or types of music just need to be encoded higher to sound good. If your ears are new to this, err on the side of encoding too high. You can always convert to a lower bit rate.

Hard drives are cheap, so storage isn't an issue like it used to be. Encoding in lossless format is also advisable. A lot of people on hear like flac, I prefer Monkey Audio for lossless http://www.monkeysaudio.com/

You can always convert to a lower bit rate also to fit more music on a portable player.

[discoleo]

I ABX-ed:
- the original (named as original): this is the VBR at 192kbps (I just didn't know which sample is used as original)
- sample 4: aka 128 (Stereo)

the question was whether the difference between 128 and 192 was audible

Unfortunately, there was NO gold standard. Comparisons between tests without having a gold standard are inherently flawed (and difficult to interpret, if somebody insists on performing them). The more relevant question would be therefore, which settings make it sound better. However, a different question, and especially important for the current scenario, is:

Do both samples fail the test? [are there artifacts in all samples]

I was able to hear a (slight) difference between 128 and 192, BUT it aknowledge that it was a difficult test. I had to listen some 10 times on average to (accurately) tell the difference. BUT as I said, this misses the point. Although I consistenlty heard a difference, I could NOT tell which one is actually better (comparison was done to 3rd sample - the reference, this is how I could assign the original). However, I found both samples pretty poor for that 27-36s sequence.

So, you see, even 192kbps is NOT enough for this particular sample. While there is NOT a real difference between 128 and 192, there could be one between 192 and 256 or 320 (... or maybe even 320 will fail on this particular sample).

My point for future tests is:
- always provide gold standard (aka non-encoded sample)
- you can easily strip unnecessary sequences (like the first ~7s in this particular sample), because I do NOT expect any differences even for 64kbps
{- because of the hiss superposition, the sequence up to 20s would be difficult to encode even in 320 kbps}
- so this sample could be replaced by the 20-43s sequence, virtually halving the space requirement, and so having space for the gold standard

[gaekwad2]

Again, the test was aimed at claims that 192k was clearly and obviously superior to 128. Your description of the difference between -V2 (generally considered to be transparent except for rare killer samples) and the lowest quality 128k setting (CBR, simple stereo) doesn't sound like that's the case.

But if you want to test the mp3s vs the original, it can be downloaded from here.
(in WavPack lossless format)

Btw, I would be careful not to jump to conclusions regarding which part is easy to encode. -V5 (sample 1, the only one of the ~128k samples in which the encoder was free to use the bitrate it deemed necessary) actually uses higher bitrates for the first ten seconds than for the rest:

[ScorLibran]

Originally posted by mistermeow
If you encode your entire music collection at 128k mp3, you will be sorry when you realize how lacking it is.

Inquiring minds want to know that what they're hearing is not a factor of suggestion. But don't worry about it. If your listening needs require no evidence, then that's fine. Just know that people experienced with the real comparison of the results of psychoacoustic audio encoding aren't going to consider your suggestions realistic or informed if you're crying "infidelity" with a normally reasonable encoding format or setting without any meaningful measurement.

The easy way - which most people take, as I did for a while, myself - is to convince yourself that you're "sure" of what's measureably a guess. "I know what I hear." But that's OK if it meets your needs. It's music. It's not nuclear energy or cancer research. You can guess, expend a bunch of hard drive space, and really be OK. And your music will sound just as fine at 900kbps as it does at the much lower perceptual transparency threshold.

Originally posted by discoleo
Unfortunately, there was NO gold standard.

You don't need a gold standard for ABX testing. You can ABX 96kbps stereo from 32kbps mono, if you like, and have perfectly reliable and meaningful test results.

All you have to do is know what you've tested.

As for recorded audio, what's a "gold standard"? CD? Analog recording? Arguments have been made against those being "perfect" as well. But seriously, a lossless rip of the original CD is going to be what you need, or a verifiably lossless copy.

The testing protocol doesn't care how "perfect" the reference is. It's a differential analysis - a test of the amount of verifiably perceptual difference between two audio samples. Period.

Originally posted by discoleo
However, I found both samples pretty poor for that 27-36s sequence.

So both samples failed? An ABX test can't confirm that. If they both sound equally bad, then all you're proving is that you can't differentiate between the samples. ABX doesn't do anything except confirm or eliminate perceptual differentiation.

Now, if you're trying to determine your perceptual transparency threshold, you'll indeed need an uncompressed (or losslessly compressed) sample. As I said above, you don't need it in order to use an ABX test. But to achieve what it sounds like you're trying to, then you will.

For your particular goal, if you don't have an original lossless sample, then you're wasting your time. It's not a "gold standard" to the ABX comparator (although you'd designate it as the "reference"), but you can consider a lossless sample to be a "gold standard" for your own needs. If you have no means of acquiring a source-to-lossless instance, then you can only live with what you have. Just don't try to transcode it.

I hope this helps.

[discoleo]

You don't need a gold standard for ABX testing. You can ABX 96kbps stereo from 32kbps mono, if you like, and have perfectly reliable and meaningful test results.
...
ABX doesn't do anything except confirm or eliminate perceptual differentiation.

I always want to know, which compression is still transparent for me, so I need the original file (aka the gold standard). As pointed out, I did not like the 192kbps sample, either. [It could be, that the original CD was broken, too, BUT I cannot judge that without the proper sample.]

Because I rip only CDs, the gold standard for me is the CD (or a lossless compression of a CD track). Others may choose different gold standards, but it is my personal belief, that most will opt for the CD. [Yes, there are even (original) CDs that don't sound good.]

Of course, you can ABX 96 kbps from 32 kbps, BUT this misses the point (and by the way, the results are NOT perfectly reliable).

Obviously, when both encodings fail, that does NOT make the lower-bitrate encoding a suitable replacement for the higher-bitrate encoding. It actually says me to go for an even higher bitrate.

{ // NOT really important
In statistical terms, you need always a gold standard to reliably compare 2 test methods.

Taking the statistical approach: if one compares 2 methods (or tests) and has NO gold-standard, the sensitivity and specificity of the test will be overestimated. You don't know however, by how much. I would also tend to compare both samples against the gold standard and then perform a receiver operating characteristics curve (so called ROC-curve) analysis. [with cutoffs set for various degrees of dissimilarity between one sample and the gold standard]

For the test results to be valid, you need a proper sample size, too. Ideally there should be 20-30 samples per music genre, which is not very realistic. If you want to code only rock, it gets easier, but IF you want something applicable to every music genre, consider 200 samples as a good starting point (and NOT just one sample). [Because of the ordinal nature of the results, the statistics used must be a rank-test (or some other non-parametric test), so, for adequate power, one has to choose even a larger population size.]

If you wish to study more than one listener, things get a little bit complicated. I would then suggest at least 30 listeners. To do this, you need first to use the kappa-statistics (for inter-observer variability). Thereafter build a multivariate model (but the population size must be increased) and test the model using ANOVA.

I feel that there is NO rigurous study on this issues and that actually everyone talks more or less from the gut.

} // END Supplimentary Information

My concern when I encode a music track is to have the most accurate encoding. This is (almost) never 128. Even 192 fails sometimes. I remember one track that failed for 320 kbps (though this happens rarely and is only of marginal significance, BUT probably some jazz fans think a little bit different on this - luckily, I'm NOT a jazz fan ).

[ScorLibran]

Originally posted by discoleo
Because I rip only CDs, the gold standard for me is the CD (or a lossless compression of a CD track). Others may choose different gold standards, but it is my personal belief, that most will opt for the CD. [Yes, there are even (original) CDs that don't sound good.]

Everyone does. I think I forgot to put the smiley in there.

edit: No. it's there.

Originally posted by discoleo
Of course, you can ABX 96 kbps from 32 kbps, BUT this misses the point (and by the way, the results are NOT perfectly reliable).

The results are perfectly reliable. The test will perfectly tell you what you can and cannot differentiate.

An ABX tool is a perceptual comparator. What YOUR needs are will determine what YOU CONSIDER meaningful.

(Sorry that I'm not very good at teaching the early stuff. I provide training on differential analysis, but this is going back a ways for me.)

Originally posted by discoleo
Obviously, when both encodings fail, that does NOT make the lower-bitrate encoding a suitable replacement for the higher-bitrate encoding. It actually says me to go for an even higher bitrate.

ABX cannot confirm that the sample AND reference in one test package are both....oh, nevermind. Already asked and answered.

Originally posted by discoleo
In statistical terms, you need always a gold standard to reliably compare 2 test methods.
...
I feel that there is NO rigurous study on this issues and that actually everyone talks more or less from the gut.

You and the testing apparatus both need a reference. The testing apparatus does not care about the quality of the reference. It's a comparator. You can compare a turd to a rock, and it'll reliably tell you the level of the tester's perceptual differentiation between the two. The meaningfulness of the test output is the result of YOUR needs. So yes, YOU need a "gold standard" if you're searching for your perceptual transparency threshold - all the comparator needs to do its job reliably is a stable reference of any fixed quality.

I really think we're making the same point, just from "different directions". I run into it all the time - I've done this so long in my career that I'm having to reach back many years to explain the basics of standardized differential analysis. Lesson 2 is "What Constitutes a 'Reference'?"

And trust me, it's studied very rigorously in the professional world. A client is not going to pay me three figures an hour to implement a method that is not THOROUGHLY proven.

Originally posted by discoleo
My concern when I encode a music track is to have the most accurate encoding.

The "most accurate encoding"? Then you must use a lossless compressor.

To know you're encoding at the most efficient level (sound quality per bitrate), you need to determine your own perceptual transparency threshold using an ABX comparator by differentiating a lossless reference against several psychoacoustically encoded samples. Once you find your PTT, then encode your music at just above that level (I recommend about a 10% differential).

As for the concern of "I'm not the only person who listens to the music I encode", don't worry - in my experience, it's EXTREMELY rare for people's PTTs to be different by any meaningful amount. Except for the occassional "audiophile" who "just knows" they can hear a difference. But for us, they're just fodder for entertainment.



(BTW - I enjoy this type discussion. It's one of the key reasons I chose my field.)

[rockouthippie]

It's all in the listening for me. 320kbps seems to be just about perfect. Even 128k sounds good most of the time. For me it isn't the average clarity of 128k that's a problem. It's those specific times when the encoding fails and artifacts that I hear it clearly.

Still, it's not "OMG, that sounds terrible!." It's "Gee, that could be better".

At 320kbps I rarely hear artifacts. For bitrates below that, they become evident in normal listening.

I think part of that is my having used compression for a long time. I hear artifacts quite easily and know it doesn't sound like it's supposed to. It might even be a single note that is weird, but I notice.

If your listeners don't know what an artifact sounds like, the error may go unnoticed. The sound of artifacting is apparent and noticeable. The bitrate at which it is almost never apparent to me is 320 kbps, although it can fail too occasionally.

Maybe the blind test is flawed. A discerning ear can hear artifacting quite easily. Perhaps the test should be whether people who are trained to hear artifacts can hear artifacts in the recording.

I really, truly hate psychoacoustic compression. Gives me a headache.

[mistermeow]

After years of listening to mp3's, and then popping in the original music CD, I definately noticed. Thats what prompted me to encode my favorite albums in lossless. I still encode them in mp3 as well for compatibility and portability reasons, but its much more pleasant to hear the original sound.

[ScorLibran]

Very good stuff.

[swingdjted]

Nice edit.

[ScorLibran]

Had to add the "very good stuff". Didn't want anyone to think I didn't fully appreciate all the humor going on.




another edit:

As was said many times before, no one needs to do any sort of testing to encode their music with reasonably consistent sound quality. Common sense will usually guide you well. ("128 seems low. No problems are jumping out at me at 192, and the files are still small. I'll use that.) Referencing already-existing test results will help, too, if you want some verifiable data to go on. No work required in either case.

But I do find it interesting that people will benchmark their graphics before openly making performance claims, and then - without any testing - cry "MP3 sounds bad!" THAT'S what keeps me smiling.

[mistermeow]

When I first decided to encode my music library in the late 90's, I used Real Jukebox (.ra) at 96k, and the encodes sounded very good. After discovering Wimamp and Sonique, and mp3 support, I converted all my *.ra files to mp3 128k. I was happy for about few days, and then I really started to hear the difference in quality :P

Needless to say, I had to delete everything and start back over.

[ScorLibran]

Mmmmm... Lossy-to-lossy transcoding. Good stuff. Been there, myself.