Old 12th December 2000, 19:54   #1
Tom
Moderator
 
Join Date: Apr 2000
Posts: 4,491
If anyone is interested I threw together a test where people can go listen to various samples and then vote for which format they feel is the best. Basically a blind listening test over the internet. If you would like to take part just go to http://players.shoutclub.net/ltestintro.php3 and follow the directions. (I'll warn you ahead of time there are 25 MB of waves to download).

If you want to keep track of the results here is the page for that: http://players.shoutclub.net/results.php3
(NOTE: Results are delayed by one day)

Okay that is all from me, I'll shut-up now.

Tom
http://players.shoutclub.net
Tom is offline   Reply With Quote
Old 12th December 2000, 21:53   #2
Flynnz
Major Dude
 
Join Date: Sep 2000
Location: mostly not here anymore
Posts: 855
Send a message via ICQ to Flynnz Send a message via AIM to Flynnz Send a message via Yahoo to Flynnz
Neato.. I plan to participate within the next two or three days. I'll have cable by tomorrow night.

[This message has been edited by Flynnz (edited December 12, 2000).]
Flynnz is offline   Reply With Quote
Old 12th December 2000, 23:42   #3
randman
Guest
 
Posts: n/a
I really couldn't tell a difference between any of them. And I can usually tell a difference between a 128kbps and a 160kbps mp3 encoded with LAME.

But I'm 45 and I've been listening to rock and roll since the 60's. Maybe my hearing is starting to go.

  Reply With Quote
Old 13th December 2000, 03:30   #4
e144539
Senior Member
 
e144539's Avatar
 
Join Date: Sep 2000
Location: Under the bridge of town
Posts: 233
Send a message via ICQ to e144539 Send a message via AIM to e144539
So how did you make the test?
encode them and then decode them back to wav's?
Doesn't that kinda defeat the whole purpose?
yes they loose info when you decode them but then the encoders guess what info went there.
A wav is a wav is a wav.

You should definitely have the option
"could not tell any difference"

------------------
Life- [{lîf} plaural lives {lîvz}] n. sexually transmitted, always fatal
e144539 is offline   Reply With Quote
Old 13th December 2000, 05:08   #5
Tom
Moderator
 
Join Date: Apr 2000
Posts: 4,491
I'll try to answer everyone here:
Flynnz,
Ahhh the joy of broadband, once you get it you'll wonder how you survive without it. Unfortunately I will be going back home over Christmas and will have to do without my dsl for a while (sad days are ahead).

randman,
That is okay, that is what this test is for. It is to see if there is a format that is better than the other or are they all perceived equally. (It might help to try with a good set of headphones if possible).

e144539,
The tests were done by encoding the original wav file into each format and then decoding it back to wav. This was done because the wav format will not allow the user to tell what each format is at a glance (as it would if I put .mp3 .ogg and .mp+ files on there). As for defeating the purpose I am not sure what you mean. The reason they were converted back to wav was so the people taking the test could not easily distinguish between .mp+ .mp3 and .ogg by sight, and the wav format retains all the information so there will not be a quality difference by doing so. That way they vote for what their ears tell them, not because of what they have heard about each format.
"yes they loose info when you decode them but then the encoders guess what info went there.
A wav is a wav is a wav."
I am not sure what you mean here. When the files are encoded each of the encoders runs them through a psychoacoustical algorithm and some data is thrown out, then some type of compression is applied to the remain file. The decoder reads that file and reproduces the sound (only the remaining part, the stuff thrown out by the encoder is lost forever). By converting these files to wav I was able to capture only the remaining data in a wav file and then these are compared (they only have the information that the encoder let through).
"You should definitely have the option
"could not tell any difference""
Unfortunately it is too late to add that to the tests. People have already cast there vote and it wouldn't be fair to add that in right now.
Tom is offline   Reply With Quote
Old 13th December 2000, 05:22   #6
randman
Guest
 
Posts: n/a
Sorry, I should have noted that I was using headphones. (Yamaha RH5MA. I think they're pretty good).

[This message has been edited by randman (edited December 12, 2000).]
  Reply With Quote
Old 13th December 2000, 05:39   #7
e144539
Senior Member
 
e144539's Avatar
 
Join Date: Sep 2000
Location: Under the bridge of town
Posts: 233
Send a message via ICQ to e144539 Send a message via AIM to e144539
ok I mean when you encode a file you go from say 40MB to say 4MB.
when you decode that mp3, mp+, ogg, whatever back into a file *.wav, you put those missing parts back in and you have a file that close to the original 40MB, you have essentially undone everything that the encoders did by decoding them.
If you want this test to work then you would have to record the output of each file while its playing in its *.mp3, *.ogg, ect. form, using a program that records whatever is going through the sound card (analog signal).
I guess there is a better way if you have the equipment, you could connect two sound cards digitally (it would probably be necessary to have two computers), one for playback the other to record. Then just connect through something like S/PDIF or if you got two verry nice systems you might have optical S/PIDF.
All I'm saying is that there isn't an ear on the planet that can hear any difference in those files. Don't kid yourself.

------------------
Life- [{lîf} plaural lives {lîvz}] n. sexually transmitted, always fatal
e144539 is offline   Reply With Quote
Old 13th December 2000, 05:44   #8
Somebody
Major Dude
 
Somebody's Avatar
 
Join Date: Oct 2000
Posts: 1,531
*watches Tom explain to e144539 about the decoding process*


btw, those missing parts do not go back

[This message has been edited by Somebody (edited December 12, 2000).]
Somebody is offline   Reply With Quote
Old 13th December 2000, 05:53   #9
e144539
Senior Member
 
e144539's Avatar
 
Join Date: Sep 2000
Location: Under the bridge of town
Posts: 233
Send a message via ICQ to e144539 Send a message via AIM to e144539
lol
e144539 is offline   Reply With Quote
Old 13th December 2000, 05:55   #10
e144539
Senior Member
 
e144539's Avatar
 
Join Date: Sep 2000
Location: Under the bridge of town
Posts: 233
Send a message via ICQ to e144539 Send a message via AIM to e144539
<BLOCKQUOTE><font size="1" face="Verdana, Arial">quote:</font><HR>Originally posted by Somebody:
btw, those missing parts do not go back<HR></BLOCKQUOTE>

then how do you explain the size of the decoded *.wav's?


------------------
Life- [{lîf} plaural lives {lîvz}] n. sexually transmitted, always fatal
e144539 is offline   Reply With Quote
Old 13th December 2000, 05:57   #11
Somebody
Major Dude
 
Somebody's Avatar
 
Join Date: Oct 2000
Posts: 1,531
<BLOCKQUOTE><font size="1" face="Verdana, Arial">quote:</font><HR>Originally posted by e144539:
then how do you explain the size of the decoded *.wav's?


<HR></BLOCKQUOTE>

It's essentially a wave file being recorded manually as you explained above.

Somebody is offline   Reply With Quote
Old 13th December 2000, 06:16   #12
e144539
Senior Member
 
e144539's Avatar
 
Join Date: Sep 2000
Location: Under the bridge of town
Posts: 233
Send a message via ICQ to e144539 Send a message via AIM to e144539
good point
ok I was wrong, it really won't matter how you record the wav's.

but I still think a high-bitrate mp3's and the wav's that they are decoded to don't sound the same when you fill in the gaps.
e144539 is offline   Reply With Quote
Old 13th December 2000, 06:16   #13
JMPZ
Member
 
Join Date: Nov 2000
Location: Melrose Park, IL
Posts: 77
Send a message via ICQ to JMPZ Send a message via AIM to JMPZ
Dadgumit! I can't stand it!

When any compressed file is decoded, it is converted to a 44.0Khz, 16-bit Stereo PCM data stream (in other words, a streaming wav file) and sent to the sound card. When an mp3 is converted back to a wav file, THE EXACT SAME DECODING OCCURS!!! Instead of being sent to the sound card, it is sent to a file. Playing the wav file and listening to the mp3 realtime produce the EXACT same thing.

Another benefit to using wav files is that it assures equal quality. All decoders are not equal. Some decoders have timing issues, or quantization issues. The truest quality test would use the reference decoder from the encoder for each file type respectively to show you the full capabilities of the format.

OK. I feel better now. Back to my pointless 4 page essay.

Have a great day y'all!
JMPZ is offline   Reply With Quote
Old 13th December 2000, 06:37   #14
e144539
Senior Member
 
e144539's Avatar
 
Join Date: Sep 2000
Location: Under the bridge of town
Posts: 233
Send a message via ICQ to e144539 Send a message via AIM to e144539
Why don't people just accept real profesionals findings http://www.math.auth.gr/~axonis/studies/audio.htm
and agree that there is no audible difference between a high quality mp3 (128kb/s on the page, 192 for my ears, and 200+ for people who think you can't have "true" stereo without at least 100 per channel ) and the original file wav if from a cd, if your talking dvd thats another story.

all-righty-o, ya'll come back now ya ear


------------------
Life- [{lîf} plaural lives {lîvz}] n. sexually transmitted, always fatal
e144539 is offline   Reply With Quote
Old 13th December 2000, 06:59   #15
Jay
Moderator Alumni
 
Jay's Avatar
 
Join Date: May 2000
Location: Next Door
Posts: 8,942
that is truely based on perception, and how the format handles certain frequencies. (which I believe this test is proving) MP3 at 160 still has a loss, in fact at the website the latest results show that MP+ is doing better than MP3 VBR. Pretty cool, now I know something about my listening tastes

I think that Fatboy.wav really spells it out. You can here a lot of differences between formats in that one.

------------------
KXRM
http://www.shoutclub.com/
"Broadcasting with the Web in Mind"
Jay is offline   Reply With Quote
Old 13th December 2000, 14:10   #16
Winsane
Moderator Alumni
 
Winsane's Avatar
 
Join Date: Jun 2000
Location: U.S. (eastern time zone)
Posts: 2,598
Also next time around, I'd include the option to select more than one as equal. On a few of those tests, I was able to eliminate one or two of the files, but couldn't distinguish between the remaining ones. And I didn't want to give one a vote that the other deserved just as much, so I just skipped those tests (and so my votes for the ones I could distinguish weren't counted). Also, do you tell the voter which they listened to after the final vote? In case not: I expect which aspects of the sound each format is good and not so good at varies, and that which aspects of the sound each listener cares the most about also varies, and so while most people might prefer one format, one person might prefer a different one, and it would be nice for that person to know which format they voted for so that they can then proceed to use that format for themself.
Is there anything I should be listening for in the hex test? I found that one the most difficult to distinguish between the files.
Winsane is offline   Reply With Quote
Old 13th December 2000, 20:50   #17
Anish
Guest
 
Posts: n/a
Would it be possible to include Psytel's mpeg 4 AAC and K&K Research's TAC formats in the review? There may be some restrictions (copyright etc). I primarily want to choose between the above two and mp+ as the format to encode my songs, but my present hardware isn't up to the mark to do a proper comparison.
  Reply With Quote
Old 14th December 2000, 19:35   #18
Tom
Moderator
 
Join Date: Apr 2000
Posts: 4,491
First off thanks guys for explaining that to e144539.

Winsane
Yes hex.wav is one of the more difficult ones. That one shows how the encoders deal with stereo-separation and transient detection. (In the original hex.wav there is a slight ringing positioned slightly to the left. After the encoded/decode the ringing has pre-echoes (swishes) and it is in the middle.) I do not tell the voter which ones they chose. This is to prevent people from taking the results and then voting for the format that they like best (unfortunate that I must do this but if I didn't the system would be abused).

Anish
In the future I will probably do some more of these, and the formats you mentioned will most likely be included. I will have to have a long talk with K&K about it before I do anything though, so who knows what will become of it. Not sure about the legal issues, but I do not think possessing a wav file that was encoded and decoded with a format is illegal. Of course how they got there may be, unless I have them "emailed" from a place where the patents do not exist.

Tom
Tom is offline   Reply With Quote
Reply
Go Back   Winamp & SHOUTcast Forums > Community Center > General Discussions

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump