64 Kbit/s Hi-lp Same Quality As 128kbit/s Mp3?

[greenmachine]

I've been wondering about Sony's claims 64 kbit/s Atrac3+ Hi-LP being comparable quality to 128 kbit/s mp3, so i compared it directly. Since it seems to be nowhere close to the mp3, i've compared the mp3 with 132 kbit/s Atrac3 LP, which seems to be still (slightly?) worse. Am i nuts?

a.) La Grange mp3 128 kbit/s

b.) La Grange Atrac3 132 kbit/s

c.) La Grange Atrac3+ 64kbit/s

La Grange Original

Edited June 8, 2005 by greenmachine

[greenmachine]

oops, wrong forum, please move

[Ishiyoshi]

Moving to "Technical Topics."

[bug80]

I voted c < b < a, altough I had some trouble testing the samples, because it was not a blind test. It is very hard to make a fair comarison if you know which one is which. The atrac samples both had severe problems with the drums, more than the mp3 file.

Does anyone have a link to the page where Sony makes the claim (including scientific evidence, hehe). I'm very curious what the test conditions were, which encoders they used (probably not LAME), background of the subjects, etc.

A company making such strange claims is not new of course, WMA @ 64 kbs is also advertised as being comparable to MP3 @ 128.

[The Low Volta]

Does anyone have a link to the page where Sony makes the claim (including scientific evidence, hehe). I'm very curious what the test conditions were, which encoders they used (probably not LAME), background of the subjects, etc.

←

I found this reference recently in a HD5 advert but of course it didn't mention any technical evidence...

Volta

[greenmachine]

I've decided not to make it a blind test because the difference seems to be too obvious (at least with good equipment).

I've used the Audition 1.5's Fraunhofer mp3 encoder with a lowpass at 15.8 kHz, m/s joint stereo enabled, intensity joint stereo disabled, cbr. It gave slightly better results than LAME 3.96 --alt preset cbr 128 in my opinion. LAME seems to be slightly worse for low bitrates, but excellent for higher bitrate vbr (--alt preset standard/extreme).

I'd really like to see a medium bitrate Atrac 3+ (about 128 kbit/s) since they claim it to work more efficiently than Atrac3. Atrac3 at this bitrate is often close to acceptable quality.

Edited June 7, 2005 by greenmachine

[dyonisus]

Here's what I found Sony's claim .

[greenmachine]

Does anyone have a link to the page where Sony makes the claim (including scientific evidence, hehe). I'm very curious what the test conditions were, which encoders they used (probably not LAME), background of the subjects, etc.

←

http://www.sony.net/Products/ATRAC3/tech/lab/index.html

[physya]

I've been wondering about Sony's claims 64 kbit/s Atrac3+ Hi-LP being comparable quality to 128 kbit/s mp3, so i compared it directly. Since it seems to be nowhere close to the mp3, i've compared the mp3 with 132 kbit/s Atrac3 LP, which seems to be still (slightly?) worse. Am i nuts?

a.) La Grange mp3 128 kbit/s

b.) La Grange Atrac3 132 kbit/s

c.) La Grange Atrac3+ 64kbit/s

←

okay. Attached are FFTs of the MP3 (a) vs ATRAC64 [c] in linear and logarythmic scales. I really do not see much of a difference. The ATRAC is a little bit cut at higher frequencies (as expected).

Perhaps, the difference you heard was more psychological. Let's do a blind test. (and with something having higher frequencies)

Edited June 7, 2005 by physya

[greenmachine]

okay. Attached are FFTs of the MP3 (a) vs ATRAC64 [c] in linear and logarythmic scales. I really do not see much of a difference. The ATRAC is a little bit cut at higher frequencies (as expected).

Perhaps, the difference you heard was more psychological. Let's do a blind test. (and with something having higher frequencies)

←

That's indeed very nice to view, but doesn't say anything about perceived sound quality. You can't judge sound quality by looking at some graphs, sorry. I'll do a blind test if it satisfies you.

[physya]

That's indeed very nice to view, but doesn't say anything about perceived sound quality. You can't judge sound quality by looking at some graphs, sorry. I'll do a blind test if it satisfies you.

←

What this graph shows is that output amplitudes on particular frequncies are the same, so physically these two waves are almost identical.

It could be that you can hear those subtle differences, but to my ears these files also sound very similar - except for high frequency cutoff in ATRAC file. (Although I don't have hi-fi sound - I use Audigy 2 sound card with creative 5+1 speakers)

still, please post a couple of unidentified file objects (UFOs) for the blind test.

Edited June 7, 2005 by physya

[greenmachine]

Did a blind test, the comparison between Atrac3+ @ 64 kbit/s and mp3 was very easy:

WinABX v0.42 test report
06/08/2005 01:03:16

A file: C:\temp\La Grange 64+.wav
B file: C:\temp\La Grange mp3 128.wav

01:03:38    1/1  p=50.0%
01:03:41    2/2  p=25.0%
01:03:43    3/3  p=12.5%
01:03:45    4/4  p=6.2%
01:03:48    5/5  p=3.1%
01:03:50    6/6  p=1.6%
01:03:52    7/7  p=0.8%
01:03:54    8/8  p=0.4%
01:03:56    9/9  p=0.2%
01:03:58  10/10  p< 0.1%
01:04:03  11/11  p< 0.1%
01:04:06  12/12  p< 0.1%
01:04:08  13/13  p< 0.1%
01:04:10  14/14  p< 0.1%
01:04:12  15/15  p< 0.1%
01:04:14  16/16  p< 0.1%
01:04:17  17/17  p< 0.1%
01:04:20  18/18  p< 0.1%
01:04:22  19/19  p< 0.1%
01:04:24  20/20  p< 0.1%
01:04:28  21/21  p< 0.1%
01:04:31  22/22  p< 0.1%
01:04:33  23/23  p< 0.1%
01:04:35  24/24  p< 0.1%
01:04:37  25/25  p< 0.1%
01:04:40  26/26  p< 0.1%
01:04:42  27/27  p< 0.1%
01:04:43  28/28  p< 0.1%
01:04:46  29/29  p< 0.1%
01:04:48  30/30  p< 0.1%
01:04:50  31/31  p< 0.1%
01:04:56  32/32  p< 0.1%
01:05:00  33/33  p< 0.1%
01:05:03  34/34  p< 0.1%
01:05:05  35/35  p< 0.1%
01:05:07  36/36  p< 0.1%
01:05:09  37/37  p< 0.1%
01:05:11  38/38  p< 0.1%
01:05:14  39/39  p< 0.1%
01:05:17  40/40  p< 0.1%
[/code]

[greenmachine]

The second one between Atrac3 @ 132 kbit/s and mp3 was significantly harder, but there's still at least a slight difference:

WinABX v0.42 test report
06/08/2005 01:30:47

A file: C:\temp\La Grange 132.wav
B file: C:\temp\La Grange mp3 128.wav

Start position 00:00.0, end position 00:24.2
01:31:54    1/1  p=50.0%
01:32:21    2/2  p=25.0%
01:33:13    3/3  p=12.5%
01:34:01    4/4  p=6.2%
01:34:25    5/5  p=3.1%
01:34:55    6/6  p=1.6%
01:35:34    7/7  p=0.8%
[/code]

[greenmachine]

What this graph shows is that output amplitudes on particular frequncies are the same, so physically these two waves are almost identical.

still, please post a couple of unidentified file objects (UFOs) for the blind test.

←

Question: If you took random noise and shaped it identical to your graph, would it sound identical to the music sample, or just like shaped noise?

There's no need to post additional samples, just pick up the given and put it into some abx machine for blind comparison:

http://www.pcabx.com/

http://www.kikeg.arrakis.es/winabx/winabx.zip

http://www.kikeg.arrakis.es/winabx/readme.txt

Edited June 8, 2005 by greenmachine

[greenmachine]

some facts about sony's tests:

-in one test they propably used the same cooledit/audition fraunhofer mp3 encoder that i've used, in the other they used musicmatch jukebox v7.5 for encoding (no idea which encoder is used there)

-they don't describe which settings they used for the mp3 encoders

-mp3 performance at very low bitrates (64 kbit/s and below) is indeed very disappointing, even if properly set, there are other codecs which perform better, including Atrac3+

-Sony itself chose the majority of the test samples

[physya]

Question: If you took random noise and shaped it identical to your graph, would it sound identical to the music sample, or just like shaped noise?

http://www.pcabx.com/

http://www.kikeg.arrakis.es/winabx/winabx.zip

←

Of course it would still sound as noise. The graph is just a power spectrum and has NO temporal information (amplitude vs. time). However, compression algorithms do just that - they simply through away some (least important) frequencies. The question is how it is done. Therefore you can estimate quality of the compression by comparing the power spectra of the same piece of music.

I am not familiar with ABX testing and from what I've read on their web page - does not sound too scientific.

[The Low Volta]

I am not familiar with ABX testing and from what I've read on their web page - does not sound too scientific.   

←

well, first of all, ABX is a method, not a company or something, so "from the ABX-website" makes as much sense as saying "from the MP3-site".

Second, if measuring and numbers are the only things hat would make something scientific...indeed ABX isn't, as it is comletely driven by the human element... but for something so subjective as "sound quality" (I have to agree with greenmachine that you can't express this in a graph or two) then you NEED this human element, as the human IS the end user...I don't give a rat's behind if my PC thinks one codec sounds better than the other if I myself can't hear it So ABX-tests were designed to suppress the subjective bias when performing listening tests, if you know you're listening to an MP3, your feelings towards MP3 will always color your evaluation. Well, ABX works for eliminating that bias. Ok, sure there still is another subjective human element "measuring" the quality, but now he's blind-testing and isn't that the whole idea!

So whether it is scientific in your book or not (it is in my book)...just try it. If you honestly can't hear a difference that means that either the fragments sound alike or your ears can't hear the difference, doesn't matter which, as for you the codecs sound alike and you can save space by using the one with the lowest bitrate... if on the other hand you want to give a general opinion that works for most people, then get a lot of people to perform the ABX-test and do some statistics on the results! But if you think you will convince anyone with a graph (no matter how correct it is) on such a subjective subject as sound quality...then ABX-tests wouldn't have been designed at all.

[bug80]

Of course it would still sound as noise. The graph is just a power spectrum and has NO temporal information (amplitude vs. time). However, compression algorithms do just that - they simply through away some (least important) frequencies. The question is how it is done. Therefore you can estimate quality of the compression by comparing the power spectra of the same piece of music.

Not entirely true. The most obvious artifacts in the samples presented here are things like pre-echo and ringing, thus temporal artifacts. It is very likely that these don't show up in a FFT analysis. Compression involves a lot more than just throwing away less important frequencies, by the way.

In the field of audio compression (and in the field of scientific acoustics in general), it is general concensus to perform listening tests to test the perceived quality of a codec or any other algorithm. A spectrum analysis (or, closely related, a correlation analysis) may only be used to couple the results of the listening tests to the mathematical properties of a signal.

[bug80]

some facts about sony's tests:

-in one test they propably used the same cooledit/audition fraunhofer mp3 encoder that i've used, in the other they used musicmatch jukebox v7.5 for encoding (no idea which encoder is used there)

-they don't describe which settings they used for the mp3 encoders

-mp3 performance at very low bitrates (64 kbit/s and below) is indeed very disappointing, even if properly set, there are other codecs which perform better, including Atrac3+

-Sony itself chose the majority of the test samples

←

I believe musicmatch uses its own encoder (not the best one around).

psyhya, I've taken a look at your graphs and besides the fact that they don't say much about perceived quality, I see a lot of differences there. Some frequencies differ with a factor of 10 in amplitude, or even higher!

[physya]

well, first of all, ABX is a method, not a company or something, so "from the ABX-website" makes as much sense as saying "from the MP3-site".

←

I meant those sites that greenmachine mentioned in the post above.

Of course, you're right about human element etc. By the way, those ABX people specifically mention the "trained listeners" and, indeed, some people will hear what other cannot. I presume that someone with absolute hearing will find the differences for any compression method. However, when you discuss sound of particular instruments, it is not clear what you compare to.Some encoders may in fact enhance these instruments and make them sound better, but I think the best encoder is the one that most accurately represents the original.

... and I am not dismissing the ABX method - I downloaded a program and will give it a try when I have time. "Not scientific" is not negative in this context - it is just not scientific, because it involves human element as a measurement device.

as for encoders - it IS all about throwing away non-imprortant frequencies and elimination of redundant information in the stream. The question is how it is done (both ways - encoding and decoding).

[The Low Volta]

"Not scientific" is not negative in this context - it is just not scientific, because it involves human element as a measurement device.

I studied chemistry/biochemistry at uni for 6 years and now study communication sciences... I do know something about the scientific methods and I think that this is really shortsighted (even though your opinion is shared by a lot of ppl)... Every and I mean EVERY measurement involves a human element! Otherwise we wouldn't be able to know what was measured.

Let me explain... If you measure the lenght of something, you use a ruler... but the ruler was made by humans, or by machines, programmed by humans, the eye reading the ruler is human or again from a machine programmed by humans, the one interpreting the measurement is human or (you get it) a machine programmed by humans, the one presenting the results...well I won't repeat it again but... at each stage a human element. Heck, even the meter was a human invention (like dB, Hz, ...)

... So you can never "eliminate" the human element from any measurement! As we all are humans, we can never know anything about the universe but through the eyes of humans. (there are even some that take this to the level that there is no reality except for that made by human interpretation) The only problem that this human element poses for scientific research is that one can't rely on one measurement to reveal the truth... only through statistical processing (by humans or PC's programmed by H's) of large numbers of tests, one can come closer to an utopical universal truth that will allow us to pose statements that will work for most situations...

as for the "trained listeners"-part, I would like to see someone untrained use some of the apparati whe used in the labs, sure there are degrees in which one can be trained to use the instrument, but that doesn't say anything about the scientific character of the instument used..

But all this still is besides the point, as I will show further...

as for encoders - it IS all about throwing away non-imprortant frequencies and elimination of redundant information in the stream. The question is how it is done (both ways - encoding and decoding).

←

and who would choose which frequencies are redundant? A computer (programmed by humans) or humans themselves... encoding/decoding is about listening, not only about measuring frequencies...so ANY test evaluating the quality of codecs should contain blind listening tests (which can be just as scientific as measuring light intensity or anything else)...

[bug80]

... and I am not dismissing the ABX method - I downloaded a program and will give it a try when I have time. "Not scientific" is not negative in this context - it is just not scientific, because it involves human element as a measurement device.

I study physics, currently working for my master degree on acoustics/sound control and I can assure you ABX testing is considered to be scientific. In fact, I used an ABX test with different subjects as a part of my research. Why? Because I had to test different algorithms that were almost identical in a mathematical sense and guess what: it turns out that people hear a difference!

[greenmachine]

Added the uncompressed file for comparison.

[greenmachine]

Thanks Volta, i think you're great in explaining correlations.

Did you read 'Die Welt als Wille und Vorstellung' by the way?

Here's something to think about:

Suppose past is infinite and future is infinite and let us think on

eternal past and eternal future.

If time go on proceeding towards future and finally reached to eternal

future, however long it may be, the distance of time between the

present and the eternal future becomes a finite specified value,

doesn't it ?

And a finite specified value is contrary to infinite, isn't it ?

This means that time can never reach to eternal future eternally.

In the same manner, when time go back to past, time can never reach to

eternal past eternally.

Then, if time has come from infinite past, how can it be possible that

it has reached to the present ?

[The Low Volta]

that makes me think we need to open a philosophy-thread here on MDCF

[sushi]

...

[bug80]

Thanks Volta, i think you're great in explaining correlations.

Did you read 'Die Welt als Wille und Vorstellung' by the way?

Here's something to think about:

Suppose past is infinite and future is infinite and let us think on

eternal past and eternal future.

If time go on proceeding towards future and finally reached to eternal

future, however long it may be, the distance of time between the

present and the eternal future becomes a finite specified value,

doesn't it ?

And a finite specified value is contrary to infinite, isn't it ?

This means that time can never reach to eternal future eternally.

In the same manner, when time go back to past, time can never reach to

eternal past eternally.

Then, if time has come from infinite past, how can it be possible that

it has reached to the present ?

←

Hm. If the eternal future is at an infinite time, it will take an infinite number of time steps to reach eternal future and thus we will never reach it. My $.02

Edit: interesting last comment about the inifite past. Thinking about the big bang, the past may not be infinite. The future may not be either.

Edited June 8, 2005 by bug80

[The Low Volta]

I really think the mods should throw these last posts in a philosophy-thread, as we are going way off topic here

Suppose ... future is infinite and let us think on... eternal future. If time go on proceeding towards future and finally reached to eternal future...

hm, this is a bit out of my league but: isn't infinite defined as unmeasurable and therefore unreachable when applied to measurable units like distance and time

This means that time can never reach to eternal future eternally.

this would still be true I guess

In the same manner, when time go back to past, time can never reach to eternal past eternally.

call me ignorant, but why wouldn't we be able to see an infinite future stemming from a finite past... if, at any point in time (no matter 'how close' to the infinite future) you choose to look forward, you'd still see an infinite future (by definition) and at the same time you would be able to look back AND measure the time between that point and the finite past...

Then how can it be possible thatitime has reached to the present ?

←

this is the easiest part to answer: time has never reached 'the present', time has never left 'the present', as time can only be 'the present'. We can not know any other time but now, as any other either does not longer exist or has not yet existed...which could be extended ad infinitum until we realize that there is no such thing as time, there is only now and it has just passed by...

Volta the Enlightened

[greenmachine]

...which would open the doors to uninhibited hedonism.

I wonder why nobody voted for c<a=b, are you all biased by my first post, eh?

[greenmachine]

And for those who think it needs extremely high quality equipment to hear a difference, just compared it through my crappy built-in laptop speakers, it was still very easy:

WinABX v0.42 test report
06/08/2005 16:39:57

A file: C:\temp\La Grange 64+.wav
B file: C:\temp\La Grange mp3 128.wav

Start position 00:00.0, end position 00:24.2
16:40:38    1/1  p=50.0%
16:40:41    2/2  p=25.0%
16:40:44    3/3  p=12.5%
16:40:46    4/4  p=6.2%
16:40:49    5/5  p=3.1%
16:40:51    6/6  p=1.6%
16:40:57    7/7  p=0.8%
16:41:00    8/8  p=0.4%
16:41:02    9/9  p=0.2%
16:41:04  10/10  p< 0.1%
[/code]

[physya]

I studied chemistry/biochemistry at uni for 6 years and now study communication sciences... I do know something about the scientific methods and I think that this is really shortsighted (even though your opinion is shared by a lot of ppl)... Every and I mean EVERY measurement involves a human element! Otherwise we wouldn't be able to know what was measured.

←

Wow! What an interesting discussion came out of a simple question. But, alas, I don't agree regarding that human element. The scientific measurement is such that if I measure something and, say some alien green men do their measurements, they'll find exactly the same as I did (of course, it might require to convert our meters into their feet). However, if we would evaluate music, I really doubt that they would agree with us.

Nevertheless, I totally agree that quality is not determined only by the frequency response. It is just a very important objective characteristic. (e.g. see the discussion on how 910 reproduces mp3 cutiing the higher frequencies). That is - if it is very different then you can talk about an intrinsic flaw in the algorithm. So, my point is that ATRAC64 retains quite nicely the original characteristics and the difference in sound quality is in how you decode the stream. It is possible that it actually adds some information during decoding (to compensate for a preceeding high compression).

P.S. ... and no, time is not infinite, because it is actually determined by the gradient of the entropy. It begins with a big bang and will either end with another one or stand still when the thermal death of the universe is reached. (I am in physics for some 20+ years by now... )

Edited June 8, 2005 by physya

[bug80]

Wow! What an interesting discussion came out of a simple question. But, alas, I don't agree regarding that human element. The scientific measurement is such that if I measure something and, say some alien green men do their measurements, they'll find exactly the same as I did (of course, it might require to convert our meters into their feet). However, if we would evaluate music, I really doubt that they would agree with us.

And that is why blind tests are so important. Compression schemes aim at the "best" results given a certain (average) bitrate. "Best" must be defined for the "avarage" listener. So, if our world gets invaded by aliens (man, where is this discussion going to? ), they must be involved in the blind tests too. Again, I'm into acoustics and blind testing is a valid scientific method and for most problems related to audio it's even the only way to go. I've encountered some examples of this during my last research. If you find this interesting you should take a look at hydrogenaudio.org. It is a place with lots of information on audio compression and blind testing.

It is possible that it actually adds some information during decoding (to compensate for a preceeding high compression).

Possible, but it is a general rule that decoders only decode, nothing more, except for things like dithering and resampling, if necessary. Everything else should be done by the encoder.

[physya]

And that is why blind tests are so important. Compression schemes aim at the "best" results given a certain (average) bitrate. "Best" must be defined for the "avarage" listener. So, if our world gets invaded by aliens (man, where is this discussion going to?  ), they must be involved in the blind tests too. Again, I'm into acoustics and blind testing is a valid scientific method and for most problems related to audio it's even the only way to go.

←

Well, no offense, but there is a difference between exact sciences and all other sciences. I only meant the former. Our alien brothers could have ultrasonic hearing and be deaf at lower frequencies....

I agree that after physical properties are shown to be comparable (spectrum etc) you need to run the acoustic tests to actually enjoy the music.

As for decoding - it is an inverse Fourier transform (with some harmonics omited) and it does add some new information - depending on how you handle the algorithm and what precision is used. Moreover, more sophisticated decoders (as ATRAC claims to be) would actually interpolate to fill in the missing frequencies.

Anyway, 5 minutes ago I got my MZ-RH910 and now I am struggling to remove that stupid plastic wrap - should probably look for a saw...

[bug80]

Well, no offense, but there is a difference between exact sciences and all other sciences. I only meant the former. Our alien brothers could have ultrasonic hearing and be deaf at lower frequencies....

Ok, there are fields within physics that are less exact than others, psycho acoustics for example. But since compression algorithms are all about psycho acoustics, it is not very strange that their quality is tested using blind tests. In fact, the models on which these algorithms are based (treshold of hearing, temporal/spectral masking, cochlea modeling etc) are all based on actual listening tests.

I still consider physics to be an exact science, however.

I just hope that the aliens stay away for a while..

[greenmachine]

I've been interested what kind of information the codecs find dispensable and throw away, so i've messed around with an audio editor and came to following results:

trash mp3

trash 132

trash 64+

Propably useless but interesting...

As expected, Atrac3+ @ 64kbps sounded 'best' in this reversed case, maybe $ony tested it this way

[physya]

Interestingly - Sony Connect sells files encoded at 132 kbps and they claim that they sell CD quality. This means that they actually accept that something less than 132 would be unsatisfactory...

[Christopher]

Not necessarily. I believe that they sell @ 132kbps ATRAC3 because it's the most compatible with all ATRAC-type players released in the last half decade. How good anything sounds is in the ear of the beholder, anyways - unfortunately alot of people get sucked up into forums like hydrogenaudio, head-fi, etc and get insane over bitrates and put enjoyment second.

[greenmachine]

If there were no such maniacs, who would develop, tune and compare all these fabulous codecs for the end-consumer?

[bug80]

unfortunately alot of people get sucked up into forums like hydrogenaudio, head-fi, etc and get insane over bitrates and put enjoyment second.

←

Some, yes, not all. I just find it very interesting to read about testing and the considerations when designing an encoder. And thanks to forums like HA, I found out what encoders and bitrates work best for my listening pleasure.

[Christopher]

That's the right way to do it, bug, but you understand the point I was trying to make.