4 ms·

I've conducted several A/B tests back when I had nothing better to do and it was a very education experience. I used a recording of a harpsichord, using lame V5

I've conducted several A/B tests back when I had nothing better to do and it was a very education experience. I used a recording of a harpsichord, using lame V5 mp3s, and came away with a statistically significant result (I remember taking 14 trials, though I don't remember what my actual true positive rate was) After all of that, though I had double blind evidence I could hear a difference, I knew I didn't really care about that difference. I couldn't even describe the difference - it was almost subliminal. And that was with a harpsichord, which is a known difficult sound to encode. I've been a lot more comfortable knowing lossy encoding isn't destroying my music enjoyment since those tests.

glandium 8y ago
There are different things that could be found through tests, but I don't think I've seen tests being really extensive. The interesting questions to me: - Can a difference be heard? - If yes, are people able to reliably identify A vs B (as in, can people identify whether it's A or B rather that whatever they listen now is not the same as the last they heard) - If they can identify a difference, which sounds better to them? - Do different headphones give different results? - Same question with headphones vs speakers?
- TheOtherHobbes 8y ago
  The answer to the last two questions is easy - yes to both. There are absolutely huge differences between budget consumer and high-end professional equipment. There are also huge differences between high-end audiophile and professional equipment - the former being designed to flatter and enhance the sound, the latter being as neutral as possible. A good few years ago a high-end audiophile CD/DAC maker was caught deliberately rolling off the top end on their equipment to make it sound smoother and "less digital." It wasn't accurate, but a lot of people liked the sound enough to pay five figures for the boxes they made. Professional audio has a different goal. The aim is to create a mix that "translates" - sounds good on as many different systems as possible, from budget earbuds to audiophile. For a long time studios used a pair of shitty Yamaha bookshelf speakers called NS10s next to their ultra-expensive reference speakers. NS10s were almost the worst speaker ever. But if a mix sounded good on the NS10s, it sounded good on anything.
- amgutier 8y ago
  Blind testing is good for the first two questions, but it's next to impossible to blind test headphones. The question of preference, though, is what matters, and that's where I am now firmly convinced lossy compression is fine for me: the sort of differences that I could hear made zero difference to my enjoyment of actual music.

glandium 8y ago

There are different things that could be found through tests, but I don't think I've seen tests being really extensive. The interesting questions to me: - Can a difference be heard? - If yes, are people able to reliably identify A vs B (as in, can people identify whether it's A or B rather that whatever they listen now is not the same as the last they heard) - If they can identify a difference, which sounds better to them? - Do different headphones give different results? - Same question with headphones vs speakers?

TheOtherHobbes 8y ago
The answer to the last two questions is easy - yes to both. There are absolutely huge differences between budget consumer and high-end professional equipment. There are also huge differences between high-end audiophile and professional equipment - the former being designed to flatter and enhance the sound, the latter being as neutral as possible. A good few years ago a high-end audiophile CD/DAC maker was caught deliberately rolling off the top end on their equipment to make it sound smoother and "less digital." It wasn't accurate, but a lot of people liked the sound enough to pay five figures for the boxes they made. Professional audio has a different goal. The aim is to create a mix that "translates" - sounds good on as many different systems as possible, from budget earbuds to audiophile. For a long time studios used a pair of shitty Yamaha bookshelf speakers called NS10s next to their ultra-expensive reference speakers. NS10s were almost the worst speaker ever. But if a mix sounded good on the NS10s, it sounded good on anything.
amgutier 8y ago
Blind testing is good for the first two questions, but it's next to impossible to blind test headphones. The question of preference, though, is what matters, and that's where I am now firmly convinced lossy compression is fine for me: the sort of differences that I could hear made zero difference to my enjoyment of actual music.