Also, something I've noticed, is that some sound tests people actually end up preferring lower quality files, as it seems to remove some slight details that can be perceived as unwanted. Extra noise and harshness people may think is being introduced by a low quality mp3, is actually on the original recording, and the higher quality mp3 doesn't mask it.
People seem to like the sound of that comes from loss of details and distortion, just look at tube amps or "warm" and "smooth" gear. So asking "what sounds better" isn't really the question that should asked.
The real question that should be asked, is "what sounds truest to the original recording?"
Also, I feel the ABX test on here is flawed, instead of FLAC being a 3rd unknown, the FLAC file should be the reference, and the 128 and 320 file should be blind tested to see which one is closer to the lossless reference file.
There is one on this forum. Though it is the 3-way blind one I spoke of which I feel is flawed.
I would create one, though I am not sure how to go about hosting it, converting files so they appear the same size.
I would have three test clips. Ideally music recorded well, with low drum kicks, as well as many high notes like cymbals/crashes/horns. The music should also be fast and fairly layered. This will help draw out an loss of sound quality from compression.
Then the FLAC sample will be the reference, or control sample. With the 128kbps and 320kbps samples being tested blind.
I feel this will yield truly accurate information on whether or not there is actually a difference between bit rates. As the idea that is being questioned is "is there a difference between bit rates"?, but the question that is being asked with current tests is "which do you think sounds better?"
So the wrong question is being asked, as many people prefer coloured (distorted) sound, therefor, some people may prefer the sound of lower bitrates.
I don't think you understand the testing I am talking about.
The test is to use FLAC as a reference, a known control sample, and then two blind samples, one 128, one 320.
The goal of the test would be to pick the blind sample that is the closest match to the known reference FLAC sample.
While not technically strictly an ABX test, it would be a variation that would be better suited to discerning differences between 128 and 320.
This would be a better test, as it asks the question of "which sounds closest to a known lossless reference?" rather than asking "which do you prefer?"
The current tests for ABX on here though ask which people prefer. Some people may prefer lower bitrate recording if it "smooths" out the sound. Just like how many people prefer coloured sounding gear and phones.
So when you pose it as an ABX and ask which do people think sounds better, you may have more people choosing 128kbps since they prefer the colouration that the lower bitrate adds.
What people are actually trying to figure out though is if there is a difference between 128, 320, and the lossless. So the comparison needs to be the 128 and 320 directly against a lossless file.
That is why a reference lossless file must be used as a comparison point, and the question needs to be, which of these compressed files sounds closest to the lossless file.
That is the only proper way to tell if there is an actual difference, since you are asking people to draw a comparison to a known value. When you only ask what sounds better across 3 unknown files, you are asking people what they prefer.
The flaw in the current testing is the assumption that people will prefer a file which is the closest to the source material. An assumption which is likely incorrect, as many people prefer non-neutral gear.
My method would remove this assumption, and would better control the variables.
Have you ever done an ABX test? It would appear that you haven't.
From
Wikipedia:
An ABX test is a method of comparing two choices of sensory stimuli to identify detectable differences between them. A subject is presented with two known samples (sample A, the first reference, and sample B, the second reference) followed by one unknown sample X that is randomly selected from either A or B. The subject is then required to identify X as either A or B. If X cannot be identified reliably with a low p-value in a predetermined number of trials, then the null hypothesis cannot be rejected and it cannot be proven that there is a perceptible difference between A and B.
ABX tests can easily be performed as double-blind trials, eliminating any possible unconscious influence from the researcher or the test supervisor. Because samples A and B are provided just prior to sample X, the difference does not have to be discerned from assumption based on long-term memory or past experience. Thus, the ABX test answers whether or not, under ideal circumstances, a perceptual difference can be found.
The purpose of ABX tests is to determine if you can distinguish between two sound samples with statistical significance, that is all.
There is no "third unknown" in the sense that you are implying, and which one you prefer has nothing to do with it. You already know that (for example) A is FLAC and B is MP3. The only unknown variable is X, which is either A or B played randomly. You can press the buttons and listen to known and unknown samples as much as you like.
In addition, you should have already listened to the samples beforehand in order to familiarize yourself with any perceived differences. (Testing out those perceptions is the only reason you would be taking the ABX test in the first place.) If the lossy file sounds more smooth and pleasant, this distinction should have been identified prior to the experiment, reducing the chances of mistaking the smoother sound of the lossy file as the lossless file. Doesn't matter anyway, because, like I mentioned, A and B are the known variables. During the test, you know whether A or B is playing, since you select it yourself; you just don't know whether X is A or B each time X is played.
If you would like to do your own ABX test, I suggest reading the instructions in the
guide that I have repeatedly linked to.
As for your variation, what you are proposing is similar to a duo-trio AXY test:
AXY – one known, two unknown (one equals A, other equals B), test is which unknown is the known: X = A (and Y = B), or Y = A (and X = B).
...Except you're not testing which unknown is the known, but instead testing which unknown is closer to the known.
The benchmark of any scientific experiment is to have one control and test one variable at a time, not two. You're not controlling the variables; you're just introducing new ones.
If you insist upon comparing both 128 and 320 to FLAC, you need to conduct two sets of ABX tests like I suggested...if you want to do it right, that is.
It's a moot point, in the end. If there is an audible difference between 128 kbps and 320 kbps MP3 (which can be verified in its own ABX), it's obvious that the latter will sound closer to lossless. Your method is misguided and unnecessary.
