[skamp]

The scientists simply choose to remember all the evidence that makes them be right, and ignore any contrary evidence.

Those are bad scientists, and their published experiments are routinely debunked.

But just as likely, B was indeed superior because it's extra detail retrieval "pulled out" that cymbal from the mix.

That's whishful thinking. You can well notice something new in the music you're listening to without switching to a better system. There's lots of reasons why you could notice it.

Finally the children were shown picture X which was either A or B and were asked to identify if picture X matched picture A or B.

In an ABX test, you're free to focus on whatever you want, and switch back and forth while focusing on the same element to detect any differences.

one of the tests was so hard that none of children could solve it even when pictures were right in front of them.

What that test showed, like any other ABX test, was that the pictures were perceivably equivalent, which is exactly one of the two outcomes I expect from an ABX test. I don't see a problem there.

Take my €65 FiiO E7 and the $1,300 Benchmark DAC1 USB. I probably couldn't ABX them because both of them are perceivably good enough for me, even though they do have measurable differences (the latter scores higher than the former). Again, no problem there. It would tell me what I want to know: do I need to spend $1,300 on a DAC?

 

[Mischa23v]

[size=medium]I would really appreciate it if you have an open mind, most of these questions you can probably answer yourself if you just thought about them. Or went back and read my previous answers.[/size]
 
[size=medium]“How come people can remember the 'differences' when the test is done sighted?” [/size]
[size=medium]Because if I place a red and black dot on piece of paper beside each other. You can actually see them at the same time. You do have to switch between complex pictures but if you know where to look and they’re beside each other, then you can see them both at same time. You definitely can see 2 letters that are on your keyboard that are beside each other at same time for example without having to rely on your memory.[/size]
 
[size=medium]“How come we can remember differences such as the difference between a radio, your car stereo and your own hifi?”[/size]
[size=medium] [/size]
[size=medium]So what you’re implying is that you can either remember something or you can’t remember anything right? Cause it’s not possible for human beings to have varying memory? It’s not really possible for some tasks to be harder to remember than other tasks? You can either remember everything in this world or not?[/size]
 
[size=medium]“How come we remember voices over the phone?”[/size]
 
[size=medium] I don’t even understand what you’re trying to imply here, I truly hope that you’re not implying that because we can remember easy tasks then we should be able to remember extremely difficult tasks as well? Your not trying to imply that just because you can remember one voice, then you are able to remember entire complex sounds of anything? [/size]
 
[size=medium]“Indeed how are we able to audition hifi and tell any differences at all if it was not for memory?”[/size]
 
[size=medium] So memory is either perfect or not? If you’re able to accomplish a task, then you should be able to accomplish any task in the world? Mind if I ask how old are you?[/size]
 
[size=medium]“Lets say it is memory, then ABX is not a fail for showing that up, it is a success. It has discovered we have a short term memory loss when it comes to sound quality differences. We should not dismiss that as a fail, we should then fit that into what we know about hearing and sound quality.”[/size]
 
[size=medium] Exactly that’s my point you don’t know if the test failed because it’s a memory problem, or because the test was too difficult for the person to distinguish between 2 audio files or that person has hearing loss. You have no idea why the person failed. Because people who conduct these tests aren’t interested in why people fail and are only interested in the result when people pass. That is why those abx tests tell you nothing useful when they fail.[/size]
 
[size=medium]"Lets say it is not memory, then again ABX is a success as it shows there is a point where differences get so small that they are now in reality inaudible. So any differences we hear when sighted are down to other senses."[/size]
 
[size=medium]Yes It’s a useful test, but no one is arguing against that. We all know that it has it’s uses.[/size]
 
[size=medium]"With that knowledge you can then make buying decisions based on what is really audible and what is not. I for one will chose to buy cheap cables and not worry about my DAC or amp and concentrate on my headphones. Others will do it differently."[/size]
 
[size=medium]This thread has nothing to do with w[size=medium]hether [/size]you can hear a differane or not and whether you should be buying expensive cables or not. Please direct your opinions in this matter else where. They are cloudy your judgment (bias doesn't belong here). [/size]
 
 
Quote:

The reason why I do not think out audible memory is that bad is because of the following.
 
How come people can remember the 'differences' when the test is done sighted?
 
How come we can remember differences such as the difference between a radio, your car stereo and your own hifi?
 
How come we remember voices over the phone?
 
Indeed how are we able to audition hifi and tell any differences at all if it was not for memory?
 
 
Lets say it is memory, then ABX is not a fail for showing that up, it is a success. It has discovered we have a short term memory loss when it comes to sound quality differences. We should not dismiss that as a fail, we should then fit that into what we know about hearing and sound quality.
 
Lets say it is not memory, then again ABX is a success as it shows there is a point where differences get so small that they are now in reality inaudible. So any differences we hear when sighted are down to other senses.
 
With that knowledge you can then make buying decisions based on what is really audible and what is not. I for one will chose to buy cheap cables and not worry about my DAC or amp and concentrate on my headphones. Others will do it differently.
 
 

 

 

[stv014]

Quote:

Originally Posted by Mischa23v /img/forum/go_quote.gif
 
[size=medium]The children were shown first picture A with circle and colors arranged in certain way for 5 seconds, and then shown picture B which had a slight different color arrangement for 5 seconds. Finally the children were shown picture X which was either A or B and were asked to identify if picture X matched picture A or B.  What I liked about this experiment was:[/size]

 
That is not quite how audio tests are done. You are allowed to listen to any part of A, B, X, or Y for as short or long time, and as many times as you want. By contrast, the above experiment is very restrictive. If the children were allowed to see the any of the pictures again with no restrictions (like it is the case for example with the foobar2000 ABX comparator), they could with some patience always have given the correct answer easily, even from 6 pictures.
 
Quote:

Originally Posted by Mischa23v /img/forum/go_quote.gif
 
[size=medium]However, most of these audio abx tests pretty much don’t really take anything into account if people fail.[/size]

 
If you fail, you (or anyone else) can try it again later. The point of the test is to find valid and statistically significant positive results.
 

 

[Prog Rock Man]

That is an interesting experiment Mischa. However, I do not get from it that ABX has failed. What I get is that depending on how you conduct the test you get different results.
 
For example if the children were audiophiles and the cards was music, from that experiment you would think that if you play unfamiliar pieces of music very briefly compared to familiar longer pieces of music, you should get more fails with the latter than the former, but that is not the case.  You would also think that simple pieces of music would be easier to tell apart than more complex ones, but that is also not the case. Tests which differentiate between audiophiles and people who do not consuder themselves to be audiophiles and you get no difference. I am not aware of any tests where people do not want to take part or are deaf, so I would not say that is a reason for invalidating audiophile blind tests.
 
We also have an accurate reason why the audiophile tests have failed, measurments of cables and the like find that so long as volume is equalised there is no measureable difference. Audiophiles are often inclined to claim audibility for things such as jitter which tests have found are beyond audibility.
 
Regarding your comment here;
 
[size=medium]"However, most of these audio abx tests pretty much don’t really take anything into account if people fail. They don’t really bother in researching why people might have failed. Most of these abx audio tests really only are interested in the results when people pass the test. That is why I’m claiming that when an audio abx tests fails, you learn absolutely nothing of importance. You have no idea why the test failed. Because no measure have been taken to see why the test has failed.[/size]"
 
I agree that not enough research has been done into why people have failed, instead far too often the test is blamed rather than accept the results and see what can be concluded. I think if people fail an ABX test we learn a lot. We do know why people failed the test, it is because there is no audible difference, which is backed up by other evidence of audibility.
 
 
 

 

[Mischa23v]

 
[size=medium]“In an ABX test, you're free to focus on whatever you want, and switch back and forth while focusing on the same element to detect any differences.”[/size]
 
[size=medium]Yes the children were allowed to go back and see the pictures again, but weren’t allowed to see all pictures at the same time. Only one at a time.[/size]
 
[size=medium]What that test showed, like any other ABX test, was that the pictures were perceivably equivalent, which is exactly one of the two outcomes I expect from an ABX test. I don't see a problem there.[/size]
[size=medium]Take my €65 FiiO E7 and the $1,300 Benchmark DAC1 USB. I probably couldn't ABX them because both of them are perceivably good enough for me, even though they do have measurable differences (the latter scores higher than the former). Again, no problem there. It would tell me what I want to know: do I need to spend $1,300 on a DAC?[/size]
 
[size=medium]No it’s not, you have no idea why you failed that test, you presumed that it’s because they are both good enough for you. It could be because you have hearing loss or any other reason. You just concluded that they were good enough and went with that.  You didn’t even bother to research why you might have failed the test. [/size]
 
[size=medium]That is my entire point from beginning that when the test fails. People don’t know why and they shouldn’t make assumptions and state them as facts without proof. That my friend is why I say those tests are flawed. [/size]
 
 
 
Quote:

In an ABX test, you're free to focus on whatever you want, and switch back and forth while focusing on the same element to detect any differences.
What that test showed, like any other ABX test, was that the pictures were perceivably equivalent, which is exactly one of the two outcomes I expect from an ABX test. I don't see a problem there.
Take my €65 FiiO E7 and the $1,300 Benchmark DAC1 USB. I probably couldn't ABX them because both of them are perceivably good enough for me, even though they do have measurable differences (the latter scores higher than the former). Again, no problem there. It would tell me what I want to know: do I need to spend $1,300 on a DAC?

 
 

 

[Prog Rock Man]

Considering the number of people who have taken part in blind tests I am sure all hearing abilities have taken part and yet, there has been no difference in the results.
 
Trick blind tests where there has been no real switching have the same results for tests where switching has taken place with either similar products or vastly different products.
 
I am quite sure that all other posibilities have been covered and that blind tests are valid and so are their results.
 
 

 

[mikeaj]

Because of all the limitations and other factors I don't think anybody reasonable would say that a negative ABX result proves anything definitely by itself.  Is the problem you have really with the ABX procedure or with overreaching conclusions based on ABX results?  It's just that based on all sorts of other side information, the lack of positive ABX results for a particular subject is further evidence (of some quality, maybe not great) that can be used in an argument to question "night and day" differences.
 
I think that we can agree that positive ABX results are more relevant, because they do prove that there is a difference that can be discerned?  Are there objections to positive ABX results?

 

[skamp]

No it’s not, you have no idea why you failed that test, you presumed that it’s because they are both good enough for you. It could be because you have hearing loss or any other reason.

I stand by what I said. Whatever the reason (hearing loss or whatever), they're effectively good enough for me. I won't claim that they're good enough for anyone else, just me. I won't claim that there are no difference either, especially if they can be measured. A failed ABX test shows what it shows: that no difference could be detected by me in that particular setting. If the setting reflects normal listening conditions, then the result is valid to me, and I can work with it.

 

[skamp]

To be clear, a failed ABX test doesn't prove that there were no differences. It only shows (and only if the person who took the test was honest) that no difference could be perceived by the person in question specifically, in specific conditions.

If the person in question has proven to have normal hearing, and the conditions could arguably be qualified as "normal listening conditions", then the test result is interesting and valuable to other people with similar hearing and training, in similar conditions.

If a single element in the setting is different for someone else (better hearing, better training, different listening conditions), then that person should run an ABX test themselves that would account for those differences.

Mischa: you seem to be under the impression that we claim more than we really do.

 

[Prog Rock Man]

I am having problems with posting today and again can't get some posts to quote. Anyway in response to Mischa
 
[size=medium]I would really appreciate it if you have an open mind, most of these questions you can probably answer yourself if you just thought about them. Or went back and read my previous answers.[/size]
 
[size=medium]I have spent a lot of time on this subject and I have come to a conclusion about blind tests, which is they are valid and produce interesting, unexpected and sadly for many unwelcome results. That does not make me closed minded. I am here to join a discussion about blind testing, not to answer my own questions and please don't be so sure you have all the answers in your previous posts.[/size]
 
[size=medium]“How come people can remember the 'differences' when the test is done sighted?”[/size]
[size=medium]Because if I place a red and black dot on piece of paper beside each other. You can actually see them at the same time. You do have to switch between complex pictures but if you know where to look and they’re beside each other, then you can see them both at same time. You definitely can see 2 letters that are on your keyboard that are beside each other at same time for example without having to rely on your memory.[/size]
 
[size=medium]I am talking about audio memory. I am sure that blind testing shows us that sight (so style and image) and knowledge of hifi (brand names, cost etc) have [/size]an affect on perceived sound quality.
 
[size=medium]“How come we can remember differences such as the difference between a radio, your car stereo and your own hifi?”[/size]
[size=medium] [/size]
[size=medium]So what you’re implying is that you can either remember something or you can’t remember anything right? Cause it’s not possible for human beings to have varying memory? It’s not really possible for some tasks to be harder to remember than other tasks? You can either remember everything in this world or not?[/size]
 
[size=medium]Strawman argument. Again I am only talking in refernece to our ability to remember differences in sound reproduced on hifi. [/size]
 
[size=medium]“How come we remember voices over the phone?”[/size]
 
[size=medium] I don’t even understand what you’re trying to imply here, I truly hope that you’re not implying that because we can remember easy tasks then we should be able to remember extremely difficult tasks as well? Your not trying to imply that just because you can remember one voice, then you are able to remember entire complex sounds of anything?[/size]
 
[size=medium]I was giving an example of how we can remember differences in sound, such as the human voice, when there really is a difference. I could also have used car engines as an example. I used to live near a Porsche dealer back in the 1980s. So many of them went past the house I could tell you if it was a 924, 944, 911 or 928 just from the sound of the engine. Indeed in our normal day to day lives we have a memory bank of thousands of sounds. Language depends on a memory for sound.  We can, (sometimes with a bit of practice) remember sounds. [/size]So why do we supposedly lose the ability to remember sounds when it comes to blind testing hifi?
 
[size=medium]“Indeed how are we able to audition hifi and tell any differences at all if it was not for memory?”[/size]
 
[size=medium] So memory is either perfect or not? If you’re able to accomplish a task, then you should be able to accomplish any task in the world? Mind if I ask how old are you?[/size]
 
[size=medium]Again, why are we able to remember thousands of different sounds and not remember the sound of one hifi compared to another in a blind test? I am mid 40s, why?[/size]
 
[size=medium]“Lets say it is memory, then ABX is not a fail for showing that up, it is a success. It has discovered we have a short term memory loss when it comes to sound quality differences. We should not dismiss that as a fail, we should then fit that into what we know about hearing and sound quality.”[/size]
 
[size=medium] Exactly that’s my point you don’t know if the test failed because it’s a memory problem, or because the test was too difficult for the person to distinguish between 2 audio files or that person has hearing loss. You have no idea why the person failed. Because people who conduct these tests aren’t interested in why people fail and are only interested in the result when people pass. That is why those abx tests tell you nothing useful when they fail.[/size]
 
[size=medium]In reverse order, enough tests have been done by enough people to be able to say hearing loss is not an issue. It would be remarkable if it turned out that every single audiophile and other who has taken part in a blind test suffers from hearing loss. I think that the tests do show that not just audio files, but other parts of the hifi chain are not audibly different. That is shown by the tests that are passed, such as with some audio files and speakers. Indeed that is one of the reasons why I do not think that we have a memory problem for sound quality, if so why are some blind tests passed?[/size]
 
[size=medium]"Lets say it is not memory, then again ABX is a success as it shows there is a point where differences get so small that they are now in reality inaudible. So any differences we hear when sighted are down to other senses."[/size]
 
[size=medium]Yes It’s a useful test, but no one is arguing against that. We all know that it has it’s uses.[/size]
 
[size=medium]OK[/size]
 
[size=medium]"With that knowledge you can then make buying decisions based on what is really audible and what is not. I for one will chose to buy cheap cables and not worry about my DAC or amp and concentrate on my headphones. Others will do it differently."[/size]
 
[size=medium]This thread has nothing to do with w[size=medium]hether[/size] you can hear a differane or not and whether you should be buying expensive cables or not. Please direct your opinions in this matter else where. They are cloudy your judgment (bias doesn't belong here).[/size]
 
OK

 

[Mischa23v]

[size=medium]Assume for a second that you failed the test because of the way you conducted the test. That if the test was conducted differently you might have passed (since most people here don’t bother investigating why they failed the test). Now not only do you not know why you failed the test. But you should have in theory passed the test. Thus, the test absolutely fails because it not only doesn’t tell you why you failed, but you actually should have passed. That is why it’s extremely important to take into consideration why you might have failed. [/size]
 

 
Quote:

I stand by what I said. Whatever the reason (hearing loss or whatever), they're effectively good enough for me. I won't claim that they're good enough for anyone else, just me. I won't claim that there are no difference either, especially if they can be measured. A failed ABX test shows what it shows: that no difference could be detected by me in that particular setting. If the setting reflects normal listening conditions, then the result is valid to me, and I can work with it.

 

 

[Mischa23v]

[size=medium]Yup I do that, unfortunately it’s one of my many flaws. [/size]
 
Quote:

Mischa: you seem to be under the impression that we claim more than we really do.

 

 

[Mischa23v]

[size=medium]Yes you’re absolutely correct that the test isn’t exactly how an audio abx test is conduct. However, it’s the only relevant data that I have. [size=medium]However, I do not agree with the fact that if they were given more time. They would have been able to correctly identify and match pictures, It might have been easier. But that depends entirely on how hard the test is.[/size][/size]
 
Quote:

Originally Posted by stv014 /img/forum/go_quote.gif

 
That is not quite how audio tests are done. You are allowed to listen to any part of A, B, X, or Y for as short or long time, and as many times as you want. By contrast, the above experiment is very restrictive. If the children were allowed to see the any of the pictures again with no restrictions (like it is the case for example with the foobar2000 ABX comparator), they could with some patience always have given the correct answer easily, even from 6 pictures.
 

 
[size=medium]It doesn’t matter how many times you take the test, if you fail the test because of the way you conducted the test.  What if you were actually supposed to pass the test? Then not only do you not know why you failed, but you also ended up with an incorrect result.[/size]
 
Quote:

 
If you fail, you (or anyone else) can try it again later. The point of the test is to find valid and statistically significant positive results.
 

 
 

 

[Mischa23v]

[size=medium]I’m not really sure who you’re directing your post to but I’ll assume it’s me. [/size]
 
[size=10pt]“Is the problem you have really with the ABX procedure or with overreaching conclusions based on ABX results?”[/size]
 
[size=10pt]Both, since we don’t know why people failed the test, there is a chance that they should have passed it if the test was conduct in a better fashion. Hence, when people fail they have the right to argue that the problem is not with their hearing but with the way abx  sound test was conducted.[/size]
 
[size=10pt]It's just that based on all sorts of other side information, the lack of positive ABX results for a particular subject is further evidence (of some quality, maybe not great) that can be used in an argument to question "night and day" differences.[/size]
 
[size=10pt]I agree[/size]
 
[size=10pt]“I think that we can agree that positive ABX results are more relevant, because they do prove that there is a difference that can be discerned? Are there objections to positive ABX results?”[/size]
 
[size=10pt]No because if you fail the test even though you were supposed to pass, it’s a huge flaw. Don’t you agree? [/size]


 
Quote:

Because of all the limitations and other factors I don't think anybody reasonable would say that a negative ABX result proves anything definitely by itself.  Is the problem you have really with the ABX procedure or with overreaching conclusions based on ABX results?  It's just that based on all sorts of other side information, the lack of positive ABX results for a particular subject is further evidence (of some quality, maybe not great) that can be used in an argument to question "night and day" differences.
 
I think that we can agree that positive ABX results are more relevant, because they do prove that there is a difference that can be discerned?  Are there objections to positive ABX results?

 

 

[skamp]

Assume for a second that you failed the test because of the way you conducted the test. That if the test was conducted differently you might have passed

I think we can agree that ABX tests prove only one thing, and only when they're successful: that there was a difference to be heard. A failed test doesn't really prove anything indeed. While I failed ABX tests with higher bitrate Ogg Vorbis files, I might succeed later on, when I have better training.

As the thread starter, let me go back to my main point: a failed ABX test is not proof of anything, but when I see claims of a difference, I expect measurements, and with claims of audible differences, I expect ABX logs. Do you find that unreasonable? I mean, I need some kind of proof, because more often than not, I will not be able to perceive claimed differences (like "256kbps AAC sounds like crap"). Do you have anything better than ABX tests that would provide such proof?