[pinnahertz]

In interpretive qualitative research, there is no usage of bias. Bias is related to positivist research, and is not used in the tradition of interpretive research. Actually, objecting to how bias is traditionally used in the nature sciences, rejecting it completely, is so comon, that there is a large number of papers dealing with it. For really good reasons. Well founded, and well argument ed reasoning.

If you've been reading the thread....

You might have discerned that the kind of testing we're talking about fits two basic forms: the statistical analysis of subjective opinion with regard to the basic question, "Is there an audible difference between A and B?", and if that analysis returns a statistically significant trend greater that random guessing, the next phase of testing, the statistical analysis of the subjective opinion with regard to the basic question, "Which is better, A or B?" can be performed. The data structure is a simple binary response in each case and does not include the more broad spectrum of data that might be generated by interpretive qualitative analysis where data is comprised of description and opinion, which of course, is not directly or easily analyzed by simple statistics. My error might be the use of the term "qualitative testing", which you then interpreted as "interpretive qualitative research", which I was not referring to.

The notion that everything needs to be statistical significant is also a highly contested argument. Also, for very good reasons. There are plenty of papers, of highly regarded scientists, that argue against the need for being statistically significant.

In this particular case, the first question, "Is there an audible difference?" can only be answered by statistical analysis of binary data. If you don't apply statistical analysis of a significant quantity of data, then if you were testing "Can a person predict the future", and your test was the ability of a subject to "call" a coin flip in advance, your assumption could be that the correct guess of a single coin-flip indicates the subjects ability to predict the future.

As for meaningless, well, what do you mean by "in fact completely meaningless"? When did meaning become a fact? How do you factually prove meaning?

In the coin-flip example, you have a single subject and a single coin flip test with a single binary answer. The resolution of that data is poor and includes a very high degree of "noise" that is equal to the data itself; the correct result could be a random correct guess OR the subject could be predicting the future. The resulting data has a noise level equivalent to the data "signal", and is therefore meaningless with regard to the question, "Can a person predict the future?" From a single correct response, conclusion could only be "yes". However, statistical analysis of a quantity of data from a number of tests increases the signal to noise ratio of the result by averaging the test responses, then returning a number rather than a binary result. If we test many subjects many times, the degree to which the results are different than random noise (guessing) will indicate a probability that someone can predict the future. In other words, the results of the test are a ratio of subject responses to random noise, which returns a probability figure, not a binary result.

This kind of testing is valid and used in scientific research all the time, most notably in drug efficacy testing. The first question, "Does use of the drug return a result different than the placebo?" could be followed by "Does the drug result in improvement in patient condition?" Both require statistical analysis of binary data. That might then be followed by full interpretive qualitative analysis of patient impressions and side-effects.

 

[gregorio]

[1] Clearly all DACs do NOT sound the same, and all DACs do not measure the same, or even close.
[2] (For example, one DAC I owned had a very dull sounding high-end, almost certainly related to the fact that its frequency response was -3 dB at 20 kHz.)
[2a] So the real question is whether DACs sound different IN WAYS NOT OBVIOUSLY RELATED TO DIFFERENCES IN MEASURED PERFORMANCE.
[3] And, in order to test that question, we must predict what specific measured differences are audible.

1. No one is saying all DACs sound the same! We're saying all modern DACs, competently designed to achieve transparency, sound the same. Therefore, obviously, not all DACs sound the same! Some are not modern, some are not competently designed and some are deliberately designed not to achieve transparency (despite claiming to be high fidelity).

2. Just saying -3dB at 20kHz doesn't necessarily tell us much, where did that roll off start and how gradual was it? But, the "high-end" is not at 20kHz, the "high-end" is roughly from about 8-9kHz and extends to as much as about 16kHz. So if you really did hear the DAC you owned as having a "very dull sounding high-end" then you should look at what's happening in the high-end, rather than at what's happening significantly beyond the high-end where you can't hear anything at all!
2a. No, the real question is what is the actual measured performance, as opposed to what is just a published spec! What's disconcerting about your posts is your determination to validate or at least give some credence to audiophile nonsense claims: Misrepresent a spec as a "MEASURED PERFORMANCE", so you can imply measurements are invalid. State that anecdotal audiophile claims and sighted tests are flawed and so are all the published scientific studies and therefore scientific studies have effectively the same level of credence as audiophile claims. This is the same old typical audiophile nonsense just presented more intelligently and less absolutely. Of course, it's your choice if you wish to give equal credence but personally, I prefer to evaluate the flaws in the published science and other evidence and judge for myself how reliable it is, how much it's conclusions are affected by those flaws and how much credence to give it. Using this rational approach is largely why this sub-forum exists and it's clear there's virtually no credible evidence to support these audiophile claims, which is particularly telling given the very significant financial incentive in providing reliable evidence over the course of nigh on 20 years!

3. And we can! While in some cases we haven't laid to rest exact figures, even the most optimistic realistic figures are often thousands or even many tens of thousands greater than those actually achieved by modern, fairly cheap units.

As consumers we do NOT have access to uncompressed video.

What's your whole post about compressed video got to do with anything? We're not talking about compressed audio and even if we were, they're different beasts, we don't data compress raw audio by up to 1,000 times! It's hard to avoid the conclusion from your posts that you have an agenda, something to sell maybe? While you're not outright saying that measurements are useless and science has it all wrong, you are suggesting it might be wrong but for that suggestion to be anything more than just wishful thinking or marketing BS you've got to provide something substantially more than fallacies, misrepresentations and a bunch of audiophile impressions and beliefs!!

G

 

[pinnahertz]

It's hard to avoid the conclusion from your posts that you have an agenda, something to sell maybe? While you're not outright saying that measurements are useless and science has it all wrong, you are suggesting it might be wrong but for that suggestion to be anything more than just wishful thinking or marketing BS you've got to provide something substantially more than fallacies, misrepresentations and a bunch of audiophile impressions and beliefs!!

I had a similar impression. In fact, my perception of a certain company has almost completely reversed as a result of this discussion.

 

[castleofargh]

If you've been reading the thread....

You might have discerned that the kind of testing we're talking about fits two basic forms: the statistical analysis of subjective opinion with regard to the basic question, "Is there an audible difference between A and B?", and if that analysis returns a statistically significant trend greater that random guessing, the next phase of testing, the statistical analysis of the subjective opinion with regard to the basic question, "Which is better, A or B?" can be performed. The data structure is a simple binary response in each case and does not include the more broad spectrum of data that might be generated by interpretive qualitative analysis where data is comprised of description and opinion, which of course, is not directly or easily analyzed by simple statistics. My error might be the use of the term "qualitative testing", which you then interpreted as "interpretive qualitative research", which I was not referring to.
In this particular case, the first question, "Is there an audible difference?" can only be answered by statistical analysis of binary data. If you don't apply statistical analysis of a significant quantity of data, then if you were testing "Can a person predict the future", and your test was the ability of a subject to "call" a coin flip in advance, your assumption could be that the correct guess of a single coin-flip indicates the subjects ability to predict the future.
In the coin-flip example, you have a single subject and a single coin flip test with a single binary answer. The resolution of that data is poor and includes a very high degree of "noise" that is equal to the data itself; the correct result could be a random correct guess OR the subject could be predicting the future. The resulting data has a noise level equivalent to the data "signal", and is therefore meaningless with regard to the question, "Can a person predict the future?" From a single correct response, conclusion could only be "yes". However, statistical analysis of a quantity of data from a number of tests increases the signal to noise ratio of the result by averaging the test responses, then returning a number rather than a binary result. If we test many subjects many times, the degree to which the results are different than random noise (guessing) will indicate a probability that someone can predict the future. In other words, the results of the test are a ratio of subject responses to random noise, which returns a probability figure, not a binary result.

This kind of testing is valid and used in scientific research all the time, most notably in drug efficacy testing. The first question, "Does use of the drug return a result different than the placebo?" could be followed by "Does the drug result in improvement in patient condition?" Both require statistical analysis of binary data. That might then be followed by full interpretive qualitative analysis of patient impressions and side-effects.

don't get dragged into his controversy bait(that's usually my job to fall for it like an idiot). the best way to test something is obviously conditioned by the question someone is trying to answer at the time.

 

[bigshot]

I think he's gone anyway... When I challenged him to prove *he* could hear a difference, it took it away from theoretical arguments into real world proof and he bailed. This comment will probably irritate his ego and he'll be back trying to drag it back from the specific to the theoretical and general, but my test will still be waiting for him.

I'd like to set up a track with two more samples... 24/96 and 24/192. I don't have any music encoded that high though myself. If anyone would like to help me set one up, I'd like to do it. Imagine 12 samples from MP3 192 all the way up to 24/192 randomly shuffled into a single 24/192 file... haha! Sort that out.

 

[JaeYoon]

I think he's gone anyway... When I challenged him to prove *he* could hear a difference, it took it away from theoretical arguments into real world proof and he bailed. This comment will probably irritate his ego and he'll be back trying to drag it back from the specific to the theoretical and general, but my test will still be waiting for him.

I'd like to set up a track with two more samples... 24/96 and 24/192. I don't have any music encoded that high though myself. If anyone would like to help me set one up, I'd like to do it. Imagine 12 samples from MP3 192 all the way up to 24/192 randomly shuffled into a single 24/192 file... haha! Sort that out.

Let me know if he accepts your challenge. I did not like his attitude at the end of the debate.

I have LAME MP3 the latest by the way, 3.100 encoded from -V3 to -V0.

 

[Strangelove424]

I was inspired to create my home theater projection system when I was able to see The Incredibles at the Frank Wells Theater at Disney.

I can't speak to the Frank Wells showing of the Incredibles, but viewing uncompressed video requires a chain built around it... drives fast enough to not drop frames, and a display system capable of processing and displaying a wide gamut. Computers can be outfitted for that pretty easily, and work in high bit color space with monitors that reliably display the extra color info. There are consumer TVs available that show wide color gamuts and many receivers and BD players that support Deep Color too. I know my own PS3 and simple Denon receiver support it. Still not sure where uncompressed films would be sourced though, as Blu Ray compression typically uses a form of h.264. Though color depth may not be of supreme importance, I think it's atleast as important as resolution. Before the market rushes headlong to 4k, they might want to make use of those extra pixels with color definition too. If the extra file size needed to deliver 4k comes at the cost of color data, that's a bad compromise to make. I'd pick an equivalent bit rate 1080p any day of the week. But we'll see where that leads to I guess.

Personally, for entertainment use, I have a PS3->DenonAVR->1080p 60hz Vizio. Nothing fancy, and I enjoy it very much. When it's time for popcorn and escapism, it certainly beats being huddled up to a hot IPS monitor. And the surround sound imo is the most important aspect of mimicing a theatre experience. The sound system brings the dimension.

 

[KeithEmo]

1) Unfortunately, the adjectives you've chosen are all subject to opinion and interpretation.... which is the problem. There are in fact many expensive DACs, being marketed by supposedly "reputable" companies, and highly thought of by a substantial number of reviewers and audiophiles, that clearly do NOT meet that criterion. And, while you and I might agree that this simply means they are poorly designed, that is obviously not sufficient to describe the difference to others. And, yes, there are clearly a number of people who believe that "any DAC made in the past ten years, and costing over $50, is going to sound audibly perfect" - and they are incorrect in that assumption.

2) You're way overanalyzing that one. My sole point was to offer that particular DAC as an example of the many instances where measured flaws in performance do in fact correlate quite well with audible flaws in performance. (I didn't measure its frequency response; however, the point remains that the manufacturer didn't even spec it to have "an audibly flat frequency response", so it would be unreasonable to treat it as if it was claimed to have one.

3) Again, you take things out of context. When I'm talking about measured performance I'm simply talking about validating your test set. Claiming that "there's no audible difference with high-res files" is meaningless if your test setup includes speakers that are "rated to 30 kHz" unless you've actually measured them and demonstrated that they meet their spec. Likewise, failing to hear an audible difference between two DACs that are CLAIMED to be flat +/-0.1 dB isn't meaningful if you don't confirm that they both do in fact meet spec. (Otherwise, it's possible that you might hear a difference simply because one meets spec and the other does not.)

In this context, I'm referring generally to the various tests claimed to determine "whether the difference between regular and high-res files is audible". In the majority of them, no measurements were taken to validate the claim that the listener was actually being presented with a MEASURABLE difference. (I don't care what someone claims. It's simple enough to pull out a test microphone and see if there's a measurable difference before going on to determine if it's audible. If there is in fact no measurable difference, then your test is invalid, and so are any results arising from it.)

4) My response about video was in response to another post about "how some types of video compression may look perfect - or may even sometimes make the video look better".

And, no, I'm not at all saying that measurements don't count.... simply that, if you do want to base your judgment on measurements, then you need to ensure that you make the proper measurements under proper conditions.
But, yes, if both are flawed, then neither can be considered to be absolutely reliable or definitive.... that's simple logic.
And, yes, a test that claims nobody can hear 25 kHz, conducted using a speaker that hasn't been tested and confirmed to be able to play 25 kHz, IS meaningless.

1. No one is saying all DACs sound the same! We're saying all modern DACs, competently designed to achieve transparency, sound the same. Therefore, obviously, not all DACs sound the same! Some are not modern, some are not competently designed and some are deliberately designed not to achieve transparency (despite claiming to be high fidelity).

2. Just saying -3dB at 20kHz doesn't necessarily tell us much, where did that roll off start and how gradual was it? But, the "high-end" is not at 20kHz, the "high-end" is roughly from about 8-9kHz and extends to as much as about 16kHz. So if you really did hear the DAC you owned as having a "very dull sounding high-end" then you should look at what's happening in the high-end, rather than at what's happening significantly beyond the high-end where you can't hear anything at all!
2a. No, the real question is what is the actual measured performance, as opposed to what is just a published spec! What's disconcerting about your posts is your determination to validate or at least give some credence to audiophile nonsense claims: Misrepresent a spec as a "MEASURED PERFORMANCE", so you can imply measurements are invalid. State that anecdotal audiophile claims and sighted tests are flawed and so are all the published scientific studies and therefore scientific studies have effectively the same level of credence as audiophile claims. This is the same old typical audiophile nonsense just presented more intelligently and less absolutely. Of course, it's your choice if you wish to give equal credence but personally, I prefer to evaluate the flaws in the published science and other evidence and judge for myself how reliable it is, how much it's conclusions are affected by those flaws and how much credence to give it. Using this rational approach is largely why this sub-forum exists and it's clear there's virtually no credible evidence to support these audiophile claims, which is particularly telling given the very significant financial incentive in providing reliable evidence over the course of nigh on 20 years!

3. And we can! While in some cases we haven't laid to rest exact figures, even the most optimistic realistic figures are often thousands or even many tens of thousands greater than those actually achieved by modern, fairly cheap units.



What's your whole post about compressed video got to do with anything? We're not talking about compressed audio and even if we were, they're different beasts, we don't data compress raw audio by up to 1,000 times! It's hard to avoid the conclusion from your posts that you have an agenda, something to sell maybe? While you're not outright saying that measurements are useless and science has it all wrong, you are suggesting it might be wrong but for that suggestion to be anything more than just wishful thinking or marketing BS you've got to provide something substantially more than fallacies, misrepresentations and a bunch of audiophile impressions and beliefs!!

G

 

[KeithEmo]

About the only place uncompressed live video can be sourced is as a direct feed from a high end camera.
You can also find various samples of uncompressed computer rendered video (Google the publicly available renderings of the "Big Buck Bunny" cartoon).
Blu-Ray discs use h.264 compression; 4k UHD discs use h.265 (HEVC), which is more efficient, and is also often optimized to deliver a smoother looking picture, sometimes even at the expense of absolute sharpness.
Even the CinePro format, used on the higher-end GoPro cameras, is compressed.
(A few industry experts have suggested that we would have been better off using moving to HDR without bothering with 4k..... but they were apparently in a minority.)

I can't speak to the Frank Wells showing of the Incredibles, but viewing uncompressed video requires a chain built around it... drives fast enough to not drop frames, and a display system capable of processing and displaying a wide gamut. Computers can be outfitted for that pretty easily, and work in high bit color space with monitors that reliably display the extra color info. There are consumer TVs available that show wide color gamuts and many receivers and BD players that support Deep Color too. I know my own PS3 and simple Denon receiver support it. Still not sure where uncompressed films would be sourced though, as Blu Ray compression typically uses a form of h.264. Though color depth may not be of supreme importance, I think it's atleast as important as resolution. Before the market rushes headlong to 4k, they might want to make use of those extra pixels with color definition too. If the extra file size needed to deliver 4k comes at the cost of color data, that's a bad compromise to make. I'd pick an equivalent bit rate 1080p any day of the week. But we'll see where that leads to I guess.

Personally, for entertainment use, I have a PS3->DenonAVR->1080p 60hz Vizio. Nothing fancy, and I enjoy it very much. When it's time for popcorn and escapism, it certainly beats being huddled up to a hot IPS monitor. And the surround sound imo is the most important aspect of mimicing a theatre experience. The sound system brings the dimension.

 

[KeithEmo]

I disagree.....

If we ask: "Can humans hear 23 kHz?" then the answer is in fact a simple binary answer...... the answer is either "Yes" or "No".

You feel free to consider any number of other questions like:
"Can MOST humans hear the difference?"
"Is the difference SIGNIFICANT?"
"Does the presence or absence of 23 kHz affect the average person's appreciation of music?"
or even
"What percentage of humans in each age group can hear 23 kHz?"
or
"Is it worthwhile designing equipment to be accurate to 23 kHz because some humans can hear it?"

However, none of them obviates the original - and quite binary - question.
(Of course, the reverse is NOT true. If the answer to the original question turns out to be "No" then the other questions become moot.)

As for perfect pitch, I simply offer it as an example of an ability that only a few humans have.
If a certain recording was off-speed, but only one in a million humans had perfect pitch accurate enough to notice the flaw in that recording, we would still have to report that "the flaw is audible to some humans".
(And, in the binary case of the question, "the flaw is audible to humans".)
The fact that we can find millions of people who cannot hear it does NOT change that binary result.)

I trust you aren't addressing that comment to me. I hate generalizations without qualification.
And as I addressed briefly in my previous post, you'd have been wise not to make that bet. The fact that a high level, high frequency test tone was audible to me up to 23kHz at one point in my life means nothing in terms of that frequency and above contributing in any way to music preproduction. You are probably aware that the high end of human hearing is highly variable and depends on many factors, the big ones being age and hearing damage. The range of 15kHz detection, for example, is over 90dB across a wide population segment. That's 90dB, not 9 or so. 90. And believe it or not, that's partially correctable! But that should tell you a bit about how human hearing averages out. Young undamaged ears may detect high levels of sustained test tones above 20kHz, but over 20 years of age, that's pretty much gone already.

Please understand that hearing high frequencies is not a binary situation. It's not "hear it or you don't", it's a question of level vs frequency vs threshold of detection. Hearing response has quite a radical curve to it, even in ideal, young, undamaged ears. But the curve gets very, very steep above 20kHz where, in fact, the threshold of hearing at the threshold of pain intersect somewhere around 140dB SPL. Pretty much no point in designing for that condition, now is there?

Perfect pitch is completely outside of this discussion. It can be developed, and then later lost. Some come by it more readily, some almost innately, others not as much. And trust me, it's not always a welcome gift. But it has nothing to do with this discussion.

 

[RRod]

It isn't a simple 'yes/no', Keith. It's a simple 'yes/no' when you reference some SPL in some environment. I can't hear 2kHz at 0.1dBSPL in my living room...

 

[KeithEmo]

I'd certainly be willing to take the test...... (but how about putting the file up on a like site like WeTransfer, and posting the link, so we can ALL take it)......
And, by all means, report the results.

I will, however request that you publish ALL the results.
I want to see the overall number of correct guesses - so we can see that percentage.
But I also want to see a list of how many correct guesses each participant made (they can be anonymous).
That way, if two or three people are consistently correct a statistically significant portion of the time, showing that at least a few people can consistently tell the difference, we will see that as well.
Since this is a binary question.... knowing that even one or two people can consistently hear the difference would be highly significant.

It would also be nice if you could include a few different pieces of music... just so we can each pick out something we're somewhat familiar with.
(And, of course, we need to know the provenance of the original, so we can confirm that, on its own, it is of sufficient quality to show differences in other factors.)

However, I will note right up front that I am NOT claiming that I will personally be able to choose correctly.
After all, I'm almost 60 years old, so it seems pretty unlikely that I have the best hearing of any human on the planet - or even close.
And, no, even if it turns out that I hear no difference on a specific sample, produced in a specific way, I will probably not generalize that to every piece of music ever made.
(So I will continue to store my music in whatever form preserves the amount of information present in the original - as I've received it.)

If I was very pressed for space, I might decide to evaluate each individual piece of music, and use lossy compression for those where I heard no difference.
However, since I am not at all pressed for space, I would be unlikely to bother (the easiest way to ensure that it will not be audibly altered is simply not to alter it at all).

I can do that very easily. I have a simple blind listening test that takes two very difficult passages of music to compress without artifacting and runs it though a variety of codecs and bit rates. It's a very easy way to determine where your own threshold of transparency lies. I can offer it to you to as a single lossless file containing ten different encodings, from lossless all the way down to MP3 192. All you have to do is listen to the samples and rank them from best to worst. Easy, right? If you'd like to take the test, all you have to do is tell me if you want FLAC or ALAC.

OK. Now it's time for your response. Will you agree to take the test? That depends on whether you really *want* to know if you can hear a difference. If you take the test and determine that you can't hear the difference with a particular codec and bitrate, will you stop ripping to a lossless format and use compressed audio? If not, why? Because you would feel more secure knowing that it's "lossless" regardless of whether it sounds identical or not.

My test doesn't just determine the level where compressed audio becomes audibly transparent. It also tests whether you will allow knowledge of the truth to override your insecurities about lossy codecs "throwing out something you might need". Most audiophiles just refuse to take the test because being wrong is preferable to them than knowing the truth. That shows that ego is the real force driving their decisions, not audio fidelity. I can test for transparency, insecurity and ego all with one simple test.

 

[bigshot]

I have multiple copies of the test in different random orders. Plus I want people to give me their ranking before they find out their results. That’s why I administer the test individually in PM

It doesn’t matter really if a person can hear above 20kHz by the way, because those ultra high frequencies add nothing to perceived audio quality in music.

 

[KeithEmo]

We seem to be talking about different semantic interpretations here....

I tend to expect that any question I state will be considered USING THE CONDITIONS I INCLUDE AS PART OF THE QUESTION.
Therefore, if I ask: "What frequency range can a human hear at 80 dB SPL" then that's the information I want.
And, if I simply ask: "What frequency range can a human hear" then I'm opening that question to be considered under ALL possible conditions.

In other words, any question that fails to include specific conditions should be interpreted as a simple yes/no encompassing all possible conditions.

It isn't a simple 'yes/no', Keith. It's a simple 'yes/no' when you reference some SPL in some environment. I can't hear 2kHz at 0.1dBSPL in my living room...

 

[bigshot]

At 80dB I would think a human could perceive super audible frequencies. They couldn’t hear them, but they certainly could feel the headache it would probably create