[Ruben123]

Thanks. Interesting. Seems to make intuitive sense to me as long as we are talking about manufactures and the like who have a vested interest in the outcome of the studies.

Indeed. Think of a group of scientist, funded by pono or Sony, both advertising with HD music. What if the outcome would be negative? Probably the results wouldn't be published then, but what if, maybe by statistic chance, the results get positive? For sure Sony will publish it. Same for medicine unfortunately

 

[Orestes1984]

  Do enlighten. The comment doesn't at all seem to relate to what I was saying about accusing academics of dishonesty without providing even evidence that their studies had been refuted.
 
 
Interesting that the Theiss study is brought up when it's relative weight is 0.71%. It is contributing basically nothing to the analysis, why analyse it as an outlier, removing it will make no difference to the study results. I do agree that it is looks to be pretty bad. It only has 3 participants. How are you supposed to generalize that?
 
My experience with audio studies is that they are all pretty terrible, at least Reiss is doing something to try to bring the standard of evidence up by using systematic methods. If we add another trial, it doesn't mean that we should discard the information from the Reiss study. Instead the new study should be used to update what we know. If it is sufficiently rigorous, it will move the needle somewhere.
 
The methodological considerations raised by Reiss are valid, and future studies should investigate them.

I'm gonna go ahead and say I haven't been following this debate closely enough.

 

[pctazhp]

Indeed. Think of a group of scientist, funded by pono or Sony, both advertising with HD music. What if the outcome would be negative? Probably the results wouldn't be published then, but what if, maybe by statistic chance, the results get positive? For sure Sony will publish it. Same for medicine unfortunately

Not a great situation for those of us interested in audio. But when it comes to medicine, which can involve life and death, it is tragic.
 
Even though I don't understand a lot of the statistical stuff, I've been enjoying the discussion in this thread. Most participants seem to be trying to stay reasonable and civil. Unlike many similar debates that date as far back as the early days of high-end audio in the late 60s.

 

[RRod]

  Those kind of claims need proof. Where's the proof? There were only 4 of 18 studies that didn't find a mean effect showings a small difference in ability to discern. You've basically just accused 14 sets of study authors of academic dishonesty. I wouldn't do that lightly. If you've got proof or citations that refute the studies than it is your duty to provide that.
 
If the argument is that only the studies with statistically significant results were being dishonest than you've still got a problem as that list is 8 deep. I checked disclosure statements for a few of the studies, including Oohashi, and for the most part it looks like studies were funded through government grants, not sponsored by industry. Maybe you have different information?

 
Well the two Jackson papers are straight-up Meridian efforts. And getting money from a grant doesn't mean you don't have ties/interest elsewhere (Reiss himself has his LandR product). None of that is proof of anything but it's reason to maintain healthy skepticism at all times.
 

  Also, as the author points out in copious detail, the methods of the larger trials, which carry the greatest weight in the analysis (82%) have a strong tendency towards type II errors, that means estimating that there is no effect when there actually is an effect. This means that the results of the meta-analysis may even be conservative overall. This is why the author felt confident in making the press release statement cited--because the evidence points to this being the correct conclusion.
 
I don't have the time or will right now to re-run this using OpenBUGS and it's been a little while since I've done a meta-analysis, but that would give us something more manageable than a p-value. There are other less finicky software to re-analyze this, but they would give you a frequentist answer, and the Bayesian answer is more interesting to me. If anyone wants to step in for my laziness, feel free.

 
All his results pertain to differentiation, *not* advantage, which is what is quoted in the press release. In a world where distortions exist, making the jump from one to the other is perilous. As far as Bayes, I am likely to use a more conservative prior than you

 

[Joe Bloggs]

glassmonkeyIt turns out I'm not able to answer your question satisfactorily at this point. I merely stated the opinion I gathered from an informal reading of community reaction, and I thought there would be plenty of people coming up to substantiate these claims when questioned. Turns out not quite, maybe because this is not an AES forum

I'm no committed enough to buy all the articles being "meta-analysed" and tell you what is wrong with each of them--but that IMHO is what we have to do for ourselves at the end of the day, inspect each reviewed experiment in detail and draw our own conclusions about the validity of each, and how it jives with the "meta" article. The fact that very few people can afford the time to do this means a meaningless pissing match ensues.

I have one question to ask you though: would you just take the meta-analysis article's word for it that a particular study is "neutral" or biased toward "type II errors"?

As for the question of the "sample size of one", it does however concern what is arguably the most important sample point at the end of the day: yourself, and whether YOU can reliably tell the difference between high-res and standard-res. At the end of the day it doesn't matter if a meta-analysis of a thousand experiments say humans in general shouldn't be able to tell high-res out, if YOU can--or vice versa.

 

[pctazhp]

If anyone wants to see an example of how insidious upgrade fever can get, check this out: http://www.head-fi.org/t/816026/breaking-news-he1000-v2

 

[Win0ver]

For me the biggest problem in all of this is that it has become hard to find useful information.
 
For example, I used to own ER4S and K501s. The ER4S pretty much need an amp to be able to listen to classical music at acceptable volume out of a smartphone. So after a bit of research, I had bought a 150-200$ usb DAC/AMP (Heardroom's Airhead or something like that) and a Little Dot II.
 
The Airhead was made by Headroom and thus specifically made for audiophiles. You'd think it would be good enough, but it couldn't drive my K501 without heavy distortion and clipping. The Little Dot II I bought also had problems (as described here: http://www.head-fi.org/t/816323/little-dot-long-term-reliability-data-collection-thread-what-have-you-experienced).
 
Both of these amps were highly praised by head-fi members. 
 
One the other hand, I never had any problem with cheap USB audio interfaces made for musicians. For about 100-150$, you get a DAC, AMP, balanced inputs, inputs with phantom power, optical out, and other bells and whistles, and they all drove my K501, SR225 and ER4S without any problem whatsoever.
 
I'm looking to buy a DAC/AMP that will be able to drive K701 and HD650, and don't want to overspend. Seeing how hard it is to find accurate info, I just might buy another generic audio inferface.
 
(I might as well ask here since people in this thread aren't out of touch with reality: would a FiiO Q1 be enough for my needs?)

 

[glassmonkey]

I have one question to ask you though: would you just take the meta-analysis article's word for it that a particular study is "neutral" or biased toward "type II errors"?

The article has supplementary data that is pretty wide open. If you look at the end of the pdf you can get a link to download lots of data, including some risk of bias assessments. I think the article has done a fairly good job explaining the potential biases, and there is a little more info on the studies in the supplementary data. Every meta-analysis has to look at risk of bias, so this is standard practice. Whether a reader trusts the results is up to them, and as has consistently been pointed out, there are all kinds of uncertainty going on here and a relatively meagre difference, albeit statistically significant, from random chance. I think there is enough in the paper to warrant future exploration with more rigourous and repeatable methods, but I can't presage which way these studies would move the needle. Thanks for the PM with the links, by the way, very informative.
 
The supplementary data gives even less reason for confidence showing even more methodological flaws than the tables in the report do. It's pretty eye-opening. I'm not clear on what effect the methodological flaws would have, though. I'm not going to be examining this in more detail. I've spent too much of this day doing not the stuff that I should be doing, work for my job that pays the bills.

 

[goodvibes]

I don't get why anyone needs studies or to prove anything when making a buy. Listen for yourself and spend what you need to tickle your fancy.  Don't assume anything, the hype or skeptics, and just listen. If you don't hear a difference, buy the cheaper one. If you do, consider if it will make you happier without a compare. If nothing has a significant enough goose bump factor, pass. Overthinking it becomes paralysis by analysis when the point is to find some pleasure.

 

[HumdrumPenguin]

Haha it is funny you saying you prefer the sound quality in your $60 Grado than on the HD800. I have a IEM (JVC HA-FX750) that costed me 251.000 Won (around USD 237), and I prefer the sound on it then on my HD700, which was CAD 800 (after tax). Indeed we can't let ourselves be "fooled", for a lack of better words, by just looking at prices alone. I do believe that there are cans that really should cost around 1000 to 1500, but more than that, seriously, I think they are just WAAAY overpricing it. As long as there are people buying it, why not keep making it more expensive, right? I'm not saying cans that cost over it sound bad or whatnot, just that they are not worth it. You can get something amazing already spending around $1000.

 

[castleofargh]

 

   
Theiss is brought up to question the author's intentions, especially since he makes it a point to laud the blindfolding aspect in the paper. And you're NOT supposed to generalize it, and making it part of a meta-analysis doesn't legitimize it. And while we are waiting for future studies to be done, the hi-res world will be running with the press release about "advantages".

Other than having some vague notion of what Type 2 errors are, I'm completely lost in this conversation. I know it would be asking to much for someone to put it in terms of DBT Protocal for Dummies, but while this interchange may be helpful for those with the knowledge to participate in it or at least read and understand it, it does nothing to further education of most lay HeadFi participants. Perhaps you guys think we great unwashed masses are too far gone to reach. You may be right.
 
As far as future studies, despite all of the talk on this thread of the billions of dollars being raked in by hi-res providers and hi-end gear manufacturers, I'm guessing there is not sufficient "academic" interest in the subject or sufficient money to give hope that future valid and informative audio DBT studies will be forthcoming.

 
for those who didn't put enough XP in their math skill while leveling up over the years, or those like myself who simply had 20 years to forget, I believe this can be of interest:

 
 
@RRod I didn't move anything because while you guys talk obvious sorcery(the spanish inquisition will knock down your door when you least expect it), I find it good from time to time to have something that goes way over our heads. I can at least talk for myself, it keeps me curious, and certainly a little less cocky for a few hours before I remember that I do deserve to rule the world(the same thing we do every night Pinky).

 

[castleofargh]

  I don't get why anyone needs studies or to prove anything when making a buy. Listen for yourself and spend what you need to tickle your fancy.  Don't assume anything, the hype or skeptics, and just listen. If you don't hear a difference, buy the cheaper one. If you do, consider if it will make you happier without a compare. If nothing has a significant enough goose bump factor, pass. Overthinking it becomes paralysis by analysis when the point is to find some pleasure.

 I'm sure many people would love to just try everything and decide by themselves, but for the guy who used the same gear for the last 4 years and comes by only because something broke(which I imagine should still be the vast majority of people), how does he even know what to try(if he can)?  the answer usually comes down to price and FOTM, 2 of the worst indicators of quality or of what someone needs for himself. I personally would love to have more specs and easier comparative tools like we have when trying to buy a graphic card, a camera, a car ...  that way a smaller list that can realistically be tested by ear could be selected.

 

[goodvibes]

No doubt. Seems to be more difficult with audio as it appears our ears are better at the examining the whole than a group of #s might indicate. Worth the wait to listen since we don't all like or appreciate the same things. Headfi should be pre-listening exploratory research and not a buying forum. Don't want to think about the dubious reviews, whether intentional or not, that started trends. I remember a (I think Scottish)custom IEM that got a best ever review with a waiting line to buy. It was later discovered to be a rebadged Heir audio 4 that got panned for a severe frequency trough before getting revised...and at twice the price. 2 woofs with a twfk that almost always sounds too hot (to me) used that way. Those measurements did result in a positive change to that Heir so it's not all subjective either. That brings another question. It was a device that wouldn't be for me but if a buyer likes the unrevised unit better, is he wrong? 

 

 

[Dillan]

Going back to bias and subjectivity - I definitely can say with confidence the majority of my favorite listening sessions have been tequila or ale influenced. Manufacturers should start shipping shots with their gear for that ultimate first impression IMO.
 

 

[pedalhead]

So true. If I'm settling in for a serious evening listening session, a snifter of whisky is the perfect companion