Proposal for a standard test when comparing equipment

Apr 25, 2009 at 3:38 AM Thread Starter Post #1 of 10

odigg

500+ Head-Fier
Joined
Apr 1, 2008
Posts
868
Likes
26
Some people don't like DBT. Some people don't like total subjectivity with no controls whatsoever. I'm not trying to debate either point in this thread so don't interpret this post as an attack or support of either stance.

For all the people on this forum audio a is a great hobby. At the same time, finding "quality" information can be a pain. You don't know who to trust, what people have heard, a person's hearing ability, or much of anything else about many members. The exceptions are the members who have been here for a while and established their credentials and ability.

At least to me, part of the problem is that there seems to be no real standard for people to go by when they test/compare equipment and report differences. I understand people want compare equipment in a way that allows them to be comfortable. We don't want to limit ourselves with how we explore our hardware and be caught in burdensome standards of testing.

But having no proper standard seems to kill the quality of the information on this board. When I first came here to Head-Fi I took in everything everybody said. Now I've identified a few people I trust and follow what they say. A lot of other stuff is really just "lesiure" reading.

Before somebody says there are no standards on Head-Fi and there should not be, let me point out that there are some basic standards here. Even if Head-Fi members say "Everybody hears differently" we still do believe there are some objective answers. If somebody said "The K701 has way too much bass" many people would wonder from what viewpoint such a statement could be made. The K701 is known to be many things, but most certainly not a bassy headphone.

So I've been thinking about a standard people can use to compare equipment so that everybody has a basic reference point to think about when they read reviews and make comments.

The standard I'm thinking of comes from the following statement about human perception.

"Increases in volume are perceived as increases in sound quality."

This has been established by scientific research. I've read a lot of stuff about this in the past. I wanted to find some links so that people could not just take my word for it, but of course I've been searching for a little while and I cannot find a good article/paper on it right now. If only the Wikipedia search engine was a little better
smily_headphones1.gif


I think this statement of how we perceive volume changes is easy enough for people to test for themselves. Lower your headphone volume. Listen for a while. Then stop your music and turn up the volume. Now listen to music at this higher volume. Does it sound better? Unless the volume change is very small, it certainly does for me.

I think this point becomes even more pertinent when we compare two sonically different headphones like the Denon D2000 and the AKG K701. Most, if not all, people would say the D2000 has more bass than the K701. Yet, if you listen to the D2000 at a low volume and the K701 at a high volume, the differences may not be so easily apparent. The higher volume of the K701 has made the bass louder since the overall volume is louder.

There's obviously a lot more to the last two paragraphs than I've stated. I'm just trying to be concise.

Given this little bit of information about how human beings perceive volume changes, it seems to me that performing volume matched comparisons between equipment would help the overall quality of reviews on Head-Fi. That way we can remove the differences we hear simply because of a volume difference and focus and report on the actual differences between equipment.

Volume matching is easy. If you've got a multimeter you can do it that way. If you've got an SPL (volume) meter you can match volumes that way as well. You could even use your ears, but I don't think our ears were designed to judge volumes accurately. If they were nobody would ever destroy their hearing at an incredibly loud rock concert because they would be able to accurately judge how loud it was in that concert hall.

Personally, I do volume matched comparisons between all my equipment. I've built myself a small box that has two inputs and a headphone out. I can easily switch between the two pieces of hardware using a toggle switch. I'm into electronics DIY so I have a multimeter and SPL meter. Although I admit on some days I just volume match with my ears. Getting all the wires and connectors together to use a multimeter can be a pain.

The switching box isn't required though. You can just as easily move your plugs/wires/headphones from once piece of equipment to another.

What do people think? Good idea?
 
Apr 25, 2009 at 4:03 PM Post #2 of 10
Recently I recorded some samples from my 4 different CD players and ran some blind tests. I discovered that I could reliably (10/10) tell any CD player from any other CD player.

However the differences between them were between 0.7db and 1.8db. When I uniformly added the average difference between any two players to the quieter of the two samples (not even fantastically accurately) getting it to within about 0.1db max, the differences vanished and I was guessing.
 
May 5, 2009 at 9:40 AM Post #3 of 10
Well, well, well. Look what we have here.

Nick, I am afraid that I must agree with your suspicions. That is of you possibly going deaf with old age, and I'm afaraid no amount of measuring is going to bring you your youthful exhuberance back - if you think it does then you can read up about placebo - many posts in here about this. If you want an opinion of what something sounds like, perhaps you should read up on the opinions of those with vastly more hearing capacity than your own as the measuring will only leave you guessing. Nevertheless its interesting to find out about what will happen to my hearing in decades time from your experiences - cheers, big-ears!
bigsmile_face.gif
 
May 5, 2009 at 11:11 AM Post #4 of 10
Quote:

Originally Posted by odigg /img/forum/go_quote.gif
Personally, I do volume matched comparisons between all my equipment. I've built myself a small box that has two inputs and a headphone out. I can easily switch between the two pieces of hardware using a toggle switch. I'm into electronics DIY so I have a multimeter and SPL meter. Although I admit on some days I just volume match with my ears. Getting all the wires and connectors together to use a multimeter can be a pain.


But your toggle switch isn't audiophile quality. If you're going to test a $4,000 CD player, then you need to spend at least $2,000 on a toggle switch.

wink.gif


Good idea - there should be a how to on volume matching, as well as how to construct a suitable switchbox. As Nick Charles has experienced, the difference between level matched digital sources is vanishingly small. I completely agree with you about the perception of volume.
 
May 5, 2009 at 3:32 PM Post #5 of 10
Oh Wow...some replies
smily_headphones1.gif


Nick - I specifically left out blind testing because of all the objections associated with it. Since I generally have nobody to help me with my various audio tests (imagine how elated my spouse would be if I asked her), I do the tests when I'm fully aware of the eqiupment.

After a few toggles, however, I'm not sure what goes where anymore. Most everything sounds the same.

I've even tried volume matching headphones, but let's not go into that particular headache.

If this is something people are going to take seriously on Head-Fi I could write up a how to for volume testing and how to make a simple toggle switch based switchbox (or just a bunch of exposed wires in my case).
 
May 5, 2009 at 4:36 PM Post #6 of 10
Nice thread odigg.

Let me point out that when you "volume match" with your ears you are doing nothing. You have to be sure you are volume matching the audio equipment correctly in order to really compare two things.

Then have you thought about the box switch enough to realise that if it is badly built, that box might add some noise, etc?

This guys from matrixhifi have built an ABX box (version 2 already). They measured THD% rates, crosstalk vs frequency, etc... and it does not add or take anything. Excellent box prepared to DBT CD players, amplifiers, etc...
Can be looked here: ABX - Unidad de Conmutación Matrix - Versión 2


(Translated via google translator here: Versión traducida de http://matrixhifi.com/contenedor_abx_ver2.htm)

As a start i think it is good enough for reviewers to have better reviews between sources. Will edit something as I am having trouble expressing myself today
biggrin.gif
 
May 5, 2009 at 4:42 PM Post #7 of 10
I would love a guide on making a input/output relay based switch box for diy beginners. Would serve the community well I think.
 
May 5, 2009 at 5:04 PM Post #8 of 10
Quote:

Originally Posted by Bullseye /img/forum/go_quote.gif
Nice thread odigg.

Let me point out that when you "volume match" with your ears you are doing nothing. You have to be sure you are volume matching the audio equipment correctly in order to really compare two things.

Then have you thought about the box switch enough to realise that if it is badly built, that box might add some noise, etc?



Hi Bullseye,

I usually volume match using a multimeter at output that will be driving the headphones. When I'm really feeling lazy I just do it with my ears but I don't take such an evaluation very seriously.

A good AB switching box would be nice but I think it would kill most of the interest in this. That device on Matrix Hi-Fi looks like a serious project. From some other threads I've gathered that non DIY people can barely justify, to themselves, a basic SPL meter or multimeter. An argument for such a complex box is difficult.

The switching setup I have is VERY simple. It has three female jacks (two inputs and one output) and a toggle switch. There are a total of 9 wires (one for each channel and one for each ground for each of the jacks).

The design could be further simplified by using two male input jacks and one female output jack.

My logic in using such a simple design was that there is not much it will do with modifying the original signal. I probably should have run an RMAA test on it to confirm.

Beyond that, it's also difficult to argue that such a simple design can modify two inputs in such a way so that both end up sounding the same. The wires or toggle switch would have to be quite magical to support such an argument.
 
May 5, 2009 at 8:23 PM Post #9 of 10
All good ideas, but the people who most need to take testing seriously will shout you down from the rooftops.

Any objective measure that tends to disprove their worldview will be attacked.
 
May 6, 2009 at 3:32 PM Post #10 of 10
Quote:

Originally Posted by Uncle Erik /img/forum/go_quote.gif
All good ideas, but the people who most need to take testing seriously will shout you down from the rooftops.

Any objective measure that tends to disprove their worldview will be attacked.



Yep. The thing is, how far down the rabbit hole do you want to go? DBT supporters on this forums have formed certain opinions, but I think even the DBT supporters stop when it encroaches questioning this entire hobby.

An example. While DBT people say all well engineered SS headphone amps sound the same, I've only seen a few people ask if a headphone amp is required in the first place. This is a question that could be answered in a volume matched blind test.

I've done all sorts of volume matched tests with my computer's onboard sound as the baseline. I've used headphones ranging from high sensitivity low impedance to low sensitivity high impedance. I don't have a dedicated headphone amp anymore or even a dedicated DAC...

I think an interesting exercise would be if a bunch of us (people who do believe in DBT) would sit down together with dacs, amps (tube and SS), headphones, a computer and an EQ. The goal would be to eliminate everything that can be recreated using a combination of other equipment.

What would we walk out of that room with? Perhaps.

1. Computer
2. Decent sound card.
3. Equalizer.
4. The few headphones that can be equalized to sound like most other headphones.

Kind of scary. This hobby would become boring pretty quickly.
 

Users who are viewing this thread

Back
Top