Head-Fi.org › Forums › Equipment Forums › Sound Science › Null testing discussion (split from 'Is "High-End" audio a scam?')
New Posts  All Forums:Forum Nav:

Null testing discussion (split from 'Is "High-End" audio a scam?') - Page 2

post #16 of 30
Quote:
Originally Posted by billybob_jcv View Post

Subtracting two signals does not tell you which is "better", only that there is a difference from your standard. 

 

Point being, that if there's no audible difference between two components, then there's nothing with which to assess which is "better," save for looks, features, design philosophy, etc., all of which can be assessed without listening at all.

 

And it's also worth noting that even if the null residual is audible, it doesn't necessarily mean that the difference between the two components will be audible.

 

Bill Waslo's blind test challenge is a pretty good illustration of this. In it, he uses two files each of the same piece of music. However one of them has a Sousa band added at something like 70dB below reference level. If you play the two files through his DiffMaker program, you can easily hear the Sousa band playing. However I don't believe anyone has been able to reliably hear the difference between the two files.

 

se

post #17 of 30
Quote:
Originally Posted by kiteki View Post

 

It won't work for finding the difference between two A/DC's either, or transducers for that matter.  If a transducer is a TV screen, let's say OLED versus LCD, you record both, and play them on an LCD, that doesn't work, same principle applies to the A/DC section.

With a sufficient colour space in the recording to exceed the ranges of both monitor outputs it's possible.

Record playback monitor one this is A

Record playback monitor two this is B.

Temporally align first frame.

Subtract luminance value from B to A.

 

Example Pixel location 250 X 250 at frame 30 has an RGB value of 150,150,150 for monitor one and 148,148,148 for monitor two. Subtraction leaves a difference of 2,2,2 too low to be visible but multiply the values by 20 and it becomes visible. You could play this difference back on anything. 

post #18 of 30
Quote:
Originally Posted by JadeEast View Post

With a sufficient colour space in the recording to exceed the ranges of both monitor outputs it's possible.

Record playback monitor one this is A

Record playback monitor two this is B.

Temporally align first frame.

Subtract luminance value from B to A.

 

Example Pixel location 250 X 250 at frame 30 has an RGB value of 150,150,150 for monitor one and 148,148,148 for monitor two. Subtraction leaves a difference of 2,2,2 too low to be visible but multiply the values by 20 and it becomes visible. You could play this difference back on anything. 

 

How are you recording the pixels?  It's not enough to record the RGB values *sent* to the pixel - you would need to use a colorimeter to *measure* the RGB values displayed at that specific pixel location.  I don't think that's so easy, and measuring each pixel of a 1080P display, one pixel at a time, across multiple frames would take a while...

post #19 of 30

Yes, it would be impractical but it is theoretically possible. I'd use a camera and let compositing software take care of the "work." 

 

 

Warning: Spoiler! (Click to show)

Comp1.jpg


Edited by JadeEast - 7/10/12 at 11:54pm
post #20 of 30

Why are we talking about pixels?

 

se

post #21 of 30

It was proposed that null testing wasn't a valid way to examine the output of a number of things, the output of monitors was used as an example. By directly contesting the use of the example I was undermining the argument against null testing. 

post #22 of 30

Yes. I know why we were talking about pixels. I was just wanting to know WHY we were talking about pixels if you get my drift. biggrin.gif

 

se

post #23 of 30
Quote:
Originally Posted by Steve Eddy View Post

Yes. I know why we were talking about pixels. I was just wanting to know WHY we were talking about pixels if you get my drift. biggrin.gif

 

se

 

I do!  

 

I want to clarify that I never said null testing wasn't a valid way to determine a difference exists, only that in the end, I'm not sure it allows you to determine which of the two high-performing devices is "better".

post #24 of 30
Quote:
Originally Posted by billybob_jcv View Post

 

I do!  

 

I want to clarify that I never said null testing wasn't a valid way to determine a difference exists, only that in the end, I'm not sure it allows you to determine which of the two high-performing devices is "better".

 

Well that goes back to what I said previously.

 

If the null reveals no audible difference between the two said components, then the only criteria for judging "better" are those things having nothing to do with sound, i.e. looks, features, etc.

 

se

post #25 of 30
Quote:
Is the difference good or bad?  This is for you to determine, but at least you will know there is a difference.  This is the first principle of experimentation.  There must be an observable change in an amplifiers output for any difference to be heard.

Taken from an article about comparing amps using null testing.

http://sound.westhost.com/sim.htm

post #26 of 30
Quote:
Originally Posted by JadeEast View Post

Taken from an article about comparing amps using null testing.
http://sound.westhost.com/sim.htm
Mr Elliot has been an interesting source for me ever since I happened upon his writings.
post #27 of 30
As a small aside on the pixel null test: it really wouldn't work. Display technologies have wildly differing arrays of phosphors to represent individual pixels as well as differing pixel densities, distance between phosphors, etc., without even getting into color temps and reflections. example (Click to show)
223

You would have to do a lot of post-processing of the captured images to allow for any approximations which would cancel each other out.
post #28 of 30

 I will agree that it's a massive technical issue to pull off null testing of monitors, it is enough of an issue to make it useless in application. 


Edited by JadeEast - 7/11/12 at 4:11pm
post #29 of 30

I think it's just a matter of semantics - if we're talking about amps, cables and any other components where the inputs and outputs can be controlled and measured in a reliable and repeatable manner, then null testing makes perfect sense for determining if there is a difference.  The ideas put forth in the Rod Elliot article about examining the differences and trying to correlate them to the things we hear also makes sense.  He points out that we might not know exactly why there is a correlation when we see & hear a difference, and we might not know why we hear a difference that we can't see in the null-test, but that's OK - and I agree!  His point about hearing/seeing harsh noise in the difference signal and then extrapolating that to mean there will be audible harshness compared to the reference is also interesting - although it's probably harder when the difference signal becomes more complex.  I think where I started to get uneasy was when thinking about using null testing on harder to measure items like headphones & speakers (and display monitors).  I just think the likelihood of anyone outside of a serious audio lab being able to control the environment, test equipment (microphones, etc), sampling intervals, etc well enough for the difference tests of those devices to be meaningful would be very slim.  We feel fortunate when we get a reasonably accurate free-air FR of a speaker, let alone anything else.  Comparing FR curves of the same earphones tested by different people is even harder. 

 

As long as we aren't going to try to make absolute judgments based solely on differences measured under questionable test conditions, then count me in!  beerchug.gif

post #30 of 30
Quote:
Originally Posted by billybob_jcv View Post
As long as we aren't going to try to make absolute judgments based solely on differences measured under questionable test conditions, then count me in!  beerchug.gif

Have you got a Windows PC that you can listen to music on?  If you do, then why not try out some null tests for yourself?  You can decide whether your test conditions are questionable or not!  L3000.gif

 

I posted this in a different thread, but I think it bears repeating here.  Here's a nifty program that you can use to conduct null tests:  http://www.libinst.com/Audio%20DiffMaker.htm   It's free.  It's less than 2 megs.  Why not download it and take it out for a spin?  The author even includes ready made "dyf" files to play on it so you too can hear the differences between different things:   http://www.libinst.com/diffmaker_example_files.htm  It's fairly easy to play the dyf files (heck, I figured it out!).  Just be sure to click the "All-functions" choice in the menu bar so that it reads "Play-only" so you can load the dyf files and then extract the difference track between them. 

 

I especially recommend trying out the test tracks in the "...and a listener challenge" section of the dyf downloads.  It's a choir singing Brahms' "Lullaby" albeit with one of the tracks having a brass band playing a Sousa march mixed in.  The Sousa march is obvious once you extract and play the difference track.

 

A lot of the "golden ears" insist that ears are the most sensitive test equipment, that human hearing can detect things instruments cannot.  The null test is an objectivist broadside into the subjectivist position that ears are superior to test equipment.  If you can't hear the brass band playing, and I'm guessing most folks can't, then it's pretty clear that test equipment is more sensitive than human hearing.  

 

I think the Audio Diffmaker could be a very interesting tool to employ in audio reviews.  You say your fancy power cord makes a "night and day" difference in a stereo?  First use Audio Diffmaker to show me that there's any difference at all and then maybe we'll talk.  I know that for me personally, if I can't hear the brass band while the choir sings, and I can't, then I'm not going to worry about things like whether the jitter in a DAC is -105 db, or -110 db.  If I don't have golden ears then there's no point in me paying golden ear prices!

 

Speaking of golden ears, perhaps the Audio Diffmaker could be used to test not just components, but golden ears themselves?  Can the golden ears reliably identify the track with the brass band at -75 db?  at -80 db? at -90 db?  We frequently have golden ears insisting there are differences between amplifiers with ruler flat response and THD+N well below -100 db.  Shouldn't they first be able to demonstrate that they have golden ears, and just how golden their ears are, before we take their word on the differences between components that seem to have no measurable differences?     

New Posts  All Forums:Forum Nav:
  Return Home
  Back to Forum: Sound Science
Head-Fi.org › Forums › Equipment Forums › Sound Science › Null testing discussion (split from 'Is "High-End" audio a scam?')