audio123 said:

  http://www.tested.com/tech/accessories/459117-science-and-myth-burning-headphones/
 
great article

Click to expand...

That article is what led me to Tyll's articles. I agree, great measured article. From the article, quoting Tyll:
 

"While the data showed only very small differences, the data was clearly above the noise, and a general trend observable," he writes. "While, it seems to me, much of the change observed could easily be due to movement, especially in the frequencies above 5kHz, some changes seem more likely due to break-in. In particular, the changes in frequency response around the fundamental resonance of the driver at 80Hz, and in %[Total Harmonic Distortion]+noise at the same frequency and at around 40Hz."

Later he performed a single-blind test and was able to distinguish between a broken-in pair of Q701s from a pair that hadn't been burned in. Hertsens writes:

"It's clear to me, having had the experience, that there is indeed an audible difference when breaking-in a pair of Q701 headphones. I've seen measured differences, and now experienced audible differences. While the measured differences are small, I believe the human perceptual system is exquisite and able to perceive, sometimes consciously and sometimes sub-consciously, subtle differences.

Click to expand...

Again, the data isn't conclusive, so prior belief will dictate how people react to this.

When I owned the P1, I noticed a very subtle difference in particular with the bass response from new to after a few hours.  I've always been in the middle, or on the fence so-to-speak between the side of subjectively and objectivity surrounding burn in phenomenon.  Both have validity and shouldn't be ignored or shunned upon, IMO.  IEM facts and opinions has just too many variables at play. 
 
Maybe I'm being to shallow-headed at this approach between the objective frequency response graphs and what the subjective listener's experience.  Would it make sense to dive deeper between the two approaches?  What about taking a single track, record and play it from start to finish with a reliable EQ software/hardware that records the track as it's being played.  Repeat the same process with the same variable(s)[bit rate, software, tips, cable...] after 50-100hrs of "burn in", overlay the two results and see if there are any changes with those minute details (assuming it's within audible range).  I'm guessing here, but it's these minute changes that's being represented in the listener's experience.

Just wanted to take a minute to say that Veritas was intended to be a tool for homebuilders who cannot afford an industry-standard measurement system. Reviewers have been using them, and I do think that some of them give the Veritas too much credit. Is it a substandard product? No. I use mine all the time to compare my builds to other Veritas graphs posted online, and it works awesome for that. It's also an $80 turnkey coupler. An industry-standard measurement system is upwards of $4000+.
 
I'm certainly not going to be using a Veritas to try to prove debated topics such as: burn in, cable impedance, tube rolling, etc. But I happily recommend them for people who want to compare earphones, test dampers, or build crossovers.
 
TLDR; Don't expect $4000 of performance from an $80 product.

@glassmonkey, a good part of the things you say or link to, are factual or at least relates to objective attempts at answering the question, and I thank you for that. it's so much better when we can get past the usual empty handed "I know what I heard!".
but IMO you seem to care about science and what we objectively know, mostly to serve your already made opinion on the matter. even on the stuff you link, you clearly do some cherry picking.
brooko may seem to do the same for his own opposing opinion, but if he doesn't feel like there was a significant difference, and if he fails to measure a significant difference, what do you expect him to do? at his own level on his own IEM the debate is as good as closed. whatever change that occurred, it's not obvious on the vibro veritas, and he failed to notice it.
burn-in in general is a different matter. there are probably enough different transducers for everybody to be both right and wrong depending on the situation.
 
 
 
a few things I believe are factual:
 
 
1/ nothing will stay as it is forever:
-so there is no point in trying to prove that something changed with time. because of course it did.
-instead the questions should be how fast a change? under which conditions? is it audible(as in really audible, not as in "I believe I remember a difference")?
 
 
 
2/ as a general approach: for speakers the coil is heavy, the size of the moving parts big, and the power used massive, so of course if there is a place to notice something, it's in the speakers. does that means we should expect the same magnitudes of impact from a tiny driver that needs to move very little air using very little power?
 
the main change on headphones seems to come from the pads. most tests done up until now seem to agree at least on that part. so anything done without removing the pads is IMO invalid for comparisons with IEMs or speakers.
 
 
 
3/ as a specific approach:  one case of that IEM changing a lot in a short time isn't evidence for all the IEMs of that model, and certainly not evidence for all transducers. what if that one IEM was defective and the significant change was a cue of it breaking very soon or at least going to crap?
at best the stuff we test will show indications of the presence or something, or fail to show it. it can point us toward a most likely possibility.
 
 
4/ changes we created ourselves:
-if like brooko and I, you try measuring stuff(even with cheap gear), you become aware of how much changing tips and the insertion of IEMs can have a very massive (measurable and audible) impact on the sound. who here can certify that he didn't change anything in his way to put on his IEMs over whatever delay he thinks the burn in manifested? it's of course a rhetorical question. that's close to impossible to be sure.  with silicone tips most IEMs move even in a single session, from jaw movements, wax, humidity...  the ear canal and even the tip may change shape a little over time if you keep using it as time passes for the "burn in".
 
-volume matching: who is sure he didn't touch his volume setting even once while testing his IEMs over the period of time when he noticed a difference? I'm pretty sure we can remove a good deal of people right there.
 
- who can be sure he didn't hit his IEM in any significant way over the burn in period? another significant group of burn in believers can be rejected again if they have the same butter fingers I was born with. if the IEM changed sound because of how you treated it, it's no burn in. same thing with ear wax, some create a little some a lot. if some wax reached the inside of the IEM and obstructed a filter a little, is that burn in? nope. I just change the filter or clean the tube and we're back to the initial signature.
 
so now how confident are people that they can vouch for those variables not being the cause of their perceived differences over time? that's what makes me uneasy when someone claims to know how things are. most audiophiles couldn't handle the preparations for a proper control over variables with the 2 sounds side by side at the same moment, but they're still confident enough to make claims about how things where 2 weeks ago or yesterday vs right now?  ignoring a all lot of variables doesn't just conveniently make them all go away.
 
 
5/ preconceptions: whatever they are, if you expect something to change, you know it will. that's how our brain does things. and it could very much be true the other way around, but that's no reason to neglect that very likely possibility for your own experience.
 
6/ memories: last but certainly not least. when did you guys become such amazing recording machines?
I can't remember a pretty girl's number long enough to enter it in my phone without having to ask her to repeat the end.
I always talk about the new toy effect, that stuff can give us the wrong idea about so many things, why should it be different for IEM burn in?
 
 
based on all this, I would personally be very reluctant to make any claim that what I remember and what happened are exactly the same, at hard to measure levels, no less. or that I I know I had no responsibility in the changes if they happened(which is still to be demonstrated for this specific model of IEM). so of course without objective data or testing, I wouldn't dare make any claim about the IEM burning in. and I'm always shocked when some people have so much self confidence that they can do it on a forum(for all the reasons above). I wonder if believing in something is really all it takes to declare it true to the world?

luke pighetti said:

  Just wanted to take a minute to say that Veritas was intended to be a tool for homebuilders who cannot afford an industry-standard measurement system. Reviewers have been using them, and I do think that some of them give the Veritas too much credit. Is it a substandard product? No. I use mine all the time to compare my builds to other Veritas graphs posted online, and it works awesome for that. It's also an $80 turnkey coupler. An industry-standard measurement system is upwards of $4000+.
 
I'm certainly not going to be using a Veritas to try to prove debated topics such as: burn in, cable impedance, tube rolling, etc. But I happily recommend them for people who want to compare earphones, test dampers, or build crossovers.
 
TLDR; Don't expect $4000 of performance from an $80 product.

Click to expand...

 
Thanks for turning up Luke.  FTR I don't expect to get the same results I would if I owned a GRAS or other TOTL product.  But as you said - it was designed for home builders of IEMs, and as such (correct me if I'm wrong) - it should be able to pick up changes of 0.5dB or more - when comparing the same IEM, same tips, same position of the IEM in the  coupler etc.  It is after all simply a mic in a coupling tube.
 
Because if you're suggesting that it can't reliably and consistently compare two measurements with the same basic parameters - then it isn't even meeting the home builders needs.  And I don't think thats what you're saying.  You'll notice when I review - I do use Veritas, and I'm very careful with disclaimers, and also to point out that it's most useful for comparative analysis.  This is in effect what I am doing here.  Simply comparing two readings - one "aged".
 
If you are suggesting Veritas can't do that reasonably reliably then you need to change your web page:
 

 Vibro Veritas is a complete IEM measurement solution. It will consistently measure frequency response for complete in-ear monitors and balanced armatures.

Click to expand...

 

  Our goal with Veritas is to jump-start the DIY IEM movement by providing cost effective tools to bridge the gap between a DIY measurement setup and a complete IEC 60318-4 ("711") compliant device. Vertias can measure CIEMs, UIEMs, earbuds, and bare armatures.

Veritas maintains exceptional consistency with back-to-back measurements. We made sure that unit-to-unit measurements are consistent enough for online collaboration.

Click to expand...

 
For the purpose of comparing two IEMs - under the same conditions - I think its delivering pretty consistent and reasonably accurate presentation of differences.
 
So suggesting this is not fit for purpose in the use I have for it is somewhat of a misnomer. It can perfectly well do what i want it to do - accurately measure the difference in frequency response between 2 IEMs.

Micah - I hope its OH - but I'm going to try and take your post in stages - and shorten it.  otherwise these posts get too long.  I'll try to also bullet point things so we get to the essence.
 
First point - you seem intent to put me in a box - so lets address that first
 

  Brooko

• I haven't measured it, therefore I'm in firm standing that it doesn't exist (this happens to be the exact same argument of Atheists to people with some sort of belief in a God, and is just as rude here)
• Provide proof that this difference exists that has nothing to do with actually listening to the headphones, i.e. lets ringfence what is considered acceptable evidence
• All subjective data is invalid (see point above)
• Dynamic drivers don't operate on the same principles when they are large and small
• Other elements have far more effect on sound (I'm not disputing this)
• The effect of burn-in is small (I'm not disputing this)

Click to expand...

 
Nope - you are jumping to some huge conclusions.

1. I have not noticed any effect that I can attribute to burn-in. I have also not measured anything I think can be attributed to burn in.  I'm not saying it cannot exist.  I am saying that I have found nothing so far which indicates the effects of burn-in are actually audible, and I would be open to changing my mind if someone can present me with something more than anecdotal "evidence".  BTW - putting me in a box and then suggesting I am being rude is pretty poor form.  Can we leave that stuff out and debate the points?
    
2. I would be more than happy to entertain any other form of empirical evidence if you can show me that it can be directly attributed to actual burn-in.  I personally don't find listening tests meet my own criteria.  We'll visit that shortly.  That addresses points 2&3
    
3. I think it would be better to say that dynamic drivers in earphones will behave differently than speakers.  We can leave this one out - as I don't have proof either way (I know they do not have a Spider though - which is one of the main components which reacts in larger speakers - its not needed in smaller speakers because they don't have the same rigidity concerns). I'm willing to leave this point out - if you are also willing to stop with the references to full sized speakers.  It's like comparing minis to tractor trailers.  Both are vehicles - but ......
    
4. 5&6 - we agree.

 
Now on yours:
 

  glassmonkey

• Not all instruments are sensitive to the difference you wish to capture
• The instruments Brooko is using are not designed for the argument he is making, as I've been told by the manufacturer of that gear
• Subjective data sometimes captures data that instruments aren't measuring, subjective experiences are not invalid and can be used to supplement quantitative data
• The amount of data necessary to show equivalence of sound before and after burn-in is insurmountable for a private citizen
• Companies are unlikely to share this data, as it is commercially sensitive
• The above factors mean that we will never be able to settle this argument
• Because we can't settle this argument we should try to be nicer to folks, i.e. not tell them that something doesn't exist just because we haven't observed it or don't have the capability to observe it.

Click to expand...

 

1. Going back to the original premise - if you can find me any measurement that will more accurate capture audible burn-in changes than frequency response, I would be more than happy to include that in any research.  All I'm looking for is truth.
2. The instruments I am using are more than capable of measuring as little of 0.5 dB change in frequency response and doing it consistently. Luke even claims it on his webpage (thanks for bringing him into this BTW - I appreciate it). If we can agree that burn-in changes generally claimed (if measured) would be in the magnitude of generally greater than 0.5-1.0 dB in frequency response (in order to be easily audible by those who make the claims), then this would save a lot of time.
3. Your point 3 I'm afraid I cannot agree with.  Can you please quantify what subjective data can capture that measurement cannot.  Please make any response relevant to the discussion at hand (ie burn-in, and specifically the MEE P1 if you can).
4. Point 4 I agree with - but that is the whole point of the debate.  Is what they think they are hearing really mechanical break-in?  Or is it brain burn-in?
5. I can't debate point 5.  All I know is that anyone who claims it should be willing to prove it with measurements.  So far no-one has (earphones).  Not one.  I think this is telling.
6. Your point 6 - then what is the point with this thread?
7. I really don't know where to go with your last point.  It appears to me as if its a dig.  And I can't see where I've made this personal.  Again - all I am looking for is truth.  So far I am not the one who hasn't "been nice".  I am quite happy to deal in facts and leave everything else aside.

 
 
More shortly.

  Brooko's primary argument, that if it isn't measurable--by his equipment, nonetheless--it doesn't exist is a fallacy as I've previously stated. Absence of evidence is not evidence of absence. Brooko's small collection of measurements on a variety of headphones with equipment that is not designed to discern minute differences, is no better than throwing forward completely subjective arguments. I provided an example of a rigorous systematic review that found that measurement instruments in quality of life did not capture elements of people's experience that subjective interviews consistently captured across numerous studies in a wide variety of geographic contexts. By Brooko's line of reasoning, I should have ignored the subjective data.

Click to expand...

 
Again - you are jumping to huge conclusions here - and I wish you would stop.  I will state again, I am open to the fact that break-in could exist, but failing conclusive evidence my findings so far is that as far as true audible results go, it does not have the effect that others propose.  I haven't heard it, I know enough about auditory memory to know that subjective haring tests are unreliable, and my proposition is that if burn-in does give the results that people calim - big changes to bass and treble - then it would show on a frequency plot.  If things like changes in position of headphones on a persons head, or depth / angle of insertion for IEMs can make audible differences and clearly be measured - why would break-in be different?
 
Question - are you saying audible break-in would not show up in a frequency plot
 
Quote:

I have argued that it may be possible to prove or disprove burn-in, but we need sensitive enough instruments to do it, which neither of us have. Asking me to produce measurements trying to dictate what that the heart of the argument is about objective measurements. What we are actually debating is whether those objective measurement tell the whole story. So, asking me to produce measurements is like saying I'm right because I don't believe in your premise for argument and I won't deign to look at alternative evidence. I've already acknowledged that the information we have available in measurable differences is not enough to support burn-in effects, but I don't think that we should ignore qualitative data, or that we should assume that our instruments are perfect instruments. Brooko's request is like saying prove me wrong using only the evidence that I want to hear. That's a bull-headed way to look at it, and I should know, because I'm frequently accused of being bull-headed.

Click to expand...

 
I've covered the equipment already - and look forward to Luke's reply.  Given the magnitude of the claims regarding burn-in (easily audible, big changes etc) - I find it unlikely that this would not show on a frequency plot.
 
You keep coming back to the "well I heard it" type of debate - and when presented with measurements, the standard response is that they aren't accurate enough.  I'd like you to remember this point as I'll cover it in my final post.
 
Quote:

I don't know how I'm supposed to debate someone who doesn't have any desire to do anything to counter my conclusion about temperance of conclusions and how people make decisions under uncertainty. Brooko has yet to provide a rigorous study of this topic, but has basically told me that I lose the debate if I can't.

Click to expand...

 
How can you make a decision under uncertainty?  You don't have the facts.  All you can do is make a guess.  If thats what you have - and you choose to discard anything else - then I agree, this thread is pointless - and I'm willing to disengage now.  The real fact is that you bring nothing new - just a return to the old "I heard it therefore it must be real".
 

I provided the example of equivalence trials, which is what would be necessary to prove that people can't actually tell the difference. Further than this you would need these trials to have a cross-over design, which is even further complicated. People would have to random order of headphone listening. There would have to be sufficient headphones to minimise the impact of confounders due to variability in manufacturing. Also, because we need to find out if people are just guessing, we need to have people repeat the evaluation multiple times. For validity in telling if someone can tell differences between how music sounds, multiple music samples would need to be available. Some of this material will be familiar with proponents of ABX testing, of which Brooko is one--a topic for another day or perhaps never. Of these elements of the study design, two reduce sample size: cross-over design, and repeat evaluations. All other factors make it very difficult and expensive to conduct the research. Here's some confounders you would have to adjust for additionally, listener expertise, age, and sex. Your randomisation could take care of those. To ask me to produce this evidence is unreasonable.

Click to expand...

 
Why would we need any of this?  Can we agree if its audible, its also measurable?  And also that if its clearly audible (again get to that shortly) then it should show up on a frequency plot?  If we can get to this point, then studies and trials you proposed aren't needed.
 

 Brooko has conveniently presented the conclusion of one of Tyll's posts, whilst ignoring the other. Tyll conducted a single-blind experiment wherein he had to identify which headphone he was listening to, the one that was burned in, or not. He identified 13 of 15, and then 5 of 5. That isn't likely to be a result that occurs by chance. That experiment had limitations: it was single blind, so it is possible that the other participant could have somehow clued Tyll into the headphone he was using, it is unclear whether pad wear was controlled for (I doubt it), there were not enough headphones, observers or observations. As I said in my first post responding to Brooko, there are too many confounders to be conclusive. Did Tyll hear a difference, yes, can we make any reliable conclusion on it, no. In this situation, whatever our previous knowledge is will dominate, under a Bayesian perspective. This is why neither Brooko, nor I are convinced to change our position.

Click to expand...

 
I actually find it funny that you presented your finding from the Matrix-Hifi post, claimed it was unlikely I'd read it, and then also put this bit about Tyll's second link - and claim that I'm ignoring it.  I read both.  I've read all of Tylls finding on burn-in (all the ones I know about anyway), because it interests me.  I was hoping I would find something more-  but all it has done is reinforced my own personal beliefs.
 
Lets look at the Matrix HiFi link.  First of all - it is all regarding speakers - not headphones, not earphones / IEMs.  By design they are different.  Also - the largest portion of the manufacturers polled (over 1/3) all agreed that any significant burn in would happen in the first few minutes.  Apply that to your observances of needing many hours for much smaller drivers - and any change not being noticeable until that time.  Yes - I read the entire thing (twice now).  I still say it has no relevance to our discussion.  How many earphones do you know with a Spider and paper surround by the way? And the burn in on the Sirius - are you really trying to say that MAtrix HiFi link has anything to contribute at all when you are talking about vast differences in driver design and material?
 
Going back to Tylls second link - and it was actually posted 4 months before the one with the measurements - not afterwards.  this is an important point I think.  To do this experiment he used two headphones.  Same manufacturer - same model - two headphones.  He did not measure both pairs before hand to see the difference in frequency.  this is important.  Here is a snipped from his conclusion at the time.
 

What Does It Mean?
Have we absolutely proven that break-in is an audible phenomenon? No. All I've proven is that I could tell one headphone from another. Proof positive is not easy to come by, and it's not something to claim lightly. However, I think this test moves us strongly in that direction.

Click to expand...

 
Interestingly in the comments - this is said (by someone who used to be a forum member here - is very conversant with measurements and things like designing EQs etc):

Manufacturing tolerances, my thought exactly.
I really respect what you're doing but that's a big problem in a test where you try to show that burn-in exists. You've seen yourself that different headphone samples (even if they have the same color and probably even if they're from the same batch) have slightly different channel balance, impedance, sensitivity/efficiency (-> different SPL).
What I'm trying to say is that I think you could achieve the same test results with two brand new Q701s or K701s.

Click to expand...

 
Tyll acknowledges this - which is partly the reason he continued to do the test in the first link which does not have the same flaws (4 months later).  You'll note that after that test his conclusions are very different.  What has happened in 4 months?  Maybe a better and more accurate means of testing break-in on a single pair of headphones.
 
This is why I ignored the subjective results from his earlier test.  It was flawed.  All he did was tell two headphones apart.  Check the AKG threads, you'll find a heap of information there about manufacturers tolerances - and OOTB AKGs sounding and measuring slightly differently.
 
Sorry - this got a bit long.  Next one coming soon.

OK for this one all I will deal with is your section:
 

On to my Bayesian perspective, which Brooko has chosen to ignore. I can think of several headphones that I've heard a difference on: HiFiMan RE0, the V-Moda XS, LH Labs Verb, Shozy Zero, Fidue Sirius. On the XS, Zero, and Sirius, I used the same basic method for burn-in, listen at baseline for a couple hours, put them in a drawer with something playing through them (I usually use a noise cocktail), listen to other headphones, check in on them over time, stop when I don't hear a change. I bolded listen to other headphones because this refutes the idea of brain burn-in to some degree. I've often found that when a pair of headphones sounds momentarily unpleasant that I've liked before, it is because I've gotten used to another pair of headphones. If I'm listening to other headphones in between, and the headphone hasn't changed, I should need brain burn-in every time I put them back on. Using this method, I've found that I don't always hear differences, and change usually stops in the first 24 hours of noise burn-in. This is consistent with some posts in the thread in matrixhifi, which I'm not convinced that Brooko even looked at. Ever since I first started burning stuff in, I've always done it since becoming a lurker on HeadFi in 2009, the differences I notice fall into the same themes: treble a little clearer, bass tighter, soundstage gets bigger, overall clarity improvement. All of these differences can be explained mechanically by reduced distortion. The select menu of possible differences presents one potential problem. I could be operating with expectation bias. The only way to know whether this is happening, is to have many individuals perform the same experiment where the potential for this bias is mitigated--not an easy task. I'm left having to trust my fallible ears. Based on this accumulation of evidence, I have a prior expectation that burn-in has a certain likelihood of affecting sound. Further experiments must modify this information. However, none of us have experiments that can do this.
 
<snip>
 
My arguments hinge on the premise that subjective qualitative data can be a valid means for interpretation. Because Brooko believes that all subjective evidence is invalid, I cannot convince him otherwise and it is pointless to debate. Neither of us have equipment that is sensitive enough to say with certainty that there is no measurable difference due to burn-in. When making decisions under uncertainty, we are informed by our prior beliefs. Brooko and I have different prior beliefs. I'm not trying to invalidate Brooko's view, it is perfectly defensible based on the evidence he believes is valid. I don't believe that his informative prior is any less valid than mine, for him. What I'm trying to get across is that he doesn't have the data to invalidate my view based on the evidence that I view as valid. This is a matter of analytical perspective and opinion, and at this point nobody has presented an argument based on indisputable objective facts.

Click to expand...

 
The issue I have with the normal "I heard it so it must exist" approach is this:
 

• We know that accurate and reliable auditory or echoic memory only lasts for around 10 seconds at most, and often is a lot less.
  https://en.wikipedia.org/wiki/Echoic_memory
  We can hold large amounts of very accurate date for a very short period of time
   
• Now look at the actual hard data we have from someone like Tyll's experiments.  Notice how minute the changes are.  You start to question how audible the changes are actually are.  Let's allow the possibility that it is audible.  Given that you last heard the headphones for a brief listening period many 10's or even 100's of hours earlier - given that we already know how highly inaccurate out actual echoic memory is - what compelling case does this give us to trust our ears - when we cannot remember in any detail the precise way the original sounded?  In effect we have no basis for comparison - none.  Just a distant memory with zero accuracy.

 
And that is my whole point about the trust your ears method.
 
Why is it legitimate for anyone to discard the actual measurements which can actually show changes pretty accurately to at least within 0.5 dB, while saying your own method is more accurate when given what is known about echoic memory actually suggests it is probably the least reliable method?
 
And why is it - that whenever these debates come up - the "trust my ears" guys always find whatever methods to obfuscate of counter measurements - whilst ignoring the elephant in the room - that anything they claim is anecdotal with zero evidence.
 
I'm willing to change my view if anyone can come up with something that has any basis in fact. So far no-one has ever done this. Do you find it strange that we have the technology that could actually measure this - and yet no-one has?
 
Yes - it is absence of evidence - but even that in itself is quite telling.

Last one - relating to the Fidue Sirius
 
Measurements - and yes I know, you refute the accuracy.  Usual disclaimer - Veritas is not calibrated, and does not give accurate raw data from 4-5 kHz north.  It seems to be pretty surprisingly good below that which is why i use it.
 
First one I took - maybe a few hours after getting them. I should have done it straight away.  It's using the Veritas recommended set-up (Startech card).  In both cases, tips are the same (Shure Olives), volume was the same (maxed on the E11K and Startech).  The settings on the PC I never touch (they are dialed in for the sound card).  Same settings in ARTA. You'll have to trust me that I'm not lying when I said I've taken them when I have - I'd like to think I have enough cred on these forums for people to believe me, and also enough ethics not to doctor the results.
 
I'd estimate the second reading (taken today) might be about the 40-50 hour mark.  No burn-in, just straight listening.
 

 
The blue is the original - red was today. And yes I had to re-seat the Sirius a few times until ti was getting a pretty accurate picture.  If that is doctoring the result - then mea culpa (i put my hand up).  To me it is simply trying to be accurate.
 
Now let's look at the Sirius another way - it has an internal bass vent (by that I mean the port is next to the nozzle - on the face that would be into your ear).
 
Completely blocking it will do this:
 

 
Neat huh?  Problem is that everyone will hear the Sirius a little (or a lot) differently due to whether your personal fit blocks or partially covers that port.
 
But here is the kicker - even making some minor adjustments in fit can affect the bass response (can change the port status).
 
Now notice how the changes are often automatically attributed to the possibility of burn-in.  What I've just posted does not disprove it - but it does suggest that the changes being heard are more likely to be something else.

It is consistent. Back to back measurements are dead on. There is some variation between Veritases (Veritae?), since I don't have a calibration system in place. I'm sure you've noticed that even the slightest change of insertion will throw the graph off more than 0.5dB. This isn't a fault of Veritas, it's the nature of any measurement coupler that doesn't have an interface made specifically to fit a specific model of earphone for end-of-line quality-assurance. (This is the main reason I stopped calibrating them.)
 
The most important thing is to not compare Veritas measurements to IEC60318-4 measurements ("711" coupler). Even though the Veritas was designed off the IEC60318-4 document, I only used the basics. If I used the entire document, it would have been prohibitively expensive for the homebuilder. 
 
But honestly I don't think that's a huge deal, since a lot of 711 measurements are weighted, corrected, raw, and often times never specified publicly. So even then it can be hard to make the $4000 instrument work for you if you're not working in industry.
 
It's important to note that the 711 exists not for reviewers to be able to tell the world what a unit sounds like with a graph... It's so that industry can collaborate on design, and have a ruler to measure earphone transducers.
 
Just the other day I was shopping for transducers from an overseas supplier and they presented me with a frequency response and THD graph. Since they used a 711 coupler, I can trust the accuracy, and test it with a 711 to confirm the quality of the units. As part of our courtship, they offered to send the toolroom quality jig that they use for testing this one particular driver so that we can proceed with future customizations and quality assurance. That's the point of a 711 coupler. You'd be insane to use Veritas for this. (And yeah, 711 coupler does approximate the impedance of the ear, but the standardization is the real value IMHO.)
 
So yeah, some guys like Speakerphone have a totally compliant 711 system, and that's awesome. But it's certainly not necessary unless you're working in industry or are a tool junkie. If all your work is "in house" then Veritas is suitable and a great value.
 
TLDR; don't compare Veritas to 711 graphs, be transparent about your data collection methods, insertion variation error is greater than Veritas precision error, 711 is a tool for industry to control product quality and allow communication between manufacturers.

luke pighetti said:

  It is consistent. Back to back measurements are dead on. There is some variation between Veritases (Veritae?), since I don't have a calibration system in place. I'm sure you've noticed that even the slightest change of insertion will throw the graph off more than 0.5dB. This isn't a fault of Veritas, it's the nature of any measurement coupler that doesn't have an interface made specifically to fit a specific model of earphone for end-of-line quality-assurance. (This is the main reason I stopped calibrating them.)
 
The most important thing is to not compare Veritas measurements to IEC60318-4 measurements ("711" coupler). Even though the Veritas was designed off the IEC60318-4 document, I only used the basics. If I used the entire document, it would have been prohibitively expensive for the homebuilder. 
 
But honestly I don't think that's a huge deal, since a lot of 711 measurements are weighted, corrected, raw, and often times never specified publicly. So even then it can be hard to make the $4000 instrument work for you if you're not working in industry.
 
It's important to note that the 711 exists not for reviewers to be able to tell the world what a unit sounds like with a graph... It's so that industry can collaborate on design, and have a ruler to measure earphone transducers.
 
Just the other day I was shopping for transducers from an overseas supplier and they presented me with a frequency response and THD graph. Since they used a 711 coupler, I can trust the accuracy, and test it with a 711 to confirm the quality of the units. As part of our courtship, they offered to send the toolroom quality jig that they use for testing this one particular driver so that we can proceed with future customizations and quality assurance. That's the point of a 711 coupler. You'd be insane to use Veritas for this. (And yeah, 711 coupler does approximate the impedance of the ear, but the standardization is the real value IMHO.)
 
So yeah, some guys like Speakerphone have a totally compliant 711 system, and that's awesome. But it's certainly not necessary unless you're working in industry or are a tool junkie. If all your work is "in house" then Veritas is suitable and a great value.
 
TLDR; don't compare Veritas to 711 graphs, be transparent about your data collection methods, insertion variation error is greater than Veritas precision error, 711 is a tool for industry to control product quality and allow communication between manufacturers.

Click to expand...

 
Thanks Luke - that is really informative.
 
So doing what I'm doing is also a reasonably reliable way to get an indicator if there is anything big going on with regard to frequency variation - as long as I'm:
 

1. Comparing rather than saying the measurements are stand alone accurate
2. Very careful with consistency in set-up, tips and insertion methods
3. Allow that minor variations can be caused by slight imperfections when coupling

 
But from what I've done so far using Veritas (for the purpose of measuring audible frequency change over time) - would you say that I should discard this data as unreliable?  It seems far more reliable and more accurate than actually "trusting my ears / echoic memory".

That's an excellent set of guidelines when using Veritas for review purposes!

brooko said:

 
Going back to Tylls second link - and it was actually posted 4 months before the one with the measurements - not afterwards.  this is an important point I think.  To do this experiment he used two headphones.  Same manufacturer - same model - two headphones.  He did not measure both pairs before hand to see the difference in frequency.  this is important.  Here is a snipped from his conclusion at the time.
 

Click to expand...

 
(Emphasis added)
 
I'd hesitate to call that an experiment, a non-random n of 2 with no control? Should totally be generalizable in its conclusion! 
 
With that out of the way, I've always thought that it was at least feasible (though probably unlikely) that dynamic headphones could experience burn in (that is audible to human ears). When people talk about burn in of all other pieces of the chain (DAPs, cables, amplifiers, DACs, etc.) it's just the ramblings of people that have no idea how electronics work. Since you hear discussion of those things burning in all over the place, I can't ever put stock in anything anyone says in regards to headphones burning in either. But because there is motion involved, and different materials whose elasticity might change over time, with dynamics it's at least possible. That said, there is so little motion in a headphone's driver, as you've mentioned, I've always found it unlikely that you'd be able to actually hear a difference, even if you could measure a difference with sufficiently sensitive equipment. 
 
My point is simply that Tyll has, perhaps, proven that 2 headphones sound different. But we have failed to control for a more likely culprit, as you've mentioned: manufacturing variance (or probably dozens of others). I'd wager we should care WAY more about that then any slight difference that might come about over time from burn in. When someone with a new headphone asks an owner that's had his or hers for a while if it sounds different after X hours listening, they're probably never going to get an answer that matters in any way. Your ears, the source material you're using, and that manufacturing variance will matter so much more (again along with probably dozens of other variables) as to make a discussion of burn-in with someone else on the internet almost pointless (at least in regards to deciding if you should keep a headphone that you don't like out of the box). Might be an interesting discussion for academic reasons, but I just can't see it mattering in any other way.

brooko said:

   
Thanks Luke - that is really informative.
 
So doing what I'm doing is also a reasonably reliable way to get an indicator if there is anything big going on with regard to frequency variation - as long as I'm:
 

1. Comparing rather than saying the measurements are stand alone accurate
2. Very careful with consistency in set-up, tips and insertion methods
3. Allow that minor variations can be caused by slight imperfections when coupling

 
But from what I've done so far using Veritas (for the purpose of measuring audible frequency change over time) - would you say that I should discard this data as unreliable?  It seems far more reliable and more accurate than actually "trusting my ears / echoic memory".

Click to expand...

I do agree that burn-in for IEMs is probably mostly psychoacoustic rather than physical changes to the IEMs. However, one thing that I haven't seen a lot of measurement on is how the decay response changes after burn-in. I.e. a frequency response curve over 100ms or something to form a 3D graph. It's not implausible that the initial and/or sustained frequency response is not changed much after burn-in, but the rate of decay (or the relative decay speed of different frequencies) might change after burn-in, which could potentially affect how people perceive the sound.

teotuf said:

 
I do agree that burn-in for IEMs is probably mostly psychoacoustic rather than physical changes to the IEMs. However, one thing that I haven't seen a lot of measurement on is how the decay response changes after burn-in. I.e. a frequency response curve over 100ms or something to form a 3D graph. It's not implausible that the initial and/or sustained frequency response is not changed much after burn-in, but the rate of decay (or the relative decay speed of different frequencies) might change after burn-in, which could potentially affect how people perceive the sound.

Click to expand...

 
Excellent point on the CSD's.  I'll start doing that as another comparison point in future.