RMAA and ABX comparison of 5G/6G iPods

[gnychis]

Hey all,

Okay, there has been plenty of speculation about what sounds better... 5G or 6G iPods. Furthermore, you can toss the iMods in to the mix and there is a whole new level. All we have are users saying "well, this sounds better!" or "that sounds better!" We all know that how something sounds to someone is subjective, but there are objective ways to measure audio quality.

So, I'm calling for an objective measurement using RMAA and ABX tools. When it comes down to it, maybe the 5G iMod is inferior or superior to the 6G dock out... maybe we prove that the headphone out of the 6G is far superior to the 5G. I just feel like theres something more we can do than posting speculation.

Anybody willing to participate? It would be interesting to have results on the following:

1. 4G iMod
2. 5G iPod Headphone
3. 5G iPod Dock
4. 5G iMod
5. 6G iPod Headphone
6. 6G iPod Dock

I have a 5.5G iPod that I can run the measurements on using a TURBO v11 dock. I've never run these type of tests, so if anyone has and can provide any sort of guidance it would be great.

Thanks!
George

 

[mirumu]

I don't think RMAA tests would be valid unless you're using the same measurement hardware for all the devices. If everyone just uses their own computer then individual tests may be corrupted in different ways due to natural device limitations and this would skew the results. You might end up concluding that the 5.5G iPod headphone jack sounds best purely because it was measured on better hardware. This would add to the confusion, not resolve it.

As far as an ABX test goes, that would mean getting all those models together in one place and doing a big shoot off. What would the methodology be? You could ask people to to tell the difference between the different iPods but to do so they would need to have spent quite some time listening to each iPod to get a feel for it's characteristics in advance otherwise they'll just be guessing. If you're getting them to choose which sounds best then you need to know up front which does sound best so that isn't going to work.

I'm sorry if I'm coming across negative here, I do appreciate what you're trying to achieve here but if it's done it has to be done in a meaningful way where the results will actually tell us something. I think some of the current speculation will die down naturally anyway when a few more people get to hear the new models and start giving their opinions.

 

[Febs]

Quote:

Originally Posted by mirumu /img/forum/go_quote.gif As far as an ABX test goes, that would mean getting all those models together in one place and doing a big shoot off. What would the methodology be? You could ask people to to tell the difference between the different iPods but to do so they would need to have spent quite some time listening to each iPod to get a feel for it's characteristics in advance otherwise they'll just be guessing. If you're getting them to choose which sounds best then you need to know up front which does sound best so that isn't going to work.

That's not how an ABX test works. In an ABX test, you listen to X, then you listen to A and B and then decide whether A = X or B = X.

 

[mirumu]

Quote:

Originally Posted by Febs /img/forum/go_quote.gif That's not how an ABX test works. In an ABX test, you listen to X, then you listen to A and B and then decide whether A = X or B = X.

Yes, I was talking more of the end results of the process in terms of what could be feasibly achieved with a large number of ABX comparisons between pairs of players. That is really my concern since ABX is really only useful to find if there is a noticeable difference between A & B or not. It wouldn't tell us which sounds best, just whether someone could tell two players apart with any accuracy. I'm suggesting that this doesn't really tell us something we could use to identify which iPod sounds best.

 

[bigshot]

I have a 3g, a 2 4s, and three 5.5gs. I haven't done A/B testing on the photo, but the others sound exactly like the CD with line level balanced AIFF files. I can't determine any sound quality difference between them at all with line out. They're all basically the same thing with different sizes of hard drives and batteries. The only difference is the headphone outs. But I use portable headphones in the headphone out, and only use my home cans with line out, so it doesn't matter.

See ya
Steve

 

[gnychis]

ahhh you're right, I didn't think about the computers being different which would effect the results. We could still do this using a pairwise methodology.

All we need is people with any two on the list. They compare those two using the same PC. That would give us an idea of A > or < B. Then if someone else compares A and D ... based on those results, its possible to place B in comparison to D.

Even if we can't compare them all in the end, it would be nice to get at least some comparisons of the 6G to the other models.

I'd be most interested in the 6G and 5.5G iMod comparison and 6G/5G dockout comparison.

Anyone have a 6G and 5G?

 

[gnychis]

bump... anyone with a 5G and 6G willing to do a comparison yet? Everyone is putting all kinds of speculation out there about 5G and 6G SQ... but no one seems willing to run any tests :\ I'm considering getting a 160gb anyways, if so I would be glad to participate.

 

[LawnGnome]

Like it's been said, the same recording device would need to be used.

I'd like to see the difference between a 1G shuffle, iMod, 5/5.5G iPod, and 6G iPod. Someone should have all of those except the iMod.