bigshot
Headphoneus Supremus
When I do my testing (between equipment or to determine different masters) I keep it simple. Level matched and double-blind with a couple friends or colleagues. Not sufficient for a peer reviewed paper but good enough for me. It's amazing the amount of times the test exposes the perception tricks of the brain.
^ This. I think everyone should do basic controlled listening tests for themselves. Too often I see people claiming tests are too hard to do, so they will just believe what they believe; or if a person's tests don't validate their preconceived ideas, they start nit-picking the technical aspects of the testing until it becomes impossible to please them. Like you say, a basic controlled test will get you consistent enough results to know which direction to go.
Good phono preamp itself can create such a large difference that you may think that you got different set of speakers.
There are things that make MUCH more of a difference than the phono preamp. It isn't rocket science to apply an RIAA curve and boost to line level. The trick is in the mechanical aspects... the size and shape of the stylus (elliptical, conical, etc.), the depth and condition of the grooves, the quality of the vinyl used, moving magnet vs moving coil, and the alignment and tracking. I have tens of thousands of records- both LPs and 78s. The amp is the easiest part of the process. Turning groove modulation into signals is the hard part. Once it's a signal, amplifying it is easy.