I have some ideas, but what would be the most efficient method to perform this measurement?
I have a cheap multimeter, is this adequate? If you have the time and patience, what are the basic steps required to do this with some accuracy?
Even a cheap multimeter can be adequate as long as it can measure AC voltage with any reasonable accuracy. It does not need to show accurate absolute levels, since it will only be used for the purpose of testing if two voltages are equal.
You will also need a splitter to be able to measure the output voltage of the amplifier while it is driving the headphones. This is not required if you know for sure that the output impedance of both amplifiers is at least 100 times lower than the lowest impedance of the headphones, but if you are not using >= 250 ohms headphones, that is quite often not the case. The splitter can be made DIY, or bought for a few dollars. It is easiest to use a 1xfemale to 2xmale TRS Y-splitter, but those can be somewhat difficult to find (note that many similar looking splitters are sold for use with headsets, to be able connect them to a separate microphone input and headphone output, but those are not suitable for this application). Alternatively, one could use a combination of a much more common 1xmale to 2xfemale TRS Y-splitter (1/8" and 1/4" versions can both be found easily), and a simple male TRS to male TRS or male TRS to 2xmale RCA cable. For unloaded measurements, having only the latter would be enough.
Make sure that there are no contact problems, and that the cables and connectors used do not have unusually high resistance (like some particularly poor quality ones do).
For generating a test tone, you can use freely available software like SineGen or Audacity. Some multimeters (especially cheap true RMS capable ones) have problems measuring AC voltage and current at much higher than 50-60 Hz. This can be tested by measuring the unloaded output voltage at different frequencies. But if there is no extreme drop in the measured signal level at 1 kHz compared to 60 Hz (even a decrease by a factor of 2 can be OK, as knowing the exact absolute level is not important), then 1 kHz may still be recommended, because full size dynamic headphones tend to have less reactance at 1-2 kHz than in the bass range, so there is less chance of inaccurate matching due to output impedance effects, and the low kHz range also better represents the perceived loudness.
There are some potential problems that can make accurate matching within an error of 0.1 dB or 1% voltage difference difficult. Especially with digital or digitally controlled volume controls, the resolution might simply not be enough to achieve that accuracy. Digital volume controls can often be adjusted in steps of 1 or at best 0.5 dB. In the case of analog volume controls, depending on the quality of the potentiometer, its taper function, and the position it is used at, it can be difficult to make sufficiently fine adjustments. Additionally, channel balance is often an issue especially with potentiometers. If at least one of the amplifiers has a balance control, it could be used to match the channel balance as well, but unfortunately analog balance controls are uncommon on headphone amplifiers. If the mismatch is relatively small but not insignificant, then it is recommended to match the overall level of the two channels, which is the sum of the squared voltages (= power), but in practice a simple average is good enough as the difference is well below 0.1 dB at 1 dB channel imbalance.
If the accuracy of matching the levels and channel balance cannot be made good enough, the input levels to one of the amplifiers (i.e. the presumably worse one for those worried about any quality loss) may need to be controlled externally, either with a pre-amplifier or simple passive attenuator, or using a multi-channel DAC/sound card with software upmixing and controlling the levels on one pair of channels.