Yes I use earbuds to listen at least 3/4 of the time. Even when I use headphones, only like 5-10% of the time is it one of my JVC SZ, i'm far more likely to use my Koss Portapros.
I am personally not really a fan of on- or over-ear headphones. Even just using earbuds there's a great improvement. The Sony XB90EX sounded great on the Walnut.
To be honest the Portapro is one of the most versatile headphones you'll find in the sub $50 region, if you don't have a set, see if you can try some out. I've had my current set for close to a decade and it's gone with me all sorts of places, shoved in bags and pockets, been all over the world. Still works perfectly and you'll find guys who have five and six digit long setups here will still enjoy Portapros with their high end gear.
Anyway. worry not, cause your headphones will sound awesome with the amping.