Yes, it's definitely worth it, but you'll get the most with a lossless format. I just re-ripped my entire library in lossless and was amazed at the difference. This comes from someone that was extremely picky with my conversion methods. The better the cans the more noticeable things get. Good and bad.
Let me guess .....
- Testing was sighted
- Testing was not volume matched (using an SPL meter) - or using software like replay gain
- Perhaps different masters were used?
What were your files originally (format / bit-rate)? What encoder did you use?
Here - if you want to objectively retest - try this : http://www.head-fi.org/t/655879/setting-up-an-abx-test-simple-guide-to-ripping-tagging-transcoding
Rip your favourite track as flac. Transocde to aac256. Blind abx. Shall look forward to the results