Lots of good replies thus far, my 2c to add:
- Modern CD players, as have been pointed out, are really into the realm of niche item, and the prices get very wonky very quickly. Like zero-to-$500 kind of wonky, and you're not getting much but a heavier chassis (whoop-de-do) or some badge engineering or similar. By contrast, DVD players (and to be even more fair, HD-DVD and Blu-ray players) are really the commodity item these days, and you can get into some very serious hardware at $200 (especially if you're okay buying something used). Folks tend to dump these things off when they become incompatible with whatever latest video standard (e.g. there are some majorly nice HD-DVD players and pre-BD Live or pre-4K Blu-ray players out there, which are perfectly fine for playing DVDs or CDs (and whatever their "original" media was), but they're not as in-demand because they won't stream Netflix or play 4K UHD or whatever - that doesn't make the internal hardware any less fancy, some examples from the land of HD-DVD include: Integra DHS8.8, Toshiba HD-XA1 and XA2; from the land of Blu-ray include: Yamaha BD-S1900, Pioneer BDP-09; from the land of straight-up DVD players include: most of Denon's DVD-#### line, like the DVD-3800). You can also go look at used CD players from an era when they were actually more common (e.g. the 1980s and 1990s), and these are more likely to have the transport controls you want ++ headphone output, but whether or not that headphone output is up to the task of driving a planar magnetic is another question entirely (that having been said, don't buy into the "they're all afterthoughts" marketing spiel - some CD players had pretty nice headphone amps in them, but again that doesn't mean they're necessarily up to running hard to drive cans). If you want to buy something brand-new, the last holdout that I'm aware of are broadcast/professional players (some of them will also be CDRW systems), like TASCAM or Denon DJ, which are still built for reliability and quality, but you may pay a bit over $200 for that.
- I completely "get" your desire for the all-in-one box - its very simple and its very easy to use. That said, you will still probably want a headphone amplifier down-chain from it, just for the sake of guaranteeing compatibility and functionality with whatever cans you have. Especially if you go the DVD/HD-DVD/Blu-ray player route, because very few of those will have headphone jacks built-in. There's tons of options here, if you don't already own such an amplifier.
- The [whatever] player will have DACs built-in for whatever it needs to decode, if it has analog outputs. There is a separate (and very niche) genre of "CD transports" that are digital-output-only devices, and assume you've got an external DAC. Newer Blu-ray players are also going this route (for an entirely different reason - analog sunset as part of their HDMI licencing requirements), and this is just something to keep in mind. If it has analog audio jacks, it has DACs for whatever its setup to play. Some of the DVD/Blu-ray players will also do DVD Audio and SACD, for example, and HD-DVD and Blu-ray players can generally decode very high bitrate lossless audio for the HD discs. That's all moot for pure CD playback, but again something to think about as you shop.
- Portable players usually are "worse" and usually in an appreciable way, because they were generally designed to be as cheap as possible, and many of them also have very aggressive anti-skip/anti-error/etc features built-in to try and make CD playback more survivable in an abusive mobile environment (these things had to survive teenagers after all). I wouldn't generally bother with them, but there's some older Sony and Philips units that are "special" for collectors, and have fairly robust guts - expect to pay for that rarity though.
- As far as "sound differences between CD players" - this can get into really contentious territory with some folks insisting you need $100k players and $50k cables and on and on just to hear "the full potential" of some $300 speakers or headphones, and anything less is just a waste of time/money/etc to use and own, and you've never "really heard" anything until you've spent so much. On the flip side you've got folks that will insist literally every player device ever made at any price point, from $20 no-name DVD players up to the $100k+ stuff, sound exactly alike no matter what, and that anything beyond the $20 device is a total scam, ripoff, fraud, etc. Yeah, both of those are extreme characterizations of very polarized and extreme viewpoints, and as with most things, the reality lies a lot closer to the middle. Specifically, I would say there can be room for sonic differences especially if you're looking at very old and very cheap (as in, it was cheap when it was new) gear versus modern gear or higher end gear, because very old/cheap digital gear wasn't the greatest stuff (and is largely responsible for most of the myths about "digititis" and whatnot these days). There's also concerns about reliability and functionality with cheap gear in general - that isn't to say all cheap stuff is bad stuff, but usually "you get what you pay for" has some truth to it. That said, multi-thousand-dollar gear just doesn't make sense imho - you're well past the point of diminishing/non-existent returns IMHO. I think your specified budget is a comfortable place to be - it isn't into the realm of 20kg chassis encapsulating 5kg off-the-shelf player and marking it up 17000%, but it should get you out of the all-plastic-fall-apart stuff.
Overall my advice would be to look at a used DVD/HD-DVD/Blu-ray player (and used gear in general), and go with an outboard headphone amplifier to simplify your life there (so you can get whatever player you want and not have to be chained to "well it also needs a headphone jack that's pretty good").
As far as bringing the PC into this: if you have the CDs already, and your PC actually has a CD drive (and I really feel old that I'm not getting my head around completely ditching removable media 100% these days), you can just rip the CDs into a lossless format (like flac or WMAL or ALAC). I like ExactAudioCopy because its simple and fast (and free), but lots of people like other software that will do the exact same thing, some of it you pay for and some of it you don't, and you can probably find folks willing to fight and argue about "what's better" at creating the same output - just get something that works and move on.

And yes, lossless rips will be 1:1 with whatever is coming off the disc, excepting if there's lots of errors (e.g. you bought used discs that have been used as hockey pucks). High bitrate lossy (e.g. 320k+ WMA/ m4a, mp3, etc) is also generally acceptably transparent, if you need to worry about space. I agree with iTunes being a pain to deal with - if you can ditch that and use something more straightforward that can dramatically improve the PC-based listening experience too.
