>> [Line out] Is it really much better than the regular headphone out?
Depends. Usually, yes. It's not trying to drive headphones, and most headphone amp circuits in PCDPs aren't exactly wonderful.
>> How does it compare to the output of home systems?
Depends. Some here will tell you any rack-style home CD player will sound better. Some (including me) believe that decent PCDPs sound nearly indistinguishable from typical rack players, when using the line out.
>> Is the main difference in the DAC?
Depends. (Is there an echo in here?) I think the analog output stage makes a bigger difference than the DAC in lower-cost players, whether portable or rack-style. But DACs can make a difference.
>> How much difference would I notice with an outboard DAC?
Depends.
If the headphones can resolve the kind of fine detail that differentiates DACs, you will notice a difference between them. An outboard DAC will have a far better analog output stage than any typical portable CD player or rack-style CD player. Personally, I think that's 75% of the difference in the first place.
DACs can't differ too much from each other without seriously coloring the sound. In other words, most inexpensive DACs, such as those used as components in typical CD players, manage to get pretty close to "accurate" as is. If a high-end, high-quality DAC sounds radically different than 99% of the consumer-grade DACs out there, you can be pretty darn sure it's the high-end DAC that's deviating from the "ideal." Of course, this might (subjectively) sound better to you. Tube amps color sound quite a bit -- it's the nature of tubes. Yet there are many folks who really like the warm (read: inaccurate, colored) sound you get from a high-end tube amp. Same with DACs.
Russ "The Peripatetic Audiophile"