Audio quality:
In the real world, we make stuff out of real-world parts (building blocks like transistors or even vacuum tubes have nonlinear electrical characteristics; other parts do too to some extent) and have to deal with things like thermal noise; it's impossible to build an amplifier or anything with ideal properties. You can't have 0 noise, 0 distortion, perfectly flat frequency and phase response, and more. You proposed something that just multiplies the input—that doesn't happen. However, people can use a wide variety of techniques to make that very close to a reality. How close? This is a matter of contention with audiophiles (in my opinion, without much merit), though for many other purposes and electronics applications, there are not the same kind of objections.
Some designers, constrained or motivated by cost, size, weight, ideals / philosophy, or goals, do not intend to get it as close as possible. There are a number of reasons why some amps (less commonly and significantly, for DACs) may sound different from one another, when driving certain headphones or IEMs.
Ex1) have a noisy amp and sensitive IEMs, you can hear the background noise.
Ex2) amp has high output impedance and headphones have impedance varying with frequency -> the amp no longer can be considered like an ideal voltage source because it is dividing the output voltage among its own output impedance and the headphones, resulting in an effective frequency response shift for the headphones.
As for some of the descriptions of sound for some of these devices? Listening impressions are valuable, but take everything with a grain of salt, especially if the listener knows what they're listening to. Often times, the perceived difference in amps is more a result of a difference in expectations or the way one is listening (or order effects, or many other factors) rather than the difference in signal produced.
Audio device size:
For desktop audio, the size of the enclosure may be significantly larger than that of the electronics. Often times, the jacks (the internal part you don't see) and thickness of the enclosure may be larger than you realize. The size is often dictated by usability and visual impression / handling points of view, rather than by the electronics. That said, there are some factors based on the electronics, detailed in the rest of this section. What takes up a lot of space are transformers, if one wants to do a linear internal power supply. The higher the power consumed by the device, the larger the transformers need to be. Likewise, the size of the capacitors needs to be larger as well. For amplifiers, some devices are a lot more powerful than others, and some are a lot less efficient than others, so more waste heat is produced. Class A amplifier operation is less efficient but has lower distortion to begin with, so don't consider low efficiency a bad thing (aside from using more power, arguably unnecessarily). Whenever there is relatively high power consumption, it is almost all going to waste heat, so space for heatsinks to dissipate the heat is necessary. Often times, the chassis itself is used as a heatsink. Also, keep in mind that high temperatures are bad for the longevity of components, so for example, a larger design might be a good idea to physically separate hot components from temperature-sensitive capacitors. At least, that would be a good practice for long-term reliability over many years. Through-hole components on a PCB take more space than small surface-mount components; the surface-mount components can be automatically soldered into place by machine.
Ex3) ODAC filters and regulates USB power, so it doesn't need transformers, and power consumption well under 1W so power circuitry can be miniscule; components are small and surface mount; only USB input and only one output, so little space required by jacks.
Ex4) E6 has low power consumption, high efficiency, low output power levels. It runs off a small battery and uses a low-power, highly integrated output chip which does amplification, volume control, and negative power supply rail generation all by itself. Compare this with the much more powerful and less efficient Lyr, which needs much larger components. It's possible to have a much more efficient design than the Lyr with similar output levels, but that's not their priority.
Audio device cost:
High costs mostly come from the relatively low volume of production—not many units are sold. Manufacturing costs can be high, even if the cost of materials is relatively low. In many designs, the cost of the chassis is higher than that of the electronics. The rest of the cost goes to profits and to pay for the R&D.
