First of all UI, go simple(Hifiman), or go full on modern(Cowon)
With the simper one, you will be giving yourself less of a headache and focus more on people that want that device JUST for audio. The simpler one will have to work extremely well. The user will have to never feel angsty at moving the controls around or experiencing weird bugs and be organized and complex to sort files and folders as a niche user may want to, but also be simple enough to put them on and browse through them with a good non angsty user experience. Weed out all the bugs and make it look nice but not distracting or with contrasting colors.
If you go modern, generally you will be like Cowon, add more features in there and appeal to a niche and non niche crowd. However the problems start getting huge! You have to design a GUI that doesn't infringe on anything (or use Android, but that poses limits as well) and make it so like the simple one, it would flow and not make the user feel mad that it isn't moving fast enough (complex UI's make the user think it's more complex and want to make it go faster in my experience) or that there isn't enough options even though it looks so good. It takes time to develop and weed out bugs and adding more features like a small group of apps or camera and video support increase costs and add more layers of complexity and failing and even price.
Storage:
8-16-32-64GB FLASH storage options(heck, doing 8,16,32 or 16 and 32 would be fine) BUT with the option to have SDHC AND OR SDXC support.....this part is important. Give us a decent or thorough amount of internal space but there better be a external storage slot that supports SDHC at the least and or SDXC.
Audio:
Get your engineers on it and do it well. Get no gimmich snake oil engineers that know audio circuitry. There have been some devices that have, through bad cirucits or not enough knowledge designed bad intergration of theDAC and other parts surrouding it that have added noise and distortion and other problems. Avoid this. Get a decent DAC, nothing too fancy but nothing too low either. As a starting device for the company, aiming at the largest target market of $200 to $300 is what you need to focus on(this can also lead UI descisions) you will need to compete with Hifiman's or Cowon's and even the iPod(in the price range) but you don't need anything too fancy for a starting variant. Something to match up to the Hifiman 601 I guess. And with a good dual op amp setup or not. Choose these things wisely. Also......if at all possible...and this does lead to a big factor, have at least an Line out option (that is of decent quality and intergration to not add more noise etc) and if at all possible with your software and or hardware. Support digital out so a user can use their own DAC's. Of course a DAC's own USB power draw will be a problem so the higher you can make it without any ill's the better, although you might not need to worry about it too much as if your device is popular...the companies will think of a way! *wink* Also. Volume stepping, some don't do it correctly at low volume. Having it be able to suit a variety of headphones with a low impendance would also be good and being able to work with sensitive IEM's would also be great. Having a powerful (thinking about price as well) amp in there would also be great with full sizers.
Extras:
Depending on what markets you want to focus on, if you are putting out many devies at differnt price ranges or one, you can choose to include camera, video support and what not to get a market outside of the niche one. This is up to you, but as you are asking on head fi. Don't let it take priority over sound if you want to make a real audio device that we would love. I can see you scaling back some options to be able to pay for the extra support and do as you see fit, but don't let these take priority if you are making an audio lovers device.
No need for wifi or other stuff if you are going simple route.
Bundled accesories:
A USB wall charger would be nice but you don't need to, charing through comptuer and using our own USB wall charger will be fine, they aren't expensive and we generally have plenty, save more room and costs.
Headphones?: If you can include a set of SoundMagics, MEElectronics or whatever through an affiliation deal, that would be awesome, but as you need to drive down costs probably. Anything or NOTHING matters to us realy. We wont be using them.
Casing:
Make it look nice! Take some cues from Apple.
Build Quality:
Make it solid, but something that is affordable and you can work with, don't want it falling apart. Whatever you feel best.
Replaceable parts and battery:
It would be great if a replaceable battery was an option, as in user replaceable, whether you will allow it in warranty or not. Sometimes headphone jacks might break to, but this section isn't crucial although some will ask about replacing the battery, either themselves or through you guys.
Pricing:
Again, target market, how many devices, UI you will choose etc. I think $200-$300 base config device (even with 8GB of internal storage but with all the storage amenities above) is fine.
Screen and physical buttons:
Don't cheap out. Wiggly weirdly glued screens are a no no. Just a regular screen realy, if you are going simple it doesn't need to do much other than be able to show you what is going on and be reliable.
The buttons should be ergonomic. IT would be best to do multi handed, but as the iPod and iPhone has shown and taught you could just design a product that, althoguh would benefit more those with a right hand, left handers could easily adapt to. The volume buttons on the iPod's and iPhones are more suited for right handers, but it is also placed in a way that left handers can and could easily adapt to using it their own way.
Having enough buttons without overload or making the device "feel" weird or other user caused problems is important.
Touchscreen?:
Target market and UI and all those stuff again. I personally don't mind but if it costs a lot of money added on, isn't responsive, breaks easier. Then, no thank you. And honestly, not having one would be better too as an audiophile.
Format support:
FLAC, ALAC, AIFF,WAV
MP3 and variants, AAC
other variants of your choice
ALAC just recently became easy to use with it's parly kinda semi open source agreement terms so you may be able to put it on, it would be great.
Overal:
As an audiophile and I think you want to build an audiophile device, building a simple UI and style device would be best, a complex UI and device would drive costs up as developing and bugging it would cost money, adding wifi and other application writing would cost money and all those other extra's some feel to include.
Internal storage as I said is fine but SDHC and SDXC support would be exactly what I would want.
Good combo of DAC and op amp(s)
Line out option at the very least, digital out would be very very good.
One more note, change your title, nobody will know what you want people to answer.
