Sounds like you bought a HEADPHONE splitter and not a HEADSET splitter. A headphone splitter splits the audio so 2 sets of headphones can be used to listen to the same source. A headset splitter splits a headset jack (i.e. iphone headset jack) into a headphone jack and a microphone jack so you can use a headset that uses a headphone plug and microphone plug.
This is the correct Y-Splitter that you should be using: http://www.amazon.com/StarTech-com-MUYHSMFF-Headset-Splitter-Adapter/dp/B004SP0WAQ/