I found that speed did matter with the Clip+ because of the time it takes for the Clip to update the library after anything is added or removed. I had a class 2 16 gb card and it took forever to update after I added something to the almost full card. Definitely try to move to a higher class card, unless you don't mind setting it aside after you add music to the Clip (either card or internal memory).
I agree that for playing the music, speed does not matter. Class 2 is sufficient.