For 2 sets of speakers, or one set of speakers and a headphone amp, splitters should have no noticeable volume or quality drop. In fact you can do the same thing to run 4 channel speakers with from one stereo output.
so lets say you have 10k on each, the load on the card will be 5k, most sound cards will drive that just fine.
I have tested it and it worked fine(poor mans 2p2t switch based stereo mirror
Do the headphones have a powered box(amp)? almost looks like.
EDIT, your post covered it(amp). you should be fine