In principle you could isolate them by putting a transformer in between each input and the place where the signals are added together. Unfortunately, that necessarily means they're hooked to each other, too, and that can easily cause one or both to blow up.
A passive answer to that would be to hook both of the inputs to the headphone at the same time.
