I have the same issue with alarm sounds playing only through headphones on my Sony Xperia 10 III (os versions 4.5.0.19 and 4.5.0.24). The same happens with phone calls also, so if the headphones are plugged in, a call ringtone will play only through the headphones.
Additionally I’ve noticed that sometimes app audio will not play through headphones at all, everything will instead use the phone speaker. (Re-plugging headphones in and app restarts don’t fix this.) I don’t know if this is issue related, but audio though headphones will work again for me after restarting the phone.