It sounds like those contacts are stored on your SIM card. Sailfish would obviously not be pushing this on you.
Duplicate of this?