From what I know about bluetooth, it stores some profile associated with some device paired in the past.
Look in /var/lib/bluetooth/. What could be tried is to reset BT - perhaps stop service remove directory start service, or reboot. Then pair again. If this does not help you could try bluetoothctl and advertise HFP (UUID: 0000111f-0000-1000-8000-00805f9b34fb)
I personally think it is a shame that bluetooth is in such shape in Xperia X. I mean Jolla, could have kept BT 4.x for a while longer as my car works perfectly well with Sailfish X 2.0 on Intex and not at all with 3.x.