[Nextcloud]: Contacts sync unreliable

REPRODUCIBILITY: 100%
BUILD ID: 3.3.0.16
HARDWARE: Xperia X
UI LANGUAGE: German
REGRESSION: unknown

DESCRIPTION:

My contacts are not synced properly using the Nextcloud integration.
After upgrading to 3.3.0 I deactivated my existing CardDAV account and enabled contacts in the Nextcloud one. Everything seemed fine, new contacts were uploaded to as well as downloaded from the server.
Now a few days ago I deleted the old CardDAV accout in order to use the NC integration only.
After this, roughly half of my contacts disappeared and the most of the remainder got unlinked, meaning only a name remained, without any further details such as number, mail or birthday.
I could not get the sync to work again. I recreated the NC account twice with contacts enabled and rebooted the device. Only when I set up once again a CardDAV account, my contacts reappeared. I had to manually relink ‘name only’ contacts on the phone with their equivalent from the CardDAV account.

On a side note:
I presume that the remaining contacts somehow are stored locally on the phone and in the cloud. I never understood how linking contacts is working in the first place. It is very hard to get how contacts from the cloud are stored and why they are treated differently from those on the phone.

PRECONDITIONS:

Nextcloud intstance, active CardDAV account plus NC integration setup.

STEPS TO REPRODUCE:

  1. Disable CardDAV account
  2. Set up Nextcloud integration
  3. Remove CardDAV account

EXPECTED RESULT:

Contacts should be synced from same server via Nextcloud account

ACTUAL RESULT:

A few contacts remain, most disappear and the rest gets unlinked. Existing contacts are not fetched from server anymore

2 Likes

I think I started a discussion on TJC (or chimed into an existing one) asking myself similar questions.
And here is what I got at that same time (not too long ago :wink: ):
adding contacts directly on phone and getting synced from server is a different beast
contacts added on phone gets uploaded to server and e.g. are available there for syncing with other devices
contacts added from server to phone during sync are ‘volatile’, means get deleted on deleting the sync account and are NOT synced back up to any other configured account (which is correct imho).

To get a consistent overall state you would need to set up all sync accounts, download all contacts to phone, do an export to vcf, disable (delete?) all accounts, import back the vcf. This way you would have all contacts as local contacts. And all would be uploaded to all accounts.

Contacts added later on server side would again be ‘property’ of that server account only.

remark:
to be eaten with a lot of salt


3 Likes

Maybe would make sense if we talked about two different servers with a different (or at least not identical) set of contacts here. But it’s the same server. Everything you describe seems to involve multiple sources (read CardDAV servers) for contacts.

Having to export and import contacts the way you describe would qualify as a major usability issue for me, so also a bug.

It should be possible to reuse the same server using the new Nectloud integration and have the same sync behavior as before with the CardDAV setup.

Updated X and XA2 to 3.4.0.22. Nextcloud credentials were broken after the update so i have re-created the account. However, the contacts did not sync. Neither with default settings nor with full path provided (/remote.php/dav/addressbooks/users/).

As a workaround I have added additional WebDAV contacts synchronization with the above path. Would be nice though to have nextcloud working out of the box.

You mean you’re using CardDAV as a workaround?
That’s unfortunate, I was hoping 3.4 might fix this. I guess the underlying problem is the carddav client.

Could anyone from the dev team comment? Or @dcaliste, do you know something ? Is the behavior described in the OP intentional? Should contacts disappear once the account is removed and not be synced back using the same server but a different client (formerly CardDAV account , now Nextcloud)?

No sorry, my ignorance is total with respect to contact sync : ( I’m still using only one address book on my device. In case you don’t get answer here, you could raise the behaviour question in a community meeting. Chris Adams, who worked a lot on this topic, may answer there. In particular he recently upgraded QtPim to latest upstream (and made the upstream ready for Qt6).

Thank you, Damien. I’ll consider the community meeting. I’d prefer to ping Adams here but I don’t know if he’s present on the forum yet.

My nextcloud install ist an official docker container, URL has the form cloud.server.tld, .well-known shows the correct DAV path. Still, If I enter only my credentials and not the optional pathes to the CalDav or CardDav I get following error (on a freshly resetted XA2): “Coud not create account. Server response could not be processed.”

Windows/Linux Clients as well as Android tablet have no problem syncing Calendars and Contacts. If I enter the dav pathes, then the contacts are not syncronized.

Contact sync is horrible on current sailfish. The database model used for contact storage is creepy, and the sync mechanism has some dark magic which never does what you would expect :confused:

The only way to get it working correctly is to do all changes on the NC server, not doing any change on the phone. That way, you got a decent contact list, you can have multiple address books working fine.

Someone mentioned in another thread that things could improve with SF4. That would be very good news.

1 Like

@erikderzweite: This sounds like an unrelated issue, so you should open another bug report for this

@JulienBlanc:
Sorry for being blunt, no offense intended, but your post is not relevant. Do you have some useful technical background information to share?

Maybe I got this wrong but bug reports are for reporting issues, not commenting on things.

Sorry if my answer does not help you. However, what you’re encountering is quite normal given the current state contact sync is in sfos. Don’t expect having more than one carddav account (nextcloud integration count as a carddav account) to work bidirectionnaly.

Linked contacts are used to group together contacts of different sources. For example, you could link a nextcloud contact with a skype/whatsapp/whatever contact. It makes a lot of sense for services which have their own contact list. It also makes sense if you wish to link contacts which are both professional (for example, in your enterprise exchange address book) and personal (in your nc account).

When you create an account on SFOS, it by default creates it on the phone. It also creates (i don’t remember exactly if it’s at creation or at synchronization) an aggregate contact (ie, a link which currently holds only one contact). At synchronization, buteo creates a carddav contact, linked with the aggregate. What properties it chooses to replicate is where starts the dark magic. Then, that contact (the newly created one) is sent to the carddav server (if you got multiple ones, it picks one by default, not sure if it is the first added or the last added). Not that after this point, if you modify your contact, property changes are likely to never reach your carddav account, because they will only be done on the local phone contact, not the carddav one (you end up with 3 entries in the database)

When you create a contact on your carddav server, and when buteo receives it, it creates two local contacts: an aggregate and a carddav one. This carddav contact acts as a local cache, and will correctly receive updates from the server. You have 2 entries in the database. However, if you modify it on the phone, the phone will create a new local contact (thus, 3 entries in the database), and put most of your changes (especially, added properties, which are not handled the same as modified properties) there, which means they’ll never reach your carddav server. You’re back at the same point, where things just don’t work.

When you delete a carddav account, the cache entries related to this carddav account are deleted from the database. You end up with only local contacts, for contacts you’ve modified or added on the phone (the ones you never changed disappear as they no longer exists). This is what you got if i understand correctly.

In your case, the nextcloud integration should have recreated the contacts, however. There may be some issues, because i think it’s a bad idea to have two accounts pointing at the same server, at the same time. The steps you should do are more:

  • delete carddav sync (contacts should disappear)
  • set up nextcloud integration
  • sync and wait for contacts to show up

If it still does not work, then it will be time to start to look at buteo’s logs (https://sailfishos.org/wiki/CalDAV_and_CardDAV_Community_Contributions#Sync_Logs provides some useful information).

My advice remains: if you want a clean address book, synced with other devices (such as with you desktop email client), don’t modify your contacts on the phone. Do the change elsewhere (directly in nc, in your mail client, anything that has a good carddav support), and wait for the phone to receive them. That way it works nicely.

Disclaimer: what i state here is based on my own experience and investigation on how contact sync works in sfos, it may be inaccurate or wrong (it is probably wrong or imprecise to some extent).

4 Likes

Wow. Now I wasn’t expecting this much technical detail. Thank you!
Your elaborate explanation helps a lot, even if it may not be ‘tech spec’ level :smiley:
It’s much more clear now how syncing and linking is supposed to work and that it is not (and should not be) the same as merging contacts.
Maybe I’ll find some time to experiment again and have a look at buteo. There is some hope the upgrade to 3.4 may improve things, though.

So now I stand corrected: I too quickly dismissed your workaround suggestion of maintaining contacts only from the Server side and drew the conclusion your post would be ‘just another one’ of these complaints that ‘everythinng is not working’ with SFOS. Sorry for that.

2 Likes

Still having this problem on 4.2.0.21

After creating Nextcloud account, the contacts do not appear in the Sailfish’s app. Doesn’t matter whether with optional DAV path or without.

EDIT: I wanted to collect logs and after a reboot it worked (with DAV path).

1 Like

Try to disable the « recent contacts » in your nextcloud installation (it is usually there if you installed the mail application, but you can disable it alone and leave the mail app). For some reasons (and from what i’ve seen it is more a nextcloud bug), the presence of this address book causes issues with sfos.

Also, note that what i’ve written earlier is now completely out of date: a lot of things has been improved on contacts management with the 4.x releases.

2 Likes

Yes, disabling that ‘Recently Contacted’ address book fixed my Nextcloud integration. To do that you need to disable the Nextcloud app called ‘Contacts Interaction’ in your Nextcloud admin account.

1 Like

In recent version of Nextcloud there is also the automatically generated “Accounts” address book. This also blocks the contact sync for me.

See this post for how to disable this (purely from the CardDAV site): How to delete the "Accounts" address book - #6 by aaz - â„č Support - Nextcloud community