Backup/restore: MMS images should be included by default and do NOT overwrite accounts if already there

Feature Requests
#1

During my preparations to move from Xperia X to a new 10 II, I have yet again battling with the issue that the photos in MMS messages are not included in the backup.

Last time I moved from Jolla 1 to X, I was able to get the pictures over by manually copying stuff from home/nemo/.local/share/commhistory over to the new device. Now in X → 10 II it’s not enough; the pictures are still missing:

Näyttökuva_20210606_001
Näyttökuva_20210606_0011080×606 173 KB

I did a quick attempt to do the symlink to nemo, which was the reported fix for the memos failing on restore, but it didn’t seem to affect the MMS photos.

Also while restoring stuff, I did get my email accounts restored from backups, that’s nice. BUT as I was restoring again (trying to fix the messages), the already-restored (and re-log’d in) accounts were over-written and I had to re-login them all.

Would be handy to have either option to skip restoring parts of the backup (ie. tick/untick messages, accounts, notes separately) or make it not overwrite accounts with same names, if they already exist on the device.

8 Likes
#2

Hi All,

There is a small number of topics on this matter, but it seems no definitive one.

My update to 4.4.0.58 went wrong, so I reflashed my device.

Prior to reflashing I had copied the contents of /home/nemo/.local/share/commhistory/ to my computer. After reflashing I copied the data back across to /home/defaultuser/.local/share/commhistory/ . This was all successful. Permissions for the files appear correct. I have tried both restarting after copying, and issueing command pkill jolla-messages after copying. I have also tried restoring from backup, and adding the data folder in manually after restore.

I can confirm all the data folder is there on the phone, and the images can be opened from the command line.

However, the messages app just shows place holders for the image as per @laustela’s post above. Clicking through on these thumbnails leads to ‘oops, cannot display image’ in the gallery.

I just had the obvious thought to see what the command line has. So I ran jolla-messages.
Clicking through to open the thumbnails as per above shows the commhistorydb is still looking in /home/nemo/… for the images, rather than /home/defaultuser/… .

So the big question is how can I alter my commhistorydb to point to the correct location after having reflashed and become defaultuser? There must be 1000’s of entries all pointing to the wrong location?

I could recreate /home/nemo/.local/share/commhistory/data/? Or I could reflash an old nemo edition of Sailfish OS, and just run through upgrades? Don’t know which one of those fixes is uglier?

Does anybody have any ideas?

Thanks everyone for looking.

2 Likes
#3

I have spent a hour or so looking at this, and have had no success. Hoping this catches someone’s helpful, knowledgeable eye. A small further update:

I have tried to dump the SQLite database as text, with the thought of using ‘find and replace’ for the data image location. But I don’t think I am doing it right. My computer can scarcely load the resulting file, and I can’t make any sense of the small section that eventually loads up.

Alternatively, looking around on the internet suggests there is an SQLite ‘Replace’ command. I haven’t managed to work out how to use it though. I am hoping I can issue a replace ‘/home/nemo/‘ with /home/defaultuser/‘? Is there anybody here familiar with SQLite and could offer some helpful suggestions?

I have also tried loading commhistory.db into Seqquel on Ubuntu, but I cannot work out how to connect to the db.

I am stuck. I wonder if Jolla Zendesk would help with something like this?

Thanks again all,
Justin.

1 Like
#4

I have some knowledge with SQLite. I can take a look on your problem, but can’t promise anything.

I think you have nothing to lost if you contact Jolla Zendesk.

2 Likes
#5

Just tested now and pictures seem to be stored at /home/defaultuser/.local/share/commhistory/data/<id>/<picture>.jpg

#6

Hi @pherjung,

Thanks so much for having a look at this. I really appreciate your help.

I will try and be clearer on my specific issue.

(1) Make a back up on SFOS 4.3, but on a phone that has been updated for some time (eg, Xperia X). The user on this phone is “nemo”.

(2) Run into some problems with upgrade to SFOS 4.4, decide to reflash, as possibly the easiest, cleanest way to move forward.

(3) Reflash Xperia X with SFOS 4.3. The user on this phone is now “defaultuser”.

(4) Restore commhistory.db (via copy and paste, or via SFOS in built backup mechanism) and copy in commhistory data folder.

(5) Existing entries in commhistory.db link to /home/nemo/.local/share/commhistory/data/<id>/<picture>.jpg . Note that it is /home/nemo/… . This location no longer exists. There is no user nemo, and no home directory for this non-existent user. All data is now stored at /home/defauluser/… . Thus, all the data links for existing messages are broken.

(6) New MMS are stored, and recalled correctly, at/from /home/defaultuser/.local/share/commhistory/data/<id>/<picture>.jpg .

Is there any way of correcting the existing data entries to point to the current storage location? The change in user from ‘nemo’ to ‘defaultuser’ has broken the links in this database.

Should I just recreate /home/nemo/.local/share/commhistory/data and store the old data there, and allow the messages app to store new data at /home/defaultuser/.local/share/commhistory/data?

This bug in the restoration of a backup made by user ‘nemo’, but restored by user ‘defaultuser’ must be a problem for more than 1 user?

I will attempt to raise a ticket with Jolla’s Zendesk after work tonight for some assistance if there are no easy answers?

Thanks again for all the help, I hope this is clear.
Justin.

1 Like
#7

First, who is the owner of your backup? nemo or defaultuser?
Second, I found two tables in commhistory.db which are related to pictures. Table Events and table MessageParts. In MessageParts, absolute path of each picture is stored. So first thing I’ll would try, is rename each field with the newer path.

1 Like
#8

I am not even sure how to tell who the owner is? ‘defaultuser’ must be able to write to it, as new SMS and MMS are successfully added?

#9

You get it using ls -l <file>.

1 Like
#10

It appears ‘defaultuser’ is the owner?

[defaultuser@XperiaXDualSIM ~]$ ls -l /home/defaultuser/
.local/share/commhistory/commhistory.db
-rw-r–r-- 1 defaultu defaultu 18722816 Mar 17 15:19
/home/defaultuser/.local/share/commhistory/commhistory.
db

#11

Be aware that I give you no guarantee !

I got something, not sure if it’s enough to fix your problem.

It should work directly on your Sailfish phone, but I prefer to modify the database on a computer then copy it on my phone.

# First go to the directory where commhistory.db is stored
cd ~/<path>/commhistory/
# Do a copy for security
cp commhistory.db commhistory.db.backup
# Then download the script : 
wget https://gist.githubusercontent.com/pherjung/476f3881e15498b4ab545416db58afb5/raw/3fcabe83a7006bb4159c9edf26a010a5dc50164c/convert.py
# Give him executable right
chmod u+x convert.py
# Execute it
./convert.py

It will replace every entry in the table MessageParts with nemo to defaultuser.

Once you transferred the database to your phone, be sure to set the good owner and group with

chown defaultuser:privileged commhistory.db
5 Likes
#12

Hi @pherjung,
This has done the job perfectly. I executed it all on device, and the script ran for about 30 seconds.
All the old MMS images now appear as expected, and new messages are all working too.
This is an amazing help, and thanks so much.
Any link to forward coffee/beer/life money?
Thanks again for the time.

2 Likes
#13

Great! I’m happy to see it worked :slight_smile:
No need of anything, I don’t have any link where to pay a beer/share money anyway. If you really wish to make a donation, I’ll be glad to see some big SailfishOS contributor receive it (coderus, adam pigg, rinigus, etc.).
Oh, but if once I visit your city I won’t refuse a beer/coffee :smiley:

1 Like
#14

Thanks, pherjung!

I used your script and now I have my old MMS images back, awesome!

However I must add; everyone, please make a full backup of commhistory. I messed up something (restored a system backup on one point as I was getting weird results in messages) and now I am missing couple of MMS message (only a couple of days and nothing super important). Should have paid more attention! Anyway I prefer the loss of two message to the fact that now I have all the old images back!

Two questions; running the convert.py creates the two extra files (commhistory.db-shm and commhistory.db-wal), should these be copied back to phone (and overwrite the old ones)? Also I didn’t have wget on my phone, and I got this error trying to run the last command: “chmod: invalid mode ‘defaultuser:privileged’”

#15

Maybe @pherjung ment chown? (CHange OWNer, basically chown user:group)

#16

mmh, effectively I meant chown…

#17

Actually, I don’t know what to do with the other files.