[Bug] /usr/libexec/sailfish-osupdateservice osupdate-check sets ssu re

Thank you for denoting that executing sfos-upgrade --verify might be used (after applying the initial workarounds to resolve mismatching release information, see above) to bring an already affected SailfishOS installation (i.e., which already has RPMs for a newer SailfishOS release installed, and you are unable to determine which ones) into a state, where one can fix things more easily, backup data etc. But as @direc85 denoted, the ultimate aim should be to upgrade to that newer SailfishOS release.

For a little detail on the --verify option of sfos-upgrade, see its description at Github.
sfos-upgrade is available at Openrepos, so it can be easily installed via Storeman.

P.S. / disclaimer: While I am the author of sfos-upgrade, the idea for the --verify option and its initial implementation was performed by @direc85. Hence all this is just shameless bragging from him and me! :wink:
More seriously, without him hinting here that sfos-upgrade --verify can be useful to rectify a SailfisOS installation to an extent, so one can perform system administration tasks and ultimately upgrade to “heal” the damage done by installing RPMs for a newer SailfishOS version (by upgrading to that version), I would have forgotten about this application of sfos-upgrade.

1 Like

An enhancement for “Workaround a.”:

  1. Issue ssu re "$(grep -F VERSION_ID /etc/os-release | cut -f 2 -d "=")" # Just another way to write the extant command above
  2. rm -f /var/cache/ssu/features.ini # New, guarantees together with step 3, that SSU uses the correct repos
  3. ssu ur # New, update repositories configured for SSU
  4. Then continue as described: Reboot, etc. etc.
1 Like

Thanks for moving this bug report from TJC to here @olf. We still have the internal report about this bug, so I’ve tagged it as “tracked” here too (and updated our internal report pointing to the new info you’ve provided here).

1 Like

@Jolla, any update on this, 6 years after this bug has been originally filed and acknowledged by a sailor?

1 Like

You wouldn’t happen to have the device in RnD mode, e.g. reported by “ssu s”?

Generally the os update should be just asking the jolla store server whether there’s an update available, no funky stuff. However if there’s RnD mode on, it does checking with PackageKit and can involve also ssu version adjustments on the update size checks etc.Though even that should be restoring the older version after done checking.

Pekka, you already asked that and we discussed it in July 2019:

… Just to make sure ‘ssu s’ doesn’t return rnd mode on? …

Your last message in this discussion thread was:

… Need to investigate if there’s any chance of things going wrong.

To which I replied with two posts with additional information.

While my device-zoo is slightly different now (one Jolla 1 died and I bought an Xperia 10), situation is still the same for all devices:
RnD mode is not set according to ssu s.

The SSU domain was always set to sales.
Also SSU’s Device registration status: not registered was set all the time on these devices.

I only did not recently check if this is still valid (should I?):

I do have the rnd-updater installed (/usr/bin/rnd-dist-upgrade, which I did not install manually), though never had ssu set to “Domain: rnd”.

It’s actually amazing that after so many years since originally reported (and then discussed so many times both on TJC and here) this so annoying and troublesome issue still appears as something apparently new and unknown to Jolla guys :anguished:

1 Like

Ah sorry, didn’t remember the tjc discussion anymore at this point.

Anyhow as this is not something I can just easily test, having devices mostly on rolling devel, I’ve been rechecking the code. Still haven’t spotted anything that suspicious if the RnD mode is not set.

The /usr/bin/rnd-dist-upgrade should be fine. It’s installed by jolla-developer-mode.

1 Like

Thank you for taking the time. Unfortunately you arrived at the same conclusion after doing the same in 2019, though SailfishOS users who have not installed the most recent OS release are still haunted by the dire effects when SSU is automagically set to a higher version than the SailfishOS release installed.

Anyhow as this is not something I can just easily test, having devices mostly on rolling devel, …

In my perception this is part of a greater issue: Nobody at Jolla seems to have a testing device with an older SailfishOS release installed, because Jolla pursues the policy “users must have the most recent SailfishOS release installed, or we do not support them”. Unfortunately it is Jolla’s disruptive changes in SailfishOS making it infeasible for users to follow this policy, because these changes regularly break a lot of third party software from OpenRepos etc. out of the sudden and regularly introduce regressions in components provided by Jolla. If Jolla would pay close attention to backward compatibility and thorough regression testing, then their “you must be on most recent SailfishOS release”-policy becomes a realistic assumption; until this is the case it is an unrealistic assessment.

Well, I am sure willing to perform any test, which does not endanger a working SailfishOS installation, if you instruct me what to do and/or check. I assume a few others are willing to do the same. But automatically and unvoluntarily I will not gather additional data points, because currently (actually for 5 years now) I prevent osupdate-check to be automatically executed on all of my SailfishOS devices.


Side note

The /usr/bin/rnd-dist-upgrade should be fine. It’s installed by jolla-developer-mode.

Thank you for the info.
Although now I wonder, why everybody who switches “developer-mode” on (i.e. root access; something most users want) is provided with /usr/bin/rnd-dist-upgrade, as RnD-mode seems to be for core SailfishOS developers only, i.e. sailors (= Jolla employees), but not even device porters from the community. But never mind, if accidentally executing the /usr/bin/rnd-dist-upgrade binary is harmless.

3 Likes

I think just got hit by this bug again, on the Jolla tablet.

That was running 4.6.0.11, and when I heard .12 was there I booted it up and ran a zypper ref; zypper up as root.

To my surprise, zypper reported some core packages to update:

All repositories have been refreshed.
Loading repository data...
Reading installed packages...

The following 3 package updates will NOT be installed:
  harbour-fahrplan2 harbour-tidings harbour-uptimed

The following 19 packages are going to be upgraded:
  apkd apkd-android-settings apkd-config-home droid-bin-tbj droid-config-tbj droid-config-tbj-bluez5
  droid-config-tbj-pulseaudio-settings droid-config-tbj-sailfish geoclue-provider-hybris-hal jolla-xt9-cp libsailfish-eas
  libsailfish-eas-common nethack nethack-keyboard patterns-sailfish-device-adaptation-tbj
  patterns-sailfish-device-configuration-common-tbj patterns-sailfish-device-configuration-tbj qmf-eas-plugin sailfish-eas

19 packages to upgrade.
Overall download size: 1.9 MiB. Already cached: 98.6 MiB. After the operation, 275.5 MiB will be freed.
Continue? [y/n/v/...? shows all options] (y): n

I made sure I was not in the “ssu has too new version” situation, but no. Both ssu version as well as repos looked OK.

[root@JollaTab nemo]# ssu re
Device release is currently: 4.6.0.11
[root@JollaTab nemo]# ssu r
Username: ^C
[root@JollaTab nemo]# ssu lr | grep 4.6
 - adaptation-common ... https://releases.jolla.com/releases/4.6.0.11/jolla-hw/adaptation-common/i486/
 - adaptation0       ... https://store-repository.jolla.com/releases/4.6.0.11/jolla-hw/adaptation-intel-tbj/i486/
 - aliendalvik       ... https://store-repository.jolla.com/releases/4.6.0.11/aliendalvik/tbj/
 - apps              ... https://releases.jolla.com/jolla-apps/4.6.0.11/i486/
 - customer-jolla    ... https://releases.jolla.com/features/4.6.0.11/customers/jolla/i486/
 - hotfixes          ... https://releases.jolla.com/releases/4.6.0.11/hotfixes/i486/
 - jolla             ... https://releases.jolla.com/releases/4.6.0.11/jolla/i486/
 - sailfish-eas      ... https://store-repository.jolla.com/features/4.6.0.11/sailfish-eas/i486/
 - xt9               ... https://store-repository.jolla.com/features/4.6.0.11/xt9/i486/
 - harbour-storeman-obs      ... https://repo.sailfishos.org/obs/home:/olf:/harbour-storeman/4.6.0.11_i486/
 - sailfishos-chum-testing   ... https://repo.sailfishos.org/obs/sailfishos:/chum:/testing/4.6_i486/
 - store                     ... https://store-repository.jolla.com/tbj/i486/?version=4.6.0.11
 - home ... https://download.jollamobile.com/home:/honeybadger/latest_i486/
[root@JollaTab nemo]#

Alll looked in order, so I did zypper up again:

The following 19 packages are going to be upgraded:
  apkd apkd-android-settings apkd-config-home droid-bin-tbj droid-config-tbj droid-config-tbj-bluez5 droid-config-tbj-pulseaudio-settings droid-config-tbj-sailfish geoclue-provider-hybris-hal
  jolla-xt9-cp libsailfish-eas libsailfish-eas-common nethack nethack-keyboard patterns-sailfish-device-adaptation-tbj patterns-sailfish-device-configuration-common-tbj
  patterns-sailfish-device-configuration-tbj qmf-eas-plugin sailfish-eas

19 packages to upgrade.
Overall download size: 1.9 MiB. Already cached: 98.6 MiB. After the operation, 275.5 MiB will be freed.
Continue? [y/n/v/...? shows all options] (y): y
In cache apkd-config-home-0.8.24-1.4.2.jolla.i486.rpm                                                                                                                     (1/19),  26.7 KiB
In cache droid-bin-tbj-0.0.18-1.1.17.jolla.i486.rpm                                                                                                                       (2/19),  94.9 MiB
In cache droid-config-tbj-bluez5-0.9.31-1.6.1.jolla.i486.rpm                                                                                                              (3/19),  45.2 KiB
In cache droid-config-tbj-pulseaudio-settings-0.9.31-1.6.1.jolla.i486.rpm                                                                                                 (4/19),  49.1 KiB
In cache geoclue-provider-hybris-hal-0.2.36-1.5.1.jolla.i486.rpm                                                                                                          (5/19),  92.8 KiB
In cache patterns-sailfish-device-configuration-common-tbj-0.9.31-1.6.1.jolla.i486.rpm                                                                                    (6/19),  38.2 KiB
In cache apkd-0.8.24-1.4.2.jolla.i486.rpm                                                                                                                                 (7/19), 116.4 KiB
In cache droid-config-tbj-0.9.31-1.6.1.jolla.i486.rpm                                                                                                                     (8/19),  69.2 KiB
In cache apkd-android-settings-0.8.24-1.4.2.jolla.i486.rpm                                                                                                                (9/19),  27.0 KiB
In cache droid-config-tbj-sailfish-0.9.31-1.6.1.jolla.noarch.rpm                                                                                                         (10/19),  40.0 KiB
In cache patterns-sailfish-device-adaptation-tbj-0.9.31-1.6.1.jolla.i486.rpm                                                                                             (11/19),  39.0 KiB
In cache patterns-sailfish-device-configuration-tbj-0.9.31-1.6.1.jolla.i486.rpm                                                                                          (12/19),  38.3 KiB
In cache libsailfish-eas-common-0.6.9-1.10.1.jolla.i486.rpm                                                                                                              (13/19),  87.5 KiB
In cache libsailfish-eas-0.6.9-1.10.1.jolla.i486.rpm                                                                                                                     (14/19), 934.2 KiB
In cache sailfish-eas-0.6.9-1.10.1.jolla.i486.rpm                                                                                                                        (15/19),  75.9 KiB
In cache qmf-eas-plugin-0.4.6-1.6.3.jolla.i486.rpm                                                                                                                       (16/19), 203.4 KiB
Retrieving: nethack-3.6.7+obs3-1.9.1.jolla.i486 (sailfishos-chum-testing)                                                                                                (17/19),   1.4 MiB
Retrieving: nethack-3.6.7+obs3-1.9.1.jolla.i486.rpm .......................................................................................................................[done (447.1 KiB/s)]
Retrieving: nethack-keyboard-3.6.7+obs3-1.9.1.jolla.noarch (sailfishos-chum-testing)                                                                                     (18/19), 460.3 KiB
Retrieving: nethack-keyboard-3.6.7+obs3-1.9.1.jolla.noarch.rpm ............................................................................................................[done (460.3 KiB/s)]
In cache jolla-xt9-cp-0.2.15-1.4.2.jolla.i486.rpm

Soo those update packages are FROM THE CACHE??? Why? How?

Anyway I did the update through zypper, and then

[root@JollaTab nemo]# sfos-upgrade 4.6.0.12
...
 Fetching and installing the SailfishOS upgrade from 4.6.0.11 to 4.6.0.12 (this may take a while):
REFRESHING CACHE AND DOWNLOADING PACKAGES
Finished transaction (status=1, runtime=76429ms)
UPGRADING SYSTEM
Finished transaction (status=1, runtime=38047ms)
FINISHING

Install (26 packages)
 - bluetooth-rfkill-event;1.1.1-1.3.5.jolla;i486;adaptation0
 - buteo-sync-plugins-sailfisheas;0.1.4-1.6.3.jolla;i486;sailfish-eas
 - droid-hal-version-tbj;1.0.0-1.4.7.jolla;i486;adaptation0
 - harbour-barcode;1.0.53-1.10.1.jolla;i486;sailfishos-chum-testing
 - harbour-fahrplan2;2.0.46-1.31.1.jolla;i486;sailfishos-chum-testing
 - harbour-foilauth;1.1.9-1.11.1.jolla;i486;sailfishos-chum-testing
 - harbour-qrclip;1.0.11-1.2.1.jolla;i486;sailfishos-chum-testing
 - harbour-tidings;1.4.2-1.15.1.jolla;i486;sailfishos-chum-testing
 - harbour-uptimed;0.4.6+git6-1.17.1.jolla;i486;sailfishos-chum-testing
 - harbour-wordle;1.0.15-1.16.1.jolla;i486;sailfishos-chum-testing
 - jolla-alarm-ui;0.2.22-1.8.1.jolla;i486;jolla
 - jolla-calendar;1.1.15.1-1.13.1.jolla;i486;apps
 - jolla-handwriting;0.1.7-1.3.3.jolla;i486;xt9
 - jolla-xt9;0.5.22-1.6.3.jolla;i486;xt9
 - lame;3.100-1.1.1.jolla;i486;sailfishos-chum-testing
 - libargon2;2019.03.26-1.1.1.jolla;i486;sailfishos-chum-testing
 - libmp3lame0;3.100-1.1.1.jolla;i486;sailfishos-chum-testing
 - libsodium;1.0.18.1-1.2.1.jolla;i486;sailfishos-chum-testing
 - python3-brotli;1.0.9+git9-1.1.1.jolla;i486;sailfishos-chum-testing
 - python3-websockets;10.1-1.1.1.jolla;noarch;sailfishos-chum-testing
 - sailfish-components-calendar-qt5;1.1.15.1-1.13.1.jolla;i486;apps
 - sailfish-hciwait;0.0.3-1.2.3.jolla;i486;adaptation0
 - sailfish-rfkill-plugin;0.0.4-1.2.11.jolla;i486;adaptation0
 - sailfish-utilities;0.1.22-1.9.1.jolla;i486;apps
 - sailfish-version-variant;4.6.0-1.9.12.jolla;noarch;jolla
 - sailfish-version;4.6.0-1.9.12.jolla;noarch;jolla

REBOOT NOW unless you need to investigate update
issues or know what you are doing (or both).

All bugs encountered until reboot are features.

Notice: After rebooting, do not miss to run post_sfos-upgrade
[root@JollaTab nemo]#

Not rebooted yet, hopefully all is fine, otherwise I’ll have to spend a week or so again to go though all of SailfishOS history on the Tablet.

1 Like

On a (just slightly) different note, as of 4.6.0.11 and .12, during the update process the GUI still says: “Please remove or revert the following package(s), as it (they) may cause problems during upgrade [and here comes a list of packages]” but then the OS update removes all of those packages by itself, without asking or notifying the user about it. Slightly confusing…

Sorry to state this so bluntly @wetab73, but this issue is completely unrelated and hence definitely off-topic.

The right thread for the issue you mention is «How to interpret “Remove the following packages” in the SailfishOS-updater?».
You may post your (absolutely correct) observation there.

1 Like

Well, not really completely, because in many cases the list of system packages to be removed shown by that dialog (often veeeery long and containing lots of system packages) is directly caused by the VERY problem this thread is about, i.e. setting ssu re to an incorrect release version. Which not only just adds to the user’s confusion, but often convinces the user to interrupt the update and try to manually remove those system packages “requested for removal”, ending up with a total mess. There are lots of stories about it on this forum, also very recent ones, encountered by people during 4.6.0.11 / 12 update.

That’s why I thought it would be useful to remind about it here.

3 Likes

Some observations from upgrading my 10III from 4.5 to 4.6:

  1. on a freshly rebooted 10III device, I triggered “Download Update”, which soon displayed warning about mapnik having to be removed => went to chum and removed it (with download still going).
  2. after a while I got another warning that about all OS packages and then some should be removed. I went to command line and checked ssu status => it was already set to 4.6, which I think to be wrong because the upgrade process was not yet called - so there was no reason for SSU to have moved forward. I think ( pardon my guess) that this confuses the upgrade, which finds v4.5 versions of packages while expecting the v4.6 as per ssu => flags them as to be removed. Proceeding with their removal would probably cause it to cut the branch under its feet and fail - but it’s speculations, I didn’t try.
    I cowardly reset the version with ssu ( in my case, ssu re 4.5.0.25). Then rebooted the phone, just in case.
  3. I started over with “download update”, then “apply update” - and this time all went smoothly, no warnings, etc
1 Like

@rtr2001, thank you very much for confirming this issue on an Xperia 10 III (until now we “only” had Jolla1, Jolla Tablet, Jolla C, Xperia X, XA2 and 10) and when upgrading from SailfishOS 4.5.0 to 4.6.0 (which Nephros already did on his Jolla Tablet): Maybe it is helpful for people with access to Jolla’s internal bug database (e.g. @pvuorela or @flypig) to denote there that this issue did not “magically go away” (why should it, as nothing was altered to address this) and is still wrecking SailfishOS installations.

As @wetab73 already documented very well, the issue with the enormously problematic message which advises users to uninstall packages when using Jolla’s GUI updater is exacerbated by this issue, because exactly that seems to let the problematic message advise users to remove many core system packages of SailfishOS.

As @wetab73 also already mentioned, it is hard to understand, why Jolla has not actively tried to research and address this issue, which seems to exist since the inception of SailfishOS (11 years ago) and is clearly documented and “tracked by Jolla” for more than 5 years.

P.S.: @rtr2001

displayed warning about mapnik having to be removed => went to chum and removed it

Usually this is not necessary, but nobody outside of Jolla knows for sure, because the algorithm to detect these packages does not seem to be documented (at least by source code, preferably by some write-up), or nobody has found that yet.

P.P.S.: @rtr2001

I […] reset the version with ssu (in my case, ssu re 4.5.0.25). Then rebooted the phone, just in case [, and] started over with “download update”, then “apply update”

IMO this is the best way to move forward, if one knows for sure, that no packages were installed or updated (including the ones which comprise the SailfishOS upgrade) during the period of time in which ssu re differed from version, because installing or updating packages during that time may result in a not working (e.g. not booting) Sailfish OS installation. If that happened, one has to identify the packages installed in or updated to a wrong version and downgrade them to the correct version before proceeding with the GUI updater. sfos-upgrade may work better in some of these situations (but sure not all, e.g. if SSU has become installed in an incorrect version), because it depends on far less other software components to work properly than Jolla’s GUI updater.

2 Likes

I’ll add these recent observations to the bug report in the Jolla database.

4 Likes

Been doing some more testing here. Forced the ssu configuration to 4.5.0, avoiding all RnD modes etc and generally trying to make what a proper 4.5.0 should be having. Then initiated check via the osupdateservice or OS update settings page.

At the same time printing out the versions in another shell: while [ true ] ; do ssu re; sleep 1; done

Result: there’s some 10-15 second window with the new version after it has found an update and doing the update size calculations. Then it rolls back to 4.5.0. Works as expected.

Can’t still figure out how this wouldn’t work. It should be restoring the ssu version being it success or failure to do the things.

For one last option, I’m pondering if there’s any chance those storeman/openrepos type of things could be messing up something. No idea how and e.g. extra repositories shouldn’t really matter. Still worth noting if someone has experienced the problem with just plain sailfish + harbour apps.

3 Likes

Let’s say that calculation fails for one reason or another - or something close to it, can it bail out past resetting the version?
Where in the code is this BTW? Looks to be doable from QML, but is that what (re-)sets it here?
(Then closing the app or something like that could be the culprit).

I think my SeaPrint sharing plugin was blamed for pulling in the update once; presumably by depending on a newer transfer-engine.

2 Likes

Should be resetting the version no matter if there was an error or not.

Code is unfortunately closed, the os update handler is part of store-client. Though it’s having a background daemon for doing its things, implemented in c++. No app to be closed.

The daemon crashing would sure explain it, but based on the initial report here, it exits with success.

Package dependencies shouldn’t matter. I’m thinking more of extra repositories or even something doing things with ssu config.

1 Like

@pvuorela, thank you very much for still trying to find the source of the disruptive behaviour of /usr/libexec/sailfish-osupdateservice osupdate-check, because IMO this issue (especially in conjunction with the “Remove the following packages”-issue) is the worst and most disruptive issue of SailfishOS which is outstanding for many years, as it results in broken SailfishOS installations which are unfixable for the average user without reflashing or a factory-reset.

  1. Extra repositories
    It is very likely that all reporters had some RPM repositories at OpenRepos enabled. SailfishOS is no fun at all without software from community repositories, so why would one not use them (maybe except for @vige).
    Additional notes:
    • For my devices I can tell for sure, that no OpenRepos repositories were / are enabled, which contain RPM packages which replace “system packages” (i.e. packages from Jolla’s RPM repos), i.e. the packages which are listed for removal when upgrading SailfishOS per GUI (known offending OpenRepos repos are lachs0r, lpr* and NielDK) since SailfishOS 3.4.0 (but this issue occurred years before SailfishOS 3.4.0), with a single exception: My Bluetooth OBEX Filter off, which I deem harmless (for reasoning, see the notes there).
    • The SailfishOS:Chum community repository (rsp. RPM repositories at SailfishOS-OBS in general) is very unlikely to contribute to this issue, because this issue occurred years before user were instructed to use RPM repos at at SailfishOS-OBS directly.
  2. SSU configuration
    I think it is likely that most (all?) reporters of these issue used the command line to upgrade SailfishOS at least once: Either by executing the commands Jolla recommends (i.e. ssu re X.Y.Z.N && version --dup as root) or by using sfos-upgrade which does exactly the same. But I assume that does not constitute “distorting the SSU configuration” for you, right?
    P.S.: I always wondered, why one is not instructed to kill the store-client, empty zypper’s dist-upgrade cache (rm -f /home/.pk-zypp-dist-upgrade-cache/*), remove Jolla’s OS-info cache file (rm -rf $HOME/.cache/sailfish-osupdateservice/os-info) and / or kill ssud followed by removing its caches (rm -rf /var/cache/ssu/*) before executing the ssu re X.Y.Z.N command (which should also automatically start ssud again, as any ssu command), and / or executing ssu ur after the ssu re X.Y.Z.N but before the version --dup? Due to Jolla’s advice in aforementioned documentation (which consistently stayed the same, now already for a decade), I ultimately decided against performing these steps before upgrading SailfishOS in sfos-upgrade proper, but let the user perform them after upgrading SailfishOS by post_sfos-upgrade.
    a. Do I interpret Jolla’s documentation correctly?
    b. Would it be beneficial to employ these considerations, which I have been dismissing all the time? Or the contrary: May these extra commands cause any harm if employed as described before issuing version --dup?

P.S. / side note: In latest rework of Jolla’s documentation how to upgrade SailfishOS at the command line some sections with basically duplicated content were created (e.g. here and there, also here and there, and here, there and there). In general the variants of these sections lower on this page are the more modern ones, but one would have to consolidate them carefully with their corresponding sections further up in order to avoid eliminating any significant information.

Additionally statements were inserted which regularly confuse users, as “For instance, 4.5.0.25 is a stop release but 4.5.0.24 is not.”: This is not really correct, because all point releases of SailfishOS 4.5.0 constitute a sub-version of the same stop release, but it is recommended to upgrade to the latest point release of a stop release from a prior release (e.g. 4.4.0.N). I.e. Jolla’s GUI updater also does upgrade from 4.5.0.24 to 4.6.0.N directly, not from 4.5.0.24 to 4.5.0.25 first and then to 4.6.0.N, AFAIK.
Another such statement is “Never try to downgrade the OS version as this could brick your device.”, which is fearmongering IMO: This might likely make the installed SailfishOS instance unusable and may make it unbootable, but it will not damage hardware (as “brick your device” suggests), prevent using the recovery console, or prevent reflashing a Sony Xperia.