Scripted download of https://jolla.zendesk.com/hc/en-us/articles/201836347 fails!

Introduction

Since some point between 2019 and spring 2020 (when SFOS 3.3.0 was released), the scripted download of https://jolla.zendesk.com/hc/en-us/articles/201836347 (e.g., per curl) started to always fail (i.e., downloading something else than the real web page; see below for details).
It was working absolutely fine before that.

Side note: It seems that someone activated a last resort measure against DDOS attacks here in 2019 or early 2020. While it is hard to believe that jolla.zendesk.com was ever DDOSed, this attack must be over now. More likely someone considered “switching on some extra protection cannot harm”; but it does, as described below.

Consequences

The aforementioned web-page contains the recent list of “stop releases”, which was downloaded and evaluated by sfos-upgrade to prevent users from “jumping over” a stop release (which can be fatal for an SFOS installation, resulting in the need to recover by the help of the recovery console, a factory reset or re-flashing the device).
Thus disabling the scripted download of https://jolla.zendesk.com/hc/en-us/articles/201836347 resulted in an issue report for sfos-upgrade, when the next SFOS version (3.3.0) was released.

Because curl now downloads a more than 8 KBytes big page with piles of JavaScript code just to output a “No JavaScript”-warning, I had to disable this safety feature of automatically downloading, extracting and using Jolla’s current list of stop releases, which is really unfortunate!
Side note: Interestingly, all web browsers I tested do display the correct page, even if JavaScript is completely switched off (or filtered by NoScript). Obviously Zendesk or Cloudflare is using some server-side magic here, because I failed to emulate the web browser’s behaviour per curl, no matter what I tried (but OTOH, it is the purpose of a DDOS protection not to be circumventable).

Proposed solution

Consequently, I kindly ask Jolla to resolve this situation by allowing for an automatic download of their current list of “stop releases”, again.
This can be achieved by

  1. Restoring the original behaviour of https://jolla.zendesk.com/hc/en-us/articles/201836347 : To be downloadable per curl.
    This is by far the easiest solution and hence IMO the preferable one.
  2. To offer the current list of “stop releases” for scripted download at a different place.
    This may even ease extracting the “stop release” versions, but is error-prone, because the two lists have to be kept in sync by Jolla. I suggest to abstain from this solution, because Jolla has had issues keeping the single list up-to-date in a timely manner.
4 Likes

Well, this is an unfortunate sub-topic, I also intend to address, because it is a recurring situation.

For example, currently (as of end of October 2021) https://jolla.zendesk.com/hc/en-us/articles/201836347#4.1 states
“• 4.0.1.48 Koli (this is the latest stop release as of February 2021)”, but Jolla released two new versions since then (February 2021):

Is one of them a stop release?
Jolla does not tell (in aforementioned / -linked release notes), but Wikipedia states, that 4.1.0 is!


TL;DR: IMO https://jolla.zendesk.com/hc/en-us/articles/201836347#4.1 shall be actively updated after each new SailfishOS release, i.e. specifically the "*(this is the latest stop release as of **Month Year**)*" statement.
1 Like

I think also the template Jolla (@jovirkku ) uses for the release notes should have a field
Stop release? Y/N

A good idea. Thanks. I will add it.

2 Likes

There was a mistake that I have fixed now. All Sailfish 4 releases so far (4.0.1, 4.1.0 and 4.2.0) are stop releases. Also, the upcoming 4.3.0 is.

3 Likes

From the user perspective:
Oh my d*#r!
That means to download 4 times, 4x install, 4x wait (4x be afraid that something goes wrong) and so on. And that only for SFOS4, not counting whatever may be needed else. Is that serious?

1 Like

Unless you have an old installation, and you want to upgrade it.

You can simply install the newest version.

Is there something I am missing ?

1 Like

Reflashing with the latest version is also always an option (with Sailfish X), letting you skip all stop releases.

@jovirkku, I would appreciate very much, if you (or another sailor) also addresses the principal topic of this thread (i.e., the initial posting):
Scripted download of https://jolla.zendesk.com/hc/en-us/articles/201836347 fails!

Thanks!

From the user perspective:
Oh my d*#r!

I concur, but not for the reasons you state!

Stop releases have technical reasons, which Jolla could try to avoid for a specific release, but that creates a technical debt and technical complications.
Hence it is understandable that Jolla does not try to accumulate changes, which require a release to become a stop release, in order to let fewer releases be stop releases. Although I have to admit that 5 stop releases in a row (3.4.0 is also a stop release) is quite extreme and might be a sign of insufficient planning. Still the convenience for users shall not be of much concern here, because updating from a very old release rarely occurs.

But having missed to provide users in an official guide with this crucial information for more than half a year (with two affected releases meanwhile), which usually results in a broken SailfishOS installation (when “jumping over” a stop release) is more than just “Oh my d*#r!”, IMO: It hints a lack of proper processes and quality assurance. :worried:

1 Like

ATM, I don’t know how to install Auto-ambience 2 on SFOS 4 (however, it works), only on SFOS 3.something… (yes, I know about situations). So, I would flash SFOS 3.something if available for the device, install apps, then upgrade…

1 Like

My guest would be the home directory path, it has change from nemo to defaultuser

You could try to make a link from defaultuser to nemo and see if it work:

devel-su ln -s /home/defaultuser /home/nemo

@jovirkku, you continued this discussion at another thread: Unstability of OS upgrades (on CLI) - #14 by jovirkku
To which I replied there in detail: Unstability of OS upgrades (on CLI) - #15 by olf

But I would really like to continue the main point here (see title: “Scripted download of https://jolla.zendesk.com/hc/en-us/articles/201836347 fails!”), because this is the proper location (in contrast to a generic thread about “Unstability of OS upgrades (on CLI)”).

I still would appreciate very much to receive a response to

Specifically, can you please address this issue at Zendesk.com rsp. their CDN-provider Cloudflare (who is causing this issue technically)?

1 Like

@jovirkku, I would still appreciate very much, if …

Also I am still curious to properly understand your former statements, even though they provide at most an awkward and fragile workaround:

Lovely, Jolla has solved this meanwhile, but their “futile to ask, because Jolla does not tell” policy is often also applied to solutions.

The list of “stop releases” was silently moved away from Zendesk to docs.sailfishos.org, which is generated from GitHub. So after three years Jolla has finally provided a machine-readable list of"stop releases" again (plus all releases), which is the solution 2 I originally suggested. I would state “a big thank you”, but I am well aware that this is just a side-effect of restructuring the information formerly at the SailfishOS-Wiki (sailfishos.org/wiki) into docs.sailfishos.org. Still this is a huge step forward, because everyone can now contribute to the SailfishOS-Documentation at GitHub.

Now I hope very much that the location and format of this information will persist, i.e., is not regularly altered by Jolla as e.g., the mountpoint of the SD-card.

P.S.: I am very glad that Jovirkku’s the suggestion to use the community maintained list of “stop releases” at Wikipedia was obviously not really meant seriously. It originally sounded like a cheap bail-out to me, along the lines “we do not have to do anything, because there is something provided by someone somewhere in the WWW”.

From the original post, which started this thread (“topic”):

@flyping provided a pointer to the corresponding “bots”-related measures the CDNs (Content Delivery Networks) Cloudflare and Akamai employ.
The documentation by Cloudflare describes very well, what I experienced for almost 3 years. As expected these measures are designed to be non-circumventable.

Side note: Both, Akamai and Cloudflare emphasise, that the bot-related measures are highly customer-configurable and can be enabled per web-page (at least for Akamai).

Nevertheless, moving Jolla’s official list of stop releases to the SailfishOS-Wiki and open it up for collaboration at GitHub is a vastly better solution.
Thanks!

1 Like

And moved again (and its source), but just to the Support sub-directory.
Jolla loves to move things around, e.g., the mount-point of SDcards multiple times over the years. :stuck_out_tongue_winking_eye: