OpenRepos and Chum integration

[rinigus]

This is a continuation of private discussion on integration of OpenRepos and Chum. As all involved were interested in it, we are continuing the discussion in public.

In short, it was suggested to find a way that would allow to rely on strength of the both platforms and provide seamless integration for developers and users.

From developer side, would be great if developer could register a application/library at OpenRepos and could somehow specify that it should be backed by Chum. As a part of this specification, developer would say the name of the package in Chum (RPM name). If backed by Chum, developer would not have to upload any RPMs, but the rest of registration stays the same. On package updates in Chum, OpenRepos would realize that the package is updated and display on its feed.

From users side, users would enable Chum repository and later would be able to install packages using Storeman as expected for any application on OpenRepos. It is possible to comment and give feedback, see application description and so on. As application is registered by developer, all notifications go to developer as well. As suggested by @basil, Storeman could provide mechanism for enabling Chum repositories for seamless integration.

@basil has proposed to use webhook on package compilation for triggering OpenRepos updates from OBS side. That would require addition of metadata on OBS package specification and ability of OpenRepos to receive such notification. @lbt has ideas on how to make it possible on OBS side using its automation facilities. Specifics of integration are not yet known and can be discussed here.

I would like to thank here all the participants via Forum messages and IRC: @basil, @piggz, @lbt, and @mentaljam . In particular, all replied fast and constructively, there is a great enthusiasm regarding this project. Thank you very much!

[rinigus]

To get back to the discussion: @basil - have you had a chance to think or implement OpenRepos webhook that we could use to notify the update of the package? Or how to register Chum-backed apps?

If we know what is expected from OBS side, we can ask @lbt for help and write down human-readable manifest on when that webhook is expected to be called.

[Basil]

@rinigus I’m thinking of pretty simple scheme:

Let’s say you have your Pure Maps app at openrepos and at chum.
While filling up(editing) your application details at openrepos, you specify your application project name you are using at Chum, in this case: “pure-maps”. If your project builds multiple rpms, I think you need to specify main rpm package name as well, “harbour-pure-maps”.
Using that info, openrepos will be able to crawl to package details project within Chum.

OBS can add extra QA post build hook (same way as lint), directly within build script or any other manner.
This hook will test package files for _openrepos file (or openrepos key within _service file), if found,the specified URL should be touched, lets say https://api.openrepos.net/v1/chum/update?pure-maps (using your chum project name) or https://api.openrepos.net/v1/chum/update?10447 (using 10447 - your openrepos appid)

The update mechanics may be as follows:
Once you push new source code to git, OBS gets notification, builds package, notifies openrepos with postbuild hook and openrepos queues package update.
Openrepos reaches https://build.merproject.org/package/show/sailfishos:chum/pure-maps verifies there are new builds, fetches repository info from latest successful build for latest sailfishos version http://repo.merproject.org/obs/sailfishos:/chum/4.1.0.24_armv7hl/ and extracts package details: version, archs, changelogs, and so on.

At this point, Storeman will get the updated version info/changelog for the packages, as well as “chum-backed” flag within application json, so it will be able to skip “enable openrepos developer repository” step.

[rinigus]

@basil, that scheme looks good. Few comments regarding it.

1. It seems to me that every project in Chum will have multiple RPMs as we have .src. and .debug. RPM generated. So, it is a good practice to specify RPM as well.

2. _openrepos (or the key in _service file) would then contain app ID in OpenRepos. To make it possible for Chum maintainers to check whether correct app is update at OpenRepos, would be better to have something that we can verify and is unique. As far as I can see, it is impossible to get OpenRepos app ID for the apps that are not mine. However, in the case of Pure Maps, https://openrepos.net/node/10447/view is available for checking so it should be sufficient.

3. The hook on OBS side will have to be a touch later. Rebuilds could be ongoing (any other apps, for example) and we should trigger the hook only after all repositories have been published again. But that is now OBS-side problem, not OpenRepos

4. Update mechanics seems to be heavy on OpenRepos side. Here “heavy” means significant amount of code, maybe we can somehow simplify it for you or contribute. Now, Chum will have host of packages for each project. So, in case of Pure Maps, it is for SFOS versions 3.3.0.16 and up. Which means that version, changelogs will have to be extracted from some dedicated repository. Maybe developer would specify something like http://repo.merproject.org/obs/sailfishos:/chum/4.1.0.24_armv7hl/ at app registration in OpenRepos. Later, developer may change it in the settings of OpenRepos.

In this bit simplified scheme, there is a small drawback. We would somehow need to convey to Storeman available packages for install depending on SFOS version and arch of the user. But maybe we can keep it for later and users would just get the same cryptic message as “failed to install” that we are all getting while trying to install packages from official store which miss device architecture (aarch64 for example).

[piggz]

I think this should be linked here, thx @mentaljam Announcing new installer for Storeman

[rinigus]

@basil: any thoughts? Have you had time to look into implementing it on your side?

[rinigus]

@basil: have you had a chance to look into the integration? can we help you on our side?

[poetaster]

Basil, if some of the work is web related, drupal I believe, I can also help.

[poetaster]

2. the node id’s can easily be obtained (they are always there in the html body classes, for instance). so it’s just a ‘view’ of the form node name, node id, author name, author id, etc. If basil comes up from under, maybe we can get there a bit more quickly.