Sfdk/mb2 handling of shared output-dir

Platform Development
#1

From the work on automated builds involving multiple SPEC files, I would like to propose few changes to mb2 and sfdk handling of the shared output directory.

It would be great to have shared output directory for RPMS organized as

<selected-shared-name>/SPECname/package1...rpm
<selected-shared-name>/repodata/...

In particular:

  • each SPEC produced RPMs will be confined in a directory.
  • an option deleting old RPMs produced by previous builds in this directory. That way we have only one set of RPMs. Delete only if the build succeeds, just before copying new RPMs there
  • process the shared directory with createrepo after each build.

This is something that we have for build scripts used in the ports. Would be good to use the same logic in application SDK.

#2

Could you please describe more specifically an use case where the current layout does not work? I do not remember seeing packages grouped by sources in regular RPM repositories, so we should have a good reason for doing differently.

#3

Could you please describe more specifically an use case where the current layout does not work?

There are two issues with the current approach:

  1. No automatic removal of older versions: With the development, you start to accumulate RPMs for different versions quite fast, as soon as new ones are produced for every try. To combat that, one can use as a version the one in SPEC without extra version info, but that is maybe not the best solution either.

  2. As soon as you start getting larger sets of RPMs, it becomes harder to track which one is coming from where. For example, collectd produces collectd-xyz.rpm, libcollectd-xyz.rpm, -devel versions and so on. If each SPEC will be partitioned into separate folder, it would be easier to track it manually.

As a result, we get also

  1. If we want to automate detection of provided symbols for automatic build systems (as in https://forum.sailfishos.org/t/automated-build-system-to-handle-multiple-packages), it would simplify scripts by having imposed SPEC->RPM list relationship. Although, this can probably be obtained reliably from rpmspec.

  2. Such layout would simplify application of (1) above. Just remove all packages from corresponding RPMS subdir. Otherwise, you may get into situation where packages could have changed in a SPEC and autoremoval on adding new version will break.

#4
  1. This would be more conveniently supported by adding an option/command to keep just the latest versions - task noted down.
  2. What would be a real world use case for this? Like in 1 I am thinking about the possibility of supporting a specific use case more conveniently. E.g. the deploy command allows to choose whether just the binaries for the current source or all binaries (--all) should be deployed. It also allows to preview the list with --dry-run as well as excluding packages with filters. Is there anything on top of this you miss or is there an use case not related to deployment?
  3. Automation of dependency handling is sooo complicated task, so that mapping binaries to sources is too negligible part to really affect the total complexity I believe.
  4. The solution here is simple. Instead of using rpmspec to list binaries for a .spec, do the opposite: filter binaries by their %{SOURCERPM} instead.
#5

Re separate folders: indeed, %{SOURCERPM} would handle my use case well. I personally don’t have other usecase for splitting and, on my side, it could be closed as that.

So, reformulating original suggestion - only autoremoval of older versions is requested.

#6

Great and thanks for your suggestion! I will ping you here when the feature is added (it’s unfortunately too late to make it into the next release).

1 Like
#7

Thanks for fast response, thinking it through, and discussion! Looking forward for when autoremoval gets implemented.

1 Like