Mb2: allow to install dependencies with vendor change

Platform Development
#1

While working on automation of SFOS port build, I encountered situations where new packages pulled into the target would have a different vendor than the package already installed there. This happens when you build projects consisting of multiple packages with some of them being dependencies for the others.

For example, AOSP10/aarch64 ports require newer ohm than is distributed with 4.0.1. So, I have added ohm into the build and will get a package that has no vendor set. To continue working with this package and build its plugins, I need to get it installed in the target as dependency. mb2 nicely detects it in the shared RPMS folder and tries to install. However, as there is a vendor change, that fails.

Similar situations can happen also with droid-config-xxx packages that provide several symbols and obsolete others. Also these packages can have issues with the changes in vendor as they are usually set to have vendor according to the device manufacturer (sony, for example).

To be able to handle these situations, would be great to have an option to mb2 that would result in installing dependencies with --allow-vendor-change. For a good measure, we should be able to specify other extra zypper options (such as GPG check control, for example).

1 Like
#2

Hey we spoke earlier on IRC but I write down what I answered there:

If you build your own variant of packages that are considered middleware which is provided by the OS and thus build with the vendor ‘meego’ you can do the same to achieve by using mb2 build --define 'vendor meego'

Those packages obsolete default configs if you install your own device config into the target that should work fine as it already does when building the ready device image.

While I understand that passing additional options to zypper might help, allowing mb2 to install dependencies that change vendor is more a hack then a solution (see further above).

1 Like
#3

@Thaodan: thank your reply and the summary of our earlier discussion.

I will get back here when I have tested with setting the vendor to meego in the project that covers a port to Sony Tama.

From the look into some packages built at OBS for AOSP9/arm32 port, I can see that the vendor is set by OBS to meego as well. For the apps, we have a separate setting where we strip vendor setting in the project. So, it makes sense to compile port RPMs using that vendor setting.

#4

From testing, I am hitting corner case(s) which are not very well covered yet. To keep track better, I will file separate issue when branching from vendor discussion.

In the ports, we have to build droid-hal-version-XXXX package (XXXX->h8324 in the example below which is one of sony tama devices). This package depends on droid-config-h8324-sailfish which, in turn, on droid-config-h8324. The latter one provides configs and defaults that replace ofono-configs-mer and usb-moded-defaults. Note that these packages are installed in SDK by default.

Now, if I test whether I can install droid-config-h8324-sailfish by zypper separately, it will complain that this includes vendor change from meego to sony. Which is correct, as droid-config-h8324 has vendor sony. If I test droid-config-h8324 then that installation will go ahead, I presume as it obsoletes those packages. Zypper will also go ahead if I try to install droid-config-h8324-sailfish and droid-config-h8324 together.

So, in the context of build automation, I am a bit of in a limbo as I cannot assume that the vendor change is allowed and, at the same time, don’t know that droid-config-h8324-sailfish is always used together with droid-config-h8324 which allows me in practice to build droid-hal-version-h8324 package.

I have few ideas on what to do about it, but as it demonstrates, it maybe good to have an ability to change vendor during pulling build requirements.

#5

To me it seems that this would be a good addition, even if it’s only rarely needed. We will consider implementing it in some future release.

#6

The issue is not the vendor but that those are installed by default. Installing the device configs into the device targets was never an issue.

#7

Indeed, if those packages would not been installed on SDK by default, I would not have that issue. But, in reality, they are installed and, I presume, it was done on purpose.

#8

In general you should build always in a clean target so that when building. I don’t see an issue building an mw package like ohm and then replacing it if you define the original vendor for those kind of packages.

When you build droid configs you should not install it into the target[1]. In general don’t install anything into the target unless it is used.

[1] https://gitlab.com/Thaodan/hadk_tools/-/blob/master/templates/mer.droid.configs.hadk#L30

#9

@Thaodan, I don’t install configs into the target anymore. But configs are pulled into it as a build dependency for droid-hal-version-XXX packages. As described above.

#10

Why do install those then?

#11

I am not sure what is this question about. Please see Mb2: allow to install dependencies with vendor change , I included all the details. Please let me know if anything is missing and not clear on why configs are pulled into the target while building some packages

#12

I asked you why do you install droid-hal-version into the target? You should not do that see also here for reference:

#13

Thanks for clarification! No, I build droid-hal-version packages and those have BuildRequires for configs. See https://github.com/sailfishos-sony-tama/droid-hal-version-sony-tama/blob/master/droid-hal-version-tama.inc and similar for official port https://github.com/mer-hybris/droid-hal-version-sony-ganges/blob/master/droid-hal-version-kirin.inc

#14

I believe the issue here is very different. Vendor change is what happens whenever you use locally built dependencies, which is a scenario I have been actively using for years. When I tried, the output was very clear about continuing despite vendor change:

The following 3 packages are going to be upgraded:
  ohm                  1.3.0-1.3.3.jolla -> 1.3.0+master.20210617065216.04c3d50-1
  ohm-configs-default  1.3.0-1.3.3.jolla -> 1.3.0+master.20210617065216.04c3d50-1
  ohm-plugin-core      1.3.0-1.3.3.jolla -> 1.3.0+master.20210617065216.04c3d50-1

The following 3 packages are going to change vendor:
  ohm                  1.3.0-1.3.3.jolla -> 1.3.0+master.20210617065216.04c3d50-1  meego -> 
  ohm-configs-default  1.3.0-1.3.3.jolla -> 1.3.0+master.20210617065216.04c3d50-1  meego -> 
  ohm-plugin-core      1.3.0-1.3.3.jolla -> 1.3.0+master.20210617065216.04c3d50-1  meego -> 

3 packages to upgrade, 3  to change vendor.
Overall download size: 0 B. Already cached: 70.9 KiB. After the operation, additional 60.0 B will be used.
Continue? [y/n/v/...? shows all options] (y): y
committing
[...]

Later it fails unpacking /etc/ohm/modules.ini:

(1/3) Installing: ohm-configs-default-1.3.0+master.20210617065216.04c3d50-1.i486 .........................................[error]
Installation of ohm-configs-default-1.3.0+master.20210617065216.04c3d50-1.i486 failed:
Error: Subprocess failed. Error: RPM failed: error: unpacking of archive failed on file /etc/ohm/modules.ini: cpio: utime failed - No such file or directory
error: ohm-configs-default-1.3.0+master.20210617065216.04c3d50-1.i486: install failed
error: ohm-configs-default-1.3.0-1.3.3.jolla.i486: erase skipped

Abort, retry, ignore? [a/r/i] (a): a
Problem occurred during or after installation or removal of packages:
Installation has been aborted as directed.
Please see the above error message for a hint.

You likely missed the error because it then ends offering possible solutions, where the vendor change is mentioned, but this is not meant to be the cause for the failure:

Problem: ohm-devel-1.3.0+master.20210617065216.04c3d50-1.i486 requires ohm = 1.3.0+master.20210617065216.04c3d50-1, but this requirement cannot be provided
  not installable providers: ohm-1.3.0+master.20210617065216.04c3d50-1.i486[~plus-repo-1]
 Solution 1: Following actions will be done:
  install ohm-1.3.0+master.20210617065216.04c3d50-1.i486 (with vendor change)
    meego  -->  
  install ohm-configs-default-1.3.0+master.20210617065216.04c3d50-1.i486 (with vendor change)
    meego  -->  
  install ohm-plugin-core-1.3.0+master.20210617065216.04c3d50-1.i486 (with vendor change)
    meego  -->  
 Solution 2: do not install ohm-devel-1.3.0+master.20210617065216.04c3d50-1.i486
 Solution 3: break ohm-devel-1.3.0+master.20210617065216.04c3d50-1.i486 by ignoring some of its dependencies

Choose from above solutions by number or cancel [1/2/3/c/d/?] (c): c

Also trying to build ohm with vendor set to meego didn’t change the result for me (except that it did not offer the possible solutions in that case).

So the actual issue here is “ohm installation fails under scratchbox2”, the symptoms being:

Error: Subprocess failed. Error: RPM failed: error: unpacking of archive failed on file /etc/ohm/modules.ini: cpio: utime failed - No such file or directory

Please consider opening a separate topic for that.

#15

After all it turned to be more likely an issue specific to the Docker-based build engine instead.

The file ownership of the preinstalled (i.e. latest) build targets is incorrect under the Docker-based build engine unless your UID on host is 1001. You do not mention using Docker-based build engine, so I am just guessing. If I am correct, simply removing and adding back each of the preinstalled build targets should fix it for you. Alternatively, you may try:

$ sfdk engine exec sudo find / -xdev -user 1001 -! -type l -exec chown "$UID" {} \;

Before applying that, you should be also able to trigger the bug simply with an attempt to reinstall the package from official repositories:

$ sfdk tools exec SailfishOS-4.1.0.24-i486 zypper in -f ohm-configs-default
[...]
The following package is going to be reinstalled:
  ohm-configs-default
[...]
(1/1) Installing: ohm-configs-default-1.3.0-1.3.3.jolla.i486 .............................................................[error]
Installation of ohm-configs-default-1.3.0-1.3.3.jolla.i486 failed:
Error: Subprocess failed. Error: RPM failed: error: unpacking of archive failed on file /etc/ohm/modules.ini: cpio: utime failed - No such file or directory
error: ohm-configs-default-1.3.0-1.3.3.jolla.i486: install failed