Docker based build environment: simulating native builds

Platform Development
#1

This is to introduce a newest app and lib build environment for SFOS based on Docker:

In contrast to official SDKs (as well as @coderus Docker images), it is not using Scratchbox2 (SB2). Instead, it relies on running either native or through QEMU hardware emulation.

What is this environment made for? It is designed to avoid SB2 bugs and possible limitations by 32-bit chroot environment used by the official SDK. It has resolved faults in QtWebEngine compilation. New build environment doesn’t induce wrong file permissions, as SB2 does.

All comes with the costs. If you don’t run it natively, QEMU has to be setup on the build server. When using QEMU emulation, expect slow compilation speeds when compared to SB2.

Scripts used in the environment are, obviously, not as refined as mb2. Some outstanding issues are:

  • updating generated version-release pair of RPMs
  • how to link it to Chum? i.e. how we can submit compiled packages into community OBS repo?

Right now, the environment is tested by compiling some larger projects, such as QtWebEngine.

Docker images are not published, so you have to generate them yourself if you want to test it. That’s easy with the included script. See documentation in project repository for details.

Edit: QtWebEngine compilation for aarch64 worked and resulted in working library. So, that fixes SB2 leads to faulty QtWebengine compilation

14 Likes
Docker based build environment: images available
Developer Announcement - Qt 5.15 available for app developer testing
Get another (Linux-) E-Mail client?
#2

That sounds interesting. The next logical step would be that some brave soul tries running that on apple silicon.

2 Likes
#3

Ah, cool. Jolla is going to move to apple silicon! Neat :slight_smile: (ducks, runs).

#4

Would be funny to run sfos on an iphone

#5

Would be great. After the launch of OS 10, I turned all my powerbooks into Debian boxes out of spite. IOs I hated from the first day.

#6

My dreams are much more modest: an aarch64 build of SDK so that I could run it on a Raspberry Pi 400 (where Docker is natively available).

#7

How long does it take to build the docker image approximatley?

#8

minutes, maybe less. Assuming that you talk about the build environment. It basically pulls RPMs and installs them.

Edit but watch out for mic messing up your QEMU. Which means that you have to comment out one section in makeimage after running in first time (the part that creates baseimage.tar). Issue: Look for alternative image generator to coderus/sailfishos-baseimage ¡ Issue #1 ¡ rinigus/docker-sailfishos-builder ¡ GitHub

#9

Just created a docker image and an i486 package this way, and it went rather smooth.

Two things I noticed:

  • The qemu-custom.conf setup did not work on debian. The related paths seems incorrect for example, and I do not think I needed this part after all. I did install qemu-user-static
  • When comparing to the CODeRUS docker image, it looks like it needs to install more packages for building a package. It was a total of 179 for me, which took some time. Perhaps adding a layer with more packages installed could speed this up for me
1 Like
#10

The build aborted after 2h for i486 with:

...
Info[04/07 13:23:06] : The new image can be found here:
  /share/base_i486_4.5.0.19.ks
  /share/baseimage.tar

Info[04/07 13:23:07] : Finished.
DEPRECATED: The legacy builder is deprecated and will be removed in a future release.
            Install the buildx component to build images with BuildKit:
            https://docs.docker.com/go/buildx/

Sending build context to Docker daemon  460.7MB
Step 1/14 : FROM scratch
 --->
Step 2/14 : ADD baseimage.tar /
 ---> db84b5e8b51c
Step 3/14 : COPY     scripts/buildrpm     scripts/rpm-install-build-deps     rpmdevtools/rpmdev-spectool     rpmdevtools/rpmdev-setuptree     /usr/bin/
COPY failed: file not found in build context or excluded by .dockerignore: stat rpmdevtools/rpmdev-spectool: file does not exist

Could this be the issue with mic?

1 Like
#11

I see. You have to clone submodule as well (the one with rpmdevtools)

1 Like
#12

Okay, retrying. Should I open a PR on the Readme for this?

Also those take the longest time for my build so far:

Info[04/07 16:58:20] : Refreshing repository: adaptation-common ...
#13

Sure, you could update README if something is not clear. Repo refreshing does take time, indeed.

#14

Re custom conf for QEMU: indeed, it would depend on distribution. Some of them may configure it automatically. @thigg was planning to make PR for README. Maybe just add that as well.

#15

Added that but I do not really have an idea what I am talking about, so please review carefully :wink:

#16

Thank you - looked good. I am checking out using zypper -root /path as an alternative to mic. Doesn’t look to be super simple, but maybe possible.

#17

Few updates:

  • Repository has been moved to GitHub - sailfishos-open/docker-sailfishos-builder: Docker based Sailfish OS build environment . It makes sense to develop it under that organization.

  • Docker images are now generated directly with zypper, without mic. This keeps mic away from messing up QEMU setup on the host. Note that I don’t pull anymore ssu into the image, it is using zypper directly. This allows skipping Qt 5.6 base and few other packages. If your app requires them, make sure it is reflected in BuildRequirements.

  • While we got rid of SB2 bugs, some QEMU issues may have an impact. I have seen failure in compilation of armv7hl code with QEMU (issue #6). I suspect, it is due to QEMU issue. Fortunately, there is a workaround. Try to get QEMU arm static compiled for 32bit host. Where to find it? … … Easy - get Jolla’s SDK and copy their qemu-arm-static and configure QEMU to use it by adjusting bitfmt config.

Edit: systemd still sneaked in - dependency of git via openssh-clients

#18

I wouldn’t worry about building time of the image. Compiling C(++) code afterwards is slower without sb2 and with QEMU. Rinigus already warned for that, and I can confirm it is al lot slower (takes at least double the time is my feeling).
Edit: actually it is more than 3 times as slow in non-native mode.

#19

I didn’t worry about the build time because I have a problem with the image build being slow. You are right, that does not really matter because you rarely do that. I was just curious if something went wrong because I waited already couple hours.

#20

There is postmarketos for the iPhone 7, not sure about other models. So porting SFOS should in theory be possible with the pinephone not the Android approach? SFOS on the old/small iphone SE would be very nice.