Compiling GCC for SDK

This thread is continuation on me trying to compile GCC for Platform SDK / Sailfish SDK. I was so impressed by the style of the blog that I decided to try it myself. Let’s see how this goes…

Quick recap:

I had already tried most of the things @flypig kindly suggested earlier, but the first issue comes from the project structure: there is no rpm folder to contain .spec file, so compilation attempts end up like this:

PlatformSDK $ mb2 build -d -p
Fatal: No RPM SPEC or YAML file found in '/home/matti/PSDK/sailfishos-gcc/rpm/'

Future me here! This was roughly the outcome with trying sfdk build too – I knew it would insist on having one single .spec file inside rpm/, so I changed to Platform SDK in hopes that mb2 would pick it up – that wasn’t the case…

(Don’t mind the commit hash, I’m not sure if I was already on a new wip branch or something.)

From there, I created the folder and copied gcc.spec there:

PlatformSDK $ mb2 build -d -p
Setting version: 8.3.0.7+copysignf.20230831155339.0034f4d
[...]
Building target platforms: aarch64-meego-linux-gnu
Building for target aarch64-meego-linux-gnu
error: Bad source: /home/matti/PSDK/sailfishos-gcc/rpm/gcc8-hack.patch: No such file or directory

Then I copied the patches there too:

Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.tOqQQQ
+ umask 022
+ cd /home/matti/PSDK/sailfishos-gcc
+ LANG=C
+ export LANG
+ unset DISPLAY
+ echo 'Patch #0 (gcc8-hack.patch):'
Patch #0 (gcc8-hack.patch):
+ patch --no-backup-if-mismatch -p0 -b --suffix .hack~ --fuzz=2
can't find file to patch at input line 3
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|--- libada/Makefile.in.jj	2009-01-14 12:07:35.000000000 +0100
|+++ libada/Makefile.in	2009-01-15 14:25:33.000000000 +0100
--------------------------
File to patch: ^C

At this point I started messing around with rpmbuild, with gradual progress:

PlatformSDK $ rpmbuild -bp gcc.spec
error: Bad source: /home/matti/rpmbuild/SOURCES/gcc-arm-src-snapshot-8.3-2019.03.tar.xz: No such file or directory

So I did that.

error: Bad source: /home/matti/rpmbuild/SOURCES/gcc8-hack.patch: No such file or directory

And that…

Executing(%prep): /bin/sh -e /var/tmp/rpm-tmp.6GLUG3
[...]
Patch #0 (gcc8-hack.patch):
+ /usr/bin/patch --no-backup-if-mismatch -p0 -b --suffix .hack~ --fuzz=0
[...]
Patch #15 (gcc8-reproducible-builds-buildid-for-checksum.patch):
+ /usr/bin/patch --no-backup-if-mismatch -p0 -b --suffix .reproducible-builds-buildid-for-checksum~ --fuzz=0
patching file gcc/c-family/c-pch.c
patching file gcc/genchecksum.c
+ echo 'Sailfish OS gcc 8.3.0-7'
+ sed -i '/UInteger Var(dwarf_version)/s/Init(2)/Init(4)/' gcc/common.opt
+ sed -i 's/\(may be either 2 or 3 or 4; the default version is \)2\./\14./' gcc/doc/invoke.texi
[...]

Finally, the patches are in! With some testing, I found out that -p1 is needed with that patch. So let’s add the new patch and try again:

$ cat gcc.spec
[...]
Patch15: gcc8-reproducible-builds-buildid-for-checksum.patch
Patch16: gcc8-copyscanf.patch # new line
[...]
%patch15 -p0 -b .reproducible-builds-buildid-for-checksum~
%patch16 -p1 -b .copyscanf~ # new line
[...]
Patch #15 (gcc8-reproducible-builds-buildid-for-checksum.patch):
+ /usr/bin/patch --no-backup-if-mismatch -p0 -b --suffix .reproducible-builds-buildid-for-checksum~ --fuzz=0
patching file gcc/c-family/c-pch.c
patching file gcc/genchecksum.c
+ echo 'Patch #16 (gcc8-copyscanf.patch):'
Patch #16 (gcc8-copyscanf.patch):
+ /usr/bin/patch --no-backup-if-mismatch -p1 -b --suffix .copyscanf~ --fuzz=0
patching file gcc/ChangeLog
Hunk #1 FAILED at 1.
1 out of 1 hunk FAILED -- saving rejects to file gcc/ChangeLog.rej
patching file gcc/config/aarch64/aarch64.md
Hunk #1 FAILED at 189.
Hunk #2 succeeded at 5438 (offset 11 lines).
1 out of 2 hunks FAILED -- saving rejects to file gcc/config/aarch64/aarch64.md.rej
patching file gcc/config/aarch64/iterators.md
Hunk #1 succeeded at 581 (offset 3 lines).
Hunk #2 succeeded at 668 (offset 3 lines).
patching file gcc/testsuite/ChangeLog
Hunk #1 FAILED at 1.
1 out of 1 hunk FAILED -- saving rejects to file gcc/testsuite/ChangeLog.rej
patching file gcc/testsuite/gcc.target/aarch64/pr90075.c
error: Bad exit status from /var/tmp/rpm-tmp.TW3siP (%prep)

Close! At this point I manually modified the patch so that it would finally apply! My approach – without any research on patching patches – I decided to make a git repo of the SOURCES folder, apply the first 15 patches, commit the files and start from there. Then I just tried to apply the patch, see the results, adjust the patch, reset the repository (this was the lifesaver trick!) and try applying it again. I am not going to replicate any of that patchwork (ha!) here, just tons of text editing, until finally:

PlatformSDK $ rpmbuild -bp gcc.spec
[...]
Patch #16 (gcc8-copyscanf.patch):
+ /usr/bin/patch --no-backup-if-mismatch -p1 -b --suffix .copyscanf~ --fuzz=0
patching file gcc/ChangeLog
patching file gcc/config/aarch64/aarch64.md
Hunk #1 succeeded at 194 (offset -2 lines).
Hunk #2 succeeded at 5439 (offset 11 lines).
patching file gcc/config/aarch64/iterators.md
Hunk #1 succeeded at 581 (offset 3 lines).
Hunk #2 succeeded at 668 (offset 3 lines).
patching file gcc/testsuite/ChangeLog
patching file gcc/testsuite/gcc.target/aarch64/pr90075.c
+ echo 'Sailfish OS gcc 8.3.0-7'
+ sed -i '/UInteger Var(dwarf_version)/s/Init(2)/Init(4)/' gcc/common.opt
+ sed -i 's/\(may be either 2 or 3 or 4; the default version is \)2\./\14./' gcc/doc/invoke.texi
+ cp -a libstdc++-v3/config/cpu/i486/atomicity.h libstdc++-v3/config/cpu/i386/atomicity.h
+ perl -pi -e 's/^all: all-redirect/ifeq (\$(MULTISUBDIR),)\nall: all-redirect\nelse\nall:\n\techo Multilib libjava build disabled\nendif/' libjava/Makefile.in
Can't open libjava/Makefile.in: No such file or directory.
+ perl -pi -e 's/^install: install-redirect/ifeq (\$(MULTISUBDIR),)\ninstall: install-redirect\nelse\ninstall:\n\techo Multilib libjava install disabled\nendif/' libjava/Makefile.in
Can't open libjava/Makefile.in: No such file or directory.
+ perl -pi -e 's/^check: check-redirect/ifeq (\$(MULTISUBDIR),)\ncheck: check-redirect\nelse\ncheck:\n\techo Multilib libjava check disabled\nendif/' libjava/Makefile.in
Can't open libjava/Makefile.in: No such file or directory.
+ perl -pi -e 's/^all: all-recursive/ifeq (\$(MULTISUBDIR),)\nall: all-recursive\nelse\nall:\n\techo Multilib libjava build disabled\nendif/' libjava/Makefile.in
Can't open libjava/Makefile.in: No such file or directory.
+ perl -pi -e 's/^install: install-recursive/ifeq (\$(MULTISUBDIR),)\ninstall: install-recursive\nelse\ninstall:\n\techo Multilib libjava install disabled\nendif/' libjava/Makefile.in
Can't open libjava/Makefile.in: No such file or directory.
+ perl -pi -e 's/^check: check-recursive/ifeq (\$(MULTISUBDIR),)\ncheck: check-recursive\nelse\ncheck:\n\techo Multilib libjava check disabled\nendif/' libjava/Makefile.in
Can't open libjava/Makefile.in: No such file or directory.
+ ./contrib/gcc_update --touch
./contrib/gcc_update: line 203: make: command not found
+ LC_ALL=C
+ sed -i -e 's/\xa0/ /' gcc/doc/options.texi
+ RPM_EC=0
++ jobs -p
+ exit 0

Victory!

It’s quite obvious that the rpmbuild route is a dead end, but ultimately I only needed to see how well the patch would apply. And that’s just as far as I got! And not an inch further…

At this point I decided that it was good enough for pushing the changes and making the pull request! I guess it won’t be merged as-is, but it should compile at least.


Oh. (Oh no.)

I had an epiphany, just now as I’m writing this.

I may not be supposed to compile it with mb2 which targets the target architecture, but instead compile it “normally” inside PlatformSDK, so that the compilation targets the Platform SDK. That makes more sense… Oh well, I still think I’ll leave it here, unless someone can point me to right direction!

7 Likes

Thanks for all your work on this @direc85; superb stuff.

I’m only just now realising that this repo has the unusual structure where the spec file isn’t inside an rpm folder. There may be a way to get this to build locally with sfossdk or sdk using the right flags, but I’m not aware of it I’m afraid (could someone else enlighten me?). I usually restructure the directories to build them locally, which isn’t a great solution.

It’ll work like this on OBS though, so I made a package using your code:
https://build.sailfishos.org/package/show/home:flypig:gecko-esr91/gcc

As I write this it’s still mid-build, so still waiting for the results.

In case it’s helpful for reference, here’s the _service file I used to create this package:

<services>
	<service name="tar_git">
	<param name="url">https://github.com/direc85/sailfishos-gcc.git</param>
	<param name="branch">copysignf</param>
	<param name="revision">0034f4df8c3ba2c62aaa1d79c26b3d2013a86d08</param>
	<param name="token"/>
	<param name="debian"/>
	<param name="dumb">Y</param>
</service></services>

In case it’s helpful for anyone reading this, the best quick-start community OBS tutorial I’m aware of is the excellent one from the Chum docs.

2 Likes

I was finally to compile, uh, something!

I copied manually the source folder contents to ./SOURCES. I then enter the build shell and tried using rpmbuild directly.

mb2 --target SailfishOS-4.5.0.18-i486 build-shell
rpmbuild -ba --define '_srcdefattr (-,root,root)' --define "_topdir `pwd`" --target=i486-meego-linux --nosignature ./SOURCES/cross-[arch]-gcc.spec

I got a list of packages missing, and I tried installing them using mb2 --target SailfishOS-4.5.0.18-i486 build-shell --maintainzypper install ...

These are the packages that are not found:

For armv7hl:

  • cross-armv7hl-glibc
  • cross-armv7hl-glibc-devel
  • cross-armv7hl-glibc-headers
  • cross-armv7hl-kernel-headers

For aarch64:

  • cross-aarch64-glibc
  • cross-aarch64-glibc-devel
  • cross-aarch64-glibc-headers
  • cross-aarch64-kernel-headers

For i486:

  • No missing packages

Fir i486, it builds and produces RPMS/i586/cross-i486-gcc-8.3.0-7.i586.rpm (with debugsource and debuginfo packages). I’m not sure if this is the complete list of packages

Additionally, I can build the “vanilla” gcc.spec and it produces a whole lot of packages (this is not useful build, but I list if for completeness):

$ mb2 --target SailfishOS-4.5.0.18-aarch64 build-shell
$ rpmbuild -ba --define '_srcdefattr (-,root,root)' --define "_topdir `pwd`" --nosignature ./SOURCES/gcc.spec
$ ls RPMS/aarch64/ -1
cpp-8.3.0-7.aarch64.rpm
gcc-8.3.0-7.aarch64.rpm
gcc-c++-8.3.0-7.aarch64.rpm
gcc-debuginfo-8.3.0-7.aarch64.rpm
gcc-debugsource-8.3.0-7.aarch64.rpm
gcc-doc-8.3.0-7.aarch64.rpm
gcc-plugin-devel-8.3.0-7.aarch64.rpm
libasan-8.3.0-7.aarch64.rpm
libasan-static-8.3.0-7.aarch64.rpm
libatomic-8.3.0-7.aarch64.rpm
libatomic-static-8.3.0-7.aarch64.rpm
libgcc-8.3.0-7.aarch64.rpm
libgomp-8.3.0-7.aarch64.rpm
libitm-8.3.0-7.aarch64.rpm
libitm-devel-8.3.0-7.aarch64.rpm
libitm-static-8.3.0-7.aarch64.rpm
liblsan-8.3.0-7.aarch64.rpm
liblsan-static-8.3.0-7.aarch64.rpm
libstdc++-8.3.0-7.aarch64.rpm
libstdc++-devel-8.3.0-7.aarch64.rpm
libstdc++-static-8.3.0-7.aarch64.rpm
libubsan-8.3.0-7.aarch64.rpm
libubsan-static-8.3.0-7.aarch64.rpm

And at the time of writing, the OBS build resulted in the same list of files – I was expecting it to produce the cross-* packages…

Edit: Stray /gcc.spec moved inside the code block

4 Likes

Great news! I installed the new gcc into my local SDK (using these packages) and re-ran the build. The error is now gone! So this patch does seem to have done the trick.

Here’s the command sequence and output so you can take a look.

$ sfdk config --session --push snapshot test
$ # Perform a build to ensure the snapshot is set up correctly
$ sfdk build -d --with git_workaround
$ # Now we do the testing
$ # First *without* the new gcc
$ sfdk engine exec
$ sb2 -R -m sdk-install -t SailfishOS-devel-aarch64.test
$ cd ./gecko-dev/gfx/wr/swgl/
$ /usr/bin/g++ -O2 -ffunction-sections -fdata-sections -fPIC -std=gnu++17 \
  -I../../../../obj-build-mer-qt-xr/dist/stl_wrappers \
  -I../../../../obj-build-mer-qt-xr/dist/system_wrappers \
  -include ../../../../gecko-dev/config/gcc_hidden.h -U_FORTIFY_SOURCE \
  -D_FORTIFY_SOURCE=2 -fstack-protector-strong -DNDEBUG=1 -DTRIMMED=1 \
  -I../../../../gecko-dev/toolkit/library/rust \
  -I../../../../obj-build-mer-qt-xr/toolkit/library/rust \
  -I../../../../obj-build-mer-qt-xr/dist/include -I/usr/include/nspr4 \
  -I/usr/include/nss3 -I/usr/include/nspr4 \
  -I../../../../obj-build-mer-qt-xr/dist/include/nss -I/usr/include/pixman-1 \
  -DMOZILLA_CLIENT -include ../../../../obj-build-mer-qt-xr/mozilla-config.h \
  -Wall -Wempty-body -Wignored-qualifiers -Wpointer-arith -Wsign-compare \
  -Wtype-limits -Wunreachable-code -Wno-invalid-offsetof -Wduplicated-cond \
  -Wimplicit-fallthrough -Wno-error=maybe-uninitialized \
  -Wno-error=deprecated-declarations -Wno-error=array-bounds \
  -Wno-error=coverage-mismatch -Wno-error=free-nonheap-object \
  -Wno-multistatement-macros -Wno-error=class-memaccess \
  -Wno-error=unused-but-set-variable -Wformat -Wformat-overflow=2 -Wno-psabi \
  -fno-sized-deallocation -fno-aligned-new -O3 -I/usr/include/freetype2 \
  -DUSE_ANDROID_OMTC_HACKS=1 -DUSE_OZONE=1 -DMOZ_UA_OS_AGNOSTIC=1 -Wno-psabi \
  -Wno-attributes -Wno-psabi -Wno-attributes -Wno-psabi -Wno-attributes \
  -Wno-psabi -Wno-attributes -Wno-psabi -Wno-attributes -Wno-psabi \
  -Wno-attributes -fno-exceptions -fno-strict-aliasing -fPIC \
  -ffunction-sections -fdata-sections -fno-exceptions -fno-math-errno -pthread \
  -pipe -gdwarf-4 -freorder-blocks -O2 -fno-omit-frame-pointer -funwind-tables \
  -DMOZILLA_CONFIG_H -I ../../../../gecko-dev/gfx/wr/webrender/res -I src \
  -I ../../../../obj-build-mer-qt-xr/aarch64-unknown-linux-gnu/release/build/swgl-c7fddee6f1578b80/out \
  -std=c++17 -fno-exceptions -fno-rtti -fno-math-errno -UMOZILLA_CONFIG_H \
  -D_GLIBCXX_USE_CXX11_ABI=0 \
  -o ../../../../obj-build-mer-qt-xr/aarch64-unknown-linux-gnu/release/build/swgl-c7fddee6f1578b80/out/src/gl.o \
  -c src/gl.cc
In file included from src/glsl.h:7,
                 from src/gl.cc:92:
src/vector_type.h: In instantiation of ‘static T glsl::Unaligned<T>::load(const P*) [with P = glsl::VectorType<float, 16>; T = glsl::vec4]’:
src/vector_type.h:532:28:   required from ‘T glsl::unaligned_load(const P*) [with T = glsl::vec4; P = glsl::VectorType<float, 16>]’
src/vector_type.h:543:27:   required from ‘D glsl::bit_cast(const S&) [with D = glsl::vec4; S = glsl::VectorType<float, 16>]’
src/blend.h:53:41:   required from here
src/vector_type.h:503:11: warning: ‘void* memcpy(void*, const void*, size_t)’ writing to an object of type ‘struct glsl::vec4’ with no trivial copy-assignment; use copy-assignment or copy-initialization instead [-Wclass-memaccess]
     memcpy(&v, p, sizeof(v));
     ~~~~~~^~~~~~~~~~~~~~~~~~
In file included from src/gl.cc:92:
src/glsl.h:1796:8: note: ‘struct glsl::vec4’ declared here
 struct vec4 {
        ^~~~
during RTL pass: expand
src/glsl.h: In function ‘glsl::vec2_scalar glsl::sign(glsl::vec2_scalar)’:
src/glsl.h:662:39: internal compiler error: Segmentation fault
 float sign(float a) { return copysignf(1.0f, a); }
                              ~~~~~~~~~^~~~~~~~~
Please submit a full bug report,
with preprocessed source if appropriate.
See <https://git.sailfishos.org/> for instructions.
$ pushd /home/flypig/gcc-packages/
$ zypper install --oldpackage cpp-8.3.0-1.3.1.jolla.aarch64.rpm gcc-8.3.0-1.3.1.jolla.aarch64.rpm gcc-c++-8.3.0-1.3.1.jolla.aarch64.rpm libstdc++-8.3.0-1.3.1.jolla.aarch64.rpm libgomp-8.3.0-1.3.1.jolla.aarch64.rpm libstdc++-devel-8.3.0-1.3.1.jolla.aarch64.rpm
[...]
(7/7) Installing: gcc-c++-8.3.0-1.3.1.jolla.aarch64 .................................................................................................[done]
$ popd
$ # Now *with* the new gcc
$ /usr/bin/g++ -O2 -ffunction-sections -fdata-sections -fPIC -std=gnu++17 \
  -I../../../../obj-build-mer-qt-xr/dist/stl_wrappers \
  -I../../../../obj-build-mer-qt-xr/dist/system_wrappers \
  -include ../../../../gecko-dev/config/gcc_hidden.h -U_FORTIFY_SOURCE \
  -D_FORTIFY_SOURCE=2 -fstack-protector-strong -DNDEBUG=1 -DTRIMMED=1 \
  -I../../../../gecko-dev/toolkit/library/rust \
  -I../../../../obj-build-mer-qt-xr/toolkit/library/rust \
  -I../../../../obj-build-mer-qt-xr/dist/include -I/usr/include/nspr4 \
  -I/usr/include/nss3 -I/usr/include/nspr4 \
  -I../../../../obj-build-mer-qt-xr/dist/include/nss -I/usr/include/pixman-1 \
  -DMOZILLA_CLIENT -include ../../../../obj-build-mer-qt-xr/mozilla-config.h \
  -Wall -Wempty-body -Wignored-qualifiers -Wpointer-arith -Wsign-compare \
  -Wtype-limits -Wunreachable-code -Wno-invalid-offsetof -Wduplicated-cond \
  -Wimplicit-fallthrough -Wno-error=maybe-uninitialized \
  -Wno-error=deprecated-declarations -Wno-error=array-bounds \
  -Wno-error=coverage-mismatch -Wno-error=free-nonheap-object \
  -Wno-multistatement-macros -Wno-error=class-memaccess \
  -Wno-error=unused-but-set-variable -Wformat -Wformat-overflow=2 -Wno-psabi \
  -fno-sized-deallocation -fno-aligned-new -O3 -I/usr/include/freetype2 \
  -DUSE_ANDROID_OMTC_HACKS=1 -DUSE_OZONE=1 -DMOZ_UA_OS_AGNOSTIC=1 -Wno-psabi \
  -Wno-attributes -Wno-psabi -Wno-attributes -Wno-psabi -Wno-attributes \
  -Wno-psabi -Wno-attributes -Wno-psabi -Wno-attributes -Wno-psabi \
  -Wno-attributes -fno-exceptions -fno-strict-aliasing -fPIC \
  -ffunction-sections -fdata-sections -fno-exceptions -fno-math-errno -pthread \
  -pipe -gdwarf-4 -freorder-blocks -O2 -fno-omit-frame-pointer -funwind-tables \
  -DMOZILLA_CONFIG_H -I ../../../../gecko-dev/gfx/wr/webrender/res -I src \
  -I ../../../../obj-build-mer-qt-xr/aarch64-unknown-linux-gnu/release/build/swgl-c7fddee6f1578b80/out \
  -std=c++17 -fno-exceptions -fno-rtti -fno-math-errno -UMOZILLA_CONFIG_H \
  -D_GLIBCXX_USE_CXX11_ABI=0 \
  -o ../../../../obj-build-mer-qt-xr/aarch64-unknown-linux-gnu/release/build/swgl-c7fddee6f1578b80/out/src/gl.o \
  -c src/gl.cc
In file included from src/glsl.h:7,
                 from src/gl.cc:92:
src/vector_type.h: In instantiation of ‘static T glsl::Unaligned<T>::load(const P*) [with P = glsl::VectorType<float, 16>; T = glsl::vec4]’:
src/vector_type.h:532:28:   required from ‘T glsl::unaligned_load(const P*) [with T = glsl::vec4; P = glsl::VectorType<float, 16>]’
src/vector_type.h:543:27:   required from ‘D glsl::bit_cast(const S&) [with D = glsl::vec4; S = glsl::VectorType<float, 16>]’
src/blend.h:53:41:   required from here
src/vector_type.h:503:11: warning: ‘void* memcpy(void*, const void*, size_t)’ writing to an object of type ‘struct glsl::vec4’ with no trivial copy-assignment; use copy-assignment or copy-initialization instead [-Wclass-memaccess]
     memcpy(&v, p, sizeof(v));
     ~~~~~~^~~~~~~~~~~~~~~~~~
In file included from src/gl.cc:92:
src/glsl.h:1796:8: note: ‘struct glsl::vec4’ declared here
 struct vec4 {
        ^~~~
$

Hopefully this will mean your PR will get merged in.

10 Likes

It is now merged in!

7 Likes

Thanks so much for finding the underlying cause and fixing it. I can now revert the workaround, which means the library can return back to full optimisation. That makes an important difference.

6 Likes