Livecasting porting notes for Zenfone 8

Devices
#1

Hello, today I bought a second-hand but unopened Asus Zenfone 8, what I deemed “a walk in the park” in my previous post.
I’m writing here to both show some transparency and to maybe also inspire other people that they can also port Sailfish to their desired devices. I am NOT writing here to ask for help - whenever I’m stuck I’ll go to the #sailfishos-porters channel. Comments are welcome, of course.

This is open-ended, failure can be one of the outcomes.

Also, this is to put some light pressure on me too. I’ll try to write a new reply whenever I spend an evening on this.

20 Likes
[Sold] Amazfit GTS in great condition, pay only shipping/packaging + cost of new strap
Flypig's gecko dev diary
#2

Day 1:
Rookie mistake - I enabled wifi, set the phone to charge and came back to it offering to reboot because Android 12 was already installed. Now not only do I have to find the asus method to unlock the device, I also have to find images with Android 11, which is what I am targeting with porting with HADK.

Luckily the Android 11 image from the early access to Android 12 is still present, down below on this page. Otherwise the support site seems to only offer OTA packages which I may not be able to extract as firmware files.
Unlocked bootloader with an apk downloaded from the same support site - worked without signing into google or Asus account (skip, skip, skip).

I’ve wasted about an hour because the script installing the Android 11 image, update_image.sh locked at

Sending sparse ‘super’ 17/22 (289528 KB)

Changing the original USB-C to USB-C cable to a ‘normal’ A-to-C fixed this…

Next going to try to install Lineage 18.1 - corresponding to Android 11. Ooops! The downloads for Lineage 18.1 are gone:) The link from ASUS ZenFone 8 - sake - LineageOS 18.1 Changelog is not available on their download server anymore. Only 19.1 version is available…
The 18.1 branches are still there on github, so I can probably build them myself to test the level of support in Lineage.

Next - install TWRP maybe?
Hmm TWRP is not a recovery.img anymore. Either needs renaming to twrp.img before uploading or 3.7 doesn’t work. Nope, even previous version doesn’t. I need to flash boot.img on boot_a and boot_b back to make original firware boot again. A quest for another day…

So these are 2 out of 4 guides I started to follow:

The next ones are:

The reason I’m trying this in more than one way is that I’ve been made curious of other ports that were done by re-using the Halium build. If I go that route, I could get two ports out of this (and see some Qt 5.15 apps along the way maybe?)
It is also because I have thought this is a “walk in the park” initially (before knowing that there are no downloadable Lineage 18.1 builds) and ‘the longer route’ seemed fun and it stuck with me.

So, for Halium (full) - I dont’ really understand instructions from Get Halium source — Halium documentation :slight_smile: What needs to be added where in the manifest xml file.
My initial version (not tested) would be

<manifest>
	<project path="device/asus/sake" name="android_device_asus_sake" remote="los" />
	<project path="kernel/asus/sm8350" name="android_kernel_asus_sm8350" remote="los" />
	<project path="vendor/asus" name="proprietary_vendor_asus" remote="them" revision="lineage-18.1" />
</manifest>

One of the things that I change with this port, from the previous one (other than the Android 10 to 11 migration), is that official Lineage is building the vendor image too. My previous one was based on an unofficial lineage which had BUILD_WITHOUT_VENDOR in the device BoardConfig.mk
This requires one to extract binaries from an actual device with a script. Or you can go and search existing ones on TheMuppets repo, available also on gitlab for those vendors issuing takedown notices;)
That is the “them” reference in the manifest.

The Lineage guide is very straightforward to follow since it’s the 2nd time for me (except the above blobs finding). This build is already hogging my computer as I’m writing this.
image

14 Likes
#3

Day 2:
In the morning I found the lineage build failed after 1h30’. The error was related to perl not loading libcrypt.so.1. After a couple of internet searches I did found that my distribution, just like in this comment, does not package libxcrypt-compat when you install perl.
It seems that there is one C file in the kernel that is generated using perl.
After having the above package installed, I continued the lineage build.
3h30’ more.
So it would’ve hit the 5 hours mark if it worked the first time, on my not-so-shabby laptop :-S (I even get warnings that 16GB may not be enough:p)

So when the evening came, a shiny new lineage-18.1-20230220-UNOFFICIAL-sake.zip was waiting for me to be flashed.

But as you know, I didn’t manage to install TWRP yesterday. In hindsight, the 2nd method might have worked, but the Asus ROM might have overwrote it(?).
So I start the not-installed TWRP with fastboot boot twrp.img, push the lineage zip, install it, reboot.

The familiar not-so-smiley loading screen appears with a bubble passing by. And it loads… and loads… and in 30 minutes it doesn’t finish loading.
Volume-Down (this time) + power button. Back to Asus recovery. Let’s fastboot boot twrp.img again… Gets stuck at ASUS logo.
Oops.
Just reboot. Stuck at lineage logo.

The error I’ve made above was that I didn’t wipe user data when switching ROMs. I knew that, in hindsight, but did not now how to do it since TWRP wasn’t starting either and ASUS recovery didn’t have a such option.

Let’s see how yesterday update-script.sh does it. Looking for “user” - there it is fastboot -w. That’ll save me.

‘Erase of userdata is not allowed in snapshotted state’

Interesting, So the device has some protection to know that the new ROM did not install correctly. I was vaguely aware that there was a slot A vs slot B on android since maybe 9th version (my current Xiaomi doesn’t use this though). But this sounds interesting. fastboot --help also says I can: snapshot-update merge or snapshot-update cancel.
Going with the first: no real difference. Going with cancel. Success. User wipe. Reboot. ASUS ROM boots, welcoming me to my new phone.

Back to ‘boot twrp.img’. Install Lineage. fastboot -w. Reboot.
And. it. smiles. Lineage has started. Skip, skip, skip. Vibration, sound, wifi, gps, compass, calls, data all seem to be working.

This should have been Day 1 :slight_smile:

The camera seems to render overshapened photos now :angry: . Maybe it’s something I can strace in the original ROM later, not a dead end. :crossed_fingers: .

While lineage was taking its hours of building, I found the time to progress a bit through the full halium track. First thing is common to HADK: use GitHub - mer-hybris/mer-kernel-check: Scripts to check kernel CONFIG_ values used by Mer script to add config flags to your kernel.

Did that as best as I could. Next is ‘fixup-mountpoints’.
This is a step requiring you to $ find device/ -iname fstab\* and also get access to the real device through adb root (good thing I have lineage now) then add some sed commands to halium/hybris-boot/fixup-mountpoints…

Wait, I don’t have halium/hybris-boot/fixup-mountpoints. Maybe I missed a step yesterday. Let’s repo sync again… (And if I hadn’t the kernel config open in Sublime, I would have lost the mer checker changes because I was not any wiser!)
But the fixup-mountpoints script is nowhere to be found. Let’s peek a bit on their telegram channel, maybe I’m not the first to make this mistake…
Result: Telegram: Contact @halium says I’m doing it wrong

It seems to me that the full halium track is not meant for the newer devices.
Maybe it’s time to switch tracks.
Where was I?

The next ones are:

14 Likes
#4

Day 3
This is a shorter update.

It is also an update which is not related to SFOS.

The Ubports documentation says that you just need to build the kernel, and the fix your boot image with some ramdisk of sorts.
The documentation states that the ‘fix’ parameters can be found by running unpack_bootimg from lineage-17.1
There are some problems with that which I found:
The lineage python script crashes with integer division or modulo by zero. It seems that it finds a page_size=0 and uses that in a division.
Peeking into the Ubports porters telegram channel, the original google unpack is advised.
This doesn’t show much info.

But looking at the lineage one and the google one I can see that the google one intetionally skips some info if version==3. Further peeking into the Ubports porters tells me people use this on vendor_boot.img, not on boot.img

This confirms what I’ve seen by scanning through the scripts, that the google one can unpack also another type of boot image. The vendor boot one.

So Android 11 has separated these things. I now think it’s called GKI, or generic kernel image.

Another thing I’ve noticed in the above channel is that if you have “gki1” (probably the android 11 version, as opposed to android 12+?) then you should just start your Ub port from… Xperia 10 IV.

That’s right. To my astonishment, there is a repo for Xperia 10 IV in ubports!!

(btw, hint for Jolla: the ubports telegram channel is an order of magnitude more populated than the SailfishOS one. Even though I know which one is based on which, and that the Ubuntu brand is bigger, I still think that people looking to free their devices may skip Sailfish because of the lack of openness, the same one being related to Qt)

Anyway, the documentation is up-to-date just as the HADK is. I should join the group tomorrow and ask around.
But I am truly hardly resisting to just go ahead with HADK/sailfishos-porters and skip this, at least for one dev day or two.

9 Likes
#5

This is a cool format, I look forward to the update every morning :wink:

I know you’re nowhere near that point yet, but are you considering making the necessary arrangements for a waydroid install (https://github.com/sailfishos-open/waydroid#porter-tasks)?

1 Like
#6

Thank you. It will not be every morning though:)

Yes, once basic system needs are fulfilled, I will adapt the kernel for waydroid.

3 Likes
#7

Day 4:
So with the deviceinfo file from Xperia 10 IV I started trust that I can start the build process of halium kernel-only.
I changed the deviceinfo a bit to point to my kernel, the name, codename etc for my device and ran build.sh from that repo. Which failed with ‘no space left on device’. which device? /tmp. Admittedly I only have 8GB there:)
./build.sh -b ../intermediate/ changes the ‘temporary’ folder to one of your choice. Did that. And then kernel got built a second time, this time with the changes I made to pass mer kernel verifier.

The out folder has 4 files:
device_sake.tar.build device_sake_usrmerge.tar.build
device_sake.tar.xz device_sake_usrmerge.tar.xz

(some other files are in …/intermediate/downloads)

Instructions not clear Installing Halium-based builds - UBports documentation, no matter how many times I read and re-read them. I try to filter them out by “Halium 9” instructions:

  • Install TWRP
  • /data partition ext4, not encrypted
  • fastboot flash boot halium-boot.img
  • Download the GSI. The link seems to point to generic_arm64 build on jenkins that does not exist.
    Using this one instead.
  • “Extract the downloaded file and locate the file android-rootfs.img in the directory system/var/lib/lxc/android. This is the GSI file you will be transferring to the device.”
  • Clone or download Jonah Brüchert / halium-install · GitLab

The chances I don’t get this right are very high.

While trying to find references to what file should go where to install on the device, I spend some time fixing some of the sony specific files I found in the repo:

Adjusting overlay/system/etc/modules-load.d/pdx225.conf → the list of modules. I rename it to sake.conf and populate it with whatever find -iname \*.ko I got.

Adjustig overlay/system//etc/udev/rules.d/70-pdx225.rules → renamed to 70-sake.rules and populated with this script:

cd lineage/out/target/product/sake
DEVICE=sake
cat system/etc/ueventd*.rc vendor/ueventd*.rc | grep ^/dev | sed -e 's/^\/dev\///' | awk '{printf "ACTION==\"add\", KERNEL==\"%s\", OWNER=\"%s\", GROUP=\"%s\", MODE=\"%s\"\n",$1,$3,$4,$2}' | sed -e 's/\r//' >/etc/udev/rules.d/70-$DEVICE.rules

Running build again.

Back to install instructions:

  • halium-install -p <mode (reference, neon, ut, ut20.04, debian-pm, debian-pm-caf, none)> <rootfs.tar.gz> <system.img>
    • where system.img is android-rootfs.img ?
  • Where is halium-boot.img? …/intermediate/tmp/partitions/boot.img ?
  • what is rootfs.tar.gz then? Telegram: Contact @ubports_porting Saikat here: just flash boot.img and system.img :slight_smile:

Asked confirmation on channel… I probably dont’ sound very clear this late in the evening:)

Ok, let’s run the instructions as I’ve understand them:

$ fastboot boot twrp-3.7.0_12-0-I006D.img
→ stuck at logo. Hmm, I’m pretty sure this used to work with Asus firmware. Ever since I got Lineage and Lineage Recovery (automatically?) this seems not to work.

Just use ASUS recovery then… fastboot -w should work, and flasing images through fastboot was never a problem. Wasn’t it?

$ sudo fastboot flash boot …/intermediate/tmp/partitions/boot.img
(OK)
$ sudo fastboot flash boot android-rootfs.img

Sending 'system_a' (459100 KB)                     OKAY [ 15.762s]
Writing 'system_a'                                 FAILED (remote: '(system_a_a) No such partition')
fastboot: error: Command failed

Oh, how things change.
Reading on some google android documentation about adb reboot fastboot was not a thing I believed. Until I read the mention again from somebody on XDA: Dynamic "super" partitions and Fastboot | XDA Forums

Doing that, and another lineage recovery pops up, this time with fastbootD logo :slight_smile:
Ook…

$ sudo fastboot flash system android-rootfs.img 
Resizing 'system_a'                                OKAY [  0.004s]
Sending sparse 'system_a' 1/2 (262068 KB)          OKAY [  9.337s]
Writing 'system_a'                                 OKAY [  0.491s]
Sending sparse 'system_a' 2/2 (196572 KB)          OKAY [  6.134s]
Writing 'system_a'                                 OKAY [  0.458s]
Finished. Total time: 16.588s

Yay…!
Booting.

…

Not. Stuck at ASUS logo again. Let’s go back to ASUS bootloader. Maybe try to get into recovery, Option doesn’t work.
Let’s try to boot twrp again. Of course it doesn’t work either.
Let’s try sudo fastboot snapshot-update cancel, it worked before, right?
Nope.

Basically I have a phone which only boots into bootloader now. No recovery and no ability to flash zips that I know of. Unless…

I flash ASUS Android 11 again. That has a super partition ready, not a subset of it, can be flashed from booloader.
Whew, phone greets me to my Zenfone again.

Not broken, but no advancing today either…
One thing that bothers me, after dodging a bricked phone again, is that I never had a chance to see what the error was from that “halium” boot. I understand that I may have misread the instructions, but I am used to get logs from my mistakes. It used to be that you’d reboot into twrp and grab /sys/fs/pstore/console-ramoops0 or something. Now I am a bit blindfolded and need to read things again and again…

9 Likes
#8

Hello,

I do not have a zenfone and doubt i will ever get one (but hey, who know)

Anyway just writing to say i love what you’re doing with these posts :slight_smile:

#9

Of course, but I’m still excited every morning, you can not prevent that anymore :wink:

But totally clear, you deserve rest and it’s also allowed to prioritize other things :wink:

#10

Thanks for the encouragement:)

#11

Day 5: Kernel first

Some morning thoughts:

These commands are documented too

./build/prepare-fake-ota.sh out/device_sake.tar.xz ota
./build/system-image-from-ota.sh ota/ubuntu_command out

in one of the repo linked from docs.

I run them and get actual out/system.img which is better than what I’ve tried to flash yesterday but the results are no different.

Could things be more boring than the first knee-jerk reaction of flashing the first output images working? They’re not, so we can now continue the saga.

I need to think and divide the problems into smaller ones.
The problems I now know are:

  • I don’t know how to handle boot.img vs vendor_boot.img - kernel doesn’t boot - no logs, no telnet access
  • System is a dynamic partition inside a “super”-partition. There’s a nice description over here and I am really fond of the “In true Google fashion” part.

But let’s take them one by one. Kernel first.
I should be able to see Mer Debug telnet on port 23 and I am not.
There’s another kernel checker here halium-boot/check-kernel-config at master · Halium/halium-boot · GitHub. I try to satisfy those requirements too.
This time I’m only flashing boot.img and when failed, flashing back boot.img of lineage, so no system is reset between the two, data remains usable. But no difference, still radio silence at dmesg -w.

The next thing I notice is that the Xperia 10 IV kernel has some commits adding CONFIG_USB_CONFIGFS_RNDIS. I should know, I received the same advice 2 years ago for my Xiaomi.
It also contains some ABI change for SYSV_IPC. I should try to import that too.
I flash this and I do get “USB device connected” but no Mer telnet working yet.

Let’s take a peek at the vendor_boot and boot.imgs of lineage and asus.
Using the unpack_bootimg tool we can get the files inside and the parameters to stdout. Also, ramdisks can be tested with the file command to see if they’re gzip or lz4 (lineage used that) and upack them to the cpio format - which I inspect with mc.

Unpacking lineage boot/vendor_boot:

  • boot.img contains kernel + a ramdisk with /system/etc/hw/init.rc and e.g. /etc/recovery.fstab, /bin/toybox executables but also /bin/recovery, /bin/fastbootd. The init seems to mount sys/fs/pstore. There’s an aditional /etc/init.asus.recovery.rc?
  • vendor_boot.img contains dtb + vendor_ramdisk with a single module in /lib/modules/focaltech_fts_zf.ko

Unpacking asus boot/boot-gki/vendor_boot

  • boot.img very simlar to lineage (or the other way around:)
  • boot-gki.img has very different kernel (28MiB, vs 44MiB) but same ramdisk (there is no actual partition called boot-gki). Using extract-ikconfig shows many different configurations - the suffix is CONFIG_LOCALVERSION="-qgki-perf"
  • vendor_boot.img contains dtb + vendor_ramdisk. Slight differences in fstab included compared to lineage (xrom?). lib/modules has a folder 5.4-gki with 40 modules! modinfo shows the modules have
    vermagic: 5.4.61 SMP preempt mod_unload modversions aarch64

Unpacking ubports boot.img:

  • boot.img has a 45MiB kernel and a ramdisk that is so Linux:)
    dtb seems to be missing also.

This dtb (device tree - binary format) is pretty important for a kernel to boot IMO. Well, at least on arm, you don’t need that on desktops.

Also, the Lineage and Asus ones have “” (empty) cmdline for boot.imgs and the vendor_boot.img has the “real” cmdline I’ve configured in Haliums’ deviceinfo file.

Part of the “Kernel first” thought is - if Lineage can make bootable images, so can Halium.
I just need to find the script that makes the image in both, compare them and edit/configure them accordingly.

Script for generating boot.img in Halium is here.
There is even a reference to “dynamic partitions” there.
Also a reference to unpacking a recovery.img ramdisk and overlaying that.
I should look up other Android 11 devices to see what other less documented options are used. (unfortunately the only android device with image version=“3” is Xperia and it doesn’t have ramdisks)

Let’s find Lineage’s one. Finding anything in Lineage is an unknown to me - imagine that there ain’t a text editor that can show the folder structure and still operate normally. Usually I grep specific filenames :slight_smile:
Creating boot images must be something driven from a .sh or .py script.
What we should find is some token from device/asus/sake/BoardConfig.mk that instructs the build to build these kind of images.
Something like lineage$ find . -type f -iname \*.py -exec grep mkboot {} \; -print finds some scripts.
I shall inspect them tomorrow when I’m more intelligent.

Sorry, no drama today. Device is still safely running Lineage:) while I get my thoughts together.

3 Likes
#12

Day 6 and a half: building vendor_boot

As I said earlier, am not very familiar with the AOSP or Lineage folder structure, so I am using brute force:
find . -type f \( -iname \*.bp -or -iname \*.sh -or -iname \*.py -or -iname \*.mk \) -exec grep "vendor_boot" {} \; -print

I find a python script in lineage/build/make/tools/releasetools which would use MKBOOTIMG env var, but that doesn’t seem to be called.

Browsing around existing repos I find that Volla 22 also has ramdisk-recovery.img and ramdisk-recovery-overlay/, and Xperia 10 IV’s kernel cmdline is just console=tty0 systempart=/dev/mapper/system. I tried to also copy the ramdisk cpio gz’d from my lineage boot but after some debugging I found it is not applied because it is destined for $deviceinfo_has_recovery_partition parameter…

Back to Lineage image creation: found a Makefile this time, in build/make/core/main.mk. It uses BOARD_CUSTOM_MKBOOTIMG if defined. meaning that I can redirect the mkbootimg executable with that define (probably in device/asus/sake/BoardConfig.mk)
But that would mean an unidentified number of targets dirtied. Better find the actual executable and make it output more info…

file out/soong/host/linux-x86/bin/mkbootimg

ELF 64-bit LSB shared object, x86-64

Hm. Not the python script I was expecting:) Plus, the fact that is in out dir may mean that if I touch it it may get rebuilt. Maybe defining a mkbootimg in BoardConfig is better though.
So I create this script and define it as “my custom mkbootimage”:

#!/bin/sh
echo MAKEBOOTIMAGE "$@" | tee -a ~/debugging/mkbootimg.log
~/lineage/out/soong/host/linux-x86/bin/mkbootimg "$@"

and run “make vendorbootimage” then “make bootimage”

(Meanwhile piggz is explaining his way of booting off halium-based SailfishOS: Telegram: Contact @sailfishos_porters. It’s interesting that he uses rpm packaging for the kernel. However, the build scripts seem to make boot.img only.)

An there they are: the two commands that build the two images are there. Their parameters are more or less:
For vendor_boot:
mkbootimg --dtb dtb.img --base 0 --pagesize 4096 --vendor_cmdline $kernel_cmdline --os_version 11 --os_patch_level 2023-02-05 --header_version 3 --vendor_ramdisk $ramdisk --vendor_boot vendor_boot.img
and for boot:
mkbootimg --kernel $kernel --ramdisk $ramdisk-recovery --os_version 11 --os_patch_level 2023-02-05 --header_version 3 --output boot.img

In the halium-generic-adaptation-build-tools the command is
mkbootimg --kernel "$KERNEL" --ramdisk "$RAMDISK" --cmdline "$deviceinfo_kernel_cmdline" --header_version $deviceinfo_bootimg_header_version -o "$OUT" --os_version $deviceinfo_bootimg_os_version --os_patch_level $deviceinfo_bootimg_os_patch_level $EXTRA_ARGS.

Let’s split it in two: the recovery ramdisk should go to boot.img and the current halium ramdisk to vendor-ramdisk, right? (maybe?)

bootloader-support GKI documentation actually says that the two ramdisks are getting concatenated and they need to be the same format (LZ4). This might be one of the reasons my device doesn’t boot, since lineage uses LZ4 but asus and ubports use GZip. And I think I’ve only ever tested ubports boot + lineage vendor_boot.

Another generic-boot doc from Google says (even though it seems Android 12+) that vendor_boot’s /first_stage_ramdisk is root folder when not in recovery.

So the plan for tomorrow is:

  • Make my ramdisk-recovery.img gzip (done earlier today), move to asus-sake root
  • copy dtb.img to root (done)
    • better build it, as piggz did above
    • for now, declaring it as prebuilt in deviceinfo (done)
  • Change halium-boot-ramdisk.img to have /first_stage_ramdisk as root folder (done)
    • (or the scripts, or add it to ramdisk-overlay?)
    • the final solution should automate this and the creation of /lib/modules/ content
  • add $deviceinfo_vendor_cmdline parameter (done)
  • detect “$deviceinfo_bootimg_header_version” -eq 3 and
    • issue the two commands above in that case
    • replacing boot.img with vendor_boot.img for one of them

Today is now ^^ “tomorrow”:
Looks like there’s an avbtool that is run too. Let’s debug that call with linege BoardConfig.mk BOARD_CUSTOM_AVBTOOL too, by wrapping it in a script as before.

For bootimage - for now, except that I don’t have a .pem key and the --prop parameters are missing, the first part up til --key is covered by the build.sh script

For vendorbootimage I add similar line as seen in Lineage.

Ok. the boot.img and vendor_boot.img generated now look better.
But the vendor ramdisk is two concatenated cpio.gzipped, I need something like
(while cpio -t ; do :; done) < vendor_ramdisk.cpio to see all the files. Hope the boot will also see them.

Flashing the two boot images…
This time asus logo does not get stuck on screen. It rapidly bootloops.
Different error! Yay! Progress!

5 Likes
#13

Wow, this is a real interesting thread. I enjoyed reading it. I don’t own the device, but thank you for your work, especially the documentation. This thread can give you a good overview of how much hard work can go into a port. Thanks, and good luck :slight_smile:

#14

Thanks Systematics:)
This is not actual documentation because I don’t know it either. It’s just a ‘story’ for now.

#15

Day 7 part 2
From telegram channel:
Q: so I was wrong when I flash system.img
A: yeah that goes inside the rootfs
A: and halium-install will do that for you

The *-ota.sh scripts above seem to be doing this system.img wrapping android in LXC in UT
asus-sake/ota/halium_halium_arm64.tar.xz - that is the UT rootfs.

Yay, another guide UBports pad (today it starts in Cyrillic. Maybe somebody pasted a translation)
Another (3rd?) advice on kernel config. This time I’m missing
CONFIG_PROC_NS
Next will be lxc-checkconfig maybe.

Back to booting: one possiblity would be that I need to revert my vendor_boot changes, switch the boot.img to version 2 and copy a similar device offsets from Poco X3 Pro.
Well it didn’t. Altough vayu (Poco X3 Pro) was launched with Android 11 it doesn’t have this image version = 3 which implies vendor_boot :hmm:

Entering fastbootD with adb restart fastboot does not help writing the partitions either

Warning: skip copying boot_b image avb footer (boot_b partition size: 100663296, boot_b image size: 134217728).
Sending 'boot_b' (131072 KB)                       OKAY [  4.383s]
Writing 'boot_b'                                   FAILED (remote: 'Value too large for defined data type')
fastboot: error: Command failed
$ fastboot flash system out/system.img 
Resizing 'system_b'                                FAILED (remote: 'Not enough space to resize partition')
fastboot: error: Command failed

The first error might be because I declared vayu’s size for boot partition. Hopefully it’s just padding. I put back the lower sake (my device) number in, and the partition is flashed (it’s interesting that the bootloader did not warn about that, but fastbootd does, hmm)

Aye aye aye, that probably overwrote something. After a couple of failed reboots, instead of switching the active slot, the device shows an even scarier screen (than the ‘unlocked’ one): your device is corrupt yadda yadda.
Changing slots is not possible because FAILED (remote: 'Slot Change is not allowed in Lock State').
And search results on that message are even scarier.
But access to bootloader is still intact (btw, I think I got my terms right now: bootloader is volUp + power, you manually enter recovery from there - which is on the boot partition, remember? and fastbootD mode from there via adb restart fastboot). At least the first bootloader mode is still accessible. What was the last time we flashed Asus’s firmware…? Time to do that again. And it boots.

Ok. Trying again an advice from UBports found earlier, to flash boot.img to vendor_boot partition and boot-gki.img (asus has one) to boot partition - does not work. Or maybe it is because I tried with the image v2? Don’t remember, need to double check.

My host distro pcieport says something like BadDLLP / device [8086:1576] error status/mask=00000080/00002000 when the phone boots. Seems related to USB.
Maybe I should correct that first. I’ve temporarily added pcie_aspm=off at rebooted the laptop. Message does go away, but no change in USB connectivity.

Back to image v3 flashed as is on vendor_boot.

$ sudo fastboot flash boot boot-gki.img
fastboot: error: Failed to identify current slot

What, what did I do again?

$ sudo fastboot set_active b
fastboot: error: Device does not support slots
$ sudo fastboot set_active a
Setting current slot to 'a' FAILED (remote: Slot Change is not allowed in Lock State
$ sudo fastboot flash vendor_boot_a boot.img 
Sending 'vendor_boot_a' (98304 KB)                 OKAY [  3.442s]
$ ./update_image.sh 
Get hwid failed!
$ // rebooting device
$ ./update_image.sh
[...]
filesize : 13680
pt_crc32_val:  img_crc32_val:E743228A
Crc32Check fail partition : multiimgoem_a

This starts to happen at random partitions / points failing while flashing.
Maybe my host kernel pcie_aspm=off parameter?
Let’s reboot without it…

Unfortunately since I wrote the above the problems I have with flashing images are consistently (though randomly) happening. I managed to finish an Asus firmware flash without CRC failures and the phone seems to work. But I need to reboot the bootloader a couple of times to write a boot image, otherwise it gets stuck, or gives one of the errors from the previous commands at random times. I tried booting twrp fastboot boot twrp.img and that never worked.
I’ll probably need to find something like a service manual to see if there’s a deeper reset, or switch cables another time, I don’t know if it was the host USB params changed, the overshoot in writing a partition or just a cable that broke (hopefully SSD storage does not break in 1 week of usage;)
Meanwhile I am waiting for somebody to answer or I’ll bump my questions about vendor_boot on the Ubports telegram channel. In the hope that I would have more info and flash less frequently broken pieces.
Or I should start building with SFOS’s HADK, wonder what would that output as boot images. :hmm:

3 Likes
#16

@vlagged, enjoying the series :slight_smile: . On Tama, we had to use fastboot reboot-bootloader before flashing in some conditions. Maybe something similar is here as well. Although, looks to be more complicated …

#17

Thank you rinigus. I’ll try that too next time I want to flash something!

#18

Day 8: HADK
Short search about fastboot problems on Zenfone 8 surfaces the thread I already knew about with the “ramdump” error. This is not an error I’m seeing but checking it nonetheless reveals

Handset is 2021.04 for MFD and C11P2003 for BATT.

This is exactly the manufacturer date and batch number of my phone! Well :crossed_fingers: let’s just proceed with caution, I’d say.

Where was I?

Next:

  • HADK, of course (most familiar…)

Next I’ll try to build the Android bits from HADK, just to see what images it generates.
Compared with the Halium “standalone kernel method” (but probably similar to Halium full build) the HADK Android bits are actually parts of AOSP and lineage interleaved with hybris and mer specific sources, built with the Android build process. There are chances that the final images built are still similar to the Lineage ones.

The HADK needs to be followed with ‘tweaks’ stated in HADK-FAQ. There’s also a list of “hot” HADK FAQ issues in the #sailfishos-porters or telegram channel.

So I am going to adjust my ol ~/.hadk.env to point to new values for asus/sake and checkout the manifests for the build.
host[hadk]$ repo init -u https://github.com/mer-hybris/android.git -b hybris-18.1
The manifest xml from day one, with my phones “device”, “kernel” and “vendor” repos now goes to hadk/.repo/local_manifests/sake.xml
Repo syncing takes many minutes, maybe 20-30, I didn’t remember a time message at the end of the proces.

Checking out hybris-patches branch hybris-18.1 and hybris-patches/apply-patches.sh --mb
Entering PlatformSDK with sfossdk then HABUILD_SDK with ubu-chroot. (Remember to install cpio in that ubuntu chroot;)

And starting the build. This should take a while, somewhat less than Lineage since the Java bits should not be included.

HABUILD_SDK[hadk]$ source build/envsetup.sh 
HABUILD_SDK[hadk]$ TEMPORARY_DISABLE_PATH_RESTRICTIONS=true breakfast sake
HABUILD_SDK[hadk]$ TEMPORARY_DISABLE_PATH_RESTRICTIONS=true make -j$(nproc --all) hybris-hal droidmedia

(that env setting is for allowing commands like cpio or python to be run by android makefiles btw. I know it from the FAQ;)

After 10 mins of build:

FAILED: ninja: ‘libhwc2_compat_layer’, needed by ‘hybris-hal’, missing and no known rule to make it

Ah, I forgot to clone libhybris. Like, the main thing.
host [hadk]$ cd external; git clone --recurse-submodules https://github.com/mer-hybris/libhybris.git; cd -

10 mins of build again:

ERROR: sake does not have mountpoint fixup data - see
Sailfish OS HADK for details on how to fix this.

Well, I this is the kind of work I was hoping to avoid by trying the GSI method.
The thing is, that at this point I am pretty sure that the partitions inside super.img do not need to be included as they won’t work anyway.
I need to mount them in systemd services or like on nabu, in the early init script. But that’s later, when I get a sign of life from kernel and telnet access.

Fixup-mountpoints replaces path found in fstab with actual /dev/sd* devices you have on your phone.
For example, if fstab has

/dev/block/by-name/metadata                             /metadata              ext4    noatime,nosuid,nodev,discard

And on device (e.g. from mount command) you have

/dev/block/sda22 on /metadata type ext4 (rw,seclabel,nosuid,nodev,noatime,discard,stripe=2)

Then the sed replacement for that entry is

            -e 's block/by-name/metadata sda22 ' \

10 mins of build later:

FAILED: ninja: ‘com.android.runtime.release’, needed by ‘hybris-hal’, missing and no known rule to make it

Let’s grep Lineage to see in which .mk file is that ‘com.android.runtime.release’. Nada.
Let’s grep hadk folder - found in hybris-boot. Oh - that would be repo I was trying to be smart and use from my fork, turns out is out of date for building 18.1. I save the old master as a hybris-17.1 branch because I need it as it’s working there, and reset master to the one in the mer-hybris org.

15 minutes of build later

sed: bad pattern ‘s block/by-name/metadata sda22’@26 (s)

Well, remember that space at the end of the string, in the fixup example above? I didn’t, when I made the multiline edits.

37 minutes of build later:

Wow, kernel build error? 
In file included from /home/vlad/hadk/kernel/asus/sm8350/arch/arm64/lib/xor-neon.c:11:
In file included from /home/vlad/hadk/kernel/asus/sm8350/arch/arm64/include/asm/neon-intrinsics.h:33:
In file included from /home/vlad/hadk/prebuilts/clang/host/linux-x86/clang-r383902b1/lib64/clang/11.0.2/include/arm_neon.h:31:
/usr/include/stdint.h:96:23: error: typedef redefinition with different types ('unsigned int' vs 'unsigned long')
typedef unsigned int            uintptr_t;
                                ^
/home/vlad/hadk/kernel/asus/sm8350/include/linux/types.h:37:24: note: previous definition is here
typedef unsigned long           uintptr_t;

I’m surprised because this is the nth time I’m building these kernel sources. At least in two configurations already - the default one that came from lineage and the one used in my “Halium standalone kernel” trial.

This must have to do with something like 1). compile flags differences, 2.) compiler (?), or 3). the host - because I’m building in an ubuntu chroot now. (maybe that last part is also optional, but I wouldn’t “make clean” and re-try from host just now. Especially since I’m on battery this evening).

  1. Differences in compile flags can be inserted by the hybris and mer pieces, presumably (doubtful). It would also mean that include/linux/types.h:37 has the typedef guarded by an #ifdef, and can have another value on #else, or something like this. This doesn’t seem to be the case (except ifndef __ASSEMBLY__).
  2. Differences in compiler would change the clang-r383902b1/lib64/clang/11.0.2/include/arm_neon.h:31 path to another/sibling one which has another value for the typedef. Needless to say, lineage includes exactly the same compilers in prebuilts/clang/host/linux-x86/. The ubports checks out a compiler named “clang-r383902” (without b1) but the lib64/clang/11.0.1/include/arm_neon.h does not differ.
  3. Differences in the host are related to exactly the TEMPORARY_DISABLE_PATH_RESTRICTIONS above - the restriction was introduced to "help ensure that build results are reproducible across different machines: The list of tools restricted is here.

When you have a multi-core build failure, you usually get output mangled from the multiple things writing to stdout. Repeating the last command has higher chance of getting just the single rebuild error.
That actually contains, in my case, a warning about „Disallowed PATH tool “ld” used: []string{"/home/vlad/hadk/out/.path/ld (…)” but that might not be the tool that introduced the error (because it was executed from buil "out"put. However, anything in the build output can be a file generated by a differing host tool from the list of tools above.

Tomorrow I will probably make clean, I don’t “need” to figure this out.

4 Likes
#19

Day 9: spot the difference

So I will build the kernel that was already succesfully built in ubports, this time on the host machine (instead of ubu-chroot)

host[hadk]$ export ANDROID_ROOT=$(pwd)
host[hadk]$ TEMPORARY_DISABLE_PATH_RESTRICTIONS=true breakfast sake
host[hadk]$ TEMPORARY_DISABLE_PATH_RESTRICTIONS=true make clean
host[hadk]$ TEMPORARY_DISABLE_PATH_RESTRICTIONS=true make -j$(nproc --all) hybris-hal droidmedia

15 mins of build later

  CC      techpack/dataipa/drivers/platform/msm/ipa/ipa_v3/ipa_hdr.o
In file included from /mnt/datas/sake/hadk/kernel/asus/sm8350/arch/arm64/lib/xor-neon.c:11:
In file included from /mnt/datas/sake/hadk/kernel/asus/sm8350/arch/arm64/include/asm/neon-intrinsics.h:33:
In file included from /mnt/datas/sake/hadk/prebuilts/clang/host/linux-x86/clang-r383902b1/lib64/clang/11.0.2/include/arm_neon.h:31:
/usr/include/stdint.h:96:23: error: typedef redefinition with different types ('unsigned int' vs 'unsigned long')
typedef unsigned int            uintptr_t;
                                ^
/mnt/datas/sake/hadk/kernel/asus/sm8350/include/linux/types.h:37:24: note: previous definition is here
typedef unsigned long           uintptr_t;
                                ^
1 error generated.

Oh wow, same error. I didn’t record if it was in this compile unit before?

Good thing that I have a commit that disables CONFIG_RNDIS_IPA, maybe that’s related and is not pushed.
Pushing that fetching back: same error, now when CC arch/arm64/lib/xor-neon.o

Interesting that arch/arm64/include/asm/neon-intrinsics.h after including <arm_neon.h> says

#ifdef CONFIG_CC_IS_CLANG
#pragma clang diagnostic ignored "-Wincompatible-pointer-types"
#endif

There is someone on the internet that says to look inside out/include/generated/compile.h to see what was used to compile. Of course I have

#define LINUX_COMPILER "Android (6877366 based on r383902b1) clang version 11.0.2 (https://android.googlesource.com/toolchain/llvm-project b397f81060ce6d701042b782172ed13bee898b79), LLD 11.0.2 (https://android.googlesource.com/toolchain/llvm-project b397f81060ce6d701042b782172ed13bee898b79)"

and the lineage kernel has exactly the same as above while ubports kernel has, which is a different linker, but the error I have is not link-time (nor LTO-time)

#define LINUX_COMPILER "Android (6443078 based on r383902) clang version 11.0.1 (https://android.googlesource.com/toolchain/llvm-project b397f81060ce6d701042b782172ed13bee898b79), GNU ld (binutils-2.27-41d8fcb) 2.27.0.20170315"

So I have two building envs the same, one building kernel with patches (hadk) one without (lineage) and another environment that builds the kernel with the same patches but doesn’t error out.

Of course it’s tempting to just ignore the error somehow or patch the header file to match the definition (and tempting still is).

How to have verbose make output in kernel build?
For ubports build.sh, I can manually add V=1 in the build-kernel.sh
That gives me an entire command for xor-neon.o build.
For lineage/hadk, I could make showcommands kernel says the internet, but that in turns says back

! The argument `showcommands` is no longer supported.
! Instead, the verbose log is always written to a compressed file in the output dir:
!   gzip -cd out/verbose.log.gz | less -R

I still don’t have the expanded command for the hadk failure.
I’ve tried as seen here
make V=1
make -n
make SHELL=“sh -x”

Interesting that make -n says that it doesn’t know the argument.
Which make? returns /usr/bin/make
/usr/bin/make --help works, make --help does not
Let’s peek into build/envsetup.sh:

function get_make_command()
{
    # If we're in the top of an Android tree, use soong_ui.bash instead of make

Oh
So I should just switch to the kernel folder and I get ‘normal’ make?
host [hadk]$ cd kernel/asus/sm8350/

make V=1

/usr/bin/env: ‘python2’: No such file or directory
***
*** Configuration file ".config" not found!
***
*** Please run some configurator (e.g. "make oldconfig" or
*** "make menuconfig" or "make xconfig").
***
Makefile:639: include/config/auto.conf: No such file or directory

Disaster:)
Why not use ubports make command then? Is as easy as

make O="$OUT" $MAKEOPTS $deviceinfo_kernel_defconfig
make O="$OUT" $MAKEOPTS -j$(nproc --all)

where MAKEOPTS are CC= and LD= defs

(long story short, I am too lazy to figure this out)

Detour:
Lahaina seems to be codename for the SoC
kernel’s build.targets includes only build.config.msm.lahaina
Adding V=1 in there does not work though.

Finally I gave up and re-run that ugly command that was output with the error message and sticking a V=1 somewhere right before -j8

PATH=/mnt/datas/sake/hadk/out/host/linux-x86/bin:$PATH PATH=/mnt/datas/sake/hadk/prebuilts/clang/host/linux-x86/clang-r383902b1/bin:$PATH LD_LIBRARY_PATH=/mnt/datas/sake/hadk/prebuilts/clang/host/linux-x86/clang-r383902b1/lib64:$LD_LIBRARY_PATH PATH=/mnt/datas/sake/hadk/prebuilts/gcc/linux-x86/aarch64/aarch64-linux-android-4.9/bin/aarch64-linux-android-/bin:$PATH PATH=/mnt/datas/sake/hadk/prebuilts/tools-lineage/linux-x86/bin:$PATH LD_LIBRARY_PATH=/mnt/datas/sake/hadk/prebuilts/tools-lineage/linux-x86/lib:$LD_LIBRARY_PATH PERL5LIB=/mnt/datas/sake/hadk/prebuilts/tools-lineage/common/perl-base BISON_PKGDATADIR=/mnt/datas/sake/hadk/prebuilts/build-tools/common/bison /mnt/datas/sake/hadk/prebuilts/build-tools/linux-x86/bin/make V=1 -j8 CFLAGS_MODULE="-fno-pic" CPATH="/usr/include:/usr/include/x86_64-linux-gnu" HOSTCFLAGS="-fuse-ld=lld" HOSTLDFLAGS="-L/usr/lib/x86_64-linux-gnu -L/usr/lib64 -fuse-ld=lld" DTC_EXT=/mnt/datas/sake/hadk/prebuilts/misc/linux-x86/dtc/dtc LLVM=1 HOSTCC=/mnt/datas/sake/hadk/prebuilts/clang/host/linux-x86/clang-r383902b/bin/clang HOSTCXX=/mnt/datas/sake/hadk/prebuilts/clang/host/linux-x86/clang-r383902b/bin/clang++ LEX=/mnt/datas/sake/hadk/prebuilts/build-tools/linux-x86/bin/flex YACC=/mnt/datas/sake/hadk/prebuilts/build-tools/linux-x86/bin/bison M4=/mnt/datas/sake/hadk/prebuilts/build-tools/linux-x86/bin/m4 -C kernel/asus/sm8350 O=/mnt/datas/sake/hadk/out/target/product/sake/obj/KERNEL_OBJ ARCH=arm64 CROSS_COMPILE="/mnt/datas/sake/hadk/prebuilts/gcc/linux-x86/aarch64/aarch64-linux-android-4.9/bin/aarch64-linux-android-" CROSS_COMPILE_ARM32="/mnt/datas/sake/hadk/prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/bin/arm-linux-androidkernel-" CROSS_COMPILE_COMPAT="/mnt/datas/sake/hadk/prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/bin/arm-linux-androidkernel-" CLANG_TRIPLE=aarch64-linux-gnu- CC=" clang" Image

and got this

clang -Wp,-MD,arch/arm64/lib/.xor-neon.o.d -nostdinc -isystem /mnt/datas/sake/hadk/prebuilts/clang/host/linux-x86/clang-r383902b1/lib64/clang/11.0.2/include -I/mnt/datas/sake/hadk/kernel/asus/sm8350/arch/arm64/include -I./arch/arm64/include/generated -I/mnt/datas/sake/hadk/kernel/asus/sm8350/include -I./include -I/mnt/datas/sake/hadk/kernel/asus/sm8350/arch/arm64/include/uapi -I./arch/arm64/include/generated/uapi -I/mnt/datas/sake/hadk/kernel/asus/sm8350/include/uapi -I./include/generated/uapi -include /mnt/datas/sake/hadk/kernel/asus/sm8350/include/linux/kconfig.h -include /mnt/datas/sake/hadk/kernel/asus/sm8350/include/linux/compiler_types.h -D__KERNEL__ -mlittle-endian -DKASAN_SHADOW_SCALE_SHIFT=3 -Qunused-arguments -DASUS_USERDEBUG_BUILD=1 -DASUS_SAKE_PROJECT=1 -DASUS_SW_VER=\"_ENG\" -Wall -Wundef -Werror=strict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -fshort-wchar -fno-PIE -Werror=implicit-function-declaration -Werror=implicit-int -Wno-format-security -std=gnu89 -Wno-misleading-indentation --target=aarch64-linux-gnu --prefix=/mnt/datas/sake/hadk/prebuilts/gcc/linux-x86/aarch64/aarch64-linux-android-4.9/bin/aarch64-linux-android- --gcc-toolchain=/mnt/datas/sake/hadk/prebuilts/gcc/linux-x86/aarch64/aarch64-linux-android-4.9 -no-integrated-as -Werror=unknown-warning-option -mno-implicit-float -DCONFIG_AS_LSE=1 -DCONFIG_CC_HAS_K_CONSTRAINT=1 -fno-asynchronous-unwind-tables -mbranch-protection=none -ffixed-x18 -DKASAN_SHADOW_SCALE_SHIFT=3 -fno-delete-null-pointer-checks -Wno-address-of-packed-member -O2 -Wframe-larger-than=2048 -fstack-protector-strong -Wno-format-invalid-specifier -Wno-gnu -Wno-tautological-compare -mno-global-merge -fno-builtin -Wno-unused-const-variable -fno-omit-frame-pointer -fno-optimize-sibling-calls -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang -g -fsanitize=shadow-call-stack -flto=thin -fsplit-lto-unit -fvisibility=default -fsanitize=cfi -fno-sanitize-cfi-canonical-jump-tables -fno-sanitize-blacklist -fsanitize-cfi-cross-dso -Wdeclaration-after-statement -Wvla -Wno-pointer-sign -Wno-array-bounds -fno-strict-overflow -fno-merge-all-constants -fno-stack-check -Werror=date-time -Werror=incompatible-pointer-types -fmacro-prefix-map=/mnt/datas/sake/hadk/kernel/asus/sm8350/= -Wno-initializer-overrides -Wno-format -Wno-sign-compare -Wno-format-zero-length -ffreestanding -I /mnt/datas/sake/hadk/kernel/asus/sm8350/arch/arm64/lib -I ./arch/arm64/lib -DKBUILD_BASENAME='"xor_neon"' -DKBUILD_MODNAME='"xor_neon"' -c -o arch/arm64/lib/xor-neon.o /mnt/datas/sake/hadk/kernel/asus/sm8350/arch/arm64/lib/xor-neon.c

I took the time to diff the two commands online for you to see: one (left) works in ubports, the right one does not in hadk:

Other than path differences, above only these flags seem to be different in the sense that they are present in hadk:

    -flto=thin
    -fsplit-lto-unit
    -fvisibility=default
    -fsanitize=cfi
    -fno-sanitize-cfi-canonical-jump-tables
    -fno-sanitize-blacklist
    -fsanitize-cfi-cross-dso

Last-minute edit: I have changed my host’s stdint.h to use __UINTPTR_TYPE__ and it reads now:
typedef __UINTPTR_TYPE__ uintptr_t;
by sudo vi /usr/include/stdint.h :pulling-hair-out: and build seems to continue now.

2 Likes
#20

Day 10: hybris-boot

So, late in Day 9 I tried these hacks:
Commenting out the typedef from
hadk/prebuilts/clang/host/linux-x86/clang-r383902b1/lib64/clang/11.0.2/include/stdint.h // comment out uintptr_t
and then next:
change my host’s /usr/include/stdint.h to also use the __UINTPTR_TYPE__.

The latter change seems to make the build continue, and I will assume nothing is very broken because it ended up using headers from the host, although I may regret this later:)

Build completed after 22 more minutes. If you add to the initial 15 mins since restarting build from outside ubu-chroot it would be under an hour (if make clean actually cleaned everything). That’s way shorter than the entire Lineage build which was ~5 hours.

Let’s see what got build as imgs:

ls hadk/out/target/product/sake/*.img
boot.img  dtb.img  dtbo.img  hybris-boot.img  hybris-recovery.img  ramdisk.img  ramdisk-recovery.img```

Hmm… no vendor_boot here either. Compare that to lineage

boot.img  dtbo.img  product.img  ramdisk.img           system_ext.img  vbmeta.img         vendor_boot.img
dtb.img   odm.img   qmcs.img     ramdisk-recovery.img  system.img      vbmeta_system.img  vendor.img

Missing vendor_boot is not the only difference. System is missing (which is expected for HADK), vbmeta and vendor too and another thing called qmcs. This may mean that the HADK may whitelist only a couple of targets and maybe the vendor_boot can still be built.
But let’s look inside the boot images:
The boot.img is similarily sized as the Lineage one, 96M.
hybris-boot.img however, and hybris-recovery, both have ‘only’ 46M.

The difference is that the hybris- ones pack a minimal filesystem for init, instead of a recovery as on my asus/Lineage boot. What I don’t like in that init script (which is readable, shell) is that I see a define DATA_PARTITION=/dev/block-dm11. Later in that script it is used to mount $DATA_PARTITION /data.

This is a failure of my mount point fixups. I do remember that I didn’t figure out the block-dm partitions, because I assume they are from the super-partition. But /data to be there is a bit surprising. Let’s double-check through adb on the (still running, original ASUS firmware) phone. (Btw, did I mention it uptimed almost 8 days, on cellular, waiting for me to resume my flashing;).

First, inspecting on device mount command, /data/ is /dev/block/dm-10, not 11. Actually there are many mount points with from that dm-10, all type f2fs, and with block device major:253, minor:N (where N=10 for dm-10). Unfortunately, I don’t have access to /proc/devices to see what 253 is, but is probably related to the dynamic partitions we talked about before.
Also, with ls -l /dev/block/platform/*/*/by-name/ from HADK, I do get userdata -> /dev/block/sda23.
Maybe I should flash lineage one more time to get adb root acces - I remember last time I had problems with not being able to boot twrp, which in turn would not allow me to flash the lineage zip…

Let’s take a look at the super.img with the tools from parse-android-dynparts
Building them as per their instruction, then simg2img super.img super-raw.ing and running parse-android-dynparts super-raw.img gives:

dynpart-system_a,,,ro,0 7010296 linear /mnt/datas/sake/debugging/asus/super-raw.img 2048;
dynpart-system_ext_a,,,ro,0 470248 linear /mnt/datas/sake/debugging/asus/super-raw.img 7012352;
dynpart-product_a,,,ro,0 2604744 linear /mnt/datas/sake/debugging/asus/super-raw.img 7483392;
dynpart-vendor_a,,,ro,0 2521232 linear /mnt/datas/sake/debugging/asus/super-raw.img 10088448;
dynpart-odm_a,,,ro,0 2176 linear /mnt/datas/sake/debugging/asus/super-raw.img 12611584

Which clearly means that /data is NOT one of those partitions. It may be that ASUS is using another dynamic partition like inside dm-10, but apart from dm-10 only these appear in mount output through adb (and they correspond to the above 5 partitions).

/dev/block/dm-5 on / type ext4 (ro,seclabel,relatime,discard)
/dev/block/dm-6 on /system_ext type ext4 (ro,seclabel,relatime,discard)
/dev/block/dm-7 on /product type ext4 (ro,seclabel,relatime,discard)
/dev/block/dm-8 on /vendor type ext4 (ro,seclabel,relatime,discard)
/dev/block/dm-9 on /odm type ext4 (ro,seclabel,relatime,discard)

Maybe dm-0 through dm-4 are the same 5 above partitions, but on the other slot.
But clearly dm-10 is nowhere to be seen.
Need to flash Lineage back to get more info.

Let’s restart to bootloader. I cannot adb shell there, no device (only fastboot access).
Then select recovery mode. I cannot adb shell here either, device unauthorised.
Let’s go to fastbootD mode. Nope, no device.
Let’s try to boot twrp. sudo fastboot boot twrp.img results in ‘Unrecognized command boot’. So I need plain bootloader for this.
sudo fastboot boot twrp.img this time works. I don’t know what happened one week before…

In twrp, /data shows as dm-5 :slight_smile: So is /sdcard.
This time, from /proc/devices (I have adb root access here) I can see that major 253 is device-mapper
Unfortunately, I don’t have dmctl to actually see where dm-5 points (this is just a guess, the command existed in ASUS rom but I was not root)
There’s a device /dev/device-mapper that can’t be ‘cat’ to stdout
There is a blockdev command though.

Let’s try to mount sda23 to see how it looks like.
mount -t ext4 /dev/block/sda23 /userdata → Device or resource busy. Same with f2fs
lsof /dev/block/sda23 → nothing. lsof /dev/block/by-name/userdata neither.
There’s got to be a way to find out what’s on that partition.

# xxd -l 512 /dev/block/sda23

gibberish
00000000: fa1e c4f1 d7df 5986 38e4 cb81 6e51 90e7  ......Y.8...nQ..
00000010: abff 69aa 2d01 4b7a 7053 ad91 d362 0afb  ..i.-.KzpS...b..
00000020: b4e5 fcd2 6ea9 dd65 9413 caad a4ea fbc3  ....n..e........
00000030: 7417 6b87 98f3 93a4 d2f3 fac6 72b3 693b  t.k.........r.i;
00000040: 456c ccf4 d4ec fb5e e462 2a58 b86d d4d7  El.....^.b*X.m..
00000050: c798 fc57 11b9 ff34 0a46 808d 4ead 565a  ...W...4.F..N.VZ
00000060: 3d0c f297 3b62 91f3 654c 710a 1ae9 6783  =...;b..eLq...g.
00000070: 13e5 f93d 8518 6398 791a 8694 649d 79ca  ...=..c.y...d.y.
00000080: 4c77 bdef 3ef6 a484 ff39 0eaf 0f24 dd9d  Lw..>....9...$..
00000090: bd91 0832 f9d9 aaa1 3dfe b5eb cabc 2cb9  ...2....=.....,.
000000a0: bf71 fa57 b901 ef82 dd7d b5c5 cf3e dd24  .q.W.....}...>.$
000000b0: a7fb 9d8d 7b3e 1324 4cf5 fc48 f6ff eabd  ....{>.$L..H....
000000c0: f0b9 46f8 d709 d9e2 657c 4df8 8603 5749  ..F.....e|M...WI
000000d0: 7961 8362 a145 81b0 b55d e3cd ca66 e404  ya.b.E...]...f..
000000e0: 3e98 fb34 0fe3 96d0 7996 d9ee 902c 0022  >..4....y....,."
000000f0: d6e4 9433 be16 7b00 967f 0431 d7bd be8e  ...3..{....1....
00000100: 5962 f9a5 6225 f781 5d75 85fa cc32 ef9d  Yb..b%..]u...2..
00000110: afe2 cbec c2f5 129c 3043 4adf 27a2 4d21  ........0CJ.'.M!
00000120: bef0 c841 4d7d bea7 dcff fced 4f4a 4a2a  ...AM}......OJJ*
00000130: 35e3 a89f ac9c a39e 1fb8 adb9 0fab 4f1e  5.............O.
00000140: 626b 9fab 84a0 9f4f f5ac e492 36e5 1494  bk.....O....6...
00000150: 84cb 66a4 c0f2 8808 6186 983c db98 df7e  ..f.....a..<...~
00000160: 0c5f c638 74a4 8079 3d2d 9d52 98bd 92bd  ._.8t..y=-.R....
00000170: bef4 0b96 dfa3 1d89 c4d1 d184 971a a5e2  ................
00000180: 7acb ca90 4cd5 4de9 c386 b21d e9b5 6b2f  z...L.M.......k/
00000190: 6c78 052c 6cc1 1b0c ea4c eab0 3c57 72dd  lx.,l....L..v.....|u..KZ.
000001f0: 3c48 ff14 5b03 a98b 75a8 2f14 b2a2 d197  <H..[...u./.....

Meh. Let’s flash Lineage though, since we got twrp.
Using twrp to wipe data → run mount again.
/dev/block/sda23 on /data type f2fs
Ta-da. This should go into our fixup mountpoints.

Being more confident that sda23 is the right partition, I am re-doing my fixups with the whole info from ls -l /dev/block/platform/*/*/by-name/ and starting a new hybris-hal boot img build.

I log into adb root onto Lineage in the meantime.
/data is again mounted on dm-5
dmctl is now available and am now root:

# dmctl list devices                                                                                                                                 
Available Device Mapper Devices:
system_ext_b         : 253:3
userdata             : 253:5
odm_b                : 253:0
system_b             : 253:2
vendor_b             : 253:4
product_b            : 253:1

As you can see, 5 is userdata. However I don’t get how the mapping is done to sda23:

# dmctl info userdata
device        : userdata
active        : true
access        : rw 
activeTable   : true
inactiveTable : false
bufferFull    : false
# dmctl getpath userdata
/dev/block/dm-5
# dmctl table userdata
Targets in the device-mapper table for userdata:
0-476106392: default-key, aes-xts-plain64 - 0 259:7 0 4 allow_discards sector_size:4096 iv_large_sectors wrappedkey_v0
# dmctl status userdata
Targets in the device-mapper table for userdata:
0-476106392: default-key

Anyways, hybris-boot now correctly uses sda23, after another 23mins build:)
Let’s flash just this on the boot partition and see if we get telnet.
If somehow we don’t I’m going to flash ASUS’s vendor_boot because it has a GZipped ramdisk too, unlike Lineage’s…

sudo fastboot flash boot hadk/hybris-boot.img in fastbootd mode. Reboot. Stuck at ASUS logo.
Vol-Dn and power, Enter bootloader, recovery mode - Stuck at ASUS logo. :facepalm: of course, recovery was on boot ramdisk before!
Let’s flash vendor_boot from ASUS in bootloader mode.
sudo fastboot flash vendor_boot asus/vendor_boot.img
Hmm… no telnet either. Let’s get back to Lineage.
sudo fastboot flash boot lineage/boot.img
sudo fastboot flash vendor_boot lineage/vendor_boot.img
Ok, It boots/works.

So, what now? We don’t have the vendor_boot generated by the HADK way either (I could look up the hybris patches / hybris-hal target to see why).
Or I could try again with wiping the userdata partition from lineage recovery.
Or I should just use the next day to ask on #sailfishos-porters for help with telnet.

2 Likes