A list of hybris-recovery.img shortcomings

robang74 · 7 July 2023 09:23

REPRODUCIBILITY: 100%
OS VERSION: 4.5.0.19
HARDWARE: Xperia 10 II
UI LANGUAGE: English
REGRESSION: no, AFAIK

DESCRIPTION:

recover-main-page

on recovery boot image:

the sshd does not work because root password is not set as supposed to be;
the factory reset does not work because the scripts are not updated for Xperia 10 II;
the fsck is based on Busiybox 1.3.4 which requires its extensions but partially installed because only .minix extension is present which is useless (cfr. UPDATE #1);
notice about telnet 10.42.66.66 is written to small to be read by an average human;
the recovery image is not supposed to stay in recovery mode forever but reboot the standard SFOS after a timeout without receiving the first telnet connection;
even better if the recovery image would NOT boot in recovery mode unless a hardware key is pressed at boot time (volume down key?) or even better unless the phone is connected to an USB cable during the reboot (this makes even more sense).

PRECONDITIONS:

Have flashed into boot_a and boot_b the hybris-recovery.img image.

STEPS TO REPRODUCE:

satisfy the preconditions above
reboot your smartphone
enjoy the recovery mode

EXPECTED RESULT:

Many but few delivered

ACTUAL RESULT:

That image is near to be useless and BTW available only after flashing the boot partitions with fastboot

MODIFICATIONS:

none

ADDITIONAL INFORMATION:

I cannot read this:

but using an ASCII art generator service it would be much easier to read and no particular changes will be required (work smart, not hard).

                                                                   #     #  #####  ######                                        
 ####   ####  #    # #    # ######  ####  #####    #####  #   #    #     # #     # #     #     ####    ##   #####  #      ###### 
#    # #    # ##   # ##   # #      #    #   #      #    #  # #     #     # #       #     #    #    #  #  #  #    # #      #      
#      #    # # #  # # #  # #####  #        #      #####    #      #     #  #####  ######     #      #    # #####  #      #####  
#      #    # #  # # #  # # #      #        #      #    #   #      #     #       # #     #    #      ###### #    # #      #      
#    # #    # #   ## #   ## #      #    #   #      #    #   #      #     # #     # #     #    #    # #    # #    # #      #      
 ####   ####  #    # #    # ######  ####    #      #####    #       #####   #####  ######      ####  #    # #####  ###### ###### 
                                                                                                                                 
                                                #     ###       #        #####       #####   #####       #####   #####  
   ##### ###### #      #    # ###### #####     ##    #   #      #    #  #     #     #     # #     #     #     # #     # 
     #   #      #      ##   # #        #      # #   #     #     #    #        #     #       #           #       #       
     #   #####  #      # #  # #####    #        #   #     #     #    #   #####      ######  ######      ######  ######  
     #   #      #      #  # # #        #        #   #     # ### ####### #       ### #     # #     # ### #     # #     # 
     #   #      #      #   ## #        #        #    #   #  ###      #  #       ### #     # #     # ### #     # #     # 
     #   ###### ###### #    # ######   #      #####   ###   ###      #  ####### ###  #####   #####  ###  #####   #####

This is a rendering readable even in a little format (25%):

About root:recovery login on SSH, it is enough that a system starting script set the password with the passwd command or set the right string into /etc/shadow and /etc/shadow-, just in case.

robang74 · 9 July 2023 08:29

UPDATE #1

There is /bin/e2fsck and it would be nice that it would linked in fsck.ext? as far as possible.

Using strings against the ARM64 binary, it reports that e2fsck belong to e2fsprogs v1.46.5 (changelog) and it supports ext2, ext3 and ext4 filesystems.

Alternatively the fsck should be removed or completed with its fsck.auto in order to have a coherent user/admin interface: who does not find fsck, will search for e2fsck or even better mkfs.auto and mkfs.ext? bring to it.

In the list of supported commands included into /bin/busybox-static, the fsck.auto is missing in 4.015.9 rootfs, boot and recovery images.

GitHub - sailfishos/busybox

Here below the Github project with the config for the SFOS busybox-static and its RPM spec.

robang74 · 9 July 2023 18:35

UPDATE #2

A enhanced version of fsck.auto is here

https://github.com/robang74/fsck.auto/blob/master/fsck.auto

this script can be completed with these links:

cd sbin
ln -sf e2fsck fsck.ext2
ln -sf e2fsck fsck.ext3
ln -sf e2fsck fsck.ext4

This configuration has not been tested, yet.

Test can be done also with telnet 10.42.66.66 then using passwd to set the root password to rescue and then use SSH to transfer the files.

robang74 · 10 July 2023 09:10

UPDATE #3

I have extracted both boot images for the SFOS v4.5.019 an check the differences. Then, I started to mod the recovery one. Here below, some hints:

the text is printed into a single row, there fore the banner approach does not work but an image can be displayed therefore I used my print_banner.sh to create such image
the miniUI on which recovery boot image rely on is on Github (here) and as you can see the only font available is code-cabled with a too tiny dimension 10x18, the best is to change in something like 25x45 for example.
after having changed the root password and reboot again, my PIN code was not valid anymore. I suppose that the root password in /etc/shadow- is used to encode something related to the PIN but correlation is not causation - I have to investigate further. This let me think that the password for root in SSH may be the PIN code, itself.
before consuming the 5 tries for the PIN code, I have re-flashed completely the smartphone. Just in case, to avoid sending to Jolla for the ultimate recovery procedure. Annoying but I can live with that until I will find why.

Here a screenshot of the boot:

Clearly, you can see the difference between the “ciao ciao” text in 10x18 font on the top and the image presentation. Unfortunately, the image will not change in case someone edit the script and change the default IP address. In such a case, s/he should recreate the PNG image but the banner script allows it in a very quick way.

Finally, the image lacks of strace and I wish to add to it because it will be very useful in the near future and for debugging purposes especially because the root filesystem (the one on which the SFOS usually run) is available from the recovery boot image.

robang74 · 11 July 2023 06:58

UPDATE #4

This is image has been created with a 6005 bytes, 121 lines shell not-optimized script.

ip-10.42.66.66.png: PNG image data, 1120 x 240, 1-bit grayscale, non-interlaced

The script creates the text banner with an arbitrary IP address in PBM format and then converts t is a 596 bytes PNG image suitable to be displayed by yamui.

On a running system the scripts takes 2 seconds to complete but on an after-boot system - equivalent to sync; echo 3 > /proc/sys/vm/drop_caches - it takes 5 seconds which is a lot of time¹. However, the image is saved and therefore nor the banner nor the image will be created anymore as long as the IP address will not change. For each IP address the script will create and store a new image without deleting the others stored and immediately available.

The conversion between PBM and PNG requires 172Kb of binaries and libraries in addition. The script can be compressed because it is used only once to export functions and therefore can keep 1770 bytes on disk which are 2kb because the smallest storage granularity is 512 bytes block.

Analysing the yamui code, I think that the best is NOT to add the PBM format to it even if the PBM format is so easy to read that an external library would be excessive but statically linking against that library. Because a PBM image has a disk footprint of 538Kb while the PNG image 589 bytes (1Kb) unless it would be compressed with gzip at 3050 bytes (3kb) which is 3x bigger than PNG format but acceptable.

To integrate the PBMz compressed format in yamui, there are some options:

integrate just the PBM format² statically linked (few kb) or add the libnetpbm.so (150kb) to the system
integrate the zlib support which the library is on the system and required by libpng (216kb) required by yamui
or create a shell script wrapper that provide decompression in /tmp before passing the PBM images to yamui
evaluate to drop the support to PNG in favor of PBMz images

The #3 makes sense only for a developers version in which PBM format is included but not yet the zlib one. About #4, why not? The PNG format is the standard most used for lossless image compression.

Why add the PBMz support? Because the PBM images can be easily creates by ASCII manipulating scripts. Those scripts can use pnmtopng (32kb) and libnetpbm.so to convert PBM in PNG but this conversion is not acceptable for on-the-fly needs. However, on-the-fly-needs can be better satisfied if yamui would keep open a file-descriptor in which a new image can be written to update the current which is not the case³, AFAIK.

The, yamui project cames with some parts and grep -rn "main(" on its git/source top folder tell us that the parts are three yaumi, yamui-screensaverd and yamui-powerkey. Having three separated binaries is not optimal for embedded systems (and a smartphone are a class of embedded system in particular their rescue/boot images). The best is the busybox/toybox approach: one single binary with several links for each functions.

conclusion

The correct approach is to change the yamui code in order to let it support multiple-lines text rendering and using the a print banner shell script that converts a text into a multi-lines banner which take tenth of seconds to run.

The best approach is like the one presented above but with a command line option that allows to rendere the 10x18 (tiny) font multiplicated for a given integer: eg. 2x (20x36, small), 3x (30x54 regular), 4x (40x72, large), …, 6x (60x108, huge), etc.

Again³ there is not a specific opened file descriptor to update this text on-the-fly, AFAIK.

notes

¹ instead of 1120x240 pixels image, a 560x120 would have fit the bill as well but 4x less of time to convert it because 4x less of data (estimation, spoiler: not so straight linear, dear!).
² the pdf417decode project on Github is a language-C example of such integration.
³ the d-bus support can fulfill this short coming on a running system but it is not the best approach for a boot/rescue image not as good as echo goodbye > /run/yamui/update.text.

spiiroin · 11 July 2023 10:10

Couple of quick thought / ideas

Yamui supports externally defined font (no, I’ve not tried if it really works), see gr_init_font() → you could skip the “generate image part” altogether and just render text in big custom font?
There is also vanilla unix socket control point. At the moment it is only used for terminating splashscreen process under circumstances where D-Bus is not (yet) available. But with some care it could be extended for changing things currently running yamui shows on screen. See unix_server_handle_client(), unix_client_terminate_server() & co

spiiroin · 11 July 2023 10:20

Which might allow yet another option that should not even require any code changes: just launch a new yamui process with updated text info - old process lets go of display and exits, new process takes over the display and shows something else.

nephros · 11 July 2023 18:15

Nice.

One more option could be to have a signal handler for e.g. USR1 (or even HUP) causing it to reread whatever it was initialized with.

robang74 · 12 July 2023 11:21

Looking at res_create_alpha_surface(), it is not a font but a PNG image and considering what I saw about the C definition of the font is a table of the ASCII printable characters (96 chars) painted in B/W. This brings us to appreciate the idea of a script that can generate B/W images in PBM format which can be easily converted into PNG. Because, it can be modified in order to create fonts for yamui.

Named pipe, it is a inter-communication process (IPC) standard way of going. Socket is specifically tailored for d-bus communication and kill -HUP for reloading a daemon configuration. BTW, with inotify() there is no need of a signal anymore as long as the write and the read are atomic, which is not a granted constrain in the most general case.

robang74 · 12 July 2023 12:23

UPDATE #5

These two commits have been developed in 2h 20m. The code is completely untested and it never have been compiled yet. Therefore, it should be considered as a proof-of-concept.

Comparing sailfishos:master...robang74:master · sailfishos/yamui · GitHub

@piiroin: I have to take care some stuff about my real-world life. In the meantime, let me know your opinion about this approach / implementation. As soon as, this code will work as expected to then the next step is to integrate the multiple-lines text rendering.

UPDATE #6

I have uploaded into the git repository also the script that prints the banner and created the related PBM and PNG images:

https://github.com/robang74/yamui/blob/master/banner/print_banner.env

This script has been modified since the last tested version and it might fail to run or to run properly. It is just a proof-of-concept by now.

robang74 · 13 July 2023 15:03

UPDATE #7

The commits I did on my yamui fork compile and the results can be downloaded from here:

I did not tested yet. If you do, you will do at your own risk.

robang74 · 15 July 2023 17:35

UPDATE #8

The recovery image, also in 4.5.0.21, have the following shortcomings:

the /bin/bash is missing but can be replaced in this way:

echo '/sbin/busybox-static ash "$@"' >/bin/bash
chmod a+x /bin/bash

the /usr/bin/fsadm is missing and it is needed by lvm resize tools. Moreover, using the one on the /rootfs which is a shell script, it fails with date
the date is an applet from busybox-static:

sfos # busybox-static  date --help 2>&1 | head -n1
BusyBox v1.34.1 (2022-09-21 00:00:00 UTC) multi-call binary.
sfos # busybox-static date -u -d"Jan 01 00:00:01 1970" +%s
date: invalid date 'Jan 01 00:00:01 1970'

but busybox date can deal with such date format, in fact:

pcos # busybox date --help 2>&1 | head -n1
BusyBox v1.30.1 (Ubuntu 1:1.30.1-7ubuntu3) multi-call binary.
pcos # busybox date -u -d"Jan 01 00:00:01 1970" +%s
1

Therefore, it is depend on the options activated into busybox applets config

robang74 · 19 July 2023 04:56

UPDATE #9

There is no any viable way to debug and fix SFOS until these three facilities will be fully functional in place unless you wish to waste your time as you were an immortal highlander.

1st - make working the recovery image because it is the starting point for everyone that seriously wants debugging / fixing their OS and for everyone that wish to experiment with the system but having a quick recovery option. For this reason the recovery boot image should be the standard and only one. Obviously, the recovery boot mode should start ONLY when the users ask for it (e.g.: USB cable connected at boot time could be a way) or when the system is bricked badly (e.g.: the UI cannot rise up, a file-format flag is set after a watchdog expired).
2nd - provide a reliable system patch manager because people who wants debug and fix their OS - included them - need to track down and revert the changes they made on the system otherwise every time is a start from scratch which currently means flash.sh because recovery does not even reset the system to SFOS factory state.
3rd - provide a reliable back-up system to let everyone is debugging and fixing their OS to quickly be back to a previous snapshot for users because the two point above should be a good starting point to deal with the root filesystem.

About backup

The rsync is our friend but a script user-friendly interface is needed and moreover rsync is not available in the userdata image and cannot be installed until the device is registered with Jolla and related repositories are available.

This means that for OS early-boot hackers the rsync is not an immediate option to go for especially because also no any internet access can be available. Therefore a backup / restore script suite to be useful on early-boot stages should necessary relies on tar an gzip because also pigz nor xz are not available nor installable even if xz-libs are installed.

Working in progress

Which is exactly what I am doing by now:

A shell scripts suite for tar/gzip backup/restore has been tested but some more development and tests is needed (cfr. users backup section in the Quick Start Guide.
A system patch manager by shell script that can un/apply permanent patches on rootfs has been used successfully to recover the system by a faulty patch installation but not released, yet
A system patch manager last version patch downloader has been implemented and successfully tested but not released, yet. It creates also a list of patch installed on the system a repository of them for future recovery or removal even in emergency off-line cases and can restart system services or daemon on-demand using a special formatted patch header.

I did not released the current version of these three tools because it is supposed that they should work together and their integration is still in early development stage. For example, the immediately after flashing system configuration shell script did not reach a reasonable maturity level.

Obviously, if - I do - these three facilities, then I do in a way to put myself in the most independent position from Jolla Oy - not necessarily against their interests or profit opportunities - simply I do not put those values in any place of my working TODO list because I am not paid to take care about their business. Which is good for the SFOS community because it brings more freedom for us.

robang74 · 5 October 2023 16:24

This version of yamui solves the problem of the telnet banner too small to be read:

GitHub yamui by robang74

Check among the new options the -m N to resize the embedded font by N times.

This version of statically compiled busybox solves some issues presented above:

GitHub busybox by robang74

The key change is about the config of the busybox.