Maybe you misunderstand; pull requests are how you constructively ask for adding new features to existing software on git projects, if not all VCS.
2 Likes
In the very best case, you’ve fixed a bug (or more), diligently documented that, and / or and added a well thought out feature that you’ve clearly explained. Well, one could go on, but, I just made a merge request which disguises a support request. It’s complicated enough, and the person on the other side knows me, so I can probably get away with it, but, it is borderline. I’m really lucky to get PRs which 99.99% of the time simply improve stuff I’m responsible for. I’m very thankful for that.
2 Likes
It is a matter of role separations. Not about feasibility or lack of will to help.
If you don’t want to create pull requests, just say so, nobody is forcing you. But we do not have fixed roles in this users community, it is all voluntarily cherry picking. If you want to help with documentation, you are welcome. Same applies for pull requests or creating apps. And your “role” (whatever you envision as your role) is already ambiguous btw, posting howto’s for beginners and asking for features, but also submitting patches for patchmanager.
2 Likes
Who did the software should take care of documenting and maintaining it because the same time that I spend to understand, documenting and fixing others people code, I can deliver a better solution or simply switch to a better product. Which most probably is what many did before me.
UPDATE #1
First of all, I wish to engage this
We always play roles: we are the father/mother, the friend, the boy/girlfriend, the son/daughter, the manager or the employee. Our lives are about the roles but this roles are something like a jail - a box in which society force to stay in. It is wonderful that a community of free people do not feel the need to fit in a box-schema.
When we approach a situation or a product or a software, we can play many roles: nerd hacker, cyber-security expert, marketing & sales, business owner, product manager, project manager, end-user, advanced user, social engineer, social manager, troll, etc.
This two way of playing roles are completely different.
The first way is about constrains and expectations that OTHERS put over us and we need to conform to them in a way or another. The second way is to put ourselves in the shoes of someone else, someone different from us without being or became someone else. Like playing Dungeon and Dragons, never made anyone being or became an elf or a wizard. However to play that roles, we need to know in deep what’s about that role (set of values) and it is not.
A bit of philosophy - Aristotle wrote, “It is the mark of an educated mind to be able to entertain a thought without accepting it.” Being able to look at & evaluate different values without necessarily adopting them is perhaps the central skill required in changing one’s own life in a meaningful way. (Source: Aristotele cited and commented on Twitter)
Few people can play multiples roles at the same time because few people are acknowledged about that roles enough to play that roles. In fact, companies usually tend to create multi-cultural heterogeneous teams because the complexity of the modern-world products cannot be faced just by 1-single PoV approach or by segmenting the design-to-delivery among separated in-line departments.
This is the reason because community-based products are usually the winner on the long terms because in the Bazaar there is many PoV. Unfortunately, this is not always true. Some company teams are better than some community-bazaars and viceversa.
end of philosophy
Now, it is my time to explain the quick answer I wrote here before going to take care of my real life and then returning back.
Settings:System → Patchmanger:Settings → Activate enabled Patches when booting
Why activating at boot time instead of made them permanent with that option?
This question is extremely relevant - not only for the consequences that brings in terms of constrains - but also because it is a design choice. This design choice should be explained in details into the documentation otherwise this software have to be re-design from scratch.
No, I cannot make a pull request to fix this because this is an information that WHO designed the software should explain. It could be a very good reason/decision or it was a good reason/decision at that time, e.g. SFOS 2.x but not anymore.
This is the reason because this design choices should be documented, they aging and they are still impacting despite aging. The temporary workaroud became the product and the product became the legacy. We need to stop this before even it begins, documenting it.
What this has to do with roles? Unless someone plays the product and project manager role in the community those PoVs are missing and in fact - AFAIK - the part of the documentation which refers to this design choice is missing.
end of theory
We evaluate a change betwee “activate at boot time” and “keep persistent”.
We can do a confrontation:
- persistence is easier because we “patch & forget”
- forgetting is not a good practice therefore checking
- checking but when?
- every time
Patch Managerpage is shown - how we can implement this check?
-
pach -Rp0 --dry-runcan fail or not - is it quick enough?
- yes → done
Here we are, we can have a Persistent Patch Manager with a little of changes. Now, it is your time to play roles or simply express yourself. What good can provide persistence and what problems can cause?
end of doing better
UPDATE #2
Why I am pedanticly collecting this suggestions in the Quick Start Guide. Because one day someone will arrive and with social engineering that will please you, will convince you to do these things or equivalent ones. In the lucky scenario otherwise not. However, the main way is the hard way: community-bazaar should learn how to play roles.
Vague philosophy lessons we can do without : check ✓
Very pedantic tone towards others : check ✓
Walls of text changed over and over again : check ✓
Redirect your “answer” to another subject in the same post : check ✓
Despite similar remarks by others I fail to see any change, and giving up hope on any self-reflection. All I can do is give you no more ammunition, so this will the last reaction you will get from me.
6 Likes
Yes, please go ahead with either of these two options; pick the one which suits you best.
This design choice should be explained in details into the documentation otherwise this software have to be re-design from scratch. […] this is an information that WHO designed the software should explain.
Neither you or anyone else is in the position to tell somebody here what should or has to be done. You may make suggestions and try to convince people (though a staccato of half-baked ideas is no very convincing), but basically it is “DIY or leave it”.
We need to stop this before even it begins, …
I wonder who comprises the “we” you keep recurring to: Do you have multiple personalities? If so, this is fine, because then each “we” means “I”; otherwise, I can only reiterate “Neither you or anyone else is in the position to tell somebody here what should or has to be done.”.
5 Likes
Just I did, moving forward for both of them. About the better solution:
- I show that it is easy and feasible to patch the filesystem (files and directory) without creating links to a temporary directory
- the
Patch Managercan move easily from “apply at boot time” in “persistent mode” with check by--dry-runoption which probably is just implemented because currently thePatch Manageris able to detect when a patched file is changed - avoid that
Patch Managerremoves patches when the system is asked to go down for shutdown or reboot
In particular about the point #3, I have tested with success and satisfaction a killall -9 patchmanager. Obviously this would not provide persistence because /tmp/patchmanager is volatile. Now, I have to make another test based on information collected with find /tmp/patchmanager -type f.
The test will be similar to the shell script code I presented here:
- collect the list of files using
find - backup all the system files when all patches are disabled (original versions) which probably is not necessary because it is reasonable that they are stored somewhere
- kill the
patchmanger - use the list of files to remove the links and replace with real files
- start again the
patchmanagerto check how is going to behave - do a system reboot instead of point #4
Some tests, just before going to edit the two scripts that apply patches and one in perl and another in shell script.
After that, I will probably discover the SFOS ill-design choice that constrain the Patch Manager to act volatile instead of providing persistence. Or in a lucky scenario, I will simple discover that volatile for Patch Manager is not a constrain (or not anymore).
In both cases the result will be a lot of fun. 
UPDATE
About the point #2, checking the /tmp/patchmanager3/patchmanager.log I found that the check with patch -Rp0 --dry-run is exactly what Patch Manager does to check that each enabled patch is applied correctly.
If i remember correctly patchmanager in the old days did modify the original files directly. Back then a patch could brake your device, for example lipstick doesn’t start anymore after a reboot. Or not unapllying all patches before a system upgrade could break the system. Then patchmanager switched to the currently used solution.
Please stop whining around, because of your dnsmasq patch. Only a few people which use SFOS are using PM and alot less are using dnsmasq. Make a RPM,make a script, make a patchmanager fork or something else, but please stop spamming the forum with total offtopic posts about your view of the world.
6 Likes
This can be a real issue, thanks for having highlighted it.
It can be solved in another way without renouncing to have a system configuration manager. This is the reason because since the first comment in this thread in which I have included some code, I suggested making a system backup of each patch.
How to revert the backup in case of system failure? It can be done with a userland watchdog (which is missing) and dsmetool seems a useful piece of software to deal with a controlled reboot after the watchdog expired. Smarter solutions might also arise.
Also this can be a real issue thanks to highlighting it.
This means that the upgrade procedure should be better implemented. After all, there is the command to uninstall all the patches immediately available: patchmanager --unapply-all which deactivates and disables (unapply) all patches.
Despite the PM patches, the SFOS upgrade can fail because the user made some kind of changes including installing RPM packages. Before the upgrade, a procedure can be created for doing a backup of the system and the recovery procedure above will also solve the system upgrade failure reverting the system to the previous state.
Sorry, here you missed the whole point but I can understand you because I wrote a lot about PM nextgen. Therefore a little recap is necessary:
-
The
dnsmasqconnmanintegration is a test (about different PoV) because the DNS Alternative is considered the way to go and it contains dnsmasq. -
The correct approach is to fix the
dnsmasqandconnmanRPMs and this is clearly stated into the patch description. Also this information should be considered acquired and accepted. -
The Quick Start Guide diverged from its original aim and started to collect parts that IMHO need to be fixed. Progressively with the time, it moved from being a “end-user” guide to a “product-manager” guide. Thus, it still reports my patch because it is a corner-case.
-
There is a huge gap between patch applications (even with a little of scripting support) and RPMs packaging because the RPMs repository - the sum of all repositories of all contributors - should NOT make a SFOS upgrade procedure fail. As you can read in the forum, the upgrade failure seems normal, instead. Or at leask a not so rare incident.
-
The
SFOScompletely lacks a system configuration manager which is an indispensable tool to have for developing a fleet management tool. The two should completely solve the problem of upgrade failures otherwise they are not good enough for that role. But they can do much more than this.
BTW the main question is: why should a community care modding the SFOS in such a manner that can support a system configuration manager and a fleet management tool?
The first and straightforward answer: a safe and friendly relationship with upgrades but there is much more related to these two tools which are missing because the SFOS is not designed to support them. That much more is also about Jolla profitability therefore Jolla should be much more interested in these design-changes than the community.
After all, unless people here wish to follow a strict policy about RPMs repository like Debian, Ubuntu, RedHat, SuSE, etc are doing which brings a lot of top-down organised work (debian is a non-profit foundation, in fact), then those two tools are the solely way to go in order to obtain something equivalent or at least, a restore system to the last working configuration.
By the ZFS has filesystem snapshots for this purpose but it is not the right approach for dealing with a fleet of IoT devices. It is tailored for servers even desktop can leverage it with some important constraints (user data, for example).
Finally: am I going to change the Patch Manager in order to make it a system configuration manager. I do not think so. Since the beginning, I have been thinking about another completely different solution, much more flexible. But it would be a shame to not learn from what has been done and learning by doing is the best way. Doing strange things under your PoV but they seem strange exactly because they are challenging the current system constraints.
UPDATE #1
This is a patch which probably will not work because I saw that Patch Manager runs in a jail and reasonably with user-privileges and not root-privileges (update: it works, after a reboot also). Despite the privileges, it is still a proof-of-concept rather than a definitive solution.
--- /usr/libexec/pm_unapply
+++ /usr/libexec/pm_unapply
@@ -25,7 +25,8 @@
PATCH_EDITED_NAME="unified_diff_${SYS_BITNESS}bit.patch"
PATCH_EDITED_BACKUP="$PM_PATCH_BACKUP_DIR"/"$PATCH_EDITED_NAME"
-ROOT_DIR="/tmp/patchmanager"
+ROOT_DIR="/"
+TMP_ROOT_DIR="/tmp/patchmanager"
# Applications
PATCH_EXEC="/usr/bin/patch"
@@ -66,6 +67,7 @@
exit 0
}
+files=""
verify_text_patch() {
if [ -f "$PATCH_FILE" ]; then
log
@@ -74,6 +76,12 @@
log "----------------------------------"
log
+ files=$(/bin/sed -ne "s,^+++ *\.*[^/]*\([^ ]*\).*,/\\1,p" "$PATCH_PATH")
+ for i in $files; do
+ [ -L "$i" ] && /bin/rm "$i"
+ [ -f "$TMP_ROOT_DIR/$i" ] && /bin/cp -arf "$TMP_ROOT_DIR/$i" "$i"
+ done
+
$PATCH_EXEC -R -p 1 -d "$ROOT_DIR" --dry-run < "$PATCH_FILE" 2>&1 | tee -a "$PM_LOG_FILE"
fi
}
@@ -87,6 +95,7 @@
log
$PATCH_EXEC -R -p 1 -d "$ROOT_DIR" --no-backup-if-mismatch < "$PATCH_FILE" 2>&1 | tee -a "$PM_LOG_FILE"
+ for i in $files; do [ -s "$i" ] || /bin/rm -f "$i"; done
fi
}
--- /usr/libexec/pm_apply
+++ /usr/libexec/pm_apply
@@ -29,7 +29,7 @@
source /etc/patchmanager/manglelist.conf
fi
-ROOT_DIR="/tmp/patchmanager"
+ROOT_DIR="/"
# Applications
PATCH_EXEC="/usr/bin/patch"
@@ -69,6 +69,13 @@
log "Test if already applied patch"
log "----------------------------------"
log
+
+ files=$(/bin/sed -ne "s,^+++ *\.*[^/]*\([^ ]*\).*,/\\1,p" "$PATCH_PATH")
+ for i in $files; do
+ if [ -L "$i" ]; then
+ /bin/rm "$i" && /bin/touch "$i"
+ fi
+ done
$PATCH_EXEC -R -p 1 -d "$ROOT_DIR" --dry-run < "$PATCH_PATH" 2>&1 | tee -a "$PM_LOG_FILE"
UPDATE #2
Instead of the current version of Patch Manager, I forked it from its github repository. Today with seven patches
- 8 patches on devel branch with an unified view (slightly tested code)
-
patched files ready to install in a tarball (
devel-su curl $link | tar xvz -C /)
I have unified the pm_apply and pm_unapply shells script in a single one pm_patch.env because most of the code was redundant.
-
pm_applydoessource pm_patch.env apply "$@" -
pm_unapplydoessource pm_patch.env unapply "$@"
This would help to maintain such shell script code in the future.
Yes, it is hard to observe wasting a years of work of several people and inventing a wheel once again.
2 Likes
About 10 years of works but there is still area of improvement about the code redundancy reduction, for example:
- 8 patches on devel branch with an unified view (slightly tested code)
-
patched files ready to install in a tarball (
devel-su curl $link | tar xvz -C /)
Possibly, I missed something: SFOS has a system configuration manager or Jolla has a fleet management system that I am not acknowledged? Don’t be shy: yes or no, might be enough.
Assuming you mean MDM which is the lets say more common term than “Fleet Management”, then yes.
2 Likes
You are devolving Patchmanager with this plan.
This is how it has worked historically (PM2.x), which resulted in bad, bad times at times.
I recommend looking into src/preload/src/preloadpatchmanager.c and try to understand what this does and why, and its connection to the perceived shortcoming of the current implementation.
You seem to have so far missed to this core piece of Patchmanager functionality.
If you want to go back in time though, there is a fork called “pm2-forever” in this repo which as far as I understand is dedicated to the old ways.
2 Likes
Go ahead and improve, the code is publicly available.
1 Like
Just, I did with 23 commits.
Powerful tools used in a wrong manner are a disgrace. The solution adopted was to constrain into a jail a potentially configuration manager tool because was not ready and safe enough to configure the whole system but just the UI.
Some few of example
Using the patch without -r /dev/null the .rej file break the restart of a daemon because that file can be read as part of a cond.d folder and mess-up everything. In fact, it did preventing the daemon to load - it is also a daemon fault that should ignore .origin, .rej, .rpmold and .rpmnew files:
A simple parsing rule for the patch file path was implemented in C++ code with a couple of loop, here:
Here below a succulent corner case:
In this project the patches v0.0.1 and v0.0.2 can be applied multiple times. While the v0.0.3 cannot. You may claim that it is irrelevant. It is not - It going to say that there is not a simple testing tool that check and test the patches in order to validate them.
About a tool to test/validate the patches and to rescue the systems from their application, I just wrote it but not published yet because it is in its early version. I wrote it because, I broke the system - but instead of re-flashing - I did a rescue tool which is a shell script that can easily replace the PM from command line and patching the system. It is far a away to be complete but good enough to let me to rescue my system from the mess that PM did because it does do what is supposed to (or it should do).
So, I confirm you that PM working at system level quickly tends to mess-up the system. However, this should be seen as a limitation not a feature. In fact, it has been reported that PM3 was developed with the purpose of to limit its functionality to the UI. I can live with that as long as and as far as, there is an alternative way to do a system configuration management with templates or a set of patches.
Lesson Learned
If we ask a fish to climb a tree and a monkey to swim, both will show poor performances.
To develop a safe and functional configuration manager tool, it is request a system administration united with an embedded system experience plus good skills in scripting, preferably bash/ash/sh with sed/awk (regular expressions) because perl is a write-only language (fast to write and easy to rewrite, again again again).
Instead, technologies like C++, d-bus and Qt5 are for the UI level but it would be better that higher level will not take decisions about how to deal with the system configuration but just help the end-user to choose the configuration s/he prefers.
This separation between the UI (user interface) and the BL (business logic) is especially true and useful for a system configuration management tool which SHOULD be available in a shell or a terminal. Otherwise, it will be unavailable when the UI is not available and this would be unacceptable which is the reason because PM2 → PM3 constrained the Patch Manager in such a way that it can apply just volatile changes.
If we put all the information and skills and experience together, everything make sense, and all the PoV fits in the same puzzle. It was not the PM that need to be jailed but it was needed a clear separation between the UI level and the system level. A clear separation about technologies involved, implementation and design (in the reverse order: design → implementation → technologies).
I hope this help for the future.
SYSTEM PATCH MANAGER
You can skip the introductory sections and jump directly to the end where the technical stuff is presented. In case you like that, you my came back here to read the premises.
Rationale
A system patch manager should be able to record the changes that are affecting the system and revert / apply them in a fail-safe way. This means that it SHOULD work from a console and it should work from a recovery image with minimum additional requisites, possibly none.
Adding a full scripting capability is a little more complex task than including such script into a package and running them because shell script can do arbitrary things and in some corner cases (and failures) can do untracked / unexpected things that cannot be reverted with counter-acting script (eg. post-install.sh → pre-remove.sh).
This is true also for RPM packages - mitigated by the fact that usually packages maintainers have a system test facility that allows them to validate the RPM packages for a wide variety of scenarios and different system configurations - plus - mitigated by the fact that RPM system is a well established way of installing / removing software which is in production since 1997.
Why do not use RPM to patch the system?
Because the RPM is a system to install software not to apply patches and do traceable system configuration changes. For sure some tools for this aim, exist out of there. Yes, they are named configuration management system. Usually they are tailored for a specific system or highly configurable for a class of system (eg. GNU/Linux distributions). The odd is that in SFOS end-users are used to use Patch Manager from the UI and a kind of integration with it would be easier to introduce them into this new dimension.
Moreover, there are many ways to track unexpected system changes, also. In particular inotify approach but can be a little tricky to use on a living system even if it will be probably a long term general solution or alternatively a filesystem that can provide a standard reliable changes log or snapshots by design.
Apply patches to the system can brick it
Breaking the system in making experiments with SFOS, which is equivalent to using it as any other GNU/Linux system as much as it can be considered similar, it is the main reason because you are using SFOS because there are no really other reason apart fairly tales and a matter of personal taste.
Backup before brick your root filesystem.
Technical approach
Therefore developing shell scripts that can be integrated with a slightly modified version of the current Patch Manager is the main way and storing the system patches into Web Catalog is also a good option to go, naturally.
Instead of start adding a generic and arbitrary shell scripting capability, I focused on system configurations and their system services. For this aim adding a header to the patch seems a reasonable easy way to cope with the most interesting and common cases.
This is a testing patch that is going for that way:
Here below an example of such a header for testing purposes:
#!/bin/bash ## this line for a patch which header is a shell script ############
##
## (C) 2023, Roberto A. Foglietta <roberto.foglietta@gmail.com>
## released under MIT (tlo.mit.edu) license terms
##
## This is a system patch header example for SailFish OS system patch manager
##
#\header #######################################################################
#
# type: system
# target: sfos
# services: dnsmasq connman
#
## optional fields using tabs instead of spaces just for test ##################
#
# name: dnsmasq-connman-integration # a comment
# provider: robang74 # another comment
# version: 0.1.1 # yes another one
#
## a repetition, a variant and an unrecognised fields just for test ############
#
# name : dnsmasq-connman-integration-not-fit # this should raise a warning
# services : dnsmasq, connman;
# string : "hello world"
#
#/header #######################################################################
#
## put the shell script body between this line and the --- end of header mark ##
---
Just a set of essential information which the vital part is:
services: dnsmasq connman
because everything else is for separate the volatile UI from the permanent system patches and to separate the application of those patches between the SFOS and your laptop/PC GNU/Linux system.
After all, the beauty of such approach is that can be used also for every GNU/Linux system and every others system which is reasonably similar and provides a shell compatible scripting environment.
Where have you found an RPM package that patching system files? As far as i know nobody did realise such crazy idea.
1 Like
I agree but it was the critic that I received in this forum.
Well, to be 100% fair - I got that critics from github by a member of this forum indicating me the documentation of the Patch Manager