Well, LMK (kernel module lowmemorykiller) use correct metrics IMHO, its logic is just not ideal - it may kill multiple apps in short interval. It would be nicer to kill one and wait for metrics update instead…
But MCE is using wrong metric. It is using cgroup’s memory.usage_in_bytes and it contains reclaimable memory.
I am not complaining about LMK, it does its job.
I am pointing out 2 things:
MCE doesn’t send any memory warning. And it definitely shouldLMK lmkd, systemd-oomd, etc. All of them don’t count the cache, only how much memory can be quickly used by apps if needed. Meanwhile MCE is the only one that uses a different metric, one that counts cache too. In my opinion it could be investigated if something that other methods use could be used here too, for more consistency and arguably better reliability. Also setting warning levels would be easier too, because it directly corresponds to used RAM, and not RAM + Cache that can also be in SWAP.Edit: Also, even in that community meeting @Thaodan mentioned, Jolla confirmed the metric used is wrong:
- 2. We agree with your conclusions, yes, but haven’t yet come to a conclusion about what API is going to work best. (flypig, 08:20:28)
So I created this bug report to remind them of that fact, that this problem still persists.
I’ve talked to @spiiroin about this issue and indeed the metric used in MCE is wrong.
It does send a warning through a DBUS brodcast thou.
The metric used both in the lmk module and MCE are differrent but brittle for different reasons.
From my personal pov the kernel PSI api is the only replacement but this requires the move from cgroup hybrid hierarchy to unified cgroup hierarchy.
That would make a lot of sense, as this is exactly the same as what systemd-oomd is using.
But I get it won’t be easy to implement it. Hope the source code for that can help, maybe it could be beased on systemd-oomd's oomd_mem_available_below for example?
That’s the idea I think.
The best help is to test unified cgroups, fix all the bugs and then proceed.
I am not very experienced with cgroups. Can someone explain to me what unified cgroups means and why it is needed? Why not to use system-wide memory pressure (PSI) information from /proc/pressure/memory?
Also could you please tell us what is a good way to monitor when MCE sends some information on DBUS? systemctl, dmesg?
There is a lot of experimenting going on by multiple people on Tuning the oom killer / low memory killer, I would love to experiment a bit and see how it behaves.
I had an oom trigger today, and I didn’t see anyhing in dmesg at least.
I think dbus-monitor or gdbus monitor, just make sure to only listen to mce or there will be a lot of spam.
Thank you. The qdbus commands from Mce | Sailfish OS Documentation are no longer working, but dbus-send and dbus-monitor commands seem to be working. It is really hard to trigger memory warning, wasn’t able to trigger them yet.
Hello guys. I was experimenting with PSI api a bit. When I try user-space example from PSI - Pressure Stall Information — The Linux Kernel documentation on x86, Linux 5.15 (with lesser threshold, 10ms), it works fine. But I am not getting any notification on Sailfish OS kernel 4.14 on Xperia 10 II. Even when I am trying different thresholds and monitoring windows. Do you have an idea why?
This feature got merged into mainline with kernel 4.20. But maybe it got into Android kernel earlier?
Because cat /proc/pressure/memory exists on my 10 III with 4.19 kernel. Does this directory exist on your device too? Then it is indeed weird if it is not fully functional.
Could it be that lkm and lkmd are so agressive you can’t get to low enough free RAM values for PSI to detect memory pressure? After all on desktop you don’t have anything like that.
Even when I lower minfree levels the lowest available value I could see with free -m is 500MB.
Yes, I have /proc/pressure on Xperia 10 II. The api allows to setup own monitoring limits, it work like that:
const char trig[] = "some 10000 500000";
struct pollfd fds;
fds.fd = open("/proc/pressure/memory", O_RDWR | O_NONBLOCK);
fds.events = POLLPRI;
write(fds.fd, trig, strlen(trig) + 1);
printf("waiting for events...\n");
while (1) {
n = poll(&fds, 1, -1);
...
}
The string some 10000 500000 means that application should be notified when there is stall 10ms in 500ms watching window. But on Sailfish OS, it never return anything from poll, even when periodic watching of memory file shows stalls more than 2%.
PSI introduced in 4.20 got backported into Android’s 4.9, 4.14 and 4.19. I tried to dig through Xperia’s configuration as well as tried to see if its implementation changed in any way in newer kernel. But it seems to all be unchanged and it should work the same as on modern x86 kernels. So I have no idea why it would work different in any way.
I think it is worth looking on https://source.android.com/devices/tech/perf/lmkd code and documentation, as it is using PSI on Android kernels. So if SFOS wants to do it too, it is good idea to copy their way of polling this data.
What must be done, other than a systemd upgrade, to enable systemd-oomd and make it functional?
I do not see lmkd as able to materialize quickly, but the former could be enabled for the Xperia 10 and onwards…
Today my screen would not turn on for 30 seconds due to the kernel OOM killer throwing lipstick under the bus along with all my 6 native apps (on 6G of RAM too…).
What you are talking about is unrelated.
MCE is informing apps that memory is almost full and the should reduce their usage as much as they can, like browser closing background tabs. And here we are discussing how it should measure memory usage/pressure.
Low memory killer is a separate thing, and SFOS devs will need to decide if they want to go with lkmd, systemd-oomd, something else or a custom solution.
But honestly I found that tuning lkm gives good enough effects, at least for me.
Sorry for confusion, I was just used wrong test executable (most likely). PSI test program is working fine.
I start experimenting with PSI usage in MCE. You may watch my mempressure-psi merge request
Mid-2023: we are waiting for Use PSI (Pressure Stall Information) api in mempressure plugin by Karry · Pull Request #14 · sailfishos/mce · GitHub to get attention, I guess?
While we’re waiting, what would be a lightweight tool to continously eyeball 
memory pressumre info in a task switcher card?
Answer is: AIDA64 and Lighthouse both have task switcher card implementations for Memory usage monitoring.
They display different numbers, maybe 10% or so.
Lighthouse is visually more polished, so I’ve been running with that.
Main observation: things get slow and sluggish instantly when Lighthouse shows ~500 MB or less memory available.
It is indeed becoming frustrating needing to keep closing Browser after every use.