Ad blocking in SailfishOS in 2023 - What are the options?

[ohnonot]

Thanks for your feedback! It is much appreciated.

• I have changed the article so that the script resides in /usr/local/bin/hostsupdate instead of the user’s home directory, and gave it a better name. It makes more sense this way.
• the necessity of hosts.head being present beforehand (and created manually) is mentioned in the article. I cannot automate ths process without complicated checks and taking the user’s freedom to have a customised hosts file (i.e. the head portion of it) but I added a check to the script that will make it fail if that file is not present.
• Yes some preliminaries are required but tbf, it is possible to do everything on the device, without ssh.

[davidrasch]

I just used your script/timer approach and went with the ad block test site Test Ad Block - Toolz from 9% to 54% of blocked ads with the sailfish browser - pretty nice.

However, for AAS Fennec with ublock I get 96%…

[ilpianista]

I get 87% with Blocky and wally3k’s ticked list

[robang74]

Check this part of this guide, I use these DNS on my laptop and they are doing a pretty well job:

@robang74

SAFE & PRIVACY DNS

About DNS, using those from your network provider (default) is not always the best choice. There are some more interesting alternatives:

• Cloudflare uses 1.1.1.1 and 1.0.0.1 for its DNS service over IPv4 and their name resolution services have some extra features for safety. It claims to be the fastest DNS in the world.
• AdGuard uses 94.140.14.14 and 94.140.15.15 and offers an {advertising, malicious, malware} hosts resolution name blocking feature plus family protections on 94.140.14.15 and 94.140.15.16 but they also offer an unfiltered service (check the link).
• Quad9 uses 9.9.9.9 and 149.112.112.112 to offer a service which a combination between the two above, in their claims (fast and safe).

The best choice is alternating two coherent DNS services together - for reliability - and using a caching system configured for leveraging a large DNS cache to gain speed, privacy and also safety depending the services you choose.

#IPv4 nameservers
nameserver  9.9.9.9 
nameserver  94.140.14.14
nameserver  149.112.112.112
nameserver  94.140.15.15

#IPv6 nameservers
nameserver  2620:fe::fe
nameserver  2a10:50c0::ad1:ff
nameserver  2620:fe::9
nameserver  2a10:50c0::ad2:ff

This is an example of /etc/resolv.conf for every host connected with an IPv4 + IPv6 network which alternates AdGuard and Quad9 ad-blocking and safe-filtered DNS .

Attention : usually the /etc/resolv.conf is not the right place to insert these values.

[ohnonot]

I get very similar results even with the strictest single list.

The developer of this tool also made this one. Not sure what the difference is, but it gives far less flattering results. I could not find any other differentiated & opensourced tools.

Using firebog.net’s meta list combined into one big hosts file I get +80% with both tools.

A slightly stricter µBlock configuration gives me 100% though, and µBlock gets a lot of its lists in AdBlock format. I will try to figure out how to best convert these to hosts format.

Anyhow, I updated my script and uploaded it to a VCS site. To be continued.

[kan_ibal]

Using (DNS alternative ) + DNScrypt-proxy blocklist + Defender II that can be used simultaneously, I get 93% for lists updated in August.

[LSolrac]

is there a way that doesn’t use DNSmasq?

[kan_ibal]

Yes, there is, You can install DNScrypt-proxy and then DNScrypt-proxy blocklist alone.

[ohnonot]

I reworked my hosts-based adblock script based on these insights:

@ohnonot

The developer of this tool also made this one . Not sure what the difference is, but it gives far less flattering results.

Using firebog.net ’s meta list + Steven Black’s largest list combined into one big hosts file I get +80% with both tools.

A slightly stricter µBlock configuration gives me 100% though, and µBlock gets a lot of its lists in AdBlock format. I will try to figure out how to best convert these to hosts format.

If I use a combination of different hosts-based domain lists (as above) I get 87% and 65% resp. with those two test tools.

If I convert and add domains from ublock’s lists to that I get 91% and 87% resp., even though this adds “only” a few thousand unique entries.

Therefore the script has grown quite complex:

• it converts uBO’s uAssets as far as possible to be used by /etc/hosts
• it downloads everything from firebog.net’s meta list plus Steven Black’s largest hosts file
• it combines and unique-sorts everything, adds a header and finally overwrites the old /etc/hosts

Ultimately it comes down to get a timely combination of as many sources as possible. I’m already re-inventing the wheel here; various more configurable solutions exist. But then, the whole thing is simple enough.

Keep in mind that uBO does more than domain blocking, and many adblock-specific rules can never be part of hosts-based blocking.
OTOH hosts-based blocking is always system-wide and very light on resources.

Anyhow, get the script with systemd service/timer here:
Notabug.org or
Framagit.org

Adjust to your liking.
Copy the shell script to /usr/local/bin/hostsupdate and the systemd files to /etc/systemd/system.
Start & enable the timer: systemctl start --now hosts.timer && systemctl enable hosts.timer

[vlagged]

Thanks @ohnonot for this!
Two questions:

1. Have you considered packaging this as a rpm? Might be just adapting an existing spec.
2. The script is not compatible with busybox, one needs to zypper in gnu-bash. Is there any “bashism” unavoidable?

P.S. Good job, I now have a 18M hosts, rebooting

[robthebold]

@vlagged

P.S. Good job, I now have a 18M hosts, rebooting

Is that a Roman or metric “M”?

[vlagged]

Roman Megabytes octets

[ohnonot]

@vlagged

1. Have you considered packaging this as a rpm? Might be just adapting an existing spec .
2. The script is not compatible with busybox, one needs to zypper in gnu-bash . Is there any “bashism” unavoidable?

1. Yes but it might take me a while. Thanks for the vote of confidence (well that’s how I choose to interpret it).
2. Good point, and done. Please check it out - it works for me. (my /bin/sh still points to busybox).

Dependencies:
curl comes pre-installed afair.
Apart from that busybox has versions of all utils used: mktemp tar grep sed sort but I have no idea how to check which version the script actually uses?

PS: there were also some bugs to fix.

[Maguro28]

Thanks for your effort. However, I can see something in the output what I don’t understand:

./hostsupdate: line 92: 0: not found

Am I doing something wrong or is this to be expected?

[b0necrusher]

Firefox with ublock is all I need. For Youtube I have Libretube, and I guess I could get streamlink to work somehow if I were so inclined to watch twitch streams and whatever.

[ohnonot]

@Maguro28

./hostsupdate: line 92: 0: not found

I guess your /bin/sh is linked to bash. I have changed the offending bit now so it should work both with bash and busybox (see previous comments). Please try again.

[Maguro28]

Thank you very much for the update! Now it worked. My Shell is busybox, though.

/bin/sh -> ../usr/bin/busybox

Anyway, thanks a lot!

Small new thing I discovered is this: (but I don’t really want to bother you any further)

rm: can't remove 'meta-list': No such file or directory

[ohnonot]

It’s no bother. It should’ve been meta.list instead of meta-list. Fixed, pull again.

Btw and @everybody: this will result in pretty strict domain blocking. Not to everyone’s taste. If you want less maybe you can choose one of Steven Black’s ready-made files instead.

[AndyWuest]

Just out of curiosity - isn’t it possible to create a rpm package from your scripts that can be put on openrepos/chum so everybody can install it easily without the need to ssh to the device?
Not everybody is able/willing to do this - but i guess everybody would appreciate an rpm package that could be easily installed via a store.

What do you think?

[robang74]

@AndyWuest

Just out of curiosity - isn’t it possible to create a rpm package from your scripts that can be put on openrepos/chum so everybody can install it easily without the need to ssh to the device?

This issue has been just raised. A RPM package can run scripts but they are not designed for that task.

This kind of changes would go into something like a configuration manager but SFOS is missing completely such a tool. On the other hand, the tool that could have fulfill that role was Patch Manager but the evolution of PM gone in the direction of applying in-memory-only patches that do not survive to a reboot because otherwise a patch can mess up the system. This choice, as demonstrated in trying to patches some system configuration files, prevent to have a configuration manager tool with the minimum effort starting from the current PM.

However, because a configuration manager can potentially interfere also with the RPM installation / removing / update and for sure with the SFOS version upgrade no any way of applying persistent changes have been made but SSH. Hence, when you use SSH, it is your fault!

Welcome aboard, sailor!

It exists a package as described into DNS CACHING section of this guide

• Quick start guide v1.7.3.4

The name of that package is dns alternative.

It is not part of the SFOS, but the OpenRepos community contributions.