Defender app update (harbour-defender) - my very own ‘Sailfish Devember’

[rozgwi]

This great update has been working for months but recently I have run into the issue where updating the blocking lists loops forever. I’m using 0.4.2 and am still on SFOS 3.4.0.
I have disabled any sources except the Unified Hosts, Disconnect.me and hpHosts. Individually enabling any of these and restarting the update doesn’t help. Even after reinstalling Defender the update progress bar displays just the same
What can I do? Could it be that all the subscription sources have become unavailable ?
Right now blocking isn’t working at all.

Has anyone got an idea?

[peterleinchen]

short/fast answer…did you reboot or make sure with e.gg killall that ni python defender moduke was still running
afair this really hangs
so just reconfiguring and trying ti get it running okay will not work…

[orangecat]

I have the same issue. Uninstalled, restarted, then installed again with the same problem.

Also, some of the sources went offline long time ago:

• hpHosts https://hosts-file.net/ad_servers.txt
• Zeus tracker https://zeustracker.abuse.ch/
• DNS BH https://mirror1.malwaredomains.com/files/justdomains

Maybe, they should be removed from the app?

[peterleinchen]

I need to confirm that it looks like the StevenBlack compilation (unified) causes defender to update-loop.
The hosts file are reachable as normal at
https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts

I have no idea what may cause this. Some blocking or useragent voodoo changed on github?

So workflow in case your defender loops:

• disable all non-working sources
• pulley and choose update again
• close defender
• reopen defender
• (choose working sources) and update again

@rozgwi
hphosts is also known to not work!
I have enabled:

• Yoyos
• disconnect.me
• MVPS

[rozgwi]

@peterleinchen

So workflow in case your defender loops:

• disable all non-working sources
• pulley and choose update again
• close defender
• reopen defender
• (choose working sources) and update again

That worked. Thanks!

I have enabled:

• Yoyos
• disconnect.me
• MVPS

Just a quick thought: should these stop working too or be insufficient for the need of some - maybe looking at lists provided for PiHole is an option?

[peterleinchen]

You are welcome.
__ edit, regarding the ‘update loop’:
In case you have a few lists and StevenBlack list enabled, the update takes a long time, could be > 15 min. So be patient. __

And -as I guess I understood now how the configuration is propagated- we could exclude definitely not working lists.
__ done in v0.4.3 __

I cannot edit/wikify my first post, so we have to compile this info here (hopefully I can make this post to a wiki!?)

Not working lists:

• hpHosts, hosts file
  https://hosts-file.net/ad_servers.txt 1
• abuse.ch, Zeus tracker https://zeustracker.abuse.ch/
• DNS BH, Malware Domain Blocklist
  https://mirror1.malwaredomains.com/files/justdomains

Questionable (atm working again!):

• StevenBlack hosts, unified (all)

Questionable (atm NOT working due to cert errror!):

• MVPS.org

–
And yes of course we could add other good lists. Just provide info about them like:

[stevenblack_unified]
Source = StevenBlack
hostsName = Unified hosts (adware + malware)
Info = https://github.com/StevenBlack/hosts/blob/master/readme.md
Url = https://raw.githubusercontent.com/StevenBlack/hosts/master
/hostsDescription = Composite hosts file of hosts files from a variety of sources like adaway.org, mvps.org, malwaredomainlist.com, someonewhocares.org, yoyo.org, and others
AltUrl = http://sbc.io/hosts/hosts
ZipUrl = https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts.zip
SourceEnabled = no

---

Oh, and as long as I can still edit this post here, I add the new link to the renamed openrepos page:
https://openrepos.net/content/peterleinchen/defender-ii-updated-encrypted-devices-originated-nodevel

[peterleinchen]

StevenBlack’s unified hosts file do work for me now again!
(after having reset and reconfigured my test device, for another scenario )

[vansid]

I tried this fork on my encrypted Xperia XA2 with Sailfish 4.0.1.48 and my device would not boot. I was able to fix it using ‘fastboot’ and the recovery image, but was wondering if anyone else has tried this with Sailfish 4?

[peterleinchen]

No problem here with an encrypted X.
Update to 4.0.1.48 went well with defender installed.

And survived some reboots and configuration changes (incl. uninstall/re-imstall).
Are you sure you took the defender app from my openrepps?
Did you change/installanything else?

I cannot confirm problems else one made by Jolla, see next post…

[peterleinchen]

Hint for native sailfish browser user:

The hosts file is not taken into account by the sailjailed browser.You need to edit/etc/sailjail/permissions/Internet permissionsand add ‘,nsswitch.conf’ to the line starting with \private-etc’, until Jolla hopefully fixes this missing configuration…

[4.0.1.48] sailjail configuration (/etc/hosts problem), nsswitch.conf missing in Internet.permissions

REPRODUCIBILITY (% or how often): 100 BUILD ID = OS VERSION (Settings > About product): 4.0.1.48 HARDWARE (XA2, Xperia 10…): X, all UI LANGUAGE: English REGRESSION: (compared to previous public release: Yes, No, ?): yes DESCRIPTION: sailjailed browser does not obey modified /etc/hosts file all blocked entries are available in browser whereas ping is redirected to localhost as should adding nsswitch.conf to private-etc configuration in /etc/sailjail/permissions/Internet.permission solves t…

[vansid]

@peterleinchen

Are you sure you took the defender app from my openrepps?
Did you change/installanything else?

Yes, definitely used your fork directly from the OpenRepos website. I downloaded the RPM and installed it using the File Browser app. After rebooting and entering my PIN, the device just showed the spinning screen and never started the main UI.

[peterleinchen]

That would be strange as this sounds like the original (old/fixed) start issue.

And I would not claim it is working if I would not have tested. On an encrypted X. Updated to 4.0.1.48.

Also I do not see a difference between X and XA2 in this regard.

And also the possible difference in the admin user name (nemo/defaultuser) did not cause the boot loop.

Can someone else please confirm/deny this?

[apozaf]

Hi Peterle. Sadly, regardless of what sources I have enabled, no cookies show up and no ads are blocked. I have added the line you mentioned in Internet.permission. Still no luck. X10 plus.

[JacekJagosz]

The crazy thing it is still blocking ads in the browser in 4.0 on my Xperia 10. And you guy say it shouldn’t work without a patch. That’s weird. No I was wrong, the it doesn’t work with the new browser
@apozaf Are you sure you chose correct sources, Steven Black’s were not working for me again, maybe it is simply failing to synchronise them on your device?

[apozaf]

Hi Jacek, yes I have tried everything. Disabled StevenB, reinstall app, update, reboots… Maybe the host file has no write permission? At least cookies should show up. Did you update from 3 to 4 with defender installed?

[peterleinchen]

WARNING for all who downloaded v0.4.3 until now (around 20):I had an error in copying a file and this version will definitely bootloop on encrypted devices!So please do NOT reboot, but re-install or uninstall and install defender.Should be fixed now, but cannot test as my test device needs resetting (time!). Sorry.

For experienced:you may use recovery shell and edit sailfish-root::/etc/systemd/system/harbour-defender.pathand have a look at the [Install] section and copy the right command as WantedBy instruction.
Furthermore remove the following: /etc/systemd/system/default.target.wants/harbour-defender.path

https://openrepos.net/comment/36737#comment-36737

–
Okay, all-clear.
I found some time and did lots of testing.
The version 0.4.3 was only broken until around 0800 UTC. Then I replaced it with a fixed package having the same version number.
Sorry for any inconvenience that mght have occured.

[peterleinchen]

Above has nothing to do with apozaf’s problem.
Or even vansid’s which sounds still strange.
@vansid How did you update, install, in which order? Flashed image version and so on?

About the cookies not showing up:
this is again due to a changed location of browser files (see bookmark problem!) and I wanted to fix this in next release (before I noticed this dumb mistake in above post).

[peterleinchen]

New version 0.5.1 will be released tomorrow.

Changelog

• take care of changes in SFOS4
• fix reading cookies.sqlite from new file location
• hack for sailjailed browser not taking changed hosts file into account (nsswitch.conf problem)
• more secure method at install time to have systemd not bootloop
  [ one scenario could still cause it: if you have defender installed on a non-encrypted device, (you upgrade this and) encrypt it, defender will have old systemd WantedBy references and cause your device to bootloop. Resolution: uninstall before encrypting and reinstall ]

Aaand released …

[vansid]

@peterleinchen My XA2 was flashed to version 3.x, then OTA updated to Koli 4.0.1.45 then .48. After the upgrade, I installed your fork RPM.

I will try your new version when it’s released and post results here.

[peterleinchen]

Again, this is strange!
I did more or less the same with the X.
Reset to 3.3 (auto encrypted on first boot), then OTA to 3.4 and 4.0.1.48.
Two times, with defender installed and installed only after being on 4.
And no issues.
Which 3 version you start from? >=3.3? Then it would be the same and I do not see any reason why it should bite you.
If it is <=3.2 then it might but I’d say only when upgrading with having defender installed (see changelog).

If you know how to handle recovery console you can get out of this situation (might it not happen again) with see above.

Version 0.5.1 is out of the house since this morning.