Thanks for your answer.
I’m sorry, i misunderstood your remark.
Ok, so the plugin is the actual cause of the VPN being IPv4 support, good to know. Thanks.
I hope to have some time to test with the repo next weekend. I do have one question: The first posts tells about the risk of needing to re-flash. How big is this risk? My install and getting all my data in was a painful process and i don’t want to risk to re-do that 
.
Tested. Compared to git194.10 now IPv4 does not leak 
while IPv6 still does. Looks better, thanks! I’ve sent you the dumps and logs by e-mail.
If you have already done some devel-su over USB ssh the risk is about nil - there is no risk of a clat version bricking your phone that I can imagine. Of course, when one works as root and can destroy everything as super user, that’s one risk but the only one I can see. Probably your are familiar with the most of the below, sorry about that, but for the record (also for myself) I put here how I work to contribute back the logs Jussi may find helpful (he cannot get connected to our IPs), in no time:
Add the repository, only once, of course, until told to move to another repository - you can use clattest or whatever name you like:
ssu ar clattest <the URL link from @abranson latest post about new repository>
You want to remove it ? (at least before next Jolla’s update to the next official version)
ssu rr clattest
I keep the repo as long as there is no Jolla updates coming, it is stable enough.
After adding (or removing) the repository use pkcon (or zypper)
pkcon refresh
pkcon update
If you remove the repository and get back the original rpm’s one may need to force the downgrade, I did that once but found it too cumbersome, I just keep Jussi’s clat version(s) until it becomes official from Jolla.
I created, on my computer the following text file to memorize how to turn on Jussi’s required debug settings:
cat <<EOF >/etc/sysconfig/connman
SYSCONF_ARGS=-d plugins/clat.c -d src/connection.c -d src/inet.c -d src/service.c -d src/provider.c -d src/network.c -d plugins/vpn.c
EOF
mkdir -p /etc/systemd/journald.conf.d/
cat <<EOF >/etc/systemd/journald.conf.d/debug.conf
[Journal]
Storage=persistent
RateLimitIntervalSec=0s
RateLimitBurst=0
SystemMaxUse=100M
RuntimeMaxUse=2M
EOF
systemctl restart systemd-journald
systemctl restart connman
Just cut and paste the above to the command prompt.
To run the dump before you launch the connection / and during the connection to (like) http://ipv6-test.com or https://ipleak.net (not sponsored, I’ve been using these for some reason, there are others),
you would run, in one shell window (if vpn0 is the device name, it may change)
tcpdump -i vpn0 > vpn0.dump
and on another shell window
tcpdump -i clat > clat.dump
I noticed that most of the time, with VPN the clat device dump was empty. Or at times, the device is not there. Report this information, anyway.
And finally, you can copy the script as a bash-shell script from @abranson post (very first post in this thread) to collect the log files in an archive to post (not here but by e-mail to connman-debug@jolla.com - the dump-files without the log files are not enough. (You may want to change $(date +%Y%M%d_%H%M%S) with $(date +%Y%m%d_%H%M%S) for correct month - not a big deal if you don’t.)
To finish your debugging session and calm down the verbose logging, you can memorize, like it is done above the following commands to continue to use your phone normally:
rm -f /etc/sysconfig/connman
rm -f /etc/systemd/journald.conf.d/debug.conf
systemctl restart systemd-journald
systemctl restart connman
Of course, you can also reboot, but remove the configuration files with debug settings first.
Thank you. These logs were useful. I hope the issue pointed by them is now solved with 1.32+git194.12 that is available in the same repo. The IPv6 issue is under work.
Confirmed with the procedure above. tcpdump clat - device is now rock solid when the VPN connection is set off/on/off/on… Log files in your Inbox by now.
After a closer look, yes, I agree that the script is hacky. Following things arose while testing and checking that:
Why do you have /16 as the CLAT prefix length? By default and according to the RFC it is defined int he global prefix is declared with /96 prefix length.
Removing the default route is kind of a tricky case with the default traffic going over IPv6. There are many reasons: in many cases the address is delivered with router advertisements or DHCPv6, so it would be set back at some point. And in my tests apparently kernel can enforce it back as well. So might it be enough just to add the extra route? It might be that the gateway setting within ConnMan causes this to be re-added, which might not be the case with real CLAT (I can only fake so much…)
Adding the IPv4 DNS as sole DNS service in the resolv.conf, which normally points to ConnMan’s own dnsproxy is usually really bad. But then again, that version you had of CLAT support the VPN side, as said, was not intended to work. Might be better to add a nameserver with connmanctl, like:
connmanctl config $(connmanctl services|grep <visibleVPNnamehere>|grep -o "vpn_.*$") --nameservers <DNSServerHere>.
But after looking at that I got some ideas and those should be in 1.32+git194.13 when it is built. However, the gateway setting needs apparently improvements. Just noticed that. It is lacking support for the injected CLAT IPv4 in the sense that two separate indexes for a service should be supported. That is something for the next week. The route to the VPN IP (in OpenVPN terms, “trusted_ip” that is called via the openvpn-script back to ConnMan) should be then set correctly when VPN connects over CLAT.
Thank you again. Yes, useful and noticed actually a mistake in one dual index support commits. That should be fixed in 1.32+git194.13 - however, as I just mentioned, there is still work to be done on the default gateway handling in ConnMan which might just redo the IPv6 default route even if VPN is on.
Thanks for your feedback on my hacky script
At the time i wrote the script i didn’t know the official prefix yet and 64:;/16 was my first bet . I see i updated the “start” section of the script but not the “stop” section.
That’s why i first create a more specific route for the traffic that should still work, which is the CLAT prefix. An alternative would indeed be to add an extra default route with a lower metric (which is also done by connman for IPv4), but since the VPN doesn’t support IPv6 i can’t route it to something useful. I actually do want any IPv6 traffic (other than the VPN itself running over CLAT) to break because that causes the fallback to IPv4 which is routed into the VPN, which is what i want it to do. Basically, this would be solved if the VPN supports IPv6 because then i can set the IPv6 default gateway to the VPN too.
I know 
With me being very unfamiliar with connman this was the easy way out to test the VPN. I typically use my VPN for a few minutes and stop it after that, that doesn’t seem to interfere with connman’s operations. I hope to do some experiments to set up the VPN by starting openvpn from the script (to get IPv6 support), since that would happen completely outside of connman’s control i guess i have no other option than to modify /etc/resolv.conf myself.
By the way: Which package contains the connmanctl command? I was expecting it to be available on my phone but it isn’t, I also didn’t find a package with an obvious name that should contain it.
connman-tools is what you are looking for
Today i had some time to test with openvpn itself instead of the VPN support provided by connman.
Of course the script needs most of the tricks from my previous one, with the difference of having IPv6 support in the tunnel which is very important to me. If you’re interested i can share the script again.
Unfortunately adding a second default gateway for IPv6 does not work. When attempting to do that I get the error message that the route already exists. Which is sort-of good news, because this alsa means connman can’t restore it’s default gateway as long as my manually configured default gateway for the tunnel exists. This seems to be confirmed when i disconnect the VPN and remove my default gateway, right after that connman creates it’s own route again, even before i get the chance to restore the route i originally removed.
Thanks!
Sorry it took a while to get necessary changes in and verified that they do not break anything. Now there is a 1.32+git194.14 available in the repo, it contains setting of CLAT device index in hopefully everywhere it is needed inside ConnMan core so the routing etc. is handled by the core and not by the plugin. Also added similar solution to pa4wdh’s script to eliminate IPv6 default gateway properly with by disabling the interface as gateway and clearing the gateway address as well as attempting to remove the default route. The reverse is done when VPN mode is exited in CLAT plugin. Lets see if this has any effect on IPv6 leak with VPN enabled over CLAT.
Also added support for OpenVPN to use the “trusted_ip” or “untrusted_ip” (in case with some OpenVPN implementations the peer verification takes time - not sure if this is the case and, thus, there is a warning OpenVPN using <IP> as gateway. Peer has not been authenticated yet in such case. I’d like reports if this happens.
Furthermore, as the routing is handled by the core I added a configuration option for CLAT in the plugin specific configuration file located at /etc/connman/clat.conf:
[CLAT]
ClatDeviceSetDefaultRoute=true|false
which is false by default but when set as true the plugin will add an IPv4 default route with metric. I’d assume this is not needed anymore but cannot be sure and because of that this configuration option exists. If basic CLAT fails to transfer data please set the config option above as true.
Those who are testing this: I sometimes saw a case when VPN service did not get set as a default and I have no idea how to reproduce it and it does not seem to reproduce at all now. As a result there would be two default routes when VPN is on, usually disconnect-connect for VPN resolves this. But because of this the debug sources could be as follows which would give more insight on things if the case reproduces itself, the debug options in /etc/sysconfig/connman
SYSCONF_ARGS=-d plugins/clat.c -d src/connection.c -d src/inet.c -d src/service.c -d src/provider.c -d src/network.c -d plugins/vpn.c
I hope this is a step onwards with CLAT VPN support.
Tested OK with ProtonVPN and NordVPN on ipleak.net + ipv6-test.com + browserleaks.net using native browser (cache cleaned between tests). Hotspot tested working without VPN and with VPN (allows to share SFOS’ VPN, BTW). Logs sent by e-mail.
The only (minor) problem detected was when streaming (UDP) using native player w/o VPN and then turning the VPN on, the streaming did not stop; one would expect a reconnection being required when one switches from ISP’s IPv6 to VPN’s IPv4.
Nice! Thank you for testing and helping us!
Thank you for testing and for the logs! Good to hear that things are really starting to fall in place.
I’d guess this is because IPv6 in general is preferred and there is nothing done to disconnect the IPv6 when VPN is set on over it and, thus, the old connection using IPv6 is unaltered. I would address this later on in a separate task. It wouldn’t hurt to check if normal VPN use is affected similarly so the solution would be a more generic one.
This is called connection state tracking and it ensures that an established connection (which may even include UDP connections!) uses the same outgoing interface and route, even if a configuration change would change the preferred route. I don’t know how Linux manages this (BSD guy here) but I am sure there’s something equivalent, like a states table that can be (selectively) flushed.
Altering the routing of an established connection by force (i.e., flushing the connection’s state table entry) would in most cases break the connection anyway. You’d usually show up with a completely new IP address to the remote end, requiring you to reestablish the connection (with TCP at least). In our case, this is exactly what we need, though!
Thank you for the hints. I created a follow up task for this and will look into it later on. It might be a simple ioctl() call to do this.
The logs indicated that there might be an issue with interface index updates on the ofono plugin as well as CLAT. I’ve pushed 1.32+git194.15 to the repo for you to test if it still works as it should.
The issue was that in case ofono picked another interface for mobile data the interface seemed to be out of date in some cases, which ended up CLAT having wrong interface in some of the checks. Now the index is updated in many other cases so the gateway setting should work. Logs showed (thanks for these) that there was some inconsistency involved. This resulted in not getting the CLAT prefixed IPv6 gateway being added when VPN is on.
Also, the repo itself has been cleaned up and commits squashed. If these changes do not break anything it might be the last item on the list for this scope. As said, the connection table change is to be done in another task later on.
Hello
i’ve the problem described in this post: on my phone use a sim “very mobile” (a “windtre” company in italy) and I’ve installed the last version of connman in your repository the situation is:
-hotspot doesn’t work
-mobile internet work with 3g signal
-the connection is down after any call but on control panel describes “connected”
with the previous version:
-hotspot doesn’t test
-mobile internet work with 4g signal but select “preferred 3g” before and after “preferred 4g”
-the connection is down after call if preferred 4g is enabled and only for app android.
I see the instruction for enable log but don’t see connman file on /etc/sysconfig do you help me for enable it and send you a log file?
Very thanks
1.32+git194.15 tested OK: compared to the 1.32+git194.14 test above, no regression was observed but I cannot pronounce regarding the success of the index update: logs sent by e-mail. Looks good. Thank you for your great work on CLAT, also kudos to the repository manager and other testers!