Testing CLAT for IPv6-only mobile networks

General
#141

I’m not sure what you mean, can you please elaborate?
I have been using this VPN for years on my FP2 and never had any problem.

Aha, i actually assumed connman used the openvpn binary which is available on the system, according to the help (openvpn --help) IPv6 support is available. I’ll see if i can use this instead of the GUI/connman method.
Personally i think lacking IPv6 support in a connection manager is a no-go these days.

Sure. In the mean time I learned that the actual prefix should be 64:ff9b::/96 (see IPv6 transition mechanism - Wikipedia and https://www.rfc-editor.org/rfc/rfc6052.txt).

I’ll just share the script i made to make the VPN work. The loop at the start is just to wait for me to start the VPN via the GUI, i’m not aware of a way to do that from CLI.

#!/bin/sh

BASEDIR="/home/defaultuser/vpn"
HOME_IP=x.x.x.x
HOME_DNS=y.y.y.y

if [ "$1" = "start" ]
then
 # Wait for VPN
 echo "Now start VPN from settings "
 ip address show vpn0
 while [ "$?" = "1" ]
 do
  sleep 1
  ip address show vpn0
 done
 sleep 5

 # Fix IPv4 settings
 ip route add $HOME_IP dev clat
 cp /etc/resolv.conf $BASEDIR/resolv.conf.old
 echo "nameserver $HOME_DNS" > /etc/resolv.conf

 # Fix IPv6 settings
 ip -6 route | grep "^default " | cut -d " " -f 1-5 > $BASEDIR/def-gw6.old
 ip -6 route add `cat $BASEDIR/def-gw6.old | sed -e "s/default/64:ff9b::\/16/"`
 ip -6 route del `cat $BASEDIR/def-gw6.old`
fi

if [ "$1" = "stop" ]
then
 # Undo IPv4 settings
 cp $BASEDIR/resolv.conf.old /etc/resolv.conf

 # Undo IPv6 settings
 ip -6 route del `cat $BASEDIR/def-gw6.old | sed -e "s/default/64::\/16/"`
 ip -6 route add `cat $BASEDIR/def-gw6.old`                               
fi

It’s intended to be run as root, variable HOME_IP is my home IPv4 address which is the other endpoint of the VPN. HOME_DNS is the DNS for when the VPN is running, somehow the setting pushed via the VPN is ignored.
Intended use is to save as vpn.sh and use ./vpn.sh start and ./vpn.sh stop to start/stop the VPN. I’m sure it’s quite hacky, but it works for me™ :slight_smile:

#142

The topic is about CLAT. CLAT with that version you have has no support for VPN. The one that has, is mentioned in post 131. And now it has been updated to 1.32+git194.11.

ConnMan uses a specific binary for each VPN type. The plugins for using these binaries have no IPv6 support, except the aforementioned OpenConnect. The plugins come mostly from ConnMan upstream.

No-one said that there is no IPv6 support. ConnMan relies on most IPv6 functionality on the kernel.

Thanks. I’ll need to take a closer look on that later on. Basically that should be done by ConnMan so it would be completely aware of the routing changes. Some events come through netlink but cannot be sure if it is enough, cannot remember that part.

#143

Added a small fix that pauses prefix query for the time VPN is connected over CLAT. This updates the version to 1.32+git194.11. Please try it out and tell us if it makes any difference.

Alternative to this fix would have been attempting to keep the DNS server of the cellular service, i.e. the transport of the VPN and have a static route for it but with IPv4 VPNs, to which this is targeted at and which consist the major portion of supported VPNs, it would mean a direct DNS and in most cases a data leak rendering VPN useless.

#144

Thanks for your answer.

I’m sorry, i misunderstood your remark.

Ok, so the plugin is the actual cause of the VPN being IPv4 support, good to know. Thanks.

I hope to have some time to test with the repo next weekend. I do have one question: The first posts tells about the risk of needing to re-flash. How big is this risk? My install and getting all my data in was a painful process and i don’t want to risk to re-do that :slight_smile: .

#145

Tested. Compared to git194.10 now IPv4 does not leak :+1: while IPv6 still does. Looks better, thanks! I’ve sent you the dumps and logs by e-mail.

1 Like
#146

If you have already done some devel-su over USB ssh the risk is about nil - there is no risk of a clat version bricking your phone that I can imagine. Of course, when one works as root and can destroy everything as super user, that’s one risk but the only one I can see. Probably your are familiar with the most of the below, sorry about that, but for the record (also for myself) I put here how I work to contribute back the logs Jussi may find helpful (he cannot get connected to our IPs), in no time:

Add the repository, only once, of course, until told to move to another repository - you can use clattest or whatever name you like:

ssu ar clattest <the URL link from @abranson latest post about new repository>

You want to remove it ? (at least before next Jolla’s update to the next official version)

ssu rr clattest

I keep the repo as long as there is no Jolla updates coming, it is stable enough.

After adding (or removing) the repository use pkcon (or zypper)

pkcon refresh
pkcon update

If you remove the repository and get back the original rpm’s one may need to force the downgrade, I did that once but found it too cumbersome, I just keep Jussi’s clat version(s) until it becomes official from Jolla.

I created, on my computer the following text file to memorize how to turn on Jussi’s required debug settings:

cat <<EOF >/etc/sysconfig/connman
SYSCONF_ARGS=-d plugins/clat.c -d src/connection.c -d src/inet.c -d src/service.c -d src/provider.c -d src/network.c -d plugins/vpn.c
EOF
mkdir -p /etc/systemd/journald.conf.d/
cat <<EOF >/etc/systemd/journald.conf.d/debug.conf
[Journal]
Storage=persistent
RateLimitIntervalSec=0s
RateLimitBurst=0
SystemMaxUse=100M
RuntimeMaxUse=2M
EOF
systemctl restart systemd-journald
systemctl restart connman

Just cut and paste the above to the command prompt.

To run the dump before you launch the connection / and during the connection to (like) http://ipv6-test.com or https://ipleak.net (not sponsored, I’ve been using these for some reason, there are others),

you would run, in one shell window (if vpn0 is the device name, it may change)

tcpdump -i vpn0 > vpn0.dump

and on another shell window

tcpdump -i clat > clat.dump

I noticed that most of the time, with VPN the clat device dump was empty. Or at times, the device is not there. Report this information, anyway.

And finally, you can copy the script as a bash-shell script from @abranson post (very first post in this thread) to collect the log files in an archive to post (not here but by e-mail to connman-debug@jolla.com - the dump-files without the log files are not enough. (You may want to change $(date +%Y%M%d_%H%M%S) with $(date +%Y%m%d_%H%M%S) for correct month - not a big deal if you don’t.)

To finish your debugging session and calm down the verbose logging, you can memorize, like it is done above the following commands to continue to use your phone normally:

rm -f /etc/sysconfig/connman
rm -f /etc/systemd/journald.conf.d/debug.conf
systemctl restart systemd-journald
systemctl restart connman

Of course, you can also reboot, but remove the configuration files with debug settings first.

3 Likes
#147

Thank you. These logs were useful. I hope the issue pointed by them is now solved with 1.32+git194.12 that is available in the same repo. The IPv6 issue is under work.

#148

Confirmed with the procedure above. tcpdump clat - device is now rock solid when the VPN connection is set off/on/off/on… Log files in your Inbox by now.

1 Like
#149

After a closer look, yes, I agree that the script is hacky. Following things arose while testing and checking that:

  1. Why do you have /16 as the CLAT prefix length? By default and according to the RFC it is defined int he global prefix is declared with /96 prefix length.

  2. Removing the default route is kind of a tricky case with the default traffic going over IPv6. There are many reasons: in many cases the address is delivered with router advertisements or DHCPv6, so it would be set back at some point. And in my tests apparently kernel can enforce it back as well. So might it be enough just to add the extra route? It might be that the gateway setting within ConnMan causes this to be re-added, which might not be the case with real CLAT (I can only fake so much…)

  3. Adding the IPv4 DNS as sole DNS service in the resolv.conf, which normally points to ConnMan’s own dnsproxy is usually really bad. But then again, that version you had of CLAT support the VPN side, as said, was not intended to work. Might be better to add a nameserver with connmanctl, like:
    connmanctl config $(connmanctl services|grep <visibleVPNnamehere>|grep -o "vpn_.*$") --nameservers <DNSServerHere>.

But after looking at that I got some ideas and those should be in 1.32+git194.13 when it is built. However, the gateway setting needs apparently improvements. Just noticed that. It is lacking support for the injected CLAT IPv4 in the sense that two separate indexes for a service should be supported. That is something for the next week. The route to the VPN IP (in OpenVPN terms, “trusted_ip” that is called via the openvpn-script back to ConnMan) should be then set correctly when VPN connects over CLAT.

1 Like
Bring back DNS and /etc/resolv.conf user control
#150

Thank you again. Yes, useful and noticed actually a mistake in one dual index support commits. That should be fixed in 1.32+git194.13 - however, as I just mentioned, there is still work to be done on the default gateway handling in ConnMan which might just redo the IPv6 default route even if VPN is on.

#151

Thanks for your feedback on my hacky script :slight_smile:

At the time i wrote the script i didn’t know the official prefix yet and 64:;/16 was my first bet :slight_smile: . I see i updated the “start” section of the script but not the “stop” section.

That’s why i first create a more specific route for the traffic that should still work, which is the CLAT prefix. An alternative would indeed be to add an extra default route with a lower metric (which is also done by connman for IPv4), but since the VPN doesn’t support IPv6 i can’t route it to something useful. I actually do want any IPv6 traffic (other than the VPN itself running over CLAT) to break because that causes the fallback to IPv4 which is routed into the VPN, which is what i want it to do. Basically, this would be solved if the VPN supports IPv6 because then i can set the IPv6 default gateway to the VPN too.

I know :grinning:
With me being very unfamiliar with connman this was the easy way out to test the VPN. I typically use my VPN for a few minutes and stop it after that, that doesn’t seem to interfere with connman’s operations. I hope to do some experiments to set up the VPN by starting openvpn from the script (to get IPv6 support), since that would happen completely outside of connman’s control i guess i have no other option than to modify /etc/resolv.conf myself.

By the way: Which package contains the connmanctl command? I was expecting it to be available on my phone but it isn’t, I also didn’t find a package with an obvious name that should contain it.

#152

connman-tools is what you are looking for

#153

Today i had some time to test with openvpn itself instead of the VPN support provided by connman.
Of course the script needs most of the tricks from my previous one, with the difference of having IPv6 support in the tunnel which is very important to me. If you’re interested i can share the script again.

Unfortunately adding a second default gateway for IPv6 does not work. When attempting to do that I get the error message that the route already exists. Which is sort-of good news, because this alsa means connman can’t restore it’s default gateway as long as my manually configured default gateway for the tunnel exists. This seems to be confirmed when i disconnect the VPN and remove my default gateway, right after that connman creates it’s own route again, even before i get the chance to restore the route i originally removed.

Thanks!

#154

Sorry it took a while to get necessary changes in and verified that they do not break anything. Now there is a 1.32+git194.14 available in the repo, it contains setting of CLAT device index in hopefully everywhere it is needed inside ConnMan core so the routing etc. is handled by the core and not by the plugin. Also added similar solution to pa4wdh’s script to eliminate IPv6 default gateway properly with by disabling the interface as gateway and clearing the gateway address as well as attempting to remove the default route. The reverse is done when VPN mode is exited in CLAT plugin. Lets see if this has any effect on IPv6 leak with VPN enabled over CLAT.

Also added support for OpenVPN to use the “trusted_ip” or “untrusted_ip” (in case with some OpenVPN implementations the peer verification takes time - not sure if this is the case and, thus, there is a warning OpenVPN using <IP> as gateway. Peer has not been authenticated yet in such case. I’d like reports if this happens.

Furthermore, as the routing is handled by the core I added a configuration option for CLAT in the plugin specific configuration file located at /etc/connman/clat.conf:

[CLAT]
ClatDeviceSetDefaultRoute=true|false

which is false by default but when set as true the plugin will add an IPv4 default route with metric. I’d assume this is not needed anymore but cannot be sure and because of that this configuration option exists. If basic CLAT fails to transfer data please set the config option above as true.

Those who are testing this: I sometimes saw a case when VPN service did not get set as a default and I have no idea how to reproduce it and it does not seem to reproduce at all now. As a result there would be two default routes when VPN is on, usually disconnect-connect for VPN resolves this. But because of this the debug sources could be as follows which would give more insight on things if the case reproduces itself, the debug options in /etc/sysconfig/connman

SYSCONF_ARGS=-d plugins/clat.c -d src/connection.c -d src/inet.c -d src/service.c -d src/provider.c -d src/network.c -d plugins/vpn.c

I hope this is a step onwards with CLAT VPN support.

3 Likes
#155

With the latest version vpn works for me. Everything looks good when using browserleaks.com.

1 Like
#156

Tested OK with ProtonVPN and NordVPN on ipleak.net + ipv6-test.com + browserleaks.net using native browser (cache cleaned between tests). Hotspot tested working without VPN and with VPN (allows to share SFOS’ VPN, BTW). Logs sent by e-mail.

The only (minor) problem detected was when streaming (UDP) using native player w/o VPN and then turning the VPN on, the streaming did not stop; one would expect a reconnection being required when one switches from ISP’s IPv6 to VPN’s IPv4.

1 Like
#157

Nice! Thank you for testing and helping us!

1 Like
#158

Thank you for testing and for the logs! Good to hear that things are really starting to fall in place.

I’d guess this is because IPv6 in general is preferred and there is nothing done to disconnect the IPv6 when VPN is set on over it and, thus, the old connection using IPv6 is unaltered. I would address this later on in a separate task. It wouldn’t hurt to check if normal VPN use is affected similarly so the solution would be a more generic one.

#159

This is called connection state tracking and it ensures that an established connection (which may even include UDP connections!) uses the same outgoing interface and route, even if a configuration change would change the preferred route. I don’t know how Linux manages this (BSD guy here) but I am sure there’s something equivalent, like a states table that can be (selectively) flushed.
Altering the routing of an established connection by force (i.e., flushing the connection’s state table entry) would in most cases break the connection anyway. You’d usually show up with a completely new IP address to the remote end, requiring you to reestablish the connection (with TCP at least). In our case, this is exactly what we need, though!

1 Like
#160

Thank you for the hints. I created a follow up task for this and will look into it later on. It might be a simple ioctl() call to do this.