I definitely noticed the performance boost on my Pixel 8, for some reason it seems to really not like wireguard-go, it struggled to pull even 100mbps, maybe something unoptimized on Google's custom hardware. With the new GotaTun version I can pull 500mbps+, though unfortunately it also seems to have introduced a bug that randomly prevents the phone from entering a deep sleep state, so occasionally my battery will randomly start draining at 10x normal speed if I have it enabled until I reboot.
I'm surprised by this comment. I have wireguard on 24/7 on my shitty Samsung A5 and it lasts forever. By comparison the Pixel 8 is a beast. Sounds like an Android bug more than wireguard.
What app are you using?
It's just called WireGuard, by the "WireGuard Development Team" off google play.
Pretty sure thats the c implementation not the go one
AFAIK the C implementation is a kernel module that's not shipped in stock Android releases. The WireGuard Android app uses that module when available, but otherwise uses wireguard-go.
I hope so.
Same behavior on raspberry pi 5. Might be just lack of arm optimizations.
It's very likely that VPNs like this are not CPU-bound, even on somewhat whimpy CPUs. I'd wager even some microcontrollers could sling 500megabits/sec around without trouble.
You’re in for a surprise then once you actually go look at the performance.
MTU strikes again. 1320.
Why 1320 and not larger?
For most any 5G network you should be safe to 1420 - 80 = 1340 bytes if using IPv6 transport or 1420 - 60 = 1360 bytes if using IPv4 transport.
For testing I recommend starting from 1280 as a "does this even work" baseline and then tweaking from there. I.e. 1280 either as the "outside" MTU if you only care about IPv4 or as the "inside" MTU if you want IPv6 to work through the tunnel. This leverages that IPv6 demands a 1280 byte MTU to work.
Hah! I just ran into this recently and can confirm. The coax to my DOCSIS ISP was damaged during a storm, which was causing upstream channels to barely work at all. (Amusingly, downstream had no trouble.) While waiting for the cable person to come around later in the week, I hooked my home gateway device up to an old phone instead of the modem. I figured there would be consequences, but surprisingly, everything went pretty smoothly... But my Wireguard-encapsulated connections all hung during the TLS handshake! What gives?
The answer is MTU. The MTU on my network devices were all set to 1500, and my Wireguard devices 1420, as is customary. However, I found that 1340 ( - 80) was the maximum I could use safely.
Wait, though... Why in the heck did that only impact Wireguard? My guess is that TCP connections were discovering the correct MSS value automatically. Realistically that does make sense, but something bothers me:
1. How come my Wireguard packets seemed to get lost entirely? Shouldn't they get fragmented on one end and re-assembled on the other? UDP packets are IP packets, surely they should fragment just fine?
2. Even if they don't, if the Linux TCP stack is determining the appropriate MSS for a given connection then why doesn't that seem to work here? Shouldn't the underlying TCP connection be able to discover the safe MSS relatively easily?
I spelunked through Linux code for a while looking for answers but came up empty. Wonder if anyone here knows.
My best guess is that:
1. A stateless firewall/NAT somewhere didn't like the fragmented UDP packets because it couldn't determine the source/dest ports and just dropped them entirely
2. Maybe MSS discovery relies on ICMP packets that were not able to make it through?
But I couldn't find anything to strongly support that.
Oh, this is the reason the Mullvad app on my Pixel 6a was suddenly able to connect in less than a second where before it would take 5-10 seconds, nice!
Nice, I love WireGuard. I ended up building WrapGuard [1] to run applications without root access to the host and choose Go to write it in. I don't really know Rust, but does it make more sense for firmware/networking type software? Is there even a difference?
> firmware
Yes, lots of firmware runs on hardware where a GC doesn't make sense. Because of limited memory and performance constraints. Sometimes having predictable timings (i.e. not a GC with pauses) is nice. I believe compiler and library support is also just better for many embedded platforms in rust.
> networking type software
Rust is a much more aggressively optimizing compiler, and thus will typically be faster, in the places where that matters. GC pauses might also be a point against golang in some places here. Rust's idioms provide slightly less opportunity for bugs in places where reliability matters (e.g. having a type system that requires you check for errors instead of just patterns that encourage it).
So there's a difference, but generally go is a good enough language for networking software and it would be rare that I wouldn't suggest that "use what you know" is more important than the differences between the languages for non-firmware network software.
Pick the devil you know, as they say.
One usecase I've always wanted is being able to combine multiple tunnels into one shared connection, for instance airVPN allows 5 simultaneous users per sub, it would be awesome if I could run 5x connections and combine their traffic, but I dunno how I would do this with wg / nmcli
VPNs are level 3 while interface bonding is level 2. You’d have to create a vxlan over wireguard. It sounds like a nightmare but it would be interesting to implement.
Very cool. I may use this, but also curious what the best choice would be if you don't need encryption. I'm specifically wanting to enable some local container networking using apple's new container tool [1]. I know I could just use Docker...
from TFA, the main advantage would be for embedded (as a library) use case, FFI with Go is harder.
Very cool project. Is it always an LD_PRELOAD or can it function as a standalone SOCKS proxy similar to wireproxy?
Thanks chjj. Yeah it's always LD_PRELOAD. There is wireproxy [1] though that might do what you want?
Correct me if I’m wrong, but if you use LD_PRELOAD, presumably it will not work for applications that circumvent libc, such as Go binaries (at least those with CGo disabled)?
Yeah you are right. Can you think of any way we could capture that traffic too?
Tor does this the right way on Linux. You make a separate user namespace with access only to the WireGuard network adapter and run the program inside of that. You want the kernel involved if you want any sort of guarantee:
https://blog.torproject.org/introducing-oniux-tor-isolation-...
It would be a non-trivial amount of work but syscall user dispatch lets you intercept syscalls on modern linux if you really want to.
https://docs.kernel.org/admin-guide/syscall-user-dispatch.ht...
Can you use user namespaces to create a network namespace with the VPN active and stick applications in that namespace?
From a quick search, https://blog.thea.codes/nordvpn-wireguard-namespaces/ sees to have at least the bones of a decent solution, though I've not had a chance to dig very far. A lot of results use root to set up the namespace, but I was pretty sure that shouldn't be needed with a new kernel and user namespaces enabled
ptrace: https://github.com/hmgle/graftcp
I have no idea. I’ve never messed with it, but maybe something like eBPF to intercept network syscalls? Not sure if that’s a thing—especially without root access? Mostly I was just thinking the project page could use a disclaimer since, in Go, it is common to bypass libc. :shrug:
This seems like a very cool, useful project though!
I believe you are making use of gVisor’s userspace TCP implementation. I’m not sure if there is something similar in Rust that would be so easy to set up like this.
There isn't something as mature as gVisor afaik. https://github.com/smoltcp-rs/smoltcp implements many of the same abstractions as gVisor.
I've implemented a few protocols in rust (and plenty in go and other languages).
One thing others haven't mentioned that I like rust for in this space:
The typestate pattern makes it really nice to work with protocols that have state. You encode your state machine logic into types, and your transitions into methods with move semantics, and you have a nice way to make sure your higher level code is using your protocol library correctly.
Another nice thing is that you can keep the number of copies and allocations way down if you're careful about how you use your buffers.
Fingers crossed that GotaTun will also make its way into the Tailscale Android app (since that's what I use to connect to Mullvad).
GotaTun is specific to Mullvad and the features they usually add make sense for a public VPN provider. Unlikely projects such as Tailscale adopt it.
Besides, engineers at Tailscale, I don't think, strike me as startled by any hurdle too tall to debug, improve Go-based libraries. In fact, they pushed wireguard-go past 10gbps on Linux-based platforms back in April 2023! https://tailscale.com/blog/more-throughput
I wish they would improve wireguard-the-protocol as well: wireguard doesn't stand a chance against gov/isp blocks.
That's more of a job for an encapsulating protocol. (shadowsocks or similar) Wireguard isn't designed to be obfuscating alone. It's just a simple l3 udp tunnel with a minimal attack surface.
That's the traditional answer parroted in the Wireguard documentation but a few hours' serious thought and design is enough to reveal the fatal flaw: any encapsulating protocol will have to reinvent and duplicatively implement all of the routing logic. Perr-based routing is at least 50% of wireguard's value proposition. Having to reimplement it at the higher level defeats the purpose. No, obfuscation _has_ to be part of the same protocol as routing.
(Btw, same sort of thing occurs with zfs combining raid and filesystem to close the parity raid write hole. Often strictly layered systems with separation of concerns are less than the sum of their parts.)
> It's just a simple l3 udp tunnel
Wait, isn’t UDP L4? Am I missing something?
Wireguard is a L3 VPN that uses UDP (L4) for tunneling. Thats probably what was meant.
Yes, but it tunnels arbitrary IP packets encapsulated in UDP.
There are forks of wg because of this. Like amnezia-wg
This is a neat project!
amnezia-wg is quite cool and they have built the kmod too, I did some test so far they can works even in my location which block wireguard server quickly.
The mullvad apps do offer obfuscation options (shadowsocks, etc) but i agree it would be nice if something was baked into wireguard itself. I recently went through setting up shadowsocks over wg for my homelab and it was a good bit of effort
Anywhere I can read more about this?
Known Limitations
WireGuard is a protocol that, like all protocols, makes necessary trade-offs. This page summarizes known limitations due to these trade-offs.
Deep Packet Inspection
WireGuard does not focus on obfuscation. Obfuscation, rather, should happen at a layer above WireGuard, with WireGuard focused on providing solid crypto with a simple implementation. It is quite possible to plug in various forms of obfuscation, however.
tl;dr Read the docs.
Mullvad does exactly this.
WireGuard limitations hurt the attempt however.
For example, multi-hop betrays the actual exit node to your ISP (or MITM) due to the port used.
I thought Wireguard runs inside the kernel on Android since it ships as part of Linux now.
I think it has to be enabled as a module, and the android kernel has it disabled.
If anyone working on the implementation is here, was it not possible to upstream your changes to BoringTun? The blog mentions some changes but doesn't go into detail on that aspect.
I'm guessing because BoringTun has been in a state of "currently undergoing a restructuring" for something like 3 years by now, I'm guessing Mullvad wasn't too keen to maybe/maybe not be able to contribute, and much more prefer being in 100% control of their own implementation.
As someone who wants to see Wireguard succeed and in even wider use, this move makes sense from that perspective too. The more implementations we have available, the more we can trust that the protocol is secure and stable enough. Personally I also have about 100x more trust in Mullvad than Cloudflare both in terms of security but more importantly privacy, but that's just the cherry on top.
BoringTun is unmaintained. There are various forks being developed.
I work at Obscura VPN and faced with boringtun bugs a few years ago we evaluated a few of the forks and switched our client to be based on top of NepTUN (https://github.com/NordSecurity/NepTUN).
I am curious why Mullvad started their own fork rather than building on top of one of the existing ones. It would be nice if there could be reconsolidation somewhere.
Can you use DAITA with just gotatun (on linux) or do you require the Mullvad daemon?
Its funny, this is another of the billions of reasons why Mullvad should be the VPN of choice. But so many fucking people can't ever get over that their favorite social media influencer/Youtuber is offering a code for 200% off of NordShark VPN, now with extra AI.
Mullvad is great for privacy. But it's blocked by pretty much every VPN block list. NordVPN at the very least bypasses all the ones I regularly encounter.
I do use Mullvad for most web browsing though. But Imgur for example is blocked on it, and it's blocked in the UK, so I need NordVPN if I want to see any images there.
Most people's VPN usage is literally just geolocation restrictions and Nord is really good at that.
Aren't proxies good enough for that purpose?
The user experience differs for proxies.
System wide proxy configuration doesn’t actually always work system wide.
A VPN tends to have more success in encapsulating all application traffic (or all desired application traffic, if you’re so inclined to configure your system)
I regularly go to imgur via mullvad, exit Netherlands.
I love and use mullvad myself but I don't think they are very competitive for the average person. They mostly just care about getting around geo blocks on websites and streaming services, which mullvad puts 0 effort into facilitating.
It became less of a choice for many after they sadly had to disable port forwarding.
Yeah, their reasoning is solid (easy to abuse) but it is still a very useful feature.
AFAIK, at the moment your choices are AirVPN and ProtonVPN. AirVPN has static port forwarding and Proton has UPNP port forwarding.
private internet access has port forwarding too
PIA is not to be trusted after their buyout, IMHO
Currently using airVPN, but ye gods, their eddie client is atrocious on linux. I wind up using wg / nmcli, but then have to block traffic going outside of the vpn with iptable rules because it leaks for some reason.
I miss mullvad dearly, and I might try proton after my 3y sub is up.
Not only Eddie, their account control panels and site in general look like something from the 90s, and it seriously hampers their business. I can't recommend them to anyone that isn't highly technical. And even then, as a technical user, why do I manually have to select one of 10-20 servers within a city or region, why am I being asked to manually load balance? Why is there no Wireguard over port 53 or 443?
It makes more sense when you know they're privacy activists first, businessmen second. But Mullvad shows you can be pro privacy and still offer great UX and a sleek site and client.
Btw, if you're managing things in CLI, you could take a look at their Hummingbird Suite. AFAIK it has a killswitch.
What sucks with Proton is that you can't share the VPN account with friends, because it is tied to your Proton account. They should create a vpn.proton.me subdomain that you can create a special managed account on that can only touch the VPN settings.
>Btw, if you're managing things in CLI, you could take a look at their Hummingbird Suite. AFAIK it has a killswitch.
Hummingbird doesn't support wireguard iirc, which is a deal breaker
I wish I could use Mullvad. But their IPs are banned from many streaming services and they don't change them often enough so I am stuck with Nord.
I would just pirate at that point. You're paying for the streaming service anyways. Use mullvad to download the torrent :). I'm pretty sure they ignore dmca requests. Not that they even know their customer's names if you pay with Mullvad amazon card.
Not to mention holding companies which snap up 15 competing VPNs and whitelabel most of them.
Mullvad seems to care and be competent about privacy, but most average VPN users aren’t seeking the most extreme privacy. They just want something cheap that lets them do geolocation things or access the most websites.
The average VPN user is knowledge-less. At best their internet usage data is being sold to third party analytics companies. At worst third parties are routing their own bots through their local connection.
You do know that NordSec maintains its own rust fork of BoringTun: https://github.com/NordSecurity/NepTUN ? :)
There is also https://github.com/firezone/boringtun which is a fork by https://www.firezone.dev/
I would love to see more root cause analysis data on the crashes they were seeing with wireguard-go. I wonder if it was bugs in the library itself, or the FFI.
For the similar reason I do not using any go based proxy code in my MintFlow app, and use rust to implement some proxy protocols.
But my app’s wireguard is natively implemented by fdio vpp plugin, so it’s based on C.
I would not have guessed that iOS allows enough access to APIs to implement anything vpp-based. Very cool to see. I also enjoyed working with vpp (for the brief 6 months that I had with it).
I was thinking that's hard, but I noticed that vpp get ported to FreeBSD using epoll shim library, and I learnt apple Darwin use some some userland of FreeBSD to do POSIX compatibility, then after some tests and hacking, most related to minor POSIX API adaptation such as mmap and one major coroutine need add some assembly code, and it work! But I think most disappointed to me is that apple do lack some vectorized network IO unless do some kernel extension or other sort non standard ways.
Is there any way to switch to this implementation for generic WireGuard users?
I tried downloading their Android app, but it's not generally usable for people who host our own WireGuard, which is fair enough.
The github repo is linked in the post which has build instructions: https://github.com/mullvad/gotatun
We rewrote it in Rust so you have to upvote it.
They didn't. They forked an old unmaintained thing already written in rust to add new features...
One meta thing I've always wondered ... Are multiple implementations of the same protocol good or bad for security?
Probably naively, I'm thinking:
- diversity: good
- doubling the attack surface: real bad
I think the general consensus is that it improves security of the protocol, but obviously that won't matter much if the implementation gets something wrong or has worse security by itself.
Issues in the protocol itself would need all implementations to change, but issues in the implementation would obviously be isolated to one implementation. For something like Wireguard, I'd wager a guess that issues in the implementations are more common than issues in the protocol, at least at this stage.
If the implementation gets it wrong that can also be a sign of ambiguity in the protocol / standard and as such result in clarifications and an overall more well specified protocol
I wouldn't say that multiple implementations are duplicating the attack surface since most users will not end up running them in parallel.
I meant at a global level (think as if you're attacking all wireguard users, not a single one)
The increased attack surface mostly only affects that one particular implementation though. So, yes, twice as many implementations that may contain exploitable bugs, but each new implementation could only be used to exploit a fraction of the total user base
> could only be used to exploit a fraction
If anything this is a even a good thing, since it means that each individual vulnerability an attacker finds is less valuable to them.
Diversity is a fantastic thing for security. It limits the impact when a bug drops and gives the possibility to migrate or run a mix of systems.
Competitions helps in multiple ways. It improve tooling, test suites, CVE response time, documentation and evolution of the protocol. There are some counter examples where compatibility suck, like DLNA but the problem often come from the spec.
That's really good because it means it will be able to have more exposure, more exposure means more improvement, more improvement eventually dig out bad bugs and reduces the attack surface in the long run
Is having Mac OS and Linux a decrease or increase in security over just having windows only?
dont fix if it ain't broken. look at sudo-rs and other rust ports.
ofc, thats a cynical view.
i personally think its a bad idea to duplicate efforts. better combine them. otherwise u risk making mistakes that were already solved. missing lessons already learnt.
sudo-rs itself is not a bad idea, Canonical’s premature shipping of it in Ubuntu was the bad idea. sudo-rs was transparent with how far it had gotten in compatibility and feature parity