There are many good people in the world, but there are also bad people, and the internet is no exception, so no matter where you are, there is a need for safe internet access. In mainland China, for reasons that cannot be described, this need is especially great.
With everyone using their own tricks, there are many ways to access the internet safely on the market, and the number of methods keeps growing.
At the beginning, and the easiest method for people to find, was using HTTP proxies. You could also find many free servers online. In the era before HTTPS was fully widespread, this basically meant handing over all your information to the proxy server, while communication between you and the proxy server was also in plaintext. Even today, when HTTPS is relatively widespread, an HTTP proxy still exposes the domain names you visit. Similar to this are plaintext SOCKS proxies.
About nine years ago, the famous Shadowsocks project was launched, ushering in the era of encrypted proxies. At first, the project was very successful, but because of certain issues with its use of cipher suites, Shadowsocks became easy to detect. And although the protocol did not have especially distinctive handshake features, it was like a van driving on the road with blacked-out windows: Shadowsocks traffic was easy to arouse suspicion and get blocked. As a result, various obfuscation methods also appeared for Shadowsocks. Similarly, there was the V2Ray project. Although it solved many of Shadowsocks' problems and was much more powerful, it was also excessively complex to configure. In recent years, the Trojan project also appeared, directly using TLS for camouflage, but it likewise failed to escape the problem of overly complex configuration.
On the VPN side, there are also many solutions on the market. A representative example is PPTP, but due to security issues, PPTP is now rarely seen. Another is L2TP/PSK-IPSec. This protocol still exists widely, but because IPSec handshakes have various issues or get blocked, it is not especially stable. In recent years, Wireguard has won over many people with its concise and elegant design, and its performance is excellent. But like other VPNs, Wireguard lives a hard life because UDP is a second-class citizen on China's internet. And Wireguard also has obvious packet characteristics, making it easy to identify. There are actually TLS-based VPN solutions too, namely Microsoft's SSTP, but this solution is relatively difficult to deploy. Natively, you need Windows Server, and there are not many open-source implementations either (SoftEther is one of them). It is also hard to hide the exposed port running the VPN service.
On the path of hiding the intent to use a proxy or VPN and avoiding ISP QoS, both proxies and VPNs ultimately point toward TLS, disguising themselves as normal website services. But neither side has an especially good implementation, so recently I spent some time thinking and wrote a lot of tools myself. As of today, I have found that writing these tools is actually not that hard, and some of the tools mentioned above, in my opinion, are too big and too comprehensive, making things overly complicated. To implement these tools, the core really does not require that much code or logic.
At the very beginning, what we used was a normal HTTP proxy, but in fact the connection to the proxy server can also be established over TLS, which hides the fact that a proxy is being used, so I wrote the go-shp project. There were already server-side implementations available—for example, Caddy 1.x—but there were not many clients that supported it. Operating systems didn't support it, and among browsers I only saw Chrome support it (Firefox might also support it), so I also hacked together a local forwarding proxy. And since Chrome supports it, I wrote a browser extension too, though I took too big a step: the function to automatically detect and use a proxy involved a lot of code, yet I still didn't write it very well.
However, HTTP proxies are inherently unable to forward traffic other than TCP. When I have that kind of need, I use Wireguard. Using native Wireguard directly is indeed too easy to recognize, so I hacked together udp-xor. But plain XOR still has recognizable features, so while practicing Rust, I wrote the udp-prepend project. UDP is nice, but because of QoS, I wrote ws-udp to stuff UDP traffic into WebSocket, which then conveniently allows the use of TLS camouflage. Of course, this inevitably introduces the TCP-over-TCP problem, but there aren't many good solutions. Still, Wireguard already has one layer of encryption, and adding TLS WebSocket on top really annoyed me, so I started missing the goodness of SSTP. But as mentioned earlier, there wasn't any especially good implementation on the market, so I wrote ws-tun as well.
At one point I wanted to implement an SSTP server, but the protocol is still a bit complicated, and it's not easy to make it blend in with normal traffic. For this reason, I chose to directly build my own tun or tap VPN. Honestly, I did find some open-source WebSocket implementations, but one used a niche language, and I gave up on another because it didn't support TLS. Besides, they were indeed written a bit too complicatedly. Between tun and tap, I didn't think tap was necessary, and it also requires root privileges to run, so I decided writing a tun implementation would be enough. Choosing WebSocket to transport tun packets was natural; after studying the WebSocket protocol, I found that its overhead is not too large, and with it I didn't need to implement my own fragmentation logic. I also didn't write the tun handling myself—I basically copied Cloudflare's boring-tun project and modified it until it worked. The only really painful part was the Rust development process: getting it to compile was miserable, and converting it to async also cost me a lot of time. This project has been tested to run harmoniously on macOS and Linux. As for configuration, the server and client only need to agree on one WebSocket address, and nothing else needs to be configured. I put both the server and client into the same program, which actually was not an especially good choice, because I designed the server to sit behind something like nginx, so TLS encryption does not need to be configured there, yet this still makes the server binary rather large. The client has no choice and must include the packages needed for TLS.
But the mobile path for ws-tun has not been so easy, because I had never written mobile apps before. This week, though, I spent a little time writing ws-tun-android, and I discovered that Android-side VPN development is actually very simple. Google provides a ToyVPN project, and after modifying it a bit, it worked. But when I was about to write the iOS version last night, I found it was not so easy. First, a developer account is mandatory, and then NetworkExtension seems to be only for enterprise users now? So I GG'd and took my leave.
Anyway, after some time, maybe I won't need to keep tinkering with this anymore. Let this article serve as a summary of many years of tinkering with networking—from using other people's services, to using other people's code to set up a service for myself, to finally writing code myself to implement one. Thanks, GFW, for teaching me a lot.