Some decent VPN/tunnel solution for the RPI?

[klesk]

I working on some sbc backup solution, preferrably through VPN however the RPIs(*) fail on this subject miserably due to the missing crypto capacity.

For example here are some results of a Raspberry PI v3 for AES-256

Code: Select all

Doing aes-256-gcm for 3s on 16 size blocks: 1651900 aes-256-gcm's in 3.00s
Doing aes-256-gcm for 3s on 64 size blocks: 478077 aes-256-gcm's in 3.00s
Doing aes-256-gcm for 3s on 256 size blocks: 126013 aes-256-gcm's in 3.00s
Doing aes-256-gcm for 3s on 1024 size blocks: 30857 aes-256-gcm's in 3.00s
Doing aes-256-gcm for 3s on 8192 size blocks: 3940 aes-256-gcm's in 3.00s
Doing aes-256-gcm for 3s on 16384 size blocks: 1958 aes-256-gcm's in 3.00s
OpenSSL 1.1.1s  1 Nov 2022
built on: Wed Nov  2 09:23:54 2022 UTC
options:bn(64,64) rc4(char) des(int) aes(partial) idea(int) blowfish(ptr) 
compiler: aarch64-linux-gnu-gcc -fPIC -pthread -Wa,--noexecstack -fstack-clash-protection -D_FORTIFY_SOURCE=2 -O2 -pipe -march=armv8-a  -g -I/usr/aarch64-linux-gnu/usr/include -fdebug-prefix-map=/builddir/openssl-1.1.1s=. -Wa,--noexecstack -fstack-clash-protection -O2 -pipe -march=armv8-a -g -fdebug-prefix-map=/builddir/openssl-1.1.1s=. -DOPENSSL_USE_NODELETE -DOPENSSL_PIC -DOPENSSL_CPUID_OBJ -DOPENSSL_BN_ASM_MONT -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DKECCAK1600_ASM -DVPAES_ASM -DECP_NISTZ256_ASM -DPOLY1305_ASM -DNDEBUG -D_FORTIFY_SOURCE=2 -I/usr/aarch64-linux-gnu/usr/include   
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
aes-256-gcm       8810.13k    10198.98k    10753.11k    10532.52k    10758.83k    10693.29k

Vs my Core i7 laptop:

Code: Select all

openssl speed -evp aes-256-cbc
Doing aes-256-cbc for 3s on 16 size blocks: 101015097 aes-256-cbc's in 2.97s
Doing aes-256-cbc for 3s on 64 size blocks: 38321830 aes-256-cbc's in 2.99s
Doing aes-256-cbc for 3s on 256 size blocks: 9957503 aes-256-cbc's in 2.98s
Doing aes-256-cbc for 3s on 1024 size blocks: 2505831 aes-256-cbc's in 2.99s
Doing aes-256-cbc for 3s on 8192 size blocks: 309705 aes-256-cbc's in 2.98s
Doing aes-256-cbc for 3s on 16384 size blocks: 157358 aes-256-cbc's in 2.98s
OpenSSL 1.1.0l  10 Sep 2019
built on: reproducible build, date unspecified
options:bn(64,64) rc4(16x,int) des(int) aes(partial) blowfish(ptr) 
compiler: gcc -DDSO_DLFCN -DHAVE_DLFCN_H -DNDEBUG -DOPENSSL_THREADS -DOPENSSL_NO_STATIC_ENGINE -DOPENSSL_PIC -DOPENSSL_IA32_SSE2 -DOPENSSL_BN_ASM_MONT -DOPENSSL_BN_ASM_MONT5 -DOPENSSL_BN_ASM_GF2m -DSHA1_ASM -DSHA256_ASM -DSHA512_ASM -DRC4_ASM -DMD5_ASM -DAES_ASM -DVPAES_ASM -DBSAES_ASM -DGHASH_ASM -DECP_NISTZ256_ASM -DPADLOCK_ASM -DPOLY1305_ASM -DOPENSSLDIR="\"/usr/lib/ssl\"" -DENGINESDIR="\"/usr/lib/x86_64-linux-gnu/engines-1.1\"" 
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
aes-256-cbc     544189.07k   820266.60k   855409.65k   858184.26k   851376.97k   865152.17k

This is 61X of the performance compared to the PI.

This shows up well on a VPN tunnel carrying capacity (openvpn).

RPI 3 test:

Code: Select all

[  1] 0.00-1.00 sec  2.63 MBytes  22.0 Mbits/sec
[  1] 1.00-2.00 sec  2.38 MBytes  19.9 Mbits/sec
[  1] 2.00-3.00 sec  2.62 MBytes  22.0 Mbits/sec
[  1] 3.00-4.00 sec  2.88 MBytes  24.1 Mbits/sec
[  1] 4.00-5.00 sec  2.12 MBytes  17.8 Mbits/sec

Core i7 laptop test:

Code: Select all

[  3]  0.0- 1.0 sec  7.88 MBytes  66.1 Mbits/sec
[  3]  1.0- 2.0 sec  5.25 MBytes  44.0 Mbits/sec
[  3]  2.0- 3.0 sec  4.12 MBytes  34.6 Mbits/sec
[  3]  3.0- 4.0 sec  5.25 MBytes  44.0 Mbits/sec
[  3]  4.0- 5.0 sec  5.25 MBytes  44.0 Mbits/sec
[  3]  5.0- 6.0 sec  5.38 MBytes  45.1 Mbits/sec
[  3]  6.0- 7.0 sec  4.12 MBytes  34.6 Mbits/sec

Results are of course a bit better on the PI4 due to higher CPU clock speed. (as I assume it does all the crypto operations from software). Lowering the encryption to AES 128 as others suggested as a tip for speeding up OpenVPN does not make any difference, neither the other openvpn tunings, I have tried them all.

I didn't open this topic for bashing on the PI.

Are there any other tunnel solutions which whilst still providing a decent encryption they can deliver throughput close to at least 100mbit/s?

Or should I just give up with this and look for other SBCs supporting AES_NI (or wait for the RPI5 which hopefully? will).

[epoch1970]

100 Mbps: over a LAN?

[graysky]

Openssl tests are relevant how? Have you tested a timed test moving your files from point A to point B (on the pi) without encryption? This will establish the max speed you can expect. I would then setup wireguard and repeat over the tunnel.

[redvli]

Have a look at https://www.wireguard.com/install/ Its core it multi-threaded, easy to get high speeds on a 4-core SoC.

I reached 150Mbps on just 1 RPi4 core with 40% load. 1 core because I ran it in a KVM with just 1 core allocated.

[bls]

Although my preferred VPN is strongSwan (IPSEC) because all client OS have the IPSEC client built-in, I’d recommend wireguard if your primary decision criteria is performance.

[carldani]

Although my preferred VPN is strongSwan (IPSEC) because all client OS have the IPSEC client built-in, I’d recommend wireguard if your primary decision criteria is performance.

I always wondered how Wireguard was supposed to have better performance than in-kernel IPsec with strongSwan. Sure, widely reported on the net, but usually it's the same ancient result by some Wireguard people everybody seems to refer to.

Then I stumbled upon these third-party measurements for IPsec, Wireguard and OpenVPN. https://rp.os3.nl/2019-2020/p71/presentation.pdf IPsec seems to have a throughput performance advantage over Wireguard, but a connection setup latency disadvantage. Both IPsec (strongSwan) and Wireguard beat OpenVPN by a large margin, but that's expected.

My own limited testing had similar results. Based on these results I built a mid-scale VPN of ~600 Raspberry Pi based on IPsec/strongSwan. (No, that VPN access is not for sale. Built for a non-profit with a tight budget for internal purposes.) I'm really happy with the performance and would build it again the same way. No passwords, just public key authentication. The one big thing I changed from the default RSA was the authentication algorithm based on desired security margin and measured CPU consumption.

[redvli]

Then I stumbled upon these third-party measurements for IPsec, Wireguard and OpenVPN. https://rp.os3.nl/2019-2020/p71/presentation.pdf IPsec seems to have a throughput performance advantage over Wireguard, but a connection setup latency disadvantage. Both IPsec (strongSwan) and Wireguard beat OpenVPN by a large margin, but that's expected.

Interesting presentation. I haven't had much focus on IPsec/StrongSwang, mainly because I had to deal with various mobile (3G/4G) connections. Some just don't support inbound connections, even though IPsec is really standard. Then OpenVPN and Wireguard are working alternatives.

[bls]

Although my preferred VPN is strongSwan (IPSEC) because all client OS have the IPSEC client built-in, I’d recommend wireguard if your primary decision criteria is performance.

I always wondered how Wireguard was supposed to have better performance than in-kernel IPsec with strongSwan. Sure, widely reported on the net, but usually it's the same ancient result by some Wireguard people everybody seems to refer to.

Then I stumbled upon these third-party measurements for IPsec, Wireguard and OpenVPN. https://rp.os3.nl/2019-2020/p71/presentation.pdf IPsec seems to have a throughput performance advantage over Wireguard, but a connection setup latency disadvantage. Both IPsec (strongSwan) and Wireguard beat OpenVPN by a large margin, but that's expected.

My own limited testing had similar results. Based on these results I built a mid-scale VPN of ~600 Raspberry Pi based on IPsec/strongSwan. (No, that VPN access is not for sale. Built for a non-profit with a tight budget for internal purposes.) I'm really happy with the performance and would build it again the same way. No passwords, just public key authentication. The one big thing I changed from the default RSA was the authentication algorithm based on desired security margin and measured CPU consumption.

Thanks for that presentation. I've seen a couple of other test results that aren't in line with these results, but probably worth pointing out that these results are likely NOT done on a Pi.

I set up a test bed a few months ago (2 routers and 2 Pi4s) and did some quick basic perf testing of strongSwan and Wireguard. WG showed much better performance on the Pi than strongSwan in my limited tests. This led me to do a look at the ciphers I'm using with strongSwan and I found that chacha was much faster than what I was using. Unfortunately, ran out of time to do any further testing on this. I still need to go implement/test the updated cipher (one line change, but building up the testbed again has apparently given me a mental block )

As far as your no password, public key auth only solution, you might find pistrong interesting. pistrong is an open source, public key cert manager using its own CA, and has scripts to build configurations for site-to-site and client-server tunnels. I'm personally aware of several long-term installations built on this, and there continues to be strong interest in it, so not everyone is flocking to WG

[memjr]

... the RPIs(*) fail on this subject miserably due to the missing crypto capacity.

....

This is 61X of the performance compared to the PI.

....

The Pi was originally made with a CPU meant to be used in routers. Their CPUs are better than that today, but still just one of those CPUs.

If you need your project to perform closer to an x86 i7 system, then a Pi is not what your project should be using.

Just because you can do it, it does not mean you should.

[redvli]

The Pi was originally made with a CPU meant to be used in routers. Their CPUs are better than that today, but still just one of those CPUs.

The CPU is just an ARM template, or I don't know what happened before 2010 (before the Pi1). Is it CPU or SoC you mean?. And why is there a complete graphics (master) domain in the '2835' series and no ethernet switch if meant for (headless) routers?
Pi SoC's are STB/TV-SoC AFAIK I know, no surprise Broadcom failed selling it in routers one could conclude. 'Older' router SoC's have some non-ARM HWblock usually to do various IP or network packet tasks which nowadays can be a PITA as no-one wants to put effort into merging all the developments in modern Linux network stack. ARMv8 version higher than what is in the Pi4 might have the proper instructions, I don't know by heart. Companies that have ARM architectural license like Apple and Qualcomm should be way ahead of what is in the RPi.

[memjr]

The Pi was originally made with a CPU meant to be used in routers. Their CPUs are better than that today, but still just one of those CPUs.

The CPU is just an ARM template, or I don't know what happened before 2010 (before the Pi1). Is it CPU or SoC you mean?. And why is there a complete graphics (master) domain in the '2835' series and no ethernet switch if meant for (headless) routers?
Pi SoC's are STB/TV-SoC AFAIK I know, no surprise Broadcom failed selling it in routers one could conclude. 'Older' router SoC's have some non-ARM HWblock usually to do various IP or network packet tasks which nowadays can be a PITA as no-one wants to put effort into merging all the developments in modern Linux network stack. ARMv8 version higher than what is in the Pi4 might have the proper instructions, I don't know by heart. Companies that have ARM architectural license like Apple and Qualcomm should be way ahead of what is in the RPi.

Yes, SoC is what I meant.

The point was to take an SoC that could be used in a tiny computer that could be sold for a low price.

That meant the chosen SoC would be lacking some stuff, in this case, encryption hardware was one of them.

The Pi was built and could be sold for $35. It was meant for people to take a tiny board, plug it to a mouse, keyboard, tv set and make something with it. Goal accomplished.

The hobby and hacking communitey just liked it so much that we started to push harder and harder by making things like container stacks, vpn, nas, openwrt, etc, etc, etc, on them.

So when people talk about how running a VPN (or whatever else) on a Pi sucks compared to an x86, they're saying without much of a valid reason. It is not the Pi's or it's designer fault that people try to make it run things it was never meant to run.

If it does not do XYZ, then you are trying to use the wrong thing for XYZ.

[swampdog]

I'm signed up for 100Mbit with my ISP. They can't deliver enough throughput to stress my single core rpi gateway. It's so old I no longer recall what it is. Someone else will know..

Code: Select all

 $ cat /proc/cpuinfo 
processor	: 0
model name	: ARMv6-compatible processor rev 7 (v6l)
BogoMIPS	: 2.00
Features	: half thumb fastmult vfp edsp java tls 
CPU implementer	: 0x41
CPU architecture: 7
CPU variant	: 0x0
CPU part	: 0xb76
CPU revision	: 7

Hardware	: BCM2708
Revision	: 000f
Serial		: 00000000d5603b60

It's running 'iptables' (for NAT) and 'sshd'. eth0 is connected to my cable modem. eth1 is a 100Mbit dongle on my intranet.

Test with 'ssh' port forwarding. eg:

Code: Select all

#!/bin/bash
IP="home ip"
ssh \
	-L 10139:smb:139 \
	-L 20139:smb1:139 \
	-Y someuser@"$IP"

..would map both my samba servers onto (say) my laptop when away from home.

Don't forget things are backward in that your ISP *upload* speed is what counts.