OpenVPN and 10Gbps ?

K

Kanade

New member
Hello there,

the backbone of our AS is currently at 10Gbps, we got aroud 8-9Gbps in download/upload with 5Gbps guaranteed.
We connected a server to the backbone and this server is currently connected in 10Gigabit Ehernet Full Duplex. But we encounter a small problem.

We want to connect several data centers on the same IP block (yes, IPv4 is expensive, we only have a /24 IPv4 and a /29 IPv6 as LIR at RIPE, so we need to use all of our datacenters servers on the same IP block), and for that, we have chosen to configure an OpenVPN in layer 2. It's efficient, it works perfectly, the only problem is that... the routed speed does not exceed 1Gbps.
We tested everything: changing the encryption used, ciphers, OpenVPN versions on the client/server side, checking the hardware (we mainly use 10-Gigabit Ethernet, and more rarely SFP+), there is really nothing abnormal. Moreover, if I do an iperf between two sites, I exceed 5Gbps without any problem.

Conclusion, I think of a software limitation on the side of OpenVPN. Our configuration is totally classic, there is nothing exceptional about it, but we would like to know if anyone has already encountered this problem.

On the other hand, if it turns out that OpenVPN is limited to layer 2 transfers of 1Gbps maximum, which VPN software would you recommend? I was told about Wireguard but it seems to be layer 3, so it doesn't suit us. And we would preferably like it to be encrypted.

So, if anyone has anything to add...
 
What CPU are you using in this server to run OpenVPN? The encryption and decryption of OpenVPN is very CPU heavy. Do you have any monitoring capability to see how the hardware is performing while trying to push as much bandwidth as possible through the server?

If CPU is OK, you could look into using multiple OpenVPN tunnels. Obviously this means you would need to put something together to handle routing over the parallel connections.

Personally I would be looking to use something else rather than a OpenVPN implementation between servers. Do you own the equipment which connects the uplinks? Depending on what you're using, you could see if it's possible to implement some sort of stretched vlan feature?
 
OpenVPN is not the best software for high speed tunnels. Also, if encryption is a must, IPsec will be definitely a better option in a long-term, having the hardware offload on high end NIC, for example Mellanox ConnectX-6 Dx with crypto unit.

If you don't need any encryption, plain GRETAP (L2) tunnel will be better, however I would like to recommend you to use plain GRE (L3) with DMVPN feature, which is also called NVGRE and is offloadable to mid-range NIC as well.

The cheapest NIC won't have neither of offload, thus you may eventually encounter packet loss due to inefficient usage of a single core of your CPU, not multiple simultaneously as it is supposed to.
 
It seems like OpenVPN may have a software limitation for layer 2 transfers above 1Gbps. Have you considered looking into other VPN software options that may better suit your needs, such as Wireguard?
 
I also tried using openvpn for enterprise vpn/routing but we also had overhead limits.

The solution we came up with was the following:

For most of our enterprise users who want to do on site location hosting, we issued them a Edge Router ER-4 with custom OS running WireGuard. Doing this we are able to issue them routable ipv4 and ipv6 addresses. The issue in this case would be this device is limited to 1gbps per uplink.

When our local office was able to offer 10gb fiber I had the same issue, I was now limited by 1gbps but in my case it was due to the hardware.

We ended up getting a bare metal jumpbox (you can search 'intel jumpbox' for this on ebay). This solved the issue of being limited to 1gbps hardware. Next we installed a minimal OpenBSD and setup WireGuard.

We are currently able to get about 3.8gbps down and 4gbps up over the WireGuard tunnel.

If you need help with the config let me know, we have it working so far with the higher speeds on OpenBSD only at this time.
 
You must log in or register to reply here.

Supporters

Back
Top