GRE, FOU, and NAT Traversal

Design notes for carrying GRE through UDP, coordinating peers, and validating tunnel behavior on Linux.

Linux ยท Networking

GRE is useful because it carries arbitrary L3 payloads with a small header. NATs are hostile to plain GRE because GRE is IP protocol 47, not TCP or UDP, and many middleboxes only track transport-layer tuples. Foo-over-UDP (FOU) wraps GRE in UDP so the path can look like ordinary UDP flow state to NAT and firewall devices.

Define the packet stack

A GRE-over-FOU packet is conceptually:

outer IP
  UDP
    GRE
      inner IP
        payload

The outer tuple is what NATs and firewalls usually understand. The inner packet is what the tunnel endpoint routes. The design has to account for both planes:

  • outer address discovery and reachability,
  • UDP source port stability,
  • GRE key or peer identity when multiplexing,
  • inner addressing and routing,
  • MTU and fragmentation,
  • keepalive cadence,
  • authentication of peer discovery messages.

If the tunnel works only on a flat LAN, it has not proven the NAT traversal design.

Separate discovery from data

The data plane should stay simple: encapsulate and forward. Discovery can be stateful and policy-heavy. A coordinator or rendezvous service can observe peer reflexive addresses, exchange candidates, and help both sides send packets at the right time.

Discovery messages need integrity. Without it, an attacker can try to redirect peers, inject stale endpoint mappings, or force traffic toward a victim. Include peer identity, nonce, expiry, protocol version, and covered endpoint data in the signed or MACed envelope.

Hole punching is timing-sensitive

UDP hole punching depends on both peers creating NAT state before the mapping expires. The implementation should record:

  • local socket bind address and port,
  • server-observed reflexive address,
  • peer candidate address,
  • first outbound punch time,
  • first inbound packet time,
  • mapping timeout estimate,
  • NAT behavior when source ports change.

Some NATs preserve source ports. Some do not. Some require traffic to the exact remote tuple before inbound packets are accepted. Symmetric NATs are often not punchable without relay.

MTU is part of correctness

Encapsulation consumes bytes. If the underlay MTU is 1500, the tunnel cannot safely pass a 1500-byte inner packet without fragmentation. Set the tunnel MTU lower and test with DF behavior.

A practical validation set:

  • small ICMP echo through the tunnel,
  • near-MTU ICMP echo with DF set,
  • TCP connection through the tunnel,
  • sustained UDP stream,
  • route removal and reconnect,
  • peer restart while the virtual interface remains configured.

Packet capture on both outer and inner interfaces is the fastest way to see whether failure is routing, encapsulation, NAT, or application behavior.

Linux control plane

On Linux, tunnel setup often touches:

  • netlink link creation and configuration,
  • FOU port registration,
  • address assignment,
  • routes and policy routing,
  • firewall allowances,
  • sysctls for forwarding when routing between interfaces.

Shelling out to ip is useful for exploration. A daemon should prefer netlink so it can handle idempotency, errors, and cleanup deliberately. Every operation needs a rollback path because half-created network state can block the next run.

Keepalives and liveness

A UDP tunnel needs liveness at two levels:

Layer Signal
NAT mapping periodic small packet on the outer tuple
peer process authenticated heartbeat or data-plane response
inner route tunnel interface state and successful payload exchange
application service-level health through the tunnel

Do not conflate all of these into one boolean. A NAT mapping can be alive while the inner route is wrong. A peer can be alive while the application is down.

Design Verification and Test, Measurement and Instrumentation, and io_uring Fixed Buffers and Parser Fuzzing.

Sources