VPC NAT Gateway#

Overview#

The VPC NAT Gateway (VpcNatGateway CRD) provides iptables/nftables-based NAT for Kube-OVN VPCs: SNAT, DNAT, floating IPs, and EIPs on an external network interface. The controller manages the lifecycle of the gateway pod workload (StatefulSet for single-replica, Deployment for HA), configures network interfaces inside the pod via nat-gateway.sh, and reconciles EIP/SNAT/DNAT resources.

Key source files:

• Controller: pkg/controller/vpc_nat_gateway.go
• EIP controller: pkg/controller/vpc_nat_gw_eip.go
• Pod annotation helpers: pkg/util/vpc_nat_gateway.go
• In-pod NAT script: dist/images/vpcnatgateway/nat-gateway.sh

Workload Mode: StatefulSet vs. Deployment#

The controller selects the workload type based on gw.Spec.Replicas via util.IsNatGwHAMode(gw) 1:

• Single replica (default): StatefulSet with replicas=1 and a static lanIP annotation.
• HA mode (replicas > 1): Deployment with configurable replicas 2, required pod anti-affinity across nodes 3, and no static lanIP annotation — skipped for HA gateways to avoid IP conflicts 4.

Initialization Race: Multus net1 Interface#

The issue: When a NAT gateway pod starts (or restarts after a node reboot), the controller calls nat-gateway.sh init before Multus has finished attaching the secondary network interface (net1 by default). The controller only waits for PodRunning phase — not for all CNI attachments to complete 5.

Error signature:

failed to ExecuteCommandInContainer errOutput: Cannot find device "net1"
failed to init route table, err: Cannot find device "net1"

nat-gateway.sh init validates that both VPC_INTERFACE (default eth0) and EXTERNAL_INTERFACE (default net1) exist via ip link show before proceeding 6. The script exits non-zero if either interface is missing, causing the controller to requeue — but the requeue interval may not align with Multus attachment completion.

In non-primary CNI mode (Kube-OVN as secondary CNI), interface names are extracted at runtime from the pod's network-status annotation before being passed to the init script 7. If this annotation is not yet populated when the controller reconciles, initialization fails with the same error.

Workaround: Delete and recreate the affected service or VpcNatGateway to trigger a fresh initialization attempt once interfaces are ready.

Secondary-CNI Mode: lanIP Injection Bug#

Issue (#6744): When Kube-OVN operates as a non-primary (secondary) CNI alongside Cilium or another primary CNI, the lanIP is not injected into the NAT gateway pod. The pod's eth0 is managed by the primary CNI; the OVN interface that should carry lanIP is never created 8.

Root cause: GenNatGwPodAnnotations() deliberately skips the v1.multus-cni.io/default-network annotation in non-primary CNI mode 9 to preserve eth0 for the primary CNI. However, without v1.multus-cni.io/default-network set, the OVN subnet's NAD is also not added to k8s.v1.cni.cncf.io/networks, so the corresponding interface is never created and lanIP is never injected 8.

Workarounds:

1. Manually add v1.multus-cni.io/default-network: "<namespace>/<nad-name>" to the NAT gateway StatefulSet (confirmed working).
2. Add the NAD reference via spec.annotations on the VpcNatGateway CR using k8s.v1.cni.cncf.io/networks (field introduced in PR #6256; verify CRD is updated to v1.16.1+) 8.

This is a known architectural limitation; related issue: #6632.

HA Mode: EIP Distribution Across Replicas#

In HA mode, EIPs use a broadcast model: createEipInPod() retrieves all running gateway pods via getNatGwPods() and applies the EIP configuration to every pod in the Deployment. There is no replica-ordinal or pod-index assignment.

All replicas receive identical EIP configuration, enabling an active-active pattern where any pod can handle outbound NAT for any EIP. EIP resources carry SubnetNameLabel, VpcNatGatewayNameLabel, and EipV4IpLabel labels but no per-pod selector 10.

nat-gateway.sh Command Reference#

The script (dist/images/vpcnatgateway/nat-gateway.sh) is invoked by the controller via kubectl exec inside the gateway pod. Key commands:

Interface defaults: VPC_INTERFACE=eth0, EXTERNAL_INTERFACE=net1. Overridable via environment variables or the persistent file /etc/kube-ovn/nat-gateway.env 17.