SYN flood: understanding and mitigating the attack with XDP in 2026

A SYN flood XDP defense is the fastest way to drop half-open connection storms before they exhaust your server's backlog queue, and in 2026 it remains the single most effective kernel-layer technique to survive volumetric attacks that target the TCP handshake. Unlike user-space daemons or iptables rules invoked after the network stack has already allocated memory, an XDP program attached to your NIC driver processes every packet in a few hundred nanoseconds and decides—block, rate-limit, or pass—before the kernel's conntrack or socket layer ever wake up. This article explains why traditional SYN cookies and firewall rules still leave you vulnerable, how XDP short-circuits the attack surface, and the exact nftables complements you need to harden stateful edge cases without breaking legitimate traffic.

Why SYN Floods Still Succeed Against Traditional Firewalls

The three-way handshake allocates kernel memory the moment a SYN arrives: the server writes an entry into the half-open connection table, replies with SYN-ACK, then waits. A flood that sends ten million SYN packets per second with randomized source IPs will fill that table in milliseconds, triggering either a kernel panic or aggressive SYN cookie fallback that degrades performance for every legitimate client.

Traditional iptables -m conntrack --ctstate NEW rules fire after Netfilter has already walked the packet through the PREROUTING chain and allocated a conntrack entry. Rate-limiting with -m limit or -m recent works, but each lookup traverses a linked list or hash table in kernel space—acceptable for thousands of packets per second, catastrophic when the attack peaks at several million pps. Even SYN cookies, enabled via net.ipv4.tcp_syncookies=1, offload state to the cookie itself but force the server to compute cryptographic hashes for every inbound SYN, burning CPU cycles and adding latency to legitimate handshakes.

The root problem is placement: by the time a packet reaches Netfilter or the TCP stack, the kernel has already performed expensive operations—SKB allocation, protocol demux, routing lookup. A SYN flood XDP hook executes in the driver's NAPI poll loop, before any of those steps, and can drop malicious packets with a single map lookup and a return code.

How XDP Drops SYN Floods in the Driver Layer

XDP (eXpress Data Path) is a kernel framework that lets you attach eBPF bytecode directly to a network interface. When a packet arrives, the NIC driver invokes your XDP program before allocating the sk_buff structure; you inspect raw Ethernet, IP, and TCP headers in a linear buffer, consult BPF maps for policy, and return one of five verdicts:

• XDP_DROP – discard immediately, zero further processing.
• XDP_PASS – hand off to the normal network stack.
• XDP_TX – bounce the packet back out the same interface (useful for reflection).
• XDP_REDIRECT – forward to another interface or CPU.
• XDP_ABORTED – trace error and drop.

A minimal SYN flood filter reads the TCP flags field; if SYN is set and ACK is clear, you increment a per-source-IP counter in a BPF hash map and compare it against a threshold. If the IP has sent more than N SYN-only packets in the current time window, return XDP_DROP. Because this logic runs in fewer than a microsecond per packet and requires no conntrack state, a single core can trivially handle several million pps without breaking a sweat.

Stateless vs. Stateful Rate-Limiting

XDP programs are stateless by design: each invocation sees one packet, with no automatic memory of previous packets. To implement rate-limiting you maintain counters in BPF maps, but those counters are eventually consistent—updated on every packet, read without locks, and subject to per-CPU batching for performance. This trade-off is acceptable for SYN floods because you care about aggregate volume, not perfect per-flow accounting.

For stricter per-connection rate-limits—"allow no more than 10 new connections per second from the same IP to the same destination port"—you layer nftables on top. The meter feature in nftables tracks a dynamic set of flows and applies burst limits with full conntrack context:

nft add rule inet pakkt input tcp dport 25565 tcp flags syn \
  meter syn_rate { ip saddr limit rate 10/second burst 20 packets } accept

This rule allows a legitimate client to open ten new connections per second with a burst allowance of twenty, then drops excess SYNs. Because it runs in the INPUT chain—after XDP has already culled the bulk of the flood—nftables processes only the subset of packets that survive the first line of defense, keeping CPU usage manageable.

Building a Dual-Layer SYN Defense: XDP + nftables

A production-grade architecture combines three elements: global packet-rate enforcement in XDP, per-port SYN rate-limiting in nftables, and conntrack-aware drop rules for invalid flag combinations. Here is the workflow:

1. XDP global PPS cap – if the interface receives more than max_pps across all protocols, drop excess packets randomly or via a token bucket. This prevents a flood from saturating the NIC or exhausting interrupt capacity.
2. XDP per-port PPS cap – each rule in the BPF map specifies a destination port range and a max_port_pps. A SYN flood targeting port 25565 will hit the port-specific threshold before the global cap, isolating the attack to one service.
3. nftables SYN meter – the rule above, keyed on source IP and destination port, enforces strict per-flow rate-limits with burst tolerance.
4. nftables invalid-flag drop – reject packets with SYN+FIN, SYN+RST, or other impossible combinations that scanners use for OS fingerprinting:

nft add rule inet pakkt input tcp flags \& (fin|syn|rst|psh|ack|urg) == (fin|syn) drop
nft add rule inet pakkt input tcp flags \& (syn|rst) == (syn|rst) drop

By running these checks in a dedicated inet pakkt table, you avoid conflicts with Docker's DOCKER-USER chain, fail2ban's dynamic blacklist insertions, or any legacy iptables-persistent rules. The kernel processes the pakkt table first; if a packet survives, it continues through the filter table where your application-specific rules live.

Why You Cannot Rely on SYN Cookies Alone

SYN cookies elegantly solve the state exhaustion problem by encoding connection parameters—MSS, timestamp, window scale—into the initial sequence number, eliminating the need for a half-open queue entry. However, they come with three penalties:

• CPU overhead – every SYN triggers an MD5 or SipHash operation; at five million SYN/s, you spend multiple cores just computing cookies.
• TCP option loss – because the server discards state, it cannot remember window scale or SACK-permitted options until the client sends the final ACK. This degrades throughput for high-BDP paths.
• No early filtering – SYN cookies activate after the packet has traversed the network stack. If the flood contains malformed headers or targets a closed port, the kernel still wastes cycles generating a cookie and sending SYN-ACK into the void.

XDP short-circuits all three issues: malformed packets never reach the TCP layer, closed ports are blackholed before cookie computation, and your CPU budget remains available for actual application logic.

Deploying and Monitoring SYN Flood XDP Rules in Production

Manual XDP development requires writing C code, compiling with Clang/LLVM, and loading the object file via ip link set dev eth0 xdp obj program.o or bpftool. Debugging involves reading bpf_trace_printk output in /sys/kernel/debug/tracing/trace_pipe and correlating interface statistics (ip -s link show eth0) with application logs—a workflow that works for kernel hackers but scales poorly across a fleet of game servers, API endpoints, or SaaS instances.

PAKKT.io solves the operational burden by shipping a single pre-compiled XDP program—the PAKKT Engine—that reads up to 256 rules from a shared BPF map. Each rule specifies a port range, protocol mask (TCP/UDP/ICMP/any), rate-limit threshold, minimum and maximum packet size, and a rule type (block, rate_limit, allow_only). You configure rules in the web dashboard; the lightweight Go agent running on your server updates the map in real time, zero downtime, zero kernel recompilation.

For SYN flood defense, you create a rule targeting your application port—say TCP 25565—with rule_type: rate_limit, max_port_pps: 50000, and optionally a minimum packet size to ignore tiny ACK floods. The engine increments a per-rule counter on every matching packet; when the rate exceeds the threshold, subsequent packets receive XDP_DROP until the next one-second window. Metrics flow into a TimescaleDB hypertable, so your dashboard graphs dropped vs. passed packets per port, per rule, and per source country in real time.

Complementary nftables Rules for Stateful Edge Cases

The PAKKT agent also writes nftables rules into the isolated inet pakkt table. Because XDP is stateless, it cannot distinguish a SYN retransmission from a new connection attempt by the same IP. For stricter enforcement, you add a meter rule:

nft add rule inet pakkt input tcp dport 25565 tcp flags syn ct state new \
  meter syn_per_ip { ip saddr limit rate 20/second burst 40 packets } accept
nft add rule inet pakkt input tcp dport 25565 tcp flags syn ct state new drop

The first rule allows twenty new connections per second per IP, with a burst of forty; the second rule drops any SYN that exceeds the meter. Combined with the XDP cap, you achieve defense in depth: the bulk of the flood dies in the driver, and any residual spikes hit the nftables meter before reaching the application's listener.

Observability: GeoIP, Top Offenders, and Audit Logs

PAKKT integrates MaxMind GeoLite2 to map source IPs to countries and plot attacks on a world map. When a SYN flood originates from a botnet spanning fifty countries, the dashboard highlights the top five by packet count, letting you decide whether to blacklist entire ASNs or tighten rate-limits. Every rule change—creation, update, deletion—appears in the audit log with timestamp, user, and agent ID. Internal agent logs capture XDP attach/detach events, map update errors, and nftables syntax warnings, so you can diagnose misconfigurations without SSH.

The PAKKT Integrations page documents the Pterodactyl v1.x plugin, which surfaces per-server drop statistics directly in the game panel. WHMCS and Pelican integrations are in development, extending the same mTLS-authenticated API to billing and orchestration workflows.

Performance Benchmarks and Real-World Capacity

Because XDP operates in the driver's NAPI context, per-packet overhead is measured in hundreds of nanoseconds. A single CPU core running a 3 GHz processor can process several million packets per second when the XDP program is a tight loop of header parsing, map lookup, and verdict return. Real-world capacity depends on NIC offload features (RSS, flow director), NUMA topology, and whether you enable XDP_REDIRECT to distribute packets across cores, but even a modest dual-core VPS can absorb SYN floods in the 5–10 Mpps range before the bottleneck shifts to memory bandwidth or interrupt coalescing.

Contrast this with iptables -m recent, which on the same hardware tops out around 200,000 pps before lock contention and list traversal dominate CPU profiles. The difference is architectural: XDP skips the entire network stack, while iptables is a hook inside ip_rcv, invoked after SKB allocation and routing decisions.

Overhead of the PAKKT Engine

The PAKKT Engine XDP program loops over an array of up to 256 rules, checking port ranges and protocol masks. Each iteration involves two BPF map lookups—one for the rule definition, one for the per-rule counter—and a handful of integer comparisons. In practice, the agent reports < 1 % CPU usage and < 5 MB RAM footprint on a typical game server, even under sustained multi-gigabit floods. Because the engine is written in restricted C and verified by the kernel's eBPF verifier, it cannot crash the kernel or leak memory.

Avoiding Common Pitfalls

Three mistakes consistently trip up first-time XDP deployments:

• Blocking all SYN packets – if you set max_port_pps too low or forget to carve out an allow rule for your monitoring infrastructure, you will lock yourself out. Always whitelist your management IPs in both the XDP map and the nftables table.
• Ignoring SYN-ACK reflections – some amplification attacks spoof your server's IP as the source, causing SYN-ACK replies to flood back. XDP can detect unexpected SYN-ACK on a port that isn't listening and drop them, but you must explicitly write that logic or rely on nftables ct state invalid drop.
• Mixing XDP drivers – not all NICs support native XDP; some fall back to generic XDP, which runs after SKB allocation and negates the performance win. Verify ip link show dev eth0 reports xdp or xdpgeneric; if generic, consider a driver upgrade or a different NIC.

Testing Your SYN Defense Without Taking Down Production

Use hping3 from a test VM to generate controlled SYN floods:

hping3 -S -p 25565 --flood --rand-source target.example.com

Monitor bpftool prog show to confirm the XDP program is attached, then check bpftool map dump name pakkt_counters to see per-rule drop counts increment. Compare those numbers with nftables counters (nft list ruleset) and your application's connection logs to verify legitimate traffic still completes the handshake.

For a deeper dive into kernel-level protection patterns and integration guides, explore the PAKKT Blog, which covers everything from ICMP rate-limiting to UDP amplification mitigation.

Conclusion

A SYN flood XDP program stops volumetric handshake attacks at the earliest possible layer, dropping millions of packets per second with sub-microsecond latency and negligible CPU overhead. By pairing stateless XDP rate-limits with stateful nftables meters and invalid-flag rules, you build a defense-in-depth posture that survives both carpet-bomb floods and low-and-slow connection exhaustion. Observability—GeoIP mapping, per-port metrics, audit logs—transforms raw drop counts into actionable intelligence, letting you refine thresholds and blacklist repeat offenders without guesswork. Whether you run game servers, API endpoints, or SaaS infrastructure, kernel-layer protection is no longer optional; it is the baseline for uptime in 2026.

FAQ

Can XDP rate-limit SYN floods per source IP without nftables?

Yes, you can maintain a BPF hash map keyed by source IP and increment counters on every SYN packet, then return XDP_DROP when the counter exceeds your threshold. However, XDP is stateless and cannot track connection state or TCP flags beyond the current packet, so you lose the ability to enforce per-flow burst allowances or distinguish SYN retransmissions from new connection attempts. For stricter per-IP limits with burst tolerance, layer an nftables meter rule on top.

Will enabling SYN cookies make XDP unnecessary?

No. SYN cookies prevent state exhaustion but activate after the packet has already traversed the network stack, consumed CPU for routing and conntrack, and triggered cookie computation. A multi-million-pps SYN flood will peg your CPU generating cookies and sending SYN-ACK replies, degrading performance for legitimate clients. XDP drops attack packets in the driver before any of that overhead, leaving your CPU free for application work.

How do I verify my NIC supports native XDP and not generic fallback?

Run ip link show dev eth0 and look for the xdp or xdpoffload flag. If you see xdpgeneric, the kernel is running your program after SKB allocation, which negates the performance benefit. Check your NIC driver documentation—Intel i40e, ixgbe, Mellanox mlx5, and Netronome nfp all support native XDP. If your driver does not, consider a NIC upgrade or contact your hosting provider for hardware options that include XDP-capable interfaces.