We all know by now that the leftmost values in the X-Forwarded-For header can be spoofed and only the rightmost IPs – added by your own reverse proxies – can be trusted. The Forwarded header (RFC 7239, 2014) has that same problem, and a new one: If the header is parsed correctly, an attacker can sabotage the whole header.
Let’s take a quick trip to understanding how that can happen and how complicated Forwarded parsing can get. (Think about how you’d parse the header as we go.)
Update 2022-03-23: Matt Layher created a Go issue about this.
Update 2022-04-14: In response to that issue, two weeks ago a change was committed to Go that makes netip.ParsePrefix behave like net.ParseCIDR: they both return an error when a zone is present. It wasn’t released in 1.18.1, but I’m guessing it’ll be in 1.18.2. So that’s great!
Does this surprise you? (Try it in the playground.)
prefix := netip.MustParsePrefix("fe80::%zone/10")
addr := netip.MustParseAddr("fe80::1%zone")
fmt.Println(prefix.Contains(addr)) // ==> false
Go’s new-as-of-1.18 netip package is better in every way than the previous net.IP, etc., but this one design decision will probably burn someone, somewhere, sometime.
This post ended up being incredibly long comprehensive. I’m afraid that many people won’t read enough to get everything that’s important, so here are the key points:
When deriving the “real client IP address” from the X-Forwarded-For header, use the rightmost IP in the list.
The leftmost IP in the XFF header is commonly considered to be “closest to the client” and “most real”, but it’s trivially spoofable. Don’t use it for anything even close to security-related.
When choosing the rightmost XFF IP, make sure to use the last instance of that header.
Using special “true client IPs” set by reverse proxies (like X-Real-IP, True-Client-IP, etc.) can be good, but it depends on the a) how the reverse proxy actually sets it, b) whether the reverse proxy sets it if it’s already present/spoofed, and c) how you’ve configured the reverse proxy (sometimes).
Any header not specifically set by your reverse proxy cannot be trusted. For example, you must not check the X-Real-IP header if you’re not behind Nginx or something else that always sets it, because you’ll be reading a spoofed value.
A lot of rate limiter implementations are using spoofable IPs and are vulnerable to rate limiter escape and memory exhaustion attacks.
If you use the “real client IP” anywhere in your code or infrastructure, you need to go check right now how you’re deriving it.
This is all explained in detail below, so keep reading. It’s a weird, scary, bumpy ride.
IPv6 rate-limiting is scarily half-baked right now. If you run a server that does any kind of IP-based rate-limiting, consider not enabling IPv6 if possible. If you do use IPv6, check how your rate-limiter actually handles it.
Every now and then I need to make a choice between using git submodules or subtrees (or nothing), or I get asked about them by coworkers. This is infrequent enough that I forget some of the details each time and need to refresh my memory. So I wrote up these notes to share with my coworkers and to help my future self. Hopefully they’re of some use to others as well.