While reviewing a co-worker’s results-paging design I realized there was a bug in some paging code I wrote¹ a few years ago. It’s unlikely to manifest and kind of subtle, but I thought that describing it here might be useful to others writing such code (including my future self). It comes down to sorting by timestamp…

These are some parenting tips/tricks/techniques that I’m capturing for a friend. I’m not pretending that the ideas are good or unique or will help anyone other that me, but… maybe?

Here’s a quick-and-dirty explanation of why two-factor authentication is good, and why U2F/WebAuthn keys (like YubiKeys) are better than the alternatives. (So I have something to point friends and family at.)

We all know by now that the leftmost values in the X-Forwarded-For header can be spoofed and only the rightmost IPs – added by your own reverse proxies – can be trusted. The Forwarded header (RFC 7239, 2014) has that same problem, and a new one: If the header is parsed correctly, an attacker can sabotage the whole header.

Let’s take a quick trip to understanding how that can happen and how complicated Forwarded parsing can get. (Think about how you’d parse the header as we go.)

There have recently been three different (but related) contexts where I have asked or been asked that question:

1. When a reverse proxy is adding the client IP to the X-Forwarded-For header.
2. When the client IP is being used for rate limiting.
3. When checking if a client IP is contained in a configured list of ranges/prefixes/CIDRs.

As I understood more about zones my opinion on this changed. This is an attempt to capture my understanding and where I ended up.

Update 2022-03-23: Matt Layher created a Go issue about this.

Update 2022-04-14: In response to that issue, two weeks ago a change was committed to Go that makes netip.ParsePrefix behave like net.ParseCIDR: they both return an error when a zone is present. It wasn’t released in 1.18.1, but I’m guessing it’ll be in 1.18.2. So that’s great!

Does this surprise you? (Try it in the playground.)

prefix := netip.MustParsePrefix("fe80::%zone/10")
addr := netip.MustParseAddr("fe80::1%zone")
fmt.Println(prefix.Contains(addr)) // ==> false

Go’s new-as-of-1.18 netip package is better in every way than the previous net.IP, etc., but this one design decision will probably burn someone, somewhere, sometime.

## Summary

This post ended up being incredibly long comprehensive. I’m afraid that many people won’t read enough to get everything that’s important, so here are the key points:

• When deriving the “real client IP address” from the X-Forwarded-For header, use the rightmost IP in the list.

• The leftmost IP in the XFF header is commonly considered to be “closest to the client” and “most real”, but it’s trivially spoofable. Don’t use it for anything even close to security-related.

• When choosing the rightmost XFF IP, make sure to use the last instance of that header.

• Using special “true client IPs” set by reverse proxies (like X-Real-IP, True-Client-IP, etc.) can be good, but it depends on the a) how the reverse proxy actually sets it, b) whether the reverse proxy sets it if it’s already present/spoofed, and c) how you’ve configured the reverse proxy (sometimes).

• Any header not specifically set by your reverse proxy cannot be trusted. For example, you must not check the X-Real-IP header if you’re not behind Nginx or something else that always sets it, because you’ll be reading a spoofed value.

• A lot of rate limiter implementations are using spoofable IPs and are vulnerable to rate limiter escape and memory exhaustion attacks.

If you use the “real client IP” anywhere in your code or infrastructure, you need to go check right now how you’re deriving it.

This is all explained in detail below, so keep reading. It’s a weird, scary, bumpy ride.

IPv6 rate-limiting is scarily half-baked right now. If you run a server that does any kind of IP-based rate-limiting, consider not enabling IPv6 if possible. If you do use IPv6, check how your rate-limiter actually handles it.

Every now and then I need to make a choice between using git submodules or subtrees (or nothing), or I get asked about them by coworkers. This is infrequent enough that I forget some of the details each time and need to refresh my memory. So I wrote up these notes to share with my coworkers and to help my future self. Hopefully they’re of some use to others as well.

Recently, I was adding timeouts to a Go HTTP server and ended up exploring how the different settings and approaches act and interact. I’m going to publish my notes here, along with the code I used for testing. Hopefully this will help someone else (or myself) in the future.

The timeout testing client can be found here: github.com/adam-p/httptimeout. There is a server in the examples directory that you can make requests to.

How should travel time be estimated? What are the ethical implications of the approach taken?

You enter your destination into your maps app. It finds a few likely routes. It determines the distance of each pretty easily. It checks traffic conditions along the routes. But we don’t yet have a travel time estimate. Time equals distance divided by speed, adjusted for traffic.

What travel speed (traffic notwithstanding) does the app use?

TL;DR: When trying to prevent timing attacks (e.g., against login username enumeration) by making a request take constant time, make sure you exclude the network read and write time. If you don’t, an attacker can slow down their request to bypass it.

I’ll be covering some background and contextual information here. If you don’t need it, skip to “Exclude network time from constant-time limiting”.

## What is a “timing attack”?

Briefly, a timing attack (in this context) is when an attacker observes the time it takes for a server to handle a request to glean some information about the validity of the input they tried. The typical target for this attack is the login request, and in that context there are – unsurprisingly – two pieces of information that can be attacked: username and password.

There’s a ton of “what to expect when you’re expecting” stuff out there that I have no intention of repeating. I found there was a lot of stuff I wasn’t prepared for – mentally or logistically – that I want to call out here.

A lot of what I’m going to say is negative. I feel like the negative experience for (some) fathers isn’t discussed very much and that that’s a major disservice to us – and a dangerous one, frankly.

For four years I carried a Yubikey NEO (USB Type-A) in my pocket, on my keychain. And then it died (would no longer be recognized by any computer).

Yubikey’s durability claim was:

Crush-resistant and waterproof, YubiKey NEO is practically indestructible during normal use, weighs only 3g, and attaches to your keychain alongside your house and car keys

So, they didn’t explicitly say “carrying it on your keychain won’t kill it”, but they sure did imply it.

I have enjoyed reading other people’s design and debugging train-of-thought posts, so after I spent two days wrestling with a code problem, I thought I’d write it up. It’s not technically exciting, but I think that describing it might be useful to someone – or my future self – someday. Or, at the very least, a little amusing.

(Bonus: While writing this I discovered an error I made while doing the actual work. See if you can spot it before I reveal it…)

In October 2017, Troy Hunt of Have I Been Pwned held a contest inviting people to do something cool with the HIBP API. I decided a) that I would kind of like the special edition ThinkPad he was giving away, and b) that I could probably whip something up pretty quickly.

I decided to create a browser extension that would simply pull HIBP breach information and show a browser notification – with the ability to view extra info – when the user visited a site that had been breached. And so was born the Breached extension. (Spoiler: I didn’t win.)

[This started as notes to myself to help clarify the problem and solution. It’s probably more suited to a Github issue than a blog post, and it may get copied into one.]

# The story so far

The Firefox and Thunderbird versions of Markdown Here both used nearly the same code – an old-style XUL extension. Tb is only capable of using a XUL extension, while Fx supports at least three extension types: XUL-based, Add-on SDK (aka Jetpack, aka jpm), and WebExtensions. WebExtensions is the newest, and is essentially an implementation of Chrome’s extension API.

While poking around in my Android phone’s developer options, I realized that if you steal a phone that’s currently unlocked because it’s in a “trusted place”, then you can force it to remain unlocked forever. (And then I got schooled about that not being a problem.)

## Security Feature: Smart Lock with Trusted Places

Android’s Smart Lock allows users to configure conditions under which to keep the phone unlocked. One of the conditions is location – you can set trusted locations where your phone shouldn’t prompt for a PIN/pattern/password when unlocking.

## Background and Motivation

I received a pull request for Markdown Here that was great: it found a bug, fixed it, and included tests for the fix. However, the PR submitter didn’t write the tests using the existing framework, so I figured I’d massage his test code into the proper form.

This is just a stub test post to allow me to try out Markdown Here in Disqus comments.

Right now MDH won’t work with Disqus in Chrome because of cross-origin restrictions. See: https://github.com/adam-p/markdown-here/issues/124

Update: The Disqus edit box is contenteditable, and MDH will render in it, but all formatting seems to get stripped out when you actually post the comment. Seems like the rich-edit-ness is probably just to support Disqus’s add-an-image feature.

Trying to get Markdown Here listed in the Safari Extensions Gallery is by far the worst browser extension “store” experience I’ve had so far. Shockingly bad.

## No hosting

First of all, but least of all: There’s no hosting. Unlike the Chrome and Mozilla stores, the Safari store doesn’t host the extension for you – it’s really more of a listing of links to wherever you host your extension files. That’s not terrible, but:

Page actions – the buttons in a browser’s address bar – are a surprising UI failure.

When adding a button for a browser extension, a choice must be made whether to make it a “page action” or a “browser action” (button on the toolbar). But browsers have failed to communicate the interactiveness of page actions, and almost no one – techy or layman – realizes that they’re clickable.

To complement the context menu item and hotkey, and to fulfil a user feature request, I decided to add a button to the Markdown Here browser extension. It turned out that simply deciding where to put the button was a big part of the effort…