Gmail said 403 when it meant slow down

A first read of a mailbox kept stopping partway. The logs said provider_list_403, and the job gave up on that mailbox until someone looked.

The error that meant “wait”

An HTTP status is a three-digit code a server sends back with its answer. A 429 means “too many requests, try later”. A 403 means “you may not do this”. Our code waited on a 429 and failed on a 403. But Google’s guide to its errors says a rate limit can arrive as either one. A 403 can also mean a daily quota, a domain policy or a missing permission, and only a short reason code in the body says which.

So I kept the reason code and nothing else from the body. The code is accepted only if it is a short run of letters:

const reasonOf = (e: unknown): string | null => {
  const errors = (
    e as { response?: { data?: { error?: { errors?: unknown } } } }
  )?.response?.data?.error?.errors;
  const reason = Array.isArray(errors)
    ? (errors[0] as { reason?: unknown } | undefined)?.reason
    : undefined;
  return typeof reason === "string" && /^[A-Za-z]{1,40}$/.test(reason)
    ? reason
    : null;
};

Two of those codes, rateLimitExceeded and userRateLimitExceeded, now count as “wait and try again”. Every other 403 still fails, and its failure code now names the reason, so the next one is easier to read.

Not meeting the limit at all

Waiting out a limit is the fallback. The better fix is to stay under it. Gmail gives each user a number of quota units a minute, and reading one message costs a fixed share of them. The quota page has the figures. Our first read went as fast as Gmail answered, which is well past that ceiling, so it hit the limit every minute.

A pacer now spaces the reads for each account. It claims its slot before it sleeps, so calls that arrive together queue up instead of all waking at once:

const interval = 60_000 / perMinute;
const next = new Map<string, number>();
return {
  async wait(key) {
    const now = clock.now();
    const at = Math.max(now, next.get(key) ?? now);
    next.set(key, at + interval);
    if (at > now) await clock.sleep(at - now);
  },
};

The tests run it on a fake clock, so they check the spacing without sleeping.

The same week’s review found a gap. Pacing says how fast a step reads, not how many messages it reads, and one page of changes can name several. A step now has a budget of reads. When it runs out partway through a page, it leaves its place where it was. The next step lists the page again and reads only what is not already stored.

Why it matters

A new mailbox now finishes its first read in one steady run, without stopping on a warning that only meant wait. That is the first thing Brood does for anyone, so it should not stall.

Noisy Brood files away the mail that doesn’t need you, so the mail that does can find you. We write up what we build each week because every fix like this one is meant to make that a little more reliable. It isn’t open to everyone yet. If you’d like to hear when it can take you, join the list.