Back to the blog
Engineering

Long-polling beats your retry loop

Six hundred requests to learn one five-digit number is a strange way to spend a rate limit. The ?wait= parameter, the timeouts you have to set, and the loop that actually works.

by Tom Okafor·Infrastructure·Published 19 Mar 2024·6 min read

About once a week a customer opens a ticket about 429s while waiting for a verification code. The code is almost always the same, and it is almost always the polling loop rather than the limit that is wrong.

The loop everyone writes first

the-slow-way.py
# Reasonable-looking. Costs 600 requests to learn a five-digit number.
-kw">while -kw">True:
    r = requests.get(f"{API}/activations/{aid}", headers=H).json()
    -kw">if r["status"] == "code_received":
        -kw">return r["messages"][-num">0]["code"]
    -kw">if r["status"] -kw">in ("expired", "cancelled"):
        -kw">return -kw">None
    time.sleep(-num">2)

The activation window is twenty minutes. At a two-second interval that is 600 requests per verification, of which 599 return "still waiting". Run four verifications concurrently and you are at 120 requests a minute, which is above the default key limit, and the 429s begin — right in the middle of the wait, which is the worst possible moment to be rate limited.

What ?wait= does

GET /api/v1/activations/{id} accepts wait, in seconds, up to 300. The connection is held open until the activation changes state or the timer runs out, whichever comes first. The polling still happens — we check the upstream every two seconds — but it happens on our side of the connection, where it does not touch your rate limit and does not cost you a round trip.

bash
# Blocks up to 240 seconds. Returns the instant the SMS lands.
-kw">curl "https://virtualsmsnumbers.com/api/v1/activations/1043872915?wait=240" \
  -H "Authorization: Bearer $VSN_KEY" \
  --max-time -num">260

# {"id":"1043872915","status":"code_received",
#  "messages":[{"code":"48219","text":"Telegram code: 48219","received_at":"..."}]}

Same twenty-minute window, five requests instead of 600. The median code arrives in 8.4 seconds, so in practice the first call usually returns almost immediately and the loop never runs a second time.

The timeouts you have to set

This is where long-polling actually goes wrong, and none of it is our fault or yours — it is the infrastructure in between deciding that a connection with no bytes on it is dead.

  • Your HTTP client. The read timeout must exceed wait, with headroom. wait=240 with a default 30-second timeout produces a client-side abort that looks exactly like an API failure. Twenty seconds of headroom is enough.
  • Reverse proxies. nginx defaults proxy_read_timeout to 60 seconds. If your worker sits behind one, either raise it or keep wait under 55.
  • Cloud load balancers. AWS ALB idles at 60 seconds by default, GCP at 600, Cloudflare's proxy will cut a response that produces nothing for 100 seconds on most plans.
  • Serverless functions. If your function has a 60-second budget, wait=240 is a guaranteed timeout, and you will be billed for the wait. Use wait=45 and loop, or move the wait to a webhook.

If you do not control the path, wait=45 is the safe universal number. If you do, wait=240 with a 260-second client timeout is what we run in our own tooling.

The loop that works

wait.py
WAIT = -num">240

-kw">def wait_for_code(aid: str) -> str | -kw">None:
    deadline = time.time() + -num">20 * -num">60
    -kw">while time.time() < deadline:
        r = requests.get(
            f"{API}/activations/{aid}",
            params={"wait": WAIT},
            headers=H,
            timeout=WAIT + -num">20,          # must exceed wait
        ).json()

        -kw">if r["status"] == "code_received":
            -kw">return r["messages"][-num">0]["code"]
        -kw">if r["status"] -kw">in ("expired", "cancelled", "refunded"):
            -kw">return -kw">None                 # already credited back, nothing to do
    -kw">return -kw">None

Note the terminal statuses. expired and refunded both mean the money is already back on your balance; there is no cleanup call and no reason to retry the same pair immediately. If you do want to retry, buy a fresh activation — preferably in a different country, since the first failure is evidence about that country.

What the server does while you wait

The connection is not free on our side either, so it is worth saying what it buys. Every two seconds the request handler reconciles the activation against the pool that issued the number and returns the instant the state changes. Holding one socket for four minutes is dramatically cheaper for us than serving 120 authenticated requests, each of which has to resolve a key, check scopes, check the rate limit and hit the database.

That is why wait is capped at 300 seconds rather than at the full twenty-minute window. Past five minutes the failure modes stop being ours — an idle NAT mapping expires, a laptop sleeps, a deploy cycles the pod — and a client that has to distinguish a dropped socket from a genuine timeout is back to writing a retry loop, which is the thing we were trying to remove.

If you still get a 429

Read the Retry-After header and sleep for exactly that long. Do not back off exponentially on top of it and do not retry immediately — both are worse than the number we gave you, because the number we gave you is when the window actually reopens. If you are hitting the limit with long-polling in place, the limit is genuinely too low for your volume and support can raise it on the key.

When a webhook is the better answer

Long-polling is for flows where something is blocked on the code: a user watching a spinner, a test that cannot proceed. If nothing is blocked — you are provisioning 3,000 numbers overnight and will process the results in the morning — hold no connections at all. Subscribe to sms.received, buy in a loop, and let the deliveries arrive. Holding 3,000 open sockets to wait for an event you are not waiting for is a strange way to spend file descriptors.

The two compose fine. Most customers running interactive flows do both: long-poll for the user's benefit, and take the webhook for the bookkeeping, deduplicating on the delivery id.

Articles are published in English.