DNS failover: TTL promises, the resolver decides

The TTL on a DNS record tells a resolver "you may keep this answer for this long". It does not say "you will refresh after this long".

In practice many resolvers rewrite TTLs. Some enforce a floor and raise 60 seconds to 300. Others enforce a ceiling. Internal resolvers on corporate networks frequently run with the oldest settings in the chain.

So when you calculate failover time, TTL is the starting point rather than the result. The real cutover time is the TTL plus the behaviour of every link in the resolver chain.

Between a user and your DNS server there are usually at least three caches: the operating system cache, the browser cache, and the ISP or corporate resolver cache.

The browser cache is the awkward one: it sometimes ignores the TTL and applies its own lifetime. A user with the page already open can keep hitting the old address for ten minutes.

Connection-level stickiness adds to this: an open TCP connection keeps talking to the old address after DNS has changed. It has to break before anything is re-resolved.

Failover is usually done with a CNAME: the `www` record points at the provider’s name and the provider returns whichever address is current. But the standard does not allow a CNAME at the apex — that is, at `example.uz`.

So the apex carries an A record instead, and an A record is static. When the provider changes address, your apex stays on the old one.

Some DNS providers paper over this with non-standard mechanisms. The reliable route is different: delegate the whole zone to the provider by changing the NS records. Then the apex and everything else are managed from one place.

The quality of a failover is decided by its check points. A check from a single location reads that location’s network problem as a global outage.

So checks run from several regions and the decision is a vote: two of three say "down", the cutover begins.

The second subtlety is what the check actually asks for. An open TCP port means nothing: a live server with a dead database is the most common outage there is. The check has to request a real page and inspect the body of the answer.

The main risk of automatic cutover is flapping: the system bounces back and forth every few minutes and a share of users lands on a different server each time.

Two thresholds prevent it: how many consecutive failed checks trigger a cutover, and how many successful ones trigger a return. The return should always be slower.

And the return is better left manual. Whether the server recovered or is merely answering the health check is a judgement for a person. An automatic return pushes traffic onto an unfinished recovery.

DNS moves traffic to a different address; it does not move user state. If sessions live on one server only, everyone signs in again after the cutover.

The same applies to data. If the standby is read-only, the site opens after the cutover but will not accept an order — sometimes worse than being down, because the failure is invisible.

A failover plan therefore has to answer two questions: does the user stay signed in, and do writes work?

None of these limits makes DNS failover useless. They move it from the promise of "we cut over in a minute" to the reality of "most users move within ten to fifteen minutes".

For most situations that is enough. When an entire data centre or provider fails, moving 80% of traffic elsewhere in ten minutes is a very good outcome.

What matters is setting the expectation correctly. If the business cannot absorb a minute of downtime, DNS is not the answer: that requires anycast or an active-active architecture, and both cost considerably more.

Do not mistake a plan for confidence. Rehearse the cutover deliberately and during working hours, not at night — at night you have neither caches nor users.

Measure three things during the exercise: when the first user reached the new address, when 90% had moved, and how long the last percent took. The third number surprises everyone.

Rehearse the return as well. Most teams practise only leaving, then take a second outage coming back during a real incident.