The bfstore private hosted zone contains:
prod.aws.internal.bfstore.example
I need a record for the internal observability ingestion service:
otel.prod.aws.internal.bfstore.example
Route 53 asks me to choose a routing policy.
Two options catch my attention:
Simple routing
Failover routing
Failover sounds more resilient.
Simple sounds less ambitious.
That wording makes it tempting to treat the decision as:
simple routing
= basic
failover routing
= production-ready
That is the wrong comparison.
Simple routing and failover routing describe different DNS contracts.
SIMPLE ROUTING
This name currently identifies
this destination.
FAILOVER ROUTING
This name has a primary destination
and a prepared secondary destination.
Route 53 may choose between them
using health or routing-control state.
Route 53 supports both simple and failover routing inside private hosted zones. Simple routing is intended for a standard record, usually pointing towards one resource. Failover routing implements active-passive selection between primary and secondary records.
Simple routing publishes an answer. Failover routing publishes a decision procedure.
The second is useful only when bfstore has something meaningful to decide.
Simple routing is the ordinary default
Suppose the name points towards an internal load balancer:
orders.prod.aws.internal.bfstore.example
│
▼
internal Application Load Balancer
│
├── order-service targets in AZ A
├── order-service targets in AZ B
└── order-service targets in AZ C
A simple alias record can point towards that load balancer.
The load balancer already:
- spans multiple Availability Zones;
- checks target health;
- stops routing requests to unhealthy targets;
- presents one stable service endpoint.
Route 53 does not need to perform another active-passive decision merely because the service is important.
The record’s job is straightforward:
orders.prod.aws.internal.bfstore.example
→ current internal order-service endpoint
Simple routing is appropriate because one resource currently performs that DNS-level function.
Route 53 describes simple routing as standard DNS routing without special weighted, latency or failover behaviour. A simple record usually points towards one resource.
The resource behind the record may itself be highly available.
SIMPLE DNS RECORD
does not imply
SINGLE SERVER
For a simple alias record, EvaluateTargetHealth does not create failover.
Route 53 still has only one record to return.
SIMPLE ALIAS RECORD
EvaluateTargetHealth
does not suppress the answer.
Route 53 returns
the only record available.
LOAD BALANCER HEALTH
still controls which
registered targets receive traffic.
The ALB’s health model remains useful because it routes application traffic only towards healthy targets. It does not turn the simple DNS record into a failover pair.
A simple record may point towards:
- a multi-AZ load balancer through an alias record;
- an RDS or Aurora endpoint through a CNAME;
- a PrivateLink interface endpoint through an alias record;
- another compatible Route 53 record in the same hosted zone;
- a service gateway.
For RDS and Aurora, I should first understand the database service’s native endpoint and failover behaviour. A private CNAME can provide a bfstore-owned service name, but it adds another DNS cache layer whose TTL must not delay reconnection after the database endpoint changes.
The routing policy describes how Route 53 selects the record.
It does not describe the complete availability architecture behind the answer.
Several IP addresses do not create failover
A simple non-alias record can contain several values.
For example:
legacy.prod.aws.internal.bfstore.example
A 10.80.10.25
A 10.80.11.25
With simple routing, Route 53 can return all the values in random order.
The recursive resolver returns them to the client, and the client decides which address to use. Route 53 does not health-check the individual values in that simple record.
This is not managed failover.
MULTIPLE SIMPLE VALUES
Route 53 returns the addresses.
The client chooses.
FAILOVER RECORDS
Route 53 evaluates health.
Route 53 chooses the eligible branch.
If 10.80.10.25 fails, Route 53 may continue returning it.
A client might try the second address.
It might not.
Its behaviour depends on:
- resolver ordering;
- application library;
- connection timeout;
- retry logic;
- address-family handling.
Several values can provide several possible destinations.
They do not establish an active-passive recovery policy.
When the requirement is to return several independently health-checked endpoints rather than maintain primary and secondary roles, Route 53 also provides multivalue answer routing.
SIMPLE MULTIPLE VALUES
one record
several values
no per-value Route 53 health
MULTIVALUE ANSWER
several health-aware records
up to eight eligible answers
FAILOVER
primary role
secondary role
one selected branch
Multivalue answer routing is an active-active multiple-answer mechanism. It is not a load balancer and it is not primary-secondary failover.
I should therefore avoid using a multi-value simple record as a homemade failover mechanism unless the client behaviour is explicitly understood and acceptable.
A basic failover pair requires two real roles
A basic failover pair contains two records with:
- the same name;
- the same record type;
- different record identifiers;
- one marked
Primary; - one marked
Secondary.
Conceptually:
dns:
name: otel.prod.aws.internal.bfstore.example
type: A
records:
- role: primary
identifier: primary-observability
target: primary-ingestion-endpoint
- role: secondary
identifier: recovery-observability
target: recovery-ingestion-endpoint
When Route 53 considers the primary healthy, it returns the primary record.
When the primary is unhealthy and the secondary is healthy, it returns the secondary. This is Route 53’s active-passive routing model.
More complex configurations may place several resources beneath either branch, using alias records and other routing policies. The top-level contract still has one primary failover role and one secondary failover role.
That arrangement introduces requirements simple routing does not have.
The secondary must be:
- deployed;
- reachable;
- authorised;
- sufficiently current;
- large enough;
- monitored;
- tested.
Failover routing cannot create those properties.
It can only select the record representing the secondary after they already exist.
ROUTE 53
can redirect discovery
ROUTE 53
cannot manufacture recovery capacity
A secondary record pointing towards an unfinished recovery environment is not resilience.
It is a highly automated redirection towards construction work.
Health is what turns failover into failover
The primary record must have a health model.
For alias records pointing towards supported AWS resources, Route 53 may be able to evaluate the target’s health.
For non-alias records, a Route 53 health check can be associated with the record.
Health checks may monitor:
- an HTTP endpoint;
- an HTTPS endpoint;
- a TCP endpoint;
- another collection of health checks;
- a CloudWatch alarm data stream;
- an Amazon Application Recovery Controller routing-control state.
Route 53 evaluates endpoint and alarm-based health continually rather than waiting for a DNS query before performing the check.
For a public endpoint, Route 53 health checkers may connect to it directly.
For a private endpoint, the health checkers sit outside the VPC and cannot directly reach a private IP address.
AWS therefore recommends representing private-service health through a CloudWatch metric and alarm, then creating a Route 53 health check based on that alarm’s data stream.
PRIVATE SERVICE
│
▼
SYNTHETIC OR APPLICATION METRIC
│
▼
CLOUDWATCH ALARM DATA
│
▼
ROUTE 53 HEALTH CHECK
│
▼
FAILOVER DECISION
The metric should represent the capability behind the name.
For the observability ingestion endpoint, that may mean:
Can an authenticated client
submit telemetry successfully?
It should not merely mean:
Does one EC2 instance
still have power?
The branch-health probe must target the branch it controls.
CONTROL-LOOP PROBES
primary-specific name
→ primary endpoint
→ primary health signal
secondary-specific name
→ secondary endpoint
→ secondary health signal
END-TO-END PROBE
stable failover name
→ currently selected branch
→ service-path and SLO evidence
The stable production name is useful for an end-to-end test.
It should not directly determine whether the primary or secondary branch is healthy. After failover, that name may resolve to the secondary, causing a successful secondary request to be mistaken for evidence that the primary has recovered.
A CloudWatch-alarm health check also has implementation boundaries. The alarm must be in the same AWS account as the Route 53 health check, must use supported standard-resolution metrics and cannot use metric math. The dedicated DNS failover article covers the wider alarm, insufficient-data and failback design.
The DNS name describes a service.
The health signal should test that service without creating a circular control loop.
A secondary with no health evaluation is treated as healthy
A secondary failover record does not have to carry a separately associated Route 53 health check.
Its effective health still depends on the record shape.
NON-ALIAS SECONDARY
no associated health check
→ treated as healthy
ALIAS SECONDARY
EvaluateTargetHealth = No
and no associated health check
→ treated as healthy
ALIAS SECONDARY
EvaluateTargetHealth = Yes
→ health inherited from alias target
An alias record may also have both an associated health check and EvaluateTargetHealth enabled. In that case, both health evaluations must be healthy for the record to be considered healthy.
When the primary is unhealthy and the secondary has neither an associated health check nor target-health evaluation, Route 53 treats the secondary as healthy and returns it.
This can be appropriate for a simple maintenance destination.
For example:
shop.bfstore.example
→ primary ecommerce application
secondary
→ static service-unavailable page
The secondary may have a narrow promise:
Explain that the shop
is temporarily unavailable.
For a private production service, an unchecked secondary is riskier.
A recovery endpoint may exist while being unable to perform useful work because:
- credentials are stale;
- data replication is behind;
- the database is read-only;
- the network route is missing;
- capacity has scaled to zero;
- a dependent service is unavailable.
Checking both destinations provides better information.
It also reveals another Route 53 behaviour.
When both are unhealthy, Route 53 returns the primary
Suppose both records in a basic failover pair have health evaluation.
Route 53 observes:
primary:
unhealthy
secondary:
unhealthy
Route 53 returns the primary record.
PRIMARY UNHEALTHY
SECONDARY UNHEALTHY
│
▼
PRIMARY ANSWER RETURNED
Failover routing therefore does not guarantee that every returned destination is healthy.
It guarantees a defined primary-secondary selection process with this last-resort response.
Other routing policies also have last-resort selection behaviour, but their exact fallback rules differ.
The application must still have:
- connection deadlines;
- bounded retries;
- circuit breaking;
- useful error handling;
- operational alerts.
DNS is selecting between destinations.
It is not promising that a functioning destination exists.
Simple routing is the honest default for most bfstore services
For most bfstore private names, I would start with simple routing.
Examples include:
orders.prod.aws.internal.bfstore.example
modules.platform.aws.internal.bfstore.example
backstage.platform.aws.internal.bfstore.example
mysql.catalog.prod.aws.internal.bfstore.example
Simple routing is appropriate where:
- one service endpoint currently owns the name;
- the target already provides suitable local high availability;
- no independent recovery destination exists;
- recovery requires operator action before traffic can move;
- changing the record is part of a larger restore procedure;
- automatic DNS movement would be unsafe.
Consider the order database.
RDS and Aurora already provide managed endpoint behaviour for their native high-availability models. A bfstore-owned name would normally point towards that endpoint using a CNAME, not a Route 53 alias to RDS.
A separate recovery database endpoint might also exist.
That does not mean Route 53 should redirect clients automatically when the primary health check fails.
Database recovery may first require:
- confirming replication state;
- promoting the secondary;
- fencing the old writer;
- validating migrations;
- reconciling event processing;
- confirming application readiness;
- changing DNS;
- recycling or reconnecting client pools.
Until that procedure is complete, simple routing may express the truthful architecture:
This name points towards
the current authorised writer.
A manual or automated recovery workflow can update the simple record after the secondary has safely become authoritative.
The CNAME TTL and the database client’s own DNS caching must fit the recovery objective. Adding a bfstore-owned name should not delay the native RDS or Aurora endpoint change.
That is not a lesser architecture.
It is an architecture where DNS follows recovery rather than trying to lead it.
Failover routing is correct at a prepared recovery boundary
Failover becomes useful when bfstore has two complete service cells.
For example:
otel.prod.aws.internal.bfstore.example
Primary:
observability ingestion cell
in the normal production path
Secondary:
independently reachable
recovery ingestion cell
Both accept:
- the same OTLP protocol;
- the same client identities;
- the same service name;
- equivalent telemetry envelopes.
Both have:
- sufficient capacity;
- valid certificates;
- usable downstream storage;
- active monitoring;
- tested network paths.
PRIMARY CELL
│
├── load balancer
├── collector gateways
├── durable queue
└── storage path
SECONDARY CELL
│
├── load balancer
├── collector gateways
├── durable queue
└── storage path
Route 53 now has a meaningful choice.
The primary and secondary are not two addresses inside the same failure domain.
They are two independently viable answers to the same service question.
That is where failover routing earns its additional complexity.
DNS caching applies to both policies
Simple and failover records both produce DNS answers that resolvers and clients may cache.
Route 53 can begin returning a new failover answer only for new DNS queries.
A client holding the old primary answer may continue using it until:
- the TTL expires;
- the application performs another lookup;
- the existing connection fails;
- the client reconnects.
ROUTE 53 CHANGES ANSWER
│
▼
RESOLVER STILL HAS OLD ANSWER
│
▼
TTL EXPIRES
│
▼
NEW QUERY
│
▼
NEW DESTINATION
The TTL control depends on the record type.
NON-ALIAS RECORD
TTL configured on the record
ALIAS TO AWS RESOURCE
TTL inherited from the target
ALIAS TO ANOTHER ROUTE 53 RECORD
TTL inherited from the referenced record
For health-checked non-alias records, the TTL is an explicit design choice.
For aliases, the effective TTL comes from the alias target and cannot be set independently on the alias record.
Failover routing is therefore not instantaneous connection migration.
A shorter effective TTL can reduce the period for which old answers remain cached.
It also increases DNS query frequency and does nothing to move existing long-lived sessions.
The recovery test must include the actual bfstore client.
A successful dig proves that Route 53 is returning the intended record.
It does not prove that a Go gRPC client with a persistent HTTP/2 connection has moved.
The cost of failover is operational, not merely financial
Failover routing adds more than a second record.
It adds:
- health checks;
- CloudWatch metrics or alarms for private services;
- secondary capacity;
- additional routing;
- security rules;
- recovery data handling;
- synthetic probes;
- failover tests;
- failback procedures;
- more monitoring;
- more failure states.
Simple routing says:
Maintain this record and target.
Failover routing says:
Maintain two destinations,
their health evidence,
their reachability,
their data behaviour,
and the decision connecting them.
That additional machinery is worthwhile when it protects a meaningful recovery objective.
It is wasteful when the secondary:
- shares the same database;
- uses the same broken Kafka cluster;
- depends on the same central route;
- cannot accept full traffic;
- has never been tested;
- exists only in the architecture diagram.
Two records are cheaper than two recovery environments.
The records are also the least important part of the cost.
My bfstore decision rule
I would select simple routing by default.
START
│
▼
Does one current endpoint
own the service name?
│
├── yes → SIMPLE ROUTING
│
└── no
│
▼
Do primary and secondary
provide the same capability?
│
├── no → redesign
│
└── yes
│
▼
Is the secondary independently
reachable and ready?
│
├── no → SIMPLE ROUTING
│ until recovery is prepared
│
└── yes
│
▼
Is there a trustworthy signal
for branch eligibility?
│
┌──────┴──────┐
│ │
YES NO
│ │
▼ ▼
automated health, SIMPLE ROUTING
calculated health and recovery-led
or ARC routing DNS update
control
│
▼
FAILOVER ROUTING
The health signal does not have to be an automated endpoint probe.
For stateful or ambiguous failure modes, an Amazon Application Recovery Controller routing control can deliberately set the Route 53 health-check state after an operator or recovery workflow has verified the destination. ARC routing controls are traffic switches rather than application-health monitors.
This keeps failover attached to evidence.
The existence of a second resource is not enough.
The secondary must be an alternative service destination, not merely another piece of infrastructure with a promising name.
The mental model I am keeping
My earlier model was:
SIMPLE ROUTING
single point of failure
FAILOVER ROUTING
high availability
The stronger model is:
PRIVATE DNS NAME
│
▼
WHAT CONTRACT DOES IT HAVE?
│
┌─────────────┴─────────────┐
│ │
▼ ▼
ONE CURRENT DESTINATION TWO PREPARED DESTINATIONS
│ │
▼ ▼
SIMPLE ROUTING FAILOVER ROUTING
│ │
▼ ▼
RETURN THE RECORD HEALTH OR ROUTING CONTROL
│
┌───────────┴───────────┐
│ │
▼ ▼
PRIMARY ELIGIBLE PRIMARY INELIGIBLE
│ │
▼ ▼
RETURN PRIMARY CHECK SECONDARY
│
┌───────────┴───────────┐
│ │
▼ ▼
SECONDARY ELIGIBLE SECONDARY INELIGIBLE
│ │
▼ ▼
RETURN SECONDARY RETURN PRIMARY
Simple routing names the current destination.
Failover routing selects between prepared branches.
Health or routing control determines branch eligibility.
TTL and client behaviour determine when the decision reaches real connections.
Simple routing is not the option bfstore uses before learning about resilience.
It is the honest policy for a name with one current service destination.
Failover routing is not a production upgrade applied to that record by default.
It is an active-passive recovery mechanism.
Use simple routing when DNS should state where the service is. Use failover routing when DNS has been given two valid places the service can be and a trustworthy signal for choosing between them.
For ordinary multi-AZ load-balanced services, simple routing is often enough.
For stateful recovery requiring promotion and reconciliation, simple routing may remain safer until the recovery workflow has completed.
For independently prepared service cells with a meaningful automated or operator-controlled signal, failover routing can preserve one stable name while Route 53 redirects future resolutions.
The second option is more powerful.
It is also a promise that bfstore has built, connected, secured and tested somewhere else for the traffic to go.
References and further reading
-
Choosing a Route 53 routing policy Describes simple, multivalue and failover routing policies.
-
Simple routing Covers simple records, multiple values and the absence of health checking for individual values.
-
Values specific for simple alias records Explains that
EvaluateTargetHealthhas no effect for simple routing. -
Multivalue answer routing Describes health-aware multiple-answer records and the limit of eight returned answers.
-
Values specific for failover alias records Defines basic failover pairs, alias health evaluation and branch-specific health-check naming.
-
How Route 53 selects health-checked records Documents unchecked-record behaviour and the result when both records in a failover pair are unhealthy.
-
Failover in a private hosted zone Explains why external Route 53 health checkers cannot directly test private endpoints and describes CloudWatch-based health checks.
-
How Route 53 determines health Covers endpoint, calculated and CloudWatch-alarm health checks, including same-account restrictions.
-
Choosing between alias and non-alias records Covers supported alias targets, CNAME use and alias TTL inheritance.
-
Amazon RDS Multi-AZ failover Describes native RDS endpoint failover and client DNS-caching considerations.
-
Amazon Application Recovery Controller routing controls Describes operator-controlled traffic switches that update Route 53 health-check state.