DrawblyTechnical explanations · Drawbly
Circuit breaker pattern: when does a request stop reaching the service?
By Drawbly ·
A circuit breaker sits in front of a dependency. It watches recent failures, pauses calls when failure looks persistent, then allows a small test call after a wait. The useful question is concrete: which requests actually reach the dependency?
Five requests to one struggling API
Imagine a checkout page asking a separate quote API for a shipping estimate. This example uses a read-only lookup, so a repeated call does not create a shipment. The fictional policy opens a breaker after three timeouts within ten seconds, waits 30 seconds from opening, then permits one test call. Those numbers explain the diagram; they are not recommended defaults.
- 0, 1 and 2 seconds: closed. Each request passes through to the quote API and times out. The third recent failure meets this example's threshold, so the breaker opens at 2 seconds.
- 3 seconds: open. The next request is rejected by the caller's breaker immediately. It never reaches the quote API. The page can honestly say that a quote is temporarily unavailable; it should not present an invented price.
- 32 seconds: half-open. After the 30-second wait measured from opening, the breaker allows one test lookup. If that lookup succeeds, this simple policy closes the breaker and normal calls resume. If it fails, the breaker opens for another wait. A production policy might require several successful probes.
A timeout does not prove the remote service did no work. The example deliberately uses a read-only operation. For a write, decide how retries and uncertain results affect duplication; the idempotency-key walkthrough shows that separate problem.
Closed, open and half-open are routing decisions
Closed routes calls through and measures the failures defined by the policy. Open stops sending calls to that dependency for a period. Half-open permits limited probes to see whether it can recover. A breaker is usually scoped to a particular dependency or operation; one failing quote provider should not automatically block an unrelated catalog read. Log state changes and rejected calls so the fast failures are visible during an incident.
The breaker cannot repair the quote API. Its immediate benefit is to avoid adding load and latency while that API is likely to fail. It also has a cost: a request that could have succeeded during the open period is still blocked. Choose thresholds and recovery behavior with real failure and latency data.
How is this different from retry?
A retry makes another attempt for an individual operation when a short fault might clear. A breaker decides whether later operations should reach the dependency at all. If every incoming request makes three attempts, three customers can create nine remote calls while the API is sick. A finite retry budget and backoff can limit that amplification; the breaker can stop later calls after the failure threshold is reached. When the breaker is open, retrying its immediate rejection in a tight loop only wastes work.
These patterns can be combined. Microsoft describes calling through a breaker with retry logic that stops when the breaker says the fault is not transient. Its circuit breaker guide defines the three states, and its transient fault guide recommends bounded retries and an aggregate retry budget. The rate-limiting example addresses a different question: controlling incoming request pressure even when a dependency is healthy.
What should the real design specify?
Failure signal: Count the timeouts or errors that indicate a struggling dependency. Do not count every client input error as service failure.
Scope and thresholds: Name the operation, observation window, minimum sample size and threshold. A few isolated failures should not trip every route.
Recovery: Set the open wait, number of concurrent half-open probes, success condition and behavior when a probe fails. Limit probes so a recovering service is not flooded.
Caller response: Define a truthful fallback or an explicit unavailable response. Monitor open/close events and the number of calls rejected locally.
A breaker may add little value when bounded retries already handle rare transient failures or the platform provides equivalent isolation. The point is to draw the real failure boundary before adding a state machine. AWS Prescriptive Guidance also describes opening after repeated timeouts or failures and testing the callee before returning to ordinary traffic.
Draw the dependency you actually have
Open the editable Drawbly timeline. Replace “quote API” with your dependency and write down the threshold, wait and probe rule. Keep the blocked locally request visible: that is the moment a circuit breaker changes the system's behavior.
Microsoft Azure Architecture Center: Circuit Breaker pattern, Transient fault handling, and AWS Prescriptive Guidance: Circuit breaker pattern. The quote API, timing and threshold are fictional.