Circuit Breaker

Requirements & System Scope

Functional Scope (In-Scope)

Resilience State-Machine Transitions: Move between states (CLOSED, OPEN, HALF_OPEN) based on failure rates and time windows.
Sliding Failures Window Counters: Track error rates over configurable sliding call windows using memory counters.
Automated Self-Recovery Pools: Automatically transition from OPEN to HALF_OPEN after a wait period, sending probe requests to test downstream health.
Graceful Fallback Handlers: Execute registered fallback suppliers to prevent user-facing errors when the circuit is OPEN.

Explicit Boundaries (Out-of-Scope)

No Automatic Network Proxy Sidecar Injectors: Does not hook directly into physical network interfaces or construct active Istio Envoy sidecars.
No Intelligent Dynamic Autoscaler Rules: Excludes auto-scaling cluster nodes or adjusting downstream pool allocations.

Class Diagram & Entity Relationships

Structural layout showing relationships between the circuit state, configuration, and sliding window monitors:

Thread-Safe States: CircuitBreaker encapsulates success/failure atomic registers alongside transition logs.
Resilience Rules: CircuitBreakerConfig defines failure thresholds, wait periods, and probe call counts.

Design Patterns & SOLID Principles

State Pattern (Breaker States): Encapsulating behaviors within discrete states (CLOSED, OPEN, HALF_OPEN) isolates state transition logic from downstream request execution, satisfying Open-Closed Principle.
Liskov Substitution Principle (LSP): Replacing remote client calls with fallback suppliers when the circuit is open ensures that callers receive expected interface structures even during failures.

Core Execution Workflows

Execution & Recovery Sequence

Caller requests downstream operation: execute(supplier).
Check current circuit state: allowCall():
1. If the state is OPEN and the wait duration has passed, transition to HALF_OPEN atomicly.
2. If the state is OPEN and the wait duration has NOT passed, execute fallback handler immediately.
Execute target remote operation:
1. On success: record result via recordSuccess().
2. On failure: record exception via recordFailure().
Evaluate states:
1. If CLOSED and the sliding error rate threshold is breached, transition to OPEN.
2. If HALF_OPEN and any request fails, transition back to OPEN.
3. If HALF_OPEN and all probe requests succeed, transition back to CLOSED.

Concurrency & Thread Safety Strategy

When thousands of threads route remote requests concurrently through the same circuit:

State Synchronization Locks: Protect state transitions using locks or atomic references to avoid parallel evaluation corruption.
Volatile Time Registers: Maintain time offsets in volatile variables to ensure they are visible across threads immediately.

Complete Clean Code Blueprint

Clean reference designs demonstrating state transitions in Java and Python:

// ─── JAVA BLUEPRINT ──────────────────────────────────────────────────────────
import java.util.*;
import java.util.concurrent.atomic.*;
import java.util.concurrent.locks.*;
import java.util.function.Supplier;

enum StateName { CLOSED, OPEN, HALF_OPEN }

class CircuitBreakerConfig {
    final double failureRateThreshold; // e.g. 50.0 for 50%
    final long waitDurationMs;         // Cooldown in open state
    final int minCallsForEvaluation;   // Minimum calls in Closed before checking threshold
    final int halfOpenMaxSuccesses;    // Successes needed in Half-Open to close circuit
    final int slidingWindowSize;       // Size of sliding call outcome history

    public CircuitBreakerConfig(double failureRateThreshold, long waitDurationMs, 
                                int minCallsForEvaluation, int halfOpenMaxSuccesses, int slidingWindowSize) {
        this.failureRateThreshold = failureRateThreshold;
        this.waitDurationMs = waitDurationMs;
        this.minCallsForEvaluation = minCallsForEvaluation;
        this.halfOpenMaxSuccesses = halfOpenMaxSuccesses;
        this.slidingWindowSize = slidingWindowSize;
    }
}

// Outcome of a single service call
enum CallOutcome { SUCCESS, FAILURE }

// Sliding Window Counter to track outcome metrics
class SlidingWindowCounter {
    private final int windowSize;
    private final Deque<CallOutcome> outcomes = new LinkedList<>();

    public SlidingWindowCounter(int windowSize) {
        this.windowSize = windowSize;
    }

    public synchronized void record(CallOutcome outcome) {
        if (outcomes.size() >= windowSize) {
            outcomes.pollFirst();
        }
        outcomes.addLast(outcome);
    }

    public synchronized int getTotalCalls() {
        return outcomes.size();
    }

    public synchronized double getFailureRate() {
        if (outcomes.isEmpty()) return 0.0;
        long failures = outcomes.stream().filter(o -> o == CallOutcome.FAILURE).count();
        return (failures * 100.0) / outcomes.size();
    }

    public synchronized void clear() {
        outcomes.clear();
    }
}

// State Pattern Interfaces
interface CircuitBreakerState {
    boolean allowCall(CircuitBreaker cb);
    void recordSuccess(CircuitBreaker cb);
    void recordFailure(CircuitBreaker cb);
    StateName getName();
}

class ClosedState implements CircuitBreakerState {
    private final SlidingWindowCounter counter;

    public ClosedState(int windowSize) {
        this.counter = new SlidingWindowCounter(windowSize);
    }

    @Override
    public boolean allowCall(CircuitBreaker cb) {
        return true;
    }

    @Override
    public void recordSuccess(CircuitBreaker cb) {
        counter.record(CallOutcome.SUCCESS);
        evaluate(cb);
    }

    @Override
    public void recordFailure(CircuitBreaker cb) {
        counter.record(CallOutcome.FAILURE);
        evaluate(cb);
    }

    private void evaluate(CircuitBreaker cb) {
        CircuitBreakerConfig config = cb.getConfig();
        if (counter.getTotalCalls() >= config.minCallsForEvaluation) {
            double rate = counter.getFailureRate();
            if (rate >= config.failureRateThreshold) {
                System.out.println("[CB State Transition] Failure rate of " + String.format("%.1f", rate) 
                                   + "% >= threshold of " + config.failureRateThreshold + "%. Tripping circuit to OPEN.");
                cb.changeState(new OpenState());
            }
        }
    }

    @Override
    public StateName getName() { return StateName.CLOSED; }
}

class OpenState implements CircuitBreakerState {
    private final long openTimeMs;

    public OpenState() {
        this.openTimeMs = System.currentTimeMillis();
    }

    @Override
    public boolean allowCall(CircuitBreaker cb) {
        long elapsed = System.currentTimeMillis() - openTimeMs;
        if (elapsed >= cb.getConfig().waitDurationMs) {
            System.out.println("[CB State Transition] Wait duration expired. Transitioning to HALF_OPEN to probe.");
            cb.changeState(new HalfOpenState());
            return true;
        }
        return false;
    }

    @Override
    public void recordSuccess(CircuitBreaker cb) {}

    @Override
    public void recordFailure(CircuitBreaker cb) {}

    @Override
    public StateName getName() { return StateName.OPEN; }
}

class HalfOpenState implements CircuitBreakerState {
    private final AtomicInteger successes = new AtomicInteger(0);

    @Override
    public boolean allowCall(CircuitBreaker cb) {
        return true;
    }

    @Override
    public void recordSuccess(CircuitBreaker cb) {
        int currentSuccesses = successes.incrementAndGet();
        if (currentSuccesses >= cb.getConfig().halfOpenMaxSuccesses) {
            System.out.println("[CB State Transition] Probes succeeded. Closing the circuit.");
            cb.changeState(new ClosedState(cb.getConfig().slidingWindowSize));
        }
    }

    @Override
    public void recordFailure(CircuitBreaker cb) {
        System.out.println("[CB State Transition] Probe call failed during HALF_OPEN. Tripping back to OPEN.");
        cb.changeState(new OpenState());
    }

    @Override
    public StateName getName() { return StateName.HALF_OPEN; }
}

class CircuitBreaker {
    private final CircuitBreakerConfig config;
    private final ReentrantLock lock = new ReentrantLock();
    private volatile CircuitBreakerState state;

    public CircuitBreaker(CircuitBreakerConfig config) {
        this.config = config;
        this.state = new ClosedState(config.slidingWindowSize);
    }

    public CircuitBreakerConfig getConfig() {
        return config;
    }

    public StateName getStateName() {
        return state.getName();
    }

    public void changeState(CircuitBreakerState newState) {
        lock.lock();
        try {
            this.state = newState;
        } finally {
            lock.unlock();
        }
    }

    public <T> T execute(Supplier<T> supplier, Supplier<T> fallback) {
        if (!state.allowCall(this)) {
            System.out.println("[CB Exec] Call blocked by CircuitBreaker (State: " + state.getName() + "). Executing fallback.");
            return fallback.get();
        }

        try {
            T result = supplier.get();
            state.recordSuccess(this);
            return result;
        } catch (Exception e) {
            state.recordFailure(this);
            System.out.println("[CB Exec] Call raised error: " + e.getMessage() + ". Logged failure. Delegating to fallback.");
            return fallback.get();
        }
    }
}

public class Main {
    public static void main(String[] args) throws InterruptedException {
        CircuitBreakerConfig config = new CircuitBreakerConfig(
            50.0,   // failureRateThreshold (50%)
            1000,   // waitDurationMs (1s)
            4,      // minCallsForEvaluation
            2,      // halfOpenMaxSuccesses
            6       // slidingWindowSize
        );

        CircuitBreaker cb = new CircuitBreaker(config);

        Supplier<String> successCall = () -> "Successful response";
        Supplier<String> failedCall = () -> {
            throw new RuntimeException("Connection Timeout");
        };
        Supplier<String> fallback = () -> "Fallback value";

        // 1. Send normal successful traffic
        System.out.println("--- Starting Normal Operations (CLOSED) ---");
        for (int i = 0; i < 3; i++) {
            System.out.println("Result: " + cb.execute(successCall, fallback));
        }

        // 2. Trigger failures to trip the circuit
        System.out.println("\n--- Sending Failures to Trip Circuit ---");
        for (int i = 0; i < 4; i++) {
            System.out.println("Result: " + cb.execute(failedCall, fallback));
        }

        // 3. Verify requests are immediately short-circuited (OPEN state)
        System.out.println("\n--- Verifying short-circuiting while OPEN ---");
        System.out.println("Result: " + cb.execute(successCall, fallback));
        System.out.println("Current State: " + cb.getStateName());

        // 4. Cooldown and check probe auto-recovery
        System.out.println("\n--- Cooldown wait... ---");
        Thread.sleep(1100);

        System.out.println("State: " + cb.getStateName() + " - Attempting probe 1");
        System.out.println("Result: " + cb.execute(successCall, fallback)); // Probe 1 success

        System.out.println("State: " + cb.getStateName() + " - Attempting probe 2");
        System.out.println("Result: " + cb.execute(successCall, fallback)); // Probe 2 success

        System.out.println("Final Circuit State: " + cb.getStateName());
    }
}

Help Us Improve

How helpful was this walkthrough?

Click a star to rate. We actively use this feedback to refine and update our system design content.

Placeholder

Discussion

Share your thoughts, ask questions, or help others.

Loading comments...

✦Requirements & System Scope

Functional Scope (In-Scope)

Explicit Boundaries (Out-of-Scope)

◫Class Diagram & Entity Relationships

⬡Design Patterns & SOLID Principles

{ }Core Execution Workflows