This problem appears in multiple sheets. Depth expectations increase as you progress:
Interview Prompt
Design Distributed Banking Ledger System.
Clarifying Questions (ask before designing)
| Question | Why it matters |
|---|---|
| Authorize-only vs capture later? Refunds and chargebacks in scope? | Sets idempotency, ledger, and reconciliation boundaries. |
| What scale should we design for — DAU, QPS, data volume? | Drives every capacity decision; shows structured thinking. |
| What are the read vs write patterns on the critical path? | Determines caching, DB choice, and replication topology. |
| What consistency and durability guarantees are required? | Separates strong-consistency paths from eventual ones — a senior differentiator. |
Scope
In scope
- Double-entry bookkeeping
- Immutable append-only ledger
- ACID at scale
- Balance snapshot optimization
- Regulatory audit
- Cross-ledger settlements
Out of scope (state explicitly)
- Fraud ML model training (#75) — rules engine is enough unless asked
- Merchant onboarding / KYC workflows
- Building a PSP or bank from scratch
Assumptions
- Strong consistency required on money/inventory paths — clarify idempotency early
- External PSP or bank APIs exist; design integration boundaries only
- 99.99% availability target for the commit/authorize path
These foundational concepts underpin the patterns used in this problem. Review them before deep-diving into component-level trade-offs.
- Double-entry bookkeeping: Every transaction creates debit and credit entries that sum to zero
- Account management: Create accounts (checking, savings, loan, revenue, expense)
- Post transactions: Record financial transactions atomically
- Balance inquiry: Real-time balance for any account
- Statement generation: Account statement for any date range
- Reconciliation: Verify all entries balance (total debits = total credits)
- Multi-currency: Support transactions in multiple currencies with exchange rates
- Immutable audit trail: Entries cannot be modified or deleted, only corrected via reversals
- ACID Compliance: Every transaction is atomic, consistent, isolated, durable
- Immutability: Ledger entries are append-only; no UPDATE or DELETE ever
- Effectively-once posting: Double-entry invariants + idempotency keys: duplicate requests return the same ledger entry
- Auditability: Complete history for regulatory compliance (7+ years retention)
- Scale: 1B+ ledger entries/day, 100M+ accounts
- Low Latency: Balance check < 10 ms; posting < 100 ms
| Metric | Calculation | Value |
|---|---|---|
| Accounts | Given | 100M |
| Ledger entries / day | Given (assumption documented in value) | 1B |
| Balance queries / sec | Derived from daily volume ÷ 86400 (+ peak factor) | 100K |
| Posting requests / sec | Derived from daily volume ÷ 86400 (+ peak factor) | 12K |
| Storage / day | 1B entries × ~500 bytes | 500 GB |
| Storage / year | Given | ~180 TB |
Loading...
Double-Entry Transaction Posting
Every financial event creates balanced entries:
Transfer $500 from Account A to Account B:
Entry 1: { account: A, type: DEBIT, amount: 500.00, tx_id: "txn-123" }
Entry 2: { account: B, type: CREDIT, amount: 500.00, tx_id: "txn-123" }
SUM = -500 + 500 = 0 ✓
Atomic posting (PostgreSQL transaction):
BEGIN;
INSERT INTO ledger_entries ... VALUES (debit), (credit);
UPDATE account_balances SET balance = balance - 500 WHERE account_id = 'A';
UPDATE account_balances SET balance = balance + 500 WHERE account_id = 'B';
SELECT balance FROM account_balances WHERE account_id = 'A';
-- If balance < 0 AND account requires non-negative: ROLLBACK
COMMIT;Balance Computation: Running Balance vs Calculated
Approach 1: Calculated (sum all entries) - slow for old accounts Approach 2: Running balance in separate table - O(1), < 1ms Approach 3: Periodic checkpoint + delta - nightly batch We use Approach 2 (running balance) for real-time + Approach 3 for reconciliation.
Immutability: How to "Fix" Errors
Entry is wrong. Cannot UPDATE or DELETE. How to correct?
Reversal: post a new entry that reverses the original.
Original (wrong): Charged $50 fee instead of $5
Entry 1: { acct: customer, DEBIT, $50 }
Entry 2: { acct: fee_rev, CREDIT, $50 }
Reversal:
Entry 3: { acct: customer, CREDIT, $50, reverses: entry_1 }
Entry 4: { acct: fee_rev, DEBIT, $50, reverses: entry_2 }
Correct posting:
Entry 5: { acct: customer, DEBIT, $5 }
Entry 6: { acct: fee_rev, CREDIT, $5 }HTTP
POST /api/v1/ledger/post
Idempotency-Key: "txn-uuid-123"
{
"transaction_id": "txn-123",
"description": "Transfer A to B",
"entries": [
{ "account_id": "acct-A", "type": "debit", "amount": 500.00, "currency": "USD" },
{ "account_id": "acct-B", "type": "credit", "amount": 500.00, "currency": "USD" }
]
}
-> 200 { "transaction_id": "txn-123", "posted_at": "...", "status": "posted" }
GET /api/v1/accounts/{account_id}/balance
-> { "account_id": "acct-A", "balance": 2450.00, "currency": "USD", "as_of": "..." }
GET /api/v1/accounts/{account_id}/statement?from=2026-03-01&to=2026-03-14
-> { "entries": [...], "opening_balance": 2950.00, "closing_balance": 2450.00 }Common Error Responses
400 Bad Request: invalid input, missing fields, or malformed JSON 401 Unauthorized: missing or invalid auth token or API key 403 Forbidden: authenticated but insufficient permissions 404 Not Found: resource ID does not exist 409 Conflict: duplicate write or version conflict; retry with idempotency key 422 Unprocessable Entity: valid syntax but invalid business logic 429 Too Many Requests: rate limit exceeded; honor Retry-After header 500 Internal Error: unexpected server fault; retry with idempotency key 503 Service Unavailable: dependency down or overloaded; use exponential backoff 402 Payment Required: insufficient funds 502 Bad Gateway: payment provider timeout; poll status endpoint
PostgreSQL: Ledger (Partitioned by Month)
SQL
CREATE TABLE ledger_entries (
entry_id UUID PRIMARY KEY, transaction_id UUID NOT NULL,
account_id UUID NOT NULL, entry_type ENUM('debit','credit') NOT NULL,
amount DECIMAL(18,2) NOT NULL CHECK (amount > 0),
currency CHAR(3) NOT NULL DEFAULT 'USD',
balance_after DECIMAL(18,2) NOT NULL,
description TEXT, reverses_entry UUID,
idempotency_key VARCHAR(64), posted_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
) PARTITION BY RANGE (posted_at);
CREATE TABLE ledger_entries_2026_03 PARTITION OF ledger_entries
FOR VALUES FROM ('2026-03-01') TO ('2026-04-01');
CREATE TABLE account_balances (
account_id UUID PRIMARY KEY,
balance DECIMAL(18,2) NOT NULL DEFAULT 0,
currency CHAR(3) DEFAULT 'USD', updated_at TIMESTAMPTZ DEFAULT NOW()
);| Concern | Solution |
|---|---|
| Partial posting | All entries in single DB transaction; all-or-nothing |
| Duplicate posting | Idempotency key with UNIQUE constraint |
| Balance drift | Nightly reconciliation: recompute all balances from entries |
| Data loss | Synchronous replication; WAL archiving; point-in-time recovery |
| Immutability violation | No UPDATE/DELETE permissions; DB user has INSERT-only privilege |
| Regulatory audit | 7-year retention; partitioned tables with cold storage |
Interview Walkthrough
- Lead with double-entry bookkeeping: every transaction produces balanced debit and credit entries — the sum of all entries is always zero.
- Explain append-only ledger_entries as immutable source of truth; account_balances is a derived cache updated in the same transaction.
- Walk through idempotent posting: check idempotency_key → insert processed_requests → post entries + update balance atomically.
- Cover cross-shard transfers via saga with suspense accounts — each shard posts a self-balanced transaction, ops reconciles if a shard fails.
- Mention pessimistic locking (
SELECT ... FOR UPDATE) plus CHECK (balance >= 0) for concurrent withdrawal race conditions. - Discuss hot-account aggregation: batch merchant deposits in Redis, flush as a single ledger entry every 5 seconds to reduce row contention.
- Common pitfall: storing only the balance column without an append-only ledger — disputes and audits have no transaction history to reconstruct.
Why PostgreSQL (Not Blockchain/DynamoDB)
Blockchain: decentralized trust, but 7-30 TPS for Bitcoin. Banking ledger doesn't need decentralized trust — the bank IS the trusted authority. PostgreSQL: 12K TPS easily. DynamoDB: no multi-row transactions. Can't atomically post debit + credit entries. Financial ledger REQUIRES strong consistency and multi-row atomic writes. PostgreSQL: ACID, serializable isolation, CHECK constraints, partitioning. The right tool for this job.
Cross-Shard Transfers
Problem: Account A (Shard 3) to Account B (Shard 11). Single PostgreSQL transaction can't span two shards. Solution: Saga Pattern with suspense accounts Shard 3: DEBIT customer:A $100, CREDIT suspense:outgoing $100 Shard 11: DEBIT suspense:incoming $100, CREDIT customer:B $100 Each shard has a self-consistent, balanced transaction. "Suspense" accounts act as intermediate holding. If Shard 11 fails -> retry with exponential backoff -> alert ops after 24h Every major payment company (Stripe, Square, PayPal) uses this pattern.
Idempotent Posting: Preventing Double Charges
Flow: 1. Check: SELECT transaction_id FROM processed_requests WHERE idempotency_key = ? 2. INSERT INTO processed_requests -> UNIQUE constraint prevents duplicates 3. INSERT ledger entries + UPDATE balances in same transaction 4. Return result The UNIQUE constraint on idempotency_key is the final safety net.
Race Condition: Concurrent Withdrawals
Account A balance: $500. Two ATM withdrawals simultaneously: $400 each. Solution: SELECT ... FOR UPDATE (pessimistic row lock) Thread 1: SELECT ... FOR UPDATE -> sees 500 -> UPDATE balance = 100 -> COMMIT Thread 2: BLOCKS -> sees 100 -> 100 < 400 -> ROLLBACK Additional safety: CHECK (balance >= 0) constraint For hot accounts (merchant receiving 1000 payments/sec): Aggregate payments in Redis, flush as single batch entry every 5 seconds.
Staff interviews expect you to articulate how the system evolves under real growth — not jump straight to the final architecture.
Phase 1: MVP (0 to 100K users)
Monolith or minimal services proving core banking ledger system flows. Optimize for shipping speed and correctness over scale.
Key components: Single region · Primary DB + Redis cache · Synchronous core path · Basic monitoring
Move to next phase when: p99 latency exceeds SLO or DB CPU sustained above 70%
Phase 2: Growth (100K to 10M users)
Split read/write paths, introduce async processing for non-critical work, add caching layers and horizontal scaling.
Key components: Read replicas or CQRS · Message queue for async work · CDN / edge caching · Service-level SLOs
Move to next phase when: Hot keys, fan-out bottlenecks, or ops toil from manual scaling
Phase 3: Scale (10M+ users)
Shard data plane, multi-region active-active or active-passive, formal DR runbooks, cost optimization.
Key components: Database sharding / partitioning · Multi-region replication · Auto-scaling + chaos testing · Dedicated platform/SRE ownership
Move to next phase when: Regional failure domain risk, compliance data residency, or linear cost growth unsustainable
SLOs & Error Budgets
| Metric | Target | Rationale |
|---|---|---|
| Core user-facing availability | 99.95% | Budget for planned maintenance + unplanned failures without user-visible outage. |
| p99 latency (critical path) | Problem-specific — state target early and tie to capacity math | Interview credibility comes from connecting SLO to architecture choices. |
| Error rate (5xx) | < 0.1% | Distinguishes transient blips from systemic failure requiring rollback. |
| Data durability | 99.999999999% (11 nines) for committed writes | Define which operations require fsync/quorum vs async replication. |
Incident Scenarios (2am reality)
| Scenario | How you detect | Mitigation |
|---|---|---|
| Primary database unavailable | Health check failures, connection pool exhaustion alerts, elevated 5xx | Failover to replica / promote standby; enable read-only degraded mode if writes impossible; queue writes if async path exists |
| Traffic spike (10× normal) | RPS anomaly alert, autoscaling lag, latency SLO burn rate | Rate limit non-critical endpoints; scale read path horizontally; pre-warm caches; shed load on expensive operations |
| Bad deploy causing elevated errors | Canary metric regression, error budget burn, deployment correlation | Automated rollback within 5 minutes; feature flag kill switch; maintain N-1 compatibility |
Cost Drivers (Staff lens)
- Egress bandwidth and CDN (often dominates media/data-heavy systems)
- Database storage + IOPS at scale (plan compaction, TTL, tiering)
- Compute for async pipelines (right-size workers, spot instances for batch)
- Managed service premiums vs operational headcount trade-off
Multi-Region & DR
Start single-region with cross-AZ redundancy. Add read replicas in secondary region for DR. Move to active-active only when latency SLO or data residency requires it — accept conflict resolution complexity explicitly.