Interview Setup
Interview Prompt
Design a payment gateway that processes card payments for merchants. Support authorization, capture, refunds, and voids. Integrate with external payment service providers (PSPs) like Stripe or Adyen. Ensure no double charging and provide reconciliation with PSP settlement files.
Clarifying Questions (ask before designing)
| Question | Why it matters |
|---|---|
| Authorization only or authorize and capture in one step? | Two phase authorize and capture is standard for e commerce with delayed fulfillment, whereas single step capture suits instant digital goods. This distinction dictates state machine complexity. |
| What's the transaction volume and peak TPS? | 20K TPS peak drives async webhook processing, idempotency store sharding, and PSP rate limit handling. |
| Do we hold funds or pass through to merchant accounts? | A marketplace platform model requires a double entry ledger and split payouts, whereas pass through routing lets the PSP handle downstream settlement directly. |
| Which PCI scope are we targeting? | Choosing between SAQ A tokenization redirects and SAQ D card data storage determines whether our infrastructure ever handles primary account numbers directly. |
Scope
In scope
- Authorize, capture, void, refund APIs
- Idempotency key handling
- PSP adapter layer (Stripe/Adyen)
- Internal ledger (double entry)
- Reconciliation with PSP settlement files
- Webhook ingestion from PSPs
- Transactional outbox for payment event publication
Out of scope (state explicitly)
- Fraud ML model training (mention rules engine + 3DS)
- Merchant onboarding/KYC
- Chargeback dispute workflow (mention as async process)
- Building a PSP from scratch
Functional Requirements
Start by asking your interviewer about payment processing, refunds, and idempotency requirements. PCI scope boundaries and multi currency are typical follow ups.
- Process payments: Accept payments through credit cards, debit cards, bank transfers, and digital wallets such as Apple Pay and Google Pay.
- Authorize and capture: Support both two phase payment flows that authorize a hold before capturing funds and single step direct charges.
- Refunds: Process full and partial refunds with balance adjustments against original transactions.
- Recurring payments: Support integration hooks for subscription billing and scheduled automatic debits. Detailed subscription lifecycle management is covered separately.
- Multi currency: Accept presentment currency from customers and settle in merchant local currency with exchange rate conversion.
- Tokenization: Replace card details with non reversible vault tokens inside the appropriate PCI scoped tokenization boundary.
- Webhooks: Deliver real time notifications of payment status changes to merchant endpoints.
- Retry failed payments: Automatically retry transient network errors with exponential backoff.
- Double entry ledger: Maintain immutable debit and credit bookkeeping records for every fund movement.
- Merchant dashboard: Provide transaction histories, settlement reports, chargeback management, and payment analytics.
Non-Functional Requirements
Your interviewer will care most about effectively once payment processing and ACID guarantees on this problem. Latency matters, but correctness and auditability come first, so emphasize that distinction before drawing the state machine.
- ACID and effectively once charges: Guarantee strong consistency for each payment while idempotency keys prevent duplicate requests from creating a second charge.
- High availability: Achieve 99.99% availability because gateway downtime directly blocks merchant revenue.
- Low latency: Target a final synchronous authorization response within 2 seconds under normal provider conditions, while unresolved provider calls can enter a pending state after a bounded timeout.
- Security: Use TLS 1.3 in transit and AES-256 at rest for protected application data, while keeping raw cardholder data inside the designated tokenization boundary.
- Idempotency: Ensure identical payment requests submitted multiple times produce one effective financial operation and return the same durable result.
- Auditability: Record every state change in an immutable append only log for financial compliance and regulatory audits.
- Scalability: Support sustained throughput of 6,000 transactions per second with peaks reaching 20,000 transactions per second.
- Fault tolerance: Tolerate sudden datacenter and PSP provider outages without losing acknowledged transactional state or creating inconsistent account balances.
Capacity Estimations
Run this math before you size the ledger database. TPS and transaction log growth tell you storage requirements and how long you must retain audit records.
| Metric | Calculation | Value |
|---|---|---|
| Transactions / day | Given (500M txn/day) | 500M |
| Transactions / sec | Given | ~6,000 (peak 20K) |
| Avg transaction size | Given (1 KB metadata per txn) | 1 KB (metadata) |
| Ledger entries / day | 500M txn x 2 entries (debit + credit) | 1B (2 entries per txn: debit + credit) |
| Ledger storage / day | 1B x 500 bytes | 500 GB |
| Ledger storage / year | Given | ~180 TB |
| Active merchants | Given (5M merchants) | 5M |
Architecture Diagram
In the room: state idempotency keys and authorize then capture before diving into PCI details.
Walk your interviewer through the diagram by payment lifecycle. Merchants submit payment requests with idempotency keys, and the payment orchestrator executes a durable state machine backed by ACID persistence. Card data is tokenized so raw PANs never reach gateway application servers. Authorization and capture run through PSP adapters, while settlements and provider callbacks are reconciled asynchronously through durable event processing. Every durable state transition is recorded in the audit log, and every financial movement is recorded in the ledger. The same PostgreSQL transaction that records the durable payment result, financial ledger entries, and audit event also writes the payment outbox record that later publishes the event to Kafka.
Component Deep Dives
Payment Orchestrator: Core State Machine
Next we walk through each box on the diagram. Idempotency, the payment state machine, and reconciliation are the three pillars interviewers probe.
The orchestrator coordinates the entire payment lifecycle, ensuring that every state transition remains durable, auditable, and strictly transactional.
A payment advances through a deterministic state machine where every durable transition executes within a PostgreSQL transaction alongside the immutable audit log and payment outbox record. Every client mutation includes an Idempotency-Key header. Before initiating any payment action, the orchestrator verifies whether the key has already been evaluated. If the key already has a completed result, the gateway returns that cached response immediately.
Idempotency: Critical Design Decision
Idempotency keys ensure that network retries and duplicate API invocations never produce multiple financial charges.
-- Atomic idempotency check, row level locking, and completion lifecycle
BEGIN TRANSACTION;
-- 1. Check existing key status and acquire an exclusive row level lock
SELECT status, response_code, response_body, request_hash, expires_at
FROM idempotency_keys
WHERE key = :idempotency_key AND merchant_id = :merchant_id
FOR UPDATE;
-- 2. If the record exists and status is 'completed', return the cached response.
-- 3. If the record exists with an unexpired 'in_flight' status, return HTTP 409 Conflict or hold until completion.
-- 4. If the reservation expired, reconcile the PSP outcome before taking ownership.
-- 5. If no record exists, insert an in flight reservation with a short bounded lease:
INSERT INTO idempotency_keys (merchant_id, key, request_hash, status, created_at, expires_at)
VALUES (:merchant_id, :idempotency_key, :request_hash, 'in_flight', NOW(), NOW() + INTERVAL '10 minutes');
-- A concurrent insert is rejected by the composite primary key. Re-read the row and compare request_hash before reuse.
COMMIT;
-- 6. Execute payment authorization or capture through the selected PSP adapter...
-- 7. Atomically persist the final response and mark the key as completed:
UPDATE idempotency_keys
SET status = 'completed',
response_code = :response_code,
response_body = :response_json,
expires_at = GREATEST(expires_at, NOW() + INTERVAL '24 hours')
WHERE key = :idempotency_key AND merchant_id = :merchant_id;Why this matters: If a merchant server experiences a network timeout or crashes after submitting a charge but before receiving the gateway response, the client will retry the request. Idempotency guarantees that retries return the cached result rather than creating a duplicate financial charge.
Tokenization Service (Card Vault)
Raw primary account numbers never touch gateway application servers. When the payment page uses a qualifying PSP hosted tokenization flow, the gateway may be eligible for SAQ A treatment, subject to the exact integration and validation requirements.
- Vault isolation: Securely store sensitive cardholder data including PAN and CVV in a physically segregated, PCI compliant card vault.
- Token exchange: Client side SDKs submit card data directly to the tokenization vault, which encrypts values using AES-256 in an HSM backed key store and returns an opaque reference token such as
tok_4242424242424242. - Compliance boundary: Only the isolated tokenization vault handles raw card data. Downstream services receive opaque tokens, while the exact PCI scope depends on the integration and validation model.
- Hardware security modules: Keep encryption keys inside dedicated HSM devices under a policy that prevents key export and limits direct key exposure.
Fraud Detection Engine
Fraud scoring evaluates transactions inline during the authorization phase so that suspicious payments are blocked before funds are reserved or moved.
- Real time evaluation: Complete inline risk scoring under 100 milliseconds using velocity caps such as more than 5 transactions in one minute, geographic anomalies, and reputation lookups against known fraudulent IP addresses and device fingerprints.
- Machine learning scoring: Score payments against models trained on historical chargeback data, evaluating features including transaction amount, merchant category, device age, and behavioral patterns.
- Configurable rules engine: Allow merchants to define custom risk thresholds, such as requiring 3D Secure verification for transactions exceeding $10,000.
Payment Processor Connector (PSP Router)
The processor connector manages integrations across multiple PSPs such as Stripe and Adyen, optimizing routing choices while preventing duplicate external charges during failover.
- Smart routing: Dynamically select the optimal payment processor for each transaction based on card network, geographic acquiring costs, historical authorization success rates, and interchange fees.
- Automated failover: If the primary provider fails before the request outcome is accepted, route the request to a secondary provider. If a timeout leaves the outcome ambiguous, query the primary provider or reconciliation state first. Never blindly replay an uncertain charge against another PSP.
- Intelligent retry handling: Distinguish soft declines such as temporary insufficient funds for scheduled retries after 24 hours from hard declines such as lost or stolen cards that must never be retried. PSP idempotency keys protect safe retries within the same provider, while ambiguous outcomes require a provider status check before another external charge is attempted.
Ledger Service: Double Entry Bookkeeping
Every financial movement posts balanced entries to an immutable double entry ledger, while non monetary state transitions are captured in the audit log. This separation preserves strict financial auditability without creating ledger entries for purely operational state changes.
Every financial movement creates balanced debit and credit entries whose totals match in the applicable currency. For example, a $100 capture can debit pending for $100, credit merchant balance for $97, and credit platform fee revenue for $3. The ledger is append only and strictly immutable, so adjustments require explicit reversal records rather than modifications. A background reconciliation job continuously verifies that debits equal credits across the platform.
Settlement Service
The settlement service aggregates captured payments and reconciles net balances with banking networks asynchronously on scheduled payout cycles.
- Settlement schedules: Aggregate captured payments per merchant on standard T+1 or T+2 schedules for batch payout processing.
- Net settlement calculation: Calculate merchant payouts using the formula
payout = sum(captures) - sum(refunds) - sum(fees). - Automated payouts: Execute daily batch jobs to aggregate net balances and dispatch bank transfers over Automated Clearing House (ACH) or SWIFT rails.
Webhook Delivery Service
Merchants receive payment status notifications asynchronously through signed webhooks backed by persistent retry queues and idempotency guards. Delivery is at least once and can arrive out of order, so merchants use the payment version or provider event ID to process state changes safely.
- Event consumption: Consumes state changes from the Kafka
payment-eventstopic and enqueues outbound HTTP POST requests to configured merchant callback URLs. - Cryptographic signing: Signs each payload with an HMAC-SHA256 signature using a shared merchant secret so recipients can verify request authenticity and prevent spoofing.
- Exponential backoff: Retries failed deliveries with exponential backoff ranging from 1 second up to 24 hours, while allowing merchants to poll payment status endpoints directly if delivery fails.
- Delivery idempotency: Dedupes delivery tasks by the provider and provider event ID pair so repeated provider retries do not trigger duplicate webhook deliveries, while distinct partial captures remain deliverable.
Event Bus Design (Kafka)
The event bus decouples real time payment processing from downstream notification and reconciliation systems while buffering large traffic bursts.
topics:
payment-events:
partitions: 64 # Partitioned by payment_id to maintain state transition ordering per payment
retention: 90d # Retained for operational replay and downstream recovery
producers:
- payment-outbox-relay # Publishes committed payment state transitions after the database transaction
consumers:
- webhook-delivery-service
- merchant-dashboard-indexer
- payment-analytics-pipeline
ordering_key: "payment_id"
deduplication_key: "event_id"
webhook-delivery:
partitions: 32 # Partitioned by merchant_id to isolate tenant delivery queues
producers:
- webhook-delivery-service # Generates outbound HTTP delivery tasks
consumers:
- webhook-worker-pool # Executes HTTP dispatches with exponential backoff up to 24 hours
settlement-events:
partitions: 16 # Partitioned by merchant_id for batch settlement grouping
producers:
- settlement-service # Publishes daily captured payment aggregates
consumers:
- ledger-reconciliation-engine
- payout-batch-system
- financial-reporting-service
cluster_configuration:
replication_factor: 3
min_insync_replicas: 2
producer_idempotence: true # Reduces duplicate Kafka records caused by producer retries. Consumers still dedupe by event_id
dead_letter_queues:
- payment-events-dlq
- webhook-delivery-dlq
execution_paths:
synchronous: "Idempotency check -> fraud evaluation -> PSP invocation -> PostgreSQL transaction for payment, ledger, audit, and outbox -> return final state when known"
asynchronous: "Outbox relay publishes payment-events, merchant webhook delivery, daily settlement batch computation, and background ledger reconciliation"API Design
Payment Gateway API Domain Signatures
The gateway API provides idempotent payment creation, two phase capture, refund, and void endpoints alongside asynchronous webhook notifications and standardized error handling.
TypeScript interfaces defining domain contracts, state enumerations, and mutation payloads across the gateway lifecycle:
type MoneyMinorUnits = number; // Nonnegative integer in the smallest currency unit
type CurrencyCode = string; // ISO 4217 code such as USD or EUR
type PaymentMethodToken = string; // Opaque PSP token reference
type PaymentId = string;
type IdempotencyKey = string;
type PaymentStatus =
| "created"
| "validated"
| "authorized"
| "pending"
| "pending_capture"
| "partially_captured"
| "captured"
| "settled"
| "partially_refunded"
| "refunded"
| "voided"
| "reversed"
| "failed";
interface CreatePaymentRequest {
amount: MoneyMinorUnits;
currency: CurrencyCode;
payment_method: PaymentMethodToken;
capture?: boolean; // Set true for immediate capture or false for authorization only hold
description?: string;
metadata?: Record<string, string>;
return_url?: string;
}
interface PaymentResponse {
payment_id: PaymentId;
status: PaymentStatus;
amount: MoneyMinorUnits;
currency: CurrencyCode;
payment_method: PaymentMethodToken;
created_at: string; // ISO 8601 timestamp
}
interface CapturePaymentRequest {
amount?: MoneyMinorUnits; // Optional amount for partial capture; defaults to full authorized amount
}
interface RefundPaymentRequest {
amount: MoneyMinorUnits;
reason?: "duplicate" | "fraudulent" | "customer_request";
}
interface VoidPaymentRequest {
reason?: "merchant_request" | "timeout_recovery" | "fraud_review";
}
interface WebhookPayload {
provider: string;
provider_event_id: string;
version: number; // Internal monotonic payment state version for ordering
event:
| "payment.authorized"
| "payment.captured"
| "payment.failed"
| "payment.refunded"
| "payment.voided"
| "payment.settled"
| "payment.chargeback";
payment_id: PaymentId;
chargeback_id?: string;
amount: MoneyMinorUnits;
currency: CurrencyCode;
timestamp: string; // ISO 8601 timestamp
}Create Payment
Submits a payment request for immediate authorization or full capture with mandatory idempotency key tracking. A definitive PSP result returns normally, while an unresolved provider timeout can return a pending state for asynchronous reconciliation. The response status may therefore be 200 for a known result or 202 when the final provider outcome remains unresolved.
POST /api/v1/payments
Idempotency-Key: idem-uuid-12345
Authorization: Bearer <merchant_api_key>
{
"amount": 10000,
"currency": "USD",
"payment_method": "tok_4242424242424242",
"capture": true,
"description": "Order #12345",
"metadata": {"order_id": "ORD-12345"},
"return_url": "https://merchant.com/payment/complete"
}
Response: 200 OK
{
"payment_id": "pay_uuid",
"status": "captured",
"amount": 10000,
"currency": "USD",
"payment_method": "tok_4242...",
"created_at": "2026-03-13T10:00:00Z"
}Capture for Authorization Only Payments
Captures previously authorized funds, supporting either the full amount or a partial capture amount.
POST /api/v1/payments/{payment_id}/capture
Idempotency-Key: capture-idem-uuid
Authorization: Bearer <merchant_api_key>
{
"amount": 10000
}Refund
Issues a full or partial refund against a captured, partially captured, or settled payment using a dedicated refund idempotency key.
POST /api/v1/payments/{payment_id}/refund
Idempotency-Key: refund-idem-uuid
Authorization: Bearer <merchant_api_key>
{
"amount": 5000,
"reason": "customer_request"
}Void
Voids an authorized payment before capture, using a dedicated idempotency key so retries cannot create conflicting provider operations.
POST /api/v1/payments/{payment_id}/void
Idempotency-Key: void-idem-uuid
Authorization: Bearer <merchant_api_key>
{
"reason": "merchant_request"
}Webhook
Delivers signed event notifications to merchant webhooks for payment authorization, capture, failure, refund, settlement, and void events.
POST https://merchant.com/webhooks/payment
X-Signature-SHA256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
{
"provider": "stripe",
"provider_event_id": "evt_123",
"version": 4,
"event": "payment.captured",
"payment_id": "pay_uuid",
"amount": 10000,
"currency": "USD",
"timestamp": "2026-03-13T10:00:01Z"
}Common Error Responses
Standardized error responses returned across payment creation, authorization, and network failure modes.
400 Bad Request: invalid input, missing required fields, or malformed JSON payload 401 Unauthorized: missing or invalid authentication token or API key 403 Forbidden: authenticated caller lacks required permissions for this resource 404 Not Found: requested resource ID does not exist 409 Conflict: duplicate write or version conflict, retry with a unique idempotency key 422 Unprocessable Entity: syntactically valid request failed semantic business validation 429 Too Many Requests: rate limit quota exceeded, client should honor Retry-After header 500 Internal Error: unexpected server failure, retry safely with an idempotency key 503 Service Unavailable: downstream dependency is unavailable or overloaded, retry with exponential backoff 402 Payment Required: account balance or payment method has insufficient funds 502 Bad Gateway: payment gateway provider timeout, poll transaction status endpoint
Data Model
PostgreSQL: Payments
The data model separates transactional payment state, idempotency records, double entry ledger bookkeeping, and reliable event publication into relational tables with strict integrity constraints.
Maintains the core payment entity and state machine lifecycle with a version column for optimistic locking and status indexing.
CREATE TABLE payments (
payment_id UUID PRIMARY KEY,
merchant_id UUID NOT NULL,
amount BIGINT NOT NULL, -- In cents
currency VARCHAR(3) NOT NULL,
status VARCHAR(24) NOT NULL, -- created, validated, authorized, pending, pending_capture, partially_captured, captured, settled, partially_refunded, refunded, voided, reversed, failed
payment_method VARCHAR(64), -- Token reference
capture_method VARCHAR(20), -- 'automatic' or 'manual'
description TEXT,
metadata JSONB,
psp_reference VARCHAR(128), -- External PSP transaction ID used for webhook/reconciliation lookup
psp_name VARCHAR(64), -- Processor handling the transaction
failure_reason TEXT,
idempotency_key VARCHAR(128),
version BIGINT NOT NULL DEFAULT 0,
created_at TIMESTAMP NOT NULL,
updated_at TIMESTAMP NOT NULL,
UNIQUE (merchant_id, idempotency_key),
UNIQUE (psp_name, psp_reference)
);
CREATE INDEX idx_merchant ON payments (merchant_id, created_at DESC);
CREATE INDEX idx_status ON payments (status);PostgreSQL: Idempotency Keys
Stores request hashes and cached responses to prevent duplicate transaction execution across network retries.
CREATE TABLE idempotency_keys (
merchant_id UUID NOT NULL,
key VARCHAR(128) NOT NULL,
request_hash VARCHAR(64) NOT NULL, -- SHA-256 hash of normalized request body
status VARCHAR(20) NOT NULL, -- 'in_flight', 'completed'
response_code INT,
response_body JSONB,
created_at TIMESTAMP NOT NULL,
expires_at TIMESTAMP NOT NULL, -- In flight lease; completed responses are retained for 24 hours
PRIMARY KEY (merchant_id, key)
);
CREATE INDEX idx_idempotency_created ON idempotency_keys (created_at);
CREATE INDEX idx_idempotency_expiry ON idempotency_keys (expires_at);PostgreSQL: Ledger (Append Only, Immutable)
Records double entry debit and credit lines for every fund movement. Materialized account balances are updated in the same transaction so balance reads do not require scanning the full ledger history.
CREATE TABLE ledger_accounts (
account_id UUID PRIMARY KEY,
merchant_id UUID,
account_type VARCHAR(32) NOT NULL,
currency VARCHAR(3) NOT NULL,
balance BIGINT NOT NULL DEFAULT 0,
updated_at TIMESTAMP NOT NULL,
UNIQUE (merchant_id, account_type, currency)
);CREATE TABLE ledger_entries (
entry_id BIGSERIAL PRIMARY KEY,
payment_id UUID NOT NULL REFERENCES payments(payment_id),
account_id UUID NOT NULL REFERENCES ledger_accounts(account_id),
entry_type VARCHAR(10) NOT NULL CHECK (entry_type IN ('debit', 'credit')),
amount BIGINT NOT NULL CHECK (amount > 0), -- In cents
currency VARCHAR(3) NOT NULL,
balance_after BIGINT, -- Running balance for the account
description TEXT,
created_at TIMESTAMP NOT NULL
);
CREATE INDEX idx_account ON ledger_entries (account_id, created_at);
CREATE INDEX idx_payment ON ledger_entries (payment_id);
-- Accounting Invariant: sum of debits = sum of credits per payment_idPostgreSQL: Payment Outbox
Stores payment events in the same database transaction as the payment, ledger, and audit updates. A separate relay publishes committed events to Kafka and safely retries unpublished rows.
CREATE TABLE payment_outbox (
outbox_id BIGSERIAL PRIMARY KEY,
payment_id UUID NOT NULL REFERENCES payments(payment_id),
version BIGINT NOT NULL,
event_id UUID NOT NULL UNIQUE,
event_type VARCHAR(64) NOT NULL,
payload JSONB NOT NULL,
created_at TIMESTAMP NOT NULL,
published_at TIMESTAMP NULL,
UNIQUE (payment_id, version)
);
CREATE INDEX idx_unpublished ON payment_outbox (published_at, created_at);PostgreSQL: Audit Log
Captures immutable historical records of all payment status transitions and triggering actors.
CREATE TABLE payment_audit_log (
log_id BIGSERIAL PRIMARY KEY,
payment_id UUID NOT NULL REFERENCES payments(payment_id),
old_status VARCHAR(20),
new_status VARCHAR(20),
actor VARCHAR(64), -- 'system', 'merchant', 'psp'
details JSONB,
created_at TIMESTAMP NOT NULL
);
-- Application roles have INSERT and SELECT permissions only; UPDATE and DELETE are denied.Kafka Topics
Buffers event streams that decouple core transaction processing from webhook delivery and batch settlement.
topics:
payment-events:
purpose: Disseminates payment state transitions to downstream indexing and analytics services
webhook-delivery:
purpose: Buffers outbound webhook delivery payloads for merchant notification workers
settlement-events:
purpose: Streams captured transactions to the daily settlement aggregation pipelineFault Tolerance
ACID Guarantees
Financial systems require defensive fault tolerance against network timeouts, third party provider downtime, and partial failure scenarios.
The relational database enforces transaction atomicity, consistency invariants, isolation boundaries, and durable replication.
| Property | How Ensured |
|---|---|
| Atomicity | PostgreSQL transactions ensure all or nothing execution across payments, ledger records, and audit events. |
| Consistency | Database constraints including CHECK, UNIQUE, and FOREIGN KEY enforce relational validity alongside application level balance invariants. |
| Isolation | Serializable isolation protects balance mutations, while Read Committed prevents dirty reads for dashboard queries without claiming repeatable read semantics. |
| Durability | PostgreSQL write-ahead logging (WAL) combined with synchronous replication to an active standby can provide zero data loss within the configured failure model, with automated failover and recovery procedures. |
Fault Tolerance Scenarios
Defensive strategies for handling provider disconnects, crashes after approval, and network partition edge cases.
| Concern | Solution |
|---|---|
| Network failure with PSP | Mark the payment as pending while a background reconciliation job queries the PSP every minute. A network failure is never treated as an automatic decline. |
| Service crash after PSP approval | The PSP sends an asynchronous webhook containing the authorization result. The webhook handler can create the missing payment record, while reconciliation catches any dropped or delayed events. |
| Double charge prevention | Assign a unique idempotency key to every mutation. Duplicate attempts return the cached result of the first completed execution, while the PSP also receives a provider idempotency key. |
| Partial failure (auth succeeded, capture failed) | Keep the payment in an appropriate pending state while the authorization hold remains valid for its provider defined lifetime. Background workers retry safely and alert on stuck captures. |
| Database failover | Synchronous replication to an active standby can provide zero data loss within the configured failure model, with automated failover and recovery procedures. |
| Webhook delivery failure | Retry deliveries using exponential backoff across 1 second, 2 seconds, 4 seconds, and up to 24 hours, while allowing merchants to poll payment status directly through the API. |
Additional Considerations
PCI DSS Compliance
Operationalizing a payment gateway requires strict adherence to regulatory standards, active dispute management, and resilient cross border settlement.
Keeping raw PAN and CVV data out of gateway application servers can reduce the gateway's PCI validation scope when the integration meets the relevant SAQ A eligibility requirements.
- When an internal tokenization vault is used, cardholder data such as primary account numbers and CVV codes remains strictly isolated within that dedicated PCI scoped boundary and is reachable only through a controlled tokenization endpoint. In the preferred PSP hosted model, the PSP vault handles the raw card data instead.
- All sensitive financial information is encrypted at rest using AES-256 and in transit using TLS 1.3.
- Network segmentation ensures the card vault runs in an isolated virtual private cloud with zero public ingress.
- Regular security verification includes quarterly automated vulnerability scans and annual third party penetration testing.
Chargeback Lifecycle
When a cardholder disputes a transaction, the payment service provider delivers a chargeback webhook to our ingestion endpoint. The gateway first verifies the cryptographic signature on the payload and freezes the disputed funds in the merchant balance. The merchant is notified to provide supporting evidence, such as signed delivery receipts or purchase records, which the gateway compiles and submits to the card network within the provider or network dispute deadline. A seven day window is an illustrative example, not a universal rule. If the dispute is won, the hold on merchant funds is released. If the dispute is lost, the gateway debits the merchant ledger, records the dispute fee, and marks the payment status as reversed. Every chargeback operation uses an idempotent chargeback_id to ensure duplicate provider webhooks never double debit merchant balances, and the normalized payment event carries that identifier for downstream deduplication.
Webhook Signature Verification
Incoming provider callbacks include an HMAC-SHA256 signature generated over the raw request payload using a shared secret. The gateway verifies this signature before parsing the JSON body to prevent deserialization attacks. Replay attacks are prevented by validating that the request timestamp falls within a five minute tolerance window, followed by recording the processed provider and provider_event_id pair in Redis using SET key value NX with a 24 hour expiration window.
Reconciliation
Multi tier reconciliation verifies mathematical integrity across internal ledgers, payment processors, and settlement bank accounts.
- Internal reconciliation: An hourly verification job confirms that internal payment records match the double entry ledger entries and that debit and credit sums balance to zero.
- External reconciliation: A daily batch process compares internal ledger captures against provider settlement files to identify fee variances, chargebacks, and dropped events.
- Bank reconciliation: A daily banking pipeline reconciles bank account statement deposits against expected net settlement amounts from payment processors.
Multi Currency Processing
Cross border commerce requires separating customer presentment currencies from merchant settlement accounts.
- Presentment currency: Accept payments in the customer local currency to maximize checkout conversion rates.
- Settlement currency: Deposit funds into merchant accounts in their domestic currency to eliminate foreign currency handling for merchants.
- Real time FX conversion: Lock exchange rates at the moment of payment capture using real time rate feeds to eliminate currency fluctuation risks.
- FX markup: Apply a standard 1% to 3% conversion fee on cross currency transactions to cover volatility buffers and provider spreads.
3D Secure (3DS) Authentication
Step up cardholder verification mitigates unauthorized transactions while fulfilling regional compliance rules.
- Cardholder verification: Provide step up authentication using one time passcodes or biometric prompts to prevent fraudulent online card use.
- 3DS 2.0 frictionless flow: Share risk data with card issuers so low risk transactions authenticate seamlessly without redirecting the customer.
- Regulatory compliance: Meet mandatory Strong Customer Authentication (SCA) requirements enforced across European Union transactions under PSD2.
Monitoring & Alerting
Operational metrics monitor financial health, provider availability, and processing latencies in real time.
- Provider success rates: Monitor authorization success rates per PSP and trigger alerts if any provider drops below 95%.
- Authorization latency: Track end to end response times and trigger alerts when average authorization latency exceeds 3 seconds.
- Fraud ratios: Monitor chargeback and fraud ratios closely, alerting operators if fraudulent transactions exceed 0.1% of total volume.
- Reconciliation discrepancies: Track automated matching rates across daily settlement runs and alert on material unmatched values, with $1 used as the example escalation threshold.
Interview Walkthrough
Structure your response to highlight the money path, idempotency guarantees, and distributed failure modes within the interview timeline.
- 25 minute cut
Focus on core money flow and state invariants before exploring edge cases.
- Requirements, scale numbers, and the idempotency golden rule (3 min)
- Authorize then capture state machine and database schema (8 min)
- Idempotency keys and concurrency locking on mutations (6 min)
- PCI scope boundaries and tokenization architecture (5 min)
- Asynchronous settlement and webhook reconciliation (3 min)
- State the golden rule immediately: payments must never double charge because every payment mutation, including creation, capture, refund, and void, uses an idempotency key.
- Model the distributed payment workflow using the Saga pattern through authorize, capture, and settlement phases, with explicit compensating actions such as voids and refunds at each failure point.
- Insist on ACID transactions in PostgreSQL for the ledger because CAP Theorem trade offs that favor eventual consistency over strong consistency are unacceptable for financial accounting.
- Minimize PCI scope by ensuring sensitive cardholder data never touches application servers, tokenizing payment details directly at the edge and storing only opaque vault references.
- Design webhook ingestion with idempotent event processing because external payment providers frequently retry webhook notifications and out of order delivery is expected.
- For recurring billing scenarios including subscriptions, invoices, and dunning workflows, defer to the Subscription Billing System, because this design focuses specifically on one time charge orchestration.
- Cover the three levels of reconciliation, including hourly internal ledger verifications, daily external settlement file matching, and proactive alerting on discrepancy rates.
- Discuss multi provider routing combined with Circuit Breaker, Retries, and Bulkheads patterns to provide seamless automated failover when a payment processor degrades.
- Highlight the common pitfall of treating the authoritative payment state as eventually consistent. PostgreSQL owns the durable payment state machine, while dashboards and webhook consumers may observe updates asynchronously.
Engineering Trade-offs
Why PostgreSQL Instead of Cassandra or DynamoDB for Payments?
Evaluate critical engineering trade offs between transactional consistency, latency, and operational complexity across the payment pipeline.
Payment processing demands strict ACID transactions to execute debit and credit pairs atomically, strong consistency for authoritative financial state, and relational querying for finance and compliance reporting. A relational database like PostgreSQL natively supports check constraints, foreign keys, and point in time auditing. In contrast, NoSQL datastores such as Apache Cassandra are less suitable for the authoritative ledger because the design requires relational constraints, transactional debit and credit updates, and financial reporting queries. Eventual consistency can still be useful for non authoritative views. Similarly, Amazon DynamoDB has transaction size and query model constraints that make PostgreSQL a more natural fit for the core financial ledger.
Authorize Then Capture vs Direct Charge
Direct charges request authorization and capture in one operation, which simplifies checkout but requires a refund when an order is cancelled after capture. In contrast, the recommended authorize then capture pattern splits the lifecycle into two phases. The first phase places an authorization hold on customer funds, and the second phase captures the money once merchant inventory or order fulfillment is confirmed. This two phase model can avoid a refund when an order is cancelled before capture, enables partial captures when items in an order ship separately, and creates a useful time window for additional fraud evaluation before funds move.
Idempotency: Three Layers of Defense
A robust payment gateway establishes three layers of duplicate charge protection. At the API layer, the gateway validates the idempotency key and uses a row level lock with SELECT FOR UPDATE to coordinate concurrent identical requests. At the provider integration layer, it forwards a stable provider idempotency key so safe network retries do not initiate duplicate external charges. At the database layer, the composite constraint across merchant_id and idempotency_key provides the final race condition barrier, while the full request hash rejects altered reuse of the same key.
Synchronous vs Asynchronous Payment Processing
Synchronous processing delivers immediate checkout decisions to users but holds open HTTP connections and makes the gateway vulnerable to provider latency spikes. Purely asynchronous processing releases client connections immediately but introduces polling or webhook complexity into consumer checkouts. The optimal design adopts a hybrid model. The gateway enforces a 2 second client response budget while the PSP adapter may keep an outstanding provider attempt alive under its own 10 second transport timeout. Under normal conditions the gateway returns the final authorization result within 2 seconds. If the client response budget is reached without a definitive provider outcome, the gateway durably records the payment as pending and hands the unresolved provider attempt to an asynchronous worker before returning an accepted pending state. If the provider attempt later times out or remains ambiguous, reconciliation resolves the outcome before any retry or compensating action. The definitive state is delivered to the merchant through webhooks.
Double Entry Bookkeeping: Why It Matters
In single entry systems, calculating platform balances requires scanning entire transaction histories and recomputing commissions, which easily introduces rounding drift and accounting errors. Double entry bookkeeping prevents these issues by recording every financial action as a balanced pair of debit and credit entries. The fundamental accounting invariant guarantees that the sum of debits equals the sum of credits for each payment within its accounting currency. If this invariant is ever violated, automated monitoring triggers an immediate critical alert. Because the ledger is strictly append only, operational corrections are never applied by updating existing rows, but rather by inserting new offsetting reversal entries that preserve a complete, tamper-evident audit history.
Smart Routing: Choosing the Right PSP Per Transaction
Smart routing continuously calculates a composite health score for each payment provider based on real time authorization success rates, transaction processing fees, availability metrics, and API latency. The router dispatches each transaction to the highest scoring provider eligible for that card network and region. If the selected provider fails before the request outcome is accepted, traffic can move to a secondary provider. If a timeout leaves the outcome ambiguous, the router first queries the original provider and reconciliation state rather than blindly replaying the charge elsewhere. In high volume platforms, this routing optimization can improve authorization rates by 2% to 5% when routing decisions are measurably better.
Review
How helpful was this walkthrough?
Click a star to rate. We actively use this feedback to refine and update our system design content.
Discussion
Share your thoughts, ask questions, or help others.