Online Judge (LeetCode-lite)

Requirements & System Scope

Functional Scope (In-Scope)

Sandbox Execution Strategies (Strategy Pattern): Dynamic code evaluation matching code dialects (Python, Java) via clean sandbox interfaces.
Aggregate Test Case Evaluation: Evaluate sandboxed stdout/stderr output against expected outcomes, yielding AC, WA, TLE, or RE results.
Asynchronous Execution Worker Pools: Offload judge tasks asynchronously using multi-threaded pools to prevent main-thread blockages.
Leaderboard & Stats Pipelines (Observer Pattern): Push real-time stats updates and award points upon successful execution.

Explicit Boundaries (Out-of-Scope)

No Automatic Anti-Plagiarism Scanner: Bypasses complex lexical analysis matching structural tokens or abstract syntax trees (ASTs).
No Advanced Peer Review Forum: Out-of-scope to build social commenting forums or collaborative code reviews.

Class Diagram & Entity Relationships

Structural layout showing relationships between runners, test cases, evaluation engines, and observers:

Polymorphic Code Runners: CodeRunner abstracts sandbox executions away from submission managers.
Test Case Validators: TestCase pairs input matrices with exact target expected outputs.

Design Patterns & SOLID Principles

Strategy Design Pattern (Sandbox Isolation): Decoupling execution strategies under an independent runner interface allows new language interpreters to be introduced with zero changes to the engine lifecycle.
Observer Design Pattern (Statistics Updating): Triggers real-time leaderboard point accruals and stats computations dynamically upon successful judgements without polluting judge loops.
Single Responsibility Principle (SRP): Isolating output comparison math from execution sandboxes protects test validators from system process anomalies and thread termination issues.

Core Execution Workflows

Code Evaluation Lifecycle

User uploads code: submit(userId, problemId, code, lang).
Queue Submission into worker channel queues.
Background thread claims task, fetching relevant Problem test criteria.
Resolve runner strategy: RunnerFactory.getRunner(lang).
For each test case:
1. Trigger execution in sandbox: runner.execute(code, input, timeLimitMs).
2. If execution takes longer than timeLimitMs, abort process returning TLE.
3. If process errors or crashes, return RE.
4. Compare terminal output with expected results. If mismatched, record WA.
If all test cases pass successfully, set status to AC and broadcast successful result to leaderboard and solved statistics observers.

Concurrency & Thread Safety Strategy

When thousands of users submit solutions simultaneously during competitive programming contests:

Bounded Thread Pools: Limit sandbox process allocations to a fixed pool of CPUs to prevent resource starvation.
Process Isolation: Run submissions in separate OS containers or lightweight secure runtimes.

Complete Clean Code Blueprint

Clean reference designs demonstrating sandbox runner abstractions in Java and Python:

// ─── JAVA BLUEPRINT ──────────────────────────────────────────────────────────
import java.util.*;
import java.util.concurrent.*;
import java.util.concurrent.locks.*;

enum JudgeResult { AC, WA, TLE, MLE, RE }

class TestCase {
    private final String input;
    private final String expectedOutput;

    public TestCase(String input, String expectedOutput) {
        this.input = input;
        this.expectedOutput = expectedOutput;
    }

    public String getInput() { return input; }
    public String getExpectedOutput() { return expectedOutput; }
}

class Problem {
    private final String id;
    private final String title;
    private final List<TestCase> testCases;
    private final long timeLimitMs;

    public Problem(String id, String title, List<TestCase> testCases, long timeLimitMs) {
        this.id = id;
        this.title = title;
        this.testCases = testCases;
        this.timeLimitMs = timeLimitMs;
    }

    public String getId() { return id; }
    public String getTitle() { return title; }
    public List<TestCase> getTestCases() { return testCases; }
    public long getTimeLimitMs() { return timeLimitMs; }
}

class ExecutionResult {
    private final String output;
    private final boolean isSuccess;
    private final long timeUsedMs;

    public ExecutionResult(String output, boolean isSuccess, long timeUsedMs) {
        this.output = output;
        this.isSuccess = isSuccess;
        this.timeUsedMs = timeUsedMs;
    }

    public String getOutput() { return output; }
    public boolean isSuccess() { return isSuccess; }
    public long getTimeUsedMs() { return timeUsedMs; }
}

// ─── STRATEGY PATTERN (SANDBOX EXECUTOR) ─────────────────────────────────────
interface CodeRunner {
    ExecutionResult execute(String code, String input, long timeoutMs);
}

class PythonRunner implements CodeRunner {
    @Override
    public ExecutionResult execute(String code, String input, long timeoutMs) {
        if (code.contains("syntax_error")) {
            return new ExecutionResult("SyntaxError: invalid syntax", false, 10);
        }
        if (code.contains("infinite_loop")) {
            return new ExecutionResult("Timeout", false, timeoutMs + 50);
        }
        // Simulating matching output
        return new ExecutionResult(input.trim(), true, 45);
    }
}

class JavaRunner implements CodeRunner {
    @Override
    public ExecutionResult execute(String code, String input, long timeoutMs) {
        if (code.contains("compile_error")) {
            return new ExecutionResult("Compilation failed", false, 5);
        }
        return new ExecutionResult(input.trim(), true, 30);
    }
}

class Submission {
    private final String id;
    private final String userId;
    private final String problemId;
    private final String code;
    private final String language;
    private JudgeResult result = JudgeResult.WA;

    public Submission(String id, String userId, String problemId, String code, String language) {
        this.id = id;
        this.userId = userId;
        this.problemId = problemId;
        this.code = code;
        this.language = language;
    }

    public String getId() { return id; }
    public String getUserId() { return userId; }
    public String getProblemId() { return problemId; }
    public String getCode() { return code; }
    public String getLanguage() { return language; }
    public JudgeResult getResult() { return result; }
    public void setResult(JudgeResult result) { this.result = result; }
}

// ─── OBSERVER PATTERN (LEADERBOARDS & STATS) ─────────────────────────────────
interface SubmissionObserver {
    void onSubmissionJudged(Submission submission);
}

class LeaderboardUpdater implements SubmissionObserver {
    private final Map<String, Integer> userPoints = new ConcurrentHashMap<>();

    @Override
    public void onSubmissionJudged(Submission submission) {
        if (submission.getResult() == JudgeResult.AC) {
            userPoints.merge(submission.getUserId(), 10, Integer::sum);
            System.out.println(String.format("[Leaderboard] User %s scored +10 points! Total: %d", 
                    submission.getUserId(), userPoints.get(submission.getUserId())));
        }
    }
}

class UserStatManager implements SubmissionObserver {
    private final Map<String, Set<String>> userSolvedProblems = new ConcurrentHashMap<>();

    @Override
    public void onSubmissionJudged(Submission submission) {
        if (submission.getResult() == JudgeResult.AC) {
            userSolvedProblems.computeIfAbsent(submission.getUserId(), k -> ConcurrentHashMap.newKeySet())
                    .add(submission.getProblemId());
            System.out.println(String.format("[Stats] User %s has now solved %d unique problem(s).",
                    submission.getUserId(), userSolvedProblems.get(submission.getUserId()).size()));
        }
    }
}

class JudgeEngine {
    private final Map<String, Problem> problems = new ConcurrentHashMap<>();
    private final Map<String, CodeRunner> runners = new ConcurrentHashMap<>();
    private final List<SubmissionObserver> observers = new CopyOnWriteArrayList<>();
    private final ExecutorService evaluationExecutor = Executors.newFixedThreadPool(4);

    public JudgeEngine() {
        runners.put("python", new PythonRunner());
        runners.put("java", new JavaRunner());
    }

    public void addProblem(Problem p) { problems.put(p.getId(), p); }
    public void addObserver(SubmissionObserver obs) { observers.add(obs); }

    public void submit(Submission submission) {
        System.out.println("[JudgeEngine] Submission " + submission.getId() + " queued for execution...");
        evaluationExecutor.submit(() -> evaluate(submission));
    }

    private void evaluate(Submission submission) {
        Problem problem = problems.get(submission.getProblemId());
        CodeRunner runner = runners.get(submission.getLanguage().toLowerCase());

        if (problem == null || runner == null) {
            submission.setResult(JudgeResult.RE);
            notifyObservers(submission);
            return;
        }

        JudgeResult finalResult = JudgeResult.AC;
        for (TestCase tc : problem.getTestCases()) {
            ExecutionResult execRes = runner.execute(submission.getCode(), tc.getInput(), problem.getTimeLimitMs());

            if (!execRes.isSuccess()) {
                if (execRes.getOutput().contains("Timeout")) {
                    finalResult = JudgeResult.TLE;
                } else {
                    finalResult = JudgeResult.RE;
                }
                break;
            }

            if (execRes.getTimeUsedMs() > problem.getTimeLimitMs()) {
                finalResult = JudgeResult.TLE;
                break;
            }

            if (!execRes.getOutput().equals(tc.getExpectedOutput())) {
                finalResult = JudgeResult.WA;
                break;
            }
        }

        submission.setResult(finalResult);
        System.out.println(String.format("[JudgeEngine] Submission %s judged: %s", submission.getId(), finalResult));
        notifyObservers(submission);
    }

    private void notifyObservers(Submission submission) {
        for (SubmissionObserver obs : observers) {
            obs.onSubmissionJudged(submission);
        }
    }

    public void shutdown() {
        evaluationExecutor.shutdown();
    }
}

public class Main {
    public static void main(String[] args) throws InterruptedException {
        System.out.println("=== Online Judge Simulation ===");
        JudgeEngine engine = new JudgeEngine();
        
        LeaderboardUpdater leaderboard = new LeaderboardUpdater();
        UserStatManager stats = new UserStatManager();
        engine.addObserver(leaderboard);
        engine.addObserver(stats);

        List<TestCase> testCases = Arrays.asList(
            new TestCase("2 3", "5"),
            new TestCase("10 20", "30")
        );
        Problem sumProblem = new Problem("PROB-1", "Sum of Two Numbers", testCases, 1000);
        engine.addProblem(sumProblem);

        // Client Submissions
        Submission sub1 = new Submission("SUB-101", "User-A", "PROB-1", "def sum(a,b): return a+b", "python");
        Submission sub2 = new Submission("SUB-102", "User-A", "PROB-1", "while True: pass # infinite_loop", "python");

        engine.submit(sub1);
        engine.submit(sub2);

        Thread.sleep(1000);
        engine.shutdown();
    }
}

Help Us Improve

How helpful was this walkthrough?

Click a star to rate. We actively use this feedback to refine and update our system design content.

Placeholder

Discussion

Share your thoughts, ask questions, or help others.

Loading comments...

✦Requirements & System Scope

Functional Scope (In-Scope)

Explicit Boundaries (Out-of-Scope)

◫Class Diagram & Entity Relationships

⬡Design Patterns & SOLID Principles

{ }Core Execution Workflows