Optiver System Design Interview: What the Bar Actually Tests

June 2, 202610 min read
interview-prepcareersystem-designalgorithms
Optiver System Design Interview: What the Bar Actually Tests
TL;DR
  • The Optiver system design interview tests microsecond-latency systems like order books and market data feeds, not generic distributed systems patterns
  • UDP multicast beats TCP for market data because retransmissions introduce latency variance that trading systems cannot tolerate
  • Memory allocation in the hot path is a hire-killer; pre-allocated pools and slab allocators are the expected answer at every level
  • The first clarifying question should always be the latency target — the entire design changes between single-digit microseconds and sub-millisecond
  • Order book design uses a doubly linked list plus hash map per price level, giving O(1) cancel and O(k) match via price-time priority
  • Kafka and eventual consistency are rejection signals; Optiver wants deterministic, low-variance latency at every layer of the stack

You prepped for system design. You know how to horizontally scale a URL shortener. You can talk about eventual consistency for forty minutes without repeating yourself. You're ready.

Then you walk into Optiver's room and they ask you to design a system that executes trades in under one microsecond, never touches the heap in the hot path, and uses UDP on purpose.

Your FAANG playbook is now dead weight. This guide covers what Optiver's system design interview actually tests, how the bar shifts by level, and what gets candidates rejected.

The Interview Loop: Where System Design Fits

The full Optiver process has four stages. System design is not an early filter. It's the last technical gate.

  1. Online assessment (75 minutes): Algorithm problems plus trading-specific problems like order book simulation or matching sequence detection, hosted on HackerRank.
  2. CS fundamentals quiz (20 minutes): 20 multiple-choice questions covering networking, OS concepts, memory management, concurrency, and systems architecture.
  3. Technical interviews (2-3 rounds): Data structures, algorithms, and implementation problems. C++ is the production language at Optiver, but they accept other languages.
  4. Final loop: One system design round plus behavioral conversations with senior engineers and a hiring manager.

System design lands in the final loop. The round runs 45-60 minutes, it's conversational, and it's explicitly domain-relevant. Optiver's own interview guide says they expect you to ask clarifying questions, understand real-world constraints, and be able to justify every design decision you make.

For a broader picture of the full process, see the Optiver software engineer interview guide.

What "System Design" Means at a Trading Firm

At Optiver, system design is performance engineering. The mental model that works for FAANG interviews, think distributed, add a cache, handle failures with retries, will get you rejected here.

The gap between the two is not subtle:

DimensionFAANGOptiver
Latency targetMilliseconds to secondsMicroseconds to nanoseconds
Primary concernScalability, consistencyDeterminism, latency floor
Network protocolHTTP/TCP everywhereUDP multicast for market data
Memory modelGC pauses acceptableNo allocation in the hot path
Failure modelRetry, eventual consistencyStrict ordering, no missed fills
ScaleBillions of usersMillions of events per second

This changes what you say in the room. When a FAANG interviewer asks "how do you handle high load?" the right answer involves horizontal scaling and load balancers. When an Optiver interviewer asks the same question, the right answer involves pre-allocated memory pools, kernel bypass, and CPU pinning. The FAANG vs quant firm interview comparison goes deeper on this divide.

A Computer Scientist holds up four fingers and lectures a Software Engineer: "I got your four basic classes, P, NP-complete, Halting Problems, and Cryptography I didn't write." The Software Engineer stares, baffled.

Your FAANG system design prep meeting the Optiver interview.

The Problems They Ask

Optiver's system design questions map directly to their production infrastructure. Based on candidate reports, these appear most frequently:

Design a market data feed system that ingests stock and options prices in real-time, distributes them to downstream trading applications, and handles late or missing packets.

Design an order matching engine with price-time priority for a single exchange. Must support add, cancel, and modify with defined latency guarantees.

Design a low-latency market-making system that detects pricing opportunities across multiple venues in microseconds and routes orders accordingly.

Design a distributed cache for market data that trading applications can read without blocking, updated as new prices arrive.

Design a backtesting system for high-frequency strategies that replays historical market data at speed and simulates order execution accurately.

These are not abstract exercises. They're the actual systems running in Optiver's infrastructure.

What an Order Matching Engine Actually Looks Like

Take the matching engine, the most commonly asked problem. Here is what a strong answer covers.

The order book data structure. Each price level holds a doubly linked list of orders in FIFO order (price-time priority) and a hash map from order ID to list node. This gives O(log n) add via a sorted price map, O(1) cancel via the hash map, and O(k) match where k is the number of fills. In C++, the price level map is typically std::map<Price, PriceLevel>.

Order book diagram showing price levels with FIFO-ordered doubly linked lists on the bid and ask sides, and an O(1) hash map for cancel lookups by order ID

Matching logic. For a new buy order at price P, scan asks from the lowest ask upward while ask price is less than or equal to P. Fill greedily. The matching loop terminates when the incoming order is fully filled or no eligible opposite orders remain.

What Optiver probes beyond the basics. Memory layout. A naive implementation with new Order() per incoming order and pointer chasing through linked list nodes is cache-hostile, and interviewers will push on this. Better: a slab allocator or pre-allocated pool of Order structs. Nodes live in a contiguous array; you index them by slot number, not raw pointer. When the matching loop iterates through fills at a price level, it's reading sequential memory. Cache lines stay hot.

Latency numbers you should know. L1 cache hit is roughly 1ns, L2 around 5ns, DRAM around 80-100ns. A co-located order add should complete in under 1 microsecond end-to-end. Every cache miss in the matching loop is a budget line item, and interviewers know this.

What a Market Data Feed Design Covers

The second common problem: design the market data pipeline.

The core insight is UDP, not TCP. Market data is multicast over UDP because TCP's handshake, acknowledgment, and retransmission introduce latency variance that trading systems cannot tolerate. You get raw speed. You lose delivery guarantees. Both are intentional.

Handling gaps. Each packet carries a sequence number. Receivers track the last seen sequence. A gap means a dropped packet. The receiver requests retransmission from a secondary replay service. The primary feed never slows down for any individual receiver. Missing one subscriber is acceptable; blocking the feed is not.

Kernel bypass. At this latency tier, the Linux kernel networking stack adds microseconds you cannot afford. Technologies like DPDK and Solarflare's Onload bypass the kernel and deliver packets directly to user space. This alone cuts latency by 10-20 microseconds. For a senior candidate, knowing these exist, why they help, and what they cost in complexity is expected. Implementing one in an interview is not.

Distribution to downstream applications. Shared memory ring buffers are faster than loopback TCP. The feed handler writes to a shared memory segment; trading applications read from it. No copies, no syscalls, no kernel involvement.

How the Bar Shifts by Level

Optiver doesn't publish explicit leveling criteria, but the pattern across candidate reports is consistent.

New grad and junior. Show you understand the order book data structure. Know why UDP beats TCP for market data. Understand concurrency basics: what a mutex does, why lock-free is faster, what a memory barrier enforces. You don't need to design a full system from scratch, but you need to engage with latency trade-offs and not reach for solutions that introduce GC pauses or blocking operations.

Mid-level. Design complete systems with multiple components and justify every major trade-off. Understand kernel bypass at a conceptual level. Calculate rough latency budgets: how many operations fit in 10 microseconds given L1 cache hit rates? Know what NUMA architecture is and why CPU affinity matters for a trading process that cannot share a core with unrelated work.

Senior. Hardware-level awareness is expected. Know what FPGA acceleration offers (processing in the NIC, sub-microsecond latency) and when the operational complexity justifies it. Understand lock-free programming in detail: compare-and-swap, the ABA problem, memory ordering models (std::memory_order_acquire vs release vs relaxed). Design for zero-copy throughout. Senior candidates are expected to identify performance bottlenecks the interviewer hasn't mentioned yet.

What Gets You Rejected

Several failure modes appear consistently across candidate reports.

Proposing generic distributed systems patterns. Optiver does not want eventual consistency in a trading system. They do not want Kafka in the critical path of an order flow. A message queue in the matching pipeline adds latency variance that makes the system unpredictable. If your answer involves "we can add a Kafka layer for reliability," you've signaled you're solving a different problem.

Ignoring memory allocation. Saying "the system allocates an Order object for each incoming order" sounds routine until the interviewer asks about heap fragmentation or GC pauses under load. Pre-allocated pools and slab allocators are the expected answer. Memory allocation in the hot path is a latency risk.

A tweet where ChatGPT confidently says the podbay doors are now open, Dave confirms they are still closed, and ChatGPT apologizes and says they are now open, while Dave watches a set of closed podbay doors.

Explaining how your design achieves low latency right before they ask where you're allocating memory.

Not asking for the latency target first. This is the first clarifying question you should ask in the room. The entire design changes between "single-digit microseconds" and "sub-millisecond." The interviewer is watching for this.

Describing what you'd build without explaining why. Optiver interviewers will ask "why did you make this design decision?" If you chose UDP without knowing why TCP is worse for market data distribution, you won't survive the follow-up. The reasoning matters as much as the choice.

Skipping capacity estimation. Trading systems have defined loads. How many orders per second? How many symbols? What's peak rate at market open? These numbers set your data structure sizing, your memory pool capacity, and your CPU budget. Skip them and your design is disconnected from reality.

How to Prepare for the Optiver System Design Interview

Four weeks out:

Study order book design in depth. Understand price-time priority, the doubly linked list plus hash map structure, and why it achieves the right complexity profile. Implement it in your language of choice. The implementation will surface the memory layout questions you haven't thought about yet.

Learn the low-latency vocabulary: kernel bypass (DPDK, Onload), NUMA, CPU affinity, lock-free queues, memory barriers. You don't need to implement these. You need to know what problem each solves and what it costs.

Study UDP multicast. Understand why it beats TCP for market data, what sequence gaps mean operationally, and how receivers handle them. Be able to explain why the reliability trade-off is intentional.

The Optiver vs Jane Street interview comparison breaks down how the two firms differ in what they weight. For problem-type coverage, the quant and HFT coding interview problems list maps the algorithmic side well.

Two weeks out:

Refresh CS fundamentals for the quiz: networking (TCP vs UDP, sockets, multicast), OS concepts (process scheduling, context switch cost, cache lines), memory management (stack vs heap, fragmentation). Twenty questions, twenty minutes. Quick recall wins, not derivation.

One week out:

Practice designing out loud under a timer. Take one of the questions from the list above, draw the architecture, talk through it, and stress-test your own justifications by asking "why?" three times for each major decision. If you want real-time feedback on how your verbal reasoning lands, SpaceComplexity runs live DSA mock interviews with rubric-based feedback, the same kind Optiver's interviewers give when they decide whether your reasoning holds up.

Further Reading