Perplexity’s older retrieval engine struggled as its index grew, producing three clear pain points for teams running large‑scale search. First, tail latency rose to near 800 ms at p99 because the dataset no longer fit in RAM, causing costly page faults on cold reads. Second, during disk index fusion the p99 latency spiked to about 1.2 seconds for ten to fifteen minutes, creating unpredictable slowdowns that hurt user experience. Third, adding or syncing a new cluster took more than a week, extending recovery time and increasing the share of partial responses. These issues made the system expensive to operate and risky to scale.
Perplexity solved these problems by building Photon, an in‑house retrieval and ranking engine written in Rust. Photon replaces the old forked engine and now handles all production traffic. Its design tackles each pain point directly. Adaptive posting lists keep short term lists inline and split longer ones into fixed‑size blocks, using sparse offset arrays or dense bitmaps so that disk reads are minimized. A WAND‑like bounded traversal algorithm computes cheap upper bounds first, reading exact term frequencies only when a candidate can beat the current threshold, which cuts unnecessary work. Record offsets are known upfront, allowing batched asynchronous reads via io_uring that overlap disk waits and eliminate sequential page faults. Indexing and serving are separated: versioned shards are built on dedicated nodes and rotated in by a controller, letting a new index be deployed in under an hour without downtime.
For users, the practical takeaway is simple. If you need low‑latency search without managing infrastructure, call the Perplexity Search API with "search_type": "fast" and pay $1 per 1,000 requests. This gives you a p50 latency of 160 ms and a p95 of 230 ms, a dramatic improvement over the old engine’s 800 ms p99. You avoid the operational burden of self‑hosting a complex retrieval system while gaining predictable performance and cost savings.