aiqbench takeAI Products

Perplexity's Photon demonstrates how optimizing retrieval infrastructure with Rust and custom data structures can drastically reduce AI search latency from 800 ms to 65 ms at p99. This shift from a maintained fork to an in-house engine highlights the critical role of low-level system engineering in scaling LLM applications, particularly when dealing with datasets that exceed available RAM. The release underscores that significant performance gains often come from architectural choices like adaptive posting lists and budgeted traversal rather than just model updates.

Source coveringMarkTechPost

What this isA model's reading of the article and related coverage, written for aiqbench. Not the publisher's words.

Perplexity Introduces Photon: A Rust-Based Retrieval Engine That Cuts p99 Latency From 800 ms to 65 ms

MarkTechPostSeptember 30, 20264 min read

Perplexity has released Photon, an in-house retrieval and ranking engine written in Rust. It replaces an open-source engine Perplexity had forked for its AI-native search stack. Photon now handles retrieval and ranking for all production traffic. It also powers a new Fast Search mode in the Perplexity Search API. Perplexity reports single-call latency of 160 […] The post Perplexity Introduces Photon: A Rust-Based Retrieval Engine That Cuts p99 Latency From 800 ms to 65 ms appeared first on MarkTechPost.

Perplexity has released Photon, an in-house retrieval and ranking engine written in Rust. It replaces an open-source engine Perplexity had forked for its AI-native search stack. Photon now handles retrieval and ranking for all production traffic. It also powers a new Fast Search mode in the Perplexity Search API. Perplexity reports single-call latency of 160 ms at p50 and 230 ms at p95. Is it deployable? Yes, as a hosted API. Set search_type: "fast" on POST /search and…

Excerpt shown under the publisher's RSS terms · 78 words · full text and figures are on the publisher's site

Read at MarkTechPost ↗Ask about this article