Sari la conținut

Română · beta. Traducerea a fost făcută automat și este încă în curs de revizuire. Citește pagina în engleză

perspectiva aiqbenchCipuri & infrastructură

analiza aiqbench

Perplexity's open-sourced Lily engine demonstrates that specialized Rust and Metal kernels can significantly outperform general-purpose MLX stacks on Apple Silicon for specific MoE models. By optimizing exclusively for Qwen3.6-35B-A3B, this runtime achieves higher throughput by bypassing generic frameworks like PyTorch or standard MLX in favor of hand-written operations tailored to the model's unique activation patterns. This approach highlights a strategic trade-off where sacrificing cross-model versatility yields measurable performance gains for local inference on high-end Mac hardware.
Sursă care acoperăMarkTechPost
Ce este aceastaLectura unui model asupra articolului și a acoperirii conexe, scrisă pentru aiqbench. Nu sunt cuvintele publicației.

Perplexity Open Sources Lily: A Rust + Metal Inference Engine for Qwen3.6-35B-A3B on Apple Silicon

MarkTechPost4 min de cititArticolul și perspectiva sunt în engleză.

Perplexity has open sourced Lily, the local inference engine behind Hybrid Compute in Perplexity Computer. Built in Rust with custom Metal kernels for one model on one chip family, it averages 1.23x MLX-LM's prefill throughput and 1.35x its decode throughput on a 40-core, 128 GB M5 Max. The post Perplexity Open Sources Lily: A Rust + Metal Inference Engine for Qwen3.6-35B-A3B on Apple Silicon appeared first on MarkTechPost.

Perplexity has open sourced Lily, the local inference engine behind Hybrid Compute in Perplexity Computer. It is a single-process runtime: a Rust layer loads the checkpoint and drives the generation loop, an OpenAI-compatible chat-completions API streams tokens, and hand-written Metal kernels execute the model. Neither PyTorch nor MLX sits in the execution path. Lily is deliberately narrow with one model, Qwen3.6-35B-A3B, on one hardware family and that narrowness is the performance argument. Is it deployable? Yes. A standalone demo…

Extras afișat conform termenilor RSS ai publicației · 79 cuvinte · textul integral și figurile sunt pe site-ul publicației

Subiecte conexe