analiza aiqbench
Perplexity's open-sourced Lily engine demonstrates that specialized Rust and Metal kernels can significantly outperform general-purpose MLX stacks on Apple Silicon for specific MoE models. By optimizing exclusively for Qwen3.6-35B-A3B, this runtime achieves higher throughput by bypassing generic frameworks like PyTorch or standard MLX in favor of hand-written operations tailored to the model's unique activation patterns. This approach highlights a strategic trade-off where sacrificing cross-model versatility yields measurable performance gains for local inference on high-end Mac hardware.
Perplexity Open Sources Lily: A Rust + Metal Inference Engine for Qwen3.6-35B-A3B on Apple Silicon

Perplexity has open sourced Lily, the local inference engine behind Hybrid Compute in Perplexity Computer. Built in Rust with custom Metal kernels for one model on one chip family, it averages 1.23x MLX-LM's prefill throughput and 1.35x its decode throughput on a 40-core, 128 GB M5 Max. The post Perplexity Open Sources Lily: A Rust + Metal Inference Engine for Qwen3.6-35B-A3B on Apple Silicon appeared first on MarkTechPost.
Perplexity has open sourced Lily, the local inference engine behind Hybrid Compute in Perplexity Computer. It is a single-process runtime: a Rust layer loads the checkpoint and drives the generation loop, an OpenAI-compatible chat-completions API streams tokens, and hand-written Metal kernels execute the model. Neither PyTorch nor MLX sits in the execution path. Lily is deliberately narrow with one model, Qwen3.6-35B-A3B, on one hardware family and that narrowness is the performance argument. Is it deployable? Yes. A standalone demo…
Extras afișat conform termenilor RSS ai publicației · 79 cuvinte · textul integral și figurile sunt pe site-ul publicației
Subiecte conexe

[AINews] The Future of Latent Space
Latent Space · 25 sept. 2026

[AINews] Opus 5.5 is good at explainer videos
Latent Space · 29 sept. 2026

[AINews] Gemini 4 Argon: GDM’s answer to Astra/Fable, with 1M output
Latent Space · 1 oct. 2026

[AINews] not much happened today
Latent Space · 3 oct. 2026