Perplexity has open sourced Lily, a Rust and Metal-based local inference engine for the Qwen3.6-35B-A3B model on Apple silicon, exposing an OpenAI-compatible chat-completions API for on-device text generation. Unlike general-purpose stacks such as MLX and MLX-LM, Lily targets a single model and hardware family, tightly integrating model structure, execution plans, and custom Metal kernels to extract performance from Apple’s unified memory architecture. Benchmarks on an M5 Max show Lily delivering about 1.23x faster prefill and 1.35x faster decode than MLX-LM, supporting deployable local inference via a lightweight HTTP server and potentially guiding future specialized runtimes.
This update represents a notable development in the Ai sector. Organizations and founders tracking this space should evaluate potential strategic and technical implications on their operations.