• 40 Posts
  • 100 Comments
Joined 4 months ago
cake
Cake day: July 29th, 2026

help-circle


  • That blog post doesn’t seem to talk about what changed in this release since v0.9.0 https://github.com/ggml-org/llama.cpp/releases/tag/v1.0.0

    llama.cpp v1.0.0 removes the deprecated public API and promotes the NextN/speculative API to llama.h, adds OCP FP8 KV caches, a host-memory MoE expert cache with a configurable CPU share, and loading of external PLE shards. It adds three text architectures (Kimi-K3.1-Pro, Ripple and xing4_0), drama-3 TTS, and MageVL-Cowork and Kimi-K3.1 vision, hardens llama-server (SSRF, CORS, DNS rebinding, MCP tool cancellation) and adds a /v1/audio/speech endpoint, ships a redesigned web UI with chat groups and a markdown editor, and updates ggml to v1.0.0.

    Some highlights:

    • llama.h: add moe_cache_size and moe_cache_cpu_share to llama_context_params (#30966)
    • FP8 KV caches (-ctk/-ctv f8_e4m3) with optional static per-tensor scales (#28392)
    • MoE cache: CPU share for misses, LRFU/ClassLRU replacement, no CPU wait on cached experts (#30966, #31233, #31888)
    • external PLE/engram loading from separate GGUF shards via --shard/–ple (#31185)
    • chat : refactor gemma4 to build loose schema rules and clean up (#31915) This one includes a compatibility break: I also took this as an opportunity to clean up the old workaround and template. The official Gemma 4 templates have addressed the issues those solved and have been out for quite a while. If anyone hasn’t updated yet, it’s on them! You can download the newer template from here https://huggingface.co/google/gemma-4-31B-it/blob/main/chat_template.jinja

    I hope they start doing a separate bleeding edge branch and stable branch. Like how Linux has rolling distros and LTS distros.








  • Sure if compaction was free then I’d do it all the time. One advancement I’d like to see is MoE models that use fewer active parameters/experts when compacting. We already have DeepSeek 4.1 that uses a different number of active parameters depending on what its doing (8b for prefill and 16b for decode) so it doesn’t seem unreasonable to use fewer for compaction. It’d be like using a REAP model, but without having to load the model separately. Or even just use the MTP/draft model for compaction.