--- title: "Local inference — AI hype score and trend · Above the Fog" canonical_url: https://abovethefog.app/topics/local_inference as_of: 2026-09-27 updated: 2026-09-28T22:02:40Z --- # Local inference (🧩 Products) [Home](https://abovethefog.app/index.md) › [Topics](https://abovethefog.app/topics.md) › Local inference Local inference is 💤 Quiet on Above the Fog’s AI hype radar: a hype score of 10 out of 100 on Sep 27, 2026, 8 points higher than 7 days before. Its attention is within the normal range across the monitored sources. Its last hype episode in the 30-day window ran from Sep 11 to Sep 11 (peak 29 on Sep 11). Running models on your own machine or server: Ollama, llama.cpp, vLLM, MLX, LM Studio, and the GGUF format. First seen on HN: front-page time (ClickHouse) · Sep 14 — the first monitored source to go above normal for it in the last 14 days. Why this stage: within the normal range across monitored sources. - **Stage:** 💤 Quiet — within the normal range across the monitored sources - **Hype score:** 10 / 100 · −3.7 in 1 day · +8.2 in 7 days - **In hype since:** not in hype now - **Peak, 30 days:** 36.9 on Sep 4, 2026 - **Category:** 🧩 Products - **Part of:** [Open-weight models](https://abovethefog.app/topics/open_weights.md) - **Sources above normal:** 1 of 27 scoring it (1 independent family) - **Stories in 7 days:** 646 - **Share of voice:** 1.1% of the radar’s attention in 7 days ## Where the attention comes from “Normal” is the median of the source’s previous 14 normal days for this topic. A source is above normal when its value is at least 30% over that level and stands out from its usual day-to-day variation (z ≥ 2); the share is the part of the hype score that source explains. “× normal” is left blank when the normal level is 0. | Source | Metric | Latest | Normal | × normal | Above normal | Share of score | |---|---|---:|---:|---:|---|---:| | HN: front-page time (ClickHouse) | frontpage hours | 6 | 0 | — | yes | 95% | | DEV Community (dev.to) | items | 13 | 10.5 | 1.2× | no | 5.1% | | Developer forums (Cursor, OpenAI, Hugging Face) | items | 1 | 0 | — | no | 0% | | Luma (SF AI events) | events | 1 (Sep 26) | 0 | — | no | 0% | | Bluesky | trending | 0 (Sep 26) | 0 | — | no | 0% | | Cerebral Valley (AI events) | events | 0 (Sep 26) | 0 | — | no | 0% | | GitHub Trending | trending | 0 (Sep 26) | 0 | — | no | 0% | | Google Trends (Trending now) | trending | 0 (Sep 26) | 0 | — | no | 0% | ## Stories behind it 1. [NVIDIA/Model-Optimizer: A unified library of SOTA model optimization techniques like quantization, distillation, pruning, neural architecture search, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize](https://github.com/NVIDIA/Model-Optimizer) — GitHub Trending (NVIDIA) · score 4,697 2. [42x Faster Prompt Lookup Drafting in llama.cpp](https://www.reddit.com/r/LocalLLaMA/comments/1wr5ylm/) — Reddit (r/LocalLLaMA) · Sep 27 · score 607 · 172 comments 3. [Ollaya – Ollama for open-source, Jev-style decision models](https://news.ycombinator.com/item?id=49848269) — Hacker News (ollaya.dev) · Sep 25 · score 609 · 145 comments 4. [ollaya-dev/ollaya: Run open decision models locally: pull and serve Laya, decider, NLI and GLiClass behind a TypeSafe-compatible API. Ollama for decision models.](https://github.com/ollaya-dev/ollaya) — GitHub (ollaya-dev) · Sep 23 · score 862 5. [Ollaya – Ollama for open-source, Jev-style decision models](https://news.ycombinator.com/item?id=49848269) — HN: front-page time (ClickHouse) (ollaya.dev) · Sep 25 · score 568 · 137 comments 6. [OrcaSAQ-2-27B](https://huggingface.co/orcarouter/OrcaSAQ-2-27B) — Hugging Face Hub (trending + orgs) (orcarouter) · Sep 24 · score 187 7. [I am currently having access to an Apple Mac M4 Max with 36GB unified RAM and ohhhh gurl does it do local LLMs! :blobcatrainbow: It does Qwen3.8 27B Q4 at 18 tokens/s and only uses 40W... completely…](https://lgbtqia.space/@m/117320431789857076) — Mastodon (fediverse) (lgbtqia.space) · Sep 23 · score 3 · 2 comments 8. [Llama-modes: load one GGUF once, then Chat, Boolean, Choice and Scale](https://discuss.huggingface.co/t/llama-modes-load-one-gguf-once-then-chat-boolean-choice-and-scale/180762) — Developer forums (Cursor, OpenAI, Hugging Face) (Hugging Face Forums) · Sep 27 · score 1 · 1 comment 9. [Depth-adaptive Inference of Looped Language Models via Continuous Depth Batching](https://huggingface.co/papers/2608.09444) — Hugging Face Daily Papers (hf-papers) · Sep 28 · score 1 · 1 comment 10. [Watermarking in vLLM](https://lobste.rs/s/bjy3mv/watermarking_vllm) — Lobste.rs (#vibecoding) · Sep 24 · score 2 · 0 comments 11. [GPT-6 ⚡, Opus 5.5 🧠, AI leaders at UN 🌐](https://tldr.tech/ai/2026-09-23) — AI newsletters (RSS) (TLDR AI) · Sep 23 12. [The Model Decision: Optimizing Cost, Control, & Speed](https://luma.com/lnnlfa87) — Luma (SF AI events) (Judy Wu) · score 0 ## Themes What its stories of the last 7 days are about: the 553 of them the Jev model kept as about the scene, by theme (a story can carry several themes): - [Models](https://abovethefog.app/themes/models.md) — 84% of its stories - [Agents](https://abovethefog.app/themes/agents.md) — 17% of its stories - [Protocols & infra](https://abovethefog.app/themes/protocols_infra.md) — 17% of its stories - [Usage & adoption](https://abovethefog.app/themes/usage_adoption.md) — 15% of its stories - [Security & safety](https://abovethefog.app/themes/security_safety.md) — 2.9% of its stories - Other tech — 0.9% of its stories ## Related topics - **Related to:** [Hermes Agent](https://abovethefog.app/topics/hermes_agent.md), [Llama](https://abovethefog.app/topics/llama.md), [Hugging Face](https://abovethefog.app/topics/hugging_face.md), [OpenCode](https://abovethefog.app/topics/opencode.md) - **Part of:** [Open-weight models](https://abovethefog.app/topics/open_weights.md) - **Mentioned with:** [Qwen](https://abovethefog.app/topics/qwen.md) (514 stories), [Gemma](https://abovethefog.app/topics/gemma.md) (94 stories), [Decision models](https://abovethefog.app/topics/decision_models.md) (32 stories), [Mistral AI](https://abovethefog.app/topics/mistral.md) (20 stories), [DeepSeek Harness](https://abovethefog.app/topics/deepseek_harness.md) (15 stories), [Muse Spark](https://abovethefog.app/topics/muse_spark.md) (9 stories) ## Hype score, last 30 days | Day | Hype score | Stage | |---|---:|---| | Sep 27, 2026 | 10 | 💤 Quiet | | Sep 26, 2026 | 13.7 | 💤 Quiet | | Sep 25, 2026 | 13 | 💤 Quiet | | Sep 24, 2026 | 5.2 | 💤 Quiet | | Sep 23, 2026 | 5.8 | 💤 Quiet | | Sep 22, 2026 | 13.6 | 💤 Quiet | | Sep 21, 2026 | 3.2 | 💤 Quiet | | Sep 20, 2026 | 1.8 | 💤 Quiet | | Sep 19, 2026 | 4.4 | 💤 Quiet | | Sep 18, 2026 | 4.7 | 💤 Quiet | | Sep 17, 2026 | 17.8 | 💤 Quiet | | Sep 16, 2026 | 5.8 | 💤 Quiet | | Sep 15, 2026 | 14.2 | 🧊 Cooling | | Sep 14, 2026 | 17.6 | 🧊 Cooling | | Sep 13, 2026 | 0 | 💤 Quiet | | Sep 12, 2026 | 16.3 | 🧊 Cooling | | Sep 11, 2026 | 28.7 | 🚀 Rising | | Sep 10, 2026 | 22 | 🧊 Cooling | | Sep 9, 2026 | 17.9 | 🧊 Cooling | | Sep 8, 2026 | 16.5 | 🧊 Cooling | | Sep 7, 2026 | 32.3 | 🔥 Hot | | Sep 6, 2026 | 11.5 | 🧊 Cooling | | Sep 5, 2026 | 21.6 | 🧊 Cooling | | Sep 4, 2026 | 36.9 | 🚀 Rising | | Sep 3, 2026 | 15.6 | 💤 Quiet | | Sep 2, 2026 | 16.4 | 🧊 Cooling | | Sep 1, 2026 | 16.8 | 🧊 Cooling | | Aug 31, 2026 | 15.6 | 🧊 Cooling | | Aug 30, 2026 | 11.8 | 🧊 Cooling | | Aug 29, 2026 | 27.5 | 🔥 Hot | [Open Local inference in the interactive map →](https://abovethefog.app/#topic=local_inference) --- Updated Sep 28, 2026, 22:02 UTC · Data: Above the Fog hype radar [HTML page](https://abovethefog.app/topics/local_inference) · [llms.txt](https://abovethefog.app/llms.txt) · [About & methodology](https://abovethefog.app/about.md) · [Privacy](https://abovethefog.app/about.md#privacy)