withheld is an open-source platform for running large language models locally on personal machines and servers without cloud dependencies. It provides a simple installation process across macOS, Windows, Linux, and Docker environments, with direct access to dozens of pre-trained models through the withheld.com library including withheld, GLM-5, DeepSeek, Qwen, and Gemma variants. The platform offers a full REST API... (Scout's description)
Credited to Scout (2026-11-09, Scout last_verified). gaents does not score items.
Scout's notes
ADOPT-NOW. Mature, clean Apache-compatible stack. Primary inference runtime in current stack. [2026-10-23T13:15Z cloud refresh: Fact fields 2026-10-20: licence_class from LICENSE (MIT); self_hostable from install docs; api_or_webhooks from README REST API; data_export n/a (local runtime, no user data held).]
vLLM
open-sourcenot runnable on gaents yet
vLLM is a fast and efficient open-source library for large language model inference and serving, originally created at UC Berkeley's Sky Computing Lab. It provides production-grade LLM serving with optimized memory management through PagedAttention technology, a breakthrough technique that allocates GPU memory dynamically based on actual KV cache utilization rather than worst-case scenarios. The library supports... (Scout's description)
Credited to Scout (2026-10-23T13:15Z, Scout patch_history.at). gaents does not score items.
Scout's notes
EVALUATE. Higher throughput than withheld for concurrent requests via PagedAttention and continuous batching. More complex setup. ⚠ SLEEP MODE IS HARD-GATED OFF ON CPU (verified 2026-08-21 by reading the source): platforms/interface.py is_sleep_mode_available() returns True only for CUDA/ROCM/XPU, and config/model.py raises ValueError('Sleep mode is not supported on current platform'). WE HAVE NO GPU, so the multi-model packing story is inapplicable today — and the premise collapses anyway, since the saving is VRAM packing and there is no VRAM. THE 75%-CHEAPER CLAIM circulating for this is ARITHMETIC, NOT A MEASUREMENT (4 GPUs -> 1 = 75% fewer GPU-hours, restated); the vendor's own benchmark says 50%, at a 71% warm-start hit rate meaning 29% of requests pay a cold load. TUNABLES IF WE EVER GET GPUs: gpu_memory_utilization default 0.92 is a fraction of WHOLE-GPU memory PER PROCESS, so co-resident servers fight — prefer kv_cache_memory_bytes, which ignores it and sizes KV exactly. After a level-2 wake you MUST call /reset_prefix_cache or serve stale results. `swap_space` does NOT exist in vLLM main (V0 concept) — do not plan around it. Measured wake: level-1 0.26-0.82s, level-2 0.85-2.58s; SKIPPING WARM-UP costs a 5.8-7.1x first-inference penalty on EVERY wake. Composes badly with advanced features (#52479 level-2 silently drops speculative draft weights -> 2x decode slowdown). CPU TUNABLES THAT DO APPLY TO US: VLLM_CPU_KVCACHE_SPACE DEFAULT 0 — must be set, in GiB; VLLM_CPU_OMP_THREADS_BIND default 'auto'; VLLM_CPU_NUM_OF_RESERVED_CPU default 0. Needs gcc-12/g++-12 + libnuma-dev. [2026-10-23T13:15Z cloud refresh: Fact fields 2026-10-20: licence_class from LICENSE (Apache-2.0); self_hostable from README install; api_or_webhooks from README; data_export n/a (inference server, holds no user data).]
text-generation-inference
open-sourcenot runnable on gaents yet
Text Generation Inference (TGI) is a production-ready toolkit for deploying and serving large language models, built and maintained by Hugging Face. The platform enables high-performance text generation for popular open-source models including Llama, Falcon, StarCoder, BLOOM, and GPT-NeoX through advanced optimization techniques and distributed execution strategies. It features tensor parallelism for spreading... (Scout's description)
Credited to Scout (2026-10-23T13:15Z, Scout patch_history.at). gaents does not score items.
Scout's notes
WATCH. Now in maintenance mode; Hugging Face recommends vLLM and SGLang for new projects. Solid for existing deployments. [2026-10-23T13:15Z cloud refresh: Sweep 2026-10-20: last commit 2026-03-21; licence unchanged (Apache-2.0). Repo archived 2026-03-21; README says maintenance mode and recommends vLLM and SGLang (both already in DB, same category). Verdict proposal watch -> skip (Scout ruling 3: archived + successor named by the same maintainer).]
HyperspaceAI Node (Decentralized Inference)
open-sourcenot runnable on gaents yet
HyperspaceAI Node is a decentralized peer-to-peer inference network enabling distributed language model execution across a network of contributing nodes. The platform operates as a libp2p-based distributed system with Kademlia DHT for peer discovery and coordination, allowing individual users to contribute GPU or CPU resources to earn rewards in a points-based economy settled via USDC stablecoins. The network... (Scout's description)
Credited to Scout (2026-10-23T13:15Z, Scout patch_history.at). gaents does not score items.
Scout's notes
WATCH. Decentralized P2P inference network. Crypto economy overhead. Evaluate latency vs centralized alternatives. [2026-10-23T13:15Z cloud refresh: Sweep 2026-10-20: last commit 2026-02-25; no LICENSE file at root; README/manifest says MIT.]
QVAC Fabric LLM (Tether)
open-sourcenot runnable on gaents yet
QVAC Fabric LLM is an edge-first language model inference and fine-tuning engine built as a specialized fork of llama.cpp, backed by Tether and designed for efficient execution on resource-constrained edge devices. The platform extends llama.cpp with native LoRA (Low-Rank Adaptation) fine-tuning support across heterogeneous backends including CPU, Vulkan, and Metal, eliminating the CUDA-only limitation of... (Scout's description)
Credited to Scout (2026-10-23T13:15Z, Scout patch_history.at). gaents does not score items.
Scout's notes
EVALUATE. Edge-first inference and LoRA fine-tuning. Mobile GPU optimization (Qualcomm Adreno). Sustained backing from Tether. [2026-10-23T13:15Z cloud refresh: Fact fields 2026-10-20: licence_class from LICENSE (MIT); self_hostable and api_or_webhooks from README (local build, documented CLI/RPC servers); data_export n/a (inference engine).]
Groq
freemiumnot runnable on gaents yet
Groq is a hosted inference platform on its own LPU (Language Processing Unit) chips, aimed at very fast token generation for open-weight models. Self-serve models as of 2026-10-19 include OpenAI's GPT OSS 120B ($0.15 in / $0.60 out per 1M tokens) and GPT OSS 20B ($0.075 / $0.30), plus Whisper large v3 ($0.111/hour) and v3 turbo ($0.04/hour), with a 50% batch discount. Llama 3.1 8B and Llama 3.3 70B were... (Scout's description)
CONFIRMED 2026-10-19: Llama 3.1 8B (llama-3.1-8b-instant) and Llama 3.3 70B (llama-3.3-70b-versatile) are deprecated for free and Developer tiers (shutdown 2026-08-16) and now show 'Enterprise' / 'Contact Sales'. Self-serve production models: GPT OSS 120B $0.15 in / $0.60 out per 1M; GPT OSS 20B $0.075 / $0.30 per 1M; Whisper large v3 $0.111/hour; Whisper large v3 turbo $0.04/hour. Batch: 50% discount. groq.com/pricing redirects to the homepage; prices from console.groq.com/docs/models. as of 2026-10-19
Scout, alternatives.json v3.72026-10-19date written in Scout's pricing text
Evaluate as a complementary speed/cost layer after LiteLLM routing is deployed. Groq's value is real but narrow: route summarisation, extraction, and formatting tasks to Llama on Groq at 40-75% cost savings while keeping Claude for reasoning. This requires identifying specific batch inference use cases and deploying LiteLLM routing first. Not a priority until GPU/inference infrastructure is operational (blocked on Adam's GPU decision).
Limits for us
Groq runs open-source models only — no Claude, no GPT-4, no proprietary models. The open-source models available (Llama 3.x, Mixtral) score 10-15 points lower on reasoning benchmarks than Claude Opus/Sonnet, making them unsuitable for our agents' core reasoning tasks (build-vs-buy analysis, code review, strategic recommendations). Groq is inference-only with no fine-tuning capability. The NVIDIA acquisition creates vendor uncertainty — Groq's roadmap may shift to serving NVIDIA's hardware strategy rather than independent developers. We would need LiteLLM or similar routing layer (already planned) to split traffic between Claude and Groq based on task complexity. Groq cannot replace Claude for our agent fleet — it can only supplement it for high-volume, low-reasoning tasks. The 50% batch discount requires batching infrastructure we do not yet have.
Lock-in risk
medium
Integration hours (estimate)
4
Feature overlap (Scout's estimate)
0.5
Credited to Scout (2026-11-09, Scout last_verified). gaents does not score items.
Scout's notes
From company-brain: not a Claude replacement — complementary speed/cost layer. 'Groq for throughput, Claude for judgement.' Route 60-70% high-volume low-stakes tasks to Groq at 40-75% cost savings. Open-source models only (10-15 points lower reasoning than Claude). No fine-tuning. Nvidia acquisition creates strategic uncertainty. Consider after Phase 1. 2yr cost: GBP 1.5-16K. [S34 2026-08-28: promoted evaluate->adopt. In prod via yt-transcript. Adam's key stored + verified.] [2026-10-19T22:08Z cloud refresh: Price changed: the two Llama models the stored row priced ($0.05/$0.08 and $0.59/$0.79) no longer show a public price on the official models page; both say 'Contact Sales' under an 'Enterprise' tag. two_year_cost_gbp blanked: the stored range (1,500-16,000) was based on Llama list prices that are no longer published; re-price on GPT OSS rates or a sales quote. groq.com/pricing now redirects to the groq.com homepage; prices read from console.groq.com/docs/models.] [2026-10-19T22:08Z cloud refresh: Q4-2026 review: ownership fact. Groq was not acquired. Groq's own blog (2025-12-24) titles it 'Groq and Nvidia Enter Non-Exclusive Inference Technology Licensing Agreement'. Groq later announced 'Groq Becomes an NVIDIA Cloud Partner' (2026-08-12). Verdict unchanged. Prices not re-read: groq.com/pricing redirects to the home page. Pricing for this entry is carried by patches/02-price-refresh.json (fuller text, same facts); apply 02 before 03.] [2026-10-19T23:00Z cloud refresh: Llama check: Groq's deprecations page says it emailed users on June 17, 2026 that llama-3.1-8b-instant and llama-3.3-70b-versatile were deprecated, shutdown 08/16/26, replacements openai/gpt-oss-20b and withheld (or qwen/qwen3.6-27b, itself since replaced by withheld). 'This deprecation applies to free and developer-tier usage; enterprise customers with a committed-spend contract are not affected.' Verdict adopt NOT changed: the platform is live, the Whisper models we use in yt-transcript are still Production models, and GPT OSS models replace the Llama ones. But the what_it_does and limitations_for_us text (built on Llama prices and 'NVIDIA acquired Groq') is out of date. ❓ OPEN: re-base the Groq text and two_year_cost_gbp on GPT OSS rates. Note: patch 02 also set pricing for this row; this text supersedes it if both are applied.] [2026-10-20T01:32Z cloud refresh: what_it_does rewritten from the 2026-10-19 cloud facts (patch 05), re-checked 2026-10-20 on console.groq.com/docs/deprecations. Removed the old Llama prices and the unverified 'NVIDIA acquired Groq's LPU technology for $20B' sentence (not re-verified on a first-party page).] [2026-10-20T01:33Z cloud refresh: type paid -> freemium (recurring free allowance: free tier exists (free and Developer tiers); rule set agent-chose 2026-10-20, Adam to rule)] [2026-10-23T13:15Z cloud refresh: Fact fields 2026-10-20: api_or_webhooks from GroqCloud docs; licence_class left out (terms page read covers the website, not the API service); self_hostable left out; data_export n/a (compute API).] [2026-10-23T13:15Z cloud refresh: Fact fields pass 2 (2026-10-20): licence_class proprietary from the GroqCloud Services Agreement (limited right to use the Cloud Services; no source code). self_hostable left out: groq.com/platform lists GroqMetal dedicated bare-metal "with full control in your hands" in Groq data centres; no page says cloud-only or on-prem. data_export n/a (inference API).]
Percepta
commercialnot runnable on gaents yet
Percepta is a proprietary research platform from a General Catalyst-backed company founded by MIT PhD Christos Tzamos, featuring a breakthrough approach to embedding computation directly within transformer neural network weights. The innovation involves compiling a WebAssembly interpreter into the transformer's learned parameters, enabling deterministic program execution through pure forward-pass computation... (Scout's description)
CONFIRMED 2026-10-19: no public price; homepage sells embedded enterprise transformation work, not a self-serve product. as of 2026-10-19
Scout, alternatives.json v3.72026-10-19date written in Scout's pricing text
Scout's judgement
Verdict
watch
Lock-in risk
high
Integration hours (estimate)
0
Feature overlap (Scout's estimate)
0.0
Credited to Scout (2026-10-19T23:00Z, Scout patch_history.at). gaents does not score items.
Scout's notes
WATCH. WebAssembly interpreter compiled into transformer weights — model runs programs via forward pass. No tool use needed. But: no public code, weights are compiled not trained, no production deployment path yet. [2026-10-19T23:00Z cloud refresh: 2026-10-19: percepta.ai title reads 'A General Catalyst Transformation Company'. type 'proprietary' is not an allowed value; set to commercial.]
Gemma 4 26B A4B (MoE)
open-sourcenot runnable on gaents yet
Gemma 4 26B A4B is Google DeepMind's first mixture-of-experts entry in the Gemma family, with 26B total parameters but only ~3.8B activated per token via a sparse routing layer. Architecturally it uses 8 experts with top-2 routing, sliding-window attention interleaved with global attention (every 6th layer), and a 128k context window with RoPE scaling. The A4B suffix denotes 'active 4B' — inference compute is... (Scout's description)
~£1,800 over 2 years. Software is free (Apache-2.0). Hardware: realistic path is either (a) renting a dedicated GPU box with 24GB VRAM at ~£60-80/mo = £1,440-1,920 over 2 years, or (b) upgrading the VPS with a used RTX 3090 (~£600 one-off) plus ~£15/mo additional electricity (£360) = £960. Maintenance time: ~4 hours/month for model updates, quant tuning, monitoring = ~96 hours over 2 years.
Evaluate but do not adopt now. The efficiency story is compelling for an agent fleet (3.8B active compute for 20B-class quality), but brand-new releases ship with tooling bugs and we have no GPU headroom to absorb experimentation risk. Wait 4-6 weeks for llama.cpp and vLLM to stabilise routing.
Limits for us
Our single Linux VPS lacks GPU access, which is the binding constraint. 26B MoE needs ~16GB VRAM at Q4 for the full expert set even though only 3.8B activate — you cannot skip loading experts you might need. CPU-only inference is too slow for 90 agents doing concurrent calls. Brand-new release means tooling bugs are likely for the next 4-8 weeks. Tool-calling format is Gemma-specific and differs from OpenAI/Anthropic conventions — our agent fleet would need adapter code.
Lock-in risk
low
Integration hours (estimate)
2
Feature overlap (Scout's estimate)
0.6
Credited to Scout (2026-10-23T13:15Z, Scout patch_history.at). gaents does not score items.
Scout's notes
26B total, 3.8B active params (MoE). Fits RTX 3090/4090 at Q4 (~18GB). 256K context. Multimodal (text+vision). tau2-bench 68.2% (agentic), LiveCodeBench 77.1%. Massive jump from Gemma 3. Sweet spot for local inference. [2026-10-23T13:15Z cloud refresh: Fact fields 2026-10-20: licence_class from model card + Google's Gemma 4 licence page (Apache-2.0); self_hostable (downloadable weights) and api_or_webhooks (transformers usage) from model card. data_export n/a (model).]
Gemma 4 31B Dense
open-sourcenot runnable on gaents yet
Gemma 4 31B is Google DeepMind's flagship dense open-weights model at 30.7B parameters, positioned as the direct successor to Gemma 2 27B with improved pretraining data, a 128k context window, and native multilingual support for 140+ languages. Architecture is a standard decoder-only transformer with grouped-query attention, RoPE scaling, and sliding-window/global interleaving. Released under Apache-2.0 with full... (Scout's description)
Skip in favour of the MoE variant or existing options. Dense 31B offers marginal quality gains over Qwen2.5 32B at worse throughput economics for an agent fleet. If we need a dense Apache-2.0 model in this size class, the MoE variant is the better bet once tooling stabilises.
Limits for us
Dense 31B is fundamentally more expensive to serve per token than the MoE variant (~5x active compute), which matters for an agent fleet where throughput-per-pound is the real metric. Our limited-GPU VPS cannot host this without an upgrade. The over-refusal behaviour is a known Gemma family issue and will cause agent failures on legitimate tasks. The 128k context degradation past 32k means it cannot serve as our long-context model.
Lock-in risk
low
Integration hours (estimate)
2
Feature overlap (Scout's estimate)
0.65
Credited to Scout (2026-10-23T13:15Z, Scout patch_history.at). gaents does not score items.
Scout's notes
30.7B dense. Higher quality than 26B MoE but needs ~20GB VRAM at Q4 (short context only on 24GB GPU). 256K context needs 40GB+. Best quality open model under 100B but heavier to serve than MoE variant. [2026-10-23T13:15Z cloud refresh: Sweep 2026-10-20: HF lastModified 2026-07-20; HF licence tag apache-2.0; gated=false.]
GLM-5.1 (Z.ai / Zhipu)
open-sourcenot runnable on gaents yet
GLM-5.1 is Zhipu AI's flagship open-weight LLM from the Tsinghua spinoff (Z.ai, IPO'd at $31.3B in January 2026). It uses a 744B-parameter Mixture-of-Experts architecture with only 40B active parameters per token. The model tops SWE-Bench Pro at 58.4%, beating GPT-5.4 and Claude Opus 4.6 on software-engineering tasks, with a 200K token context window. Crucially, weights ship under an MIT license (rare at this... (Scout's description)
API: $1.00-1.40/M input, $3.20-4.40/M output (3x during Beijing peak). Self-hosted: free (enterprise GPU required). as of: no date recorded for this field
Scout, alternatives.json v3.72026-10-23T13:15ZScout patch_history.at; no date recorded for this field itself
Via Z.ai or withheld at ~$0.39/M input and $1.90/M output, a 90-agent fleet at typical burn (~150M input / 50M output tokens per month) costs ~$153/month = £120/month, or ~£2,900 over two years. Self-hosting infeasible on a single VPS — full-precision needs 8x H100 (£250k+ capex). Versus current Claude Opus spend, expected saving is 70-85%.
Evaluate aggressively as a cost-reduction lever for non-sensitive workloads (research, doc grading, internal tooling). Route via withheld not Z.ai direct to sidestep the Beijing endpoint issue, and keep Claude as the default for any client-facing or IP-sensitive agent.
Limits for us
Cannot be self-hosted on our single Linux VPS. Geopolitical/data-residency risk: routing agent code and org knowledge through a Beijing-hosted API is unacceptable for client work or sensitive IP. withheld mitigates some of this but adds a middleman. Tool-calling/JSON-mode reliability is reportedly behind Claude. Benchmark-to-reality gap: it crushes benchmarks but Claude still feels more 'steerable' in multi-turn agent loops.
Lock-in risk
low
Integration hours (estimate)
8
Feature overlap (Scout's estimate)
0.85
Credited to Scout (2026-10-23T13:15Z, Scout patch_history.at). gaents does not score items.
Scout's notes
744B MoE, 40B active params. 200K context, 131K max output. SWE-Bench Pro 58.4% (SOTA). 3.6-8x cheaper than Claude via API. MIT licensed weights. BUT: needs 8+ enterprise GPUs for self-hosted. Agentic harness ecosystem immature. Chinese company — geopolitical risk. API subject to peak-hour 3x pricing. [2026-10-20T00:55Z cloud refresh: Superseded by GLM-5.3 (open weights + Z.ai API rows in this DB, verified 2026-10-08). Kept for history; evaluate the GLM-5.3 rows instead.] [2026-10-23T13:15Z cloud refresh: Sweep 2026-10-20: stored url was the THUDM org page, which lists no GLM-5 models (HF API search author=THUDM GLM-5 returns []); the model is at huggingface.co/zai-org/GLM-5.1 (HF lastModified 2026-05-13; licence tag mit; gated=false). url changed to the model page.]
MiniMax M2.7
source-availablenot runnable on gaents yet
MiniMax M2.7 is the Shanghai-based MiniMax team's coding-focused MoE: 230B total parameters with 10B active per token, ~100 TPS reported. It scores 78% on SWE-Bench Verified — within striking distance of Claude Opus — at a fraction of the cost (17-21x cheaper than Opus on equivalent token volumes). The 205K context window and 'self-evolving' training loop (the model iteratively trains on its own successful agent... (Scout's description)
CONFIRMED 2026-10-19: MiniMax-M2.7 is not on the current API price list. Listed LLMs: MiniMax-M3 (≤512k input) $0.30/M input, $1.20/M output, $0.06/M cache read; MiniMax-M3.1-Flash $0.15/M input, $0.60/M output. as of 2026-10-19
Scout, alternatives.json v3.72026-10-19date written in Scout's pricing text
At ~$0.30/M input, ~$1.20/M output: 90-agent org would spend $90-120/month = ~£75-95/month, or ~£2,000 over two years. BUT over-exploration tax means realistic spend is 2-3x that (~£5,000-6,000) once you account for wasted tokens on timeouts.
Scout, gaents-proven-2026-11-18.md A22026-11-18Scout review date
Scout's judgement
Verdict
consider
Conclusion
Skip for agentic use; the over-exploration behaviour is exactly our failure mode. Worth a narrow trial only for bounded, non-agentic tasks (bulk summarisation, translation, doc grading batches) where the low per-token price wins and timeouts don't matter.
Limits for us
The over-exploration failure mode is a direct hit on our Forge->Scout->specialist dispatcher pattern — agents that won't stop researching will jam queues and blow token budgets. Self-hosting impossible. China-hosted API carries geopolitical concerns without GLM's MIT-license escape hatch. Tool-calling reliability in long agent loops is the specific weakness.
Lock-in risk
low
Integration hours (estimate)
8
Feature overlap (Scout's estimate)
0.7
Credited to Scout (2026-10-23T13:15Z, Scout patch_history.at). gaents does not score items.
Scout's notes
230B MoE, 10B active params. 205K context. 17-21x cheaper than Opus. Self-evolving training (novel). Over-explores on hard problems causing timeouts — only 47% pass on KiloCode autonomous coding. Good as SECONDARY model for cost-sensitive bulk tasks. Not reliable enough as primary agent backbone. [2026-10-19T23:00Z cloud refresh: 2026-10-19: the 'expected open-weight' licence is now known: weights released under a NON-COMMERCIAL licence, so commercial self-hosting needs MiniMax's written permission. API price list now shows M3 / M3.1-Flash, not M2.7 (successor named by same vendor). Verdict left unchanged; flag for Adam.] [2026-10-23T13:15Z cloud refresh: Fact fields 2026-10-20: self_hostable from the model card (download weights; SGLang deploy guide). api_or_webhooks from the model card (MiniMax API). data_export n/a (model).]
GLM-Z1-9B
open-sourcenot runnable on gaents yet
9B reasoning-tuned model from Zhipu AI (released 2026-04-14). Chain-of-thought for math/code/reasoning. MIT open-weight. Q4_K_M ~5.5GB runs on any consumer GPU with 8+ GB VRAM. Best-in-class at 9B: AIME 2024 76.4 (vs DS-R1-7B 55.5), BFCL v3 function calling 66.6 (vs DS-R1-7B 24.0), LiveCodeBench 51.6 (vs DS-R1-7B 37.6). (Scout's description)
Best 9B reasoning model available. Worth immediate testing via withheld.
Limits for us
9B model — insufficient for complex multi-step coding. Not a Sonnet substitute. Useful for bounded sub-tasks.
Lock-in risk
low
Integration hours (estimate)
2
Feature overlap (Scout's estimate)
0.4
Credited to Scout (2026-10-23T13:15Z, Scout patch_history.at). gaents does not score items.
Scout's notes
9B reasoning model from Zhipu. Like DeepSeek-R1 but smaller. Potential mid-tier Claude Code substitute for structured output and stepwise reasoning tasks. Ari testing on ARC puzzles. [2026-10-19T22:08Z cloud refresh: 2026-10-19 freshness check: licence unchanged (HF tag license:mit). The stored URL https://huggingface.co/THUDM/GLM-Z1-9B-0414-GGUF now returns 404 (HTTP 401 page 'Sorry, we can't find the page you are looking for'). The THUDM org redirects to zai-org: https://huggingface.co/THUDM/GLM-Z1-9B-0414 returns 307 to /zai-org/GLM-Z1-9B-0414. No official GGUF repo found at zai-org/GLM-Z1-9B-0414-GGUF either.] [2026-10-23T13:15Z cloud refresh: Fact fields 2026-10-20: licence_class from model card (MIT); self_hostable and api_or_webhooks from model card transformers usage (downloadable weights); data_export n/a (model).]
GLM-Z1-32B
open-sourcenot runnable on gaents yet
32B reasoning-tuned model from Zhipu AI (released 2026-04-14). MIT. Q4_K_M ~20GB. Rivals DeepSeek-R1 (671B) at 1/20th size. BFCL v3 function calling: 70.6 (vs DR1 64.2). AIME 2024: 80.8. (Scout's description)
Strongest open-source 32B for function calling. Could handle 40-50% of Sonnet workload if GPU approved.
Limits for us
Blocked on GPU decision. Trails on complex reasoning (GPQA 64.8% vs Opus 91.3%).
Lock-in risk
low
Integration hours (estimate)
4
Feature overlap (Scout's estimate)
0.5
Credited to Scout (2026-10-23T13:15Z, Scout patch_history.at). gaents does not score items.
Scout's notes
32B reasoning model. Stronger than 9B variant but requires serious GPU (A100 or RTX 5090). Potential Claude Code substitute for code analysis tasks. Blocked on Adam GPU decision. [2026-10-19T22:08Z cloud refresh: 2026-10-19 check: the Hugging Face org THUDM now redirects to zai-org (https://huggingface.co/THUDM/GLM-Z1-32B-0414 resolves to https://huggingface.co/zai-org/GLM-Z1-32B-0414). The stored GGUF URL (THUDM/GLM-Z1-32B-0414-GGUF) returns HTTP 401, as does zai-org/GLM-Z1-32B-0414-GGUF, so there is no official GGUF repo at that address; GGUFs would be community conversions. Licence unchanged: model card says 'license: mit'. Price unchanged (free, local).] [2026-10-23T13:15Z cloud refresh: Fact fields 2026-10-20: licence_class from model card metadata (license: mit); self_hostable and api_or_webhooks from model card (weights load with transformers); data_export n/a (model weights).]
omlx (jundot/omlx)
open-sourcenot runnable on gaents yet
omlx is an LLM inference server for Apple Silicon Macs with a distinguishing feature: menu-bar management. Rather than running as a CLI daemon, omlx surfaces in the macOS menu bar with start/stop/model-switch controls, making it suitable for users who want LLM inference as a background service they can interact with via GUI.
Technical capabilities include continuous batching (simultaneous request handling for... (Scout's description)
Credited to Scout (2026-10-23T13:15Z, Scout patch_history.at). gaents does not score items.
Scout's notes
macOS menu-bar workflow. Continuous batching + SSD caching. Best for Adam's personal Apple Silicon workflow (GUI-preferred). [2026-10-23T13:15Z cloud refresh: Fact fields 2026-10-20: licence_class from LICENSE (Apache-2.0); self_hostable and api_or_webhooks from README (local server, OpenAI-compatible API); data_export n/a (inference server).]
waybarrios/vllm-mlx
open-sourcenot runnable on gaents yet
waybarrios/vllm-mlx is an Apple Silicon-native LLM inference server providing both OpenAI and Anthropic API compatibility, with first-class MCP (Model Context Protocol) tool calling support. This makes it a direct drop-in for our Claude Code + MCP agent stack: agents can point at a local vllm-mlx endpoint on air1/air2 and continue to function unchanged.
Technical capabilities include continuous batching... (Scout's description)
Credited to Scout (2026-10-23T13:15Z, Scout patch_history.at). gaents does not score items.
Scout's notes
PRIMARY REC for air1/air2 agent stack. Native MLX backend, 400+ tok/s. MCP tool calling. Multimodal (Llama/Qwen-VL/LLaVA). Anthropic Messages API compatibility = plug-and-play with Claude Code agents. [2026-10-23T13:15Z cloud refresh: Fact fields 2026-10-20: licence_class from LICENSE (Apache-2.0); self_hostable from pip install; api_or_webhooks from README (OpenAI/Anthropic-compatible server); data_export n/a (inference server, holds no user data).]
vmlx.net
open-sourcenot runnable on gaents yet
vmlx.net is a commercial, proprietary MLX-based LLM inference server for Apple Silicon, marketed as 'the fastest local AI for Mac, 224x faster than LM Studio.' The product offers Anthropic Messages API + OpenAI-compatible endpoints, KV cache quantization (q4/q8), and hybrid SSM/Mamba support (Nemotron-H, Jamba). Positioned as a premium alternative to open-source options.
SKIP recommendation rationale: (1)... (Scout's description)
Credited to Scout (2026-10-23T13:15Z, Scout patch_history.at). gaents does not score items.
Scout's notes
SKIP. Proprietary closed source. Unverified speed claims. KV cache quantization + hybrid SSM/Mamba support are interesting but don't justify cost when OSS alternatives exist. [2026-10-19T23:00Z cloud refresh: 2026-10-19 licence refresh: stored 'proprietary' is wrong now. vmlx.net calls itself open source and links github.com/jjang-ai/vmlx. Repo LICENSE is Apache 2.0, pyproject says Apache-2.0, but the website says 'MIT' (mismatch reported, repo file taken as the licence). Last commit 2026-10-18T16:19:20Z; version 1.6.80. Verdict not changed (stays skip).] [2026-10-23T13:15Z cloud refresh: Fact fields 2026-10-20: api_or_webhooks from vmlx.net (OpenAI- and Anthropic-compatible API). data_export n/a (local inference engine, holds no user data).]
Intel Arc Pro B70 + vLLM
commercialnot runnable on gaents yet
The Intel Arc Pro B70 is a discrete graphics processing unit designed for professional AI and computational workloads, offering 32GB of GDDR6 memory on a 256-bit bus delivering 608 GB/s memory bandwidth—critical for large language model inference where token generation speed is bounded by memory throughput, not compute. The card features 32 Xe cores running at 2800 MHz with 22.9 TFLOPS of FP32 peak performance... (Scout's description)
UNVERIFIED: Intel's official B70 pages list no price (no Recommended Customer Price on the spec page; the product page links to retailers only). The stored $949 is not confirmed. Spec facts CONFIRMED 2026-10-20: 32 GB GDDR6, 608 GB/s, 230 W TBP, launched Q1'26. vLLM is Apache-2.0 open source (stored, not re-read today). as of 2026-10-23unverified in Scout's data
Credited to Scout (2026-10-23T13:15Z, Scout patch_history.at). gaents does not score items.
Scout's notes
Hardware+software combo: HW lock-in to Intel Xe2 architecture, SW is Intel's vLLM fork (Apache-2.0). Consumer-grade 32GB VRAM, sub-$1K vs NVIDIA RTX 4000 Pro ($1,800, 24GB). Niche for local LLM inference. [2026-10-23T13:15Z cloud refresh: type: was 'hybrid'. Set to commercial because the row's paid item is the GPU, which is sold only through retailers (Intel page lists shops, no free tier). The serving software (intel/llm-scaler, vLLM-based Docker image) is Apache-2.0, so the software half alone would be open-source. Scout may prefer a separate 'hardware' type.] [2026-10-23T13:15Z cloud refresh: 2026-10-20: $949 not found on intel.com; Intel shows retailer links (Newegg, Micro Center, Proshop etc.). type left for the type agent.] [2026-10-23T13:15Z cloud refresh: Fact fields 2026-10-20: row mixes Intel hardware with vLLM; fields describe the vLLM software part. licence_class from vLLM LICENSE (Apache 2.0); self_hostable from vLLM GPU install docs (Intel XPU supported); api_or_webhooks from vLLM online serving docs (HTTP server). Intel product page returned 403. data_export n/a (inference engine).]
Not a product — a personal, reproducible bootstrap recipe for bringing a freshly-rented GPU instance up to serving DeepSeek V4 Flash locally. The chain is: keennay/gpu-cluster-setup (host bootstrap scripts) -> KVCache-AI KT-Kernel + their SGLang fork (CPU/GPU hybrid MoE expert offload) -> DeepSeek-V4-Flash weights from HuggingFace -> FlashMLA (DeepSeek's own MLA attention kernel, requires CUDA 12.9) ->... (Scout's description)
MIXED: the setup repo (moved from keennay/gpu-cluster-setup to yoniq-org/compute) now has an Apache-2.0 LICENSE (Copyright 2026 YONIQ LLC); other parts as stored before and not re-read here: kvcache-ai/ktransformers Apache-2.0, DeepSeek-V4-Flash weights MIT, deepseek-ai/FlashMLA MIT, local-inference-lab/llm-inference-bench no licence
Software free. Real cost is GPU: the recipe pins tensor-parallel-size 2 with NUMA-pinned CPU offload (30 CPU-inference threads, 134 GPU experts), i.e. a 2-GPU box plus substantial host RAM, rented or owned. No free tier exists for this — the entire proposition is capex/rental displacing per-token API spend. as of: no date recorded for this field
Scout, alternatives.json v3.72026-10-23T13:15ZScout patch_history.at; no date recorded for this field itself
GitHub stars
53 as of: no date recorded for this field
Scout, alternatives.json v3.72026-10-23T13:15ZScout patch_history.at; no date recorded for this field itself
BRITTLE BY CONSTRUCTION. The recipe pins exact versions at nearly every step — tilelang==0.1.8, apache-tvm-ffi==0.1.9, transformers==4.57.1, nvidia-cudnn-cu12==9.16.0.29, CUDA 12.9, and flash-mla installed from a specific git SHA — plus a launch command carrying roughly thirty tuned flags including eight SGLANG_* environment variables, numactl CPU/memory binding, --kt-num-gpu-experts 134 and --kt-max-deferred-experts-per-token 2. Every one of those is a future breakage. This is an expert-operated stack, not an installable one, and there is no vendor to escalate to. The orchestrating repo carries no licence and 51 stars — that is one person's working notes published, not maintained infrastructure, and we should not build a dependency on it. The benchmark repo (llm-inference-bench, 59 stars) is likewise unlicensed. On the build-vs-buy axis the case is not obviously favourable: we already hold NVIDIA build.nvidia.com free tier (~133 listed, 55/82 returned 404 on 09-22; the '$900/mo displacement' figure was never measured, withheld 09-23) plus Groq and withheld as zero-ops routes, and self-hosting only wins past a high sustained-volume crossover that nobody has computed. My integration estimate of 40 hours is to first reliable serve, and explicitly excludes ongoing operational burden, which for a stack this pinned is the larger number.
Lock-in risk
medium
Credited to Scout (2026-10-23T13:15Z, Scout patch_history.at). gaents does not score items.
Scout's notes
Composite item — one tweet, five repos plus a HuggingFace model. When merging, do NOT create an entry for keennay/gpu-cluster-setup (51 stars, unlicensed, personal). DO consider two durable entries under LLM Serving (Local): (1) KTransformers, Apache-2.0, 19,016 stars, as the CPU/GPU hybrid MoE expert-offload runtime that makes large MoE models fit on modest GPU counts; (2) DeepSeek-V4-Flash, MIT weights, 1M context, 3.16M HF downloads, as a self-hostable frontier-class model. Route both to withheld for the compute-economics call before assigning any verdict stronger than park. Cross-reference the existing NVIDIA build.nvidia.com ADOPT-NOW entry under Model Routing / LLM Proxy — that is the zero-ops comparator this stack has to beat, and stating the comparison explicitly is what keeps the build-vs-buy question honest. [2026-10-23T13:15Z cloud refresh: Sweep 2026-10-20: last commit 2026-10-20; licence changed: MIXED: the setup repo. repo moved to github.com/yoniq-org/compute. licence_class left OPEN (row mixes several repos and weights).]
ODS (Osmantic Deployment System)
open-sourcenot runnable on gaents yet
All-in-one local AI server installer. Detects hardware, downloads appropriate model, starts inference, launches chat UI. Bundles: LLM inference, chat UI, voice, agents, workflows, RAG, search, image generation, privacy tooling, operator control. Single installer for the full local AI stack. Python-based, runs on PC/Mac/Linux. (Scout's description)
Interesting for greenfield local AI deployments. Less relevant for us since we already run withheld + individual components. Watch for maturity.
Limits for us
We already have withheld (adopt-now) for model serving, plus individual tools for each component ODS bundles. ODS's value is the single-installer experience, which matters less when you already have the pieces running. 3.8K stars is respectable but much smaller than withheld (137K+).
Credited to Scout (2026-10-23T13:15Z, Scout patch_history.at). gaents does not score items.
Scout's notes
[2026-10-20T01:33Z cloud refresh: type None -> open-source (license text: Apache; rule set agent-chose 2026-10-20, Adam to rule)] [2026-10-23T13:15Z cloud refresh: Sweep 2026-10-20: last commit 2026-10-19; licence unchanged (Apache-2.0).]
Superlinked SIE (Inference Engine)
open-sourcenot runnable on gaents yet
Serves several small models (embedder, reranker, extractor/NER, small generator) from one process with LRU residency, instead of one server per model. (Scout's description)
Credited to Scout (2026-10-23T13:15Z, Scout patch_history.at). gaents does not score items.
Scout's notes
THE 75% CLAIM IS ARITHMETIC, NOT A MEASUREMENT. Traced to the author's own article: 4 models on 4 cards -> 1 card means 'three of those terms disappear... the bill drops by 75%'. That is 4->1 GPU-hours restated, assuming perfect packing and zero eviction thrash. THE VENDOR'S OWN BENCHMARK SAYS 50% ($2,304/mo on 4 GPUs vs $1,152 on 2), and elsewhere quotes a 71% warm-start hit rate — i.e. 29% of requests pay a cold load. ALSO: the source tweet is not about vLLM sleep mode at all; it promotes SIE and names vLLM as part of the PROBLEM. Two different mechanisms, and the 75% belongs to neither. TUNABLES: `pressure_threshold` — DOCS ARE SELF-INCONSISTENT, 0.85 in the blog vs 0.95 in the deployment guide; verify empirically before tuning. Capacity rule of thumb: reserve 2-3x model weight size. ⚠ TELEMETRY IS ON BY DEFAULT — disable with SIE_TELEMETRY_DISABLED=1 or DO_NOT_TRACK=1. Repo created 2023-11-07 but repurposed from the older Superlinked framework, so age does not imply SIE maturity. [2026-10-20T01:33Z cloud refresh: type None -> open-source (license text: Apache; rule set agent-chose 2026-10-20, Adam to rule)] [2026-10-23T13:15Z cloud refresh: Fact fields 2026-10-20: licence_class from LICENSE (Apache-2.0); self_hostable and api_or_webhooks from README. data_export n/a (inference engine).]
local-ai-registry
open-sourcenot runnable on gaents yet
See research/tweet-triage-2026-10-02-scout.md (section for local-ai-registry). (Scout's description)
One triage pass; re-verify stars, licence and price before acting.
Limits for us
See research/tweet-triage-2026-10-02-scout.md (section for local-ai-registry).
Credited to Scout (2026-10-23T13:15Z, Scout patch_history.at). gaents does not score items.
Scout's notes
Merged from research/tweet-triage-2026-10-02-scout.md by merge_triage_table.py. [2026-10-20T01:33Z cloud refresh: type None -> open-source (license text: MIT; rule set agent-chose 2026-10-20, Adam to rule)] [2026-10-23T13:15Z cloud refresh: Sweep 2026-10-20: last commit 2026-10-20; licence unchanged (MIT). repo moved to github.com/sybil-solutions/local-ai-registry.]
llmfit
open-sourcenot runnable on gaents yet
See research/tweet-triage-2026-10-02-scout.md (section for llmfit). (Scout's description)
One triage pass; re-verify stars, licence and price before acting.
Limits for us
See research/tweet-triage-2026-10-02-scout.md (section for llmfit).
Credited to Scout (2026-10-23T13:15Z, Scout patch_history.at). gaents does not score items.
Scout's notes
Merged from research/tweet-triage-2026-10-02-scout.md by merge_triage_table.py. [2026-10-20T01:33Z cloud refresh: type None -> open-source (license text: MIT; rule set agent-chose 2026-10-20, Adam to rule)] [2026-10-23T13:15Z cloud refresh: Sweep 2026-10-20: last commit 2026-10-20; licence unchanged (MIT).]
turboquant
open-sourcenot runnable on gaents yet
See research/tweet-triage-2026-10-02-scout.md (section for turboquant). (Scout's description)
One triage pass; re-verify stars, licence and price before acting.
Limits for us
See research/tweet-triage-2026-10-02-scout.md (section for turboquant).
Credited to Scout (2026-10-23T13:15Z, Scout patch_history.at). gaents does not score items.
Scout's notes
Merged from research/tweet-triage-2026-10-02-scout.md by merge_triage_table.py. [2026-10-20T01:33Z cloud refresh: type None -> open-source (license text: GPL; rule set agent-chose 2026-10-20, Adam to rule)] [2026-10-23T13:15Z cloud refresh: Sweep 2026-10-20: last commit 2026-09-03; licence unchanged (GPL-3.0).]
GLM-5.3 open weights (self-hosted)
source-availablenot runnable on gaents yet
Zhipu's flagship open-weight model; successor to GLM-5.1 in this category. (Scout's description)
Credited to Scout (2026-11-09T08:48Z, Scout patch_history.at). gaents does not score items.
Scout's notes
Self-hosting only pays when data must not leave us; on price the API wins. [2026-10-23T13:15Z cloud refresh: Sweep 2026-10-20: HF lastModified 2026-10-14; HF licence tag other; gated=false. licence_class source-available (agent-chose): MIT-style terms plus a Z.AI security review for Model-as-a-Service firms over USD 10B revenue (a use restriction, like Llama-style terms). type open-source -> source-available: licence has use restrictions (see licence_class note).] [2026-11-09T08:48Z cloud refresh: Tweet triage 2026-11-04 #14 (2026-11-09): External datapoint (The Register, 2026-11-03, from David Abitbol's AI Engineer NYC talk): Revolut self-hosts GLM (started 5.1, moved to 5.3) on 56 Nvidia B200s for ~2,000 engineers; '$0.30 per million input/output tokens' blended, about a tenth of Codex/Claude Code; 70% lead-time improvement. GLM is text-only so vision goes to other models via a router; The Register does not say Claude Code/Codex were put on an 'exception list' (that is the tweet's wording). The $0.30 excludes staff/ops per commentators. At our scale (no GPUs) the API stays cheaper; withheld/Fathom lead.]
withheld open weights (self-hosted)
open-sourceno public gaents link
Smaller, MIT-licensed GLM-5.3 sibling; the cheaper self-host candidate withheld did not price. (Scout's description)
One triage pass; re-verify stars, licence and price before acting.
Limits for us
See research/tweet-triage-2026-10-14-scout.md (section for Kolibri-1 (Aleph Alpha)).
Credited to Scout (2026-10-23T13:15Z, Scout patch_history.at). gaents does not score items.
Scout's notes
Merged from research/tweet-triage-2026-10-14-scout.md by merge_triage_table.py. [2026-10-20T01:33Z cloud refresh: type None -> open-source (license text: Apache; rule set agent-chose 2026-10-20, Adam to rule)] [2026-10-23T13:15Z cloud refresh: Sweep 2026-10-20: HF lastModified 2026-10-20; HF licence tag apache-2.0; gated=false.]
Mistral Large 4 "Le Chonk"
commercialnot runnable on gaents yet
See research/tweet-triage-2026-10-14-scout.md (section for Mistral Large 4 "Le Chonk"). (Scout's description)
CONFIRMED 2026-10-19: public preview API $1.36 per M input tokens, $4.18 per M output tokens. Weights promised 'by the end of the month' (announcement dated 2026-10-06); docs list licence only as 'Open'. as of 2026-10-19
Scout, alternatives.json v3.72026-10-19date written in Scout's pricing text
One triage pass; re-verify stars, licence and price before acting.
Limits for us
See research/tweet-triage-2026-10-14-scout.md (section for Mistral Large 4 "Le Chonk").
Credited to Scout (2026-10-23T13:15Z, Scout patch_history.at). gaents does not score items.
Scout's notes
Merged from research/tweet-triage-2026-10-14-scout.md by merge_triage_table.py. [2026-10-20T01:33Z cloud refresh: type None -> commercial (weights not released; paid API; rule set agent-chose 2026-10-20, Adam to rule)] [2026-10-23T13:15Z cloud refresh: Fact fields 2026-10-20: api_or_webhooks from the launch post (preview API on Mistral Studio). licence_class and self_hostable left out: weights not yet released and no licence named on the launch page ('We will release the weights by the end of the month'). data_export n/a (model).]
MiniCPM-V 4.7
open-sourcenot runnable on gaents yet
See research/tweet-triage-2026-10-14-scout.md (section for MiniCPM-V 4.7). (Scout's description)
One triage pass; re-verify stars, licence and price before acting.
Limits for us
See research/tweet-triage-2026-10-14-scout.md (section for MiniCPM-V 4.7).
Credited to Scout (2026-10-23T13:15Z, Scout patch_history.at). gaents does not score items.
Scout's notes
Merged from research/tweet-triage-2026-10-14-scout.md by merge_triage_table.py. [2026-10-20T01:33Z cloud refresh: type None -> open-source (license text: Apache; rule set agent-chose 2026-10-20, Adam to rule)] [2026-10-23T13:15Z cloud refresh: Fact fields 2026-10-20: licence_class from repo model-licence section (weights and code Apache-2.0); self_hostable from README (withheld); api_or_webhooks from transformers inference docs; data_export n/a (model).]
AMD Ryzen AI Max+ 395 mini PC (128 GB)
commercialnot runnable on gaents yet
See research/tweet-triage-2026-10-14-scout.md (section for AMD Ryzen AI Max+ 395 mini PC (128 GB)). (Scout's description)
From ~$1,499 (third-party, e.g. GMKtec EVO-X2); AMD Halo box $3,999.99 as of: no date recorded for this field
Scout, alternatives.json v3.72026-10-19T23:00ZScout patch_history.at; no date recorded for this field itself
Scout's judgement
Verdict
consider
Reasoning (de-assume)
One triage pass; re-verify stars, licence and price before acting.
Limits for us
See research/tweet-triage-2026-10-14-scout.md (section for AMD Ryzen AI Max+ 395 mini PC (128 GB)).
Credited to Scout (2026-10-19T23:00Z, Scout patch_history.at). gaents does not score items.
Scout's notes
Merged from research/tweet-triage-2026-10-14-scout.md by merge_triage_table.py. [2026-10-19T23:00Z cloud refresh: 2026-10-19: official AMD Ryzen AI Halo page found (replaces the unverified amd.com URL). Page shows no price, so the $3,999.99 Halo box price stays UNVERIFIED. Page says it is for the US market only: 'This product is available for purchase and use in the United States.' Chip spec page: https://www.amd.com/en/products/processors/laptop/ryzen/ai-300-series/amd-ryzen-ai-max-plus-395.html ('Max. Memory 128 GB').]
Magnitude
open-sourcenot runnable on gaents yet
See research/tweet-triage-2026-10-18-scout.md (section for Magnitude). (Scout's description)
One triage pass; re-verify stars, licence and price before acting.
Limits for us
See research/tweet-triage-2026-10-18-scout.md (section for Magnitude).
Credited to Scout (2026-10-23T13:15Z, Scout patch_history.at). gaents does not score items.
Scout's notes
Merged from research/tweet-triage-2026-10-18-scout.md by merge_triage_table.py. [2026-10-20T01:33Z cloud refresh: type None -> open-source (license text: Apache; rule set agent-chose 2026-10-20, Adam to rule)] [2026-10-23T13:15Z cloud refresh: Fact fields 2026-10-20: licence_class from LICENSE (Apache-2.0); self_hostable from README (runs on your hardware, desktop download); api_or_webhooks from README (OpenAI-compatible API); data_export n/a (inference engine).]
llama.cpp (llama-server)
open-sourcenot runnable on gaents yet
See research/tweet-triage-2026-10-18-scout.md (section for llama.cpp (llama-server)). (Scout's description)
One triage pass; re-verify stars, licence and price before acting.
Limits for us
See research/tweet-triage-2026-10-18-scout.md (section for llama.cpp (llama-server)).
Credited to Scout (2026-11-09, Scout last_verified). gaents does not score items.
Scout's notes
Merged from research/tweet-triage-2026-10-18-scout.md by merge_triage_table.py. [2026-10-20T01:33Z cloud refresh: type None -> open-source (license text: MIT; rule set agent-chose 2026-10-20, Adam to rule)] [2026-10-23T13:15Z cloud refresh: Fact fields 2026-10-20: licence_class from LICENSE (MIT); self_hostable and api_or_webhooks from README (local OpenAI-compatible server); data_export n/a (inference server).]
ik_llama.cpp
open-sourcenot runnable on gaents yet
See research/tweet-triage-2026-10-18-scout.md (section for ik_llama.cpp). (Scout's description)
One triage pass; re-verify stars, licence and price before acting.
Limits for us
See research/tweet-triage-2026-10-18-scout.md (section for ik_llama.cpp).
Credited to Scout (2026-10-23T13:15Z, Scout patch_history.at). gaents does not score items.
Scout's notes
Merged from research/tweet-triage-2026-10-18-scout.md by merge_triage_table.py. [2026-10-20T01:33Z cloud refresh: type None -> open-source (license text: MIT; rule set agent-chose 2026-10-20, Adam to rule)] [2026-10-23T13:15Z cloud refresh: Sweep 2026-10-20: last commit 2026-10-20; licence unchanged (MIT).]
LM Studio
freemiumnot runnable on gaents yet
See research/tweet-triage-2026-10-18-scout.md (section for LM Studio). (Scout's description)
One triage pass; re-verify stars, licence and price before acting.
Limits for us
See research/tweet-triage-2026-10-18-scout.md (section for LM Studio).
Credited to Scout (2026-10-23T13:15Z, Scout patch_history.at). gaents does not score items.
Scout's notes
Merged from research/tweet-triage-2026-10-18-scout.md by merge_triage_table.py. [2026-10-19T23:00Z cloud refresh: 2026-10-19: LM Studio's desktop app is now marketed as 'LM Studio Bionic' (an agent) with an optional paid cloud tier Bionic+.] [2026-10-23T13:15Z cloud refresh: Fact fields 2026-10-20: licence_class proprietary for the app (App Terms licence grant); the lms CLI is MIT per stored licence. self_hostable from download page. api_or_webhooks from developer docs (local REST API, SDKs). data_export left out (no export doc read).]
mistral.rs
open-sourcenot runnable on gaents yet
See research/tweet-triage-2026-10-18-scout.md (section for mistral.rs). (Scout's description)
One triage pass; re-verify stars, licence and price before acting.
Limits for us
See research/tweet-triage-2026-10-18-scout.md (section for mistral.rs).
Credited to Scout (2026-10-23T13:15Z, Scout patch_history.at). gaents does not score items.
Scout's notes
Merged from research/tweet-triage-2026-10-18-scout.md by merge_triage_table.py. [2026-10-20T01:33Z cloud refresh: type None -> open-source (license text: MIT; rule set agent-chose 2026-10-20, Adam to rule)] [2026-10-23T13:15Z cloud refresh: Sweep 2026-10-20: last commit 2026-10-20; licence unchanged (MIT).]
SGLang
open-sourcenot runnable on gaents yet
See research/tweet-triage-2026-10-18-scout.md (section for SGLang). (Scout's description)
One triage pass; re-verify stars, licence and price before acting.
Limits for us
See research/tweet-triage-2026-10-18-scout.md (section for SGLang).
Credited to Scout (2026-10-23T13:15Z, Scout patch_history.at). gaents does not score items.
Scout's notes
Merged from research/tweet-triage-2026-10-18-scout.md by merge_triage_table.py. [2026-10-20T01:33Z cloud refresh: type None -> open-source (license text: Apache; rule set agent-chose 2026-10-20, Adam to rule)] [2026-10-23T13:15Z cloud refresh: Sweep 2026-10-20: last commit 2026-10-20; licence unchanged (Apache-2.0).]
ExLlamaV3
open-sourcenot runnable on gaents yet
See research/tweet-triage-2026-10-18-scout.md (section for ExLlamaV3). (Scout's description)
One triage pass; re-verify stars, licence and price before acting.
Limits for us
See research/tweet-triage-2026-10-18-scout.md (section for ExLlamaV3).
Credited to Scout (2026-10-23T13:15Z, Scout patch_history.at). gaents does not score items.
Scout's notes
Merged from research/tweet-triage-2026-10-18-scout.md by merge_triage_table.py. [2026-10-20T01:33Z cloud refresh: type None -> open-source (license text: MIT; rule set agent-chose 2026-10-20, Adam to rule)] [2026-10-23T13:15Z cloud refresh: Sweep 2026-10-20: last commit 2026-10-17; licence unchanged (MIT).]
TensorRT-LLM
open-sourcenot runnable on gaents yet
See research/tweet-triage-2026-10-18-scout.md (section for TensorRT-LLM). (Scout's description)
One triage pass; re-verify stars, licence and price before acting.
Limits for us
See research/tweet-triage-2026-10-18-scout.md (section for TensorRT-LLM).
Credited to Scout (2026-10-23T13:15Z, Scout patch_history.at). gaents does not score items.
Scout's notes
Merged from research/tweet-triage-2026-10-18-scout.md by merge_triage_table.py. [2026-10-20T01:33Z cloud refresh: type None -> open-source (license text: Apache; rule set agent-chose 2026-10-20, Adam to rule)] [2026-10-23T13:15Z cloud refresh: Sweep 2026-10-20: last commit 2026-10-20; licence unchanged (Apache-2.0).]
DeepInfer (DeepSeek)
open-sourcenot runnable on gaents yet
DeepSeek's production inference framework, open-sourced 2026-10-20. Supports one model: DeepSeek-V4.1-Flash. Derived mainly from vLLM with ideas from SGLang and TensorRT-LLM. (Scout's description)
Cloud research pass 2026-10-19; re-verify price and licence before acting.
Limits for us
Hardware: NVIDIA SM100 (Blackwell) + CUDA 13, or Ascend 950 + CANN 9.2. Benchmarks use 4 GPUs prefill + 64 decode. Open release omits KV cache system, PD disaggregation and DP load balancing ('required for a complete inference service'). None of our boxes qualify (withheld RTX 2070 is Turing; main has no GPU; air1/air2 are Macs).
Lock-in risk
low
Integration hours (estimate)
40
Feature overlap (Scout's estimate)
0.3
Credited to Scout (2026-10-23, Scout evidence.read_at). gaents does not score items.
Tweet triage 2026-10-23 #10. For us: use DeepSeek-V4.1-Flash via API or vLLM/SGLang; DeepInfer only matters if we ever rent Blackwell nodes. withheld/Fathom lead on model side.
RWKV-7 G1 (RNN LLM)
open-sourcenot runnable on gaents yet
Attention-free RNN LLM family ('Goose'), linear time and constant memory per token; small checkpoints suitable for CPU/Mac tests. (Scout's description)
Cloud research pass 2026-10-19; re-verify price and licence before acting.
Limits for us
Not the Memory Caching architecture (Google released no code/weights for that); quality below same-size transformers on many tasks; tooling support narrower than llama.cpp mainstream models.
Lock-in risk
low
Integration hours (estimate)
3
Feature overlap (Scout's estimate)
0.2
Credited to Scout (2026-10-23, Scout evidence.read_at). gaents does not score items.
https://arxiv.org/abs/2602.24281 (read 2026-10-23): “Memory Caching: RNNs with Growing Memory (no code or weights link on the abstract page)”
Scout's notes
Tweet triage 2026-10-23 #11: the testable stand-in for 'RNN with long memory'. Fathom could run a small G1 on air2 vs a same-size Qwen on our held-out sets.
Falcon-H1 (hybrid attention+Mamba)
open-sourcenot runnable on gaents yet
TII hybrid transformer+Mamba-2 model family, 0.5B to 34B, base and instruct checkpoints on HF. (Scout's description)
Tweet triage 2026-10-23 #13: the only open Meta byte model. The 'Breaking the Token Ceiling' paper (arXiv 2609.12303) released no code or weights, and its 'surpasses Llama 3.2-1B' numbers are extrapolated scaling-law predictions.
EvaByte
open-sourcenot runnable on gaents yet
Open byte-level LM (6.5B per its model card; size not re-checked 2026-10-23), base and SFT checkpoints. (Scout's description)
Tweet triage 2026-10-23 #13 alternative set (open byte-level model).
Apple Foundation Models (on-device, fm CLI)
commercialnot runnable on gaents yet
Apple's ~3B on-device language model framework (macOS/iOS 26+), with structured (schema-guided) output and tool calling; macOS 27 adds an fm Terminal CLI (respond, chat, schema, serve = OpenAI-compatible local endpoint). 4,096-token session context shared by prompt and answer. (Scout's description)
Free on Apple-silicon devices with Apple Intelligence enabled as of: no date recorded for this field
Scout, alternatives.json v3.72026-11-17T22:13ZScout patch_history.at; no date recorded for this field itself
Scout's judgement
Verdict
watch
Reasoning (de-assume)
Scout local research pass (2026-11-09-tweet-triage-1103.json); re-verify price and licence before acting.
Limits for us
4K context breaks our batched Roar calls (needed max_tokens 4096). --model pcc runs on Apple Private Cloud Compute, so 'offline' holds only for the system model. Fathom's probe on air2 (macOS 26.4.1) returned appleIntelligenceNotEnabled, so no quality number yet.
Lock-in risk
medium
Integration hours (estimate)
4
Feature overlap (Scout's estimate)
0.2
Credited to Scout (2026-11-17T22:13Z, Scout patch_history.at). gaents does not score items.
Scout tweet triage 2026-11-03 (research/tweet-triage-2026-11-03-scout.md) item 5. Fathom leads measurement (reply-Fathom.md item 1): needs Adam to enable Apple Intelligence on air2. Value is the client Tier 1 offline Mac case, not our own cost (GLM-5.3 Flash is already fractions of a cent per 1K items).
LiquidAI LFM3 family (1.7B, VL-2B, Audio-2B, Omni-2B)
source-availablenot runnable on gaents yet
Small on-device models: LFM3-1.7B text (256K context, tool calling), LFM3-VL-2B vision with video input, LFM3-Audio-2B, LFM3-Omni-2B (text/image/audio in, text+audio out). Omni-2B runs fully in a browser via WebGPU (HF Space, ~1.6 GB download). (Scout's description)
Scout local research pass (2026-11-09-tweet-triage-1103.json); re-verify price and licence before acting.
Limits for us
1.7B card says not for agentic coding or complex multi-step reasoning. We have no omni/voice task to measure on; it cannot beat GLM-5.3 Flash on cost for us. Not on withheld.
Lock-in risk
low
Integration hours (estimate)
4
Feature overlap (Scout's estimate)
0.2
Credited to Scout (2026-11-17T22:13Z, Scout patch_history.at). gaents does not score items.
Scout tweet triage 2026-11-03 (research/tweet-triage-2026-11-03-scout.md) item 16 and 31. '~280 tok/s on M4 Max' and '90 ms first token' (xenova) are the demo author's numbers, unverified. Useful for a client offline voice-assistant quote. Fathom leads measurement.
uzu (Mirai)
open-sourcenot runnable on gaents yet
Rust + Metal inference engine for Apple silicon with Swift/Python/TypeScript/Rust bindings; speculative decoding with a 16-token draft tree (DFlash drafter + 56.7M Weaver adapter), int8 verification on M5, Mirai's own 4-bit checkpoints. (Scout's description)
Scout tweet triage 2026-11-03 (research/tweet-triage-2026-11-03-scout.md) item 18. Tweet's 92 vs 22 tok/s compares uzu with speculative decoding against llama.cpp/MLX without it; ceiling arithmetic (153 GB/s / 5.2 GB ~ 29 tok/s) checks out. Fathom proposes a $0 reproduction on air2 (reply-Fathom.md item 4).
kimi-k3-in-c
open-sourcenot runnable on gaents yet
Portable C99 inference engine (176 KB, no BLAS/GPU) that runs the 2.78T-parameter Kimi K3 on one CPU by streaming weights from disk; measured peak RSS 8.24 GB. (Scout's description)
Scout local research pass (2026-11-09-tweet-triage-1104.json); re-verify price and licence before acting.
Limits for us
README's own timings: 26.5 s per token on an 8 GB laptop, 19.8 s at 64 GB, 5.6 s at 128 GB+ (124-core box, fast NVMe). 1.56 TB checkpoint. A proof of concept, not usable throughput.
Lock-in risk
low
Integration hours (estimate)
6
Feature overlap (Scout's estimate)
0.0
Credited to Scout (2026-11-17T22:13Z, Scout patch_history.at). gaents does not score items.
https://github.com/FareedKhan-dev/kimi-k3-in-c (read 2026-11-09): “2.78T parameters | 1.56 TB checkpoint on disk | 8.24 GB peak RSS, measured | 176 KB the whole engine ... an ordinary laptop 8 GB 26.5 s ... a heavy workstation 128 GB+ 5.6 s”
Scout's notes
Tweet triage 2026-11-04 #18. Tweet (@k2sbhai) is literally true on RAM but omits seconds-per-token and the 1.56 TB disk. withheld/Fathom lead. Details: research/tweet-triage-2026-11-04-scout.md #18.