Local LLM Hardware VRAM & Quantization Calculator
Size exact model weights, GQA KV-cache overhead, and estimated decode throughput (tok/s) across Apple Unified Memory, NVIDIA RTX 4090/5090, H100, and Exo distributed clusters before provisioning hardware.
1. Configure Model & Context
FITS_NATIVELY (20.1 tok/s)
Live Local Inference & Agent Runtime Telemetry (GitHub Baseline)
Includes live Machined Obsidian & Emerald/Amber SVG ShieldsAutonomous multi-step AI agent orchestration runtime with local & hybrid model execution.
Get up and running with OpenAI gpt-oss, DeepSeek-R1, Gemma 3 and other models.
LLM inference in C/C++ with zero dependencies across Metal, CUDA, and Vulkan.
A high-throughput and memory-efficient inference and serving engine for LLMs.
Official Model Context Protocol (MCP) reference servers for local filesystem, Git, and DB tool execution.
Unified LLM proxy and cost-routing gateway across 100+ local (Ollama/vLLM) and frontier APIs.
Run frontier AI locally with distributed Thunderbolt RDMA across consumer devices.
Fast, Pythonic framework for building zero-latency Model Context Protocol servers and tools.
