Why 8B Models Loop on Agents: 3 Hermes Tunings for Low VRAM (2026)

In 2026, running Llama 3.1/3.2 8B, Qwen 2.5 7B, or Hermes 3 8B locally feels like a bargain—until the agent enters a death loop: malformed JSON tool calls, repeated terminal invocations, or “I will now run the tests…” without ever calling a tool. The GPU is fine; the sampling + context + tool surface is not.

Hermes Agent (NousResearch/hermes-agent, MIT) is built for frontier models but runs on OpenRouter, Ollama, and other OpenAI-compatible endpoints. This guide explains why small models break on agents and gives three hard config levers—plus a seven-step runbook—for stable long tasks on 16–24 GB VRAM Macs or Linux boxes.

Why 8B models choke on function calling

Small instruct models are trained for single-shot chat, not fifty-turn tool transcripts. Agent loops punish three weaknesses:

Quotable definition: On Hermes, a “tool call” is a structured message the runtime parses and executes; when an 8B model emits narrative text instead of that structure, the loop never advances and context fills with duplicate plans.

Frontier models recover with size; 8B models need a narrower tool menu, lower reasoning overhead, and aggressive context hygiene. Pair this article with trajectory compression so middle-turn bloat does not amplify loops. When sizing unified memory, see Apple Mac mini specs.

Architecture: what Hermes actually sends each turn

Hermes applies model-specific temperature contracts internally. For local servers, set sampling on the inference server (Ollama PARAMETER temperature 0.2, llama.cpp --temp 0.2) and keep Hermes-side reasoning and tool knobs as the main levers until #17565 exposes user temperature in YAML.

Three tunings that matter more than raw temperature

Tuning 1 — Kill hidden reasoning tax (reasoning_effort: none)

Default Hermes reasoning is medium, which helps Opus-class models but often bloats small locals:

agent: reasoning_effort: none # or minimal for light planning

display: show_reasoning: false

/reasoning /reasoning none

Symptom fixed: long blank “thinking” stretches before malformed tool JSON appears.

Tuning 2 — Force real tool calls (tool_use_enforcement)

8B Llama/Qwen frequently narrate actions. Hermes can inject enforcement prompts:

agent: tool_use_enforcement: true

provider_routing: require_parameters: true

Default "auto" enables enforcement for GPT/Gemini families only—local 8B stacks are off unless you opt in. See provider routing docs.

Tuning 3 — Skill diet: two skills, minimal toolsets

Skills use progressive disclosure, but the index alone costs ~3k tokens. Optional MCP catalogs add more.

Launch pattern for 8B:

hermes chat --toolsets "terminal,file" \ -s github-pr-workflow,plan \ -m "qwen/qwen-2.5-7b-instruct"

• ≤2 skills preloaded (-s skill-a,skill-b); load others manually only when needed
• Disable heavy toolsets globally—see snippet below
• Do not install every optional-mcps/ entry on an 8B box—each MCP expands schemas and failure modes (MCP Mac guide)

agent: disabled_toolsets: - web - browser - image_gen - moa - memory

Optional: blank skill slate at profile create:

hermes profile create local-8b --no-skills

VRAM presets: what fits on Mac / single-GPU Linux

Not every “8B” setup is the same. Quantization (Q4_K_M vs Q8), gateway overhead, and whether you run browser MCP sidecars change stability more than the model name on the tin.

If X, do Y: If you see two failed JSON tool calls in a row, drop to one skill and halve tool_output.max_bytes. If loops persist after enforcement + low server temperature, the task likely needs a 14B+ tool model or cloud fallback for planning only.

Three loop patterns (and the fix)

• Parser loop — invalid JSON → retry → longer thread. Fix: tool_use_enforcement: true, fewer tools, colder server sampling.
• Plan loop — restates plan without tools. Fix: reasoning_effort: none, explicit max tool count in prompt.
• Stdout loop — reads giant logs repeatedly. Fix: lower tool_output caps, compress at 40% context, pre-truncate with head in terminal.

Log each pattern in /usage after a failed session—teams that track “tokens per successful tool call” spot regressions when they change quant or context length.

Seven-step runbook: stable 8B agent on Hermes

Step 1 — Install and pick a realistic context window

curl -fsSL https://raw.githubusercontent.com/NousResearch/hermes-agent/main/scripts/install.sh | bash source ~/.zshrc hermes doctor

Set model.context_length to match your actual local context (e.g. 8192 or 16384), not the theoretical 128k marketing number:

model: context_length: 16384

Prerequisite: Hermes Mac setup.

Step 2 — Point Hermes at your local endpoint

hermes model hermes chat -q "Reply OK only" -m "your-local-model-id"

Step 3 — Apply the three tunings in config.yaml

agent: reasoning_effort: none tool_use_enforcement: true disabled_toolsets: - web - browser - image_gen - memory compression: enabled: true threshold: 0.40 tool_output: max_bytes: 20000 max_lines: 500 file_read_max_chars: 30000

Lower thresholds than defaults—small windows fill fast.

Step 4 — Set server-side sampling (until config exposes temperature)

On Ollama Modelfile or API, prefer low temperature and modest top_p:

PARAMETER temperature 0.2 PARAMETER top_p 0.9

Hermes may still apply per-model overrides; server-side low entropy reduces JSON hallucinations.

Step 5 — Run a bounded tool task with two skills

hermes chat --toolsets "terminal,file" \ -s plan \ -m "your-local-model-id"

Task: list top 5 largest files in ./src, write paths to /tmp/top5.txt. Rules: max 8 tool calls; if stuck, stop and report blocker.

Watch the status bar context %; manually /compress if orange before auto fires.

Step 6 — Detect loops early with /usage and /stop

/usage /stop

If tool calls repeat with identical arguments three times, abort and narrow the prompt—8B models rarely self-correct without human steering.

Step 7 — Optional: gateway on an always-on Mac mini

For 24/7 automations, move gateway to hardware that does not sleep; keep 8B inference on GPU, Hermes gateway on CPU. See cron digest guide only after loops are stable—scheduled jobs amplify runaway spend.

Troubleshooting

JSON parse errors on every tool call

Symptom: Failed to parse tool call or tool name appears inside prose.

Fix: Enable tool_use_enforcement: true; reduce toolsets to terminal,file; lower server temperature; switch to a tool-tuned quant (e.g. Qwen2.5-Instruct vs base). Avoid parallel tool calls in the prompt.

Infinite “I will…” without tools

Symptom: Model describes steps; no terminal icon in TUI.

Fix: Same as above plus explicit system rule in SOUL.md: “Always invoke tools; never promise future actions.” Reduce skills to one. Check you are not on reasoning_effort: high.

Context full at turn 4 on 16K models

Symptom: Red context bar; garbled answers.

Fix: Set compression.threshold: 0.35, cut tool_output.max_bytes, disable web/browser, enable trajectory compression habits (/compress before big pastes).

FAQ

Related reading

• Headroom: local LLM tool output compression
• Hermes Mac setup guide
• Hermes MCP dev toolchain
• Hermes Trajectory Compressor
• Hermes cron Discord digest