dsh-local-perf

dsh-local-perf

Durable DeepSeek Harness plugin bundle carrying the local-model performance
tuning process as a re-installable layer — so it survives dsh updates
instead of living in hand-edited patch files that a version bump can wipe or
whose rationale dies with the author.

Install once, re-apply forever:

# from the GitHub repository
git clone https://github.com/flowingboy/dsh-local-perf.git /tmp/dsh-local-perf
dsh plugin --profile web add file:/tmp/dsh-local-perf

# or from a checkout: dsh plugin --profile web add file:./dsh-local-perf

The bundle joins the profile's layer stack after dsh-base / dsh-web-app,
re-applies all tuning rows on every boot, and vendors its own copy of the
text-toolcall-guard plugin (self-contained, no dsh checkout required).

The complete performance process (why every knob exists)

Everything below was learned the hard way on an M5 Max running four local
OpenAI-compatible servers (Ollama / MLX / Rapid-MLX / MLX-DSpark) against the
DSH web GUI. The bundle encodes the conclusions; this README preserves the
reasoning.

Incident log

Layer 1 — model configs (settings.example.yaml → ~/.dsh/settings.yaml)

Machine-specific (paths, ports, model ids), so the bundle carries them as a
template, not a runtime patch. Copy the llm-pi-ai section into
~/.dsh/settings.yaml on a fresh machine.

The recurring principles:

• Timeout ≠ prefill tolerance. A slow local server needs timeoutMs +
  streamIdleTimeoutMs ≥ worst-case prefill + reasoning + decode. Gemma 4 31B
  prefills at ~180 tok/s (system prompt + tool schemas ≈ 13k tokens → ~70s),
  mlx-dspark buffers whole generations, so both budgets sit at 300–600s.
• Retry only TRANSPORT. Connection-level failures happen before prefill
  and are cheap; a TIMEOUT must never re-prefill a long prompt.
• Context window ≤ practical prefill budget. 262144 tokens at ~180 tok/s
  is minutes of prefill. Lower to 32K–64K; compaction at 0.6× keeps sessions
  safely under the server limit.
• maxTokens ≤ decode budget. At ~27 tok/s decode, 16K output is ~10 min.
  Cap at 8192–16384 so one step's worst case fits the timeout budgets.
• Reasoning effort default "off" (or the server's lowest level) for
  quick, low-latency local loops; the local model's thinking stream still
  renders as a DSH reasoning block when enabled.

Layer 2 — cordis rows (cordis.patch.yml)

Layer 3 — text-toolcall guard (plugins/text-toolcall-guard)

Vendored from @deepseek-ai/dsh-text-toolcall-guard (built lib/ + src/).
When a step closes with no native tool calls but the assistant text carries
<tool_call> / <function=…> markers at line starts, the guard steers a
corrective message so the model re-issues the call natively. Bounded to
maxCorrections (2) per turn per agent; marker detection requires line-start
placement so prose that merely quotes the format is not corrected. Peers are
resolved at runtime from the dsh installation's profiles/node_modules
fallback (the designed out-of-tree-plugin path), so no registry fetch needed.

Installation

# from this directory
dsh plugin --profile web add file:$(pwd)
# verify the layer joined the stack
dsh --profile web --dump-config | grep -A3 "dsh-local-perf"

The web profile's cordis.patch.yml should then only hold rows this bundle
does NOT own (currently: none — everything moved into the bundle).

Updating the bundle

git pull                       # or edit locally
cd plugins/text-toolcall-guard # rebuild the vendored guard if its src changed
pnpm exec tsc -p tsconfig.json --outDir lib --declarationDir lib/types
# reinstall the layer
dsh plugin --profile web add file:$(pwd)

Publishing (GitHub / dsh-plugin ecosystem)

Published at https://github.com/flowingboy/dsh-local-perf (public, main,
topics: dsh-plugin). To re-publish after local edits:

git add -A && git commit -m "dsh-local-perf: ..."
git push origin main
# topics (once)
gh repo edit dsh-local-perf --add-topic dsh-plugin

Layout

cordis.patch.yml                  the perf layer (all tuning rows)
settings.example.yaml             model-config template (copy to ~/.dsh/settings.yaml)
plugins/text-toolcall-guard/      vendored guard plugin (lib + src)
README.md                         this document — the preserved process