dsh-kb-rag

@dsh-external/dsh-kb-rag

Local-first RAG knowledge tools for DSH — thin tool wrapper around the
kb-rag Python pipeline (Ollama bge-m3 +
ChromaDB + SiYuan). Four tools, all on-premises, zero API cost, your documents
never leave the machine.

Tool	What it does	Backing script
kb_query	Semantic search over your local vector DB (top-k chunks with source + similarity)	query.py --json
kb_ingest	Ingest a docs directory into ChromaDB (incremental, manifest hash-idempotent; Shiyu notes auto-stripped of front-matter + machine layer)	ingest.py
kb_crawl	Batch-crawl URLs to clean Markdown (Scrapling + markdownify)	crawl.py
kb_related	Related-document recommendations: retrieval-style, graph-style knowledge walk, or from a known doc	related.py

The plugin is a pure forwarder: it locates Python, spawns the kb-rag scripts and
parses their output. All knowledge-base logic lives in the Python project, which
you can also use standalone.

Install

# requires a dsh build with the `dsh plugin` subcommand; <profile> = tui/headless/web/...
dsh plugin --profile <profile> add github:YuMu247/dsh-kb-rag
dsh --profile <profile>        # restart: kb_query / kb_ingest / kb_crawl / kb_related injected

Peer dependencies (@deepseek-ai/dsh-tools, @deepseek-ai/cordis) are provided
by the dsh profile.

Prerequisites (one-time, on the host)

# 1. the Python pipeline
git clone https://github.com/YuMu247/kb-rag
pip install -r kb-rag/requirements.txt

# 2. Ollama + embedding model
ollama pull bge-m3        # ~1.2GB

# 3. tell the plugin where kb-rag lives (either one)
#    a) plugin config:  kbRagDir: "/path/to/kb-rag"
#    b) env var:        KB_RAG_DIR=/path/to/kb-rag
#    c) default:        ./kb-rag under the session working directory

# 4. ingest something, then ask
python kb-rag/ingest.py --docs /path/to/docs

Optional (Windows daily sync): examples/sync_daily.cmd in the kb-rag repo
starts Ollama, syncs Shiyu video notes, ingests and imports into SiYuan.

Tool arguments

• kb_query — query (required), k (default 5, max 20). Returns {ok, count, chunks:[{source, chunk, similarity, text}]}.
• kb_ingest — docs (default <kb-rag>/docs), force, reset, heading, chunkSize (default 512), overlap (default 64). Returns {ok, summary}.
• kb_crawl — urls (array) or urlsFile (path), out (default <kb-rag>/out), delay (default 1.5s, min 0.2). Returns {ok, summary, outDir}.
• kb_related — seed or fromDoc (exactly one), k (default 5), graph (>0 enables graph-style walk). Returns {ok, text}.

Plugin config

All keys optional, set in the profile plugin config:

Key	Default	Meaning
pythonCmd	python (win) / python3 (other)	Python executable
kbRagDir	KB_RAG_DIR env, then <cwd>/kb-rag	kb-rag checkout directory

Development

npm install                 # typescript + @types/node + dsh-tools/cordis types (compile only)
npm run typecheck           # tsc -b

Runtime has zero npm dependencies; src/ is the source of truth and lib/
the compiled copy (both shipped).

中文说明

DSH 的本地优先知识库插件:kb_query(语义检索)/ kb_ingest(增量入库)/
kb_crawl(网页批量抓取)/ kb_related(关联推荐),底层调用
kb-rag Python 管线(Ollama bge-m3 +
ChromaDB),全程本地、零 API 成本。安装:dsh plugin --profile <profile> add github:YuMu247/dsh-kb-rag。前置:克隆 kb-rag、pip install -r requirements.txt、
ollama pull bge-m3,并用配置 kbRagDir 或环境变量 KB_RAG_DIR 指明 kb-rag 路径。

License

MIT