dsh-qwen-multimodal

dsh-qwen-multimodal

A DSH bundle that gives text-only main models (e.g. DeepSeek) **three multimodal skills in one plugin** through Qwen APIs: vision, speech-to-text, and text-to-image — with a built-in generate-then-verify quality loop. | Tool | Capability | Backend | |---|---|---| | describe_image | Image / screenshot / OCR / chart understanding (multiple images at once) | Qwen VL (default qwen3-vl-flash) | | transcribe_audio | Speech / recording transcription (wav/mp3/m4a/aac/flac/ogg/amr) | Qwen3-ASR (qwen3-asr-flash) | | generate_image | Generate images from text and save them locally | Qwen-Image (qwen-image-plus) | Media never enters the main model context: visual/audio content is converted to text, and generated images are saved to local files with paths returned by the tool.

How it works

All API calls reuse the original Python scripts in skills/deepseek-vision/scripts/*.py and the .env configuration (vision/audio use the Alibaba Cloud Bailian OpenAI-compatible endpoint; image generation uses the native multimodal-generation endpoint). The plugin itself is a pure-JS Cordis bundle depending only on the host's mounted subprocess / tools services — **no build step required for git installs**.

Install

From GitHub

``sh dsh plugin --profile demo add github:wuwangmao/dsh-qwen-multimodal `

Local checkout / tarball

`sh dsh plugin --profile demo add ./dsh-qwen-multimodal

or

pnpm pack # then dsh plugin --profile demo add ./dsh-qwen-multimodal-0.1.0.tgz ` Before first use, configure your API key: copy skills/deepseek-vision/.env.example to skills/deepseek-vision/.env and fill in VISION_API_KEY (create one in the Alibaba Cloud Bailian console; new users get free quota, college students get a ¥300 annual voucher). Vision/audio/image reuse the same key by default, or configure them separately (see .env.example).

Python

**Python 3.10+ is required** (the image-generation script uses int | None type-annotation syntax). python is resolved from the system PATH by default. If it cannot be resolved, restate the plugin row in your profile's cordis.patch.yml and set config.pythonPath.

Configuration overrides

The plugin uses the bundled skill directory by default. To point it at an external directory (e.g. to reuse an existing .env and scripts, or to keep your key outside node_modules), restate the row: `yaml - insert: - id: qwen-multimodal name: dsh-qwen-multimodal config: skillDir: 'D:/qwen-vision' pythonPath: 'C:/path/to/python.exe' `

Usage

Once loaded, the model can call the three tools directly: - describe_image({ images: ['screenshot.png'] }) — verbatim extraction of text/code/errors in images - describe_image({ images: ['chart.png'], prompt: '逐字提取图中所有文字，保留原样' }) — custom prompt - transcribe_audio({ audios: ['recording.m4a'], language: 'zh' }) — specify language for accuracy - generate_image({ prompt: 'a cute orange cat on a windowsill watching the sunset', out_dir: './out' }) — generate and save locally - generate_image({ prompt: '...', out_dir: './out', verify: true }) — generate, then automatically re-check the result with Qwen VL against the prompt (quality loop)

Layout

` dsh-qwen-multimodal/ ├── package.json # dsh.bundle manifest ├── cordis.patch.yml # bundle layer: inserts the plugin row ├── src/index.js # plugin: registers the three model tools (pure JS) ├── scripts/selfcheck.mjs # self-check: node scripts/selfcheck.mjs └── skills/deepseek-vision/ # skill assets: SKILL.md + Python scripts + .env.example ``

License

MIT