AI software

• llama.cpp Works Upstream builds run with qemu-riscv64 -cpu max,vlen=256; SpacemiT's docs give the full command to run llama-cli in QEMU. Functional validation only — the speed means nothing. github.comraw.githubusercontent.com
• onnxruntime Partial build_riscv64.sh uses QEMU to validate cross-compiled artifacts; it runs, but very slowly. raw.githubusercontent.com
• pytorch Partial The RISE wheels explicitly say they are not built for QEMU; under QEMU you can only check that things work. riseproject.dev
• opencv Works Runs in emulation; performance figures are meaningless. github.com
• ncnn Works CI passes on riscv64. github.com

NPU / acceleratorNone (emulation only)

Only for validating compilation and functionality without hardware; it says nothing about performance.

• llama.cpp No public evidence No public Unmatched-specific benchmark found; upstream only ships source (no riscv64 binaries in releases), and U74 has no RVV, so it is scalar-only and expected to be slow. github.comgithub.com
• onnxruntime Partial Upstream includes an MLAS RVV kernel and tools/scripts/build_riscv64.sh (contributed by SiFive), but there are no riscv64 prebuilt packages, so cross-compile yourself; the RVV kernel targets RVV 1.0. raw.githubusercontent.comgithub.com
• pytorch Partial PyPI has no riscv64 wheels; RISE provides a torch 2.13.0 manylinux_2_39_riscv64 wheel (needs glibc >= 2.39, e.g. Ubuntu 24.04/Bianbu). riseproject.devpypi.riseproject.dev
• opencv Works Upstream 4.x supports RISC-V and the RVV HAL; most distros can install it or build it with the RVV backend. No per-board performance validation. github.com
• ncnn Partial Upstream has linux-riscv64 CI and RVV implementations; U74 has no RVV, so ncnn runs scalar-only. github.comgithub.com
• tengine Partial tengine-lite provides RVV convolution kernels for risc-v/lp64dv and an rv64-c906 toolchain; only some operators are optimised, so support is experimental. raw.githubusercontent.comgithub.com

NPU / acceleratorNo NPU

ruyi provides OpenBSD/Ubuntu/openKylin/openWrt images; a long-standing ecosystem, but with no AI accelerator.

• llama.cpp Partial The community can build and run it, but issue #14926 reports Illegal instruction on the VF2, so set the RVV/CPU build flags correctly; with no V extension it is pure scalar inference and very slow. github.comgithub.com
• onnxruntime Partial Upstream includes an MLAS RVV kernel and tools/scripts/build_riscv64.sh (contributed by SiFive), but there are no riscv64 prebuilt packages, so cross-compile yourself; the RVV kernel targets RVV 1.0. raw.githubusercontent.comgithub.com
• pytorch Partial PyPI has no riscv64 wheels; RISE provides a torch 2.13.0 manylinux_2_39_riscv64 wheel (needs glibc >= 2.39, e.g. Ubuntu 24.04/Bianbu). riseproject.devpypi.riseproject.dev
• opencv Works Upstream 4.x supports RISC-V and the RVV HAL; most distros can install it or build it with the RVV backend. No per-board performance validation. github.com
• ncnn Partial Upstream has linux-riscv64 CI and RVV implementations; JH7110 has no RVV, so ncnn runs scalar-only. github.comgithub.com
• tengine Partial tengine-lite provides RVV convolution kernels for risc-v/lp64dv and an rv64-c906 toolchain; only some operators are optimised, so support is experimental. raw.githubusercontent.comgithub.com

NPU / acceleratorNo NPU (GPU/video codec only)

One of the most mature starter boards (ruyi offers Armbian/openEIC/OpenBSD images), but the SoC has no vector extension, so LLMs are "it runs, slowly".

• llama.cpp Works Sipeed's official wiki has a llama.cpp section: about 6s/token for a 7B model (without the V extension). C910 is RVV 0.7.1, so upstream's RVV 1.0 optimisation path does not apply and it stays scalar. raw.githubusercontent.comgithub.com
• onnxruntime Partial Upstream's riscv64 MLAS RVV kernel targets RVV 1.0; the TH1520's 0.7.1 vectors can't use it, so it falls back to scalar. raw.githubusercontent.com
• pytorch Partial RISE wheels need glibc >= 2.39 and are optimised for RVV 1.0; they install on RevyOS/Debian but give no vector acceleration. riseproject.dev
• opencv Works Works as usual; the RVV HAL has limited 0.7.1 support. github.com
• ncnn Partial Builds, but the RVV optimisations target 1.0 and degrade to scalar on 0.7.1. github.com
• tengine Partial tengine-lite ships an rv64-c906 toolchain and RVV convolution kernels built for 0.7.1 cores like the C906/C910, but only some operators are covered. github.com
• npu-sdk (HHB) Partial The official wiki documents the NPU flow: the T-Head HHB compiler turns ONNX into NPU executables (int8/int16 fixed point) inside a dedicated Docker image; the toolchain is closed source and fixed-point only, so it is unsuitable for LLMs. raw.githubusercontent.com

NPU / accelerator4 TOPS@INT8, vendor closed-source HHB toolchain only (int8/int16 fixed point), no upstream open-source driver

Running an LLM on the CPU works but is slow (~6s/token); the NPU only has the closed T-Head HHB path and cannot run Transformer LLMs.

• llama.cpp Partial Same TH1520 as the LicheePi 4A and officially supported by RevyOS; llama.cpp follows the same 0.7.1 vector/scalar path (about 6s/token measured on the LPi4A). No Meles-specific public benchmark found. milkv.ioraw.githubusercontent.comrevyos.github.io
• onnxruntime Partial Same TH1520 story: upstream's RVV kernel targets 1.0, so it degrades to scalar here. raw.githubusercontent.com
• pytorch Partial RISE wheels install (RevyOS is Debian-based), with no vector acceleration. riseproject.dev
• opencv Works Works as usual. github.com
• ncnn Partial Builds, with the RVV optimisations degrading on 0.7.1. github.com
• tengine Partial The rv64-c906 toolchain matches these cores, covering only some operators. github.com
• npu-sdk (HHB) Partial Milk-V's specs confirm a 4TOPS@INT8 NPU with TensorFlow/ONNX/Caffe support, but it goes through the closed T-Head toolchain with no open-source driver. milkv.io

NPU / accelerator4 TOPS@INT8 (TH1520), closed-source HHB toolchain

ruyi can flash RevyOS directly (with 4G/8G/16G uboot packages); the AI situation matches the LPi4A.

• llama.cpp Works Upstream llama.cpp has a dedicated build-riscv64-spacemit.md: A100 cores use RISCV64_SPACEMIT_IME2, and the measured qwen3 0.6B Q4_0 reaches pp128 565.8 / tg128 55.8 t/s while qwen3 4B Q4_0 reaches pp128 79.7 / tg128 11.3 t/s (8 threads). raw.githubusercontent.com
• spacemit-llama.cpp (vendor) Works SpacemiT ships a prebuilt spacemit-llama.cpp.riscv64 (now 0.0.9) with the model library on archive.spacemit.com; Bit-Brick's docs show llama-cli/llama-server usage. archive.spacemit.comdocs.bit-brick.com
• onnxruntime Works The RVV 1.0 cores fully use upstream's MLAS RVV kernel, but you must build it yourself. raw.githubusercontent.com
• pytorch Works glibc >= 2.39 distros such as Bianbu can install RISE's torch 2.13.0 riscv64 wheel directly (it is manylinux_2_39_riscv64). riseproject.devpypi.riseproject.dev
• opencv Works RVV 1.0 plus the RVV HAL — the best combination. github.com
• ncnn Works RVV 1.0, covered by upstream CI. github.comgithub.com
• tengine Partial Works, but the optimisations target the 0.7.1 C906 and give little on 1.0. github.com
• spacemit-ai-sdk Works archive.spacemit.com/spacemit-ai/ offers ModelZoo, llama.cpp, localai, dify and more — one of the most complete vendor AI stacks on RISC-V today. archive.spacemit.com

NPU / acceleratorK3 leans on CPU+AI cores (SpacemiT IME extensions) with no standalone general-purpose NPU programming model; acceleration goes through SpacemiT's proprietary instructions and AI SDK

Currently the most complete AI software stack among RISC-V SBCs: upstream llama.cpp ships its kernels directly.

• llama.cpp No public evidence No public llama.cpp benchmark on the A210; the C920 is RVV 0.7.1, so upstream's 1.0 optimisation path is unavailable. cnx-software.com
• onnxruntime No public evidence No public evidence for this board (the chip is new, with upstream support only in 2026). raw.githubusercontent.com
• pytorch No public evidence No public evidence for this board. riseproject.dev
• opencv Partial RVV 0.7.1, so scalar/limited vector only; no benchmarks for this board. github.com
• ncnn No public evidence No public evidence for this board. github.com
• tengine No public evidence No public evidence for this board. github.com
• torq-toolkit (vendor) Partial CNX reports: the official Linux SDK plus TORQ-Toolkit handle model conversion and deployment, the NPU supports INT4/INT8/INT16/FP8/FP16/BF16, and DeepSeek-7B is claimed at 8 tok/s; the toolchain is closed and its docs are not publicly available. cnx-software.com

NPU / accelerator12 TOPS@INT8 (supports FP16/BF16/FP8), via the vendor's closed-source TORQ-Toolkit

ruyi already offers Debian/buildroot images for zhihe-a210, but on the AI side there are only vendor claims and no publicly reproducible benchmark.

• llama.cpp No public evidence The K230 targets vision inference (KPU) and no public llama.cpp benchmark exists; with 512MB/1GB of RAM an LLM is unrealistic. kendryte.com
• onnxruntime No public evidence No public evidence (the vendor pushes nncase). kendryte.com
• pytorch No public evidence No public evidence; not applicable for training. kendryte.com
• opencv Works The CanMV/K230 SDK includes image processing and a camera pipeline, and OpenCV-style APIs work. kendryte.com
• ncnn No public evidence No public evidence. kendryte.com
• nncase + KPU Works kendryte/nncase is the official open-source deep-learning compiler that turns ONNX/TFLite into KPU kmodels; the K230 SDK docs cover the whole AI workflow. github.comkendryte.com

NPU / acceleratorKPU (about 1 TOPS class); kmodels are compiled with nncase, and the driver comes from the vendor SDK (not upstream)

A textbook "strong at vision AI, unsuited to LLMs" board: nncase is open source, but the runtime is still tied to the vendor SDK.

• llama.cpp No public evidence The P550 has no RVV and there is no public llama.cpp benchmark on the Megrez; the official AI path is the NPU, not the CPU. milkv.io
• onnxruntime No public evidence No public evidence. milkv.io
• pytorch No public evidence No public evidence. milkv.io
• opencv Partial No RVV, so scalar only; RockOS can install it but there is no public benchmark. github.com
• ncnn No public evidence No public evidence. github.com
• ennp-sdk (vendor) Works The ESWIN ENNP platform (closed source) provides EsQuant quantisation, EsAAC compilation and EsSimulator emulation; the official runtime sample gives Qwen2 and DeepSeek-R1 examples on the NPU (apt install es-sdk-sample-npu-qwen, model ZIFENG278/Qwen2-0.5B_ENNP). milkv.iomilkv.io

NPU / accelerator19.95 TOPS@INT8 (INT16/FP16) through the closed-source ENNP SDK, though official Qwen2/DeepSeek-R1 samples exist

Rare on RISC-V in that its NPU can run LLMs and the vendor ships samples; the cost is total lock-in to ESWIN's closed stack.

• llama.cpp No public evidence No public llama.cpp benchmark on the Titan; the board is new and ships in small numbers. milkv.io
• onnxruntime No public evidence No public evidence. milkv.io
• pytorch No public evidence No public evidence. milkv.io
• opencv Partial No RVV, so scalar only; no public benchmark. milkv.io
• ncnn No public evidence No public evidence. milkv.io

NPU / acceleratorNo onboard NPU; it is designed to take a GPU/AI accelerator over PCIe Gen4 x16

Positioned as a Mini-ITX workstation (dual DDR4 ECC up to 64GB, BMC), so AI needs a PCIe card — but there is no public evidence of a working GPU/NPU driver stack on RISC-V yet.

• llama.cpp Works arXiv 2503.17422 (V-Seek) optimises LLM inference on the SG2042: DeepSeek R1 Distill Llama 8B / QWEN 14B reach 4.32 / 2.29 token/s generation and 6.54 / 3.68 token/s prefill. Milk-V also has its own InferLLM docs. arxiv.orgmilkv.io
• onnxruntime Partial 64 cores run in parallel, but RVV 0.7.1 can't use upstream's 1.0 kernel. raw.githubusercontent.com
• pytorch Partial RISE wheels install, but there is no 0.7.1 vector acceleration. riseproject.dev
• opencv Works Multi-core scalar works. github.com
• ncnn Partial Builds, degrading to scalar on 0.7.1. github.com
• tengine Partial The rv64-c906 kernel targets 0.7.1, covering only some operators. github.com
• inferllm Works Milk-V's docs cover deploying InferLLM (MegEngine) on the Pioneer. milkv.io

NPU / acceleratorNo NPU (64-core CPU only)

The only RISC-V board with both large memory (up to 128GB) and 64 cores, so 70B-class models fit — but throughput is single-digit token/s.

• llama.cpp Works The X60 is the same core as the BPI-F3; upstream's build-riscv64-spacemit.md has an X60 results table (qwen3 0.6B Q4_0: pp128 49.15 / tg128 11.73 t/s). There is also an experimental llama.cpp repo for the Orange Pi RV2. raw.githubusercontent.comgithub.com
• spacemit-llama.cpp (vendor) Works SpacemiT's prebuilt packages work (same K1 silicon). archive.spacemit.com
• onnxruntime Works RVV 1.0, upstream kernel usable. raw.githubusercontent.com
• pytorch Partial glibc >= 2.39 (e.g. Ubuntu 24.04 / recent Armbian) can install RISE wheels; older images have too old a glibc. riseproject.dev
• opencv Works RVV 1.0 plus the RVV HAL. github.com
• ncnn Works RVV 1.0, covered by upstream CI. github.comgithub.com
• tengine Partial Works; the optimisations are not 1.0-specific. github.com

NPU / acceleratorNo standalone NPU (K1's AI comes from SpacemiT's proprietary vector/AI extensions)

ruyi provides an armbian-orangepi-rv2-minimal image; one of the best value boards that genuinely runs LLMs.

• llama.cpp Works Upstream llama.cpp's riscv64-spacemit docs give a measured X60 (K1) table and a quantisation support matrix (Q4_0/Q4_1/Q4_K/Q8_0 work); Bit-Brick's docs cover full usage on Bianbu. raw.githubusercontent.comdocs.bit-brick.com
• spacemit-llama.cpp (vendor) Works SpacemiT's prebuilt spacemit-llama.cpp.riscv64.0.0.4+ (0.0.9 on the archive) supports llama-cli / llama-server / an OpenAI-compatible API. archive.spacemit.comdocs.bit-brick.com
• onnxruntime Works RVV 1.0, upstream MLAS RVV kernel works (self-built). raw.githubusercontent.com
• pytorch Works glibc >= 2.39 distros such as Bianbu can install the RISE torch 2.13.0 riscv64 wheel (manylinux_2_39_riscv64). riseproject.devpypi.riseproject.dev
• opencv Works RVV 1.0, the OpenCV RVV HAL is active. github.com
• ncnn Works RVV 1.0, covered by upstream CI. github.comgithub.com
• tengine Partial Works, but not optimal. github.com
• spacemit-ai-sdk Works SpacemiT's AI artifact repository (ModelZoo/llama.cpp/localai/dify) is ready to use. archive.spacemit.com

NPU / acceleratorNo standalone NPU; AI runs on the X60's vector/AI extensions plus the proprietary AI SDK

The current poster board for "LLMs that just run": upstream llama.cpp, vendor prebuilt packages and Bianbu are all in place.

• llama.cpp Does not work With 64MB/256MB of RAM, no RVV and no FPU headroom, LLMs are not practically feasible; no public success report was found. milkv.io
• onnxruntime Does not work Not enough memory or compute; the official AI path is the TPU rather than a general runtime. milkv.io
• pytorch Does not work Not applicable. milkv.io
• opencv Partial Basic image processing works, but memory is limited and the official samples go through the TDL SDK. milkv.io
• ncnn No public evidence No public evidence (not enough memory to load common models). milkv.io
• cvitek-tpu / tdl-sdk Works Milk-V's docs: the Duo series SoCs include a CVITEK TPU and the TDL SDK is recommended; YOLOv5/v8/11/12, MobileNetV2, ResNet18 and other samples are provided. The TPU stack is closed source. milkv.io

NPU / acceleratorCVITEK TPU (about 0.5 TOPS) with the closed-source TDL SDK / cvitek toolchain

A classic MCU-class vision board: fine for small YOLO-style models, thoroughly unsuited to LLMs.