一个mORMot原生的AI扩展，适用于Free Pascal和Delphi 7到13——完全一样 编译器范围为 mORMot2。它涵盖模型上下文协议、LLM 客户端、嵌入， 向量搜索和RAG，并且它带来了自己的推理引擎，用纯Object Pascal： 运行时没有 Python，没有 llama.cpp，运行时没有外部 .so/[.dll。mORMot2 的 HTTP 栈，JSON， RTTI、SQLite和加密是基础，而非我们容忍的依赖。

状态，2026-10-19： MCP、LLM客户端、代理环路、RAG和原生向量存储 已经完成并正在使用。本地推理引擎从该推理引擎运行BGE-M3嵌入 GGUF 文件为 lembed，质量相同，速度为 x86_64（Linux / Windows）上的 1.6 到 2 倍; 它通过了所有套件、质量门槛和Linux x86_64的90分钟浸入测试， ，它是消费后端的推荐供应商。三种解码器家族 在ILlmClient后面跑——Qwen3，Qwen3-MoE（专家混合，Qwen3-30B-A3B） 和Qwen2.5——在FP32中，其强势输出与transformers相同（代表MoE） 在微小参考模型上;30B模型与CPU路径进行比对）：

mormot2-ai — the mormot.ai.* extension for mORMot2 (FPC · Delphi 7 … 13)

A mORMot-native AI extension for Free Pascal and Delphi 7 to 13 — the same compiler range as mORMot2. It covers the Model Context Protocol, LLM clients, embeddings, vector search and RAG, and it brings its own inference engine in plain Object Pascal: no Python, no llama.cpp, no foreign .so/.dll at run time. mORMot2’s HTTP stack, JSON, RTTI, SQLite and crypto are the foundation, not a dependency we tolerate.

Status, 2026-10-19: MCP, LLM clients, agent loop, RAG and the native vector store are done and in use. The native inference engine runs BGE-M3 embeddings from the same GGUF file as lembed, with the same quality and 1.6 to 2 times its speed on x86_64 (Linux / Windows); it passes all suites, its quality gate and a 90-minute soak test on Linux x86_64 too, and it is the recommended provider of the consuming backend. Three decoder families run behind ILlmClient - Qwen3, Qwen3-MoE (mixture of experts, Qwen3-30B-A3B) and Qwen2.5 - with greedy output in FP32 token-identical to transformers (for MoE on tiny reference models; the 30B model is checked against the CPU path): the agent loop calls tools with a model on this machine, greedy or sampled, with a prompt cache for the next turn. Weights come from safetensors or llama.cpp GGUF files (Q8_0, Q4_0, Q4_K_M, with the Q5_0 matrices llama.cpp writes for widths like Qwen2.5’s 896). The whole forward pass runs on the CPU (NEON, AVX2), on Vulkan (tested on an RTX 5090, an Adreno X1-85 and an M2 Pro through MoltenVK) and on CUDA. An RTX 5090 decodes Qwen3-0.6B at about 400 tokens/s (CUDA) and Qwen3-30B-A3B at about 97 (CUDA; Vulkan 90), choosing its experts on the GPU. Sliding-window attention and layers without rotary positions run on the CPU, Vulkan and CUDA (SmolLM3, so far without a chat template). gpt-oss-20b runs on the CPU from its published MXFP4 checkpoint or GGUF file (attention sinks, YaRN, biases, the clamped SwiGLU) and on Vulkan and CUDA with its MXFP4 experts (an RTX 5090 decodes it at about 125 tokens/s), and chats through ILlmClient in its Harmony format, tool calls included. A decoder also takes embedding rows instead of token ids and loads as one stack of a larger checkpoint - the talker and code predictor of Qwen3-TTS match PyTorch on the CPU, Vulkan and CUDA. Three test suites - about 7200 assertions on Windows x64, fewer where a feature is missing - green on every platform in the table below.

What it does