livekit/agents

在找JS/TS库吗？可以看看AgentsJS

什么是特工？

代理框架旨在构建实时、可编程的参与者 运行在服务器上。用它来创建对话式、多模态语音 能够看见、听见并理解的特工。

特色

• 灵活集成：一个全面的生态系统，可根据您的使用场景灵活搭配合适的 STT、LLM、TTS 和实时 API。
• 集成作业调度：内置任务调度和分发，配备调度API，连接终端用户与代理。
• 广泛的WebRTC客户端：利用LiveKit的开源SDK生态系统构建客户端应用，支持所有主要平台。
• 电话集成：与LiveKit的电话协议栈无缝配合，允许您的座席拨打电话或接听电话。
• 与客户端交换数据：利用RPC及其他数据API与客户端无缝交换数据。
• 语义回合检测：使用变换器模型检测用户回合结束，有助于减少中断。
• MCP 支持：原生支持 MCP。将 MCP 服务器提供的工具与一行代码集成。

Looking for the JS/TS library? Check out AgentsJS

What is Agents?

The Agent Framework is designed for building realtime, programmable participants that run on servers. Use it to create conversational, multi-modal voice agents that can see, hear, and understand.

Features

• Flexible integrations: A comprehensive ecosystem to mix and match the right STT, LLM, TTS, and Realtime API to suit your use case.
• Integrated job scheduling: Built-in task scheduling and distribution with dispatch APIs to connect end users to agents.
• Extensive WebRTC clients: Build client applications using LiveKit’s open-source SDK ecosystem, supporting all major platforms.
• Telephony integration: Works seamlessly with LiveKit’s telephony stack, allowing your agent to make calls to or receive calls from phones.
• Exchange data with clients: Use RPCs and other Data APIs to seamlessly exchange data with clients.
• Semantic turn detection: Uses a transformer model to detect when a user is done with their turn, helps to reduce interruptions.
• MCP support: Native support for MCP. Integrate tools provided by MCP servers with one line of code.
• Builtin test framework: Write tests and use judges to ensure your agent is performing as expected.
• Open-source: Fully open-source, allowing you to run the entire stack on your own servers, including LiveKit server, one of the most widely used WebRTC media servers.

Installation

To install the core Agents library, along with plugins for popular model providers: