Compare commits

..
2 Commits
Author SHA1 Message Date
openbot 9ce0ba0e13 feat(channels): 媒体入站链路接入,出站骨架就位(未接线)+ 空内容判定修复
媒体功能是上一轮智能体自改代码时留下的 WIP(那轮步数耗尽被中断)。
本次把手头完成度较高的部分整理到可提交状态,并修掉其中两处判定 bug。

入站(已可用):
- 新增 flows/media_input.py:附件先落盘再进上下文,小图内联为 base64,
  大图与音视频退化为「路径 + 元信息」,附件级失败只降级为说明而不打断整轮
- processor 改用 prepare_inbound,文本 / 多模态输入据此决定发纯文本还是
  content blocks

出站(骨架就位,尚未接线):
- 新增 agent/outbox.py、channels/media.py(大小与类型校验、限长下载 / 读取)
- 新增 send_image / send_file 工具、ChannelServices.send_content、
  MessageEvent.content_event / media_event
- 目前还没有任何调用方创建 MediaOutbox,因此这两个工具一律返回
  「当前不在渠道会话中」;渠道侧的 _send_media 由各平台子类实现

修复:
- prepare_inbound 不再塞「(无内容)」占位块:占位块会让 has_content 恒真,
  空消息也能一路跑到 run_task,白烧一次模型调用
- 新增 PreparedInbound.is_multimodal 取代 len(blocks) > 1 的猜测:
  「只有一张图、没有正文」时 blocks 只有 1 个,原逻辑会把附件丢掉
- 附件全部接收失败时回「附件未能接收:<原因>」,不再误报「空消息」
- 顺手清掉 ruff 告警(未用导入、__all__ 未排序、嵌套 if / with)

测试:test_processor 拆成三个用例(空消息 / 附件全失败 / 仅附件必须发
blocks,最后一条是防回潮的回归测试),全量 243 passed,ruff 全绿。
2026-11-09 21:33:50 +08:00
openbot 5da55f8d17 fix(flows): 步数预算与图硬上限分离,预算耗尽时优雅收尾
max_steps 以前同时充当软预算和 recursion_limit,于是预算耗尽的那一刻
必然触发 GraphRecursionError:模型连一句收尾都来不及说,用户只收到
「处理消息时出错,请稍后再试」,已产出的内容也不会落库。

- 新增 StepBudgetMiddleware:按模型调用轮数计账(agent_rounds =
  max_steps // 2),剩余 <=4 轮时注入一次「立刻停止调用工具、给出结论」
  的收尾提示,只提醒一次
- 新增 STEP_GRACE(20),recursion_limit = max_steps + STEP_GRACE,把余量
  留给收尾动作
- GraphRecursionError 统一转成 StepBudgetExceeded,ainvoke 与流式两条
  路径都覆盖
- AgentReply 把「步数用尽」渲染成可操作说明,接在已产出内容之后一并
  下发并落库;用 _reported 标记让 on_error 不再补发通用错误文案,任务
  状态仍如实记 failed
- 同步 README / docs/architecture.md / db/schema.py 中 max_steps 的语义

测试:新增 tests/test_budget.py(7 个用例),全量 241 passed。
2026-11-09 21:32:29 +08:00
20 changed files with 1717 additions and 62 deletions

No files matched your search

+5 -3
View File
@@ -154,8 +154,10 @@ bot serve --channel wecom
规则: 规则:
- 未设置 `BOT042_MODEL` 时,使用 `models` 表中 `is_default=1` 的模型; - 未设置 `BOT042_MODEL` 时,使用 `models` 表中 `is_default=1` 的模型;
- 单次任务步数上限同理:`settings.agent.max_steps`(默认 200),`BOT042_MAX_STEPS` 优先; - 单次任务步数预算同理:`settings.agent.max_steps`(默认 200),`BOT042_MAX_STEPS` 优先;
联网检索类任务步数消耗快,触顶会抛 `GraphRecursionError`,表现为「处理消息时出错」; 预算将尽时智能体先收到一条「立刻收尾」提示,把已完成的部分总结出来;交给 LangGraph
的硬上限是 `max_steps + 20` 个图步,真撞上时用户看到的是「本轮执行步数用尽(上限 N 步)」
加已产出的内容,而不是笼统的「处理消息时出错」;
- `OPENAI_API_KEY` / `OPENAI_BASE_URL` 为空时,从该模型所属 `providers` 行补齐; - `OPENAI_API_KEY` / `OPENAI_BASE_URL` 为空时,从该模型所属 `providers` 行补齐;
- 环境变量指定的模型与库中默认模型**不一致**时,不会套用库里的密钥 - 环境变量指定的模型与库中默认模型**不一致**时,不会套用库里的密钥
(错用别的 key 只会得到 401); (错用别的 key 只会得到 401);
@@ -169,7 +171,7 @@ bot serve --channel wecom
| `BOT042_MODEL` | langchain 模型标识(`provider:model`) | `openai:gpt-4o-mini` | | `BOT042_MODEL` | langchain 模型标识(`provider:model`) | `openai:gpt-4o-mini` |
| `OPENAI_API_KEY` | OpenAI 兼容 API Key | — | | `OPENAI_API_KEY` | OpenAI 兼容 API Key | — |
| `OPENAI_BASE_URL` | 兼容网关地址(DeepSeek / Kimi 等均可) | 官方地址 | | `OPENAI_BASE_URL` | 兼容网关地址(DeepSeek / Kimi 等均可) | 官方地址 |
| `BOT042_MAX_STEPS` | 单次任务最大步数(recursion_limit) | `200` | | `BOT042_MAX_STEPS` | 单次任务步数预算(软上限,硬上限为其 +20) | `200` |
| `BOT042_LOG_LEVEL` | 日志级别 | `INFO` | | `BOT042_LOG_LEVEL` | 日志级别 | `INFO` |
使用 DeepSeek 等 OpenAI 兼容服务示例: 使用 DeepSeek 等 OpenAI 兼容服务示例:
+13 -4
View File
@@ -457,13 +457,22 @@ langchain 从环境变量取凭据。
一次 `ainvoke`,但缓存由持有者(`ChannelServices`)决定生命周期,不做模块级单例, 一次 `ainvoke`,但缓存由持有者(`ChannelServices`)决定生命周期,不做模块级单例,
避免配置不同或测试替换 `create_agent` 时串味。 避免配置不同或测试替换 `create_agent` 时串味。
`run_task()` 把历史转成 `{"messages": [...]}`,以 `recursion_limit=settings.max_steps` 调用 `ainvoke`, `run_task()` 把历史转成 `{"messages": [...]}`,以 `recursion_limit=settings.recursion_limit` 调用 `ainvoke`,
返回最终 state;`last_message_text()` 负责从多模态 content 中只取文本块(按行拼接)。 返回最终 state;`last_message_text()` 负责从多模态 content 中只取文本块(按行拼接)。
`max_steps` 默认 200,优先级 **环境变量 `BOT042_MAX_STEPS` > `settings.agent.max_steps` > 默认**: `max_steps` 默认 200,优先级 **环境变量 `BOT042_MAX_STEPS` > `settings.agent.max_steps` > 默认**:
联网检索类任务一轮要连续消耗十几个图步数(搜索 → 抓取 → 再搜索 → 汇总),上限过低时 联网检索类任务一轮要连续消耗十几个图步数(搜索 → 抓取 → 再搜索 → 汇总)。它是**软预算**,
`GraphRecursionError` 会被渠道兜底成用户看到的「处理消息时出错,请稍后再试。」, 不是 `recursion_limit` 本身——
所以它和模型配置一起由 `resolve_model_settings()` 解析进 `Settings`。
- `StepBudgetMiddleware`(`agent/middlewares/budget.py`)按模型调用轮数计账(`agent_rounds = max_steps // 2`,
一次「模型 → 工具」往返约耗 2 个图步),剩余 ≤4 轮时注入一条「立刻停止调用工具、给出结论」
的提示,且只提醒一次;
- 交给 LangGraph 的硬上限是 `recursion_limit = max_steps + STEP_GRACE(20)`,留给收尾动作;
- 真撞上硬上限(模型不听话、或子智能体的步数不在计数内)时,`GraphRecursionError` 会被转成
`StepBudgetExceeded`;`AgentReply` 把它渲染成「本轮执行步数用尽(上限 N 步)」+ 已产出的部分内容,
一并落库为 assistant 消息,并标记 `_reported` 让 `on_error` 不再补发通用错误文案。
两者以前是同值,于是预算耗尽的那一刻必然抛异常,模型连一句话都来不及说——这是 v0.1 的 bug。
## 7. 存储层(`db/`) ## 7. 存储层(`db/`)
+4 -2
View File
@@ -11,6 +11,7 @@ from deepagents.backends import LocalShellBackend
from langchain.agents.middleware.types import AgentMiddleware from langchain.agents.middleware.types import AgentMiddleware
from langgraph.graph.state import CompiledStateGraph from langgraph.graph.state import CompiledStateGraph
from bot042.agent.middlewares.budget import StepBudgetMiddleware
from bot042.agent.middlewares.steering import SteeringMiddleware from bot042.agent.middlewares.steering import SteeringMiddleware
from bot042.agent.prompts import SYSTEM_PROMPT from bot042.agent.prompts import SYSTEM_PROMPT
from bot042.agent.tools import get_tools from bot042.agent.tools import get_tools
@@ -90,12 +91,13 @@ def create_agent(
"openai:gpt-4o-mini"、"openai:deepseek-chat"(配合 OPENAI_BASE_URL)等。 "openai:gpt-4o-mini"、"openai:deepseek-chat"(配合 OPENAI_BASE_URL)等。
""" """
settings = settings or Settings.from_env() settings = settings or Settings.from_env()
# SteeringMiddleware 放最前:每轮模型调用前都先消化插话 # SteeringMiddleware 放最前:每轮模型调用前都先消化插话;
# StepBudgetMiddleware 紧随其后:预算将尽时注入收尾提示,避免撞硬上限
return create_deep_agent( return create_deep_agent(
model=_build_model(settings), model=_build_model(settings),
tools=get_tools(), tools=get_tools(),
system_prompt=SYSTEM_PROMPT, system_prompt=SYSTEM_PROMPT,
backend=_build_backend(settings), backend=_build_backend(settings),
middleware=[SteeringMiddleware(), *middleware], middleware=[SteeringMiddleware(), StepBudgetMiddleware(), *middleware],
interrupt_on=interrupt_on, interrupt_on=interrupt_on,
) )
+126
View File
@@ -0,0 +1,126 @@
"""步数预算:在硬上限之前提醒模型收尾。
LangGraph 的 ``recursion_limit`` 是**硬中断**——撞上就抛
``GraphRecursionError``,模型没有任何机会整理已做的工作、给出结论。所以这里
在触到硬上限之前先注入一条收尾提示,让模型自己把半成品收口:说清做到哪一步、
结论是什么、还差什么。
预算按**模型调用轮数**计(一个「模型 → 工具」往返约耗 2 个图步),轮数预算由
:attr:`bot042.config.Settings.agent_rounds` 从 ``max_steps`` 换算而来;硬上限
则是 :attr:`bot042.config.Settings.recursion_limit`,比预算多留
:data:`bot042.config.STEP_GRACE` 步给收尾动作。
与插话一样用 :mod:`contextvars` 绑定:``AgentCache`` 让同一张图被多个会话复用,
轮数记在实例属性上会串台。
限制:中间件只数**主智能体**的模型调用;子智能体(``task`` 工具派发的)消耗的
步数不在计数内。所以硬上限不能省——它是唯一能兜住子智能体跑飞的东西。
"""
from __future__ import annotations
import contextvars
from dataclasses import dataclass
from typing import Any
from langchain.agents.middleware.types import AgentMiddleware
from langchain_core.messages import HumanMessage
__all__ = [
"BUDGET_PREFIX",
"BUDGET_PROMPT",
"BUDGET_WARN_MARGIN",
"StepBudget",
"StepBudgetExceeded",
"StepBudgetMiddleware",
"bind",
"current_budget",
"unbind",
]
BUDGET_WARN_MARGIN = 4
"""剩余轮数 ≤ 该值时开始提醒收尾(留够 4 轮写下结论)。"""
BUDGET_PREFIX = "[系统 · 步数预算] "
"""收尾提示的前缀,让模型能识别这不是用户的新指令。"""
BUDGET_PROMPT = (
"本次任务的执行预算即将用尽(剩余约 {remaining} 轮模型调用)。"
"请立刻停止调用工具,基于已经完成的工作给出最终答复:"
"说明做到了哪一步、结论是什么、还差什么没做。不要再发起新的探索或修改。"
)
class StepBudgetExceeded(RuntimeError):
"""图步数硬上限(``recursion_limit``)被耗尽。
这是**兜底**路径:正常情况下预算中间件会先提醒模型收尾;走到这里说明模型
没有遵守提醒,或子智能体吃掉了大量步数。
"""
def __init__(self, max_steps: int) -> None:
self.max_steps = max_steps
super().__init__(f"任务步数用尽(max_steps={max_steps})")
@dataclass
class StepBudget:
"""一次运行的轮数预算(绑定单个 asyncio 任务,不加锁)。"""
rounds: int
"""允许的模型调用轮数。"""
used: int = 0
"""已发生的模型调用次数。"""
warned: bool = False
"""是否已经注入过收尾提示(只提醒一次,避免反复打断)。"""
@property
def remaining(self) -> int:
"""剩余轮数(不为负)。"""
return max(0, self.rounds - self.used)
def take(self) -> int:
"""登记一次模型调用,返回登记后的剩余轮数。"""
self.used += 1
return self.remaining
_current: contextvars.ContextVar[StepBudget | None] = contextvars.ContextVar(
"bot042_step_budget", default=None
)
def current_budget() -> StepBudget | None:
"""当前 asyncio 上下文的步数预算(无运行中任务时为 ``None``)。"""
return _current.get()
def bind(budget: StepBudget) -> contextvars.Token[StepBudget | None]:
"""把预算绑定到当前上下文,返回用于还原的 token。"""
return _current.set(budget)
def unbind(token: contextvars.Token[StepBudget | None]) -> None:
"""还原上下文绑定。"""
_current.reset(token)
class StepBudgetMiddleware(AgentMiddleware):
"""在预算将尽时注入一次收尾提示。"""
async def abefore_model(self, state: Any, runtime: Any) -> dict[str, Any] | None:
_ = state, runtime
budget = _current.get()
if budget is None or budget.warned:
return None
remaining = budget.take()
if remaining > BUDGET_WARN_MARGIN:
return None
budget.warned = True
return {
"messages": [
HumanMessage(content=BUDGET_PREFIX + BUDGET_PROMPT.format(remaining=remaining))
]
}
+169
View File
@@ -0,0 +1,169 @@
"""出站媒体信箱:让智能体在回复过程中登记要发给用户的媒体。
问题:智能体跑在 langgraph 里,工具是被模型调用的普通函数,拿不到渠道路由
(``channel_id`` / ``subject_id`` / 被动回复帧)。而渠道那一层又只能拿到
"智能体跑完后的最终文本"。
做法与 :mod:`bot042.agent.middlewares.steering` 同构:用 ``contextvars`` 放一个
每轮一次的**信箱**,工具往里放分片,流程层在 ``run_task`` 之后取走并通过
``channel.send`` RPC 下发。用 contextvar 而不是模块级单例,是因为同一张图会被
多个会话复用,实例属性会串台。
安全边界(明确写出来):
- 大小与类型在**这里**和渠道层各校验一次,任一处拒绝都不会上传;
- 路径限制(``allowed_roots``)默认不启用:智能体本来就有宿主机 shell 权限,
文件路径白名单挡不住它读任何文件,给了只会是"看起来安全"。要收紧应该从
工具权限入手(``interrupt_on`` / 去掉 ``execute``),而不是在这里假装设防;
- 真正有效的是**大小上限**与**可执行扩展名拒绝**,它们限制了"一次误操作能造成
多大外泄"以及"会不会把可执行文件推到对方机器上"。
"""
from __future__ import annotations
import contextvars
from collections import deque
from dataclasses import dataclass, field
from pathlib import Path
from typing import Final
from bot042.channels.media import (
MediaLimits,
MediaRejected,
check_mime,
read_file_limited,
sanitize_filename,
)
from bot042.channels.models import (
ContentPart,
FileContent,
ImageContent,
)
__all__ = [
"MediaOutbox",
"Outbox",
"bind",
"current_outbox",
"make_file_part",
"make_image_part",
"unbind",
]
MAX_PENDING: Final[int] = 8
"""单轮回复最多登记多少个附件;超过的直接拒绝,避免模型刷屏式调用。"""
@dataclass
class MediaOutbox:
"""一轮回复期间登记的待发媒体(绑定单个 asyncio 上下文)。"""
limits: MediaLimits = field(default_factory=MediaLimits)
allowed_roots: tuple[Path, ...] | None = None
max_pending: int = MAX_PENDING
_items: deque[ContentPart] = field(default_factory=deque, repr=False)
def push(self, part: ContentPart) -> None:
"""登记一个分片;超过 ``max_pending`` 抛 :class:`MediaRejected`。"""
if len(self._items) >= self.max_pending:
raise MediaRejected(
"本轮附件数量超限",
f"最多 {self.max_pending} 个,已拒绝额外的 {part.type}",
)
self._items.append(part)
def drain(self) -> list[ContentPart]:
"""取走并清空全部已登记分片。"""
items = list(self._items)
self._items.clear()
return items
def __len__(self) -> int:
return len(self._items)
def __bool__(self) -> bool:
return bool(self._items)
Outbox = MediaOutbox
"""别名:与 ``MediaOutbox`` 同一类型,供调用方用更短的写法。"""
_current: contextvars.ContextVar[MediaOutbox | None] = contextvars.ContextVar(
"bot042_outbox", default=None
)
def current_outbox() -> MediaOutbox | None:
"""当前上下文的信箱(无运行中回复时为 ``None``)。"""
return _current.get()
def bind(outbox: MediaOutbox) -> contextvars.Token:
"""把信箱绑定到当前上下文,返回用于还原的 token。"""
return _current.set(outbox)
def unbind(token: contextvars.Token) -> None:
"""还原上下文绑定。"""
_current.reset(token)
def _resolve(path_text: str, allowed_roots: tuple[Path, ...] | None) -> Path:
"""解析并校验待发文件路径(存在性 + 可选根目录限制)。"""
raw = (path_text or "").strip()
if not raw:
raise MediaRejected("文件路径为空", "")
path = Path(raw).expanduser()
try:
resolved = path.resolve(strict=True)
except (OSError, RuntimeError) as exc:
raise MediaRejected("文件不存在或不可访问", f"{raw}: {exc}") from exc
if not resolved.is_file():
raise MediaRejected("不是普通文件", str(resolved))
if allowed_roots and not any(
resolved == root or root in resolved.parents
for root in (r.resolve() for r in allowed_roots)
):
raise MediaRejected("路径不在允许的目录内", str(resolved))
return resolved
def make_image_part(
path_text: str,
*,
limits: MediaLimits | None = None,
allowed_roots: tuple[Path, ...] | None = None,
caption: str = "",
) -> ImageContent:
"""把本地图片文件做成 :class:`ImageContent` 分片(校验大小与类型)。"""
limits = limits or MediaLimits()
path = _resolve(path_text, allowed_roots)
raw = read_file_limited(path, limits.image)
mime = check_mime(ImageContent(), "", raw[:32])
return ImageContent(
local_path=str(path),
size=len(raw),
mime_type=mime,
alt_text=caption,
)
def make_file_part(
path_text: str,
*,
limits: MediaLimits | None = None,
allowed_roots: tuple[Path, ...] | None = None,
filename: str = "",
) -> FileContent:
"""把本地文件做成 :class:`FileContent` 分片(校验大小与类型)。"""
limits = limits or MediaLimits()
path = _resolve(path_text, allowed_roots)
raw = read_file_limited(path, limits.file)
name = sanitize_filename(filename or path.name)
mime = check_mime(FileContent(filename=name), "", raw[:32])
return FileContent(
local_path=str(path),
filename=name,
size=len(raw),
mime_type=mime,
)
+60 -1
View File
@@ -8,7 +8,13 @@ from typing import Any
from langchain_core.tools import tool from langchain_core.tools import tool
from bot042.agent.execution import run_python from bot042.agent.execution import run_python
from bot042.agent.outbox import (
current_outbox,
make_file_part,
make_image_part,
)
from bot042.agent.web import get_web_tools from bot042.agent.web import get_web_tools
from bot042.channels.media import MediaRejected, format_size
_VALID_DIRECTIONS = ("north", "south", "east", "west") _VALID_DIRECTIONS = ("north", "south", "east", "west")
_VALID_DIRECTIONS_CN = {"北": "north", "南": "south", "东": "east", "西": "west"} _VALID_DIRECTIONS_CN = {"北": "north", "南": "south", "东": "east", "西": "west"}
@@ -84,13 +90,66 @@ async def speak(text: str) -> str:
return f"[播报] {text}" return f"[播报] {text}"
@tool
async def send_image(path: str, caption: str = "") -> str:
"""把本地图片发给用户(作为原生图片消息,而不是文本链接)。
仅支持 png/jpeg/gif/webp/bmp,且不超过大小上限;超限或类型不符会被拒绝
并返回原因。同一轮回复里最多登记 8 个附件。
Args:
path: 本地图片文件路径(绝对路径或相对当前工作目录)。
caption: 附在图片上的说明,可留空。
"""
outbox = current_outbox()
if outbox is None:
return "[发送失败] 当前不在渠道会话中,无法发送图片"
try:
part = make_image_part(
path,
limits=outbox.limits,
allowed_roots=outbox.allowed_roots,
caption=caption,
)
outbox.push(part)
except MediaRejected as exc:
return f"[发送失败] {exc.reason}" + (f":{exc.detail}" if exc.detail else "")
return f"[已登记图片] {part.local_path}({format_size(part.size or 0)},将在本轮回复后发送)"
@tool
async def send_file(path: str, filename: str = "") -> str:
"""把本地文件发给用户(作为文件消息)。
可执行/脚本类扩展名(exe/bat/ps1/dll 等)会被拒绝。
Args:
path: 本地文件路径。
filename: 对方看到的文件名,留空则用原文件名。
"""
outbox = current_outbox()
if outbox is None:
return "[发送失败] 当前不在渠道会话中,无法发送文件"
try:
part = make_file_part(
path,
limits=outbox.limits,
allowed_roots=outbox.allowed_roots,
filename=filename,
)
outbox.push(part)
except MediaRejected as exc:
return f"[发送失败] {exc.reason}" + (f":{exc.detail}" if exc.detail else "")
return f"[已登记文件] {part.filename}({format_size(part.size or 0)},将在本轮回复后发送)"
def get_tools() -> list[Any]: def get_tools() -> list[Any]:
"""返回机器人工具集(世界操作 + 联网 + 本地执行)。 """返回机器人工具集(世界操作 + 联网 + 本地执行)。
shell 执行(``execute``)由 deepagents 的 FilesystemMiddleware 依据后端能力 shell 执行(``execute``)由 deepagents 的 FilesystemMiddleware 依据后端能力
自动挂载,不在这里重复注册。 自动挂载,不在这里重复注册。
""" """
return [perceive, move, speak, run_python, *get_web_tools()] return [perceive, move, speak, send_image, send_file, run_python, *get_web_tools()]
def reset_robot() -> None: def reset_robot() -> None:
+50 -11
View File
@@ -16,8 +16,6 @@
from __future__ import annotations from __future__ import annotations
import asyncio import asyncio
import base64
import binascii
import logging import logging
import mimetypes import mimetypes
import time import time
@@ -29,6 +27,14 @@ from typing import Any
import httpx import httpx
from bot042.channels.media import (
MediaLimits,
MediaRejected,
decode_base64_limited,
download_limited,
format_size,
read_file_limited,
)
from bot042.channels.models import ( from bot042.channels.models import (
ChannelConfig, ChannelConfig,
ChannelMessage, ChannelMessage,
@@ -141,6 +147,15 @@ class BaseChannel(ABC):
"""是否已配置消息处理器。""" """是否已配置消息处理器。"""
return self._processor is not None return self._processor is not None
@property
def media_limits(self) -> MediaLimits:
"""本渠道的媒体大小上限(子类可覆写以适配平台限制)。"""
limits = getattr(self, "_media_limits", None)
if limits is None:
limits = MediaLimits()
self._media_limits = limits
return limits
@property @property
def is_connected(self) -> bool: def is_connected(self) -> bool:
"""平台连接是否在线(供管理器刷新运行时状态)。""" """平台连接是否在线(供管理器刷新运行时状态)。"""
@@ -599,6 +614,9 @@ class BaseChannel(ABC):
# ========================================================================= # =========================================================================
# 媒体字节读取(data / local_path / url 三级优先) # 媒体字节读取(data / local_path / url 三级优先)
# ========================================================================= # =========================================================================
def media_download_headers(self) -> dict[str, str]:
"""下载入站媒体时附带的请求头(平台鉴权等);默认无。"""
return {}
async def fetch_remote_media(self, url: str) -> tuple[bytes, str]: async def fetch_remote_media(self, url: str) -> tuple[bytes, str]:
"""下载远端媒体并返回 ``(bytes, content_type)``;子类可覆写加平台鉴权。""" """下载远端媒体并返回 ``(bytes, content_type)``;子类可覆写加平台鉴权。"""
@@ -610,30 +628,51 @@ class BaseChannel(ABC):
) )
return resp.content, content_type return resp.content, content_type
async def load_media_bytes(self, part: ContentPart) -> tuple[bytes, str]: async def load_media_bytes(
self, part: ContentPart, *, limits: MediaLimits | None = None
) -> tuple[bytes, str]:
"""把媒体分片解析为 ``(bytes, content_type)``。 """把媒体分片解析为 ``(bytes, content_type)``。
解析顺序:``data``(base64 内联字节)→ ``local_path``(本地文件)→ 解析顺序:``data``(base64 内联字节)→ ``local_path``(本地文件)→
``url``(远端下载)。三者皆空时报 ValueError。 ``url``(远端下载)。三者皆空时报 ValueError。
**全程带大小闸**(:class:`~bot042.channels.media.MediaLimits`):
入站 URL 由平台给出、出站 ``local_path`` 由智能体给出,长度都不可信,
整份读进内存会把进程打爆。超限抛
:class:`~bot042.channels.media.MediaRejected`(``ValueError`` 子类)。
""" """
limits = limits or self.media_limits
if isinstance(part, TextContent): if isinstance(part, TextContent):
raise TypeError("load_media_bytes 不能用于文本分片") raise TypeError("load_media_bytes 不能用于文本分片")
limit = limits.for_part(part)
if part.data: if part.data:
mime = part.mime_type or "application/octet-stream" mime = part.mime_type or "application/octet-stream"
try: return decode_base64_limited(part.data, limit), mime
return base64.b64decode(part.data, validate=False), mime
except (binascii.Error, ValueError) as exc:
raise ValueError(f"ContentPart.data 不是合法 base64: {exc}") from exc
if part.local_path: if part.local_path:
path = Path(part.local_path) path = Path(part.local_path)
mime = part.mime_type or mimetypes.guess_type(str(path))[0] or "application/octet-stream" mime = part.mime_type or mimetypes.guess_type(str(path))[0] or "application/octet-stream"
return path.read_bytes(), mime return read_file_limited(path, limit), mime
if part.url: if part.url:
mime = part.mime_type mime = part.mime_type or mimetypes.guess_type(part.url)[0]
raw, guessed = await self.fetch_remote_media(part.url) try:
return raw, mime or guessed raw, guessed = await download_limited(
part.url,
limit=limit,
chunk=limits.chunk,
timeout=limits.timeout,
headers=self.media_download_headers(),
)
except httpx.HTTPError:
# 平台鉴权只能由子类覆写 fetch_remote_media 提供;这里退回它取字节
raw, guessed = await self.fetch_remote_media(part.url)
if len(raw) > limit:
raise MediaRejected(
"媒体超过大小上限",
f"{format_size(len(raw))} > {format_size(limit)}",
) from None
return raw, mime or guessed or "application/octet-stream"
raise ValueError(f"分片缺少 data/local_path/url: {part}") raise ValueError(f"分片缺少 data/local_path/url: {part}")
+503
View File
@@ -0,0 +1,503 @@
"""媒体安全策略:大小上限、MIME 校验、文件名净化与落盘隔离。
入站媒体在进入智能体上下文之前必须经过这一层,理由有三条:
1. **内存**:平台给的 URL 可能是任意大的文件,``resp.content`` 会把整份
内容读进内存。这里一律按块读取,超限立即中断;
2. **路径**:文件名来自平台,属于不可信输入,``../../.ssh/authorized_keys``
这类名字必须净化后才能拼进落盘路径;
3. **类型**:声明类型与实际字节可能不一致,需要按魔数嗅探交叉校验。
设计取舍(明确写出来,避免被当成完备防护):
- MIME 白名单是纵深防御,不是内容扫描:它只保证「不把明显不是图片的字节
当图片处理」,不做病毒扫描、不做转码、不做像素级校验;
- 可执行扩展名黑名单天然不完备(总会有新扩展名),只拦常见的那批;
- 落盘后不做权限收紧(沿用进程 umask),隔离靠目录边界而非 DACL。
"""
from __future__ import annotations
import base64
import binascii
import hashlib
import mimetypes
import re
import unicodedata
from dataclasses import dataclass
from pathlib import Path
from typing import Final
import httpx
from bot042.channels.models import (
AudioContent,
ContentPart,
FileContent,
ImageContent,
TextContent,
VideoContent,
)
__all__ = [
"ALLOWED_AUDIO_MIME",
"ALLOWED_IMAGE_MIME",
"ALLOWED_VIDEO_MIME",
"BLOCKED_EXTENSIONS",
"MediaLimits",
"MediaRejected",
"StoredMedia",
"check_mime",
"decode_base64_limited",
"download_limited",
"format_size",
"guess_mime",
"read_file_limited",
"safe_join",
"sanitize_filename",
"store_media",
]
MB: Final[int] = 1024 * 1024
# ---------------------------------------------------------------------------
# 类型白名单
# ---------------------------------------------------------------------------
ALLOWED_IMAGE_MIME: Final[frozenset[str]] = frozenset(
{"image/png", "image/jpeg", "image/gif", "image/webp", "image/bmp"}
)
ALLOWED_AUDIO_MIME: Final[frozenset[str]] = frozenset(
{
"audio/amr",
"audio/mpeg",
"audio/wav",
"audio/x-wav",
"audio/ogg",
"audio/mp4",
"audio/aac",
"audio/silk",
}
)
ALLOWED_VIDEO_MIME: Final[frozenset[str]] = frozenset(
{"video/mp4", "video/quicktime", "video/webm", "video/x-msvideo"}
)
BLOCKED_EXTENSIONS: Final[frozenset[str]] = frozenset(
{
".exe", ".bat", ".cmd", ".com", ".scr", ".pif", ".msi", ".dll",
".ps1", ".vbs", ".vbe", ".js", ".jse", ".wsf", ".hta", ".lnk",
".jar", ".sh", ".bash",
}
)
"""可执行/脚本类扩展名,入站与出站都拒绝。
这是**黑名单**,不完备;只拦常见的那批。真正的防线是「不自动执行内容」,
而不是这张表。
"""
_IMAGE_EXT: Final[dict[str, str]] = {
".png": "image/png",
".jpg": "image/jpeg",
".jpeg": "image/jpeg",
".gif": "image/gif",
".webp": "image/webp",
".bmp": "image/bmp",
}
_VIDEO_EXT: Final[dict[str, str]] = {
".mp4": "video/mp4",
".mov": "video/quicktime",
".webm": "video/webm",
".avi": "video/x-msvideo",
}
_AUDIO_EXT: Final[dict[str, str]] = {
".amr": "audio/amr",
".mp3": "audio/mpeg",
".wav": "audio/wav",
".ogg": "audio/ogg",
".m4a": "audio/mp4",
".aac": "audio/aac",
".silk": "audio/silk",
}
def guess_mime(filename: str) -> str | None:
"""按扩展名推断 MIME;未知返回 ``None``。"""
ext = Path(filename).suffix.lower()
for table in (_IMAGE_EXT, _VIDEO_EXT, _AUDIO_EXT):
if ext in table:
return table[ext]
guessed, _ = mimetypes.guess_type(filename)
return guessed
def sniff_mime(head: bytes) -> str | None:
"""按魔数嗅探 MIME;无法判定返回 ``None``。
只覆盖本项目实际会碰到的格式,不做通用实现。
"""
if head.startswith(b"\x89PNG\r\n\x1a\n"):
return "image/png"
if head.startswith(b"\xff\xd8\xff"):
return "image/jpeg"
if head[:6] in (b"GIF87a", b"GIF89a"):
return "image/gif"
if head.startswith(b"BM"):
return "image/bmp"
if head.startswith(b"RIFF") and len(head) >= 12:
form = head[8:12]
if form == b"WEBP":
return "image/webp"
if form == b"WAVE":
return "audio/wav"
if form == b"AVI ":
return "video/x-msvideo"
if head.startswith(b"#!AMR"):
return "audio/amr"
if head.startswith(b"OggS"):
return "audio/ogg"
if head.startswith(b"ID3") or head[:2] in (b"\xff\xfb", b"\xff\xf3"):
return "audio/mpeg"
if head.startswith(b"\x1aE\xdf\xa3"):
return "video/webm"
if len(head) >= 12 and head[4:8] == b"ftyp":
brand = head[8:12]
if brand.startswith(b"qt"):
return "video/quicktime"
return "video/mp4"
if head.startswith(b"#!SILK"):
return "audio/silk"
return None
def check_mime(part: ContentPart, declared: str, head: bytes = b"") -> str:
"""校验媒体类型是否可接受,返回最终采信的 MIME。
采信顺序:魔数嗅探 > 声明类型 > 扩展名推断。三者都没有时按
``application/octet-stream`` 处理——此时只有 :class:`FileContent`
放行,图片/音视频一律拒绝(无法确认它真的是媒体)。
Raises:
MediaRejected: 类型不在该类媒体允许的集合内,或扩展名被拉黑。
"""
sniffed = sniff_mime(head) if head else None
candidate = (sniffed or declared or "").split(";")[0].strip().lower()
if isinstance(part, FileContent):
name = part.filename or ""
ext = Path(name).suffix.lower()
if ext in BLOCKED_EXTENSIONS:
raise MediaRejected("文件类型被拒绝", f"可执行/脚本扩展名:{ext}")
return candidate or guess_mime(name) or "application/octet-stream"
if isinstance(part, ImageContent):
allowed, label = ALLOWED_IMAGE_MIME, "图片"
elif isinstance(part, AudioContent):
allowed, label = ALLOWED_AUDIO_MIME, "音频"
elif isinstance(part, VideoContent):
allowed, label = ALLOWED_VIDEO_MIME, "视频"
else: # pragma: no cover - ContentPart 联合已由类型系统约束
raise MediaRejected("不支持的分片类型", type(part).__name__)
if candidate in allowed:
return candidate
fallback = guess_mime(getattr(part, "filename", "") or "")
if fallback in allowed:
return fallback
raise MediaRejected(
f"{label}类型不被接受",
f"收到 {candidate or '未知'},允许:{sorted(allowed)}",
)
# ---------------------------------------------------------------------------
# 上限
# ---------------------------------------------------------------------------
def format_size(num: int) -> str:
"""人类可读的字节数(1 位小数)。"""
value = float(num)
for unit in ("B", "KB", "MB", "GB"):
if value < 1024 or unit == "GB":
return f"{value:.1f} {unit}" if unit != "B" else f"{int(value)} B"
value /= 1024
return f"{int(num)} B" # pragma: no cover
@dataclass(frozen=True)
class MediaLimits:
"""各类媒体的字节上限与下载参数。
默认值是按「单进程内存占用 + 交互体验」权衡设定的**本项目策略值**,
不是平台官方限制;部署时按实际内存调。
"""
image: int = 10 * MB
audio: int = 20 * MB
video: int = 50 * MB
file: int = 20 * MB
chunk: int = 64 * 1024
"""按块读取的块大小,决定下载时的内存粒度。"""
timeout: float = 30.0
"""单次远端下载的总超时(秒)。"""
def for_part(self, part: ContentPart) -> int:
"""返回该分片适用的字节上限。"""
if isinstance(part, ImageContent):
return self.image
if isinstance(part, AudioContent):
return self.audio
if isinstance(part, VideoContent):
return self.video
if isinstance(part, FileContent):
return self.file
raise TypeError(f"文本分片没有大小上限:{type(part).__name__}")
class MediaRejected(ValueError):
"""媒体被安全策略拒绝。
``reason`` 是给用户看的一句话,``detail`` 是给日志的技术细节。
调用方应把 ``player_reason`` 降级成文本提示,而不是向上抛异常打断整轮回复。
"""
def __init__(self, reason: str, detail: str = "") -> None:
self.reason = reason
self.detail = detail
super().__init__(f"{reason}({detail})" if detail else reason)
@property
def player_reason(self) -> str:
"""面向用户的简短说明。"""
return self.reason
# ---------------------------------------------------------------------------
# 受限读取
# ---------------------------------------------------------------------------
def read_file_limited(path: Path, limit: int) -> bytes:
"""读取本地文件,超过 ``limit`` 抛 :class:`MediaRejected`。
只多读 1 字节用于判定越界,不会把超大文件整个读进内存。
"""
if not path.exists():
raise MediaRejected("文件不存在", str(path))
if not path.is_file():
raise MediaRejected("不是普通文件", str(path))
with path.open("rb") as handle:
data = handle.read(limit + 1)
if len(data) > limit:
raise MediaRejected(
"媒体超过大小上限", f"大于 {format_size(limit)}(上限 {format_size(limit)})"
)
return data
def decode_base64_limited(data: str, limit: int) -> bytes:
"""解码 base64 内联字节,超过 ``limit`` 抛 :class:`MediaRejected`。
先按 4/3 膨胀率粗判长度,避免为一个必然超限的字符串分配大缓冲。
"""
if len(data) > (limit * 4) // 3 + 16:
raise MediaRejected("媒体超过大小上限", f"base64 长度 {len(data)}")
try:
raw = base64.b64decode(data, validate=False)
except (binascii.Error, ValueError) as exc:
raise MediaRejected("媒体 base64 解码失败", str(exc)) from exc
if len(raw) > limit:
raise MediaRejected(
"媒体超过大小上限", f"{format_size(len(raw))} > {format_size(limit)}"
)
return raw
async def download_limited(
url: str,
*,
limit: int,
chunk: int = 64 * 1024,
timeout: float = 30.0,
headers: dict[str, str] | None = None,
) -> tuple[bytes, str]:
"""流式下载远端媒体,``(bytes, content_type)``。
两道闸:``Content-Length`` 提前拒绝(省一次完整下载),以及按块累计的
硬上限(服务端谎报长度或不分块时仍然生效)。
Raises:
MediaRejected: 超限。
httpx.HTTPError: 网络/状态码错误,由调用方决定是否降级。
"""
async with httpx.AsyncClient(
follow_redirects=True, timeout=timeout, headers=headers
) as client, client.stream("GET", url) as resp:
resp.raise_for_status()
declared = resp.headers.get("content-length", "")
if declared.isdigit() and int(declared) > limit:
raise MediaRejected(
"媒体超过大小上限",
f"Content-Length {format_size(int(declared))} > {format_size(limit)}",
)
buffer = bytearray()
async for piece in resp.aiter_bytes(chunk):
buffer.extend(piece)
if len(buffer) > limit:
raise MediaRejected(
"媒体超过大小上限",
f"下载超过 {format_size(limit)} 已中断",
)
content_type = (resp.headers.get("content-type") or "").split(";")[0].strip()
return bytes(buffer), content_type or ""
# ---------------------------------------------------------------------------
# 文件名与落盘路径
# ---------------------------------------------------------------------------
_UNSAFE_CHARS = re.compile(r"[^\w.\- ]+", re.UNICODE)
_DOTS = re.compile(r"\.{2,}")
def sanitize_filename(name: str, *, default: str = "media.bin", max_len: int = 120) -> str:
"""把平台给的文件名净化成可安全落盘的单段名字。
处理顺序:NFKC 归一(同形字)→ 去掉路径成分 → 替换控制字符与分隔符 →
合并连续点(``..`` 穿越)→ 去首尾空白与点 → 截断。
返回的值**不含目录分隔符**,仍须经 :func:`safe_join` 拼接。
"""
raw = unicodedata.normalize("NFKC", name or "")
# 只取最后一段:同时覆盖 "/" 与 "\\"
raw = raw.replace("\\", "/").split("/")[-1]
raw = "".join(ch for ch in raw if ch.isprintable() and ch not in "\x00")
raw = _UNSAFE_CHARS.sub("_", raw)
raw = _DOTS.sub(".", raw).strip(" .")
if not raw:
return default
if len(raw) > max_len:
stem, dot, suffix = raw.rpartition(".")
if dot and len(suffix) <= 10:
keep = max(1, max_len - len(suffix) - 1)
raw = f"{stem[:keep]}.{suffix}"
else:
raw = raw[:max_len]
return raw or default
def safe_join(root: Path, filename: str) -> Path:
"""把单段文件名拼到 ``root`` 下,并确认解析结果仍在 ``root`` 内。
``root`` 不存在时按父目录解析——落盘前由调用方 ``mkdir``。
Raises:
MediaRejected: 路径逃出 ``root``。
"""
base = root.resolve()
name = sanitize_filename(filename)
candidate = (base / name).resolve()
if candidate != base and base not in candidate.parents:
raise MediaRejected("落盘路径非法", f"{name!r} 解析到 {candidate}")
if candidate == base:
raise MediaRejected("落盘路径非法", f"{name!r} 解析为目录本身")
return candidate
# ---------------------------------------------------------------------------
# 入站落盘
# ---------------------------------------------------------------------------
@dataclass(frozen=True)
class StoredMedia:
"""一次入站落盘的结果。"""
part: ContentPart
"""更新后的分片:``local_path`` 已填、``url``/``data`` 已清空。"""
path: Path
size: int
mime: str
sha256: str
def _default_name_for(part: ContentPart, mime: str) -> str:
"""没有文件名时按类型与 MIME 造一个。"""
ext = mimetypes.guess_extension(mime) if mime else None
if isinstance(part, ImageContent):
return f"image{ext or '.bin'}"
if isinstance(part, AudioContent):
return f"audio{ext or '.bin'}"
if isinstance(part, VideoContent):
return f"video{ext or '.bin'}"
if isinstance(part, FileContent) and part.filename:
return part.filename
return f"file{ext or '.bin'}"
async def store_media(
part: ContentPart,
*,
root: Path,
limits: MediaLimits | None = None,
session_key: str = "",
headers: dict[str, str] | None = None,
) -> StoredMedia:
"""把媒体分片落盘到 ``root/<session>/``,返回更新后的分片。
字节来源优先级与 ``BaseChannel.load_media_bytes`` 一致
(``data`` → ``local_path`` → ``url``),但每一步都带大小闸;
落盘后分片只保留 ``local_path``,让下游(智能体、出站回显)共用一份字节。
Raises:
MediaRejected: 超限、类型不符、路径非法。
httpx.HTTPError: 远端下载失败(调用方降级为文本提示)。
"""
limits = limits or MediaLimits()
if isinstance(part, TextContent):
raise TypeError("store_media 不能用于文本分片")
limit = limits.for_part(part)
mime_declared = (part.mime_type or "").split(";")[0].strip().lower()
name_hint = getattr(part, "filename", "") or ""
if part.data:
raw = decode_base64_limited(part.data, limit)
mime = check_mime(part, mime_declared or guess_mime(name_hint) or "", raw[:32])
elif part.local_path:
source = Path(part.local_path)
raw = read_file_limited(source, limit)
mime = check_mime(part, mime_declared or guess_mime(str(source)) or "", raw[:32])
name_hint = name_hint or source.name
elif part.url:
raw, fetched = await download_limited(
part.url,
limit=limit,
chunk=limits.chunk,
timeout=limits.timeout,
headers=headers,
)
mime = check_mime(part, mime_declared or fetched or guess_mime(name_hint) or "", raw[:32])
else:
raise MediaRejected("媒体缺少字节来源", "data/local_path/url 三者皆空")
folder = root
if session_key:
folder = root / sanitize_filename(session_key, default="session", max_len=60)
folder.mkdir(parents=True, exist_ok=True)
target = safe_join(folder, sanitize_filename(name_hint) if name_hint else _default_name_for(part, mime))
digest = hashlib.sha256(raw).hexdigest()
# 同名不同内容时加哈希前缀,避免后到的覆盖先到的(也避免穿越后仍撞名)
if target.exists():
target = safe_join(folder, f"{digest[:12]}-{target.name}")
# 写临时文件再原子改名:中途失败不会留下半份字节
temp = target.with_name(target.name + ".part")
temp.write_bytes(raw)
temp.replace(target)
updated = part.model_copy(update={"local_path": str(target), "url": "", "data": None,
"size": len(raw), "mime_type": mime})
return StoredMedia(part=updated, path=target, size=len(raw), mime=mime, sha256=digest)
+26
View File
@@ -268,6 +268,32 @@ class MessageEvent(BaseModel):
"""ERROR 事件:携带错误消息。""" """ERROR 事件:携带错误消息。"""
return cls(type=MessageEventType.ERROR, error=message) return cls(type=MessageEventType.ERROR, error=message)
@classmethod
def content_event(cls, parts: list[ContentPart], **kwargs: Any) -> MessageEvent:
"""MESSAGE 事件:一次投递任意分片组合(文本 + 图片/文件/音视频)。
出站管线(``BaseChannel._deliver_event``)已按「先文本、再逐个媒体」
下发媒体分片,这里只是让调用方能真正构造出带媒体的 MESSAGE 事件——
在此之前 :class:`MediaContent` 在 ``src/`` 内没有任何构造入口。
示例::
yield MessageEvent.content_event([
TextContent(text="这是你要的图"),
ImageContent(local_path="/tmp/a.png", mime_type="image/png"),
])
"""
return cls(type=MessageEventType.MESSAGE, content=list(parts), **kwargs)
@classmethod
def media_event(cls, media: MediaContent, caption: str = "", **kwargs: Any) -> MessageEvent:
"""MESSAGE 事件:单个媒体分片,可选配一句说明文本。"""
parts: list[ContentPart] = []
if caption:
parts.append(TextContent(text=caption))
parts.append(media)
return cls(type=MessageEventType.MESSAGE, content=parts, **kwargs)
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# 总线事件载荷 # 总线事件载荷
+27 -3
View File
@@ -8,6 +8,13 @@ from pathlib import Path
from bot042.home import db_path, resolve_home from bot042.home import db_path, resolve_home
STEP_GRACE = 20
"""图步数硬上限相对 ``max_steps`` 的余量。
留给「预算将尽」提示被遵守的空间:预算用尽与硬上限如果相等,撞上限的那一刻
直接抛 ``GraphRecursionError``,模型连一句话都来不及说。
"""
@dataclass(frozen=True) @dataclass(frozen=True)
class Settings: class Settings:
@@ -21,10 +28,13 @@ class Settings:
model: str = "openai:gpt-4o-mini" model: str = "openai:gpt-4o-mini"
max_steps: int = 200 max_steps: int = 200
"""单次任务的图步数上限(recursion_limit)。 """单次任务的**软**步数预算。
联网检索类任务一轮要跑很多次工具调用(搜索 → 抓取 → 再搜索),50 步很容易 这是给模型看的预算:预算将尽时中间件会注入一条「立刻收尾」提示
在给出答案前耗尽并抛 ``GraphRecursionError``,因此默认给到 200。 (见 :mod:`bot042.agent.middlewares.budget`),交给 LangGraph 的硬上限是
:attr:`recursion_limit`(比它多 :data:`STEP_GRACE` 步)。联网检索类任务一轮
要跑很多次工具调用(搜索 → 抓取 → 再搜索),50 步很容易在给出答案前耗尽,
因此默认给到 200。
""" """
log_level: str = "INFO" log_level: str = "INFO"
home: Path = field(default_factory=resolve_home) home: Path = field(default_factory=resolve_home)
@@ -40,6 +50,20 @@ class Settings:
"""home 目录下的 bot.db 路径。""" """home 目录下的 bot.db 路径。"""
return db_path(self.home) return db_path(self.home)
@property
def recursion_limit(self) -> int:
"""交给 LangGraph 的硬步数上限。
硬上限必须**大于**软预算:预算用尽时模型需要几步把结论写完,直接等值会
让收尾提示和中断同时到达,退化回「抛异常、无结论」。
"""
return self.max_steps + STEP_GRACE
@property
def agent_rounds(self) -> int:
"""模型调用轮数预算:一次「模型 → 工具」往返约耗 2 个图步。"""
return max(1, self.max_steps // 2)
@classmethod @classmethod
def from_env(cls) -> Settings: def from_env(cls) -> Settings:
return cls( return cls(
+1 -1
View File
@@ -100,7 +100,7 @@ _CHANNEL_SEEDS = [
_SETTING_SEEDS = [ _SETTING_SEEDS = [
("agent.default_model", "openai:gpt-4o-mini", "默认模型"), ("agent.default_model", "openai:gpt-4o-mini", "默认模型"),
("agent.max_steps", "200", "单次任务最大步数"), ("agent.max_steps", "200", "单次任务步数预算"),
("log.level", "INFO", "日志级别"), ("log.level", "INFO", "日志级别"),
] ]
+62 -11
View File
@@ -8,11 +8,23 @@ from dataclasses import dataclass, field
from typing import TYPE_CHECKING, Any from typing import TYPE_CHECKING, Any
from langchain_core.messages import AIMessageChunk from langchain_core.messages import AIMessageChunk
from langgraph.errors import GraphRecursionError
from bot042.agent import create_agent from bot042.agent import create_agent
from bot042.agent.middlewares.steering import SteeringBuffer, bind, unbind from bot042.agent.middlewares.budget import StepBudget, StepBudgetExceeded
from bot042.agent.middlewares.budget import bind as bind_budget
from bot042.agent.middlewares.budget import unbind as unbind_budget
from bot042.agent.middlewares.steering import SteeringBuffer
from bot042.agent.middlewares.steering import bind as bind_steering
from bot042.agent.middlewares.steering import unbind as unbind_steering
from bot042.agent.outbox import MediaOutbox
from bot042.agent.outbox import bind as bind_outbox
from bot042.agent.outbox import unbind as unbind_outbox
from bot042.config import Settings from bot042.config import Settings
TaskInput = str | list[dict[str, Any]]
"""任务输入:纯文本,或 langchain 多模态内容块列表(图片 base64 + 文本)。"""
if TYPE_CHECKING: if TYPE_CHECKING:
from langgraph.graph.state import CompiledStateGraph from langgraph.graph.state import CompiledStateGraph
@@ -37,6 +49,18 @@ def _content_text(content: object) -> str:
return "" return ""
def _preview(task: TaskInput) -> str:
"""日志用的任务摘要:多模态时只取文本块,避免把 base64 打进日志。"""
if isinstance(task, str):
return task
return _content_text(task)
def _run_config(settings: Settings) -> dict[str, Any]:
"""图调用配置:硬步数上限 = 软预算 + 余量(见 ``Settings.recursion_limit``)。"""
return {"recursion_limit": settings.recursion_limit}
def last_message_text(result: dict[str, Any]) -> str: def last_message_text(result: dict[str, Any]) -> str:
"""从智能体最终状态中提取最后一条消息的文本(多模态只取文本块)。""" """从智能体最终状态中提取最后一条消息的文本(多模态只取文本块)。"""
messages = result.get("messages") or [] messages = result.get("messages") or []
@@ -69,12 +93,13 @@ class AgentCache:
async def run_task( async def run_task(
task: str, task: TaskInput,
settings: Settings | None = None, settings: Settings | None = None,
history: list[dict[str, str]] | None = None, history: list[dict[str, str]] | None = None,
*, *,
cache: AgentCache | None = None, cache: AgentCache | None = None,
steering: SteeringBuffer | None = None, steering: SteeringBuffer | None = None,
outbox: MediaOutbox | None = None,
) -> dict[str, Any]: ) -> dict[str, Any]:
"""异步执行单个任务,返回智能体最终状态(含 messages)。 """异步执行单个任务,返回智能体最终状态(含 messages)。
@@ -84,22 +109,35 @@ async def run_task(
history: 历史消息(不含本轮)。 history: 历史消息(不含本轮)。
cache: 智能体缓存;给定则复用其中的实例,缺省每次新建(一次性调用用)。 cache: 智能体缓存;给定则复用其中的实例,缺省每次新建(一次性调用用)。
steering: 插话缓冲;给定则该次运行期间的插话会在下一轮模型调用前注入。 steering: 插话缓冲;给定则该次运行期间的插话会在下一轮模型调用前注入。
outbox: 出站媒体信箱;给定则本次运行中 ``send_image`` / ``send_file``
登记的附件留在其中,由调用方取走下发。
Raises:
StepBudgetExceeded: 图步数硬上限被耗尽(软预算的收尾提示没起作用)。
""" """
settings = settings or Settings.from_env() settings = settings or Settings.from_env()
agent = cache.get(settings) if cache is not None else create_agent(settings) agent = cache.get(settings) if cache is not None else create_agent(settings)
messages = list(history or []) messages = list(history or [])
messages.append({"role": "user", "content": task}) messages.append({"role": "user", "content": task})
logger.debug("执行任务:%s", task) logger.debug("执行任务:%s", _preview(task))
token = bind(steering) if steering is not None else None token = bind_steering(steering) if steering is not None else None
out_token = bind_outbox(outbox) if outbox is not None else None
budget_token = bind_budget(StepBudget(settings.agent_rounds))
try: try:
return await agent.ainvoke( return await agent.ainvoke(
{"messages": messages}, {"messages": messages},
config={"recursion_limit": settings.max_steps}, config=_run_config(settings),
) )
except GraphRecursionError as exc:
# 硬上限只是兜底:走到这里说明收尾提示没被遵守,或子智能体吃掉了步数
raise StepBudgetExceeded(settings.max_steps) from exc
finally: finally:
unbind_budget(budget_token)
if out_token is not None:
unbind_outbox(out_token)
if token is not None: if token is not None:
unbind(token) unbind_steering(token)
@dataclass @dataclass
@@ -111,12 +149,13 @@ class _Round:
async def stream_task( async def stream_task(
task: str, task: TaskInput,
settings: Settings | None = None, settings: Settings | None = None,
history: list[dict[str, str]] | None = None, history: list[dict[str, str]] | None = None,
*, *,
cache: AgentCache | None = None, cache: AgentCache | None = None,
steering: SteeringBuffer | None = None, steering: SteeringBuffer | None = None,
outbox: MediaOutbox | None = None,
) -> AsyncIterator[str]: ) -> AsyncIterator[str]:
"""流式执行任务,逐步产出**累积文本**(不是增量)。 """流式执行任务,逐步产出**累积文本**(不是增量)。
@@ -133,20 +172,26 @@ async def stream_task(
history: 历史消息(不含本轮)。 history: 历史消息(不含本轮)。
cache: 智能体缓存;给定则复用其中的实例。 cache: 智能体缓存;给定则复用其中的实例。
steering: 插话缓冲;给定则该次运行期间的插话会在下一轮模型调用前注入。 steering: 插话缓冲;给定则该次运行期间的插话会在下一轮模型调用前注入。
outbox: 出站媒体信箱;给定则登记其中(流式路径不重复发送)。
Raises:
StepBudgetExceeded: 图步数硬上限被耗尽;调用方可用已产出的部分文本拼提示。
""" """
settings = settings or Settings.from_env() settings = settings or Settings.from_env()
agent = cache.get(settings) if cache is not None else create_agent(settings) agent = cache.get(settings) if cache is not None else create_agent(settings)
messages = list(history or []) messages = list(history or [])
messages.append({"role": "user", "content": task}) messages.append({"role": "user", "content": task})
logger.debug("流式执行任务:%s", task) logger.debug("流式执行任务:%s", _preview(task))
token = bind(steering) if steering is not None else None token = bind_steering(steering) if steering is not None else None
out_token = bind_outbox(outbox) if outbox is not None else None
budget_token = bind_budget(StepBudget(settings.agent_rounds))
rounds: dict[str, _Round] = {} rounds: dict[str, _Round] = {}
order: list[str] = [] order: list[str] = []
try: try:
async for chunk, _meta in agent.astream( async for chunk, _meta in agent.astream(
{"messages": messages}, {"messages": messages},
config={"recursion_limit": settings.max_steps}, config=_run_config(settings),
stream_mode="messages", stream_mode="messages",
): ):
# messages 模式同时会吐出 ToolMessage(工具返回值)与 HumanMessage, # messages 模式同时会吐出 ToolMessage(工具返回值)与 HumanMessage,
@@ -166,6 +211,12 @@ async def stream_task(
visible = "".join(rounds[k].text for k in order if not rounds[k].tool) visible = "".join(rounds[k].text for k in order if not rounds[k].tool)
if visible: if visible:
yield visible yield visible
except GraphRecursionError as exc:
# 已产出的文本由调用方持有(累积式产出),这里只把异常换成可读的错误类型
raise StepBudgetExceeded(settings.max_steps) from exc
finally: finally:
unbind_budget(budget_token)
if out_token is not None:
unbind_outbox(out_token)
if token is not None: if token is not None:
unbind(token) unbind_steering(token)
+85 -15
View File
@@ -20,11 +20,14 @@ from dataclasses import dataclass, field
from pathlib import Path from pathlib import Path
from typing import Any from typing import Any
from bot042.agent.middlewares.budget import StepBudgetExceeded
from bot042.agent.middlewares.steering import SteeringBuffer from bot042.agent.middlewares.steering import SteeringBuffer
from bot042.channels.manager import ChannelManager from bot042.channels.manager import ChannelManager
from bot042.channels.models import ( from bot042.channels.models import (
ChannelMessage, ChannelMessage,
ContentPart,
InboundMessage, InboundMessage,
OutboundContent,
OutboundStream, OutboundStream,
OutboundText, OutboundText,
) )
@@ -62,10 +65,24 @@ ACK_TEXT = "收到,正在处理…"
ERROR_REPLY = "处理消息时出错,请稍后再试。" ERROR_REPLY = "处理消息时出错,请稍后再试。"
BUDGET_REPLY = (
"本轮执行步数用尽(上限 {steps} 步),我先停在这里。\n"
"要继续的话,把需求拆小一点重新发一条,或者直接说「接着做」,我从当前进度往下推。"
)
"""步数用尽时的说明(接在已完成的部分回答之后)。"""
REPLY_META_KEYS = ("_frame", "response_url", "chat_id", "_ws_client") REPLY_META_KEYS = ("_frame", "response_url", "chat_id", "_ws_client")
"""入站 metadata 中需要回传给渠道的回复上下文(IM 被动回复用)。""" """入站 metadata 中需要回传给渠道的回复上下文(IM 被动回复用)。"""
def _with_notice(answer: str, aborted: StepBudgetExceeded | None) -> str:
"""把「步数用尽」的说明接到部分回答后面;没有部分回答时只给说明。"""
if aborted is None:
return answer
notice = BUDGET_REPLY.format(steps=aborted.max_steps)
return f"{answer}\n\n{notice}" if answer.strip() else notice
def reply_metadata(message: InboundMessage) -> dict[str, Any]: def reply_metadata(message: InboundMessage) -> dict[str, Any]:
"""挑出入站消息里的回复上下文,随出站 RPC 回传给渠道。 """挑出入站消息里的回复上下文,随出站 RPC 回传给渠道。
@@ -129,6 +146,33 @@ class ChannelServices:
return return
await self.channels.send_text(channel_id, subject_id, text) await self.channels.send_text(channel_id, subject_id, text)
async def send_content(
self,
channel_id: str,
subject_id: str,
parts: list[ContentPart],
*,
metadata: dict[str, Any] | None = None,
) -> None:
"""发送富内容(文本 + 媒体);出站管线按分片顺序下发。
媒体分片自带 ``local_path``(入站落盘或智能体工具产出),
平台侧由 ``_send_media`` 上传后按原生消息投递。
"""
if self.hub is not None:
_ = await self.hub.call(
RPC_SEND,
OutboundContent(
channel_id=channel_id,
subject_id=subject_id,
parts=parts,
metadata=dict(metadata or {}),
),
timeout=self.send_timeout,
)
return
await self.channels.send_content(channel_id, subject_id, parts)
async def stream_text( async def stream_text(
self, self,
channel_id: str, channel_id: str,
@@ -205,6 +249,8 @@ class AgentReply(Task):
super().__init__(context) super().__init__(context)
self._steering = SteeringBuffer() self._steering = SteeringBuffer()
self._steered: list[str] = [] self._steered: list[str] = []
self._reported = False
"""是否已自行回话并落库(失败时 ``on_error`` 不再发通用文案)。"""
@property @property
def steering(self) -> SteeringBuffer: def steering(self) -> SteeringBuffer:
@@ -294,26 +340,36 @@ class AgentReply(Task):
) )
if stream is None: if stream is None:
result = await run_task( answer = ""
text, aborted: StepBudgetExceeded | None = None
services.settings, try:
history=history, result = await run_task(
cache=services.cache, text,
steering=self._steering, services.settings,
) history=history,
answer = last_message_text(result) cache=services.cache,
steering=self._steering,
)
answer = last_message_text(result)
except StepBudgetExceeded as exc:
aborted = exc
await self._persist_steered(services, session_id, channel) await self._persist_steered(services, session_id, channel)
if answer: reply = _with_notice(answer, aborted)
if reply:
await repo.add_message( await repo.add_message(
services.db, services.db,
session_id=session_id, session_id=session_id,
role="assistant", role="assistant",
content=answer, content=reply,
channel=channel, channel=channel,
) )
await services.send_text( await services.send_text(
payload.channel_id, payload.subject_id, answer or EMPTY_REPLY, metadata=meta payload.channel_id, payload.subject_id, reply or EMPTY_REPLY, metadata=meta
) )
if aborted is not None:
# 已回话并落库:任务仍记失败(步数用尽是真的没做完),但不再发通用错误
self._reported = True
raise aborted
return return
# 支持流式:先给回执,再边生成边覆盖同一条消息。 # 支持流式:先给回执,再边生成边覆盖同一条消息。
@@ -321,6 +377,7 @@ class AgentReply(Task):
await stream.update(ACK_TEXT, force=True) await stream.update(ACK_TEXT, force=True)
answer = "" answer = ""
first = True first = True
aborted = None
try: try:
async for partial in stream_task( async for partial in stream_task(
text, text,
@@ -333,20 +390,33 @@ class AgentReply(Task):
# 首段跳过节流:让用户在 ack 之后立刻看到内容开始出现 # 首段跳过节流:让用户在 ack 之后立刻看到内容开始出现
await stream.update(partial, force=first) await stream.update(partial, force=first)
first = False first = False
except StepBudgetExceeded as exc:
aborted = exc
finally: finally:
await stream.finish(answer or EMPTY_REPLY) # 步数用尽时把说明补在已经流出的内容后面,不必再单独发一条
final = _with_notice(answer, aborted)
await stream.finish(final or EMPTY_REPLY)
await self._persist_steered(services, session_id, channel) await self._persist_steered(services, session_id, channel)
if answer: if final:
await repo.add_message( await repo.add_message(
services.db, services.db,
session_id=session_id, session_id=session_id,
role="assistant", role="assistant",
content=answer, content=final,
channel=channel, channel=channel,
) )
if aborted is not None:
self._reported = True
raise aborted
async def on_error(self, exc: BaseException) -> None: async def on_error(self, exc: BaseException) -> None:
"""失败时尽量回一句可读提示(发送本身失败只记日志)。""" """失败时尽量回一句可读提示(发送本身失败只记日志)。
已经自行回话并落库的失败(如步数用尽)跳过——否则用户会先收到准确说明,
再收到一句没信息量的「处理消息时出错」。
"""
if self._reported:
return
services = self.deps services = self.deps
payload = self.context.payload payload = self.context.payload
if not (isinstance(services, ChannelServices) and isinstance(payload, ChannelMessage)): if not (isinstance(services, ChannelServices) and isinstance(payload, ChannelMessage)):
+251
View File
@@ -0,0 +1,251 @@
"""入站媒体 → 智能体上下文:落盘、降级与多模态内容块。
这是 issue #1 断点 1 的实现层:媒体消息不再被"暂仅支持文本"挡在门外,
而是按类型分三条路进入上下文:
==================== ==========================================================
类型 进入上下文的方式
==================== ==========================================================
图片(小) ``image_url`` 块(data URI,base64 内联)
图片(大/超量) 文本说明 + 落盘路径,模型可用文件工具自行读取
文本类文件 抽取的正文(有字符上限),进 ``text`` 块
音视频 / 二进制文件 元信息(类型、大小、时长)+ 落盘路径
==================== ==========================================================
**为什么不一概内联**:base64 进上下文会按 ~1.33 倍膨胀,一张 10MB 的图足以
吃光窗口;因此内联有独立的字节与数量上限(:class:`InlinePolicy`),超过就
退化成"路径 + 元信息",把"读不读、怎么读"交给模型按需处理。
**失败一律降级,不上抛**:单个附件超限/类型拒绝/下载失败只记录一条
``notes``,其余附件与该消息的文本照常进入上下文。用户看到的是部分可用,
而不是整轮报错。
"""
from __future__ import annotations
import base64
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any
import httpx
from bot042.channels.media import (
MB,
MediaLimits,
MediaRejected,
format_size,
store_media,
)
from bot042.channels.models import (
AudioContent,
ContentPart,
FileContent,
ImageContent,
InboundMessage,
TextContent,
VideoContent,
)
__all__ = ["InlinePolicy", "PreparedInbound", "prepare_inbound"]
_TEXT_MIME_PREFIXES = ("text/",)
_TEXT_EXTENSIONS = frozenset(
{
".txt", ".md", ".csv", ".tsv", ".json", ".yaml", ".yml", ".toml", ".ini",
".log", ".py", ".js", ".ts", ".html", ".htm", ".xml", ".sql", ".sh",
".cfg", ".conf", ".rst", ".tex",
}
)
@dataclass(frozen=True)
class InlinePolicy:
"""图片内联进上下文的策略。
``max_bytes`` 与 ``max_count`` 是**本项目策略值**,用于换取上下文可用性,
不是平台限制。超出的图片退化为路径说明,功能不丢失。
"""
max_bytes: int = 2 * MB
max_count: int = 4
max_text_chars: int = 20000
"""文本类文件抽取的字符上限(超出截断并注明)。"""
@dataclass
class PreparedInbound:
"""入站消息的智能体可用形态。"""
text: str
"""纯文本部分(所有 ``TextContent`` 拼接)。"""
blocks: list[dict[str, Any]] = field(default_factory=list)
"""发给模型的内容块(langchain 多模态格式)。仅文本时为一个 text 块;
文本与附件都没有时为空列表(不留占位块)。"""
summary: str = ""
"""用于落库的文本(正文 + 附件清单),不含字节。"""
notes: list[str] = field(default_factory=list)
"""降级说明(超限 / 类型拒绝 / 下载失败),会并入 summary。"""
stored: list[Path] = field(default_factory=list)
@property
def has_content(self) -> bool:
"""是否拿到了任何可进上下文的内容(文本或附件)。
**不要**为了"总是有个块"而塞占位文本:占位块会让本属性恒为真,空消息
就再也走不到拒绝分支,用户发个空白消息会白白烧掉一轮模型调用。
"""
return bool(self.text.strip()) or bool(self.blocks) or bool(self.stored)
@property
def is_multimodal(self) -> bool:
"""除正文块之外还有别的内容块(即确实带了附件)。
用来决定交给模型的是内容块列表还是纯文本:``len(blocks) > 1`` 这种判断
在"只有一张图、没有正文"时会退化成纯文本,把附件整个丢掉。
"""
body_blocks = 1 if self.text else 0
return len(self.blocks) > body_blocks
def _is_text_file(part: FileContent) -> bool:
mime = (part.mime_type or "").lower()
if any(mime.startswith(p) for p in _TEXT_MIME_PREFIXES):
return True
return Path(part.filename or "").suffix.lower() in _TEXT_EXTENSIONS
def _decode_text(raw: bytes, limit: int) -> str | None:
"""尝试把字节当 UTF-8 文本解码;含 NUL 或解码失败时返回 ``None``。"""
head = raw[:limit]
if b"\x00" in head:
return None
try:
return head.decode("utf-8")
except UnicodeDecodeError:
try:
return head.decode("gb18030")
except UnicodeDecodeError:
return None
def _media_label(part: ContentPart) -> str:
if isinstance(part, ImageContent):
return "图片"
if isinstance(part, AudioContent):
return "音频"
if isinstance(part, VideoContent):
return "视频"
if isinstance(part, FileContent):
return f"文件 {part.filename or ''}".strip()
return "媒体"
def _describe(part: ContentPart, size: int, mime: str, path: Path) -> str:
"""生成给模型看的附件说明行(不含字节)。"""
bits = [f"{_media_label(part)}", f"{format_size(size)}", mime or "未知类型"]
duration = getattr(part, "duration", None)
if isinstance(duration, int) and duration > 0:
bits.append(f"{duration / 1000:.1f}s")
return f"{' / '.join(bits)},已保存到 {path}"
async def prepare_inbound(
message: InboundMessage,
*,
root: Path,
limits: MediaLimits | None = None,
policy: InlinePolicy | None = None,
headers: dict[str, str] | None = None,
) -> PreparedInbound:
"""把入站消息整理成 ``(text, blocks, summary, notes)``,媒体先落盘。
Args:
message: 入站消息。
root: 媒体落盘根目录(按会话建子目录)。
limits: 字节上限;缺省 :class:`MediaLimits`。
policy: 图片内联策略;缺省 :class:`InlinePolicy`。
headers: 下载远端媒体时附带的请求头(平台鉴权由渠道提供)。
Returns:
:class:`PreparedInbound`。附件层面的失败只进 ``notes``,不抛异常。
"""
limits = limits or MediaLimits()
policy = policy or InlinePolicy()
text = message.text.strip()
prepared = PreparedInbound(text=text)
blocks: list[dict[str, Any]] = []
descriptions: list[str] = []
inline_images = 0
session_key = message.db_session_id
for part in message.content:
if isinstance(part, TextContent):
continue
try:
stored = await store_media(
part,
root=root,
limits=limits,
session_key=session_key,
headers=headers,
)
except MediaRejected as exc:
prepared.notes.append(f"{_media_label(part)}未接收:{exc.reason}")
continue
except (httpx.HTTPError, OSError) as exc:
prepared.notes.append(
f"{_media_label(part)}未接收:下载失败({type(exc).__name__})"
)
continue
prepared.stored.append(stored.path)
description = _describe(part, stored.size, stored.mime, stored.path)
descriptions.append(description)
if isinstance(stored.part, ImageContent) and stored.mime.startswith("image/"):
if stored.size <= policy.max_bytes and inline_images < policy.max_count:
payload = base64.b64encode(stored.path.read_bytes()).decode("ascii")
blocks.append(
{
"type": "image_url",
"image_url": {"url": f"data:{stored.mime};base64,{payload}"},
}
)
inline_images += 1
else:
blocks.append(
{"type": "text", "text": f"[图片未内联] {description}(可用文件工具读取)"}
)
continue
if isinstance(stored.part, FileContent) and _is_text_file(stored.part):
raw = stored.path.read_bytes()[: policy.max_text_chars * 4]
decoded = _decode_text(raw, policy.max_text_chars * 4)
if decoded:
clipped = decoded[: policy.max_text_chars]
suffix = "" if len(decoded) <= policy.max_text_chars else "\n…(内容已截断)"
blocks.append(
{
"type": "text",
"text": f"【附件 {stored.part.filename or stored.path.name}】\n{clipped}{suffix}",
}
)
continue
blocks.append({"type": "text", "text": f"【附件】{description}(可用文件工具读取)"})
# 正文块一定排在最前:多数模型对首块是文本的提示更稳
if text:
blocks.insert(0, {"type": "text", "text": text})
# 无任何内容时 blocks 保持为空:占位块会让 has_content 恒真,见该属性的说明
prepared.blocks = blocks
summary_parts = [text] if text else []
if descriptions:
summary_parts.append("[附件] " + ";".join(descriptions))
if prepared.notes:
summary_parts.append("[降级] " + ";".join(prepared.notes))
prepared.summary = "\n".join(p for p in summary_parts if p).strip()
return prepared
+18 -4
View File
@@ -27,6 +27,7 @@ from bot042.config import Settings
from bot042.db import repo from bot042.db import repo
from bot042.db.schema import init_db from bot042.db.schema import init_db
from bot042.flows.agent_task import AgentCache, last_message_text, run_task from bot042.flows.agent_task import AgentCache, last_message_text, run_task
from bot042.flows.media_input import prepare_inbound
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
@@ -89,6 +90,7 @@ def make_processor(
""" """
settings = settings or Settings.from_env() settings = settings or Settings.from_env()
db_file = Path(db) if db is not None else settings.database db_file = Path(db) if db is not None else settings.database
media_root = settings.home / "data" / "inbound"
db_ready = False db_ready = False
# 智能体按处理器复用:图构建开销大,每条消息重建是纯浪费 # 智能体按处理器复用:图构建开销大,每条消息重建是纯浪费
agent_cache = AgentCache() agent_cache = AgentCache()
@@ -97,10 +99,17 @@ def make_processor(
nonlocal db_ready nonlocal db_ready
yield MessageEvent.typing() yield MessageEvent.typing()
text = message.text.strip() # 媒体先落盘再进上下文:小图内联、大图与音视频退化为「路径 + 元信息」。
if not text: # 附件级失败只降级为说明,不打断整轮(见 media_input.prepare_inbound)。
yield MessageEvent.error_event("(空消息,已忽略)" if not message.has_media else "暂仅支持文本消息") prepared = await prepare_inbound(message, root=media_root)
if not prepared.has_content:
# 有附件但全部没接收成功:说清原因,别让用户以为是「发了条空消息」
if prepared.notes:
yield MessageEvent.error_event("附件未能接收:" + ";".join(prepared.notes))
else:
yield MessageEvent.error_event("(空消息,已忽略)")
return return
text = prepared.summary
session_id, channel = _session_key(message) session_id, channel = _session_key(message)
try: try:
@@ -115,7 +124,12 @@ def make_processor(
await repo.add_message( await repo.add_message(
db_file, session_id=session_id, role="user", content=text, channel=channel db_file, session_id=session_id, role="user", content=text, channel=channel
) )
result = await run_task(text, settings, history=history, cache=agent_cache) # 只有正文时发纯文本,带附件才发多模态块列表;用 blocks 数量猜会
# 在「只有一张图、没有正文」时把附件丢掉(见 PreparedInbound.is_multimodal)
task_input = prepared.blocks if prepared.is_multimodal else text
result = await run_task(
task_input, settings, history=history, cache=agent_cache
)
answer = last_message_text(result) answer = last_message_text(result)
if answer: if answer:
await repo.add_message( await repo.add_message(
+139
View File
@@ -0,0 +1,139 @@
"""步数预算测试:预算换算、收尾提示注入、硬上限转成可读失败。"""
from __future__ import annotations
import asyncio
from typing import Any
import pytest
from langgraph.errors import GraphRecursionError
from bot042.agent.middlewares.budget import (
BUDGET_PREFIX,
BUDGET_WARN_MARGIN,
StepBudget,
StepBudgetExceeded,
StepBudgetMiddleware,
bind,
current_budget,
unbind,
)
from bot042.config import STEP_GRACE, Settings
from bot042.flows import agent_task as agent_task_module
from bot042.flows.agent_task import run_task, stream_task
class RecordingAgent:
"""记录调用配置,并顺带把当时绑定的预算读出来。"""
def __init__(self) -> None:
self.config: dict[str, Any] | None = None
self.remaining: int | None = None
async def ainvoke(
self, state: dict[str, Any], config: dict[str, Any] | None = None
) -> dict[str, Any]:
_ = state
self.config = config
budget = current_budget()
self.remaining = budget.remaining if budget is not None else None
return {"messages": []}
class RecursionBoomAgent:
"""模拟 LangGraph 撞上硬上限。"""
async def ainvoke(
self, state: dict[str, Any], config: dict[str, Any] | None = None
) -> dict[str, Any]:
_ = state, config
raise GraphRecursionError("Recursion limit of 27 reached")
async def astream(self, *args: Any, **kwargs: Any) -> Any:
_ = args, kwargs
raise GraphRecursionError("Recursion limit of 27 reached")
yield # pragma: no cover - 让本方法成为异步生成器
def test_settings_derive_recursion_limit_and_rounds() -> None:
"""硬上限必须大于软预算,轮数预算按「一次往返约 2 步」换算。"""
settings = Settings(max_steps=200)
assert settings.recursion_limit == 200 + STEP_GRACE
assert settings.agent_rounds == 100
# 极小预算也要留出至少一轮,否则中间件永远来不及提醒
assert Settings(max_steps=1).agent_rounds == 1
def test_budget_take_counts_down() -> None:
budget = StepBudget(rounds=2)
assert budget.take() == 1
assert budget.take() == 0
assert budget.take() == 0 # 不为负
assert budget.used == 3
assert budget.remaining == 0
def test_middleware_noop_without_budget() -> None:
middleware = StepBudgetMiddleware()
assert asyncio.run(middleware.abefore_model({}, None)) is None # type: ignore[arg-type]
def test_middleware_warns_once_when_budget_runs_low() -> None:
budget = StepBudget(rounds=BUDGET_WARN_MARGIN + 2)
token = bind(budget)
try:
middleware = StepBudgetMiddleware()
# 剩余 5 → 安静;剩余 4 → 注入收尾提示
assert asyncio.run(middleware.abefore_model({}, None)) is None # type: ignore[arg-type]
update = asyncio.run(middleware.abefore_model({}, None)) # type: ignore[arg-type]
assert update is not None
message = update["messages"][0]
assert isinstance(message.content, str)
assert message.content.startswith(BUDGET_PREFIX)
assert "剩余约 4 轮" in message.content
assert budget.warned is True
# 只提醒一次:反复打断只会让模型更答不完
assert asyncio.run(middleware.abefore_model({}, None)) is None # type: ignore[arg-type]
finally:
unbind(token)
def test_run_task_passes_derived_limit_and_binds_budget(monkeypatch) -> None:
agent = RecordingAgent()
monkeypatch.setattr(
agent_task_module, "create_agent", lambda settings=None: agent
)
settings = Settings(max_steps=7)
asyncio.run(run_task("hi", settings))
assert agent.config == {"recursion_limit": 7 + STEP_GRACE}
assert agent.remaining == settings.agent_rounds # 预算已绑定且未被计数
# 运行结束必须解绑,否则下一个任务会读到上一轮的预算
assert current_budget() is None
def test_run_task_translates_recursion_error(monkeypatch) -> None:
monkeypatch.setattr(
agent_task_module, "create_agent", lambda settings=None: RecursionBoomAgent()
)
with pytest.raises(StepBudgetExceeded) as info:
asyncio.run(run_task("hi", Settings(max_steps=7)))
assert info.value.max_steps == 7
assert "max_steps=7" in str(info.value)
assert current_budget() is None
def test_stream_task_translates_recursion_error(monkeypatch) -> None:
monkeypatch.setattr(
agent_task_module, "create_agent", lambda settings=None: RecursionBoomAgent()
)
async def _consume() -> None:
async for _ in stream_task("hi", Settings(max_steps=7)):
pass
with pytest.raises(StepBudgetExceeded):
asyncio.run(_consume())
assert current_budget() is None
+57 -1
View File
@@ -9,6 +9,7 @@ from typing import Any
import pytest import pytest
from bot042.agent.middlewares.budget import StepBudgetExceeded
from bot042.channels.base import BaseChannel from bot042.channels.base import BaseChannel
from bot042.channels.manager import ChannelManager from bot042.channels.manager import ChannelManager
from bot042.channels.models import ( from bot042.channels.models import (
@@ -20,7 +21,13 @@ from bot042.channels.models import (
from bot042.config import Settings from bot042.config import Settings
from bot042.db import repo from bot042.db import repo
from bot042.flows import builtin from bot042.flows import builtin
from bot042.flows.builtin import ERROR_REPLY, RPC_SEND, AgentReply, ChannelServices from bot042.flows.builtin import (
BUDGET_REPLY,
ERROR_REPLY,
RPC_SEND,
AgentReply,
ChannelServices,
)
from bot042.flows.bus import EventBus from bot042.flows.bus import EventBus
from bot042.flows.rpc import RpcHub from bot042.flows.rpc import RpcHub
from bot042.flows.scheduler import CN_TZ from bot042.flows.scheduler import CN_TZ
@@ -209,3 +216,52 @@ def test_agent_reply_sends_error_hint_on_failure(
sent, statuses = asyncio.run(_run()) sent, statuses = asyncio.run(_run())
assert sent == [ERROR_REPLY] assert sent == [ERROR_REPLY]
assert statuses == [RunStatus.FAILED] assert statuses == [RunStatus.FAILED]
def test_agent_reply_explains_step_budget_without_generic_error(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""步数用尽:给出可操作说明并落库,且不再补发通用错误文案。"""
async def exhausted(task, settings=None, history=None, *, cache=None, steering=None):
raise StepBudgetExceeded(200)
monkeypatch.setattr(builtin, "run_task", exhausted)
async def _run() -> tuple[list[str], list[RunStatus]]:
bus = EventBus()
bus.bind_loop()
hub = RpcHub(bus)
manager = ChannelManager()
channel = EchoChannel()
await manager.add_channel(channel)
await manager.start()
_ = hub.serve(RPC_SEND, manager.rpc_send)
services = ChannelServices(
channels=manager, settings=Settings(), db=tmp_path / "bot.db", hub=hub
)
runner = TaskRunner(bus, deps=services)
runner.register(AgentReply)
_ = runner.add_task(AgentReply.name, EventTrigger(event="channel.inbound"))
await runner.start()
bus.emit("channel.inbound", _envelope("把整站重构一遍"))
await runner.wait_idle(timeout=2)
await bus.wait_for_complete()
statuses = [run.status for run in runner.runs(AgentReply.name)]
await runner.stop()
hub.close()
await manager.stop()
return channel.sent, statuses
sent, statuses = asyncio.run(_run())
notice = BUDGET_REPLY.format(steps=200)
assert sent == [notice] # 只有说明,没有多余的「处理消息时出错」
assert statuses == [RunStatus.FAILED]
# 失败轮也要落库:用户已经看到这段内容,历史缺了它会前后不接
rows = asyncio.run(repo.list_messages(tmp_path / "bot.db", session_id="s1"))
assert [(r["role"], r["content"]) for r in rows] == [
("user", "把整站重构一遍"),
("assistant", notice),
]
+3 -2
View File
@@ -8,7 +8,7 @@ from typing import Any
from bot042.channels.base import BaseChannel from bot042.channels.base import BaseChannel
from bot042.channels.models import ChannelSubject, InboundMessage, TextContent from bot042.channels.models import ChannelSubject, InboundMessage, TextContent
from bot042.config import Settings from bot042.config import STEP_GRACE, Settings
from bot042.flows.agent_task import last_message_text, run_task from bot042.flows.agent_task import last_message_text, run_task
from bot042.flows.session import run_session from bot042.flows.session import run_session
@@ -86,7 +86,8 @@ def test_run_task_uses_async_ainvoke(monkeypatch) -> None:
assert last_message_text(result) == "done" assert last_message_text(result) == "done"
state, config = fake.calls[0] state, config = fake.calls[0]
assert state["messages"][-1] == {"role": "user", "content": "hi"} assert state["messages"][-1] == {"role": "user", "content": "hi"}
assert config == {"recursion_limit": 7} # 硬步数上限 = 软预算 + 收尾余量(见 Settings.recursion_limit)
assert config == {"recursion_limit": 7 + STEP_GRACE}
def test_last_message_text_multimodal_content() -> None: def test_last_message_text_multimodal_content() -> None:
+52 -2
View File
@@ -16,6 +16,7 @@ from bot042.channels.models import (
) )
from bot042.db import repo from bot042.db import repo
from bot042.flows import processor as processor_mod from bot042.flows import processor as processor_mod
from bot042.flows.media_input import PreparedInbound
from bot042.flows.processor import make_processor from bot042.flows.processor import make_processor
@@ -210,7 +211,7 @@ def test_build_history_rules() -> None:
] ]
def test_processor_rejects_empty_and_media( def test_processor_rejects_empty_message(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None: ) -> None:
async def fake_run_task(task, settings=None, history=None, *, cache=None): # pragma: no cover async def fake_run_task(task, settings=None, history=None, *, cache=None): # pragma: no cover
@@ -223,6 +224,55 @@ def test_processor_rejects_empty_and_media(
assert events[-1].type == MessageEventType.ERROR assert events[-1].type == MessageEventType.ERROR
assert "空消息" in (events[-1].error or "") assert "空消息" in (events[-1].error or "")
def test_processor_explains_unreadable_attachments(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""附件全部没接收成功:给附件级说明,不要含糊成「空消息」。"""
async def fake_prepare(message, *, root, **kwargs):
_ = message, root, kwargs
return PreparedInbound(text="", notes=["图片未接收:超过大小上限"])
async def fake_run_task(task, settings=None, history=None, *, cache=None): # pragma: no cover
raise AssertionError("不应执行任务")
monkeypatch.setattr(processor_mod, "prepare_inbound", fake_prepare)
monkeypatch.setattr(processor_mod, "run_task", fake_run_task)
processor = make_processor(db=tmp_path / "bot.db")
events = _events(processor, _message("", media=True)) events = _events(processor, _message("", media=True))
assert events[-1].type == MessageEventType.ERROR assert events[-1].type == MessageEventType.ERROR
assert "仅支持文本" in (events[-1].error or "") error = events[-1].error or ""
assert "附件未能接收" in error
assert "超过大小上限" in error
def test_processor_sends_blocks_for_attachment_only_message(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""只有一张图、没有正文时也必须发多模态块,否则附件被静默丢掉。"""
async def fake_prepare(message, *, root, **kwargs):
_ = message, root, kwargs
return PreparedInbound(
text="",
blocks=[
{"type": "image_url", "image_url": {"url": "data:image/png;base64,AA=="}}
],
)
seen: list[object] = []
async def fake_run_task(task, settings=None, history=None, *, cache=None):
seen.append(task)
return {"messages": [{"role": "assistant", "content": "看到了"}]}
monkeypatch.setattr(processor_mod, "prepare_inbound", fake_prepare)
monkeypatch.setattr(processor_mod, "run_task", fake_run_task)
processor = make_processor(db=tmp_path / "bot.db")
events = _events(processor, _message("", media=True))
assert isinstance(seen[0], list)
assert seen[0][0]["type"] == "image_url"
assert events[-1].type == MessageEventType.COMPLETED
+66 -2
View File
@@ -9,6 +9,7 @@ from typing import Any, ClassVar
import pytest import pytest
from bot042.agent.middlewares.budget import StepBudgetExceeded
from bot042.channels.base import BaseChannel from bot042.channels.base import BaseChannel
from bot042.channels.manager import ChannelManager from bot042.channels.manager import ChannelManager
from bot042.channels.models import ( from bot042.channels.models import (
@@ -21,12 +22,13 @@ from bot042.channels.models import (
) )
from bot042.channels.registry import register_channel from bot042.channels.registry import register_channel
from bot042.config import Settings from bot042.config import Settings
from bot042.db import repo
from bot042.flows import builtin from bot042.flows import builtin
from bot042.flows.builtin import RPC_STREAM, AgentReply, ChannelServices from bot042.flows.builtin import BUDGET_REPLY, RPC_STREAM, AgentReply, ChannelServices
from bot042.flows.bus import EventBus from bot042.flows.bus import EventBus
from bot042.flows.rpc import RpcHub from bot042.flows.rpc import RpcHub
from bot042.flows.stream import ReplyStream from bot042.flows.stream import ReplyStream
from bot042.flows.tasks import EventTrigger, TaskRunner from bot042.flows.tasks import EventTrigger, RunStatus, TaskRunner
def test_stream_sends_ack_and_throttles() -> None: def test_stream_sends_ack_and_throttles() -> None:
@@ -220,6 +222,68 @@ def test_agent_reply_acks_then_streams(
assert frames[-1] == ("你好", True) assert frames[-1] == ("你好", True)
def test_agent_reply_streams_budget_notice_after_partial(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""步数用尽:说明补在已流出的内容后面,落库,且不再补发通用错误文案。"""
StreamChannel.instances.clear()
async def exhausted(task, settings=None, history=None, *, cache=None, steering=None):
yield "做到一半"
raise StepBudgetExceeded(200)
monkeypatch.setattr(builtin, "stream_task", exhausted)
async def _run() -> tuple[list[tuple[str, bool]], list[RunStatus]]:
bus = EventBus()
bus.bind_loop()
hub = RpcHub(bus)
manager = ChannelManager()
channel = StreamChannel(channel_id="s1")
await manager.add_channel(channel)
await manager.start()
_ = hub.serve(RPC_STREAM, manager.rpc_stream)
services = ChannelServices(
channels=manager,
settings=Settings(),
db=tmp_path / "bot.db",
hub=hub,
)
runner = TaskRunner(bus, deps=services)
runner.register(AgentReply)
_ = runner.add_task(AgentReply.name, EventTrigger(event="channel.inbound"))
await runner.start()
bus.emit(
"channel.inbound",
ChannelMessage.of(
InboundMessage(
channel_id="s1",
channel_type="stream-test",
channel_subject=ChannelSubject(subject_id="u1"),
content=[TextContent(text="把整站重构一遍")],
)
),
)
await asyncio.sleep(0.05)
await runner.wait_idle()
statuses = [run.status for run in runner.runs(AgentReply.name)]
await runner.stop()
hub.close()
await manager.stop()
return channel.frames, statuses
frames, statuses = asyncio.run(_run())
final = "做到一半\n\n" + BUDGET_REPLY.format(steps=200)
assert frames[0] == (builtin.ACK_TEXT, False)
assert frames[-1] == (final, True) # 收尾帧保留了部分内容 + 说明
assert statuses == [RunStatus.FAILED]
rows = asyncio.run(repo.list_messages(tmp_path / "bot.db", session_id="s1:u1"))
assert rows[-1]["role"] == "assistant"
assert rows[-1]["content"] == final
def test_outbound_stream_payload_defaults() -> None: def test_outbound_stream_payload_defaults() -> None:
payload = OutboundStream( payload = OutboundStream(
channel_id="c", subject_id="u", stream_id="s", text="x" channel_id="c", subject_id="u", stream_id="s", text="x"