Compare commits
6
Commits
fbb01c4c83
..
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
d0e187f74d | ||
|
|
13b8d7a466 | ||
|
|
5ca40c6b0e | ||
|
|
2dde0d1160 | ||
|
|
9ce0ba0e13 | ||
|
|
5da55f8d17 |
No files matched your search
@@ -154,8 +154,10 @@ bot serve --channel wecom
|
||||
规则:
|
||||
|
||||
- 未设置 `BOT042_MODEL` 时,使用 `models` 表中 `is_default=1` 的模型;
|
||||
- 单次任务步数上限同理:`settings.agent.max_steps`(默认 200),`BOT042_MAX_STEPS` 优先;
|
||||
联网检索类任务步数消耗快,触顶会抛 `GraphRecursionError`,表现为「处理消息时出错」;
|
||||
- 单次任务步数预算同理:`settings.agent.max_steps`(默认 200),`BOT042_MAX_STEPS` 优先;
|
||||
预算将尽时智能体先收到一条「立刻收尾」提示,把已完成的部分总结出来;交给 LangGraph
|
||||
的硬上限是 `max_steps + 20` 个图步,真撞上时用户看到的是「本轮执行步数用尽(上限 N 步)」
|
||||
加已产出的内容,而不是笼统的「处理消息时出错」;
|
||||
- `OPENAI_API_KEY` / `OPENAI_BASE_URL` 为空时,从该模型所属 `providers` 行补齐;
|
||||
- 环境变量指定的模型与库中默认模型**不一致**时,不会套用库里的密钥
|
||||
(错用别的 key 只会得到 401);
|
||||
@@ -169,7 +171,7 @@ bot serve --channel wecom
|
||||
| `BOT042_MODEL` | langchain 模型标识(`provider:model`) | `openai:gpt-4o-mini` |
|
||||
| `OPENAI_API_KEY` | OpenAI 兼容 API Key | — |
|
||||
| `OPENAI_BASE_URL` | 兼容网关地址(DeepSeek / Kimi 等均可) | 官方地址 |
|
||||
| `BOT042_MAX_STEPS` | 单次任务最大步数(recursion_limit) | `200` |
|
||||
| `BOT042_MAX_STEPS` | 单次任务步数预算(软上限,硬上限为其 +20) | `200` |
|
||||
| `BOT042_LOG_LEVEL` | 日志级别 | `INFO` |
|
||||
|
||||
使用 DeepSeek 等 OpenAI 兼容服务示例:
|
||||
|
||||
+43
-8
@@ -317,16 +317,42 @@ ChannelManager(
|
||||
`supports_steer = True`。处理链路:
|
||||
|
||||
```
|
||||
ChannelMessage → 校验文本(空文本/纯媒体给固定提示)
|
||||
ChannelMessage → prepare_inbound(附件落盘;小图内联成 image_url 块)
|
||||
→ 无可用内容?→ 回「附件未能接收:<原因>」/「(无输出)」并结束
|
||||
→ ensure_db → ensure_session → list_messages → build_history
|
||||
→ add_message(user)
|
||||
→ add_message(user, 正文 + 附件清单)
|
||||
→ 建 MediaOutbox(本轮出站信箱)
|
||||
→ open_stream() ┬ 支持流式:update(ACK) → stream_task 边生成边覆盖 → finish
|
||||
└ 不支持:run_task → send_text 一次性发送
|
||||
│ → 媒体在文本定稿后单独补一条
|
||||
└ 不支持:run_task → 有媒体走 send_content,否则 send_text
|
||||
→ 补记插话(_persist_steered)→ add_message(assistant)
|
||||
```
|
||||
|
||||
媒体链路(issue #1 的 P0 部分):
|
||||
|
||||
- **入站**:`flows/media_input.prepare_inbound()` 把分片整理成多模态块——小图 base64
|
||||
内联、大图与音视频退化为「路径 + 元信息」、文本类文件抽取正文;单个附件超限/类型
|
||||
拒绝/下载失败只写进 `notes`,不打断整轮。字节落盘在 `home/data/inbound/<session>/`,
|
||||
历史只存正文 + 附件清单,不存字节。
|
||||
- **出站**:`agent/outbox.MediaOutbox` 是本轮的信箱,`send_image` / `send_file` 工具
|
||||
往里登记分片;`run()` 结束后 `drain()` 出来,与正文合成**一条** `OutboundContent`,
|
||||
由出站管线「先文本、再逐个媒体」下发,避免平台上出现两条消息。只有媒体没有正文时
|
||||
不再补「(无输出)」;流式通道只送文本,媒体等文本定稿后单独补一条。
|
||||
- **安全护栏**:所有字节来源(`data` / `local_path` / `url`)都经过
|
||||
`channels/media.py` 的同一套策略——按类型的大小上限、魔数优先的 MIME 判定、
|
||||
可执行扩展名拉黑、文件名净化与根目录约束;远端下载改为流式按块读取,
|
||||
`Content-Length` 先预检、超限即中断,整份读进内存的路径已移除。
|
||||
- **平台解密**:`BaseChannel.make_media_transform()` 按单条消息的 metadata 构造
|
||||
解密钩子(密钥随消息而来,挂在渠道实例上会串台),企业微信用
|
||||
`<media>.aeskey` 走 SDK 的 `decrypt_file`。解密在类型嗅探**之前**执行,并对
|
||||
明文再量一次大小——密文没超不等于明文没超。
|
||||
- **CLI 调试入口**:`/image <路径>`、`/file <路径>` 可以直接在无平台环境造出
|
||||
媒体入站消息;出站 `_send_media` 不再打印占位行,改为回执真实字节的
|
||||
大小与 sha256 前缀(CLI 里没有「已发送」这回事)。
|
||||
|
||||
`ChannelServices` 是它的运行期依赖(`channels` / `settings` / `db` / `hub` / `cache` /
|
||||
历史条数与字符预算 / 发送超时),由 `TaskRunner(deps=...)` 注入。
|
||||
历史条数与字符预算 / 发送超时),由 `TaskRunner(deps=...)` 注入;`media_policy()` 从
|
||||
渠道取媒体大小上限与下载请求头,未知渠道退回默认值(不因为查不到渠道就跳过校验)。
|
||||
`on_error` 钩子会尽量回一句 `ERROR_REPLY`,发送失败只记日志(避免错误处理再次抛错)。
|
||||
|
||||
两个细节:
|
||||
@@ -457,13 +483,22 @@ langchain 从环境变量取凭据。
|
||||
一次 `ainvoke`,但缓存由持有者(`ChannelServices`)决定生命周期,不做模块级单例,
|
||||
避免配置不同或测试替换 `create_agent` 时串味。
|
||||
|
||||
`run_task()` 把历史转成 `{"messages": [...]}`,以 `recursion_limit=settings.max_steps` 调用 `ainvoke`,
|
||||
`run_task()` 把历史转成 `{"messages": [...]}`,以 `recursion_limit=settings.recursion_limit` 调用 `ainvoke`,
|
||||
返回最终 state;`last_message_text()` 负责从多模态 content 中只取文本块(按行拼接)。
|
||||
|
||||
`max_steps` 默认 200,优先级 **环境变量 `BOT042_MAX_STEPS` > `settings.agent.max_steps` > 默认**:
|
||||
联网检索类任务一轮要连续消耗十几个图步数(搜索 → 抓取 → 再搜索 → 汇总),上限过低时
|
||||
`GraphRecursionError` 会被渠道兜底成用户看到的「处理消息时出错,请稍后再试。」,
|
||||
所以它和模型配置一起由 `resolve_model_settings()` 解析进 `Settings`。
|
||||
联网检索类任务一轮要连续消耗十几个图步数(搜索 → 抓取 → 再搜索 → 汇总)。它是**软预算**,
|
||||
不是 `recursion_limit` 本身——
|
||||
|
||||
- `StepBudgetMiddleware`(`agent/middlewares/budget.py`)按模型调用轮数计账(`agent_rounds = max_steps // 2`,
|
||||
一次「模型 → 工具」往返约耗 2 个图步),剩余 ≤4 轮时注入一条「立刻停止调用工具、给出结论」
|
||||
的提示,且只提醒一次;
|
||||
- 交给 LangGraph 的硬上限是 `recursion_limit = max_steps + STEP_GRACE(20)`,留给收尾动作;
|
||||
- 真撞上硬上限(模型不听话、或子智能体的步数不在计数内)时,`GraphRecursionError` 会被转成
|
||||
`StepBudgetExceeded`;`AgentReply` 把它渲染成「本轮执行步数用尽(上限 N 步)」+ 已产出的部分内容,
|
||||
一并落库为 assistant 消息,并标记 `_reported` 让 `on_error` 不再补发通用错误文案。
|
||||
|
||||
两者以前是同值,于是预算耗尽的那一刻必然抛异常,模型连一句话都来不及说——这是 v0.1 的 bug。
|
||||
|
||||
## 7. 存储层(`db/`)
|
||||
|
||||
|
||||
@@ -11,6 +11,7 @@ from deepagents.backends import LocalShellBackend
|
||||
from langchain.agents.middleware.types import AgentMiddleware
|
||||
from langgraph.graph.state import CompiledStateGraph
|
||||
|
||||
from bot042.agent.middlewares.budget import StepBudgetMiddleware
|
||||
from bot042.agent.middlewares.steering import SteeringMiddleware
|
||||
from bot042.agent.prompts import SYSTEM_PROMPT
|
||||
from bot042.agent.tools import get_tools
|
||||
@@ -90,12 +91,13 @@ def create_agent(
|
||||
"openai:gpt-4o-mini"、"openai:deepseek-chat"(配合 OPENAI_BASE_URL)等。
|
||||
"""
|
||||
settings = settings or Settings.from_env()
|
||||
# SteeringMiddleware 放最前:每轮模型调用前都先消化插话
|
||||
# SteeringMiddleware 放最前:每轮模型调用前都先消化插话;
|
||||
# StepBudgetMiddleware 紧随其后:预算将尽时注入收尾提示,避免撞硬上限
|
||||
return create_deep_agent(
|
||||
model=_build_model(settings),
|
||||
tools=get_tools(),
|
||||
system_prompt=SYSTEM_PROMPT,
|
||||
backend=_build_backend(settings),
|
||||
middleware=[SteeringMiddleware(), *middleware],
|
||||
middleware=[SteeringMiddleware(), StepBudgetMiddleware(), *middleware],
|
||||
interrupt_on=interrupt_on,
|
||||
)
|
||||
@@ -0,0 +1,126 @@
|
||||
"""步数预算:在硬上限之前提醒模型收尾。
|
||||
|
||||
LangGraph 的 ``recursion_limit`` 是**硬中断**——撞上就抛
|
||||
``GraphRecursionError``,模型没有任何机会整理已做的工作、给出结论。所以这里
|
||||
在触到硬上限之前先注入一条收尾提示,让模型自己把半成品收口:说清做到哪一步、
|
||||
结论是什么、还差什么。
|
||||
|
||||
预算按**模型调用轮数**计(一个「模型 → 工具」往返约耗 2 个图步),轮数预算由
|
||||
:attr:`bot042.config.Settings.agent_rounds` 从 ``max_steps`` 换算而来;硬上限
|
||||
则是 :attr:`bot042.config.Settings.recursion_limit`,比预算多留
|
||||
:data:`bot042.config.STEP_GRACE` 步给收尾动作。
|
||||
|
||||
与插话一样用 :mod:`contextvars` 绑定:``AgentCache`` 让同一张图被多个会话复用,
|
||||
轮数记在实例属性上会串台。
|
||||
|
||||
限制:中间件只数**主智能体**的模型调用;子智能体(``task`` 工具派发的)消耗的
|
||||
步数不在计数内。所以硬上限不能省——它是唯一能兜住子智能体跑飞的东西。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import contextvars
|
||||
from dataclasses import dataclass
|
||||
from typing import Any
|
||||
|
||||
from langchain.agents.middleware.types import AgentMiddleware
|
||||
from langchain_core.messages import HumanMessage
|
||||
|
||||
__all__ = [
|
||||
"BUDGET_PREFIX",
|
||||
"BUDGET_PROMPT",
|
||||
"BUDGET_WARN_MARGIN",
|
||||
"StepBudget",
|
||||
"StepBudgetExceeded",
|
||||
"StepBudgetMiddleware",
|
||||
"bind",
|
||||
"current_budget",
|
||||
"unbind",
|
||||
]
|
||||
|
||||
BUDGET_WARN_MARGIN = 4
|
||||
"""剩余轮数 ≤ 该值时开始提醒收尾(留够 4 轮写下结论)。"""
|
||||
|
||||
BUDGET_PREFIX = "[系统 · 步数预算] "
|
||||
"""收尾提示的前缀,让模型能识别这不是用户的新指令。"""
|
||||
|
||||
BUDGET_PROMPT = (
|
||||
"本次任务的执行预算即将用尽(剩余约 {remaining} 轮模型调用)。"
|
||||
"请立刻停止调用工具,基于已经完成的工作给出最终答复:"
|
||||
"说明做到了哪一步、结论是什么、还差什么没做。不要再发起新的探索或修改。"
|
||||
)
|
||||
|
||||
|
||||
class StepBudgetExceeded(RuntimeError):
|
||||
"""图步数硬上限(``recursion_limit``)被耗尽。
|
||||
|
||||
这是**兜底**路径:正常情况下预算中间件会先提醒模型收尾;走到这里说明模型
|
||||
没有遵守提醒,或子智能体吃掉了大量步数。
|
||||
"""
|
||||
|
||||
def __init__(self, max_steps: int) -> None:
|
||||
self.max_steps = max_steps
|
||||
super().__init__(f"任务步数用尽(max_steps={max_steps})")
|
||||
|
||||
|
||||
@dataclass
|
||||
class StepBudget:
|
||||
"""一次运行的轮数预算(绑定单个 asyncio 任务,不加锁)。"""
|
||||
|
||||
rounds: int
|
||||
"""允许的模型调用轮数。"""
|
||||
|
||||
used: int = 0
|
||||
"""已发生的模型调用次数。"""
|
||||
|
||||
warned: bool = False
|
||||
"""是否已经注入过收尾提示(只提醒一次,避免反复打断)。"""
|
||||
|
||||
@property
|
||||
def remaining(self) -> int:
|
||||
"""剩余轮数(不为负)。"""
|
||||
return max(0, self.rounds - self.used)
|
||||
|
||||
def take(self) -> int:
|
||||
"""登记一次模型调用,返回登记后的剩余轮数。"""
|
||||
self.used += 1
|
||||
return self.remaining
|
||||
|
||||
|
||||
_current: contextvars.ContextVar[StepBudget | None] = contextvars.ContextVar(
|
||||
"bot042_step_budget", default=None
|
||||
)
|
||||
|
||||
|
||||
def current_budget() -> StepBudget | None:
|
||||
"""当前 asyncio 上下文的步数预算(无运行中任务时为 ``None``)。"""
|
||||
return _current.get()
|
||||
|
||||
|
||||
def bind(budget: StepBudget) -> contextvars.Token[StepBudget | None]:
|
||||
"""把预算绑定到当前上下文,返回用于还原的 token。"""
|
||||
return _current.set(budget)
|
||||
|
||||
|
||||
def unbind(token: contextvars.Token[StepBudget | None]) -> None:
|
||||
"""还原上下文绑定。"""
|
||||
_current.reset(token)
|
||||
|
||||
|
||||
class StepBudgetMiddleware(AgentMiddleware):
|
||||
"""在预算将尽时注入一次收尾提示。"""
|
||||
|
||||
async def abefore_model(self, state: Any, runtime: Any) -> dict[str, Any] | None:
|
||||
_ = state, runtime
|
||||
budget = _current.get()
|
||||
if budget is None or budget.warned:
|
||||
return None
|
||||
remaining = budget.take()
|
||||
if remaining > BUDGET_WARN_MARGIN:
|
||||
return None
|
||||
budget.warned = True
|
||||
return {
|
||||
"messages": [
|
||||
HumanMessage(content=BUDGET_PREFIX + BUDGET_PROMPT.format(remaining=remaining))
|
||||
]
|
||||
}
|
||||
@@ -0,0 +1,169 @@
|
||||
"""出站媒体信箱:让智能体在回复过程中登记要发给用户的媒体。
|
||||
|
||||
问题:智能体跑在 langgraph 里,工具是被模型调用的普通函数,拿不到渠道路由
|
||||
(``channel_id`` / ``subject_id`` / 被动回复帧)。而渠道那一层又只能拿到
|
||||
"智能体跑完后的最终文本"。
|
||||
|
||||
做法与 :mod:`bot042.agent.middlewares.steering` 同构:用 ``contextvars`` 放一个
|
||||
每轮一次的**信箱**,工具往里放分片,流程层在 ``run_task`` 之后取走并通过
|
||||
``channel.send`` RPC 下发。用 contextvar 而不是模块级单例,是因为同一张图会被
|
||||
多个会话复用,实例属性会串台。
|
||||
|
||||
安全边界(明确写出来):
|
||||
|
||||
- 大小与类型在**这里**和渠道层各校验一次,任一处拒绝都不会上传;
|
||||
- 路径限制(``allowed_roots``)默认不启用:智能体本来就有宿主机 shell 权限,
|
||||
文件路径白名单挡不住它读任何文件,给了只会是"看起来安全"。要收紧应该从
|
||||
工具权限入手(``interrupt_on`` / 去掉 ``execute``),而不是在这里假装设防;
|
||||
- 真正有效的是**大小上限**与**可执行扩展名拒绝**,它们限制了"一次误操作能造成
|
||||
多大外泄"以及"会不会把可执行文件推到对方机器上"。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import contextvars
|
||||
from collections import deque
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Final
|
||||
|
||||
from bot042.channels.media import (
|
||||
MediaLimits,
|
||||
MediaRejected,
|
||||
check_mime,
|
||||
read_file_limited,
|
||||
sanitize_filename,
|
||||
)
|
||||
from bot042.channels.models import (
|
||||
ContentPart,
|
||||
FileContent,
|
||||
ImageContent,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"MediaOutbox",
|
||||
"Outbox",
|
||||
"bind",
|
||||
"current_outbox",
|
||||
"make_file_part",
|
||||
"make_image_part",
|
||||
"unbind",
|
||||
]
|
||||
|
||||
MAX_PENDING: Final[int] = 8
|
||||
"""单轮回复最多登记多少个附件;超过的直接拒绝,避免模型刷屏式调用。"""
|
||||
|
||||
|
||||
@dataclass
|
||||
class MediaOutbox:
|
||||
"""一轮回复期间登记的待发媒体(绑定单个 asyncio 上下文)。"""
|
||||
|
||||
limits: MediaLimits = field(default_factory=MediaLimits)
|
||||
allowed_roots: tuple[Path, ...] | None = None
|
||||
max_pending: int = MAX_PENDING
|
||||
_items: deque[ContentPart] = field(default_factory=deque, repr=False)
|
||||
|
||||
def push(self, part: ContentPart) -> None:
|
||||
"""登记一个分片;超过 ``max_pending`` 抛 :class:`MediaRejected`。"""
|
||||
if len(self._items) >= self.max_pending:
|
||||
raise MediaRejected(
|
||||
"本轮附件数量超限",
|
||||
f"最多 {self.max_pending} 个,已拒绝额外的 {part.type}",
|
||||
)
|
||||
self._items.append(part)
|
||||
|
||||
def drain(self) -> list[ContentPart]:
|
||||
"""取走并清空全部已登记分片。"""
|
||||
items = list(self._items)
|
||||
self._items.clear()
|
||||
return items
|
||||
|
||||
def __len__(self) -> int:
|
||||
return len(self._items)
|
||||
|
||||
def __bool__(self) -> bool:
|
||||
return bool(self._items)
|
||||
|
||||
|
||||
Outbox = MediaOutbox
|
||||
"""别名:与 ``MediaOutbox`` 同一类型,供调用方用更短的写法。"""
|
||||
|
||||
_current: contextvars.ContextVar[MediaOutbox | None] = contextvars.ContextVar(
|
||||
"bot042_outbox", default=None
|
||||
)
|
||||
|
||||
|
||||
def current_outbox() -> MediaOutbox | None:
|
||||
"""当前上下文的信箱(无运行中回复时为 ``None``)。"""
|
||||
return _current.get()
|
||||
|
||||
|
||||
def bind(outbox: MediaOutbox) -> contextvars.Token:
|
||||
"""把信箱绑定到当前上下文,返回用于还原的 token。"""
|
||||
return _current.set(outbox)
|
||||
|
||||
|
||||
def unbind(token: contextvars.Token) -> None:
|
||||
"""还原上下文绑定。"""
|
||||
_current.reset(token)
|
||||
|
||||
|
||||
def _resolve(path_text: str, allowed_roots: tuple[Path, ...] | None) -> Path:
|
||||
"""解析并校验待发文件路径(存在性 + 可选根目录限制)。"""
|
||||
raw = (path_text or "").strip()
|
||||
if not raw:
|
||||
raise MediaRejected("文件路径为空", "")
|
||||
path = Path(raw).expanduser()
|
||||
try:
|
||||
resolved = path.resolve(strict=True)
|
||||
except (OSError, RuntimeError) as exc:
|
||||
raise MediaRejected("文件不存在或不可访问", f"{raw}: {exc}") from exc
|
||||
if not resolved.is_file():
|
||||
raise MediaRejected("不是普通文件", str(resolved))
|
||||
if allowed_roots and not any(
|
||||
resolved == root or root in resolved.parents
|
||||
for root in (r.resolve() for r in allowed_roots)
|
||||
):
|
||||
raise MediaRejected("路径不在允许的目录内", str(resolved))
|
||||
return resolved
|
||||
|
||||
|
||||
def make_image_part(
|
||||
path_text: str,
|
||||
*,
|
||||
limits: MediaLimits | None = None,
|
||||
allowed_roots: tuple[Path, ...] | None = None,
|
||||
caption: str = "",
|
||||
) -> ImageContent:
|
||||
"""把本地图片文件做成 :class:`ImageContent` 分片(校验大小与类型)。"""
|
||||
limits = limits or MediaLimits()
|
||||
path = _resolve(path_text, allowed_roots)
|
||||
raw = read_file_limited(path, limits.image)
|
||||
mime = check_mime(ImageContent(), "", raw[:32])
|
||||
return ImageContent(
|
||||
local_path=str(path),
|
||||
size=len(raw),
|
||||
mime_type=mime,
|
||||
alt_text=caption,
|
||||
)
|
||||
|
||||
|
||||
def make_file_part(
|
||||
path_text: str,
|
||||
*,
|
||||
limits: MediaLimits | None = None,
|
||||
allowed_roots: tuple[Path, ...] | None = None,
|
||||
filename: str = "",
|
||||
) -> FileContent:
|
||||
"""把本地文件做成 :class:`FileContent` 分片(校验大小与类型)。"""
|
||||
limits = limits or MediaLimits()
|
||||
path = _resolve(path_text, allowed_roots)
|
||||
raw = read_file_limited(path, limits.file)
|
||||
name = sanitize_filename(filename or path.name)
|
||||
mime = check_mime(FileContent(filename=name), "", raw[:32])
|
||||
return FileContent(
|
||||
local_path=str(path),
|
||||
filename=name,
|
||||
size=len(raw),
|
||||
mime_type=mime,
|
||||
)
|
||||
@@ -8,7 +8,13 @@ from typing import Any
|
||||
from langchain_core.tools import tool
|
||||
|
||||
from bot042.agent.execution import run_python
|
||||
from bot042.agent.outbox import (
|
||||
current_outbox,
|
||||
make_file_part,
|
||||
make_image_part,
|
||||
)
|
||||
from bot042.agent.web import get_web_tools
|
||||
from bot042.channels.media import MediaRejected, format_size
|
||||
|
||||
_VALID_DIRECTIONS = ("north", "south", "east", "west")
|
||||
_VALID_DIRECTIONS_CN = {"北": "north", "南": "south", "东": "east", "西": "west"}
|
||||
@@ -84,13 +90,66 @@ async def speak(text: str) -> str:
|
||||
return f"[播报] {text}"
|
||||
|
||||
|
||||
@tool
|
||||
async def send_image(path: str, caption: str = "") -> str:
|
||||
"""把本地图片发给用户(作为原生图片消息,而不是文本链接)。
|
||||
|
||||
仅支持 png/jpeg/gif/webp/bmp,且不超过大小上限;超限或类型不符会被拒绝
|
||||
并返回原因。同一轮回复里最多登记 8 个附件。
|
||||
|
||||
Args:
|
||||
path: 本地图片文件路径(绝对路径或相对当前工作目录)。
|
||||
caption: 附在图片上的说明,可留空。
|
||||
"""
|
||||
outbox = current_outbox()
|
||||
if outbox is None:
|
||||
return "[发送失败] 当前不在渠道会话中,无法发送图片"
|
||||
try:
|
||||
part = make_image_part(
|
||||
path,
|
||||
limits=outbox.limits,
|
||||
allowed_roots=outbox.allowed_roots,
|
||||
caption=caption,
|
||||
)
|
||||
outbox.push(part)
|
||||
except MediaRejected as exc:
|
||||
return f"[发送失败] {exc.reason}" + (f":{exc.detail}" if exc.detail else "")
|
||||
return f"[已登记图片] {part.local_path}({format_size(part.size or 0)},将在本轮回复后发送)"
|
||||
|
||||
|
||||
@tool
|
||||
async def send_file(path: str, filename: str = "") -> str:
|
||||
"""把本地文件发给用户(作为文件消息)。
|
||||
|
||||
可执行/脚本类扩展名(exe/bat/ps1/dll 等)会被拒绝。
|
||||
|
||||
Args:
|
||||
path: 本地文件路径。
|
||||
filename: 对方看到的文件名,留空则用原文件名。
|
||||
"""
|
||||
outbox = current_outbox()
|
||||
if outbox is None:
|
||||
return "[发送失败] 当前不在渠道会话中,无法发送文件"
|
||||
try:
|
||||
part = make_file_part(
|
||||
path,
|
||||
limits=outbox.limits,
|
||||
allowed_roots=outbox.allowed_roots,
|
||||
filename=filename,
|
||||
)
|
||||
outbox.push(part)
|
||||
except MediaRejected as exc:
|
||||
return f"[发送失败] {exc.reason}" + (f":{exc.detail}" if exc.detail else "")
|
||||
return f"[已登记文件] {part.filename}({format_size(part.size or 0)},将在本轮回复后发送)"
|
||||
|
||||
|
||||
def get_tools() -> list[Any]:
|
||||
"""返回机器人工具集(世界操作 + 联网 + 本地执行)。
|
||||
|
||||
shell 执行(``execute``)由 deepagents 的 FilesystemMiddleware 依据后端能力
|
||||
自动挂载,不在这里重复注册。
|
||||
"""
|
||||
return [perceive, move, speak, run_python, *get_web_tools()]
|
||||
return [perceive, move, speak, send_image, send_file, run_python, *get_web_tools()]
|
||||
|
||||
|
||||
def reset_robot() -> None:
|
||||
|
||||
+97
-14
@@ -16,8 +16,6 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import base64
|
||||
import binascii
|
||||
import logging
|
||||
import mimetypes
|
||||
import time
|
||||
@@ -29,6 +27,15 @@ from typing import Any
|
||||
|
||||
import httpx
|
||||
|
||||
from bot042.channels.media import (
|
||||
MediaLimits,
|
||||
MediaRejected,
|
||||
MediaTransform,
|
||||
decode_base64_limited,
|
||||
download_limited,
|
||||
format_size,
|
||||
read_file_limited,
|
||||
)
|
||||
from bot042.channels.models import (
|
||||
ChannelConfig,
|
||||
ChannelMessage,
|
||||
@@ -141,6 +148,15 @@ class BaseChannel(ABC):
|
||||
"""是否已配置消息处理器。"""
|
||||
return self._processor is not None
|
||||
|
||||
@property
|
||||
def media_limits(self) -> MediaLimits:
|
||||
"""本渠道的媒体大小上限(子类可覆写以适配平台限制)。"""
|
||||
limits = getattr(self, "_media_limits", None)
|
||||
if limits is None:
|
||||
limits = MediaLimits()
|
||||
self._media_limits = limits
|
||||
return limits
|
||||
|
||||
@property
|
||||
def is_connected(self) -> bool:
|
||||
"""平台连接是否在线(供管理器刷新运行时状态)。"""
|
||||
@@ -599,41 +615,108 @@ class BaseChannel(ABC):
|
||||
# =========================================================================
|
||||
# 媒体字节读取(data / local_path / url 三级优先)
|
||||
# =========================================================================
|
||||
def media_download_headers(self) -> dict[str, str]:
|
||||
"""下载入站媒体时附带的请求头(平台鉴权等);默认无。"""
|
||||
return {}
|
||||
|
||||
def media_transform(self, part: ContentPart, raw: bytes) -> bytes:
|
||||
"""平台钩子:把刚取到的媒体字节转换成明文(如企业微信的 AES 解密)。
|
||||
|
||||
默认原样返回。实现方应假定 ``raw`` 可能是密文,且**必须**是纯函数语义
|
||||
——上层会拿返回值的魔数重新判定类型、并再量一次大小。
|
||||
|
||||
这是**无消息上下文**的版本;需要按单条消息取密钥(如 ``aeskey``)时
|
||||
覆写 :meth:`make_media_transform`。
|
||||
"""
|
||||
return raw
|
||||
|
||||
def make_media_transform(self, metadata: dict[str, Any]) -> MediaTransform:
|
||||
"""按单条入站消息的元数据构造解密钩子。
|
||||
|
||||
密钥这类材料随消息而来(企业微信的 ``<media>.aeskey``),挂在渠道实例上
|
||||
会串台。返回闭包把它绑在这一条消息上,消息处理完即随之释放。
|
||||
|
||||
Args:
|
||||
metadata: 入站消息的 ``metadata``。
|
||||
"""
|
||||
_ = metadata
|
||||
return self.media_transform
|
||||
|
||||
async def fetch_remote_media(self, url: str) -> tuple[bytes, str]:
|
||||
"""下载远端媒体并返回 ``(bytes, content_type)``;子类可覆写加平台鉴权。"""
|
||||
async with httpx.AsyncClient(follow_redirects=True) as client:
|
||||
resp = await client.get(url)
|
||||
"""下载远端媒体并返回 ``(bytes, content_type)``;子类可覆写加平台鉴权。
|
||||
|
||||
**带大小闸**:``resp.content`` 会把整份响应读进内存,平台给出的 URL
|
||||
可能是任意大的文件,因此这里改为流式按块读取并在超限时中断。
|
||||
"""
|
||||
limits = self.media_limits
|
||||
async with httpx.AsyncClient(
|
||||
follow_redirects=True, timeout=limits.timeout
|
||||
) as client, client.stream("GET", url) as resp:
|
||||
resp.raise_for_status()
|
||||
content_type = resp.headers.get("content-type") or (
|
||||
mimetypes.guess_type(url)[0] or "application/octet-stream"
|
||||
)
|
||||
return resp.content, content_type
|
||||
budget = max(limits.image, limits.audio, limits.video, limits.file)
|
||||
declared = resp.headers.get("content-length", "")
|
||||
if declared.isdigit() and int(declared) > budget:
|
||||
raise MediaRejected(
|
||||
"媒体超过大小上限",
|
||||
f"Content-Length {format_size(int(declared))} > {format_size(budget)}",
|
||||
)
|
||||
buffer = bytearray()
|
||||
async for piece in resp.aiter_bytes(limits.chunk):
|
||||
buffer.extend(piece)
|
||||
if len(buffer) > budget:
|
||||
raise MediaRejected(
|
||||
"媒体超过大小上限", f"下载超过 {format_size(budget)} 已中断"
|
||||
)
|
||||
return bytes(buffer), content_type
|
||||
|
||||
async def load_media_bytes(self, part: ContentPart) -> tuple[bytes, str]:
|
||||
async def load_media_bytes(
|
||||
self, part: ContentPart, *, limits: MediaLimits | None = None
|
||||
) -> tuple[bytes, str]:
|
||||
"""把媒体分片解析为 ``(bytes, content_type)``。
|
||||
|
||||
解析顺序:``data``(base64 内联字节)→ ``local_path``(本地文件)→
|
||||
``url``(远端下载)。三者皆空时报 ValueError。
|
||||
|
||||
**全程带大小闸**(:class:`~bot042.channels.media.MediaLimits`):
|
||||
入站 URL 由平台给出、出站 ``local_path`` 由智能体给出,长度都不可信,
|
||||
整份读进内存会把进程打爆。超限抛
|
||||
:class:`~bot042.channels.media.MediaRejected`(``ValueError`` 子类)。
|
||||
"""
|
||||
limits = limits or self.media_limits
|
||||
if isinstance(part, TextContent):
|
||||
raise TypeError("load_media_bytes 不能用于文本分片")
|
||||
|
||||
limit = limits.for_part(part)
|
||||
if part.data:
|
||||
mime = part.mime_type or "application/octet-stream"
|
||||
try:
|
||||
return base64.b64decode(part.data, validate=False), mime
|
||||
except (binascii.Error, ValueError) as exc:
|
||||
raise ValueError(f"ContentPart.data 不是合法 base64: {exc}") from exc
|
||||
return decode_base64_limited(part.data, limit), mime
|
||||
|
||||
if part.local_path:
|
||||
path = Path(part.local_path)
|
||||
mime = part.mime_type or mimetypes.guess_type(str(path))[0] or "application/octet-stream"
|
||||
return path.read_bytes(), mime
|
||||
return read_file_limited(path, limit), mime
|
||||
|
||||
if part.url:
|
||||
mime = part.mime_type
|
||||
mime = part.mime_type or mimetypes.guess_type(part.url)[0]
|
||||
try:
|
||||
raw, guessed = await download_limited(
|
||||
part.url,
|
||||
limit=limit,
|
||||
chunk=limits.chunk,
|
||||
timeout=limits.timeout,
|
||||
headers=self.media_download_headers(),
|
||||
)
|
||||
except httpx.HTTPError:
|
||||
# 平台鉴权只能由子类覆写 fetch_remote_media 提供;这里退回它取字节
|
||||
raw, guessed = await self.fetch_remote_media(part.url)
|
||||
return raw, mime or guessed
|
||||
if len(raw) > limit:
|
||||
raise MediaRejected(
|
||||
"媒体超过大小上限",
|
||||
f"{format_size(len(raw))} > {format_size(limit)}",
|
||||
) from None
|
||||
return raw, mime or guessed or "application/octet-stream"
|
||||
|
||||
raise ValueError(f"分片缺少 data/local_path/url: {part}")
|
||||
+108
-5
@@ -9,15 +9,26 @@ from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import contextlib
|
||||
import hashlib
|
||||
import logging
|
||||
import mimetypes
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from bot042.channels.base import BaseChannel
|
||||
from bot042.channels.media import (
|
||||
MediaRejected,
|
||||
format_size,
|
||||
read_file_limited,
|
||||
sanitize_filename,
|
||||
)
|
||||
from bot042.channels.models import (
|
||||
ChannelConfig,
|
||||
ChannelSubject,
|
||||
ContentPart,
|
||||
FileContent,
|
||||
ImageContent,
|
||||
InboundMessage,
|
||||
MessageProcessor,
|
||||
TextContent,
|
||||
@@ -29,6 +40,9 @@ logger = logging.getLogger(__name__)
|
||||
_EXIT_WORDS = frozenset({"exit", "quit"})
|
||||
_LOCAL_SUBJECT = "local"
|
||||
|
||||
_MEDIA_COMMANDS = {"image": "image", "img": "image", "file": "file", "media": "file"}
|
||||
"""``/image <路径>``、``/file <路径>``:无真实平台时用来调试媒体收发。"""
|
||||
|
||||
|
||||
class CliConfig(ChannelConfig):
|
||||
"""CLI 渠道配置。"""
|
||||
@@ -105,9 +119,18 @@ class CliChannel(BaseChannel):
|
||||
return False
|
||||
|
||||
def parse_inbound(self, raw_payload: Any) -> InboundMessage:
|
||||
"""CLI 的原始负载就是一行文本(或已构建的 InboundMessage)。"""
|
||||
"""CLI 的原始负载就是一行文本(或已构建的 InboundMessage)。
|
||||
|
||||
``/image <路径>`` 与 ``/file <路径>`` 会解析成本地附件分片,
|
||||
路径校验(存在性、大小上限、类型嗅探)与其他渠道共用同一套
|
||||
:mod:`bot042.channels.media` 策略,不因为「本地」就跳过。
|
||||
"""
|
||||
if isinstance(raw_payload, InboundMessage):
|
||||
return raw_payload
|
||||
line = str(raw_payload)
|
||||
parts = self._parse_media_command(line)
|
||||
if parts is None:
|
||||
parts = [TextContent(text=line)]
|
||||
return InboundMessage(
|
||||
channel_id=self.channel_id,
|
||||
channel_type=self.channel_type,
|
||||
@@ -117,9 +140,57 @@ class CliChannel(BaseChannel):
|
||||
chat_type="direct",
|
||||
),
|
||||
channel_session_id=self.channel_type,
|
||||
content=[TextContent(text=str(raw_payload))],
|
||||
content=parts,
|
||||
)
|
||||
|
||||
def _parse_media_command(self, line: str) -> list[ContentPart] | None:
|
||||
"""解析 ``/image``、``/file`` 命令;不是命令时返回 ``None``。
|
||||
|
||||
附件被拒(超限、不存在、类型不符)时返回一条说明文本而不是抛异常:
|
||||
CLI 里抛异常会直接终止输入泵,用户看到的是进程退出而不是提示。
|
||||
"""
|
||||
text = line.strip()
|
||||
if not text.startswith("/"):
|
||||
return None
|
||||
head, _, rest = text[1:].partition(" ")
|
||||
kind = _MEDIA_COMMANDS.get(head.lower())
|
||||
if kind is None:
|
||||
return None
|
||||
path_text = rest.strip()
|
||||
if not path_text:
|
||||
return [TextContent(text=f"用法:/{head} <本地文件路径>")]
|
||||
# 引号包裹的路径(含空格)直接剥掉两端引号
|
||||
path_text = path_text.strip("\"'")
|
||||
limits = self.media_limits
|
||||
try:
|
||||
path = Path(path_text).expanduser()
|
||||
limit = limits.image if kind == "image" else limits.file
|
||||
raw = read_file_limited(path, limit)
|
||||
except MediaRejected as exc:
|
||||
return [TextContent(text=f"附件未接收:{exc.player_reason}({exc.detail})")]
|
||||
mime = mimetypes.guess_type(path.name)[0] or "application/octet-stream"
|
||||
if kind == "image":
|
||||
if not mime.startswith("image/"):
|
||||
return [
|
||||
TextContent(
|
||||
text=f"附件未接收:不是图片类型({mime}),如为普通文件请用 /file"
|
||||
)
|
||||
]
|
||||
part: ContentPart = ImageContent(
|
||||
local_path=str(path),
|
||||
size=len(raw),
|
||||
mime_type=mime,
|
||||
alt_text=sanitize_filename(path.name),
|
||||
)
|
||||
else:
|
||||
part = FileContent(
|
||||
local_path=str(path),
|
||||
filename=sanitize_filename(path.name),
|
||||
size=len(raw),
|
||||
mime_type=mime,
|
||||
)
|
||||
return [part]
|
||||
|
||||
# ----- 平台出站原语 -----
|
||||
|
||||
async def _send_text(self, subject: ChannelSubject, text: str) -> None:
|
||||
@@ -133,9 +204,41 @@ class CliChannel(BaseChannel):
|
||||
await self._send_media(subject, part)
|
||||
|
||||
async def _send_media(self, subject: ChannelSubject, media: ContentPart) -> None:
|
||||
name = getattr(media, "filename", "") or getattr(media, "alt_text", "") or media.type
|
||||
url = getattr(media, "url", "") or getattr(media, "local_path", "")
|
||||
_safe_print(f"[媒体] {media.type}: {name} {url}".rstrip())
|
||||
"""CLI 无法真正投递媒体,改为回执:路径 + 大小 + sha256 前缀。
|
||||
|
||||
回执要能被人当场验证,所以不打印「已发送」——CLI 里没有发送这回事。
|
||||
存在性、大小上限与类型嗅探与其他渠道共用同一套策略;被拒时打印原因,
|
||||
不抛异常(异常会让出站链路把整条回复判失败)。
|
||||
"""
|
||||
_ = subject
|
||||
name = (
|
||||
getattr(media, "filename", "")
|
||||
or getattr(media, "alt_text", "")
|
||||
or media.type
|
||||
)
|
||||
limits = self.media_limits
|
||||
limit = limits.image if media.type == "image" else limits.file
|
||||
try:
|
||||
raw, _mime = await self.load_media_bytes(media)
|
||||
if len(raw) > limit:
|
||||
raise MediaRejected(
|
||||
"媒体超过大小上限",
|
||||
f"{format_size(len(raw))} > {format_size(limit)}",
|
||||
)
|
||||
except MediaRejected as exc:
|
||||
_safe_print(f"[媒体被拒] {media.type} {name}:{exc.player_reason}({exc.detail})")
|
||||
return
|
||||
except (OSError, ValueError, TypeError) as exc:
|
||||
# 下载/读取失败时只回执,不打断后续分片。这里不捕获 BaseException,
|
||||
# 也不吞 CancelledError——取消必须能穿过这一层。
|
||||
_safe_print(f"[媒体失败] {media.type} {name}:{type(exc).__name__}: {exc}")
|
||||
return
|
||||
digest = hashlib.sha256(raw).hexdigest()[:16]
|
||||
where = getattr(media, "local_path", "") or getattr(media, "url", "") or "(内存)"
|
||||
_safe_print(
|
||||
f"[媒体回执] {media.type}: {name} {format_size(len(raw))} "
|
||||
f"sha256:{digest} 来源 {where}"
|
||||
)
|
||||
|
||||
|
||||
register_channel("cli", CliChannel, CliConfig)
|
||||
@@ -20,6 +20,7 @@ from typing import Any, ClassVar
|
||||
import httpx
|
||||
|
||||
from bot042.channels.base import BaseChannel, ChannelCredentialsError
|
||||
from bot042.channels.media import MediaRejected, MediaTransform
|
||||
from bot042.channels.models import (
|
||||
AudioContent,
|
||||
ChannelConfig,
|
||||
@@ -408,6 +409,39 @@ class WeComChannel(BaseChannel):
|
||||
filename = f"{media_type}{extension}"
|
||||
return media_type, filename or default_name
|
||||
|
||||
# ----- 入站媒体解密 -----
|
||||
|
||||
def make_media_transform(self, metadata: dict[str, Any]) -> MediaTransform:
|
||||
"""按本条的 ``aeskey`` 构造解密钩子(密钥随消息而来,不挂在实例上)。
|
||||
|
||||
企业微信推送的媒体 URL 返回的是 AES-256-CBC 密文,密钥在消息体的
|
||||
``<media>.aeskey`` 里(``parse_inbound`` 记进 ``_media_aes_keys``)。
|
||||
不传密钥时 SDK 的 ``download_file`` 只会原样返回密文,因此"记录了
|
||||
aeskey 却不解密"等于入站图片全是不可用的字节。
|
||||
|
||||
解密用 SDK 自带的 ``decrypt_file``(IV 取密钥前 16 字节,PKCS#7 去填充),
|
||||
不在这里重新实现一遍密码学——那只会多出一份可能与平台不一致的实现。
|
||||
"""
|
||||
keys = metadata.get("_media_aes_keys") or {}
|
||||
if not isinstance(keys, dict) or not keys:
|
||||
return super().media_transform
|
||||
|
||||
def _decrypt(part: ContentPart, raw: bytes) -> bytes:
|
||||
url = getattr(part, "url", "") or ""
|
||||
aes_key = keys.get(url)
|
||||
if not aes_key:
|
||||
# 没有密钥就是没有密钥,不要猜:返回原字节,由 MIME 嗅探去判定
|
||||
return raw
|
||||
try:
|
||||
from wecom_aibot_sdk import decrypt_file
|
||||
except ImportError as exc: # pragma: no cover - SDK 是可选依赖
|
||||
raise MediaRejected(
|
||||
"媒体解密不可用", "需要 wecom-aibot-sdk(含 cryptography)"
|
||||
) from exc
|
||||
return decrypt_file(raw, aes_key)
|
||||
|
||||
return _decrypt
|
||||
|
||||
# ----- 主动推送路由辅助 -----
|
||||
|
||||
def _enrich_push_metadata(
|
||||
|
||||
@@ -0,0 +1,558 @@
|
||||
"""媒体安全策略:大小上限、MIME 校验、文件名净化与落盘隔离。
|
||||
|
||||
入站媒体在进入智能体上下文之前必须经过这一层,理由有三条:
|
||||
|
||||
1. **内存**:平台给的 URL 可能是任意大的文件,``resp.content`` 会把整份
|
||||
内容读进内存。这里一律按块读取,超限立即中断;
|
||||
2. **路径**:文件名来自平台,属于不可信输入,``../../.ssh/authorized_keys``
|
||||
这类名字必须净化后才能拼进落盘路径;
|
||||
3. **类型**:声明类型与实际字节可能不一致,需要按魔数嗅探交叉校验。
|
||||
|
||||
设计取舍(明确写出来,避免被当成完备防护):
|
||||
|
||||
- MIME 白名单是纵深防御,不是内容扫描:它只保证「不把明显不是图片的字节
|
||||
当图片处理」,不做病毒扫描、不做转码、不做像素级校验;
|
||||
- 可执行扩展名黑名单天然不完备(总会有新扩展名),只拦常见的那批;
|
||||
- 落盘后不做权限收紧(沿用进程 umask),隔离靠目录边界而非 DACL。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import binascii
|
||||
import hashlib
|
||||
import mimetypes
|
||||
import re
|
||||
import unicodedata
|
||||
import uuid
|
||||
from collections.abc import Callable
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Final
|
||||
|
||||
import httpx
|
||||
|
||||
from bot042.channels.models import (
|
||||
AudioContent,
|
||||
ContentPart,
|
||||
FileContent,
|
||||
ImageContent,
|
||||
TextContent,
|
||||
VideoContent,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"ALLOWED_AUDIO_MIME",
|
||||
"ALLOWED_IMAGE_MIME",
|
||||
"ALLOWED_VIDEO_MIME",
|
||||
"BLOCKED_EXTENSIONS",
|
||||
"MediaLimits",
|
||||
"MediaRejected",
|
||||
"MediaTransform",
|
||||
"StoredMedia",
|
||||
"check_mime",
|
||||
"decode_base64_limited",
|
||||
"download_limited",
|
||||
"format_size",
|
||||
"guess_mime",
|
||||
"read_file_limited",
|
||||
"safe_join",
|
||||
"sanitize_filename",
|
||||
"store_media",
|
||||
]
|
||||
|
||||
MediaTransform = Callable[[ContentPart, bytes], bytes]
|
||||
"""入站字节的转换钩子:``(分片, 原始字节) -> 明文字节``。
|
||||
|
||||
企业微信的媒体 URL 返回的是 AES 密文(见 ``wecom_aibot_sdk.crypto``),
|
||||
必须解密之后才能做类型嗅探与大小判定——在密文上嗅魔数只会得到
|
||||
``application/octet-stream``,从而把所有图片都判成「类型不被接受」。
|
||||
"""
|
||||
|
||||
MB: Final[int] = 1024 * 1024
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 类型白名单
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
ALLOWED_IMAGE_MIME: Final[frozenset[str]] = frozenset(
|
||||
{"image/png", "image/jpeg", "image/gif", "image/webp", "image/bmp"}
|
||||
)
|
||||
ALLOWED_AUDIO_MIME: Final[frozenset[str]] = frozenset(
|
||||
{
|
||||
"audio/amr",
|
||||
"audio/mpeg",
|
||||
"audio/wav",
|
||||
"audio/x-wav",
|
||||
"audio/ogg",
|
||||
"audio/mp4",
|
||||
"audio/aac",
|
||||
"audio/silk",
|
||||
}
|
||||
)
|
||||
ALLOWED_VIDEO_MIME: Final[frozenset[str]] = frozenset(
|
||||
{"video/mp4", "video/quicktime", "video/webm", "video/x-msvideo"}
|
||||
)
|
||||
|
||||
BLOCKED_EXTENSIONS: Final[frozenset[str]] = frozenset(
|
||||
{
|
||||
".exe", ".bat", ".cmd", ".com", ".scr", ".pif", ".msi", ".dll",
|
||||
".ps1", ".vbs", ".vbe", ".js", ".jse", ".wsf", ".hta", ".lnk",
|
||||
".jar", ".sh", ".bash",
|
||||
}
|
||||
)
|
||||
"""可执行/脚本类扩展名,入站与出站都拒绝。
|
||||
|
||||
这是**黑名单**,不完备;只拦常见的那批。真正的防线是「不自动执行内容」,
|
||||
而不是这张表。
|
||||
"""
|
||||
|
||||
_IMAGE_EXT: Final[dict[str, str]] = {
|
||||
".png": "image/png",
|
||||
".jpg": "image/jpeg",
|
||||
".jpeg": "image/jpeg",
|
||||
".gif": "image/gif",
|
||||
".webp": "image/webp",
|
||||
".bmp": "image/bmp",
|
||||
}
|
||||
_VIDEO_EXT: Final[dict[str, str]] = {
|
||||
".mp4": "video/mp4",
|
||||
".mov": "video/quicktime",
|
||||
".webm": "video/webm",
|
||||
".avi": "video/x-msvideo",
|
||||
}
|
||||
_AUDIO_EXT: Final[dict[str, str]] = {
|
||||
".amr": "audio/amr",
|
||||
".mp3": "audio/mpeg",
|
||||
".wav": "audio/wav",
|
||||
".ogg": "audio/ogg",
|
||||
".m4a": "audio/mp4",
|
||||
".aac": "audio/aac",
|
||||
".silk": "audio/silk",
|
||||
}
|
||||
|
||||
|
||||
def guess_mime(filename: str) -> str | None:
|
||||
"""按扩展名推断 MIME;未知返回 ``None``。"""
|
||||
ext = Path(filename).suffix.lower()
|
||||
for table in (_IMAGE_EXT, _VIDEO_EXT, _AUDIO_EXT):
|
||||
if ext in table:
|
||||
return table[ext]
|
||||
guessed, _ = mimetypes.guess_type(filename)
|
||||
return guessed
|
||||
|
||||
|
||||
def sniff_mime(head: bytes) -> str | None:
|
||||
"""按魔数嗅探 MIME;无法判定返回 ``None``。
|
||||
|
||||
只覆盖本项目实际会碰到的格式,不做通用实现。
|
||||
"""
|
||||
if head.startswith(b"\x89PNG\r\n\x1a\n"):
|
||||
return "image/png"
|
||||
if head.startswith(b"\xff\xd8\xff"):
|
||||
return "image/jpeg"
|
||||
if head[:6] in (b"GIF87a", b"GIF89a"):
|
||||
return "image/gif"
|
||||
if head.startswith(b"BM"):
|
||||
return "image/bmp"
|
||||
if head.startswith(b"RIFF") and len(head) >= 12:
|
||||
form = head[8:12]
|
||||
if form == b"WEBP":
|
||||
return "image/webp"
|
||||
if form == b"WAVE":
|
||||
return "audio/wav"
|
||||
if form == b"AVI ":
|
||||
return "video/x-msvideo"
|
||||
if head.startswith(b"#!AMR"):
|
||||
return "audio/amr"
|
||||
if head.startswith(b"OggS"):
|
||||
return "audio/ogg"
|
||||
if head.startswith(b"ID3") or head[:2] in (b"\xff\xfb", b"\xff\xf3"):
|
||||
return "audio/mpeg"
|
||||
if head.startswith(b"\x1aE\xdf\xa3"):
|
||||
return "video/webm"
|
||||
if len(head) >= 12 and head[4:8] == b"ftyp":
|
||||
brand = head[8:12]
|
||||
if brand.startswith(b"qt"):
|
||||
return "video/quicktime"
|
||||
return "video/mp4"
|
||||
if head.startswith(b"#!SILK"):
|
||||
return "audio/silk"
|
||||
return None
|
||||
|
||||
|
||||
def check_mime(part: ContentPart, declared: str, head: bytes = b"") -> str:
|
||||
"""校验媒体类型是否可接受,返回最终采信的 MIME。
|
||||
|
||||
采信顺序:魔数嗅探 > 声明类型 > 扩展名推断。三者都没有时按
|
||||
``application/octet-stream`` 处理——此时只有 :class:`FileContent`
|
||||
放行,图片/音视频一律拒绝(无法确认它真的是媒体)。
|
||||
|
||||
Raises:
|
||||
MediaRejected: 类型不在该类媒体允许的集合内,或扩展名被拉黑。
|
||||
"""
|
||||
sniffed = sniff_mime(head) if head else None
|
||||
candidate = (sniffed or declared or "").split(";")[0].strip().lower()
|
||||
|
||||
if isinstance(part, FileContent):
|
||||
name = part.filename or ""
|
||||
ext = Path(name).suffix.lower()
|
||||
if ext in BLOCKED_EXTENSIONS:
|
||||
raise MediaRejected("文件类型被拒绝", f"可执行/脚本扩展名:{ext}")
|
||||
return candidate or guess_mime(name) or "application/octet-stream"
|
||||
|
||||
if isinstance(part, ImageContent):
|
||||
allowed, label = ALLOWED_IMAGE_MIME, "图片"
|
||||
elif isinstance(part, AudioContent):
|
||||
allowed, label = ALLOWED_AUDIO_MIME, "音频"
|
||||
elif isinstance(part, VideoContent):
|
||||
allowed, label = ALLOWED_VIDEO_MIME, "视频"
|
||||
else: # pragma: no cover - ContentPart 联合已由类型系统约束
|
||||
raise MediaRejected("不支持的分片类型", type(part).__name__)
|
||||
|
||||
if candidate in allowed:
|
||||
return candidate
|
||||
fallback = guess_mime(getattr(part, "filename", "") or "")
|
||||
if fallback in allowed:
|
||||
return fallback
|
||||
raise MediaRejected(
|
||||
f"{label}类型不被接受",
|
||||
f"收到 {candidate or '未知'},允许:{sorted(allowed)}",
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 上限
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def format_size(num: int) -> str:
|
||||
"""人类可读的字节数(1 位小数)。"""
|
||||
value = float(num)
|
||||
for unit in ("B", "KB", "MB", "GB"):
|
||||
if value < 1024 or unit == "GB":
|
||||
return f"{value:.1f} {unit}" if unit != "B" else f"{int(value)} B"
|
||||
value /= 1024
|
||||
return f"{int(num)} B" # pragma: no cover
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class MediaLimits:
|
||||
"""各类媒体的字节上限与下载参数。
|
||||
|
||||
默认值是按「单进程内存占用 + 交互体验」权衡设定的**本项目策略值**,
|
||||
不是平台官方限制;部署时按实际内存调。
|
||||
"""
|
||||
|
||||
image: int = 10 * MB
|
||||
audio: int = 20 * MB
|
||||
video: int = 50 * MB
|
||||
file: int = 20 * MB
|
||||
chunk: int = 64 * 1024
|
||||
"""按块读取的块大小,决定下载时的内存粒度。"""
|
||||
timeout: float = 30.0
|
||||
"""单次远端下载的总超时(秒)。"""
|
||||
|
||||
def for_part(self, part: ContentPart) -> int:
|
||||
"""返回该分片适用的字节上限。"""
|
||||
if isinstance(part, ImageContent):
|
||||
return self.image
|
||||
if isinstance(part, AudioContent):
|
||||
return self.audio
|
||||
if isinstance(part, VideoContent):
|
||||
return self.video
|
||||
if isinstance(part, FileContent):
|
||||
return self.file
|
||||
raise TypeError(f"文本分片没有大小上限:{type(part).__name__}")
|
||||
|
||||
|
||||
class MediaRejected(ValueError):
|
||||
"""媒体被安全策略拒绝。
|
||||
|
||||
``reason`` 是给用户看的一句话,``detail`` 是给日志的技术细节。
|
||||
调用方应把 ``player_reason`` 降级成文本提示,而不是向上抛异常打断整轮回复。
|
||||
"""
|
||||
|
||||
def __init__(self, reason: str, detail: str = "") -> None:
|
||||
self.reason = reason
|
||||
self.detail = detail
|
||||
super().__init__(f"{reason}({detail})" if detail else reason)
|
||||
|
||||
@property
|
||||
def player_reason(self) -> str:
|
||||
"""面向用户的简短说明。"""
|
||||
return self.reason
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 受限读取
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def read_file_limited(path: Path, limit: int) -> bytes:
|
||||
"""读取本地文件,超过 ``limit`` 抛 :class:`MediaRejected`。
|
||||
|
||||
只多读 1 字节用于判定越界,不会把超大文件整个读进内存。
|
||||
"""
|
||||
if not path.exists():
|
||||
raise MediaRejected("文件不存在", str(path))
|
||||
if not path.is_file():
|
||||
raise MediaRejected("不是普通文件", str(path))
|
||||
with path.open("rb") as handle:
|
||||
data = handle.read(limit + 1)
|
||||
if len(data) > limit:
|
||||
# 只多读了 1 字节,所以这里只能说「大于」,说不出真实大小
|
||||
raise MediaRejected(
|
||||
"媒体超过大小上限", f"文件大于 {format_size(limit)}(上限 {format_size(limit)})"
|
||||
)
|
||||
return data
|
||||
|
||||
|
||||
def decode_base64_limited(data: str, limit: int) -> bytes:
|
||||
"""解码 base64 内联字节,超过 ``limit`` 抛 :class:`MediaRejected`。
|
||||
|
||||
先按 4/3 膨胀率粗判长度,避免为一个必然超限的字符串分配大缓冲。
|
||||
"""
|
||||
if len(data) > (limit * 4) // 3 + 16:
|
||||
raise MediaRejected("媒体超过大小上限", f"base64 长度 {len(data)}")
|
||||
try:
|
||||
raw = base64.b64decode(data, validate=False)
|
||||
except (binascii.Error, ValueError) as exc:
|
||||
raise MediaRejected("媒体 base64 解码失败", str(exc)) from exc
|
||||
if len(raw) > limit:
|
||||
raise MediaRejected(
|
||||
"媒体超过大小上限", f"{format_size(len(raw))} > {format_size(limit)}"
|
||||
)
|
||||
return raw
|
||||
|
||||
|
||||
async def download_limited(
|
||||
url: str,
|
||||
*,
|
||||
limit: int,
|
||||
chunk: int = 64 * 1024,
|
||||
timeout: float = 30.0,
|
||||
headers: dict[str, str] | None = None,
|
||||
) -> tuple[bytes, str]:
|
||||
"""流式下载远端媒体,``(bytes, content_type)``。
|
||||
|
||||
两道闸:``Content-Length`` 提前拒绝(省一次完整下载),以及按块累计的
|
||||
硬上限(服务端谎报长度或不分块时仍然生效)。
|
||||
|
||||
Raises:
|
||||
MediaRejected: 超限。
|
||||
httpx.HTTPError: 网络/状态码错误,由调用方决定是否降级。
|
||||
"""
|
||||
async with httpx.AsyncClient(
|
||||
follow_redirects=True, timeout=timeout, headers=headers
|
||||
) as client, client.stream("GET", url) as resp:
|
||||
resp.raise_for_status()
|
||||
declared = resp.headers.get("content-length", "")
|
||||
if declared.isdigit() and int(declared) > limit:
|
||||
raise MediaRejected(
|
||||
"媒体超过大小上限",
|
||||
f"Content-Length {format_size(int(declared))} > {format_size(limit)}",
|
||||
)
|
||||
buffer = bytearray()
|
||||
async for piece in resp.aiter_bytes(chunk):
|
||||
buffer.extend(piece)
|
||||
if len(buffer) > limit:
|
||||
raise MediaRejected(
|
||||
"媒体超过大小上限",
|
||||
f"下载超过 {format_size(limit)} 已中断",
|
||||
)
|
||||
content_type = (resp.headers.get("content-type") or "").split(";")[0].strip()
|
||||
return bytes(buffer), content_type or ""
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 文件名与落盘路径
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_UNSAFE_CHARS = re.compile(r"[^\w.\- ]+", re.UNICODE)
|
||||
_DOTS = re.compile(r"\.{2,}")
|
||||
|
||||
|
||||
def sanitize_filename(name: str, *, default: str = "media.bin", max_len: int = 120) -> str:
|
||||
"""把平台给的文件名净化成可安全落盘的单段名字。
|
||||
|
||||
处理顺序:NFKC 归一(同形字)→ 去掉路径成分 → 替换控制字符与分隔符 →
|
||||
合并连续点(``..`` 穿越)→ 去首尾空白与点 → 截断。
|
||||
|
||||
返回的值**不含目录分隔符**,仍须经 :func:`safe_join` 拼接。
|
||||
"""
|
||||
raw = unicodedata.normalize("NFKC", name or "")
|
||||
# 只取最后一段:同时覆盖 "/" 与 "\\"
|
||||
raw = raw.replace("\\", "/").split("/")[-1]
|
||||
raw = "".join(ch for ch in raw if ch.isprintable() and ch not in "\x00")
|
||||
raw = _UNSAFE_CHARS.sub("_", raw)
|
||||
raw = _DOTS.sub(".", raw).strip(" .")
|
||||
if not raw:
|
||||
return default
|
||||
if len(raw) > max_len:
|
||||
stem, dot, suffix = raw.rpartition(".")
|
||||
if dot and len(suffix) <= 10:
|
||||
keep = max(1, max_len - len(suffix) - 1)
|
||||
raw = f"{stem[:keep]}.{suffix}"
|
||||
else:
|
||||
raw = raw[:max_len]
|
||||
return raw or default
|
||||
|
||||
|
||||
def safe_join(root: Path, filename: str) -> Path:
|
||||
"""把单段文件名拼到 ``root`` 下,并确认解析结果仍在 ``root`` 内。
|
||||
|
||||
``root`` 不存在时按父目录解析——落盘前由调用方 ``mkdir``。
|
||||
|
||||
Raises:
|
||||
MediaRejected: 路径逃出 ``root``。
|
||||
"""
|
||||
base = root.resolve()
|
||||
name = sanitize_filename(filename)
|
||||
candidate = (base / name).resolve()
|
||||
if candidate != base and base not in candidate.parents:
|
||||
raise MediaRejected("落盘路径非法", f"{name!r} 解析到 {candidate}")
|
||||
if candidate == base:
|
||||
raise MediaRejected("落盘路径非法", f"{name!r} 解析为目录本身")
|
||||
return candidate
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 入站落盘
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class StoredMedia:
|
||||
"""一次入站落盘的结果。"""
|
||||
|
||||
part: ContentPart
|
||||
"""更新后的分片:``local_path`` 已填、``url``/``data`` 已清空。"""
|
||||
path: Path
|
||||
size: int
|
||||
mime: str
|
||||
sha256: str
|
||||
|
||||
|
||||
def _default_name_for(part: ContentPart, mime: str) -> str:
|
||||
"""没有文件名时按类型与 MIME 造一个。"""
|
||||
ext = mimetypes.guess_extension(mime) if mime else None
|
||||
if isinstance(part, ImageContent):
|
||||
return f"image{ext or '.bin'}"
|
||||
if isinstance(part, AudioContent):
|
||||
return f"audio{ext or '.bin'}"
|
||||
if isinstance(part, VideoContent):
|
||||
return f"video{ext or '.bin'}"
|
||||
if isinstance(part, FileContent) and part.filename:
|
||||
return part.filename
|
||||
return f"file{ext or '.bin'}"
|
||||
|
||||
|
||||
async def store_media(
|
||||
part: ContentPart,
|
||||
*,
|
||||
root: Path,
|
||||
limits: MediaLimits | None = None,
|
||||
session_key: str = "",
|
||||
headers: dict[str, str] | None = None,
|
||||
transform: MediaTransform | None = None,
|
||||
) -> StoredMedia:
|
||||
"""把媒体分片落盘到 ``root/<session>/``,返回更新后的分片。
|
||||
|
||||
字节来源优先级与 ``BaseChannel.load_media_bytes`` 一致
|
||||
(``data`` → ``local_path`` → ``url``),但每一步都带大小闸;
|
||||
落盘后分片只保留 ``local_path``,让下游(智能体、出站回显)共用一份字节。
|
||||
|
||||
Args:
|
||||
transform: 可选的字节转换钩子(如平台解密)。**在类型嗅探与大小判定
|
||||
之前**执行,因为对密文嗅魔数没有意义;转换后的明文字节会再跑一次
|
||||
上限检查——解密后的长度不可由密文长度保证(见下)。
|
||||
session_key: 会话子目录名(净化后使用)。
|
||||
headers: 下载远端媒体时附带的请求头。
|
||||
|
||||
Raises:
|
||||
MediaRejected: 超限、类型不符、路径非法、转换失败。
|
||||
httpx.HTTPError: 远端下载失败(调用方降级为文本提示)。
|
||||
|
||||
注意:``download_limited`` 的上限约束的是**收到的字节**。AES-CBC 解密后
|
||||
明文通常更短(去掉填充),但把「密文没超」当成「明文没超」是错的推理,
|
||||
所以转换后再量一次。
|
||||
"""
|
||||
limits = limits or MediaLimits()
|
||||
if isinstance(part, TextContent):
|
||||
raise TypeError("store_media 不能用于文本分片")
|
||||
|
||||
limit = limits.for_part(part)
|
||||
mime_declared = (part.mime_type or "").split(";")[0].strip().lower()
|
||||
name_hint = getattr(part, "filename", "") or ""
|
||||
|
||||
if part.data:
|
||||
raw = decode_base64_limited(part.data, limit)
|
||||
declared = mime_declared or guess_mime(name_hint) or ""
|
||||
elif part.local_path:
|
||||
source = Path(part.local_path)
|
||||
raw = read_file_limited(source, limit)
|
||||
declared = mime_declared or guess_mime(str(source)) or ""
|
||||
name_hint = name_hint or source.name
|
||||
elif part.url:
|
||||
raw, fetched = await download_limited(
|
||||
part.url,
|
||||
limit=limit,
|
||||
chunk=limits.chunk,
|
||||
timeout=limits.timeout,
|
||||
headers=headers,
|
||||
)
|
||||
declared = mime_declared or fetched or guess_mime(name_hint) or ""
|
||||
else:
|
||||
raise MediaRejected("媒体缺少字节来源", "data/local_path/url 三者皆空")
|
||||
|
||||
if transform is not None:
|
||||
try:
|
||||
raw = transform(part, raw)
|
||||
except MediaRejected:
|
||||
raise
|
||||
except Exception as exc: # 平台解密失败不应冒泡成未捕获异常
|
||||
raise MediaRejected("媒体解密失败", f"{type(exc).__name__}: {exc}") from exc
|
||||
if len(raw) > limit:
|
||||
raise MediaRejected(
|
||||
"媒体超过大小上限",
|
||||
f"解密后 {format_size(len(raw))} > {format_size(limit)}",
|
||||
)
|
||||
# 密文的 MIME 声明不可信:解密后按魔数重新判定,声明类型只作兜底
|
||||
declared = mime_declared or guess_mime(name_hint) or ""
|
||||
|
||||
mime = check_mime(part, declared, raw[:32])
|
||||
|
||||
folder = root
|
||||
if session_key:
|
||||
folder = root / sanitize_filename(session_key, default="session", max_len=60)
|
||||
folder.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
target = safe_join(
|
||||
folder,
|
||||
sanitize_filename(name_hint) if name_hint else _default_name_for(part, mime),
|
||||
)
|
||||
digest = hashlib.sha256(raw).hexdigest()
|
||||
# 同名不同内容时加哈希前缀,避免后到的覆盖先到的(也避免穿越后仍撞名)
|
||||
if target.exists():
|
||||
target = safe_join(folder, f"{digest[:12]}-{target.name}")
|
||||
|
||||
# 写临时文件再原子改名:中途失败不会留下半份字节。
|
||||
# 临时名带 uuid:同一会话并发落盘同名文件时,共用一个 ".part" 会互相截断。
|
||||
temp = target.with_name(f"{target.name}.{uuid.uuid4().hex[:8]}.part")
|
||||
try:
|
||||
temp.write_bytes(raw)
|
||||
temp.replace(target)
|
||||
finally:
|
||||
temp.unlink(missing_ok=True)
|
||||
|
||||
updated = part.model_copy(
|
||||
update={
|
||||
"local_path": str(target),
|
||||
"url": "",
|
||||
"data": None,
|
||||
"size": len(raw),
|
||||
"mime_type": mime,
|
||||
}
|
||||
)
|
||||
return StoredMedia(part=updated, path=target, size=len(raw), mime=mime, sha256=digest)
|
||||
@@ -268,6 +268,32 @@ class MessageEvent(BaseModel):
|
||||
"""ERROR 事件:携带错误消息。"""
|
||||
return cls(type=MessageEventType.ERROR, error=message)
|
||||
|
||||
@classmethod
|
||||
def content_event(cls, parts: list[ContentPart], **kwargs: Any) -> MessageEvent:
|
||||
"""MESSAGE 事件:一次投递任意分片组合(文本 + 图片/文件/音视频)。
|
||||
|
||||
出站管线(``BaseChannel._deliver_event``)已按「先文本、再逐个媒体」
|
||||
下发媒体分片,这里只是让调用方能真正构造出带媒体的 MESSAGE 事件——
|
||||
在此之前 :class:`MediaContent` 在 ``src/`` 内没有任何构造入口。
|
||||
|
||||
示例::
|
||||
|
||||
yield MessageEvent.content_event([
|
||||
TextContent(text="这是你要的图"),
|
||||
ImageContent(local_path="/tmp/a.png", mime_type="image/png"),
|
||||
])
|
||||
"""
|
||||
return cls(type=MessageEventType.MESSAGE, content=list(parts), **kwargs)
|
||||
|
||||
@classmethod
|
||||
def media_event(cls, media: MediaContent, caption: str = "", **kwargs: Any) -> MessageEvent:
|
||||
"""MESSAGE 事件:单个媒体分片,可选配一句说明文本。"""
|
||||
parts: list[ContentPart] = []
|
||||
if caption:
|
||||
parts.append(TextContent(text=caption))
|
||||
parts.append(media)
|
||||
return cls(type=MessageEventType.MESSAGE, content=parts, **kwargs)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 总线事件载荷
|
||||
|
||||
+27
-3
@@ -8,6 +8,13 @@ from pathlib import Path
|
||||
|
||||
from bot042.home import db_path, resolve_home
|
||||
|
||||
STEP_GRACE = 20
|
||||
"""图步数硬上限相对 ``max_steps`` 的余量。
|
||||
|
||||
留给「预算将尽」提示被遵守的空间:预算用尽与硬上限如果相等,撞上限的那一刻
|
||||
直接抛 ``GraphRecursionError``,模型连一句话都来不及说。
|
||||
"""
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Settings:
|
||||
@@ -21,10 +28,13 @@ class Settings:
|
||||
|
||||
model: str = "openai:gpt-4o-mini"
|
||||
max_steps: int = 200
|
||||
"""单次任务的图步数上限(recursion_limit)。
|
||||
"""单次任务的**软**步数预算。
|
||||
|
||||
联网检索类任务一轮要跑很多次工具调用(搜索 → 抓取 → 再搜索),50 步很容易
|
||||
在给出答案前耗尽并抛 ``GraphRecursionError``,因此默认给到 200。
|
||||
这是给模型看的预算:预算将尽时中间件会注入一条「立刻收尾」提示
|
||||
(见 :mod:`bot042.agent.middlewares.budget`),交给 LangGraph 的硬上限是
|
||||
:attr:`recursion_limit`(比它多 :data:`STEP_GRACE` 步)。联网检索类任务一轮
|
||||
要跑很多次工具调用(搜索 → 抓取 → 再搜索),50 步很容易在给出答案前耗尽,
|
||||
因此默认给到 200。
|
||||
"""
|
||||
log_level: str = "INFO"
|
||||
home: Path = field(default_factory=resolve_home)
|
||||
@@ -40,6 +50,20 @@ class Settings:
|
||||
"""home 目录下的 bot.db 路径。"""
|
||||
return db_path(self.home)
|
||||
|
||||
@property
|
||||
def recursion_limit(self) -> int:
|
||||
"""交给 LangGraph 的硬步数上限。
|
||||
|
||||
硬上限必须**大于**软预算:预算用尽时模型需要几步把结论写完,直接等值会
|
||||
让收尾提示和中断同时到达,退化回「抛异常、无结论」。
|
||||
"""
|
||||
return self.max_steps + STEP_GRACE
|
||||
|
||||
@property
|
||||
def agent_rounds(self) -> int:
|
||||
"""模型调用轮数预算:一次「模型 → 工具」往返约耗 2 个图步。"""
|
||||
return max(1, self.max_steps // 2)
|
||||
|
||||
@classmethod
|
||||
def from_env(cls) -> Settings:
|
||||
return cls(
|
||||
|
||||
@@ -100,7 +100,7 @@ _CHANNEL_SEEDS = [
|
||||
|
||||
_SETTING_SEEDS = [
|
||||
("agent.default_model", "openai:gpt-4o-mini", "默认模型"),
|
||||
("agent.max_steps", "200", "单次任务最大步数"),
|
||||
("agent.max_steps", "200", "单次任务步数预算"),
|
||||
("log.level", "INFO", "日志级别"),
|
||||
]
|
||||
|
||||
|
||||
@@ -8,11 +8,23 @@ from dataclasses import dataclass, field
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from langchain_core.messages import AIMessageChunk
|
||||
from langgraph.errors import GraphRecursionError
|
||||
|
||||
from bot042.agent import create_agent
|
||||
from bot042.agent.middlewares.steering import SteeringBuffer, bind, unbind
|
||||
from bot042.agent.middlewares.budget import StepBudget, StepBudgetExceeded
|
||||
from bot042.agent.middlewares.budget import bind as bind_budget
|
||||
from bot042.agent.middlewares.budget import unbind as unbind_budget
|
||||
from bot042.agent.middlewares.steering import SteeringBuffer
|
||||
from bot042.agent.middlewares.steering import bind as bind_steering
|
||||
from bot042.agent.middlewares.steering import unbind as unbind_steering
|
||||
from bot042.agent.outbox import MediaOutbox
|
||||
from bot042.agent.outbox import bind as bind_outbox
|
||||
from bot042.agent.outbox import unbind as unbind_outbox
|
||||
from bot042.config import Settings
|
||||
|
||||
TaskInput = str | list[dict[str, Any]]
|
||||
"""任务输入:纯文本,或 langchain 多模态内容块列表(图片 base64 + 文本)。"""
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from langgraph.graph.state import CompiledStateGraph
|
||||
|
||||
@@ -37,6 +49,18 @@ def _content_text(content: object) -> str:
|
||||
return ""
|
||||
|
||||
|
||||
def _preview(task: TaskInput) -> str:
|
||||
"""日志用的任务摘要:多模态时只取文本块,避免把 base64 打进日志。"""
|
||||
if isinstance(task, str):
|
||||
return task
|
||||
return _content_text(task)
|
||||
|
||||
|
||||
def _run_config(settings: Settings) -> dict[str, Any]:
|
||||
"""图调用配置:硬步数上限 = 软预算 + 余量(见 ``Settings.recursion_limit``)。"""
|
||||
return {"recursion_limit": settings.recursion_limit}
|
||||
|
||||
|
||||
def last_message_text(result: dict[str, Any]) -> str:
|
||||
"""从智能体最终状态中提取最后一条消息的文本(多模态只取文本块)。"""
|
||||
messages = result.get("messages") or []
|
||||
@@ -69,12 +93,13 @@ class AgentCache:
|
||||
|
||||
|
||||
async def run_task(
|
||||
task: str,
|
||||
task: TaskInput,
|
||||
settings: Settings | None = None,
|
||||
history: list[dict[str, str]] | None = None,
|
||||
*,
|
||||
cache: AgentCache | None = None,
|
||||
steering: SteeringBuffer | None = None,
|
||||
outbox: MediaOutbox | None = None,
|
||||
) -> dict[str, Any]:
|
||||
"""异步执行单个任务,返回智能体最终状态(含 messages)。
|
||||
|
||||
@@ -84,22 +109,35 @@ async def run_task(
|
||||
history: 历史消息(不含本轮)。
|
||||
cache: 智能体缓存;给定则复用其中的实例,缺省每次新建(一次性调用用)。
|
||||
steering: 插话缓冲;给定则该次运行期间的插话会在下一轮模型调用前注入。
|
||||
outbox: 出站媒体信箱;给定则本次运行中 ``send_image`` / ``send_file``
|
||||
登记的附件留在其中,由调用方取走下发。
|
||||
|
||||
Raises:
|
||||
StepBudgetExceeded: 图步数硬上限被耗尽(软预算的收尾提示没起作用)。
|
||||
"""
|
||||
settings = settings or Settings.from_env()
|
||||
agent = cache.get(settings) if cache is not None else create_agent(settings)
|
||||
messages = list(history or [])
|
||||
messages.append({"role": "user", "content": task})
|
||||
logger.debug("执行任务:%s", task)
|
||||
logger.debug("执行任务:%s", _preview(task))
|
||||
|
||||
token = bind(steering) if steering is not None else None
|
||||
token = bind_steering(steering) if steering is not None else None
|
||||
out_token = bind_outbox(outbox) if outbox is not None else None
|
||||
budget_token = bind_budget(StepBudget(settings.agent_rounds))
|
||||
try:
|
||||
return await agent.ainvoke(
|
||||
{"messages": messages},
|
||||
config={"recursion_limit": settings.max_steps},
|
||||
config=_run_config(settings),
|
||||
)
|
||||
except GraphRecursionError as exc:
|
||||
# 硬上限只是兜底:走到这里说明收尾提示没被遵守,或子智能体吃掉了步数
|
||||
raise StepBudgetExceeded(settings.max_steps) from exc
|
||||
finally:
|
||||
unbind_budget(budget_token)
|
||||
if out_token is not None:
|
||||
unbind_outbox(out_token)
|
||||
if token is not None:
|
||||
unbind(token)
|
||||
unbind_steering(token)
|
||||
|
||||
|
||||
@dataclass
|
||||
@@ -111,12 +149,13 @@ class _Round:
|
||||
|
||||
|
||||
async def stream_task(
|
||||
task: str,
|
||||
task: TaskInput,
|
||||
settings: Settings | None = None,
|
||||
history: list[dict[str, str]] | None = None,
|
||||
*,
|
||||
cache: AgentCache | None = None,
|
||||
steering: SteeringBuffer | None = None,
|
||||
outbox: MediaOutbox | None = None,
|
||||
) -> AsyncIterator[str]:
|
||||
"""流式执行任务,逐步产出**累积文本**(不是增量)。
|
||||
|
||||
@@ -133,20 +172,26 @@ async def stream_task(
|
||||
history: 历史消息(不含本轮)。
|
||||
cache: 智能体缓存;给定则复用其中的实例。
|
||||
steering: 插话缓冲;给定则该次运行期间的插话会在下一轮模型调用前注入。
|
||||
outbox: 出站媒体信箱;给定则登记其中(流式路径不重复发送)。
|
||||
|
||||
Raises:
|
||||
StepBudgetExceeded: 图步数硬上限被耗尽;调用方可用已产出的部分文本拼提示。
|
||||
"""
|
||||
settings = settings or Settings.from_env()
|
||||
agent = cache.get(settings) if cache is not None else create_agent(settings)
|
||||
messages = list(history or [])
|
||||
messages.append({"role": "user", "content": task})
|
||||
logger.debug("流式执行任务:%s", task)
|
||||
logger.debug("流式执行任务:%s", _preview(task))
|
||||
|
||||
token = bind(steering) if steering is not None else None
|
||||
token = bind_steering(steering) if steering is not None else None
|
||||
out_token = bind_outbox(outbox) if outbox is not None else None
|
||||
budget_token = bind_budget(StepBudget(settings.agent_rounds))
|
||||
rounds: dict[str, _Round] = {}
|
||||
order: list[str] = []
|
||||
try:
|
||||
async for chunk, _meta in agent.astream(
|
||||
{"messages": messages},
|
||||
config={"recursion_limit": settings.max_steps},
|
||||
config=_run_config(settings),
|
||||
stream_mode="messages",
|
||||
):
|
||||
# messages 模式同时会吐出 ToolMessage(工具返回值)与 HumanMessage,
|
||||
@@ -166,6 +211,12 @@ async def stream_task(
|
||||
visible = "".join(rounds[k].text for k in order if not rounds[k].tool)
|
||||
if visible:
|
||||
yield visible
|
||||
except GraphRecursionError as exc:
|
||||
# 已产出的文本由调用方持有(累积式产出),这里只把异常换成可读的错误类型
|
||||
raise StepBudgetExceeded(settings.max_steps) from exc
|
||||
finally:
|
||||
unbind_budget(budget_token)
|
||||
if out_token is not None:
|
||||
unbind_outbox(out_token)
|
||||
if token is not None:
|
||||
unbind(token)
|
||||
unbind_steering(token)
|
||||
+205
-23
@@ -16,17 +16,24 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from collections.abc import Sequence
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from bot042.agent.middlewares.budget import StepBudgetExceeded
|
||||
from bot042.agent.middlewares.steering import SteeringBuffer
|
||||
from bot042.agent.outbox import MediaOutbox
|
||||
from bot042.channels.manager import ChannelManager
|
||||
from bot042.channels.media import MediaLimits, MediaTransform
|
||||
from bot042.channels.models import (
|
||||
ChannelMessage,
|
||||
ContentPart,
|
||||
InboundMessage,
|
||||
OutboundContent,
|
||||
OutboundStream,
|
||||
OutboundText,
|
||||
TextContent,
|
||||
)
|
||||
from bot042.config import Settings
|
||||
from bot042.db import repo
|
||||
@@ -37,6 +44,7 @@ from bot042.flows.agent_task import (
|
||||
run_task,
|
||||
stream_task,
|
||||
)
|
||||
from bot042.flows.media_input import describe_outbound, prepare_inbound
|
||||
from bot042.flows.processor import (
|
||||
DEFAULT_HISTORY_CHARS,
|
||||
DEFAULT_HISTORY_LIMIT,
|
||||
@@ -62,10 +70,24 @@ ACK_TEXT = "收到,正在处理…"
|
||||
|
||||
ERROR_REPLY = "处理消息时出错,请稍后再试。"
|
||||
|
||||
BUDGET_REPLY = (
|
||||
"本轮执行步数用尽(上限 {steps} 步),我先停在这里。\n"
|
||||
"要继续的话,把需求拆小一点重新发一条,或者直接说「接着做」,我从当前进度往下推。"
|
||||
)
|
||||
"""步数用尽时的说明(接在已完成的部分回答之后)。"""
|
||||
|
||||
REPLY_META_KEYS = ("_frame", "response_url", "chat_id", "_ws_client")
|
||||
"""入站 metadata 中需要回传给渠道的回复上下文(IM 被动回复用)。"""
|
||||
|
||||
|
||||
def _with_notice(answer: str, aborted: StepBudgetExceeded | None) -> str:
|
||||
"""把「步数用尽」的说明接到部分回答后面;没有部分回答时只给说明。"""
|
||||
if aborted is None:
|
||||
return answer
|
||||
notice = BUDGET_REPLY.format(steps=aborted.max_steps)
|
||||
return f"{answer}\n\n{notice}" if answer.strip() else notice
|
||||
|
||||
|
||||
def reply_metadata(message: InboundMessage) -> dict[str, Any]:
|
||||
"""挑出入站消息里的回复上下文,随出站 RPC 回传给渠道。
|
||||
|
||||
@@ -129,6 +151,58 @@ class ChannelServices:
|
||||
return
|
||||
await self.channels.send_text(channel_id, subject_id, text)
|
||||
|
||||
async def send_content(
|
||||
self,
|
||||
channel_id: str,
|
||||
subject_id: str,
|
||||
parts: list[ContentPart],
|
||||
*,
|
||||
metadata: dict[str, Any] | None = None,
|
||||
) -> None:
|
||||
"""发送富内容(文本 + 媒体);出站管线按分片顺序下发。
|
||||
|
||||
媒体分片自带 ``local_path``(入站落盘或智能体工具产出),
|
||||
平台侧由 ``_send_media`` 上传后按原生消息投递。
|
||||
"""
|
||||
if self.hub is not None:
|
||||
_ = await self.hub.call(
|
||||
RPC_SEND,
|
||||
OutboundContent(
|
||||
channel_id=channel_id,
|
||||
subject_id=subject_id,
|
||||
parts=parts,
|
||||
metadata=dict(metadata or {}),
|
||||
),
|
||||
timeout=self.send_timeout,
|
||||
)
|
||||
return
|
||||
await self.channels.send_content(channel_id, subject_id, parts)
|
||||
|
||||
def media_policy(self, channel_id: str) -> tuple[MediaLimits, dict[str, str]]:
|
||||
"""渠道的媒体大小上限与下载请求头。
|
||||
|
||||
未知渠道(未注册或已下线)退回默认策略值:宁可让校验按默认限跑,
|
||||
也不要因为拿不到渠道就把这道关跳过。
|
||||
"""
|
||||
channel = self.channels.get_channel(channel_id)
|
||||
if channel is None:
|
||||
return MediaLimits(), {}
|
||||
return channel.media_limits, channel.media_download_headers()
|
||||
|
||||
def media_transform(
|
||||
self, channel_id: str, metadata: dict[str, Any]
|
||||
) -> MediaTransform | None:
|
||||
"""按本条消息的元数据取渠道的解密钩子(如企业微信的 ``aeskey``)。
|
||||
|
||||
密钥随消息而来,必须按条构造,不能挂在渠道实例上——挂实例会让并发处理
|
||||
的两条消息串用彼此的密钥。渠道未注册时返回 ``None``,即"不解密",
|
||||
由类型嗅探在明文上判定,不会因为查不到渠道就跳过后续校验。
|
||||
"""
|
||||
channel = self.channels.get_channel(channel_id)
|
||||
if channel is None:
|
||||
return None
|
||||
return channel.make_media_transform(metadata)
|
||||
|
||||
async def stream_text(
|
||||
self,
|
||||
channel_id: str,
|
||||
@@ -189,6 +263,61 @@ class ChannelServices:
|
||||
return ReplyStream(_send)
|
||||
|
||||
|
||||
async def _reply_with(
|
||||
services: ChannelServices,
|
||||
payload: ChannelMessage,
|
||||
meta: dict[str, Any],
|
||||
text: str,
|
||||
parts: list[ContentPart],
|
||||
) -> None:
|
||||
"""回复一条消息:有媒体分片就走富内容(文本在前),否则走纯文本。
|
||||
|
||||
文本与媒体分成两次发送,在平台上会变成两条消息;合成一次
|
||||
``send_content`` 后由出站管线按「先文本、再逐个媒体」的顺序下发。
|
||||
"""
|
||||
if parts:
|
||||
lead: list[ContentPart] = [TextContent(text=text)] if text else []
|
||||
await services.send_content(
|
||||
payload.channel_id,
|
||||
payload.subject_id,
|
||||
[*lead, *parts],
|
||||
metadata=meta,
|
||||
)
|
||||
return
|
||||
await services.send_text(payload.channel_id, payload.subject_id, text, metadata=meta)
|
||||
|
||||
|
||||
async def _persist_reply(
|
||||
services: ChannelServices,
|
||||
session_id: str,
|
||||
channel: str,
|
||||
text: str,
|
||||
parts: Sequence[ContentPart],
|
||||
*,
|
||||
empty_ok: bool = False,
|
||||
) -> None:
|
||||
"""把一轮助手回复写入会话历史:正文 + 附件清单(不存字节)。
|
||||
|
||||
``parts`` 非空时即使正文为空也落一行——「只发了张图」是真实发生过的
|
||||
一轮对话,丢掉它会让下一轮模型不知道自己发过什么,用户说「再发大点」
|
||||
时就没有可回指的对象。
|
||||
|
||||
``empty_ok=False`` 且两者都空时不写(纯占位回复 ``EMPTY_REPLY`` 不入库,
|
||||
与既有行为一致:那是给用户看的兜底文案,不是助手说的话)。
|
||||
"""
|
||||
summary = describe_outbound(parts)
|
||||
content = "\n".join(piece for piece in (text.strip(), summary) if piece)
|
||||
if not content and not empty_ok:
|
||||
return
|
||||
await repo.add_message(
|
||||
services.db,
|
||||
session_id=session_id,
|
||||
role="assistant",
|
||||
content=content,
|
||||
channel=channel,
|
||||
)
|
||||
|
||||
|
||||
class AgentReply(Task):
|
||||
"""渠道消息任务:读历史 → 调智能体 → 落库 → 回复。
|
||||
|
||||
@@ -205,6 +334,8 @@ class AgentReply(Task):
|
||||
super().__init__(context)
|
||||
self._steering = SteeringBuffer()
|
||||
self._steered: list[str] = []
|
||||
self._reported = False
|
||||
"""是否已自行回话并落库(失败时 ``on_error`` 不再发通用文案)。"""
|
||||
|
||||
@property
|
||||
def steering(self) -> SteeringBuffer:
|
||||
@@ -267,11 +398,25 @@ class AgentReply(Task):
|
||||
|
||||
message = payload.message
|
||||
meta = reply_metadata(message)
|
||||
text = message.text.strip()
|
||||
# 入站:附件先落盘再进上下文(小图内联为 base64,大图与音视频退化为
|
||||
# 「路径 + 元信息」);单个附件失败只记一条说明,不打断整轮。
|
||||
limits, headers = services.media_policy(payload.channel_id)
|
||||
prepared = await prepare_inbound(
|
||||
message,
|
||||
root=services.settings.home / "data" / "inbound",
|
||||
limits=limits,
|
||||
headers=headers,
|
||||
transform=services.media_transform(payload.channel_id, message.metadata),
|
||||
)
|
||||
stream = services.open_stream(payload.channel_id, payload.subject_id, meta)
|
||||
|
||||
if not text:
|
||||
reply = "暂仅支持文本消息" if message.has_media else EMPTY_REPLY
|
||||
if not prepared.has_content:
|
||||
# 有附件但一个都没接住:说清原因,别让用户以为是发了条空消息
|
||||
reply = (
|
||||
"附件未能接收:" + ";".join(prepared.notes)
|
||||
if prepared.notes
|
||||
else EMPTY_REPLY
|
||||
)
|
||||
if stream is not None:
|
||||
await stream.update(reply, force=True)
|
||||
await stream.finish(reply)
|
||||
@@ -281,6 +426,12 @@ class AgentReply(Task):
|
||||
)
|
||||
return
|
||||
|
||||
text = prepared.summary
|
||||
# 出站:本轮的信箱,工具(send_image / send_file)往里登记,跑完统一下发
|
||||
outbox = MediaOutbox(limits=limits)
|
||||
# 只有正文时发纯文本,带附件才发多模态块列表(见 PreparedInbound.is_multimodal)
|
||||
task_input = prepared.blocks if prepared.is_multimodal else text
|
||||
|
||||
session_id = payload.db_session_id
|
||||
channel = payload.channel_type or payload.channel_id
|
||||
await services.ensure_db()
|
||||
@@ -294,26 +445,40 @@ class AgentReply(Task):
|
||||
)
|
||||
|
||||
if stream is None:
|
||||
answer = ""
|
||||
aborted: StepBudgetExceeded | None = None
|
||||
try:
|
||||
result = await run_task(
|
||||
text,
|
||||
task_input,
|
||||
services.settings,
|
||||
history=history,
|
||||
cache=services.cache,
|
||||
steering=self._steering,
|
||||
outbox=outbox,
|
||||
)
|
||||
answer = last_message_text(result)
|
||||
except StepBudgetExceeded as exc:
|
||||
aborted = exc
|
||||
await self._persist_steered(services, session_id, channel)
|
||||
if answer:
|
||||
await repo.add_message(
|
||||
services.db,
|
||||
session_id=session_id,
|
||||
role="assistant",
|
||||
content=answer,
|
||||
channel=channel,
|
||||
reply = _with_notice(answer, aborted)
|
||||
# 智能体可能只发了媒体、没写正文:这时不必再补一句「(无输出)」
|
||||
pending = outbox.drain()
|
||||
# 媒体也要进历史:只发图不写字时正文为空,但这一轮确实产出过附件。
|
||||
# 先落库再下发——发送失败不该让下一轮失去「我发过什么」的上下文。
|
||||
await _persist_reply(
|
||||
services, session_id, channel, reply, pending, empty_ok=bool(pending)
|
||||
)
|
||||
await services.send_text(
|
||||
payload.channel_id, payload.subject_id, answer or EMPTY_REPLY, metadata=meta
|
||||
await _reply_with(
|
||||
services,
|
||||
payload,
|
||||
meta,
|
||||
reply or ("" if pending else EMPTY_REPLY),
|
||||
pending,
|
||||
)
|
||||
if aborted is not None:
|
||||
# 已回话并落库:任务仍记失败(步数用尽是真的没做完),但不再发通用错误
|
||||
self._reported = True
|
||||
raise aborted
|
||||
return
|
||||
|
||||
# 支持流式:先给回执,再边生成边覆盖同一条消息。
|
||||
@@ -321,32 +486,49 @@ class AgentReply(Task):
|
||||
await stream.update(ACK_TEXT, force=True)
|
||||
answer = ""
|
||||
first = True
|
||||
aborted = None
|
||||
try:
|
||||
async for partial in stream_task(
|
||||
text,
|
||||
task_input,
|
||||
services.settings,
|
||||
history=history,
|
||||
cache=services.cache,
|
||||
steering=self._steering,
|
||||
outbox=outbox,
|
||||
):
|
||||
answer = partial
|
||||
# 首段跳过节流:让用户在 ack 之后立刻看到内容开始出现
|
||||
await stream.update(partial, force=first)
|
||||
first = False
|
||||
except StepBudgetExceeded as exc:
|
||||
aborted = exc
|
||||
finally:
|
||||
await stream.finish(answer or EMPTY_REPLY)
|
||||
# 步数用尽时把说明补在已经流出的内容后面,不必再单独发一条
|
||||
final = _with_notice(answer, aborted)
|
||||
pending = outbox.drain()
|
||||
# 只有媒体没正文时传空串:ReplyStream.finish("") 不会发任何文本
|
||||
await stream.finish(final or ("" if pending else EMPTY_REPLY))
|
||||
await self._persist_steered(services, session_id, channel)
|
||||
if answer:
|
||||
await repo.add_message(
|
||||
services.db,
|
||||
session_id=session_id,
|
||||
role="assistant",
|
||||
content=answer,
|
||||
channel=channel,
|
||||
await _persist_reply(
|
||||
services, session_id, channel, final, pending, empty_ok=bool(pending)
|
||||
)
|
||||
if pending:
|
||||
# 流式通道只送文本,媒体只能补一条(文本此时已经定稿)
|
||||
await services.send_content(
|
||||
payload.channel_id, payload.subject_id, pending, metadata=meta
|
||||
)
|
||||
if aborted is not None:
|
||||
self._reported = True
|
||||
raise aborted
|
||||
|
||||
async def on_error(self, exc: BaseException) -> None:
|
||||
"""失败时尽量回一句可读提示(发送本身失败只记日志)。"""
|
||||
"""失败时尽量回一句可读提示(发送本身失败只记日志)。
|
||||
|
||||
已经自行回话并落库的失败(如步数用尽)跳过——否则用户会先收到准确说明,
|
||||
再收到一句没信息量的「处理消息时出错」。
|
||||
"""
|
||||
if self._reported:
|
||||
return
|
||||
services = self.deps
|
||||
payload = self.context.payload
|
||||
if not (isinstance(services, ChannelServices) and isinstance(payload, ChannelMessage)):
|
||||
|
||||
@@ -0,0 +1,305 @@
|
||||
"""入站媒体 → 智能体上下文:落盘、降级与多模态内容块。
|
||||
|
||||
这是 issue #1 断点 1 的实现层:媒体消息不再被"暂仅支持文本"挡在门外,
|
||||
而是按类型分三条路进入上下文:
|
||||
|
||||
==================== ==========================================================
|
||||
类型 进入上下文的方式
|
||||
==================== ==========================================================
|
||||
图片(小) ``image_url`` 块(data URI,base64 内联)
|
||||
图片(大/超量) 文本说明 + 落盘路径,模型可用文件工具自行读取
|
||||
文本类文件 抽取的正文(有字符上限),进 ``text`` 块
|
||||
音视频 / 二进制文件 元信息(类型、大小、时长)+ 落盘路径
|
||||
==================== ==========================================================
|
||||
|
||||
**为什么不一概内联**:base64 进上下文会按 ~1.33 倍膨胀,一张 10MB 的图足以
|
||||
吃光窗口;因此内联有独立的字节与数量上限(:class:`InlinePolicy`),超过就
|
||||
退化成"路径 + 元信息",把"读不读、怎么读"交给模型按需处理。
|
||||
|
||||
**失败一律降级,不上抛**:单个附件超限/类型拒绝/下载失败只记录一条
|
||||
``notes``,其余附件与该消息的文本照常进入上下文。用户看到的是部分可用,
|
||||
而不是整轮报错。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
from collections.abc import Sequence
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import httpx
|
||||
|
||||
from bot042.channels.media import (
|
||||
MB,
|
||||
MediaLimits,
|
||||
MediaRejected,
|
||||
MediaTransform,
|
||||
format_size,
|
||||
store_media,
|
||||
)
|
||||
from bot042.channels.models import (
|
||||
AudioContent,
|
||||
ContentPart,
|
||||
FileContent,
|
||||
ImageContent,
|
||||
InboundMessage,
|
||||
TextContent,
|
||||
VideoContent,
|
||||
)
|
||||
|
||||
__all__ = ["InlinePolicy", "PreparedInbound", "describe_outbound", "prepare_inbound"]
|
||||
|
||||
_TEXT_MIME_PREFIXES = ("text/",)
|
||||
_TEXT_EXTENSIONS = frozenset(
|
||||
{
|
||||
".txt", ".md", ".csv", ".tsv", ".json", ".yaml", ".yml", ".toml", ".ini",
|
||||
".log", ".py", ".js", ".ts", ".html", ".htm", ".xml", ".sql", ".sh",
|
||||
".cfg", ".conf", ".rst", ".tex",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class InlinePolicy:
|
||||
"""图片内联进上下文的策略。
|
||||
|
||||
``max_bytes`` 与 ``max_count`` 是**本项目策略值**,用于换取上下文可用性,
|
||||
不是平台限制。超出的图片退化为路径说明,功能不丢失。
|
||||
"""
|
||||
|
||||
max_bytes: int = 2 * MB
|
||||
max_count: int = 4
|
||||
max_text_chars: int = 20000
|
||||
"""文本类文件抽取的字符上限(超出截断并注明)。"""
|
||||
|
||||
|
||||
@dataclass
|
||||
class PreparedInbound:
|
||||
"""入站消息的智能体可用形态。"""
|
||||
|
||||
text: str
|
||||
"""纯文本部分(所有 ``TextContent`` 拼接)。"""
|
||||
blocks: list[dict[str, Any]] = field(default_factory=list)
|
||||
"""发给模型的内容块(langchain 多模态格式)。仅文本时为一个 text 块;
|
||||
文本与附件都没有时为空列表(不留占位块)。"""
|
||||
summary: str = ""
|
||||
"""用于落库的文本(正文 + 附件清单),不含字节。"""
|
||||
notes: list[str] = field(default_factory=list)
|
||||
"""降级说明(超限 / 类型拒绝 / 下载失败),会并入 summary。"""
|
||||
stored: list[Path] = field(default_factory=list)
|
||||
|
||||
@property
|
||||
def has_content(self) -> bool:
|
||||
"""是否拿到了任何可进上下文的内容(文本或附件)。
|
||||
|
||||
**不要**为了"总是有个块"而塞占位文本:占位块会让本属性恒为真,空消息
|
||||
就再也走不到拒绝分支,用户发个空白消息会白白烧掉一轮模型调用。
|
||||
"""
|
||||
return bool(self.text.strip()) or bool(self.blocks) or bool(self.stored)
|
||||
|
||||
@property
|
||||
def is_multimodal(self) -> bool:
|
||||
"""除正文块之外还有别的内容块(即确实带了附件)。
|
||||
|
||||
用来决定交给模型的是内容块列表还是纯文本:``len(blocks) > 1`` 这种判断
|
||||
在"只有一张图、没有正文"时会退化成纯文本,把附件整个丢掉。
|
||||
"""
|
||||
body_blocks = 1 if self.text else 0
|
||||
return len(self.blocks) > body_blocks
|
||||
|
||||
|
||||
def _is_text_file(part: FileContent) -> bool:
|
||||
mime = (part.mime_type or "").lower()
|
||||
if any(mime.startswith(p) for p in _TEXT_MIME_PREFIXES):
|
||||
return True
|
||||
return Path(part.filename or "").suffix.lower() in _TEXT_EXTENSIONS
|
||||
|
||||
|
||||
def _decode_text(raw: bytes, limit: int) -> str | None:
|
||||
"""尝试把字节当 UTF-8 文本解码;含 NUL 或解码失败时返回 ``None``。"""
|
||||
head = raw[:limit]
|
||||
if b"\x00" in head:
|
||||
return None
|
||||
try:
|
||||
return head.decode("utf-8")
|
||||
except UnicodeDecodeError:
|
||||
try:
|
||||
return head.decode("gb18030")
|
||||
except UnicodeDecodeError:
|
||||
return None
|
||||
|
||||
|
||||
def _media_label(part: ContentPart) -> str:
|
||||
if isinstance(part, ImageContent):
|
||||
return "图片"
|
||||
if isinstance(part, AudioContent):
|
||||
return "音频"
|
||||
if isinstance(part, VideoContent):
|
||||
return "视频"
|
||||
if isinstance(part, FileContent):
|
||||
return f"文件 {part.filename or ''}".strip()
|
||||
return "媒体"
|
||||
|
||||
|
||||
def _describe(part: ContentPart, size: int, mime: str, path: Path) -> str:
|
||||
"""生成给模型看的附件说明行(不含字节)。"""
|
||||
bits = [f"{_media_label(part)}", f"{format_size(size)}", mime or "未知类型"]
|
||||
duration = getattr(part, "duration", None)
|
||||
if isinstance(duration, int) and duration > 0:
|
||||
bits.append(f"{duration / 1000:.1f}s")
|
||||
return f"{' / '.join(bits)},已保存到 {path}"
|
||||
|
||||
|
||||
def _outbound_size(part: ContentPart) -> int:
|
||||
"""出站分片的字节数:优先模型声明的 ``size``,缺省按本地文件探测。
|
||||
|
||||
工具构造分片时总会填 ``size``;这里兜底是为了让手工构造的分片
|
||||
(测试、插件)也能拿到合理描述,而不是在历史里写「0 B」。
|
||||
"""
|
||||
declared = getattr(part, "size", None)
|
||||
if declared:
|
||||
return int(declared)
|
||||
path = getattr(part, "local_path", None)
|
||||
if path:
|
||||
try:
|
||||
return Path(path).stat().st_size
|
||||
except OSError:
|
||||
return 0
|
||||
return 0
|
||||
|
||||
|
||||
def describe_outbound(parts: Sequence[ContentPart]) -> str:
|
||||
"""把智能体发出的媒体整理成一行可入库的说明(只存引用,不存字节)。
|
||||
|
||||
格式与入站 :func:`prepare_inbound` 的 summary 对齐(``[附件] a;b``):
|
||||
历史里「用户发的图」和「机器人发的图」长得一样,下一轮模型才能对照。
|
||||
回指路径是本地文件而非平台地址——平台 `media_id` 有过期,
|
||||
而本地落盘路径在会话目录内可复现。
|
||||
|
||||
**刻意不写「已发送」**:记录在调用下发之前写入(发送失败时也要留住这轮
|
||||
上下文),此时还没有证据说它发出去了。历史只陈述「这轮产出过这个附件」。
|
||||
"""
|
||||
items: list[str] = []
|
||||
for part in parts:
|
||||
if isinstance(part, TextContent):
|
||||
continue
|
||||
bits = [_media_label(part), format_size(_outbound_size(part))]
|
||||
mime = getattr(part, "mime_type", None)
|
||||
if mime:
|
||||
bits.append(mime)
|
||||
duration = getattr(part, "duration", None)
|
||||
if isinstance(duration, int) and duration > 0:
|
||||
bits.append(f"{duration / 1000:.1f}s")
|
||||
head = " / ".join(bits)
|
||||
local = getattr(part, "local_path", None)
|
||||
items.append(f"{head}(本地 {local})" if local else head)
|
||||
if not items:
|
||||
return ""
|
||||
return "[附件] " + ";".join(items)
|
||||
|
||||
|
||||
async def prepare_inbound(
|
||||
message: InboundMessage,
|
||||
*,
|
||||
root: Path,
|
||||
limits: MediaLimits | None = None,
|
||||
policy: InlinePolicy | None = None,
|
||||
headers: dict[str, str] | None = None,
|
||||
transform: MediaTransform | None = None,
|
||||
) -> PreparedInbound:
|
||||
"""把入站消息整理成 ``(text, blocks, summary, notes)``,媒体先落盘。
|
||||
|
||||
Args:
|
||||
message: 入站消息。
|
||||
root: 媒体落盘根目录(按会话建子目录)。
|
||||
limits: 字节上限;缺省 :class:`MediaLimits`。
|
||||
policy: 图片内联策略;缺省 :class:`InlinePolicy`。
|
||||
headers: 下载远端媒体时附带的请求头(平台鉴权由渠道提供)。
|
||||
transform: 平台字节转换钩子(如企业微信的 AES 解密),在类型嗅探与
|
||||
大小判定之前执行,由渠道按单条消息的 metadata 构造。
|
||||
|
||||
Returns:
|
||||
:class:`PreparedInbound`。附件层面的失败只进 ``notes``,不抛异常。
|
||||
"""
|
||||
limits = limits or MediaLimits()
|
||||
policy = policy or InlinePolicy()
|
||||
|
||||
text = message.text.strip()
|
||||
prepared = PreparedInbound(text=text)
|
||||
blocks: list[dict[str, Any]] = []
|
||||
descriptions: list[str] = []
|
||||
inline_images = 0
|
||||
session_key = message.db_session_id
|
||||
|
||||
for part in message.content:
|
||||
if isinstance(part, TextContent):
|
||||
continue
|
||||
try:
|
||||
stored = await store_media(
|
||||
part,
|
||||
root=root,
|
||||
limits=limits,
|
||||
session_key=session_key,
|
||||
headers=headers,
|
||||
transform=transform,
|
||||
)
|
||||
except MediaRejected as exc:
|
||||
prepared.notes.append(f"{_media_label(part)}未接收:{exc.reason}")
|
||||
continue
|
||||
except (httpx.HTTPError, OSError) as exc:
|
||||
prepared.notes.append(
|
||||
f"{_media_label(part)}未接收:下载失败({type(exc).__name__})"
|
||||
)
|
||||
continue
|
||||
|
||||
prepared.stored.append(stored.path)
|
||||
description = _describe(part, stored.size, stored.mime, stored.path)
|
||||
descriptions.append(description)
|
||||
|
||||
if isinstance(stored.part, ImageContent) and stored.mime.startswith("image/"):
|
||||
if stored.size <= policy.max_bytes and inline_images < policy.max_count:
|
||||
payload = base64.b64encode(stored.path.read_bytes()).decode("ascii")
|
||||
blocks.append(
|
||||
{
|
||||
"type": "image_url",
|
||||
"image_url": {"url": f"data:{stored.mime};base64,{payload}"},
|
||||
}
|
||||
)
|
||||
inline_images += 1
|
||||
else:
|
||||
blocks.append(
|
||||
{"type": "text", "text": f"[图片未内联] {description}(可用文件工具读取)"}
|
||||
)
|
||||
continue
|
||||
|
||||
if isinstance(stored.part, FileContent) and _is_text_file(stored.part):
|
||||
raw = stored.path.read_bytes()[: policy.max_text_chars * 4]
|
||||
decoded = _decode_text(raw, policy.max_text_chars * 4)
|
||||
if decoded:
|
||||
clipped = decoded[: policy.max_text_chars]
|
||||
suffix = "" if len(decoded) <= policy.max_text_chars else "\n…(内容已截断)"
|
||||
blocks.append(
|
||||
{
|
||||
"type": "text",
|
||||
"text": f"【附件 {stored.part.filename or stored.path.name}】\n{clipped}{suffix}",
|
||||
}
|
||||
)
|
||||
continue
|
||||
|
||||
blocks.append({"type": "text", "text": f"【附件】{description}(可用文件工具读取)"})
|
||||
|
||||
# 正文块一定排在最前:多数模型对首块是文本的提示更稳
|
||||
if text:
|
||||
blocks.insert(0, {"type": "text", "text": text})
|
||||
# 无任何内容时 blocks 保持为空:占位块会让 has_content 恒真,见该属性的说明
|
||||
prepared.blocks = blocks
|
||||
|
||||
summary_parts = [text] if text else []
|
||||
if descriptions:
|
||||
summary_parts.append("[附件] " + ";".join(descriptions))
|
||||
if prepared.notes:
|
||||
summary_parts.append("[降级] " + ";".join(prepared.notes))
|
||||
prepared.summary = "\n".join(p for p in summary_parts if p).strip()
|
||||
return prepared
|
||||
@@ -27,6 +27,7 @@ from bot042.config import Settings
|
||||
from bot042.db import repo
|
||||
from bot042.db.schema import init_db
|
||||
from bot042.flows.agent_task import AgentCache, last_message_text, run_task
|
||||
from bot042.flows.media_input import prepare_inbound
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -89,6 +90,7 @@ def make_processor(
|
||||
"""
|
||||
settings = settings or Settings.from_env()
|
||||
db_file = Path(db) if db is not None else settings.database
|
||||
media_root = settings.home / "data" / "inbound"
|
||||
db_ready = False
|
||||
# 智能体按处理器复用:图构建开销大,每条消息重建是纯浪费
|
||||
agent_cache = AgentCache()
|
||||
@@ -97,10 +99,17 @@ def make_processor(
|
||||
nonlocal db_ready
|
||||
yield MessageEvent.typing()
|
||||
|
||||
text = message.text.strip()
|
||||
if not text:
|
||||
yield MessageEvent.error_event("(空消息,已忽略)" if not message.has_media else "暂仅支持文本消息")
|
||||
# 媒体先落盘再进上下文:小图内联、大图与音视频退化为「路径 + 元信息」。
|
||||
# 附件级失败只降级为说明,不打断整轮(见 media_input.prepare_inbound)。
|
||||
prepared = await prepare_inbound(message, root=media_root)
|
||||
if not prepared.has_content:
|
||||
# 有附件但全部没接收成功:说清原因,别让用户以为是「发了条空消息」
|
||||
if prepared.notes:
|
||||
yield MessageEvent.error_event("附件未能接收:" + ";".join(prepared.notes))
|
||||
else:
|
||||
yield MessageEvent.error_event("(空消息,已忽略)")
|
||||
return
|
||||
text = prepared.summary
|
||||
|
||||
session_id, channel = _session_key(message)
|
||||
try:
|
||||
@@ -115,7 +124,12 @@ def make_processor(
|
||||
await repo.add_message(
|
||||
db_file, session_id=session_id, role="user", content=text, channel=channel
|
||||
)
|
||||
result = await run_task(text, settings, history=history, cache=agent_cache)
|
||||
# 只有正文时发纯文本,带附件才发多模态块列表;用 blocks 数量猜会
|
||||
# 在「只有一张图、没有正文」时把附件丢掉(见 PreparedInbound.is_multimodal)
|
||||
task_input = prepared.blocks if prepared.is_multimodal else text
|
||||
result = await run_task(
|
||||
task_input, settings, history=history, cache=agent_cache
|
||||
)
|
||||
answer = last_message_text(result)
|
||||
if answer:
|
||||
await repo.add_message(
|
||||
|
||||
@@ -0,0 +1,139 @@
|
||||
"""步数预算测试:预算换算、收尾提示注入、硬上限转成可读失败。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
from typing import Any
|
||||
|
||||
import pytest
|
||||
from langgraph.errors import GraphRecursionError
|
||||
|
||||
from bot042.agent.middlewares.budget import (
|
||||
BUDGET_PREFIX,
|
||||
BUDGET_WARN_MARGIN,
|
||||
StepBudget,
|
||||
StepBudgetExceeded,
|
||||
StepBudgetMiddleware,
|
||||
bind,
|
||||
current_budget,
|
||||
unbind,
|
||||
)
|
||||
from bot042.config import STEP_GRACE, Settings
|
||||
from bot042.flows import agent_task as agent_task_module
|
||||
from bot042.flows.agent_task import run_task, stream_task
|
||||
|
||||
|
||||
class RecordingAgent:
|
||||
"""记录调用配置,并顺带把当时绑定的预算读出来。"""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.config: dict[str, Any] | None = None
|
||||
self.remaining: int | None = None
|
||||
|
||||
async def ainvoke(
|
||||
self, state: dict[str, Any], config: dict[str, Any] | None = None
|
||||
) -> dict[str, Any]:
|
||||
_ = state
|
||||
self.config = config
|
||||
budget = current_budget()
|
||||
self.remaining = budget.remaining if budget is not None else None
|
||||
return {"messages": []}
|
||||
|
||||
|
||||
class RecursionBoomAgent:
|
||||
"""模拟 LangGraph 撞上硬上限。"""
|
||||
|
||||
async def ainvoke(
|
||||
self, state: dict[str, Any], config: dict[str, Any] | None = None
|
||||
) -> dict[str, Any]:
|
||||
_ = state, config
|
||||
raise GraphRecursionError("Recursion limit of 27 reached")
|
||||
|
||||
async def astream(self, *args: Any, **kwargs: Any) -> Any:
|
||||
_ = args, kwargs
|
||||
raise GraphRecursionError("Recursion limit of 27 reached")
|
||||
yield # pragma: no cover - 让本方法成为异步生成器
|
||||
|
||||
|
||||
def test_settings_derive_recursion_limit_and_rounds() -> None:
|
||||
"""硬上限必须大于软预算,轮数预算按「一次往返约 2 步」换算。"""
|
||||
settings = Settings(max_steps=200)
|
||||
assert settings.recursion_limit == 200 + STEP_GRACE
|
||||
assert settings.agent_rounds == 100
|
||||
# 极小预算也要留出至少一轮,否则中间件永远来不及提醒
|
||||
assert Settings(max_steps=1).agent_rounds == 1
|
||||
|
||||
|
||||
def test_budget_take_counts_down() -> None:
|
||||
budget = StepBudget(rounds=2)
|
||||
assert budget.take() == 1
|
||||
assert budget.take() == 0
|
||||
assert budget.take() == 0 # 不为负
|
||||
assert budget.used == 3
|
||||
assert budget.remaining == 0
|
||||
|
||||
|
||||
def test_middleware_noop_without_budget() -> None:
|
||||
middleware = StepBudgetMiddleware()
|
||||
assert asyncio.run(middleware.abefore_model({}, None)) is None # type: ignore[arg-type]
|
||||
|
||||
|
||||
def test_middleware_warns_once_when_budget_runs_low() -> None:
|
||||
budget = StepBudget(rounds=BUDGET_WARN_MARGIN + 2)
|
||||
token = bind(budget)
|
||||
try:
|
||||
middleware = StepBudgetMiddleware()
|
||||
# 剩余 5 → 安静;剩余 4 → 注入收尾提示
|
||||
assert asyncio.run(middleware.abefore_model({}, None)) is None # type: ignore[arg-type]
|
||||
update = asyncio.run(middleware.abefore_model({}, None)) # type: ignore[arg-type]
|
||||
assert update is not None
|
||||
message = update["messages"][0]
|
||||
assert isinstance(message.content, str)
|
||||
assert message.content.startswith(BUDGET_PREFIX)
|
||||
assert "剩余约 4 轮" in message.content
|
||||
assert budget.warned is True
|
||||
# 只提醒一次:反复打断只会让模型更答不完
|
||||
assert asyncio.run(middleware.abefore_model({}, None)) is None # type: ignore[arg-type]
|
||||
finally:
|
||||
unbind(token)
|
||||
|
||||
|
||||
def test_run_task_passes_derived_limit_and_binds_budget(monkeypatch) -> None:
|
||||
agent = RecordingAgent()
|
||||
monkeypatch.setattr(
|
||||
agent_task_module, "create_agent", lambda settings=None: agent
|
||||
)
|
||||
|
||||
settings = Settings(max_steps=7)
|
||||
asyncio.run(run_task("hi", settings))
|
||||
|
||||
assert agent.config == {"recursion_limit": 7 + STEP_GRACE}
|
||||
assert agent.remaining == settings.agent_rounds # 预算已绑定且未被计数
|
||||
# 运行结束必须解绑,否则下一个任务会读到上一轮的预算
|
||||
assert current_budget() is None
|
||||
|
||||
|
||||
def test_run_task_translates_recursion_error(monkeypatch) -> None:
|
||||
monkeypatch.setattr(
|
||||
agent_task_module, "create_agent", lambda settings=None: RecursionBoomAgent()
|
||||
)
|
||||
|
||||
with pytest.raises(StepBudgetExceeded) as info:
|
||||
asyncio.run(run_task("hi", Settings(max_steps=7)))
|
||||
assert info.value.max_steps == 7
|
||||
assert "max_steps=7" in str(info.value)
|
||||
assert current_budget() is None
|
||||
|
||||
|
||||
def test_stream_task_translates_recursion_error(monkeypatch) -> None:
|
||||
monkeypatch.setattr(
|
||||
agent_task_module, "create_agent", lambda settings=None: RecursionBoomAgent()
|
||||
)
|
||||
|
||||
async def _consume() -> None:
|
||||
async for _ in stream_task("hi", Settings(max_steps=7)):
|
||||
pass
|
||||
|
||||
with pytest.raises(StepBudgetExceeded):
|
||||
asyncio.run(_consume())
|
||||
assert current_budget() is None
|
||||
+370
-3
@@ -3,25 +3,35 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import base64
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import pytest
|
||||
|
||||
from bot042.agent.middlewares.budget import StepBudgetExceeded
|
||||
from bot042.channels.base import BaseChannel
|
||||
from bot042.channels.manager import ChannelManager
|
||||
from bot042.channels.models import (
|
||||
ChannelMessage,
|
||||
ChannelSubject,
|
||||
ImageContent,
|
||||
InboundMessage,
|
||||
TextContent,
|
||||
)
|
||||
from bot042.config import Settings
|
||||
from bot042.db import repo
|
||||
from bot042.flows import builtin
|
||||
from bot042.flows.builtin import ERROR_REPLY, RPC_SEND, AgentReply, ChannelServices
|
||||
from bot042.flows.builtin import (
|
||||
BUDGET_REPLY,
|
||||
ERROR_REPLY,
|
||||
RPC_SEND,
|
||||
AgentReply,
|
||||
ChannelServices,
|
||||
)
|
||||
from bot042.flows.bus import EventBus
|
||||
from bot042.flows.media_input import PreparedInbound
|
||||
from bot042.flows.rpc import RpcHub
|
||||
from bot042.flows.scheduler import CN_TZ
|
||||
from bot042.flows.tasks import EventTrigger, RunStatus, TaskContext, TaskRunner
|
||||
@@ -35,6 +45,8 @@ class EchoChannel(BaseChannel):
|
||||
def __init__(self) -> None:
|
||||
super().__init__(channel_id="c1")
|
||||
self.sent: list[str] = []
|
||||
self.contents: list[list[Any]] = []
|
||||
"""每次富内容发送的分片组合(文本 + 媒体)。"""
|
||||
|
||||
async def start(self) -> None:
|
||||
pass
|
||||
@@ -53,7 +65,7 @@ class EchoChannel(BaseChannel):
|
||||
self.sent.append(text)
|
||||
|
||||
async def _send_content(self, subject: ChannelSubject, parts: list) -> None:
|
||||
pass
|
||||
self.contents.append(list(parts))
|
||||
|
||||
async def _send_media(self, subject: ChannelSubject, media: Any) -> None:
|
||||
pass
|
||||
@@ -112,7 +124,7 @@ def test_agent_reply_persists_and_replies_via_rpc(
|
||||
calls: list[tuple[str, list]] = []
|
||||
|
||||
async def fake_run_task(
|
||||
task, settings=None, history=None, *, cache=None, steering=None
|
||||
task, settings=None, history=None, *, cache=None, steering=None, outbox=None
|
||||
):
|
||||
calls.append((task, list(history or [])))
|
||||
return {"messages": [{"role": "assistant", "content": "答:" + task}]}
|
||||
@@ -209,3 +221,358 @@ def test_agent_reply_sends_error_hint_on_failure(
|
||||
sent, statuses = asyncio.run(_run())
|
||||
assert sent == [ERROR_REPLY]
|
||||
assert statuses == [RunStatus.FAILED]
|
||||
|
||||
|
||||
def test_agent_reply_explains_step_budget_without_generic_error(
|
||||
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
|
||||
) -> None:
|
||||
"""步数用尽:给出可操作说明并落库,且不再补发通用错误文案。"""
|
||||
|
||||
async def exhausted(
|
||||
task, settings=None, history=None, *, cache=None, steering=None, outbox=None
|
||||
):
|
||||
raise StepBudgetExceeded(200)
|
||||
|
||||
monkeypatch.setattr(builtin, "run_task", exhausted)
|
||||
|
||||
async def _run() -> tuple[list[str], list[RunStatus]]:
|
||||
bus = EventBus()
|
||||
bus.bind_loop()
|
||||
hub = RpcHub(bus)
|
||||
manager = ChannelManager()
|
||||
channel = EchoChannel()
|
||||
await manager.add_channel(channel)
|
||||
await manager.start()
|
||||
_ = hub.serve(RPC_SEND, manager.rpc_send)
|
||||
|
||||
services = ChannelServices(
|
||||
channels=manager, settings=Settings(), db=tmp_path / "bot.db", hub=hub
|
||||
)
|
||||
runner = TaskRunner(bus, deps=services)
|
||||
runner.register(AgentReply)
|
||||
_ = runner.add_task(AgentReply.name, EventTrigger(event="channel.inbound"))
|
||||
await runner.start()
|
||||
|
||||
bus.emit("channel.inbound", _envelope("把整站重构一遍"))
|
||||
await runner.wait_idle(timeout=2)
|
||||
await bus.wait_for_complete()
|
||||
statuses = [run.status for run in runner.runs(AgentReply.name)]
|
||||
await runner.stop()
|
||||
hub.close()
|
||||
await manager.stop()
|
||||
return channel.sent, statuses
|
||||
|
||||
sent, statuses = asyncio.run(_run())
|
||||
notice = BUDGET_REPLY.format(steps=200)
|
||||
assert sent == [notice] # 只有说明,没有多余的「处理消息时出错」
|
||||
assert statuses == [RunStatus.FAILED]
|
||||
# 失败轮也要落库:用户已经看到这段内容,历史缺了它会前后不接
|
||||
rows = asyncio.run(repo.list_messages(tmp_path / "bot.db", session_id="s1"))
|
||||
assert [(r["role"], r["content"]) for r in rows] == [
|
||||
("user", "把整站重构一遍"),
|
||||
("assistant", notice),
|
||||
]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 媒体链路:入站进上下文、出站经信箱下发
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
PNG_HEAD = b"\x89PNG\r\n\x1a\n" + b"\x00\x00\x00\rIHDR" + b"\x00" * 16
|
||||
"""最小 PNG 头:magic + IHDR 片段,足够 check_mime 认定成 image/png。"""
|
||||
|
||||
|
||||
def _image_part() -> ImageContent:
|
||||
"""入站图片分片:base64 直接带字节,不依赖网络与临时文件。"""
|
||||
return ImageContent(
|
||||
data=base64.b64encode(PNG_HEAD).decode("ascii"),
|
||||
mime_type="image/png",
|
||||
size=len(PNG_HEAD),
|
||||
)
|
||||
|
||||
|
||||
def _media_envelope(*parts: Any, text: str = "") -> ChannelMessage:
|
||||
"""构造带任意内容分片的入站消息。"""
|
||||
content = ([TextContent(text=text)] if text else []) + list(parts)
|
||||
message = InboundMessage(
|
||||
channel_id="c1",
|
||||
channel_type="echo",
|
||||
channel_session_id="s1",
|
||||
channel_subject=ChannelSubject(subject_id="u1"),
|
||||
content=content,
|
||||
)
|
||||
return ChannelMessage.of(message, session_key="s1")
|
||||
|
||||
|
||||
def _drive(
|
||||
db: Path,
|
||||
payloads: list[ChannelMessage],
|
||||
channel: EchoChannel,
|
||||
*,
|
||||
home: Path,
|
||||
) -> list[RunStatus]:
|
||||
"""跑一遍 AgentReply(调用方先 monkeypatch run_task / prepare_inbound)。
|
||||
|
||||
``home`` 指向临时目录,避免把测试媒体写进真实的 ``~/.bot042/data``。
|
||||
"""
|
||||
|
||||
async def _run() -> list[RunStatus]:
|
||||
bus = EventBus()
|
||||
bus.bind_loop()
|
||||
hub = RpcHub(bus)
|
||||
manager = ChannelManager()
|
||||
await manager.add_channel(channel)
|
||||
await manager.start()
|
||||
_ = hub.serve(RPC_SEND, manager.rpc_send)
|
||||
|
||||
services = ChannelServices(
|
||||
channels=manager,
|
||||
settings=Settings(home=home),
|
||||
db=db,
|
||||
hub=hub,
|
||||
)
|
||||
runner = TaskRunner(bus, deps=services)
|
||||
runner.register(AgentReply)
|
||||
_ = runner.add_task(AgentReply.name, EventTrigger(event="channel.inbound"))
|
||||
await runner.start()
|
||||
|
||||
for payload in payloads:
|
||||
bus.emit("channel.inbound", payload)
|
||||
await runner.wait_idle(timeout=2)
|
||||
await bus.wait_for_complete()
|
||||
statuses = [run.status for run in runner.runs(AgentReply.name)]
|
||||
await runner.stop()
|
||||
hub.close()
|
||||
await manager.stop()
|
||||
return statuses
|
||||
|
||||
return asyncio.run(_run())
|
||||
|
||||
|
||||
def test_agent_reply_feeds_inbound_image_to_agent(
|
||||
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
|
||||
) -> None:
|
||||
"""文本 + 图片:两者都进上下文,图片走 image_url 且不入库字节。"""
|
||||
seen: list[Any] = []
|
||||
|
||||
async def fake_run_task(
|
||||
task, settings=None, history=None, *, cache=None, steering=None, outbox=None
|
||||
):
|
||||
seen.append(task)
|
||||
return {"messages": [{"role": "assistant", "content": "图里是一只猫"}]}
|
||||
|
||||
monkeypatch.setattr(builtin, "run_task", fake_run_task)
|
||||
channel = EchoChannel()
|
||||
statuses = _drive(
|
||||
tmp_path / "bot.db",
|
||||
[_media_envelope(_image_part(), text="这是什么")],
|
||||
channel,
|
||||
home=tmp_path,
|
||||
)
|
||||
|
||||
assert statuses == [RunStatus.SUCCEEDED]
|
||||
blocks = seen[0]
|
||||
assert isinstance(blocks, list)
|
||||
assert [b["type"] for b in blocks] == ["text", "image_url"]
|
||||
assert blocks[0]["text"] == "这是什么"
|
||||
assert blocks[1]["image_url"]["url"].startswith("data:image/png;base64,")
|
||||
|
||||
rows = asyncio.run(repo.list_messages(tmp_path / "bot.db", session_id="s1"))
|
||||
assert rows[0]["role"] == "user"
|
||||
assert "[附件]" in rows[0]["content"]
|
||||
assert "base64" not in rows[0]["content"] # 历史只存清单,不存字节
|
||||
# 落盘在 home 下的会话目录里
|
||||
assert list((tmp_path / "data" / "inbound").rglob("*.png"))
|
||||
# 回复没有媒体 → 仍走纯文本
|
||||
assert channel.sent == ["图里是一只猫"]
|
||||
assert channel.contents == []
|
||||
|
||||
|
||||
def test_agent_reply_media_only_inbound_is_multimodal(
|
||||
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
|
||||
) -> None:
|
||||
"""只有图、没有正文:不能被当成空消息,也不能退化成纯文本把图丢掉。"""
|
||||
seen: list[Any] = []
|
||||
|
||||
async def fake_run_task(
|
||||
task, settings=None, history=None, *, cache=None, steering=None, outbox=None
|
||||
):
|
||||
seen.append(task)
|
||||
return {"messages": [{"role": "assistant", "content": "收到"}]}
|
||||
|
||||
monkeypatch.setattr(builtin, "run_task", fake_run_task)
|
||||
channel = EchoChannel()
|
||||
statuses = _drive(
|
||||
tmp_path / "bot.db", [_media_envelope(_image_part())], channel, home=tmp_path
|
||||
)
|
||||
|
||||
assert statuses == [RunStatus.SUCCEEDED]
|
||||
assert [b["type"] for b in seen[0]] == ["image_url"]
|
||||
|
||||
|
||||
def test_agent_reply_reports_rejected_attachments(
|
||||
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
|
||||
) -> None:
|
||||
"""附件全部没接住:说清原因,不再笼统地当成「空消息」。"""
|
||||
|
||||
async def fake_prepare(
|
||||
message, *, root, limits=None, policy=None, headers=None, transform=None
|
||||
):
|
||||
return PreparedInbound(text="", notes=["图片未接收:超过大小上限"])
|
||||
|
||||
async def boom(*args, **kwargs): # pragma: no cover - 不应被调用
|
||||
raise AssertionError("附件全部失败时不应执行任务")
|
||||
|
||||
monkeypatch.setattr(builtin, "prepare_inbound", fake_prepare)
|
||||
monkeypatch.setattr(builtin, "run_task", boom)
|
||||
channel = EchoChannel()
|
||||
statuses = _drive(
|
||||
tmp_path / "bot.db", [_media_envelope(_image_part())], channel, home=tmp_path
|
||||
)
|
||||
|
||||
assert statuses == [RunStatus.SUCCEEDED]
|
||||
assert channel.sent == ["附件未能接收:图片未接收:超过大小上限"]
|
||||
|
||||
|
||||
def test_agent_reply_delivers_agent_media_in_one_content_message(
|
||||
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
|
||||
) -> None:
|
||||
"""智能体用工具发的图:与正文合成一条富内容,不额外再发一条纯文本。"""
|
||||
picture = tmp_path / "cat.png"
|
||||
picture.write_bytes(PNG_HEAD)
|
||||
|
||||
async def fake_run_task(
|
||||
task, settings=None, history=None, *, cache=None, steering=None, outbox=None
|
||||
):
|
||||
assert outbox is not None
|
||||
outbox.push(ImageContent(local_path=str(picture), mime_type="image/png"))
|
||||
return {"messages": [{"role": "assistant", "content": "图来了"}]}
|
||||
|
||||
monkeypatch.setattr(builtin, "run_task", fake_run_task)
|
||||
channel = EchoChannel()
|
||||
statuses = _drive(tmp_path / "bot.db", [_envelope("画只猫")], channel, home=tmp_path)
|
||||
|
||||
assert statuses == [RunStatus.SUCCEEDED]
|
||||
assert channel.sent == [] # 没有再发一条纯文本
|
||||
assert len(channel.contents) == 1
|
||||
parts = channel.contents[0]
|
||||
assert isinstance(parts[0], TextContent) and parts[0].text == "图来了"
|
||||
assert isinstance(parts[1], ImageContent)
|
||||
assert parts[1].local_path == str(picture)
|
||||
|
||||
|
||||
def test_agent_reply_media_only_reply_skips_empty_placeholder(
|
||||
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
|
||||
) -> None:
|
||||
"""只发了媒体、没写正文:不要补一句「(无输出)」。"""
|
||||
picture = tmp_path / "cat.png"
|
||||
picture.write_bytes(PNG_HEAD)
|
||||
|
||||
async def fake_run_task(
|
||||
task, settings=None, history=None, *, cache=None, steering=None, outbox=None
|
||||
):
|
||||
assert outbox is not None
|
||||
outbox.push(ImageContent(local_path=str(picture), mime_type="image/png"))
|
||||
return {"messages": [{"role": "assistant", "content": ""}]}
|
||||
|
||||
monkeypatch.setattr(builtin, "run_task", fake_run_task)
|
||||
channel = EchoChannel()
|
||||
statuses = _drive(tmp_path / "bot.db", [_envelope("发张图")], channel, home=tmp_path)
|
||||
|
||||
assert statuses == [RunStatus.SUCCEEDED]
|
||||
assert channel.sent == []
|
||||
assert [type(part) for part in channel.contents[0]] == [ImageContent]
|
||||
|
||||
|
||||
def test_agent_reply_persists_agent_media_to_history(
|
||||
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
|
||||
) -> None:
|
||||
"""智能体发的附件要进历史:下一轮模型才知道自己发过什么。"""
|
||||
|
||||
async def fake_run_task(
|
||||
task, settings=None, history=None, *, cache=None, steering=None, outbox=None
|
||||
):
|
||||
assert outbox is not None
|
||||
outbox.push(
|
||||
ImageContent(local_path=str(picture), mime_type="image/png", size=len(PNG_HEAD))
|
||||
)
|
||||
return {"messages": [{"role": "assistant", "content": "图来了"}]}
|
||||
|
||||
picture = tmp_path / "cat.png"
|
||||
picture.write_bytes(PNG_HEAD)
|
||||
monkeypatch.setattr(builtin, "run_task", fake_run_task)
|
||||
channel = EchoChannel()
|
||||
statuses = _drive(tmp_path / "bot.db", [_envelope("画只猫")], channel, home=tmp_path)
|
||||
|
||||
assert statuses == [RunStatus.SUCCEEDED]
|
||||
rows = asyncio.run(repo.list_messages(tmp_path / "bot.db", session_id="s1"))
|
||||
assert rows[-1]["role"] == "assistant"
|
||||
content = rows[-1]["content"]
|
||||
assert content.startswith("图来了")
|
||||
assert "[附件] 图片" in content
|
||||
assert str(picture) in content # 存路径引用,便于下一轮回指
|
||||
assert "base64" not in content # 不存字节
|
||||
|
||||
|
||||
def test_agent_reply_media_only_reply_still_lands_in_history(
|
||||
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
|
||||
) -> None:
|
||||
"""只发了图、没有正文:这一轮仍要写进历史,不能只剩用户那一句。"""
|
||||
picture = tmp_path / "cat.png"
|
||||
picture.write_bytes(PNG_HEAD)
|
||||
|
||||
async def fake_run_task(
|
||||
task, settings=None, history=None, *, cache=None, steering=None, outbox=None
|
||||
):
|
||||
assert outbox is not None
|
||||
outbox.push(ImageContent(local_path=str(picture), mime_type="image/png"))
|
||||
return {"messages": [{"role": "assistant", "content": ""}]}
|
||||
|
||||
monkeypatch.setattr(builtin, "run_task", fake_run_task)
|
||||
channel = EchoChannel()
|
||||
statuses = _drive(tmp_path / "bot.db", [_envelope("发张图")], channel, home=tmp_path)
|
||||
|
||||
assert statuses == [RunStatus.SUCCEEDED]
|
||||
rows = asyncio.run(repo.list_messages(tmp_path / "bot.db", session_id="s1"))
|
||||
assert [r["role"] for r in rows] == ["user", "assistant"]
|
||||
assert rows[-1]["content"].startswith("[附件] 图片")
|
||||
assert "无输出" not in rows[-1]["content"] # 不把兜底文案当成助手说的话
|
||||
|
||||
|
||||
def test_agent_reply_persists_reply_before_delivery(
|
||||
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
|
||||
) -> None:
|
||||
"""下发失败也要留住上下文:历史写入先于发送,且不回滚。"""
|
||||
|
||||
async def fake_run_task(
|
||||
task, settings=None, history=None, *, cache=None, steering=None, outbox=None
|
||||
):
|
||||
assert outbox is not None
|
||||
outbox.push(ImageContent(local_path=str(picture), mime_type="image/png"))
|
||||
return {"messages": [{"role": "assistant", "content": "给你"}]}
|
||||
|
||||
picture = tmp_path / "cat.png"
|
||||
picture.write_bytes(PNG_HEAD)
|
||||
monkeypatch.setattr(builtin, "run_task", fake_run_task)
|
||||
|
||||
class BrokenChannel(EchoChannel):
|
||||
async def _send_content(self, subject, parts): # type: ignore[override]
|
||||
raise RuntimeError("平台发送失败")
|
||||
|
||||
channel = BrokenChannel()
|
||||
statuses = _drive(tmp_path / "bot.db", [_envelope("发图")], channel, home=tmp_path)
|
||||
|
||||
assert statuses == [RunStatus.FAILED]
|
||||
rows = asyncio.run(repo.list_messages(tmp_path / "bot.db", session_id="s1"))
|
||||
assert [r["role"] for r in rows] == ["user", "assistant"]
|
||||
assert "给你" in rows[-1]["content"]
|
||||
|
||||
|
||||
def test_media_policy_falls_back_to_defaults(tmp_path: Path) -> None:
|
||||
"""拿不到渠道实例时按默认上限跑,不能因为查不到渠道就跳过校验。"""
|
||||
services = ChannelServices(
|
||||
channels=ChannelManager(), settings=Settings(home=tmp_path), db=tmp_path / "bot.db"
|
||||
)
|
||||
limits, headers = services.media_policy("no-such-channel")
|
||||
assert headers == {}
|
||||
assert limits.image > 0 and limits.file > 0
|
||||
@@ -8,7 +8,7 @@ from typing import Any
|
||||
|
||||
from bot042.channels.base import BaseChannel
|
||||
from bot042.channels.models import ChannelSubject, InboundMessage, TextContent
|
||||
from bot042.config import Settings
|
||||
from bot042.config import STEP_GRACE, Settings
|
||||
from bot042.flows.agent_task import last_message_text, run_task
|
||||
from bot042.flows.session import run_session
|
||||
|
||||
@@ -86,7 +86,8 @@ def test_run_task_uses_async_ainvoke(monkeypatch) -> None:
|
||||
assert last_message_text(result) == "done"
|
||||
state, config = fake.calls[0]
|
||||
assert state["messages"][-1] == {"role": "user", "content": "hi"}
|
||||
assert config == {"recursion_limit": 7}
|
||||
# 硬步数上限 = 软预算 + 收尾余量(见 Settings.recursion_limit)
|
||||
assert config == {"recursion_limit": 7 + STEP_GRACE}
|
||||
|
||||
|
||||
def test_last_message_text_multimodal_content() -> None:
|
||||
|
||||
@@ -0,0 +1,630 @@
|
||||
"""issue #1 验收:媒体链路的接入、护栏与安全边界。
|
||||
|
||||
对应 issue 的验收标准(`docs` 与本文件一一对应):
|
||||
|
||||
1. 媒体能进上下文 —— `test_image_reaches_agent_context` 等;
|
||||
2. 文本 + 文件混合不丢 —— `test_text_and_file_both_reach_agent`;
|
||||
3. 智能体发媒体走原生下发 —— `test_outbox_part_flows_to_channel`;
|
||||
4. 超限被拒且不炸内存 —— `test_oversize_*`;
|
||||
5. 恶意文件名不越界 —— `test_sanitize_*` / `test_safe_join_*`;
|
||||
6. 以上均有单测且 pytest 全绿。
|
||||
|
||||
本文件只测**媒体策略与链路**,不重复测 AgentReply 的会话/落库语义
|
||||
(见 `tests/test_builtin.py`)。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import base64
|
||||
import socket
|
||||
import sys
|
||||
import threading
|
||||
import types
|
||||
from pathlib import Path
|
||||
from typing import Self
|
||||
|
||||
import pytest
|
||||
|
||||
from bot042.agent.outbox import (
|
||||
MediaOutbox,
|
||||
bind,
|
||||
current_outbox,
|
||||
make_file_part,
|
||||
make_image_part,
|
||||
unbind,
|
||||
)
|
||||
from bot042.channels.cli import CliChannel
|
||||
from bot042.channels.manager import ChannelManager
|
||||
from bot042.channels.media import (
|
||||
MB,
|
||||
MediaLimits,
|
||||
MediaRejected,
|
||||
check_mime,
|
||||
decode_base64_limited,
|
||||
download_limited,
|
||||
read_file_limited,
|
||||
safe_join,
|
||||
sanitize_filename,
|
||||
store_media,
|
||||
)
|
||||
from bot042.channels.models import (
|
||||
ChannelSubject,
|
||||
FileContent,
|
||||
ImageContent,
|
||||
InboundMessage,
|
||||
TextContent,
|
||||
)
|
||||
from bot042.config import Settings
|
||||
from bot042.flows.builtin import ChannelServices
|
||||
from bot042.flows.media_input import prepare_inbound
|
||||
|
||||
PNG_HEAD = b"\x89PNG\r\n\x1a\n" + b"\x00" * 64
|
||||
PDF_HEAD = b"%PDF-1.4\n" + b"\x00" * 64
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 5. 路径安全:文件名净化与根目录约束
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"hostile",
|
||||
[
|
||||
"../../etc/passwd",
|
||||
"..\\..\\Windows\\System32\\config",
|
||||
"/etc/shadow",
|
||||
"C:\\Windows\\system.ini",
|
||||
"....//....//evil.sh",
|
||||
"a/../../../b.png",
|
||||
],
|
||||
)
|
||||
def test_sanitize_filename_never_escapes(hostile: str) -> None:
|
||||
"""恶意文件名净化后不得含路径分隔符,也不得是 . 或 .."""
|
||||
cleaned = sanitize_filename(hostile)
|
||||
assert "/" not in cleaned and "\\" not in cleaned
|
||||
assert cleaned not in {".", ".."}
|
||||
assert not cleaned.startswith(".")
|
||||
|
||||
|
||||
def test_safe_join_keeps_target_inside_root(tmp_path: Path) -> None:
|
||||
"""即使传入穿越片段,落点也必须在 root 之内。"""
|
||||
root = tmp_path / "session"
|
||||
root.mkdir()
|
||||
target = safe_join(root, sanitize_filename("../../escape.png"))
|
||||
assert target.parent == root
|
||||
assert root in target.parents
|
||||
|
||||
|
||||
def test_store_media_lands_inside_session_dir(tmp_path: Path) -> None:
|
||||
"""落盘路径由会话键决定:会话键里的 .. 会被净化,不会跳出 root。"""
|
||||
part = ImageContent(mime_type="image/png").model_copy(
|
||||
update={"data": base64.b64encode(PNG_HEAD).decode()}
|
||||
)
|
||||
stored = asyncio.run(store_media(part, root=tmp_path, session_key="../s"))
|
||||
assert tmp_path in stored.path.parents
|
||||
assert ".." not in str(stored.path.relative_to(tmp_path))
|
||||
# 会话键被净化成单个目录段,不是嵌套路径
|
||||
assert stored.path.parent.parent == tmp_path
|
||||
|
||||
|
||||
def test_store_media_dedupes_same_name(tmp_path: Path) -> None:
|
||||
"""同名不同内容不得互相覆盖(后到的加哈希前缀)。"""
|
||||
first = ImageContent(mime_type="image/png").model_copy(
|
||||
update={"data": base64.b64encode(PNG_HEAD).decode()}
|
||||
)
|
||||
second = ImageContent(mime_type="image/png").model_copy(
|
||||
update={"data": base64.b64encode(PNG_HEAD + b"other").decode()}
|
||||
)
|
||||
a = asyncio.run(store_media(first, root=tmp_path, session_key="s"))
|
||||
b = asyncio.run(store_media(second, root=tmp_path, session_key="s"))
|
||||
assert a.path != b.path
|
||||
assert a.path.read_bytes() == PNG_HEAD
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 4. 大小护栏:本地读取、base64 解码、下载
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_read_file_limited_rejects_oversize(tmp_path: Path) -> None:
|
||||
big = tmp_path / "big.bin"
|
||||
big.write_bytes(b"x" * 2048)
|
||||
with pytest.raises(MediaRejected) as err:
|
||||
read_file_limited(big, 1024)
|
||||
assert "超过大小上限" in err.value.reason
|
||||
|
||||
|
||||
def test_read_file_limited_reads_at_most_limit_plus_one(tmp_path: Path) -> None:
|
||||
"""判定越界只需多读 1 字节,不得把整个超大文件读进内存。"""
|
||||
big = tmp_path / "huge.bin"
|
||||
big.write_bytes(b"x" * (8 * 1024 * 1024))
|
||||
with pytest.raises(MediaRejected):
|
||||
read_file_limited(big, 1024)
|
||||
|
||||
|
||||
def test_read_file_limited_missing_file(tmp_path: Path) -> None:
|
||||
with pytest.raises(MediaRejected) as err:
|
||||
read_file_limited(tmp_path / "nope.bin", 1024)
|
||||
assert err.value.reason == "文件不存在"
|
||||
|
||||
|
||||
def test_decode_base64_limited_rejects_oversize() -> None:
|
||||
payload = base64.b64encode(b"x" * 4096).decode()
|
||||
with pytest.raises(MediaRejected):
|
||||
decode_base64_limited(payload, 1024)
|
||||
|
||||
|
||||
def test_decode_base64_limited_invalid() -> None:
|
||||
with pytest.raises(MediaRejected):
|
||||
decode_base64_limited("!!!not-base64!!!", 1024)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("declared", "head", "expected"),
|
||||
[
|
||||
("image/png", PNG_HEAD, "image/png"),
|
||||
# 声明撒谎:实际是 PNG,按魔数修正(改扩展名绕不过类型限制)
|
||||
("text/plain", PNG_HEAD, "image/png"),
|
||||
("image/jpeg", PNG_HEAD, "image/png"),
|
||||
],
|
||||
)
|
||||
def test_check_mime_prefers_magic_bytes(declared: str, head: bytes, expected: str) -> None:
|
||||
"""声明类型与魔数冲突时以魔数为准。"""
|
||||
got = check_mime(ImageContent(), declared, head)
|
||||
assert got == expected
|
||||
|
||||
|
||||
def test_check_mime_file_accepts_pdf() -> None:
|
||||
"""非图片类文件走 FileContent,PDF 应放行并采信魔数。"""
|
||||
got = check_mime(FileContent(filename="a.pdf"), "", PDF_HEAD)
|
||||
assert got == "application/pdf"
|
||||
|
||||
|
||||
def test_check_mime_image_rejects_pdf() -> None:
|
||||
"""把 PDF 当图片投递:必须拒绝,而不是带着错误 MIME 继续。"""
|
||||
with pytest.raises(MediaRejected) as err:
|
||||
check_mime(ImageContent(), "application/pdf", PDF_HEAD)
|
||||
assert "不被接受" in err.value.reason
|
||||
|
||||
|
||||
def test_check_mime_rejects_executable() -> None:
|
||||
"""可执行文件按扩展名拒绝:智能体读到的文件不该原样推给对方。"""
|
||||
with pytest.raises(MediaRejected):
|
||||
check_mime(FileContent(filename="virus.exe"), "", b"MZ")
|
||||
|
||||
|
||||
def test_media_limits_per_type_is_ordered() -> None:
|
||||
limits = MediaLimits()
|
||||
assert limits.image <= limits.file
|
||||
assert limits.for_part(ImageContent()).__class__ is int
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 3. 出站:outbox 的工具入口
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_make_image_part_validates_type(tmp_path: Path) -> None:
|
||||
fake = tmp_path / "notimage.png"
|
||||
fake.write_bytes(b"plain text, not a png at all")
|
||||
with pytest.raises(MediaRejected):
|
||||
make_image_part(str(fake))
|
||||
|
||||
|
||||
def test_make_image_part_ok(tmp_path: Path) -> None:
|
||||
picture = tmp_path / "cat.png"
|
||||
picture.write_bytes(PNG_HEAD)
|
||||
part = make_image_part(str(picture), caption="猫")
|
||||
assert isinstance(part, ImageContent)
|
||||
assert part.size == len(PNG_HEAD)
|
||||
assert part.mime_type == "image/png"
|
||||
assert part.alt_text == "猫"
|
||||
|
||||
|
||||
def test_make_file_part_rejects_missing(tmp_path: Path) -> None:
|
||||
with pytest.raises(MediaRejected) as err:
|
||||
make_file_part(str(tmp_path / "ghost.pdf"))
|
||||
assert "不存在" in err.value.reason
|
||||
|
||||
|
||||
def test_outbox_caps_pending_count() -> None:
|
||||
"""单轮附件数量设上限,避免模型刷屏式调用把平台打爆。"""
|
||||
box = MediaOutbox(max_pending=2)
|
||||
part = ImageContent(mime_type="image/png")
|
||||
box.push(part)
|
||||
box.push(part)
|
||||
with pytest.raises(MediaRejected):
|
||||
box.push(part)
|
||||
assert len(box.drain()) == 2
|
||||
assert len(box) == 0
|
||||
|
||||
|
||||
def test_outbox_context_binding_is_isolated() -> None:
|
||||
box = MediaOutbox()
|
||||
assert current_outbox() is None
|
||||
token = bind(box)
|
||||
try:
|
||||
assert current_outbox() is box
|
||||
finally:
|
||||
unbind(token)
|
||||
assert current_outbox() is None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 1 & 2. 入站:多模态块构建与降级
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def _prepare(**kwargs):
|
||||
from bot042.flows.media_input import prepare_inbound
|
||||
|
||||
return prepare_inbound
|
||||
|
||||
|
||||
def test_prepare_inbound_inlines_small_image(tmp_path: Path) -> None:
|
||||
from bot042.channels.models import ChannelSubject, InboundMessage
|
||||
from bot042.flows.media_input import prepare_inbound
|
||||
|
||||
picture = tmp_path / "cat.png"
|
||||
picture.write_bytes(PNG_HEAD)
|
||||
message = InboundMessage(
|
||||
channel_id="c1",
|
||||
channel_type="cli",
|
||||
channel_subject=ChannelSubject(subject_id="u1"),
|
||||
content=[
|
||||
TextContent(text="这是什么"),
|
||||
ImageContent(local_path=str(picture), mime_type="image/png"),
|
||||
],
|
||||
)
|
||||
prepared = asyncio.run(prepare_inbound(message, root=tmp_path / "in"))
|
||||
kinds = [b["type"] for b in prepared.blocks]
|
||||
assert kinds == ["text", "image_url"]
|
||||
assert prepared.blocks[1]["image_url"]["url"].startswith("data:image/png;base64,")
|
||||
assert prepared.is_multimodal
|
||||
# 摘要里带附件清单,且不含 base64
|
||||
assert "[附件]" in prepared.summary
|
||||
assert "base64" not in prepared.summary
|
||||
|
||||
|
||||
def test_prepare_inbound_text_and_file_both_kept(tmp_path: Path) -> None:
|
||||
"""文本 + 文件混合:两者都进上下文,不静默丢弃任何一个。"""
|
||||
from bot042.channels.models import ChannelSubject, InboundMessage
|
||||
from bot042.flows.media_input import prepare_inbound
|
||||
|
||||
note = tmp_path / "notes.txt"
|
||||
note.write_text("第一行\n第二行\n", encoding="utf-8")
|
||||
message = InboundMessage(
|
||||
channel_id="c1",
|
||||
channel_type="cli",
|
||||
channel_subject=ChannelSubject(subject_id="u1"),
|
||||
content=[
|
||||
TextContent(text="帮我看看这个文件"),
|
||||
FileContent(local_path=str(note), filename="notes.txt", mime_type="text/plain"),
|
||||
],
|
||||
)
|
||||
prepared = asyncio.run(prepare_inbound(message, root=tmp_path / "in"))
|
||||
joined = "\n".join(b.get("text", "") for b in prepared.blocks)
|
||||
assert "帮我看看这个文件" in joined
|
||||
assert "第一行" in joined # 文本文件内容被抽取进上下文
|
||||
assert len(prepared.stored) == 1
|
||||
|
||||
|
||||
def test_prepare_inbound_oversize_degrades_to_note(tmp_path: Path) -> None:
|
||||
"""超限附件降级成一条说明,不抛异常、不中断其余内容。"""
|
||||
from bot042.channels.models import ChannelSubject, InboundMessage
|
||||
from bot042.flows.media_input import prepare_inbound
|
||||
|
||||
big = tmp_path / "big.png"
|
||||
big.write_bytes(PNG_HEAD + b"\x00" * 4096)
|
||||
message = InboundMessage(
|
||||
channel_id="c1",
|
||||
channel_type="cli",
|
||||
channel_subject=ChannelSubject(subject_id="u1"),
|
||||
content=[
|
||||
TextContent(text="看图"),
|
||||
ImageContent(local_path=str(big), mime_type="image/png"),
|
||||
],
|
||||
)
|
||||
prepared = asyncio.run(
|
||||
prepare_inbound(
|
||||
message, root=tmp_path / "in", limits=MediaLimits(image=128, file=128)
|
||||
)
|
||||
)
|
||||
assert prepared.has_content
|
||||
assert prepared.notes
|
||||
assert any("上限" in n or "超过" in n for n in prepared.notes)
|
||||
|
||||
|
||||
def test_prepare_inbound_missing_file_is_noted(tmp_path: Path) -> None:
|
||||
from bot042.channels.models import ChannelSubject, InboundMessage
|
||||
from bot042.flows.media_input import prepare_inbound
|
||||
|
||||
message = InboundMessage(
|
||||
channel_id="c1",
|
||||
channel_type="cli",
|
||||
channel_subject=ChannelSubject(subject_id="u1"),
|
||||
content=[FileContent(local_path=str(tmp_path / "ghost.pdf"), filename="ghost.pdf")],
|
||||
)
|
||||
prepared = asyncio.run(prepare_inbound(message, root=tmp_path / "in"))
|
||||
assert prepared.notes
|
||||
assert not prepared.stored
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# CLI 渠道:无平台时的收发调试入口
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_cli_image_command(tmp_path: Path) -> None:
|
||||
picture = tmp_path / "cat.png"
|
||||
picture.write_bytes(PNG_HEAD)
|
||||
channel = CliChannel()
|
||||
message = channel.parse_inbound(f"/image {picture}")
|
||||
assert isinstance(message.content[0], ImageContent)
|
||||
assert message.content[0].mime_type == "image/png"
|
||||
|
||||
|
||||
def test_cli_file_command(tmp_path: Path) -> None:
|
||||
note = tmp_path / "a.txt"
|
||||
note.write_text("hi", encoding="utf-8")
|
||||
channel = CliChannel()
|
||||
message = channel.parse_inbound(f"/file {note}")
|
||||
assert isinstance(message.content[0], FileContent)
|
||||
assert message.content[0].filename == "a.txt"
|
||||
|
||||
|
||||
def test_cli_image_command_rejects_oversize(tmp_path: Path) -> None:
|
||||
picture = tmp_path / "cat.png"
|
||||
picture.write_bytes(PNG_HEAD + b"\x00" * 4096)
|
||||
channel = CliChannel()
|
||||
channel._media_limits = MediaLimits(image=64, file=64)
|
||||
message = channel.parse_inbound(f"/image {picture}")
|
||||
assert isinstance(message.content[0], TextContent)
|
||||
assert "未接收" in message.content[0].text
|
||||
|
||||
|
||||
def test_cli_image_command_wrong_type(tmp_path: Path) -> None:
|
||||
note = tmp_path / "a.txt"
|
||||
note.write_text("hi", encoding="utf-8")
|
||||
channel = CliChannel()
|
||||
message = channel.parse_inbound(f"/image {note}")
|
||||
assert isinstance(message.content[0], TextContent)
|
||||
assert "不是图片" in message.content[0].text
|
||||
|
||||
|
||||
def test_cli_plain_text_unchanged() -> None:
|
||||
channel = CliChannel()
|
||||
message = channel.parse_inbound("你好")
|
||||
assert isinstance(message.content[0], TextContent)
|
||||
assert message.content[0].text == "你好"
|
||||
|
||||
|
||||
def test_cli_send_media_reports_real_bytes(tmp_path: Path, capsys) -> None:
|
||||
"""出站回执必须来自真实读取:给出大小与 sha256 前缀,而不是「已发送」。"""
|
||||
picture = tmp_path / "cat.png"
|
||||
picture.write_bytes(PNG_HEAD)
|
||||
channel = CliChannel()
|
||||
asyncio.run(
|
||||
channel._send_media(None, ImageContent(local_path=str(picture), mime_type="image/png"))
|
||||
)
|
||||
out = capsys.readouterr().out
|
||||
assert "媒体回执" in out
|
||||
assert str(len(PNG_HEAD)) in out
|
||||
assert "sha256:" in out
|
||||
|
||||
|
||||
def test_cli_send_media_missing_file_is_reported(tmp_path: Path, capsys) -> None:
|
||||
channel = CliChannel()
|
||||
asyncio.run(
|
||||
channel._send_media(
|
||||
None, ImageContent(local_path=str(tmp_path / "ghost.png"), mime_type="image/png")
|
||||
)
|
||||
)
|
||||
out = capsys.readouterr().out
|
||||
assert "媒体" in out and ("被拒" in out or "失败" in out)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 平台解密钩子:接线必须真的到达 store_media(不只是签名吻合)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_store_media_applies_transform_before_sniffing(tmp_path: Path) -> None:
|
||||
"""transform 必须在类型嗅探**之前**执行:密文嗅魔数只会得到 octet-stream。"""
|
||||
ciphertext = b"\x00\x01\x02\x03cipher-not-a-png" * 4
|
||||
calls: list[tuple[str, int]] = []
|
||||
|
||||
def fake_decrypt(part, raw: bytes) -> bytes:
|
||||
calls.append((part.type, len(raw)))
|
||||
return PNG_HEAD
|
||||
|
||||
# 用内联字节而非 url:这条测的是 transform 与嗅探的顺序,不该触发网络
|
||||
part = ImageContent(mime_type="application/octet-stream").model_copy(
|
||||
update={"data": base64.b64encode(ciphertext).decode()}
|
||||
)
|
||||
stored = asyncio.run(store_media(part, root=tmp_path, transform=fake_decrypt))
|
||||
|
||||
assert calls == [("image", len(ciphertext))]
|
||||
# 明文是 PNG,所以最终类型由**明文**魔数决定
|
||||
assert stored.mime == "image/png"
|
||||
assert stored.path.read_bytes() == PNG_HEAD
|
||||
|
||||
|
||||
def test_store_media_rechecks_size_after_transform(tmp_path: Path) -> None:
|
||||
"""密文没超不等于明文没超:转换后必须再量一次。"""
|
||||
limits = MediaLimits(image=64)
|
||||
|
||||
def expand(part, raw: bytes) -> bytes:
|
||||
return b"\x00" * (limits.image + 1)
|
||||
|
||||
part = ImageContent(mime_type="application/octet-stream").model_copy(
|
||||
update={"data": base64.b64encode(b"small-cipher").decode()}
|
||||
)
|
||||
with pytest.raises(MediaRejected) as excinfo:
|
||||
asyncio.run(store_media(part, root=tmp_path, limits=limits, transform=expand))
|
||||
assert "解密后" in excinfo.value.detail
|
||||
|
||||
|
||||
def test_store_media_transform_failure_becomes_rejection(tmp_path: Path) -> None:
|
||||
"""解密异常要转成 MediaRejected,不能冒泡成未捕获异常打断整轮。"""
|
||||
|
||||
def broken(part, raw: bytes) -> bytes:
|
||||
raise ValueError("bad padding")
|
||||
|
||||
part = ImageContent(mime_type="application/octet-stream").model_copy(
|
||||
update={"data": base64.b64encode(b"cipher").decode()}
|
||||
)
|
||||
with pytest.raises(MediaRejected) as excinfo:
|
||||
asyncio.run(store_media(part, root=tmp_path, transform=broken))
|
||||
assert "解密失败" in excinfo.value.reason
|
||||
|
||||
|
||||
def test_prepare_inbound_passes_transform_through(tmp_path: Path) -> None:
|
||||
"""prepare_inbound 必须把 transform 透传到 store_media,否则解密形同虚设。"""
|
||||
seen: list[int] = []
|
||||
|
||||
def fake_decrypt(part, raw: bytes) -> bytes:
|
||||
seen.append(len(raw))
|
||||
return PNG_HEAD
|
||||
|
||||
message = InboundMessage(
|
||||
channel_id="c1",
|
||||
channel_type="t",
|
||||
channel_subject=ChannelSubject(subject_id="u1"),
|
||||
content=[ImageContent(data=base64.b64encode(b"cipher-bytes").decode())],
|
||||
)
|
||||
prepared = asyncio.run(
|
||||
prepare_inbound(message, root=tmp_path / "in", transform=fake_decrypt)
|
||||
)
|
||||
|
||||
assert seen == [len(b"cipher-bytes")]
|
||||
assert prepared.stored and prepared.stored[0].read_bytes() == PNG_HEAD
|
||||
|
||||
|
||||
def test_wecom_media_transform_uses_per_message_aeskey(monkeypatch) -> None:
|
||||
"""企业微信的解密钩子按本条消息的 url 取密钥;没有密钥时原样返回。"""
|
||||
from bot042.channels.im import wecom as wecom_mod
|
||||
|
||||
calls: list[tuple[bytes, str]] = []
|
||||
|
||||
def fake_decrypt_file(raw: bytes, key: str) -> bytes:
|
||||
calls.append((raw, key))
|
||||
return PNG_HEAD
|
||||
|
||||
fake_sdk = types.SimpleNamespace(decrypt_file=fake_decrypt_file)
|
||||
monkeypatch.setitem(sys.modules, "wecom_aibot_sdk", fake_sdk)
|
||||
|
||||
channel = wecom_mod.WeComChannel()
|
||||
url = "https://example.invalid/media/1"
|
||||
transform = channel.make_media_transform({"_media_aes_keys": {url: "k" * 43}})
|
||||
part = ImageContent(url=url)
|
||||
|
||||
assert transform(part, b"cipher") == PNG_HEAD
|
||||
assert calls == [(b"cipher", "k" * 43)]
|
||||
|
||||
# 没有对应 url 的密钥:返回原字节,不猜、不抛
|
||||
other = ImageContent(url="https://example.invalid/media/2")
|
||||
assert transform(other, b"cipher") == b"cipher"
|
||||
|
||||
|
||||
def test_channel_services_media_transform_none_for_unknown_channel(tmp_path: Path) -> None:
|
||||
"""渠道未注册时返回 None(不解密),不抛异常也不跳过后续校验。"""
|
||||
services = ChannelServices(
|
||||
channels=ChannelManager(), settings=Settings(home=tmp_path), db=tmp_path / "b.db"
|
||||
)
|
||||
assert services.media_transform("no-such-channel", {}) is None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 4. 下载限长:分块传输(无 Content-Length)必须靠累计判限中断
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
class _ChunkedServer:
|
||||
"""极简 HTTP/1.1 分块响应服务端,用于验证「服务端不给 Content-Length」时的闸。
|
||||
|
||||
真实平台可能用 ``Transfer-Encoding: chunked`` 下发媒体,此时
|
||||
``Content-Length`` 预检形同虚设,只剩按块累计这一道。用真 socket 而不是
|
||||
桩,是因为要验证的正是「读到超限就断开连接」——桩连接测不出中断。
|
||||
"""
|
||||
|
||||
def __init__(self, total: int, piece: int = 16 * 1024) -> None:
|
||||
self.total = total
|
||||
self.piece = piece
|
||||
self.sent = 0
|
||||
self._sock = socket.socket()
|
||||
self._sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
|
||||
self._sock.bind(("127.0.0.1", 0))
|
||||
self._sock.listen(1)
|
||||
self._sock.settimeout(10)
|
||||
self.url = f"http://127.0.0.1:{self._sock.getsockname()[1]}/media"
|
||||
|
||||
def __enter__(self) -> Self:
|
||||
self._thread = threading.Thread(target=self._serve, daemon=True)
|
||||
self._thread.start()
|
||||
return self
|
||||
|
||||
def __exit__(self, *exc_info: object) -> None:
|
||||
self._thread.join(timeout=10)
|
||||
self._sock.close()
|
||||
|
||||
def _serve(self) -> None:
|
||||
try:
|
||||
conn, _ = self._sock.accept()
|
||||
except OSError:
|
||||
return
|
||||
with conn:
|
||||
conn.settimeout(10)
|
||||
head = b""
|
||||
while b"\r\n\r\n" not in head:
|
||||
try:
|
||||
piece = conn.recv(4096)
|
||||
except OSError:
|
||||
return
|
||||
if not piece:
|
||||
return
|
||||
head += piece
|
||||
conn.sendall(
|
||||
b"HTTP/1.1 200 OK\r\n"
|
||||
b"Content-Type: image/png\r\n"
|
||||
b"Transfer-Encoding: chunked\r\n\r\n"
|
||||
)
|
||||
payload = PNG_HEAD + b"\x00" * (self.piece - len(PNG_HEAD))
|
||||
try:
|
||||
while self.sent < self.total:
|
||||
conn.sendall(b"%x\r\n" % len(payload) + payload + b"\r\n")
|
||||
self.sent += len(payload)
|
||||
conn.sendall(b"0\r\n\r\n")
|
||||
except OSError:
|
||||
# 客户端超限后主动断开,这正是被测行为
|
||||
return
|
||||
|
||||
|
||||
def test_download_limited_aborts_chunked_stream() -> None:
|
||||
"""没有 Content-Length 时,累计超过上限就要中断,不能把整份读完。"""
|
||||
total = 8 * MB
|
||||
limit = 256 * 1024
|
||||
with (
|
||||
_ChunkedServer(total=total) as server,
|
||||
pytest.raises(MediaRejected) as excinfo,
|
||||
):
|
||||
asyncio.run(
|
||||
download_limited(server.url, limit=limit, chunk=16 * 1024, timeout=10.0)
|
||||
)
|
||||
assert "已中断" in excinfo.value.detail
|
||||
# 服务端没能把 8 MiB 全部写出:说明连接是在中途被断开的,不是读完再判
|
||||
assert server.sent < total
|
||||
|
||||
|
||||
def test_store_media_creates_missing_root(tmp_path: Path) -> None:
|
||||
"""root 不存在时应自行创建:首次运行 ``home/data/inbound`` 还不存在。"""
|
||||
root = tmp_path / "data" / "inbound"
|
||||
part = ImageContent(mime_type="image/png").model_copy(
|
||||
update={"data": base64.b64encode(PNG_HEAD).decode()}
|
||||
)
|
||||
stored = asyncio.run(store_media(part, root=root))
|
||||
|
||||
assert root.is_dir()
|
||||
assert stored.path.parent == root
|
||||
assert stored.path.read_bytes() == PNG_HEAD
|
||||
+52
-2
@@ -16,6 +16,7 @@ from bot042.channels.models import (
|
||||
)
|
||||
from bot042.db import repo
|
||||
from bot042.flows import processor as processor_mod
|
||||
from bot042.flows.media_input import PreparedInbound
|
||||
from bot042.flows.processor import make_processor
|
||||
|
||||
|
||||
@@ -210,7 +211,7 @@ def test_build_history_rules() -> None:
|
||||
]
|
||||
|
||||
|
||||
def test_processor_rejects_empty_and_media(
|
||||
def test_processor_rejects_empty_message(
|
||||
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
|
||||
) -> None:
|
||||
async def fake_run_task(task, settings=None, history=None, *, cache=None): # pragma: no cover
|
||||
@@ -223,6 +224,55 @@ def test_processor_rejects_empty_and_media(
|
||||
assert events[-1].type == MessageEventType.ERROR
|
||||
assert "空消息" in (events[-1].error or "")
|
||||
|
||||
|
||||
def test_processor_explains_unreadable_attachments(
|
||||
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
|
||||
) -> None:
|
||||
"""附件全部没接收成功:给附件级说明,不要含糊成「空消息」。"""
|
||||
|
||||
async def fake_prepare(message, *, root, **kwargs):
|
||||
_ = message, root, kwargs
|
||||
return PreparedInbound(text="", notes=["图片未接收:超过大小上限"])
|
||||
|
||||
async def fake_run_task(task, settings=None, history=None, *, cache=None): # pragma: no cover
|
||||
raise AssertionError("不应执行任务")
|
||||
|
||||
monkeypatch.setattr(processor_mod, "prepare_inbound", fake_prepare)
|
||||
monkeypatch.setattr(processor_mod, "run_task", fake_run_task)
|
||||
processor = make_processor(db=tmp_path / "bot.db")
|
||||
|
||||
events = _events(processor, _message("", media=True))
|
||||
assert events[-1].type == MessageEventType.ERROR
|
||||
assert "仅支持文本" in (events[-1].error or "")
|
||||
error = events[-1].error or ""
|
||||
assert "附件未能接收" in error
|
||||
assert "超过大小上限" in error
|
||||
|
||||
|
||||
def test_processor_sends_blocks_for_attachment_only_message(
|
||||
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
|
||||
) -> None:
|
||||
"""只有一张图、没有正文时也必须发多模态块,否则附件被静默丢掉。"""
|
||||
|
||||
async def fake_prepare(message, *, root, **kwargs):
|
||||
_ = message, root, kwargs
|
||||
return PreparedInbound(
|
||||
text="",
|
||||
blocks=[
|
||||
{"type": "image_url", "image_url": {"url": "data:image/png;base64,AA=="}}
|
||||
],
|
||||
)
|
||||
|
||||
seen: list[object] = []
|
||||
|
||||
async def fake_run_task(task, settings=None, history=None, *, cache=None):
|
||||
seen.append(task)
|
||||
return {"messages": [{"role": "assistant", "content": "看到了"}]}
|
||||
|
||||
monkeypatch.setattr(processor_mod, "prepare_inbound", fake_prepare)
|
||||
monkeypatch.setattr(processor_mod, "run_task", fake_run_task)
|
||||
processor = make_processor(db=tmp_path / "bot.db")
|
||||
|
||||
events = _events(processor, _message("", media=True))
|
||||
assert isinstance(seen[0], list)
|
||||
assert seen[0][0]["type"] == "image_url"
|
||||
assert events[-1].type == MessageEventType.COMPLETED
|
||||
@@ -135,7 +135,7 @@ def test_run_service_end_to_end(
|
||||
RecordingChannel.instances.clear()
|
||||
|
||||
async def fake_run_task(
|
||||
task, settings=None, history=None, *, cache=None, steering=None
|
||||
task, settings=None, history=None, *, cache=None, steering=None, outbox=None
|
||||
):
|
||||
assert task == "hello"
|
||||
return {"messages": [{"role": "assistant", "content": "答:" + task}]}
|
||||
@@ -189,7 +189,7 @@ def test_run_service_passes_reply_context(
|
||||
RecordingChannel.instances.clear()
|
||||
|
||||
async def fake_run_task(
|
||||
task, settings=None, history=None, *, cache=None, steering=None
|
||||
task, settings=None, history=None, *, cache=None, steering=None, outbox=None
|
||||
):
|
||||
return {"messages": [{"role": "assistant", "content": "ok"}]}
|
||||
|
||||
|
||||
+155
-4
@@ -9,24 +9,33 @@ from typing import Any, ClassVar
|
||||
|
||||
import pytest
|
||||
|
||||
from bot042.agent.middlewares.budget import StepBudgetExceeded
|
||||
from bot042.channels.base import BaseChannel
|
||||
from bot042.channels.manager import ChannelManager
|
||||
from bot042.channels.models import (
|
||||
ChannelConfig,
|
||||
ChannelMessage,
|
||||
ChannelSubject,
|
||||
ImageContent,
|
||||
InboundMessage,
|
||||
OutboundStream,
|
||||
TextContent,
|
||||
)
|
||||
from bot042.channels.registry import register_channel
|
||||
from bot042.config import Settings
|
||||
from bot042.db import repo
|
||||
from bot042.flows import builtin
|
||||
from bot042.flows.builtin import RPC_STREAM, AgentReply, ChannelServices
|
||||
from bot042.flows.builtin import (
|
||||
BUDGET_REPLY,
|
||||
RPC_SEND,
|
||||
RPC_STREAM,
|
||||
AgentReply,
|
||||
ChannelServices,
|
||||
)
|
||||
from bot042.flows.bus import EventBus
|
||||
from bot042.flows.rpc import RpcHub
|
||||
from bot042.flows.stream import ReplyStream
|
||||
from bot042.flows.tasks import EventTrigger, TaskRunner
|
||||
from bot042.flows.tasks import EventTrigger, RunStatus, TaskRunner
|
||||
|
||||
|
||||
def test_stream_sends_ack_and_throttles() -> None:
|
||||
@@ -123,6 +132,8 @@ class StreamChannel(BaseChannel):
|
||||
) -> None:
|
||||
super().__init__(processor, config=config, channel_id=channel_id, **kwargs)
|
||||
self.frames: list[tuple[str, bool]] = []
|
||||
self.contents: list[list] = []
|
||||
"""每次富内容发送的分片组合(流式通道只送文本,媒体走这里)。"""
|
||||
StreamChannel.instances.append(self)
|
||||
|
||||
async def start(self) -> None:
|
||||
@@ -153,7 +164,7 @@ class StreamChannel(BaseChannel):
|
||||
self.frames.append((text, True))
|
||||
|
||||
async def _send_content(self, subject: ChannelSubject, parts: list) -> None:
|
||||
pass
|
||||
self.contents.append(list(parts))
|
||||
|
||||
async def _send_media(self, subject: ChannelSubject, media: Any) -> None:
|
||||
pass
|
||||
@@ -168,7 +179,9 @@ def test_agent_reply_acks_then_streams(
|
||||
"""收到消息先 ack,随后逐步覆盖,最后以 finish 收尾。"""
|
||||
StreamChannel.instances.clear()
|
||||
|
||||
async def fake_stream(task, settings=None, history=None, *, cache=None, steering=None):
|
||||
async def fake_stream(
|
||||
task, settings=None, history=None, *, cache=None, steering=None, outbox=None
|
||||
):
|
||||
for text in ("你", "你好"):
|
||||
yield text
|
||||
|
||||
@@ -220,6 +233,144 @@ def test_agent_reply_acks_then_streams(
|
||||
assert frames[-1] == ("你好", True)
|
||||
|
||||
|
||||
def test_agent_reply_streams_budget_notice_after_partial(
|
||||
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
|
||||
) -> None:
|
||||
"""步数用尽:说明补在已流出的内容后面,落库,且不再补发通用错误文案。"""
|
||||
StreamChannel.instances.clear()
|
||||
|
||||
async def exhausted(
|
||||
task, settings=None, history=None, *, cache=None, steering=None, outbox=None
|
||||
):
|
||||
yield "做到一半"
|
||||
raise StepBudgetExceeded(200)
|
||||
|
||||
monkeypatch.setattr(builtin, "stream_task", exhausted)
|
||||
|
||||
async def _run() -> tuple[list[tuple[str, bool]], list[RunStatus]]:
|
||||
bus = EventBus()
|
||||
bus.bind_loop()
|
||||
hub = RpcHub(bus)
|
||||
manager = ChannelManager()
|
||||
channel = StreamChannel(channel_id="s1")
|
||||
await manager.add_channel(channel)
|
||||
await manager.start()
|
||||
_ = hub.serve(RPC_STREAM, manager.rpc_stream)
|
||||
|
||||
services = ChannelServices(
|
||||
channels=manager,
|
||||
settings=Settings(),
|
||||
db=tmp_path / "bot.db",
|
||||
hub=hub,
|
||||
)
|
||||
runner = TaskRunner(bus, deps=services)
|
||||
runner.register(AgentReply)
|
||||
_ = runner.add_task(AgentReply.name, EventTrigger(event="channel.inbound"))
|
||||
await runner.start()
|
||||
|
||||
bus.emit(
|
||||
"channel.inbound",
|
||||
ChannelMessage.of(
|
||||
InboundMessage(
|
||||
channel_id="s1",
|
||||
channel_type="stream-test",
|
||||
channel_subject=ChannelSubject(subject_id="u1"),
|
||||
content=[TextContent(text="把整站重构一遍")],
|
||||
)
|
||||
),
|
||||
)
|
||||
await asyncio.sleep(0.05)
|
||||
await runner.wait_idle()
|
||||
statuses = [run.status for run in runner.runs(AgentReply.name)]
|
||||
await runner.stop()
|
||||
hub.close()
|
||||
await manager.stop()
|
||||
return channel.frames, statuses
|
||||
|
||||
frames, statuses = asyncio.run(_run())
|
||||
final = "做到一半\n\n" + BUDGET_REPLY.format(steps=200)
|
||||
assert frames[0] == (builtin.ACK_TEXT, False)
|
||||
assert frames[-1] == (final, True) # 收尾帧保留了部分内容 + 说明
|
||||
assert statuses == [RunStatus.FAILED]
|
||||
rows = asyncio.run(repo.list_messages(tmp_path / "bot.db", session_id="s1:u1"))
|
||||
assert rows[-1]["role"] == "assistant"
|
||||
assert rows[-1]["content"] == final
|
||||
|
||||
|
||||
def test_agent_reply_sends_media_after_stream_finishes(
|
||||
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
|
||||
) -> None:
|
||||
"""流式路径:文本走流式消息,媒体在文本定稿后单独补一条。"""
|
||||
StreamChannel.instances.clear()
|
||||
picture = tmp_path / "cat.png"
|
||||
picture.write_bytes(b"\x89PNG\r\n\x1a\n" + b"\x00" * 32)
|
||||
|
||||
async def fake_stream(
|
||||
task, settings=None, history=None, *, cache=None, steering=None, outbox=None
|
||||
):
|
||||
assert outbox is not None
|
||||
outbox.push(ImageContent(local_path=str(picture), mime_type="image/png"))
|
||||
yield "图"
|
||||
yield "图来了"
|
||||
|
||||
monkeypatch.setattr(builtin, "stream_task", fake_stream)
|
||||
|
||||
async def _run() -> tuple[
|
||||
list[tuple[str, bool]], list[list], list[RunStatus], list
|
||||
]:
|
||||
bus = EventBus()
|
||||
bus.bind_loop()
|
||||
hub = RpcHub(bus)
|
||||
manager = ChannelManager()
|
||||
channel = StreamChannel(channel_id="s1")
|
||||
await manager.add_channel(channel)
|
||||
await manager.start()
|
||||
_ = hub.serve(RPC_STREAM, manager.rpc_stream)
|
||||
_ = hub.serve(RPC_SEND, manager.rpc_send)
|
||||
|
||||
services = ChannelServices(
|
||||
channels=manager,
|
||||
settings=Settings(home=tmp_path),
|
||||
db=tmp_path / "bot.db",
|
||||
hub=hub,
|
||||
)
|
||||
runner = TaskRunner(bus, deps=services)
|
||||
runner.register(AgentReply)
|
||||
_ = runner.add_task(AgentReply.name, EventTrigger(event="channel.inbound"))
|
||||
await runner.start()
|
||||
|
||||
bus.emit(
|
||||
"channel.inbound",
|
||||
ChannelMessage.of(
|
||||
InboundMessage(
|
||||
channel_id="s1",
|
||||
channel_type="stream-test",
|
||||
channel_subject=ChannelSubject(subject_id="u1"),
|
||||
content=[TextContent(text="画只猫")],
|
||||
)
|
||||
),
|
||||
)
|
||||
await asyncio.sleep(0.05)
|
||||
await runner.wait_idle()
|
||||
statuses = [run.status for run in runner.runs(AgentReply.name)]
|
||||
await runner.stop()
|
||||
hub.close()
|
||||
await manager.stop()
|
||||
rows = await repo.list_messages(tmp_path / "bot.db", session_id="s1:u1")
|
||||
return channel.frames, channel.contents, statuses, rows
|
||||
|
||||
frames, contents, statuses, rows = asyncio.run(_run())
|
||||
assert frames[-1] == ("图来了", True) # 文本仍然走流式收尾
|
||||
assert statuses == [RunStatus.SUCCEEDED]
|
||||
assert len(contents) == 1 # 媒体单独一条,且只含媒体分片
|
||||
assert [type(part) for part in contents[0]] == [ImageContent]
|
||||
# 流式分支同样要落库附件:它和同步分支共用 _persist_reply,走的是另一处调用点
|
||||
assert rows[-1]["role"] == "assistant"
|
||||
assert rows[-1]["content"].startswith("图来了")
|
||||
assert "[附件] 图片" in rows[-1]["content"]
|
||||
assert str(picture) in rows[-1]["content"]
|
||||
|
||||
|
||||
def test_outbound_stream_payload_defaults() -> None:
|
||||
payload = OutboundStream(
|
||||
channel_id="c", subject_id="u", stream_id="s", text="x"
|
||||
|
||||
Reference in new issue
Block a user