Compare commits

...
4 Commits
Author SHA1 Message Date
openbot d0e187f74d feat(flows): 助手出站媒体写入会话历史(issue #1 验收标准 8)
智能体发的附件此前只下发不落库:下一轮模型不知道上一轮发过什么,用户说
「再发大点」时没有可回指的对象。新增 describe_outbound() 把出站分片整理成
一行引用说明(类型/大小/MIME/时长/本地路径),与入站 prepare_inbound 的
summary 格式对齐,只存引用不存字节。

两条处理链都改走 _persist_reply():
- 同步链:原先只在 reply 非空时落库,纯媒体回复会让这一轮完全消失
- 流式链:原先同样只写正文

历史写入保持在发送之前——发送失败不该让下一轮失去上下文,因此不回滚,
且刻意不写「已发送」。

tests: test_builtin 3 条(附件入库、纯媒体入库、发送失败仍入库),
test_stream 补断流式分支的落库断言。变异验证:把两处调用点改回旧写法,
对应断言分别变红。
2026-11-10 14:09:29 +08:00
openbot 13b8d7a466 test(media): 补分块下载中断与 root 自动创建的实测
上一轮两条「未验证」的敞口,本次用可执行的测试消掉:

分块传输(Transfer-Encoding: chunked):
- download_limited 的 Content-Length 预检对分块响应无效,只剩按块累计
  这一道闸。此前只有代码没有测试,无法证明超限时连接真的被中断。
- 新增 _ChunkedServer:起一个真 socket 的最小 HTTP/1.1 服务端,不下发
  Content-Length、按 16 KiB 分块持续写 8 MiB。用真连接而不是桩,因为
  要验证的正是「读到超限就断开」——桩测不出中断。
- 断言两条:抛 MediaRejected 且 detail 含「已中断」;服务端 sent < total
  说明它没能把 8 MiB 写完,即连接在中途被断开,而不是读完再判。
- 变异验证:临时移除 media.py 里的累计判限后,该用例从通过变为
  DID NOT RAISE,确认它测的确实是这段逻辑;随后已还原,media.py 与
  HEAD 无差异(git diff 为空)。

root 不存在:
- store_media 用 mkdir(parents=True, exist_ok=True) 建目录,但此前没有
  断言覆盖「首次运行时 home/data/inbound 还不存在」这条路径。

全量 295 passed,ruff 全绿,新增两条连续跑 3 次均稳定(涉及真 socket
与线程,排除偶发)。
2026-11-10 10:22:21 +08:00
openbot 5ca40c6b0e feat(channels): 平台解密、下载限长与 CLI 媒体命令(issue #1 P1)
P0 接好了链路与钩子抽象,但几个实际会让功能不可用的点没处理:企业微信的
aeskey 记录下来后被直接丢弃、远端下载把整份响应读进内存、没有真实平台时
无法造出媒体消息来调试。本次补齐这三块。

企业微信解密:
- WeComChannel.make_media_transform 按本条消息的 <media>.aeskey 构造闭包,
  用 SDK 的 decrypt_file 解密(IV 取密钥前 16 字节、PKCS#7 去填充)。
  不在本仓库重写密码学——那只会多出第二份可能与平台不一致的实现
- 没有对应 url 的密钥时原样返回字节,不猜、不抛:由 MIME 嗅探去判定
- SDK 缺失时抛 MediaRejected(含安装提示),不让 ImportError 冒泡打断整轮

下载限长:
- BaseChannel.fetch_remote_media 从 resp.content 整份读改成 client.stream
  按块累计,Content-Length 先预检、流式累计超限即中断
- 上限取各类型限制的最大值:此方法拿不到分片类型,宁可放到最宽的一档,
  精确的按类型判定仍由 store_media 负责

CLI 媒体入口:
- /image <路径>、/file <路径> 解析成本地附件分片,走与其他渠道同一套
  media.py 策略(存在性、大小上限、MIME 嗅探、文件名净化),不因为
  「本地」就跳过
- 拒绝时降级为一条说明文本而非抛异常:CLI 里抛异常会直接终止输入泵
- _send_media 从打印占位行改为回执真实字节的大小与 sha256 前缀。
  CLI 里不存在「发送」这回事,所以不打印「已发送」

顺带修的健壮性问题:
- store_media 的临时文件名加 uuid 后缀。原先固定 .part,同一会话并发落盘
  同名文件时会互相截断,且 try/finally 保证失败不留半份字节
- read_file_limited 的超限文案原先把同一个数写了两遍(「大于 8 MiB
  (上限 8 MiB)」),改为「文件大于 8 MiB(上限 8 MiB)」,并注明只多读了
  1 字节所以只能说「大于」
- cli.py 的 except Exception 收窄为 (OSError, ValueError, TypeError):
  CancelledError 必须能穿过这一层

测试:新增 tests/test_media_pipeline.py 43 条,按 issue 的六条验收标准逐条
对应(路径穿越与净化、读取与下载限长、MIME 魔数优先、outbox 上限与上下文
隔离、prepare_inbound 多模态与降级、CLI 收发)。文档 architecture.md 补
安全护栏、平台解密、CLI 入口三段。

未验证(见 PR 描述):
- 没有一条真实的企业微信加密媒体消息跑过解密路径,单测里这一环是桩
- 分块传输(Transfer-Encoding: chunked)未构造响应实测中断逻辑
2026-11-10 09:47:30 +08:00
openbot 2dde0d1160 feat(flows): 媒体入站进出站链路接线(issue #1 P0)
上一轮提交 9ce0ba0 把 media.py / outbox.py 的骨架铺好了,但没有任何调用方:
入站附件进不了上下文,出站 MediaOutbox 从不被创建,send_image / send_file
两个工具因此恒定返回「当前不在渠道会话中」。本次把链路接上。

入站:
- AgentReply 改用 prepare_inbound 整理分片:小图内联为 image_url 块,
  大图与音视频退化为「路径 + 元信息」,文本类文件抽取正文进上下文
- 附件全部失败时回「附件未能接收:<原因>」,不再误报成「(无输出)」
- 正文用 prepared.summary(正文 + 附件清单),历史只存清单不存字节
- 只有正文发纯文本,带附件才发 content blocks(is_multimodal)

出站:
- 每轮创建一个 MediaOutbox 传给 run_task / stream_task,工具往里登记分片,
  跑完 drain() 出来与正文合成一条 OutboundContent
- 只有媒体没有正文时不再补「(无输出)」;流式通道文本定稿后单独补发媒体
- ChannelServices.send_content 统一走「先文本、再逐个媒体」

解密钩子的抽象层:
- BaseChannel.media_transform(part, raw) 默认原样返回
- BaseChannel.make_media_transform(metadata) 按单条消息构造钩子。密钥随消息
  而来,挂在渠道实例上会让并发处理的两条消息串用彼此的密钥
- ChannelServices.media_transform 从渠道取钩子,渠道未注册时返回 None
  (不解密,但也不跳过后续的类型与大小校验)
- media.py 新增 MediaTransform 类型与 store_media(transform=)。钩子在类型嗅探
  与大小判定之前执行——对密文嗅魔数只会得到 octet-stream——并在转换后对明文
  再量一次上限,因为密文没超不等于明文没超

测试:test_builtin 增加 6 条媒体用例(文本+图、纯图、附件全失败、出站下发、
仅媒体不补空文案、流式补发),test_service / test_stream 的 fake_run_task 补
outbox 形参。全量 250 passed,ruff 全绿。

P0 边界说明:本提交只做接线与抽象,不含企业微信的具体解密实现、下载限长、
CLI 媒体命令与 docs 更新,这些在下一个提交(P1)。
2026-11-10 09:46:16 +08:00
11 changed files with 1511 additions and 55 deletions

No files matched your search

+30 -4
View File
@@ -317,16 +317,42 @@ ChannelManager(
`supports_steer = True`。处理链路:
```
ChannelMessage → 校验文本(空文本/纯媒体给固定提示)
ChannelMessage → prepare_inbound(附件落盘;小图内联成 image_url 块)
→ 无可用内容?→ 回「附件未能接收:<原因>」/「(无输出)」并结束
→ ensure_db → ensure_session → list_messages → build_history
→ add_message(user)
→ add_message(user, 正文 + 附件清单)
→ 建 MediaOutbox(本轮出站信箱)
→ open_stream() ┬ 支持流式:update(ACK) → stream_task 边生成边覆盖 → finish
└ 不支持:run_task → send_text 一次性发送
│ → 媒体在文本定稿后单独补一条
└ 不支持:run_task → 有媒体走 send_content,否则 send_text
→ 补记插话(_persist_steered)→ add_message(assistant)
```
媒体链路(issue #1 的 P0 部分):
- **入站**:`flows/media_input.prepare_inbound()` 把分片整理成多模态块——小图 base64
内联、大图与音视频退化为「路径 + 元信息」、文本类文件抽取正文;单个附件超限/类型
拒绝/下载失败只写进 `notes`,不打断整轮。字节落盘在 `home/data/inbound/<session>/`,
历史只存正文 + 附件清单,不存字节。
- **出站**:`agent/outbox.MediaOutbox` 是本轮的信箱,`send_image` / `send_file` 工具
往里登记分片;`run()` 结束后 `drain()` 出来,与正文合成**一条** `OutboundContent`,
由出站管线「先文本、再逐个媒体」下发,避免平台上出现两条消息。只有媒体没有正文时
不再补「(无输出)」;流式通道只送文本,媒体等文本定稿后单独补一条。
- **安全护栏**:所有字节来源(`data` / `local_path` / `url`)都经过
`channels/media.py` 的同一套策略——按类型的大小上限、魔数优先的 MIME 判定、
可执行扩展名拉黑、文件名净化与根目录约束;远端下载改为流式按块读取,
`Content-Length` 先预检、超限即中断,整份读进内存的路径已移除。
- **平台解密**:`BaseChannel.make_media_transform()` 按单条消息的 metadata 构造
解密钩子(密钥随消息而来,挂在渠道实例上会串台),企业微信用
`<media>.aeskey` 走 SDK 的 `decrypt_file`。解密在类型嗅探**之前**执行,并对
明文再量一次大小——密文没超不等于明文没超。
- **CLI 调试入口**:`/image <路径>`、`/file <路径>` 可以直接在无平台环境造出
媒体入站消息;出站 `_send_media` 不再打印占位行,改为回执真实字节的
大小与 sha256 前缀(CLI 里没有「已发送」这回事)。
`ChannelServices` 是它的运行期依赖(`channels` / `settings` / `db` / `hub` / `cache` /
历史条数与字符预算 / 发送超时),由 `TaskRunner(deps=...)` 注入。
历史条数与字符预算 / 发送超时),由 `TaskRunner(deps=...)` 注入;`media_policy()` 从
渠道取媒体大小上限与下载请求头,未知渠道退回默认值(不因为查不到渠道就跳过校验)。
`on_error` 钩子会尽量回一句 `ERROR_REPLY`,发送失败只记日志(避免错误处理再次抛错)。
两个细节:
+48 -4
View File
@@ -30,6 +30,7 @@ import httpx
from bot042.channels.media import (
MediaLimits,
MediaRejected,
MediaTransform,
decode_base64_limited,
download_limited,
format_size,
@@ -618,15 +619,58 @@ class BaseChannel(ABC):
"""下载入站媒体时附带的请求头(平台鉴权等);默认无。"""
return {}
def media_transform(self, part: ContentPart, raw: bytes) -> bytes:
"""平台钩子:把刚取到的媒体字节转换成明文(如企业微信的 AES 解密)。
默认原样返回。实现方应假定 ``raw`` 可能是密文,且**必须**是纯函数语义
——上层会拿返回值的魔数重新判定类型、并再量一次大小。
这是**无消息上下文**的版本;需要按单条消息取密钥(如 ``aeskey``)时
覆写 :meth:`make_media_transform`。
"""
return raw
def make_media_transform(self, metadata: dict[str, Any]) -> MediaTransform:
"""按单条入站消息的元数据构造解密钩子。
密钥这类材料随消息而来(企业微信的 ``<media>.aeskey``),挂在渠道实例上
会串台。返回闭包把它绑在这一条消息上,消息处理完即随之释放。
Args:
metadata: 入站消息的 ``metadata``。
"""
_ = metadata
return self.media_transform
async def fetch_remote_media(self, url: str) -> tuple[bytes, str]:
"""下载远端媒体并返回 ``(bytes, content_type)``;子类可覆写加平台鉴权。"""
async with httpx.AsyncClient(follow_redirects=True) as client:
resp = await client.get(url)
"""下载远端媒体并返回 ``(bytes, content_type)``;子类可覆写加平台鉴权。
**带大小闸**:``resp.content`` 会把整份响应读进内存,平台给出的 URL
可能是任意大的文件,因此这里改为流式按块读取并在超限时中断。
"""
limits = self.media_limits
async with httpx.AsyncClient(
follow_redirects=True, timeout=limits.timeout
) as client, client.stream("GET", url) as resp:
resp.raise_for_status()
content_type = resp.headers.get("content-type") or (
mimetypes.guess_type(url)[0] or "application/octet-stream"
)
return resp.content, content_type
budget = max(limits.image, limits.audio, limits.video, limits.file)
declared = resp.headers.get("content-length", "")
if declared.isdigit() and int(declared) > budget:
raise MediaRejected(
"媒体超过大小上限",
f"Content-Length {format_size(int(declared))} > {format_size(budget)}",
)
buffer = bytearray()
async for piece in resp.aiter_bytes(limits.chunk):
buffer.extend(piece)
if len(buffer) > budget:
raise MediaRejected(
"媒体超过大小上限", f"下载超过 {format_size(budget)} 已中断"
)
return bytes(buffer), content_type
async def load_media_bytes(
self, part: ContentPart, *, limits: MediaLimits | None = None
+108 -5
View File
@@ -9,15 +9,26 @@ from __future__ import annotations
import asyncio
import contextlib
import hashlib
import logging
import mimetypes
import sys
from pathlib import Path
from typing import Any
from bot042.channels.base import BaseChannel
from bot042.channels.media import (
MediaRejected,
format_size,
read_file_limited,
sanitize_filename,
)
from bot042.channels.models import (
ChannelConfig,
ChannelSubject,
ContentPart,
FileContent,
ImageContent,
InboundMessage,
MessageProcessor,
TextContent,
@@ -29,6 +40,9 @@ logger = logging.getLogger(__name__)
_EXIT_WORDS = frozenset({"exit", "quit"})
_LOCAL_SUBJECT = "local"
_MEDIA_COMMANDS = {"image": "image", "img": "image", "file": "file", "media": "file"}
"""``/image <路径>``、``/file <路径>``:无真实平台时用来调试媒体收发。"""
class CliConfig(ChannelConfig):
"""CLI 渠道配置。"""
@@ -105,9 +119,18 @@ class CliChannel(BaseChannel):
return False
def parse_inbound(self, raw_payload: Any) -> InboundMessage:
"""CLI 的原始负载就是一行文本(或已构建的 InboundMessage)。"""
"""CLI 的原始负载就是一行文本(或已构建的 InboundMessage)。
``/image <路径>`` 与 ``/file <路径>`` 会解析成本地附件分片,
路径校验(存在性、大小上限、类型嗅探)与其他渠道共用同一套
:mod:`bot042.channels.media` 策略,不因为「本地」就跳过。
"""
if isinstance(raw_payload, InboundMessage):
return raw_payload
line = str(raw_payload)
parts = self._parse_media_command(line)
if parts is None:
parts = [TextContent(text=line)]
return InboundMessage(
channel_id=self.channel_id,
channel_type=self.channel_type,
@@ -117,9 +140,57 @@ class CliChannel(BaseChannel):
chat_type="direct",
),
channel_session_id=self.channel_type,
content=[TextContent(text=str(raw_payload))],
content=parts,
)
def _parse_media_command(self, line: str) -> list[ContentPart] | None:
"""解析 ``/image``、``/file`` 命令;不是命令时返回 ``None``。
附件被拒(超限、不存在、类型不符)时返回一条说明文本而不是抛异常:
CLI 里抛异常会直接终止输入泵,用户看到的是进程退出而不是提示。
"""
text = line.strip()
if not text.startswith("/"):
return None
head, _, rest = text[1:].partition(" ")
kind = _MEDIA_COMMANDS.get(head.lower())
if kind is None:
return None
path_text = rest.strip()
if not path_text:
return [TextContent(text=f"用法:/{head} <本地文件路径>")]
# 引号包裹的路径(含空格)直接剥掉两端引号
path_text = path_text.strip("\"'")
limits = self.media_limits
try:
path = Path(path_text).expanduser()
limit = limits.image if kind == "image" else limits.file
raw = read_file_limited(path, limit)
except MediaRejected as exc:
return [TextContent(text=f"附件未接收:{exc.player_reason}({exc.detail})")]
mime = mimetypes.guess_type(path.name)[0] or "application/octet-stream"
if kind == "image":
if not mime.startswith("image/"):
return [
TextContent(
text=f"附件未接收:不是图片类型({mime}),如为普通文件请用 /file"
)
]
part: ContentPart = ImageContent(
local_path=str(path),
size=len(raw),
mime_type=mime,
alt_text=sanitize_filename(path.name),
)
else:
part = FileContent(
local_path=str(path),
filename=sanitize_filename(path.name),
size=len(raw),
mime_type=mime,
)
return [part]
# ----- 平台出站原语 -----
async def _send_text(self, subject: ChannelSubject, text: str) -> None:
@@ -133,9 +204,41 @@ class CliChannel(BaseChannel):
await self._send_media(subject, part)
async def _send_media(self, subject: ChannelSubject, media: ContentPart) -> None:
name = getattr(media, "filename", "") or getattr(media, "alt_text", "") or media.type
url = getattr(media, "url", "") or getattr(media, "local_path", "")
_safe_print(f"[媒体] {media.type}: {name} {url}".rstrip())
"""CLI 无法真正投递媒体,改为回执:路径 + 大小 + sha256 前缀。
回执要能被人当场验证,所以不打印「已发送」——CLI 里没有发送这回事。
存在性、大小上限与类型嗅探与其他渠道共用同一套策略;被拒时打印原因,
不抛异常(异常会让出站链路把整条回复判失败)。
"""
_ = subject
name = (
getattr(media, "filename", "")
or getattr(media, "alt_text", "")
or media.type
)
limits = self.media_limits
limit = limits.image if media.type == "image" else limits.file
try:
raw, _mime = await self.load_media_bytes(media)
if len(raw) > limit:
raise MediaRejected(
"媒体超过大小上限",
f"{format_size(len(raw))} > {format_size(limit)}",
)
except MediaRejected as exc:
_safe_print(f"[媒体被拒] {media.type} {name}:{exc.player_reason}({exc.detail})")
return
except (OSError, ValueError, TypeError) as exc:
# 下载/读取失败时只回执,不打断后续分片。这里不捕获 BaseException,
# 也不吞 CancelledError——取消必须能穿过这一层。
_safe_print(f"[媒体失败] {media.type} {name}:{type(exc).__name__}: {exc}")
return
digest = hashlib.sha256(raw).hexdigest()[:16]
where = getattr(media, "local_path", "") or getattr(media, "url", "") or "(内存)"
_safe_print(
f"[媒体回执] {media.type}: {name} {format_size(len(raw))} "
f"sha256:{digest} 来源 {where}"
)
register_channel("cli", CliChannel, CliConfig)
+34
View File
@@ -20,6 +20,7 @@ from typing import Any, ClassVar
import httpx
from bot042.channels.base import BaseChannel, ChannelCredentialsError
from bot042.channels.media import MediaRejected, MediaTransform
from bot042.channels.models import (
AudioContent,
ChannelConfig,
@@ -408,6 +409,39 @@ class WeComChannel(BaseChannel):
filename = f"{media_type}{extension}"
return media_type, filename or default_name
# ----- 入站媒体解密 -----
def make_media_transform(self, metadata: dict[str, Any]) -> MediaTransform:
"""按本条的 ``aeskey`` 构造解密钩子(密钥随消息而来,不挂在实例上)。
企业微信推送的媒体 URL 返回的是 AES-256-CBC 密文,密钥在消息体的
``<media>.aeskey`` 里(``parse_inbound`` 记进 ``_media_aes_keys``)。
不传密钥时 SDK 的 ``download_file`` 只会原样返回密文,因此"记录了
aeskey 却不解密"等于入站图片全是不可用的字节。
解密用 SDK 自带的 ``decrypt_file``(IV 取密钥前 16 字节,PKCS#7 去填充),
不在这里重新实现一遍密码学——那只会多出一份可能与平台不一致的实现。
"""
keys = metadata.get("_media_aes_keys") or {}
if not isinstance(keys, dict) or not keys:
return super().media_transform
def _decrypt(part: ContentPart, raw: bytes) -> bytes:
url = getattr(part, "url", "") or ""
aes_key = keys.get(url)
if not aes_key:
# 没有密钥就是没有密钥,不要猜:返回原字节,由 MIME 嗅探去判定
return raw
try:
from wecom_aibot_sdk import decrypt_file
except ImportError as exc: # pragma: no cover - SDK 是可选依赖
raise MediaRejected(
"媒体解密不可用", "需要 wecom-aibot-sdk(含 cryptography)"
) from exc
return decrypt_file(raw, aes_key)
return _decrypt
# ----- 主动推送路由辅助 -----
def _enrich_push_metadata(
+65 -10
View File
@@ -24,6 +24,8 @@ import hashlib
import mimetypes
import re
import unicodedata
import uuid
from collections.abc import Callable
from dataclasses import dataclass
from pathlib import Path
from typing import Final
@@ -46,6 +48,7 @@ __all__ = [
"BLOCKED_EXTENSIONS",
"MediaLimits",
"MediaRejected",
"MediaTransform",
"StoredMedia",
"check_mime",
"decode_base64_limited",
@@ -58,6 +61,14 @@ __all__ = [
"store_media",
]
MediaTransform = Callable[[ContentPart, bytes], bytes]
"""入站字节的转换钩子:``(分片, 原始字节) -> 明文字节``。
企业微信的媒体 URL 返回的是 AES 密文(见 ``wecom_aibot_sdk.crypto``),
必须解密之后才能做类型嗅探与大小判定——在密文上嗅魔数只会得到
``application/octet-stream``,从而把所有图片都判成「类型不被接受」。
"""
MB: Final[int] = 1024 * 1024
# ---------------------------------------------------------------------------
@@ -290,8 +301,9 @@ def read_file_limited(path: Path, limit: int) -> bytes:
with path.open("rb") as handle:
data = handle.read(limit + 1)
if len(data) > limit:
# 只多读了 1 字节,所以这里只能说「大于」,说不出真实大小
raise MediaRejected(
"媒体超过大小上限", f"大于 {format_size(limit)}(上限 {format_size(limit)})"
"媒体超过大小上限", f"文件大于 {format_size(limit)}(上限 {format_size(limit)})"
)
return data
@@ -443,6 +455,7 @@ async def store_media(
limits: MediaLimits | None = None,
session_key: str = "",
headers: dict[str, str] | None = None,
transform: MediaTransform | None = None,
) -> StoredMedia:
"""把媒体分片落盘到 ``root/<session>/``,返回更新后的分片。
@@ -450,9 +463,20 @@ async def store_media(
(``data`` → ``local_path`` → ``url``),但每一步都带大小闸;
落盘后分片只保留 ``local_path``,让下游(智能体、出站回显)共用一份字节。
Args:
transform: 可选的字节转换钩子(如平台解密)。**在类型嗅探与大小判定
之前**执行,因为对密文嗅魔数没有意义;转换后的明文字节会再跑一次
上限检查——解密后的长度不可由密文长度保证(见下)。
session_key: 会话子目录名(净化后使用)。
headers: 下载远端媒体时附带的请求头。
Raises:
MediaRejected: 超限、类型不符、路径非法。
MediaRejected: 超限、类型不符、路径非法、转换失败。
httpx.HTTPError: 远端下载失败(调用方降级为文本提示)。
注意:``download_limited`` 的上限约束的是**收到的字节**。AES-CBC 解密后
明文通常更短(去掉填充),但把「密文没超」当成「明文没超」是错的推理,
所以转换后再量一次。
"""
limits = limits or MediaLimits()
if isinstance(part, TextContent):
@@ -464,11 +488,11 @@ async def store_media(
if part.data:
raw = decode_base64_limited(part.data, limit)
mime = check_mime(part, mime_declared or guess_mime(name_hint) or "", raw[:32])
declared = mime_declared or guess_mime(name_hint) or ""
elif part.local_path:
source = Path(part.local_path)
raw = read_file_limited(source, limit)
mime = check_mime(part, mime_declared or guess_mime(str(source)) or "", raw[:32])
declared = mime_declared or guess_mime(str(source)) or ""
name_hint = name_hint or source.name
elif part.url:
raw, fetched = await download_limited(
@@ -478,26 +502,57 @@ async def store_media(
timeout=limits.timeout,
headers=headers,
)
mime = check_mime(part, mime_declared or fetched or guess_mime(name_hint) or "", raw[:32])
declared = mime_declared or fetched or guess_mime(name_hint) or ""
else:
raise MediaRejected("媒体缺少字节来源", "data/local_path/url 三者皆空")
if transform is not None:
try:
raw = transform(part, raw)
except MediaRejected:
raise
except Exception as exc: # 平台解密失败不应冒泡成未捕获异常
raise MediaRejected("媒体解密失败", f"{type(exc).__name__}: {exc}") from exc
if len(raw) > limit:
raise MediaRejected(
"媒体超过大小上限",
f"解密后 {format_size(len(raw))} > {format_size(limit)}",
)
# 密文的 MIME 声明不可信:解密后按魔数重新判定,声明类型只作兜底
declared = mime_declared or guess_mime(name_hint) or ""
mime = check_mime(part, declared, raw[:32])
folder = root
if session_key:
folder = root / sanitize_filename(session_key, default="session", max_len=60)
folder.mkdir(parents=True, exist_ok=True)
target = safe_join(folder, sanitize_filename(name_hint) if name_hint else _default_name_for(part, mime))
target = safe_join(
folder,
sanitize_filename(name_hint) if name_hint else _default_name_for(part, mime),
)
digest = hashlib.sha256(raw).hexdigest()
# 同名不同内容时加哈希前缀,避免后到的覆盖先到的(也避免穿越后仍撞名)
if target.exists():
target = safe_join(folder, f"{digest[:12]}-{target.name}")
# 写临时文件再原子改名:中途失败不会留下半份字节
temp = target.with_name(target.name + ".part")
# 写临时文件再原子改名:中途失败不会留下半份字节。
# 临时名带 uuid:同一会话并发落盘同名文件时,共用一个 ".part" 会互相截断。
temp = target.with_name(f"{target.name}.{uuid.uuid4().hex[:8]}.part")
try:
temp.write_bytes(raw)
temp.replace(target)
finally:
temp.unlink(missing_ok=True)
updated = part.model_copy(update={"local_path": str(target), "url": "", "data": None,
"size": len(raw), "mime_type": mime})
updated = part.model_copy(
update={
"local_path": str(target),
"url": "",
"data": None,
"size": len(raw),
"mime_type": mime,
}
)
return StoredMedia(part=updated, path=target, size=len(raw), mime=mime, sha256=digest)
+134 -22
View File
@@ -16,13 +16,16 @@
from __future__ import annotations
import logging
from collections.abc import Sequence
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any
from bot042.agent.middlewares.budget import StepBudgetExceeded
from bot042.agent.middlewares.steering import SteeringBuffer
from bot042.agent.outbox import MediaOutbox
from bot042.channels.manager import ChannelManager
from bot042.channels.media import MediaLimits, MediaTransform
from bot042.channels.models import (
ChannelMessage,
ContentPart,
@@ -30,6 +33,7 @@ from bot042.channels.models import (
OutboundContent,
OutboundStream,
OutboundText,
TextContent,
)
from bot042.config import Settings
from bot042.db import repo
@@ -40,6 +44,7 @@ from bot042.flows.agent_task import (
run_task,
stream_task,
)
from bot042.flows.media_input import describe_outbound, prepare_inbound
from bot042.flows.processor import (
DEFAULT_HISTORY_CHARS,
DEFAULT_HISTORY_LIMIT,
@@ -173,6 +178,31 @@ class ChannelServices:
return
await self.channels.send_content(channel_id, subject_id, parts)
def media_policy(self, channel_id: str) -> tuple[MediaLimits, dict[str, str]]:
"""渠道的媒体大小上限与下载请求头。
未知渠道(未注册或已下线)退回默认策略值:宁可让校验按默认限跑,
也不要因为拿不到渠道就把这道关跳过。
"""
channel = self.channels.get_channel(channel_id)
if channel is None:
return MediaLimits(), {}
return channel.media_limits, channel.media_download_headers()
def media_transform(
self, channel_id: str, metadata: dict[str, Any]
) -> MediaTransform | None:
"""按本条消息的元数据取渠道的解密钩子(如企业微信的 ``aeskey``)。
密钥随消息而来,必须按条构造,不能挂在渠道实例上——挂实例会让并发处理
的两条消息串用彼此的密钥。渠道未注册时返回 ``None``,即"不解密",
由类型嗅探在明文上判定,不会因为查不到渠道就跳过后续校验。
"""
channel = self.channels.get_channel(channel_id)
if channel is None:
return None
return channel.make_media_transform(metadata)
async def stream_text(
self,
channel_id: str,
@@ -233,6 +263,61 @@ class ChannelServices:
return ReplyStream(_send)
async def _reply_with(
services: ChannelServices,
payload: ChannelMessage,
meta: dict[str, Any],
text: str,
parts: list[ContentPart],
) -> None:
"""回复一条消息:有媒体分片就走富内容(文本在前),否则走纯文本。
文本与媒体分成两次发送,在平台上会变成两条消息;合成一次
``send_content`` 后由出站管线按「先文本、再逐个媒体」的顺序下发。
"""
if parts:
lead: list[ContentPart] = [TextContent(text=text)] if text else []
await services.send_content(
payload.channel_id,
payload.subject_id,
[*lead, *parts],
metadata=meta,
)
return
await services.send_text(payload.channel_id, payload.subject_id, text, metadata=meta)
async def _persist_reply(
services: ChannelServices,
session_id: str,
channel: str,
text: str,
parts: Sequence[ContentPart],
*,
empty_ok: bool = False,
) -> None:
"""把一轮助手回复写入会话历史:正文 + 附件清单(不存字节)。
``parts`` 非空时即使正文为空也落一行——「只发了张图」是真实发生过的
一轮对话,丢掉它会让下一轮模型不知道自己发过什么,用户说「再发大点」
时就没有可回指的对象。
``empty_ok=False`` 且两者都空时不写(纯占位回复 ``EMPTY_REPLY`` 不入库,
与既有行为一致:那是给用户看的兜底文案,不是助手说的话)。
"""
summary = describe_outbound(parts)
content = "\n".join(piece for piece in (text.strip(), summary) if piece)
if not content and not empty_ok:
return
await repo.add_message(
services.db,
session_id=session_id,
role="assistant",
content=content,
channel=channel,
)
class AgentReply(Task):
"""渠道消息任务:读历史 → 调智能体 → 落库 → 回复。
@@ -313,11 +398,25 @@ class AgentReply(Task):
message = payload.message
meta = reply_metadata(message)
text = message.text.strip()
# 入站:附件先落盘再进上下文(小图内联为 base64,大图与音视频退化为
# 「路径 + 元信息」);单个附件失败只记一条说明,不打断整轮。
limits, headers = services.media_policy(payload.channel_id)
prepared = await prepare_inbound(
message,
root=services.settings.home / "data" / "inbound",
limits=limits,
headers=headers,
transform=services.media_transform(payload.channel_id, message.metadata),
)
stream = services.open_stream(payload.channel_id, payload.subject_id, meta)
if not text:
reply = "暂仅支持文本消息" if message.has_media else EMPTY_REPLY
if not prepared.has_content:
# 有附件但一个都没接住:说清原因,别让用户以为是发了条空消息
reply = (
"附件未能接收:" + ";".join(prepared.notes)
if prepared.notes
else EMPTY_REPLY
)
if stream is not None:
await stream.update(reply, force=True)
await stream.finish(reply)
@@ -327,6 +426,12 @@ class AgentReply(Task):
)
return
text = prepared.summary
# 出站:本轮的信箱,工具(send_image / send_file)往里登记,跑完统一下发
outbox = MediaOutbox(limits=limits)
# 只有正文时发纯文本,带附件才发多模态块列表(见 PreparedInbound.is_multimodal)
task_input = prepared.blocks if prepared.is_multimodal else text
session_id = payload.db_session_id
channel = payload.channel_type or payload.channel_id
await services.ensure_db()
@@ -344,27 +449,31 @@ class AgentReply(Task):
aborted: StepBudgetExceeded | None = None
try:
result = await run_task(
text,
task_input,
services.settings,
history=history,
cache=services.cache,
steering=self._steering,
outbox=outbox,
)
answer = last_message_text(result)
except StepBudgetExceeded as exc:
aborted = exc
await self._persist_steered(services, session_id, channel)
reply = _with_notice(answer, aborted)
if reply:
await repo.add_message(
services.db,
session_id=session_id,
role="assistant",
content=reply,
channel=channel,
# 智能体可能只发了媒体、没写正文:这时不必再补一句「(无输出)」
pending = outbox.drain()
# 媒体也要进历史:只发图不写字时正文为空,但这一轮确实产出过附件。
# 先落库再下发——发送失败不该让下一轮失去「我发过什么」的上下文。
await _persist_reply(
services, session_id, channel, reply, pending, empty_ok=bool(pending)
)
await services.send_text(
payload.channel_id, payload.subject_id, reply or EMPTY_REPLY, metadata=meta
await _reply_with(
services,
payload,
meta,
reply or ("" if pending else EMPTY_REPLY),
pending,
)
if aborted is not None:
# 已回话并落库:任务仍记失败(步数用尽是真的没做完),但不再发通用错误
@@ -380,11 +489,12 @@ class AgentReply(Task):
aborted = None
try:
async for partial in stream_task(
text,
task_input,
services.settings,
history=history,
cache=services.cache,
steering=self._steering,
outbox=outbox,
):
answer = partial
# 首段跳过节流:让用户在 ack 之后立刻看到内容开始出现
@@ -395,15 +505,17 @@ class AgentReply(Task):
finally:
# 步数用尽时把说明补在已经流出的内容后面,不必再单独发一条
final = _with_notice(answer, aborted)
await stream.finish(final or EMPTY_REPLY)
pending = outbox.drain()
# 只有媒体没正文时传空串:ReplyStream.finish("") 不会发任何文本
await stream.finish(final or ("" if pending else EMPTY_REPLY))
await self._persist_steered(services, session_id, channel)
if final:
await repo.add_message(
services.db,
session_id=session_id,
role="assistant",
content=final,
channel=channel,
await _persist_reply(
services, session_id, channel, final, pending, empty_ok=bool(pending)
)
if pending:
# 流式通道只送文本,媒体只能补一条(文本此时已经定稿)
await services.send_content(
payload.channel_id, payload.subject_id, pending, metadata=meta
)
if aborted is not None:
self._reported = True
+55 -1
View File
@@ -24,6 +24,7 @@
from __future__ import annotations
import base64
from collections.abc import Sequence
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any
@@ -34,6 +35,7 @@ from bot042.channels.media import (
MB,
MediaLimits,
MediaRejected,
MediaTransform,
format_size,
store_media,
)
@@ -47,7 +49,7 @@ from bot042.channels.models import (
VideoContent,
)
__all__ = ["InlinePolicy", "PreparedInbound", "prepare_inbound"]
__all__ = ["InlinePolicy", "PreparedInbound", "describe_outbound", "prepare_inbound"]
_TEXT_MIME_PREFIXES = ("text/",)
_TEXT_EXTENSIONS = frozenset(
@@ -150,6 +152,54 @@ def _describe(part: ContentPart, size: int, mime: str, path: Path) -> str:
return f"{' / '.join(bits)},已保存到 {path}"
def _outbound_size(part: ContentPart) -> int:
"""出站分片的字节数:优先模型声明的 ``size``,缺省按本地文件探测。
工具构造分片时总会填 ``size``;这里兜底是为了让手工构造的分片
(测试、插件)也能拿到合理描述,而不是在历史里写「0 B」。
"""
declared = getattr(part, "size", None)
if declared:
return int(declared)
path = getattr(part, "local_path", None)
if path:
try:
return Path(path).stat().st_size
except OSError:
return 0
return 0
def describe_outbound(parts: Sequence[ContentPart]) -> str:
"""把智能体发出的媒体整理成一行可入库的说明(只存引用,不存字节)。
格式与入站 :func:`prepare_inbound` 的 summary 对齐(``[附件] a;b``):
历史里「用户发的图」和「机器人发的图」长得一样,下一轮模型才能对照。
回指路径是本地文件而非平台地址——平台 `media_id` 有过期,
而本地落盘路径在会话目录内可复现。
**刻意不写「已发送」**:记录在调用下发之前写入(发送失败时也要留住这轮
上下文),此时还没有证据说它发出去了。历史只陈述「这轮产出过这个附件」。
"""
items: list[str] = []
for part in parts:
if isinstance(part, TextContent):
continue
bits = [_media_label(part), format_size(_outbound_size(part))]
mime = getattr(part, "mime_type", None)
if mime:
bits.append(mime)
duration = getattr(part, "duration", None)
if isinstance(duration, int) and duration > 0:
bits.append(f"{duration / 1000:.1f}s")
head = " / ".join(bits)
local = getattr(part, "local_path", None)
items.append(f"{head}(本地 {local})" if local else head)
if not items:
return ""
return "[附件] " + ";".join(items)
async def prepare_inbound(
message: InboundMessage,
*,
@@ -157,6 +207,7 @@ async def prepare_inbound(
limits: MediaLimits | None = None,
policy: InlinePolicy | None = None,
headers: dict[str, str] | None = None,
transform: MediaTransform | None = None,
) -> PreparedInbound:
"""把入站消息整理成 ``(text, blocks, summary, notes)``,媒体先落盘。
@@ -166,6 +217,8 @@ async def prepare_inbound(
limits: 字节上限;缺省 :class:`MediaLimits`。
policy: 图片内联策略;缺省 :class:`InlinePolicy`。
headers: 下载远端媒体时附带的请求头(平台鉴权由渠道提供)。
transform: 平台字节转换钩子(如企业微信的 AES 解密),在类型嗅探与
大小判定之前执行,由渠道按单条消息的 metadata 构造。
Returns:
:class:`PreparedInbound`。附件层面的失败只进 ``notes``,不抛异常。
@@ -190,6 +243,7 @@ async def prepare_inbound(
limits=limits,
session_key=session_key,
headers=headers,
transform=transform,
)
except MediaRejected as exc:
prepared.notes.append(f"{_media_label(part)}未接收:{exc.reason}")
+314 -3
View File
@@ -3,6 +3,7 @@
from __future__ import annotations
import asyncio
import base64
from datetime import datetime
from pathlib import Path
from typing import Any
@@ -15,6 +16,7 @@ from bot042.channels.manager import ChannelManager
from bot042.channels.models import (
ChannelMessage,
ChannelSubject,
ImageContent,
InboundMessage,
TextContent,
)
@@ -29,6 +31,7 @@ from bot042.flows.builtin import (
ChannelServices,
)
from bot042.flows.bus import EventBus
from bot042.flows.media_input import PreparedInbound
from bot042.flows.rpc import RpcHub
from bot042.flows.scheduler import CN_TZ
from bot042.flows.tasks import EventTrigger, RunStatus, TaskContext, TaskRunner
@@ -42,6 +45,8 @@ class EchoChannel(BaseChannel):
def __init__(self) -> None:
super().__init__(channel_id="c1")
self.sent: list[str] = []
self.contents: list[list[Any]] = []
"""每次富内容发送的分片组合(文本 + 媒体)。"""
async def start(self) -> None:
pass
@@ -60,7 +65,7 @@ class EchoChannel(BaseChannel):
self.sent.append(text)
async def _send_content(self, subject: ChannelSubject, parts: list) -> None:
pass
self.contents.append(list(parts))
async def _send_media(self, subject: ChannelSubject, media: Any) -> None:
pass
@@ -119,7 +124,7 @@ def test_agent_reply_persists_and_replies_via_rpc(
calls: list[tuple[str, list]] = []
async def fake_run_task(
task, settings=None, history=None, *, cache=None, steering=None
task, settings=None, history=None, *, cache=None, steering=None, outbox=None
):
calls.append((task, list(history or [])))
return {"messages": [{"role": "assistant", "content": "答:" + task}]}
@@ -223,7 +228,9 @@ def test_agent_reply_explains_step_budget_without_generic_error(
) -> None:
"""步数用尽:给出可操作说明并落库,且不再补发通用错误文案。"""
async def exhausted(task, settings=None, history=None, *, cache=None, steering=None):
async def exhausted(
task, settings=None, history=None, *, cache=None, steering=None, outbox=None
):
raise StepBudgetExceeded(200)
monkeypatch.setattr(builtin, "run_task", exhausted)
@@ -265,3 +272,307 @@ def test_agent_reply_explains_step_budget_without_generic_error(
("user", "把整站重构一遍"),
("assistant", notice),
]
# ---------------------------------------------------------------------------
# 媒体链路:入站进上下文、出站经信箱下发
# ---------------------------------------------------------------------------
PNG_HEAD = b"\x89PNG\r\n\x1a\n" + b"\x00\x00\x00\rIHDR" + b"\x00" * 16
"""最小 PNG 头:magic + IHDR 片段,足够 check_mime 认定成 image/png。"""
def _image_part() -> ImageContent:
"""入站图片分片:base64 直接带字节,不依赖网络与临时文件。"""
return ImageContent(
data=base64.b64encode(PNG_HEAD).decode("ascii"),
mime_type="image/png",
size=len(PNG_HEAD),
)
def _media_envelope(*parts: Any, text: str = "") -> ChannelMessage:
"""构造带任意内容分片的入站消息。"""
content = ([TextContent(text=text)] if text else []) + list(parts)
message = InboundMessage(
channel_id="c1",
channel_type="echo",
channel_session_id="s1",
channel_subject=ChannelSubject(subject_id="u1"),
content=content,
)
return ChannelMessage.of(message, session_key="s1")
def _drive(
db: Path,
payloads: list[ChannelMessage],
channel: EchoChannel,
*,
home: Path,
) -> list[RunStatus]:
"""跑一遍 AgentReply(调用方先 monkeypatch run_task / prepare_inbound)。
``home`` 指向临时目录,避免把测试媒体写进真实的 ``~/.bot042/data``。
"""
async def _run() -> list[RunStatus]:
bus = EventBus()
bus.bind_loop()
hub = RpcHub(bus)
manager = ChannelManager()
await manager.add_channel(channel)
await manager.start()
_ = hub.serve(RPC_SEND, manager.rpc_send)
services = ChannelServices(
channels=manager,
settings=Settings(home=home),
db=db,
hub=hub,
)
runner = TaskRunner(bus, deps=services)
runner.register(AgentReply)
_ = runner.add_task(AgentReply.name, EventTrigger(event="channel.inbound"))
await runner.start()
for payload in payloads:
bus.emit("channel.inbound", payload)
await runner.wait_idle(timeout=2)
await bus.wait_for_complete()
statuses = [run.status for run in runner.runs(AgentReply.name)]
await runner.stop()
hub.close()
await manager.stop()
return statuses
return asyncio.run(_run())
def test_agent_reply_feeds_inbound_image_to_agent(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""文本 + 图片:两者都进上下文,图片走 image_url 且不入库字节。"""
seen: list[Any] = []
async def fake_run_task(
task, settings=None, history=None, *, cache=None, steering=None, outbox=None
):
seen.append(task)
return {"messages": [{"role": "assistant", "content": "图里是一只猫"}]}
monkeypatch.setattr(builtin, "run_task", fake_run_task)
channel = EchoChannel()
statuses = _drive(
tmp_path / "bot.db",
[_media_envelope(_image_part(), text="这是什么")],
channel,
home=tmp_path,
)
assert statuses == [RunStatus.SUCCEEDED]
blocks = seen[0]
assert isinstance(blocks, list)
assert [b["type"] for b in blocks] == ["text", "image_url"]
assert blocks[0]["text"] == "这是什么"
assert blocks[1]["image_url"]["url"].startswith("data:image/png;base64,")
rows = asyncio.run(repo.list_messages(tmp_path / "bot.db", session_id="s1"))
assert rows[0]["role"] == "user"
assert "[附件]" in rows[0]["content"]
assert "base64" not in rows[0]["content"] # 历史只存清单,不存字节
# 落盘在 home 下的会话目录里
assert list((tmp_path / "data" / "inbound").rglob("*.png"))
# 回复没有媒体 → 仍走纯文本
assert channel.sent == ["图里是一只猫"]
assert channel.contents == []
def test_agent_reply_media_only_inbound_is_multimodal(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""只有图、没有正文:不能被当成空消息,也不能退化成纯文本把图丢掉。"""
seen: list[Any] = []
async def fake_run_task(
task, settings=None, history=None, *, cache=None, steering=None, outbox=None
):
seen.append(task)
return {"messages": [{"role": "assistant", "content": "收到"}]}
monkeypatch.setattr(builtin, "run_task", fake_run_task)
channel = EchoChannel()
statuses = _drive(
tmp_path / "bot.db", [_media_envelope(_image_part())], channel, home=tmp_path
)
assert statuses == [RunStatus.SUCCEEDED]
assert [b["type"] for b in seen[0]] == ["image_url"]
def test_agent_reply_reports_rejected_attachments(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""附件全部没接住:说清原因,不再笼统地当成「空消息」。"""
async def fake_prepare(
message, *, root, limits=None, policy=None, headers=None, transform=None
):
return PreparedInbound(text="", notes=["图片未接收:超过大小上限"])
async def boom(*args, **kwargs): # pragma: no cover - 不应被调用
raise AssertionError("附件全部失败时不应执行任务")
monkeypatch.setattr(builtin, "prepare_inbound", fake_prepare)
monkeypatch.setattr(builtin, "run_task", boom)
channel = EchoChannel()
statuses = _drive(
tmp_path / "bot.db", [_media_envelope(_image_part())], channel, home=tmp_path
)
assert statuses == [RunStatus.SUCCEEDED]
assert channel.sent == ["附件未能接收:图片未接收:超过大小上限"]
def test_agent_reply_delivers_agent_media_in_one_content_message(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""智能体用工具发的图:与正文合成一条富内容,不额外再发一条纯文本。"""
picture = tmp_path / "cat.png"
picture.write_bytes(PNG_HEAD)
async def fake_run_task(
task, settings=None, history=None, *, cache=None, steering=None, outbox=None
):
assert outbox is not None
outbox.push(ImageContent(local_path=str(picture), mime_type="image/png"))
return {"messages": [{"role": "assistant", "content": "图来了"}]}
monkeypatch.setattr(builtin, "run_task", fake_run_task)
channel = EchoChannel()
statuses = _drive(tmp_path / "bot.db", [_envelope("画只猫")], channel, home=tmp_path)
assert statuses == [RunStatus.SUCCEEDED]
assert channel.sent == [] # 没有再发一条纯文本
assert len(channel.contents) == 1
parts = channel.contents[0]
assert isinstance(parts[0], TextContent) and parts[0].text == "图来了"
assert isinstance(parts[1], ImageContent)
assert parts[1].local_path == str(picture)
def test_agent_reply_media_only_reply_skips_empty_placeholder(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""只发了媒体、没写正文:不要补一句「(无输出)」。"""
picture = tmp_path / "cat.png"
picture.write_bytes(PNG_HEAD)
async def fake_run_task(
task, settings=None, history=None, *, cache=None, steering=None, outbox=None
):
assert outbox is not None
outbox.push(ImageContent(local_path=str(picture), mime_type="image/png"))
return {"messages": [{"role": "assistant", "content": ""}]}
monkeypatch.setattr(builtin, "run_task", fake_run_task)
channel = EchoChannel()
statuses = _drive(tmp_path / "bot.db", [_envelope("发张图")], channel, home=tmp_path)
assert statuses == [RunStatus.SUCCEEDED]
assert channel.sent == []
assert [type(part) for part in channel.contents[0]] == [ImageContent]
def test_agent_reply_persists_agent_media_to_history(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""智能体发的附件要进历史:下一轮模型才知道自己发过什么。"""
async def fake_run_task(
task, settings=None, history=None, *, cache=None, steering=None, outbox=None
):
assert outbox is not None
outbox.push(
ImageContent(local_path=str(picture), mime_type="image/png", size=len(PNG_HEAD))
)
return {"messages": [{"role": "assistant", "content": "图来了"}]}
picture = tmp_path / "cat.png"
picture.write_bytes(PNG_HEAD)
monkeypatch.setattr(builtin, "run_task", fake_run_task)
channel = EchoChannel()
statuses = _drive(tmp_path / "bot.db", [_envelope("画只猫")], channel, home=tmp_path)
assert statuses == [RunStatus.SUCCEEDED]
rows = asyncio.run(repo.list_messages(tmp_path / "bot.db", session_id="s1"))
assert rows[-1]["role"] == "assistant"
content = rows[-1]["content"]
assert content.startswith("图来了")
assert "[附件] 图片" in content
assert str(picture) in content # 存路径引用,便于下一轮回指
assert "base64" not in content # 不存字节
def test_agent_reply_media_only_reply_still_lands_in_history(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""只发了图、没有正文:这一轮仍要写进历史,不能只剩用户那一句。"""
picture = tmp_path / "cat.png"
picture.write_bytes(PNG_HEAD)
async def fake_run_task(
task, settings=None, history=None, *, cache=None, steering=None, outbox=None
):
assert outbox is not None
outbox.push(ImageContent(local_path=str(picture), mime_type="image/png"))
return {"messages": [{"role": "assistant", "content": ""}]}
monkeypatch.setattr(builtin, "run_task", fake_run_task)
channel = EchoChannel()
statuses = _drive(tmp_path / "bot.db", [_envelope("发张图")], channel, home=tmp_path)
assert statuses == [RunStatus.SUCCEEDED]
rows = asyncio.run(repo.list_messages(tmp_path / "bot.db", session_id="s1"))
assert [r["role"] for r in rows] == ["user", "assistant"]
assert rows[-1]["content"].startswith("[附件] 图片")
assert "无输出" not in rows[-1]["content"] # 不把兜底文案当成助手说的话
def test_agent_reply_persists_reply_before_delivery(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""下发失败也要留住上下文:历史写入先于发送,且不回滚。"""
async def fake_run_task(
task, settings=None, history=None, *, cache=None, steering=None, outbox=None
):
assert outbox is not None
outbox.push(ImageContent(local_path=str(picture), mime_type="image/png"))
return {"messages": [{"role": "assistant", "content": "给你"}]}
picture = tmp_path / "cat.png"
picture.write_bytes(PNG_HEAD)
monkeypatch.setattr(builtin, "run_task", fake_run_task)
class BrokenChannel(EchoChannel):
async def _send_content(self, subject, parts): # type: ignore[override]
raise RuntimeError("平台发送失败")
channel = BrokenChannel()
statuses = _drive(tmp_path / "bot.db", [_envelope("发图")], channel, home=tmp_path)
assert statuses == [RunStatus.FAILED]
rows = asyncio.run(repo.list_messages(tmp_path / "bot.db", session_id="s1"))
assert [r["role"] for r in rows] == ["user", "assistant"]
assert "给你" in rows[-1]["content"]
def test_media_policy_falls_back_to_defaults(tmp_path: Path) -> None:
"""拿不到渠道实例时按默认上限跑,不能因为查不到渠道就跳过校验。"""
services = ChannelServices(
channels=ChannelManager(), settings=Settings(home=tmp_path), db=tmp_path / "bot.db"
)
limits, headers = services.media_policy("no-such-channel")
assert headers == {}
assert limits.image > 0 and limits.file > 0
+630
View File
@@ -0,0 +1,630 @@
"""issue #1 验收:媒体链路的接入、护栏与安全边界。
对应 issue 的验收标准(`docs` 与本文件一一对应):
1. 媒体能进上下文 —— `test_image_reaches_agent_context` 等;
2. 文本 + 文件混合不丢 —— `test_text_and_file_both_reach_agent`;
3. 智能体发媒体走原生下发 —— `test_outbox_part_flows_to_channel`;
4. 超限被拒且不炸内存 —— `test_oversize_*`;
5. 恶意文件名不越界 —— `test_sanitize_*` / `test_safe_join_*`;
6. 以上均有单测且 pytest 全绿。
本文件只测**媒体策略与链路**,不重复测 AgentReply 的会话/落库语义
(见 `tests/test_builtin.py`)。
"""
from __future__ import annotations
import asyncio
import base64
import socket
import sys
import threading
import types
from pathlib import Path
from typing import Self
import pytest
from bot042.agent.outbox import (
MediaOutbox,
bind,
current_outbox,
make_file_part,
make_image_part,
unbind,
)
from bot042.channels.cli import CliChannel
from bot042.channels.manager import ChannelManager
from bot042.channels.media import (
MB,
MediaLimits,
MediaRejected,
check_mime,
decode_base64_limited,
download_limited,
read_file_limited,
safe_join,
sanitize_filename,
store_media,
)
from bot042.channels.models import (
ChannelSubject,
FileContent,
ImageContent,
InboundMessage,
TextContent,
)
from bot042.config import Settings
from bot042.flows.builtin import ChannelServices
from bot042.flows.media_input import prepare_inbound
PNG_HEAD = b"\x89PNG\r\n\x1a\n" + b"\x00" * 64
PDF_HEAD = b"%PDF-1.4\n" + b"\x00" * 64
# ---------------------------------------------------------------------------
# 5. 路径安全:文件名净化与根目录约束
# ---------------------------------------------------------------------------
@pytest.mark.parametrize(
"hostile",
[
"../../etc/passwd",
"..\\..\\Windows\\System32\\config",
"/etc/shadow",
"C:\\Windows\\system.ini",
"....//....//evil.sh",
"a/../../../b.png",
],
)
def test_sanitize_filename_never_escapes(hostile: str) -> None:
"""恶意文件名净化后不得含路径分隔符,也不得是 . 或 .."""
cleaned = sanitize_filename(hostile)
assert "/" not in cleaned and "\\" not in cleaned
assert cleaned not in {".", ".."}
assert not cleaned.startswith(".")
def test_safe_join_keeps_target_inside_root(tmp_path: Path) -> None:
"""即使传入穿越片段,落点也必须在 root 之内。"""
root = tmp_path / "session"
root.mkdir()
target = safe_join(root, sanitize_filename("../../escape.png"))
assert target.parent == root
assert root in target.parents
def test_store_media_lands_inside_session_dir(tmp_path: Path) -> None:
"""落盘路径由会话键决定:会话键里的 .. 会被净化,不会跳出 root。"""
part = ImageContent(mime_type="image/png").model_copy(
update={"data": base64.b64encode(PNG_HEAD).decode()}
)
stored = asyncio.run(store_media(part, root=tmp_path, session_key="../s"))
assert tmp_path in stored.path.parents
assert ".." not in str(stored.path.relative_to(tmp_path))
# 会话键被净化成单个目录段,不是嵌套路径
assert stored.path.parent.parent == tmp_path
def test_store_media_dedupes_same_name(tmp_path: Path) -> None:
"""同名不同内容不得互相覆盖(后到的加哈希前缀)。"""
first = ImageContent(mime_type="image/png").model_copy(
update={"data": base64.b64encode(PNG_HEAD).decode()}
)
second = ImageContent(mime_type="image/png").model_copy(
update={"data": base64.b64encode(PNG_HEAD + b"other").decode()}
)
a = asyncio.run(store_media(first, root=tmp_path, session_key="s"))
b = asyncio.run(store_media(second, root=tmp_path, session_key="s"))
assert a.path != b.path
assert a.path.read_bytes() == PNG_HEAD
# ---------------------------------------------------------------------------
# 4. 大小护栏:本地读取、base64 解码、下载
# ---------------------------------------------------------------------------
def test_read_file_limited_rejects_oversize(tmp_path: Path) -> None:
big = tmp_path / "big.bin"
big.write_bytes(b"x" * 2048)
with pytest.raises(MediaRejected) as err:
read_file_limited(big, 1024)
assert "超过大小上限" in err.value.reason
def test_read_file_limited_reads_at_most_limit_plus_one(tmp_path: Path) -> None:
"""判定越界只需多读 1 字节,不得把整个超大文件读进内存。"""
big = tmp_path / "huge.bin"
big.write_bytes(b"x" * (8 * 1024 * 1024))
with pytest.raises(MediaRejected):
read_file_limited(big, 1024)
def test_read_file_limited_missing_file(tmp_path: Path) -> None:
with pytest.raises(MediaRejected) as err:
read_file_limited(tmp_path / "nope.bin", 1024)
assert err.value.reason == "文件不存在"
def test_decode_base64_limited_rejects_oversize() -> None:
payload = base64.b64encode(b"x" * 4096).decode()
with pytest.raises(MediaRejected):
decode_base64_limited(payload, 1024)
def test_decode_base64_limited_invalid() -> None:
with pytest.raises(MediaRejected):
decode_base64_limited("!!!not-base64!!!", 1024)
@pytest.mark.parametrize(
("declared", "head", "expected"),
[
("image/png", PNG_HEAD, "image/png"),
# 声明撒谎:实际是 PNG,按魔数修正(改扩展名绕不过类型限制)
("text/plain", PNG_HEAD, "image/png"),
("image/jpeg", PNG_HEAD, "image/png"),
],
)
def test_check_mime_prefers_magic_bytes(declared: str, head: bytes, expected: str) -> None:
"""声明类型与魔数冲突时以魔数为准。"""
got = check_mime(ImageContent(), declared, head)
assert got == expected
def test_check_mime_file_accepts_pdf() -> None:
"""非图片类文件走 FileContent,PDF 应放行并采信魔数。"""
got = check_mime(FileContent(filename="a.pdf"), "", PDF_HEAD)
assert got == "application/pdf"
def test_check_mime_image_rejects_pdf() -> None:
"""把 PDF 当图片投递:必须拒绝,而不是带着错误 MIME 继续。"""
with pytest.raises(MediaRejected) as err:
check_mime(ImageContent(), "application/pdf", PDF_HEAD)
assert "不被接受" in err.value.reason
def test_check_mime_rejects_executable() -> None:
"""可执行文件按扩展名拒绝:智能体读到的文件不该原样推给对方。"""
with pytest.raises(MediaRejected):
check_mime(FileContent(filename="virus.exe"), "", b"MZ")
def test_media_limits_per_type_is_ordered() -> None:
limits = MediaLimits()
assert limits.image <= limits.file
assert limits.for_part(ImageContent()).__class__ is int
# ---------------------------------------------------------------------------
# 3. 出站:outbox 的工具入口
# ---------------------------------------------------------------------------
def test_make_image_part_validates_type(tmp_path: Path) -> None:
fake = tmp_path / "notimage.png"
fake.write_bytes(b"plain text, not a png at all")
with pytest.raises(MediaRejected):
make_image_part(str(fake))
def test_make_image_part_ok(tmp_path: Path) -> None:
picture = tmp_path / "cat.png"
picture.write_bytes(PNG_HEAD)
part = make_image_part(str(picture), caption="猫")
assert isinstance(part, ImageContent)
assert part.size == len(PNG_HEAD)
assert part.mime_type == "image/png"
assert part.alt_text == "猫"
def test_make_file_part_rejects_missing(tmp_path: Path) -> None:
with pytest.raises(MediaRejected) as err:
make_file_part(str(tmp_path / "ghost.pdf"))
assert "不存在" in err.value.reason
def test_outbox_caps_pending_count() -> None:
"""单轮附件数量设上限,避免模型刷屏式调用把平台打爆。"""
box = MediaOutbox(max_pending=2)
part = ImageContent(mime_type="image/png")
box.push(part)
box.push(part)
with pytest.raises(MediaRejected):
box.push(part)
assert len(box.drain()) == 2
assert len(box) == 0
def test_outbox_context_binding_is_isolated() -> None:
box = MediaOutbox()
assert current_outbox() is None
token = bind(box)
try:
assert current_outbox() is box
finally:
unbind(token)
assert current_outbox() is None
# ---------------------------------------------------------------------------
# 1 & 2. 入站:多模态块构建与降级
# ---------------------------------------------------------------------------
def _prepare(**kwargs):
from bot042.flows.media_input import prepare_inbound
return prepare_inbound
def test_prepare_inbound_inlines_small_image(tmp_path: Path) -> None:
from bot042.channels.models import ChannelSubject, InboundMessage
from bot042.flows.media_input import prepare_inbound
picture = tmp_path / "cat.png"
picture.write_bytes(PNG_HEAD)
message = InboundMessage(
channel_id="c1",
channel_type="cli",
channel_subject=ChannelSubject(subject_id="u1"),
content=[
TextContent(text="这是什么"),
ImageContent(local_path=str(picture), mime_type="image/png"),
],
)
prepared = asyncio.run(prepare_inbound(message, root=tmp_path / "in"))
kinds = [b["type"] for b in prepared.blocks]
assert kinds == ["text", "image_url"]
assert prepared.blocks[1]["image_url"]["url"].startswith("data:image/png;base64,")
assert prepared.is_multimodal
# 摘要里带附件清单,且不含 base64
assert "[附件]" in prepared.summary
assert "base64" not in prepared.summary
def test_prepare_inbound_text_and_file_both_kept(tmp_path: Path) -> None:
"""文本 + 文件混合:两者都进上下文,不静默丢弃任何一个。"""
from bot042.channels.models import ChannelSubject, InboundMessage
from bot042.flows.media_input import prepare_inbound
note = tmp_path / "notes.txt"
note.write_text("第一行\n第二行\n", encoding="utf-8")
message = InboundMessage(
channel_id="c1",
channel_type="cli",
channel_subject=ChannelSubject(subject_id="u1"),
content=[
TextContent(text="帮我看看这个文件"),
FileContent(local_path=str(note), filename="notes.txt", mime_type="text/plain"),
],
)
prepared = asyncio.run(prepare_inbound(message, root=tmp_path / "in"))
joined = "\n".join(b.get("text", "") for b in prepared.blocks)
assert "帮我看看这个文件" in joined
assert "第一行" in joined # 文本文件内容被抽取进上下文
assert len(prepared.stored) == 1
def test_prepare_inbound_oversize_degrades_to_note(tmp_path: Path) -> None:
"""超限附件降级成一条说明,不抛异常、不中断其余内容。"""
from bot042.channels.models import ChannelSubject, InboundMessage
from bot042.flows.media_input import prepare_inbound
big = tmp_path / "big.png"
big.write_bytes(PNG_HEAD + b"\x00" * 4096)
message = InboundMessage(
channel_id="c1",
channel_type="cli",
channel_subject=ChannelSubject(subject_id="u1"),
content=[
TextContent(text="看图"),
ImageContent(local_path=str(big), mime_type="image/png"),
],
)
prepared = asyncio.run(
prepare_inbound(
message, root=tmp_path / "in", limits=MediaLimits(image=128, file=128)
)
)
assert prepared.has_content
assert prepared.notes
assert any("上限" in n or "超过" in n for n in prepared.notes)
def test_prepare_inbound_missing_file_is_noted(tmp_path: Path) -> None:
from bot042.channels.models import ChannelSubject, InboundMessage
from bot042.flows.media_input import prepare_inbound
message = InboundMessage(
channel_id="c1",
channel_type="cli",
channel_subject=ChannelSubject(subject_id="u1"),
content=[FileContent(local_path=str(tmp_path / "ghost.pdf"), filename="ghost.pdf")],
)
prepared = asyncio.run(prepare_inbound(message, root=tmp_path / "in"))
assert prepared.notes
assert not prepared.stored
# ---------------------------------------------------------------------------
# CLI 渠道:无平台时的收发调试入口
# ---------------------------------------------------------------------------
def test_cli_image_command(tmp_path: Path) -> None:
picture = tmp_path / "cat.png"
picture.write_bytes(PNG_HEAD)
channel = CliChannel()
message = channel.parse_inbound(f"/image {picture}")
assert isinstance(message.content[0], ImageContent)
assert message.content[0].mime_type == "image/png"
def test_cli_file_command(tmp_path: Path) -> None:
note = tmp_path / "a.txt"
note.write_text("hi", encoding="utf-8")
channel = CliChannel()
message = channel.parse_inbound(f"/file {note}")
assert isinstance(message.content[0], FileContent)
assert message.content[0].filename == "a.txt"
def test_cli_image_command_rejects_oversize(tmp_path: Path) -> None:
picture = tmp_path / "cat.png"
picture.write_bytes(PNG_HEAD + b"\x00" * 4096)
channel = CliChannel()
channel._media_limits = MediaLimits(image=64, file=64)
message = channel.parse_inbound(f"/image {picture}")
assert isinstance(message.content[0], TextContent)
assert "未接收" in message.content[0].text
def test_cli_image_command_wrong_type(tmp_path: Path) -> None:
note = tmp_path / "a.txt"
note.write_text("hi", encoding="utf-8")
channel = CliChannel()
message = channel.parse_inbound(f"/image {note}")
assert isinstance(message.content[0], TextContent)
assert "不是图片" in message.content[0].text
def test_cli_plain_text_unchanged() -> None:
channel = CliChannel()
message = channel.parse_inbound("你好")
assert isinstance(message.content[0], TextContent)
assert message.content[0].text == "你好"
def test_cli_send_media_reports_real_bytes(tmp_path: Path, capsys) -> None:
"""出站回执必须来自真实读取:给出大小与 sha256 前缀,而不是「已发送」。"""
picture = tmp_path / "cat.png"
picture.write_bytes(PNG_HEAD)
channel = CliChannel()
asyncio.run(
channel._send_media(None, ImageContent(local_path=str(picture), mime_type="image/png"))
)
out = capsys.readouterr().out
assert "媒体回执" in out
assert str(len(PNG_HEAD)) in out
assert "sha256:" in out
def test_cli_send_media_missing_file_is_reported(tmp_path: Path, capsys) -> None:
channel = CliChannel()
asyncio.run(
channel._send_media(
None, ImageContent(local_path=str(tmp_path / "ghost.png"), mime_type="image/png")
)
)
out = capsys.readouterr().out
assert "媒体" in out and ("被拒" in out or "失败" in out)
# ---------------------------------------------------------------------------
# 平台解密钩子:接线必须真的到达 store_media(不只是签名吻合)
# ---------------------------------------------------------------------------
def test_store_media_applies_transform_before_sniffing(tmp_path: Path) -> None:
"""transform 必须在类型嗅探**之前**执行:密文嗅魔数只会得到 octet-stream。"""
ciphertext = b"\x00\x01\x02\x03cipher-not-a-png" * 4
calls: list[tuple[str, int]] = []
def fake_decrypt(part, raw: bytes) -> bytes:
calls.append((part.type, len(raw)))
return PNG_HEAD
# 用内联字节而非 url:这条测的是 transform 与嗅探的顺序,不该触发网络
part = ImageContent(mime_type="application/octet-stream").model_copy(
update={"data": base64.b64encode(ciphertext).decode()}
)
stored = asyncio.run(store_media(part, root=tmp_path, transform=fake_decrypt))
assert calls == [("image", len(ciphertext))]
# 明文是 PNG,所以最终类型由**明文**魔数决定
assert stored.mime == "image/png"
assert stored.path.read_bytes() == PNG_HEAD
def test_store_media_rechecks_size_after_transform(tmp_path: Path) -> None:
"""密文没超不等于明文没超:转换后必须再量一次。"""
limits = MediaLimits(image=64)
def expand(part, raw: bytes) -> bytes:
return b"\x00" * (limits.image + 1)
part = ImageContent(mime_type="application/octet-stream").model_copy(
update={"data": base64.b64encode(b"small-cipher").decode()}
)
with pytest.raises(MediaRejected) as excinfo:
asyncio.run(store_media(part, root=tmp_path, limits=limits, transform=expand))
assert "解密后" in excinfo.value.detail
def test_store_media_transform_failure_becomes_rejection(tmp_path: Path) -> None:
"""解密异常要转成 MediaRejected,不能冒泡成未捕获异常打断整轮。"""
def broken(part, raw: bytes) -> bytes:
raise ValueError("bad padding")
part = ImageContent(mime_type="application/octet-stream").model_copy(
update={"data": base64.b64encode(b"cipher").decode()}
)
with pytest.raises(MediaRejected) as excinfo:
asyncio.run(store_media(part, root=tmp_path, transform=broken))
assert "解密失败" in excinfo.value.reason
def test_prepare_inbound_passes_transform_through(tmp_path: Path) -> None:
"""prepare_inbound 必须把 transform 透传到 store_media,否则解密形同虚设。"""
seen: list[int] = []
def fake_decrypt(part, raw: bytes) -> bytes:
seen.append(len(raw))
return PNG_HEAD
message = InboundMessage(
channel_id="c1",
channel_type="t",
channel_subject=ChannelSubject(subject_id="u1"),
content=[ImageContent(data=base64.b64encode(b"cipher-bytes").decode())],
)
prepared = asyncio.run(
prepare_inbound(message, root=tmp_path / "in", transform=fake_decrypt)
)
assert seen == [len(b"cipher-bytes")]
assert prepared.stored and prepared.stored[0].read_bytes() == PNG_HEAD
def test_wecom_media_transform_uses_per_message_aeskey(monkeypatch) -> None:
"""企业微信的解密钩子按本条消息的 url 取密钥;没有密钥时原样返回。"""
from bot042.channels.im import wecom as wecom_mod
calls: list[tuple[bytes, str]] = []
def fake_decrypt_file(raw: bytes, key: str) -> bytes:
calls.append((raw, key))
return PNG_HEAD
fake_sdk = types.SimpleNamespace(decrypt_file=fake_decrypt_file)
monkeypatch.setitem(sys.modules, "wecom_aibot_sdk", fake_sdk)
channel = wecom_mod.WeComChannel()
url = "https://example.invalid/media/1"
transform = channel.make_media_transform({"_media_aes_keys": {url: "k" * 43}})
part = ImageContent(url=url)
assert transform(part, b"cipher") == PNG_HEAD
assert calls == [(b"cipher", "k" * 43)]
# 没有对应 url 的密钥:返回原字节,不猜、不抛
other = ImageContent(url="https://example.invalid/media/2")
assert transform(other, b"cipher") == b"cipher"
def test_channel_services_media_transform_none_for_unknown_channel(tmp_path: Path) -> None:
"""渠道未注册时返回 None(不解密),不抛异常也不跳过后续校验。"""
services = ChannelServices(
channels=ChannelManager(), settings=Settings(home=tmp_path), db=tmp_path / "b.db"
)
assert services.media_transform("no-such-channel", {}) is None
# ---------------------------------------------------------------------------
# 4. 下载限长:分块传输(无 Content-Length)必须靠累计判限中断
# ---------------------------------------------------------------------------
class _ChunkedServer:
"""极简 HTTP/1.1 分块响应服务端,用于验证「服务端不给 Content-Length」时的闸。
真实平台可能用 ``Transfer-Encoding: chunked`` 下发媒体,此时
``Content-Length`` 预检形同虚设,只剩按块累计这一道。用真 socket 而不是
桩,是因为要验证的正是「读到超限就断开连接」——桩连接测不出中断。
"""
def __init__(self, total: int, piece: int = 16 * 1024) -> None:
self.total = total
self.piece = piece
self.sent = 0
self._sock = socket.socket()
self._sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self._sock.bind(("127.0.0.1", 0))
self._sock.listen(1)
self._sock.settimeout(10)
self.url = f"http://127.0.0.1:{self._sock.getsockname()[1]}/media"
def __enter__(self) -> Self:
self._thread = threading.Thread(target=self._serve, daemon=True)
self._thread.start()
return self
def __exit__(self, *exc_info: object) -> None:
self._thread.join(timeout=10)
self._sock.close()
def _serve(self) -> None:
try:
conn, _ = self._sock.accept()
except OSError:
return
with conn:
conn.settimeout(10)
head = b""
while b"\r\n\r\n" not in head:
try:
piece = conn.recv(4096)
except OSError:
return
if not piece:
return
head += piece
conn.sendall(
b"HTTP/1.1 200 OK\r\n"
b"Content-Type: image/png\r\n"
b"Transfer-Encoding: chunked\r\n\r\n"
)
payload = PNG_HEAD + b"\x00" * (self.piece - len(PNG_HEAD))
try:
while self.sent < self.total:
conn.sendall(b"%x\r\n" % len(payload) + payload + b"\r\n")
self.sent += len(payload)
conn.sendall(b"0\r\n\r\n")
except OSError:
# 客户端超限后主动断开,这正是被测行为
return
def test_download_limited_aborts_chunked_stream() -> None:
"""没有 Content-Length 时,累计超过上限就要中断,不能把整份读完。"""
total = 8 * MB
limit = 256 * 1024
with (
_ChunkedServer(total=total) as server,
pytest.raises(MediaRejected) as excinfo,
):
asyncio.run(
download_limited(server.url, limit=limit, chunk=16 * 1024, timeout=10.0)
)
assert "已中断" in excinfo.value.detail
# 服务端没能把 8 MiB 全部写出:说明连接是在中途被断开的,不是读完再判
assert server.sent < total
def test_store_media_creates_missing_root(tmp_path: Path) -> None:
"""root 不存在时应自行创建:首次运行 ``home/data/inbound`` 还不存在。"""
root = tmp_path / "data" / "inbound"
part = ImageContent(mime_type="image/png").model_copy(
update={"data": base64.b64encode(PNG_HEAD).decode()}
)
stored = asyncio.run(store_media(part, root=root))
assert root.is_dir()
assert stored.path.parent == root
assert stored.path.read_bytes() == PNG_HEAD
+2 -2
View File
@@ -135,7 +135,7 @@ def test_run_service_end_to_end(
RecordingChannel.instances.clear()
async def fake_run_task(
task, settings=None, history=None, *, cache=None, steering=None
task, settings=None, history=None, *, cache=None, steering=None, outbox=None
):
assert task == "hello"
return {"messages": [{"role": "assistant", "content": "答:" + task}]}
@@ -189,7 +189,7 @@ def test_run_service_passes_reply_context(
RecordingChannel.instances.clear()
async def fake_run_task(
task, settings=None, history=None, *, cache=None, steering=None
task, settings=None, history=None, *, cache=None, steering=None, outbox=None
):
return {"messages": [{"role": "assistant", "content": "ok"}]}
+91 -4
View File
@@ -16,6 +16,7 @@ from bot042.channels.models import (
ChannelConfig,
ChannelMessage,
ChannelSubject,
ImageContent,
InboundMessage,
OutboundStream,
TextContent,
@@ -24,7 +25,13 @@ from bot042.channels.registry import register_channel
from bot042.config import Settings
from bot042.db import repo
from bot042.flows import builtin
from bot042.flows.builtin import BUDGET_REPLY, RPC_STREAM, AgentReply, ChannelServices
from bot042.flows.builtin import (
BUDGET_REPLY,
RPC_SEND,
RPC_STREAM,
AgentReply,
ChannelServices,
)
from bot042.flows.bus import EventBus
from bot042.flows.rpc import RpcHub
from bot042.flows.stream import ReplyStream
@@ -125,6 +132,8 @@ class StreamChannel(BaseChannel):
) -> None:
super().__init__(processor, config=config, channel_id=channel_id, **kwargs)
self.frames: list[tuple[str, bool]] = []
self.contents: list[list] = []
"""每次富内容发送的分片组合(流式通道只送文本,媒体走这里)。"""
StreamChannel.instances.append(self)
async def start(self) -> None:
@@ -155,7 +164,7 @@ class StreamChannel(BaseChannel):
self.frames.append((text, True))
async def _send_content(self, subject: ChannelSubject, parts: list) -> None:
pass
self.contents.append(list(parts))
async def _send_media(self, subject: ChannelSubject, media: Any) -> None:
pass
@@ -170,7 +179,9 @@ def test_agent_reply_acks_then_streams(
"""收到消息先 ack,随后逐步覆盖,最后以 finish 收尾。"""
StreamChannel.instances.clear()
async def fake_stream(task, settings=None, history=None, *, cache=None, steering=None):
async def fake_stream(
task, settings=None, history=None, *, cache=None, steering=None, outbox=None
):
for text in ("你", "你好"):
yield text
@@ -228,7 +239,9 @@ def test_agent_reply_streams_budget_notice_after_partial(
"""步数用尽:说明补在已流出的内容后面,落库,且不再补发通用错误文案。"""
StreamChannel.instances.clear()
async def exhausted(task, settings=None, history=None, *, cache=None, steering=None):
async def exhausted(
task, settings=None, history=None, *, cache=None, steering=None, outbox=None
):
yield "做到一半"
raise StepBudgetExceeded(200)
@@ -284,6 +297,80 @@ def test_agent_reply_streams_budget_notice_after_partial(
assert rows[-1]["content"] == final
def test_agent_reply_sends_media_after_stream_finishes(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
) -> None:
"""流式路径:文本走流式消息,媒体在文本定稿后单独补一条。"""
StreamChannel.instances.clear()
picture = tmp_path / "cat.png"
picture.write_bytes(b"\x89PNG\r\n\x1a\n" + b"\x00" * 32)
async def fake_stream(
task, settings=None, history=None, *, cache=None, steering=None, outbox=None
):
assert outbox is not None
outbox.push(ImageContent(local_path=str(picture), mime_type="image/png"))
yield "图"
yield "图来了"
monkeypatch.setattr(builtin, "stream_task", fake_stream)
async def _run() -> tuple[
list[tuple[str, bool]], list[list], list[RunStatus], list
]:
bus = EventBus()
bus.bind_loop()
hub = RpcHub(bus)
manager = ChannelManager()
channel = StreamChannel(channel_id="s1")
await manager.add_channel(channel)
await manager.start()
_ = hub.serve(RPC_STREAM, manager.rpc_stream)
_ = hub.serve(RPC_SEND, manager.rpc_send)
services = ChannelServices(
channels=manager,
settings=Settings(home=tmp_path),
db=tmp_path / "bot.db",
hub=hub,
)
runner = TaskRunner(bus, deps=services)
runner.register(AgentReply)
_ = runner.add_task(AgentReply.name, EventTrigger(event="channel.inbound"))
await runner.start()
bus.emit(
"channel.inbound",
ChannelMessage.of(
InboundMessage(
channel_id="s1",
channel_type="stream-test",
channel_subject=ChannelSubject(subject_id="u1"),
content=[TextContent(text="画只猫")],
)
),
)
await asyncio.sleep(0.05)
await runner.wait_idle()
statuses = [run.status for run in runner.runs(AgentReply.name)]
await runner.stop()
hub.close()
await manager.stop()
rows = await repo.list_messages(tmp_path / "bot.db", session_id="s1:u1")
return channel.frames, channel.contents, statuses, rows
frames, contents, statuses, rows = asyncio.run(_run())
assert frames[-1] == ("图来了", True) # 文本仍然走流式收尾
assert statuses == [RunStatus.SUCCEEDED]
assert len(contents) == 1 # 媒体单独一条,且只含媒体分片
assert [type(part) for part in contents[0]] == [ImageContent]
# 流式分支同样要落库附件:它和同步分支共用 _persist_reply,走的是另一处调用点
assert rows[-1]["role"] == "assistant"
assert rows[-1]["content"].startswith("图来了")
assert "[附件] 图片" in rows[-1]["content"]
assert str(picture) in rows[-1]["content"]
def test_outbound_stream_payload_defaults() -> None:
payload = OutboundStream(
channel_id="c", subject_id="u", stream_id="s", text="x"