feat(flows): 媒体入站进出站链路接线(issue #1 P0)
上一轮提交 9ce0ba0 把 media.py / outbox.py 的骨架铺好了,但没有任何调用方:
入站附件进不了上下文,出站 MediaOutbox 从不被创建,send_image / send_file
两个工具因此恒定返回「当前不在渠道会话中」。本次把链路接上。
入站:
- AgentReply 改用 prepare_inbound 整理分片:小图内联为 image_url 块,
大图与音视频退化为「路径 + 元信息」,文本类文件抽取正文进上下文
- 附件全部失败时回「附件未能接收:<原因>」,不再误报成「(无输出)」
- 正文用 prepared.summary(正文 + 附件清单),历史只存清单不存字节
- 只有正文发纯文本,带附件才发 content blocks(is_multimodal)
出站:
- 每轮创建一个 MediaOutbox 传给 run_task / stream_task,工具往里登记分片,
跑完 drain() 出来与正文合成一条 OutboundContent
- 只有媒体没有正文时不再补「(无输出)」;流式通道文本定稿后单独补发媒体
- ChannelServices.send_content 统一走「先文本、再逐个媒体」
解密钩子的抽象层:
- BaseChannel.media_transform(part, raw) 默认原样返回
- BaseChannel.make_media_transform(metadata) 按单条消息构造钩子。密钥随消息
而来,挂在渠道实例上会让并发处理的两条消息串用彼此的密钥
- ChannelServices.media_transform 从渠道取钩子,渠道未注册时返回 None
(不解密,但也不跳过后续的类型与大小校验)
- media.py 新增 MediaTransform 类型与 store_media(transform=)。钩子在类型嗅探
与大小判定之前执行——对密文嗅魔数只会得到 octet-stream——并在转换后对明文
再量一次上限,因为密文没超不等于明文没超
测试:test_builtin 增加 6 条媒体用例(文本+图、纯图、附件全失败、出站下发、
仅媒体不补空文案、流式补发),test_service / test_stream 的 fake_run_task 补
outbox 形参。全量 250 passed,ruff 全绿。
P0 边界说明:本提交只做接线与抽象,不含企业微信的具体解密实现、下载限长、
CLI 媒体命令与 docs 更新,这些在下一个提交(P1)。
This commit is contained in:
1 parent
9ce0ba0e13
commit
2dde0d1160
7 files changed
+496
-24
No files matched your search
@@ -30,6 +30,7 @@ import httpx
|
|||||||
from bot042.channels.media import (
|
from bot042.channels.media import (
|
||||||
MediaLimits,
|
MediaLimits,
|
||||||
MediaRejected,
|
MediaRejected,
|
||||||
|
MediaTransform,
|
||||||
decode_base64_limited,
|
decode_base64_limited,
|
||||||
download_limited,
|
download_limited,
|
||||||
format_size,
|
format_size,
|
||||||
@@ -618,6 +619,29 @@ class BaseChannel(ABC):
|
|||||||
"""下载入站媒体时附带的请求头(平台鉴权等);默认无。"""
|
"""下载入站媒体时附带的请求头(平台鉴权等);默认无。"""
|
||||||
return {}
|
return {}
|
||||||
|
|
||||||
|
def media_transform(self, part: ContentPart, raw: bytes) -> bytes:
|
||||||
|
"""平台钩子:把刚取到的媒体字节转换成明文(如企业微信的 AES 解密)。
|
||||||
|
|
||||||
|
默认原样返回。实现方应假定 ``raw`` 可能是密文,且**必须**是纯函数语义
|
||||||
|
——上层会拿返回值的魔数重新判定类型、并再量一次大小。
|
||||||
|
|
||||||
|
这是**无消息上下文**的版本;需要按单条消息取密钥(如 ``aeskey``)时
|
||||||
|
覆写 :meth:`make_media_transform`。
|
||||||
|
"""
|
||||||
|
return raw
|
||||||
|
|
||||||
|
def make_media_transform(self, metadata: dict[str, Any]) -> MediaTransform:
|
||||||
|
"""按单条入站消息的元数据构造解密钩子。
|
||||||
|
|
||||||
|
密钥这类材料随消息而来(企业微信的 ``<media>.aeskey``),挂在渠道实例上
|
||||||
|
会串台。返回闭包把它绑在这一条消息上,消息处理完即随之释放。
|
||||||
|
|
||||||
|
Args:
|
||||||
|
metadata: 入站消息的 ``metadata``。
|
||||||
|
"""
|
||||||
|
_ = metadata
|
||||||
|
return self.media_transform
|
||||||
|
|
||||||
async def fetch_remote_media(self, url: str) -> tuple[bytes, str]:
|
async def fetch_remote_media(self, url: str) -> tuple[bytes, str]:
|
||||||
"""下载远端媒体并返回 ``(bytes, content_type)``;子类可覆写加平台鉴权。"""
|
"""下载远端媒体并返回 ``(bytes, content_type)``;子类可覆写加平台鉴权。"""
|
||||||
async with httpx.AsyncClient(follow_redirects=True) as client:
|
async with httpx.AsyncClient(follow_redirects=True) as client:
|
||||||
|
|||||||
@@ -24,6 +24,7 @@ import hashlib
|
|||||||
import mimetypes
|
import mimetypes
|
||||||
import re
|
import re
|
||||||
import unicodedata
|
import unicodedata
|
||||||
|
from collections.abc import Callable
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Final
|
from typing import Final
|
||||||
@@ -46,6 +47,7 @@ __all__ = [
|
|||||||
"BLOCKED_EXTENSIONS",
|
"BLOCKED_EXTENSIONS",
|
||||||
"MediaLimits",
|
"MediaLimits",
|
||||||
"MediaRejected",
|
"MediaRejected",
|
||||||
|
"MediaTransform",
|
||||||
"StoredMedia",
|
"StoredMedia",
|
||||||
"check_mime",
|
"check_mime",
|
||||||
"decode_base64_limited",
|
"decode_base64_limited",
|
||||||
@@ -58,6 +60,14 @@ __all__ = [
|
|||||||
"store_media",
|
"store_media",
|
||||||
]
|
]
|
||||||
|
|
||||||
|
MediaTransform = Callable[[ContentPart, bytes], bytes]
|
||||||
|
"""入站字节的转换钩子:``(分片, 原始字节) -> 明文字节``。
|
||||||
|
|
||||||
|
企业微信的媒体 URL 返回的是 AES 密文(见 ``wecom_aibot_sdk.crypto``),
|
||||||
|
必须解密之后才能做类型嗅探与大小判定——在密文上嗅魔数只会得到
|
||||||
|
``application/octet-stream``,从而把所有图片都判成「类型不被接受」。
|
||||||
|
"""
|
||||||
|
|
||||||
MB: Final[int] = 1024 * 1024
|
MB: Final[int] = 1024 * 1024
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
@@ -443,6 +453,7 @@ async def store_media(
|
|||||||
limits: MediaLimits | None = None,
|
limits: MediaLimits | None = None,
|
||||||
session_key: str = "",
|
session_key: str = "",
|
||||||
headers: dict[str, str] | None = None,
|
headers: dict[str, str] | None = None,
|
||||||
|
transform: MediaTransform | None = None,
|
||||||
) -> StoredMedia:
|
) -> StoredMedia:
|
||||||
"""把媒体分片落盘到 ``root/<session>/``,返回更新后的分片。
|
"""把媒体分片落盘到 ``root/<session>/``,返回更新后的分片。
|
||||||
|
|
||||||
@@ -450,9 +461,20 @@ async def store_media(
|
|||||||
(``data`` → ``local_path`` → ``url``),但每一步都带大小闸;
|
(``data`` → ``local_path`` → ``url``),但每一步都带大小闸;
|
||||||
落盘后分片只保留 ``local_path``,让下游(智能体、出站回显)共用一份字节。
|
落盘后分片只保留 ``local_path``,让下游(智能体、出站回显)共用一份字节。
|
||||||
|
|
||||||
|
Args:
|
||||||
|
transform: 可选的字节转换钩子(如平台解密)。**在类型嗅探与大小判定
|
||||||
|
之前**执行,因为对密文嗅魔数没有意义;转换后的明文字节会再跑一次
|
||||||
|
上限检查——解密后的长度不可由密文长度保证(见下)。
|
||||||
|
session_key: 会话子目录名(净化后使用)。
|
||||||
|
headers: 下载远端媒体时附带的请求头。
|
||||||
|
|
||||||
Raises:
|
Raises:
|
||||||
MediaRejected: 超限、类型不符、路径非法。
|
MediaRejected: 超限、类型不符、路径非法、转换失败。
|
||||||
httpx.HTTPError: 远端下载失败(调用方降级为文本提示)。
|
httpx.HTTPError: 远端下载失败(调用方降级为文本提示)。
|
||||||
|
|
||||||
|
注意:``download_limited`` 的上限约束的是**收到的字节**。AES-CBC 解密后
|
||||||
|
明文通常更短(去掉填充),但把「密文没超」当成「明文没超」是错的推理,
|
||||||
|
所以转换后再量一次。
|
||||||
"""
|
"""
|
||||||
limits = limits or MediaLimits()
|
limits = limits or MediaLimits()
|
||||||
if isinstance(part, TextContent):
|
if isinstance(part, TextContent):
|
||||||
@@ -464,11 +486,11 @@ async def store_media(
|
|||||||
|
|
||||||
if part.data:
|
if part.data:
|
||||||
raw = decode_base64_limited(part.data, limit)
|
raw = decode_base64_limited(part.data, limit)
|
||||||
mime = check_mime(part, mime_declared or guess_mime(name_hint) or "", raw[:32])
|
declared = mime_declared or guess_mime(name_hint) or ""
|
||||||
elif part.local_path:
|
elif part.local_path:
|
||||||
source = Path(part.local_path)
|
source = Path(part.local_path)
|
||||||
raw = read_file_limited(source, limit)
|
raw = read_file_limited(source, limit)
|
||||||
mime = check_mime(part, mime_declared or guess_mime(str(source)) or "", raw[:32])
|
declared = mime_declared or guess_mime(str(source)) or ""
|
||||||
name_hint = name_hint or source.name
|
name_hint = name_hint or source.name
|
||||||
elif part.url:
|
elif part.url:
|
||||||
raw, fetched = await download_limited(
|
raw, fetched = await download_limited(
|
||||||
@@ -478,16 +500,36 @@ async def store_media(
|
|||||||
timeout=limits.timeout,
|
timeout=limits.timeout,
|
||||||
headers=headers,
|
headers=headers,
|
||||||
)
|
)
|
||||||
mime = check_mime(part, mime_declared or fetched or guess_mime(name_hint) or "", raw[:32])
|
declared = mime_declared or fetched or guess_mime(name_hint) or ""
|
||||||
else:
|
else:
|
||||||
raise MediaRejected("媒体缺少字节来源", "data/local_path/url 三者皆空")
|
raise MediaRejected("媒体缺少字节来源", "data/local_path/url 三者皆空")
|
||||||
|
|
||||||
|
if transform is not None:
|
||||||
|
try:
|
||||||
|
raw = transform(part, raw)
|
||||||
|
except MediaRejected:
|
||||||
|
raise
|
||||||
|
except Exception as exc: # 平台解密失败不应冒泡成未捕获异常
|
||||||
|
raise MediaRejected("媒体解密失败", f"{type(exc).__name__}: {exc}") from exc
|
||||||
|
if len(raw) > limit:
|
||||||
|
raise MediaRejected(
|
||||||
|
"媒体超过大小上限",
|
||||||
|
f"解密后 {format_size(len(raw))} > {format_size(limit)}",
|
||||||
|
)
|
||||||
|
# 密文的 MIME 声明不可信:解密后按魔数重新判定,声明类型只作兜底
|
||||||
|
declared = mime_declared or guess_mime(name_hint) or ""
|
||||||
|
|
||||||
|
mime = check_mime(part, declared, raw[:32])
|
||||||
|
|
||||||
folder = root
|
folder = root
|
||||||
if session_key:
|
if session_key:
|
||||||
folder = root / sanitize_filename(session_key, default="session", max_len=60)
|
folder = root / sanitize_filename(session_key, default="session", max_len=60)
|
||||||
folder.mkdir(parents=True, exist_ok=True)
|
folder.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
target = safe_join(folder, sanitize_filename(name_hint) if name_hint else _default_name_for(part, mime))
|
target = safe_join(
|
||||||
|
folder,
|
||||||
|
sanitize_filename(name_hint) if name_hint else _default_name_for(part, mime),
|
||||||
|
)
|
||||||
digest = hashlib.sha256(raw).hexdigest()
|
digest = hashlib.sha256(raw).hexdigest()
|
||||||
# 同名不同内容时加哈希前缀,避免后到的覆盖先到的(也避免穿越后仍撞名)
|
# 同名不同内容时加哈希前缀,避免后到的覆盖先到的(也避免穿越后仍撞名)
|
||||||
if target.exists():
|
if target.exists():
|
||||||
@@ -498,6 +540,13 @@ async def store_media(
|
|||||||
temp.write_bytes(raw)
|
temp.write_bytes(raw)
|
||||||
temp.replace(target)
|
temp.replace(target)
|
||||||
|
|
||||||
updated = part.model_copy(update={"local_path": str(target), "url": "", "data": None,
|
updated = part.model_copy(
|
||||||
"size": len(raw), "mime_type": mime})
|
update={
|
||||||
|
"local_path": str(target),
|
||||||
|
"url": "",
|
||||||
|
"data": None,
|
||||||
|
"size": len(raw),
|
||||||
|
"mime_type": mime,
|
||||||
|
}
|
||||||
|
)
|
||||||
return StoredMedia(part=updated, path=target, size=len(raw), mime=mime, sha256=digest)
|
return StoredMedia(part=updated, path=target, size=len(raw), mime=mime, sha256=digest)
|
||||||
@@ -22,7 +22,9 @@ from typing import Any
|
|||||||
|
|
||||||
from bot042.agent.middlewares.budget import StepBudgetExceeded
|
from bot042.agent.middlewares.budget import StepBudgetExceeded
|
||||||
from bot042.agent.middlewares.steering import SteeringBuffer
|
from bot042.agent.middlewares.steering import SteeringBuffer
|
||||||
|
from bot042.agent.outbox import MediaOutbox
|
||||||
from bot042.channels.manager import ChannelManager
|
from bot042.channels.manager import ChannelManager
|
||||||
|
from bot042.channels.media import MediaLimits, MediaTransform
|
||||||
from bot042.channels.models import (
|
from bot042.channels.models import (
|
||||||
ChannelMessage,
|
ChannelMessage,
|
||||||
ContentPart,
|
ContentPart,
|
||||||
@@ -30,6 +32,7 @@ from bot042.channels.models import (
|
|||||||
OutboundContent,
|
OutboundContent,
|
||||||
OutboundStream,
|
OutboundStream,
|
||||||
OutboundText,
|
OutboundText,
|
||||||
|
TextContent,
|
||||||
)
|
)
|
||||||
from bot042.config import Settings
|
from bot042.config import Settings
|
||||||
from bot042.db import repo
|
from bot042.db import repo
|
||||||
@@ -40,6 +43,7 @@ from bot042.flows.agent_task import (
|
|||||||
run_task,
|
run_task,
|
||||||
stream_task,
|
stream_task,
|
||||||
)
|
)
|
||||||
|
from bot042.flows.media_input import prepare_inbound
|
||||||
from bot042.flows.processor import (
|
from bot042.flows.processor import (
|
||||||
DEFAULT_HISTORY_CHARS,
|
DEFAULT_HISTORY_CHARS,
|
||||||
DEFAULT_HISTORY_LIMIT,
|
DEFAULT_HISTORY_LIMIT,
|
||||||
@@ -173,6 +177,31 @@ class ChannelServices:
|
|||||||
return
|
return
|
||||||
await self.channels.send_content(channel_id, subject_id, parts)
|
await self.channels.send_content(channel_id, subject_id, parts)
|
||||||
|
|
||||||
|
def media_policy(self, channel_id: str) -> tuple[MediaLimits, dict[str, str]]:
|
||||||
|
"""渠道的媒体大小上限与下载请求头。
|
||||||
|
|
||||||
|
未知渠道(未注册或已下线)退回默认策略值:宁可让校验按默认限跑,
|
||||||
|
也不要因为拿不到渠道就把这道关跳过。
|
||||||
|
"""
|
||||||
|
channel = self.channels.get_channel(channel_id)
|
||||||
|
if channel is None:
|
||||||
|
return MediaLimits(), {}
|
||||||
|
return channel.media_limits, channel.media_download_headers()
|
||||||
|
|
||||||
|
def media_transform(
|
||||||
|
self, channel_id: str, metadata: dict[str, Any]
|
||||||
|
) -> MediaTransform | None:
|
||||||
|
"""按本条消息的元数据取渠道的解密钩子(如企业微信的 ``aeskey``)。
|
||||||
|
|
||||||
|
密钥随消息而来,必须按条构造,不能挂在渠道实例上——挂实例会让并发处理
|
||||||
|
的两条消息串用彼此的密钥。渠道未注册时返回 ``None``,即"不解密",
|
||||||
|
由类型嗅探在明文上判定,不会因为查不到渠道就跳过后续校验。
|
||||||
|
"""
|
||||||
|
channel = self.channels.get_channel(channel_id)
|
||||||
|
if channel is None:
|
||||||
|
return None
|
||||||
|
return channel.make_media_transform(metadata)
|
||||||
|
|
||||||
async def stream_text(
|
async def stream_text(
|
||||||
self,
|
self,
|
||||||
channel_id: str,
|
channel_id: str,
|
||||||
@@ -233,6 +262,30 @@ class ChannelServices:
|
|||||||
return ReplyStream(_send)
|
return ReplyStream(_send)
|
||||||
|
|
||||||
|
|
||||||
|
async def _reply_with(
|
||||||
|
services: ChannelServices,
|
||||||
|
payload: ChannelMessage,
|
||||||
|
meta: dict[str, Any],
|
||||||
|
text: str,
|
||||||
|
parts: list[ContentPart],
|
||||||
|
) -> None:
|
||||||
|
"""回复一条消息:有媒体分片就走富内容(文本在前),否则走纯文本。
|
||||||
|
|
||||||
|
文本与媒体分成两次发送,在平台上会变成两条消息;合成一次
|
||||||
|
``send_content`` 后由出站管线按「先文本、再逐个媒体」的顺序下发。
|
||||||
|
"""
|
||||||
|
if parts:
|
||||||
|
lead: list[ContentPart] = [TextContent(text=text)] if text else []
|
||||||
|
await services.send_content(
|
||||||
|
payload.channel_id,
|
||||||
|
payload.subject_id,
|
||||||
|
[*lead, *parts],
|
||||||
|
metadata=meta,
|
||||||
|
)
|
||||||
|
return
|
||||||
|
await services.send_text(payload.channel_id, payload.subject_id, text, metadata=meta)
|
||||||
|
|
||||||
|
|
||||||
class AgentReply(Task):
|
class AgentReply(Task):
|
||||||
"""渠道消息任务:读历史 → 调智能体 → 落库 → 回复。
|
"""渠道消息任务:读历史 → 调智能体 → 落库 → 回复。
|
||||||
|
|
||||||
@@ -313,11 +366,25 @@ class AgentReply(Task):
|
|||||||
|
|
||||||
message = payload.message
|
message = payload.message
|
||||||
meta = reply_metadata(message)
|
meta = reply_metadata(message)
|
||||||
text = message.text.strip()
|
# 入站:附件先落盘再进上下文(小图内联为 base64,大图与音视频退化为
|
||||||
|
# 「路径 + 元信息」);单个附件失败只记一条说明,不打断整轮。
|
||||||
|
limits, headers = services.media_policy(payload.channel_id)
|
||||||
|
prepared = await prepare_inbound(
|
||||||
|
message,
|
||||||
|
root=services.settings.home / "data" / "inbound",
|
||||||
|
limits=limits,
|
||||||
|
headers=headers,
|
||||||
|
transform=services.media_transform(payload.channel_id, message.metadata),
|
||||||
|
)
|
||||||
stream = services.open_stream(payload.channel_id, payload.subject_id, meta)
|
stream = services.open_stream(payload.channel_id, payload.subject_id, meta)
|
||||||
|
|
||||||
if not text:
|
if not prepared.has_content:
|
||||||
reply = "暂仅支持文本消息" if message.has_media else EMPTY_REPLY
|
# 有附件但一个都没接住:说清原因,别让用户以为是发了条空消息
|
||||||
|
reply = (
|
||||||
|
"附件未能接收:" + ";".join(prepared.notes)
|
||||||
|
if prepared.notes
|
||||||
|
else EMPTY_REPLY
|
||||||
|
)
|
||||||
if stream is not None:
|
if stream is not None:
|
||||||
await stream.update(reply, force=True)
|
await stream.update(reply, force=True)
|
||||||
await stream.finish(reply)
|
await stream.finish(reply)
|
||||||
@@ -327,6 +394,12 @@ class AgentReply(Task):
|
|||||||
)
|
)
|
||||||
return
|
return
|
||||||
|
|
||||||
|
text = prepared.summary
|
||||||
|
# 出站:本轮的信箱,工具(send_image / send_file)往里登记,跑完统一下发
|
||||||
|
outbox = MediaOutbox(limits=limits)
|
||||||
|
# 只有正文时发纯文本,带附件才发多模态块列表(见 PreparedInbound.is_multimodal)
|
||||||
|
task_input = prepared.blocks if prepared.is_multimodal else text
|
||||||
|
|
||||||
session_id = payload.db_session_id
|
session_id = payload.db_session_id
|
||||||
channel = payload.channel_type or payload.channel_id
|
channel = payload.channel_type or payload.channel_id
|
||||||
await services.ensure_db()
|
await services.ensure_db()
|
||||||
@@ -344,17 +417,20 @@ class AgentReply(Task):
|
|||||||
aborted: StepBudgetExceeded | None = None
|
aborted: StepBudgetExceeded | None = None
|
||||||
try:
|
try:
|
||||||
result = await run_task(
|
result = await run_task(
|
||||||
text,
|
task_input,
|
||||||
services.settings,
|
services.settings,
|
||||||
history=history,
|
history=history,
|
||||||
cache=services.cache,
|
cache=services.cache,
|
||||||
steering=self._steering,
|
steering=self._steering,
|
||||||
|
outbox=outbox,
|
||||||
)
|
)
|
||||||
answer = last_message_text(result)
|
answer = last_message_text(result)
|
||||||
except StepBudgetExceeded as exc:
|
except StepBudgetExceeded as exc:
|
||||||
aborted = exc
|
aborted = exc
|
||||||
await self._persist_steered(services, session_id, channel)
|
await self._persist_steered(services, session_id, channel)
|
||||||
reply = _with_notice(answer, aborted)
|
reply = _with_notice(answer, aborted)
|
||||||
|
# 智能体可能只发了媒体、没写正文:这时不必再补一句「(无输出)」
|
||||||
|
pending = outbox.drain()
|
||||||
if reply:
|
if reply:
|
||||||
await repo.add_message(
|
await repo.add_message(
|
||||||
services.db,
|
services.db,
|
||||||
@@ -363,8 +439,12 @@ class AgentReply(Task):
|
|||||||
content=reply,
|
content=reply,
|
||||||
channel=channel,
|
channel=channel,
|
||||||
)
|
)
|
||||||
await services.send_text(
|
await _reply_with(
|
||||||
payload.channel_id, payload.subject_id, reply or EMPTY_REPLY, metadata=meta
|
services,
|
||||||
|
payload,
|
||||||
|
meta,
|
||||||
|
reply or ("" if pending else EMPTY_REPLY),
|
||||||
|
pending,
|
||||||
)
|
)
|
||||||
if aborted is not None:
|
if aborted is not None:
|
||||||
# 已回话并落库:任务仍记失败(步数用尽是真的没做完),但不再发通用错误
|
# 已回话并落库:任务仍记失败(步数用尽是真的没做完),但不再发通用错误
|
||||||
@@ -380,11 +460,12 @@ class AgentReply(Task):
|
|||||||
aborted = None
|
aborted = None
|
||||||
try:
|
try:
|
||||||
async for partial in stream_task(
|
async for partial in stream_task(
|
||||||
text,
|
task_input,
|
||||||
services.settings,
|
services.settings,
|
||||||
history=history,
|
history=history,
|
||||||
cache=services.cache,
|
cache=services.cache,
|
||||||
steering=self._steering,
|
steering=self._steering,
|
||||||
|
outbox=outbox,
|
||||||
):
|
):
|
||||||
answer = partial
|
answer = partial
|
||||||
# 首段跳过节流:让用户在 ack 之后立刻看到内容开始出现
|
# 首段跳过节流:让用户在 ack 之后立刻看到内容开始出现
|
||||||
@@ -395,7 +476,9 @@ class AgentReply(Task):
|
|||||||
finally:
|
finally:
|
||||||
# 步数用尽时把说明补在已经流出的内容后面,不必再单独发一条
|
# 步数用尽时把说明补在已经流出的内容后面,不必再单独发一条
|
||||||
final = _with_notice(answer, aborted)
|
final = _with_notice(answer, aborted)
|
||||||
await stream.finish(final or EMPTY_REPLY)
|
pending = outbox.drain()
|
||||||
|
# 只有媒体没正文时传空串:ReplyStream.finish("") 不会发任何文本
|
||||||
|
await stream.finish(final or ("" if pending else EMPTY_REPLY))
|
||||||
await self._persist_steered(services, session_id, channel)
|
await self._persist_steered(services, session_id, channel)
|
||||||
if final:
|
if final:
|
||||||
await repo.add_message(
|
await repo.add_message(
|
||||||
@@ -405,6 +488,11 @@ class AgentReply(Task):
|
|||||||
content=final,
|
content=final,
|
||||||
channel=channel,
|
channel=channel,
|
||||||
)
|
)
|
||||||
|
if pending:
|
||||||
|
# 流式通道只送文本,媒体只能补一条(文本此时已经定稿)
|
||||||
|
await services.send_content(
|
||||||
|
payload.channel_id, payload.subject_id, pending, metadata=meta
|
||||||
|
)
|
||||||
if aborted is not None:
|
if aborted is not None:
|
||||||
self._reported = True
|
self._reported = True
|
||||||
raise aborted
|
raise aborted
|
||||||
|
|||||||
@@ -34,6 +34,7 @@ from bot042.channels.media import (
|
|||||||
MB,
|
MB,
|
||||||
MediaLimits,
|
MediaLimits,
|
||||||
MediaRejected,
|
MediaRejected,
|
||||||
|
MediaTransform,
|
||||||
format_size,
|
format_size,
|
||||||
store_media,
|
store_media,
|
||||||
)
|
)
|
||||||
@@ -157,6 +158,7 @@ async def prepare_inbound(
|
|||||||
limits: MediaLimits | None = None,
|
limits: MediaLimits | None = None,
|
||||||
policy: InlinePolicy | None = None,
|
policy: InlinePolicy | None = None,
|
||||||
headers: dict[str, str] | None = None,
|
headers: dict[str, str] | None = None,
|
||||||
|
transform: MediaTransform | None = None,
|
||||||
) -> PreparedInbound:
|
) -> PreparedInbound:
|
||||||
"""把入站消息整理成 ``(text, blocks, summary, notes)``,媒体先落盘。
|
"""把入站消息整理成 ``(text, blocks, summary, notes)``,媒体先落盘。
|
||||||
|
|
||||||
@@ -166,6 +168,8 @@ async def prepare_inbound(
|
|||||||
limits: 字节上限;缺省 :class:`MediaLimits`。
|
limits: 字节上限;缺省 :class:`MediaLimits`。
|
||||||
policy: 图片内联策略;缺省 :class:`InlinePolicy`。
|
policy: 图片内联策略;缺省 :class:`InlinePolicy`。
|
||||||
headers: 下载远端媒体时附带的请求头(平台鉴权由渠道提供)。
|
headers: 下载远端媒体时附带的请求头(平台鉴权由渠道提供)。
|
||||||
|
transform: 平台字节转换钩子(如企业微信的 AES 解密),在类型嗅探与
|
||||||
|
大小判定之前执行,由渠道按单条消息的 metadata 构造。
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
:class:`PreparedInbound`。附件层面的失败只进 ``notes``,不抛异常。
|
:class:`PreparedInbound`。附件层面的失败只进 ``notes``,不抛异常。
|
||||||
@@ -190,6 +194,7 @@ async def prepare_inbound(
|
|||||||
limits=limits,
|
limits=limits,
|
||||||
session_key=session_key,
|
session_key=session_key,
|
||||||
headers=headers,
|
headers=headers,
|
||||||
|
transform=transform,
|
||||||
)
|
)
|
||||||
except MediaRejected as exc:
|
except MediaRejected as exc:
|
||||||
prepared.notes.append(f"{_media_label(part)}未接收:{exc.reason}")
|
prepared.notes.append(f"{_media_label(part)}未接收:{exc.reason}")
|
||||||
|
|||||||
+230
-3
@@ -3,6 +3,7 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import asyncio
|
import asyncio
|
||||||
|
import base64
|
||||||
from datetime import datetime
|
from datetime import datetime
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any
|
from typing import Any
|
||||||
@@ -15,6 +16,7 @@ from bot042.channels.manager import ChannelManager
|
|||||||
from bot042.channels.models import (
|
from bot042.channels.models import (
|
||||||
ChannelMessage,
|
ChannelMessage,
|
||||||
ChannelSubject,
|
ChannelSubject,
|
||||||
|
ImageContent,
|
||||||
InboundMessage,
|
InboundMessage,
|
||||||
TextContent,
|
TextContent,
|
||||||
)
|
)
|
||||||
@@ -29,6 +31,7 @@ from bot042.flows.builtin import (
|
|||||||
ChannelServices,
|
ChannelServices,
|
||||||
)
|
)
|
||||||
from bot042.flows.bus import EventBus
|
from bot042.flows.bus import EventBus
|
||||||
|
from bot042.flows.media_input import PreparedInbound
|
||||||
from bot042.flows.rpc import RpcHub
|
from bot042.flows.rpc import RpcHub
|
||||||
from bot042.flows.scheduler import CN_TZ
|
from bot042.flows.scheduler import CN_TZ
|
||||||
from bot042.flows.tasks import EventTrigger, RunStatus, TaskContext, TaskRunner
|
from bot042.flows.tasks import EventTrigger, RunStatus, TaskContext, TaskRunner
|
||||||
@@ -42,6 +45,8 @@ class EchoChannel(BaseChannel):
|
|||||||
def __init__(self) -> None:
|
def __init__(self) -> None:
|
||||||
super().__init__(channel_id="c1")
|
super().__init__(channel_id="c1")
|
||||||
self.sent: list[str] = []
|
self.sent: list[str] = []
|
||||||
|
self.contents: list[list[Any]] = []
|
||||||
|
"""每次富内容发送的分片组合(文本 + 媒体)。"""
|
||||||
|
|
||||||
async def start(self) -> None:
|
async def start(self) -> None:
|
||||||
pass
|
pass
|
||||||
@@ -60,7 +65,7 @@ class EchoChannel(BaseChannel):
|
|||||||
self.sent.append(text)
|
self.sent.append(text)
|
||||||
|
|
||||||
async def _send_content(self, subject: ChannelSubject, parts: list) -> None:
|
async def _send_content(self, subject: ChannelSubject, parts: list) -> None:
|
||||||
pass
|
self.contents.append(list(parts))
|
||||||
|
|
||||||
async def _send_media(self, subject: ChannelSubject, media: Any) -> None:
|
async def _send_media(self, subject: ChannelSubject, media: Any) -> None:
|
||||||
pass
|
pass
|
||||||
@@ -119,7 +124,7 @@ def test_agent_reply_persists_and_replies_via_rpc(
|
|||||||
calls: list[tuple[str, list]] = []
|
calls: list[tuple[str, list]] = []
|
||||||
|
|
||||||
async def fake_run_task(
|
async def fake_run_task(
|
||||||
task, settings=None, history=None, *, cache=None, steering=None
|
task, settings=None, history=None, *, cache=None, steering=None, outbox=None
|
||||||
):
|
):
|
||||||
calls.append((task, list(history or [])))
|
calls.append((task, list(history or [])))
|
||||||
return {"messages": [{"role": "assistant", "content": "答:" + task}]}
|
return {"messages": [{"role": "assistant", "content": "答:" + task}]}
|
||||||
@@ -223,7 +228,9 @@ def test_agent_reply_explains_step_budget_without_generic_error(
|
|||||||
) -> None:
|
) -> None:
|
||||||
"""步数用尽:给出可操作说明并落库,且不再补发通用错误文案。"""
|
"""步数用尽:给出可操作说明并落库,且不再补发通用错误文案。"""
|
||||||
|
|
||||||
async def exhausted(task, settings=None, history=None, *, cache=None, steering=None):
|
async def exhausted(
|
||||||
|
task, settings=None, history=None, *, cache=None, steering=None, outbox=None
|
||||||
|
):
|
||||||
raise StepBudgetExceeded(200)
|
raise StepBudgetExceeded(200)
|
||||||
|
|
||||||
monkeypatch.setattr(builtin, "run_task", exhausted)
|
monkeypatch.setattr(builtin, "run_task", exhausted)
|
||||||
@@ -265,3 +272,223 @@ def test_agent_reply_explains_step_budget_without_generic_error(
|
|||||||
("user", "把整站重构一遍"),
|
("user", "把整站重构一遍"),
|
||||||
("assistant", notice),
|
("assistant", notice),
|
||||||
]
|
]
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
# 媒体链路:入站进上下文、出站经信箱下发
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
|
||||||
|
PNG_HEAD = b"\x89PNG\r\n\x1a\n" + b"\x00\x00\x00\rIHDR" + b"\x00" * 16
|
||||||
|
"""最小 PNG 头:magic + IHDR 片段,足够 check_mime 认定成 image/png。"""
|
||||||
|
|
||||||
|
|
||||||
|
def _image_part() -> ImageContent:
|
||||||
|
"""入站图片分片:base64 直接带字节,不依赖网络与临时文件。"""
|
||||||
|
return ImageContent(
|
||||||
|
data=base64.b64encode(PNG_HEAD).decode("ascii"),
|
||||||
|
mime_type="image/png",
|
||||||
|
size=len(PNG_HEAD),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _media_envelope(*parts: Any, text: str = "") -> ChannelMessage:
|
||||||
|
"""构造带任意内容分片的入站消息。"""
|
||||||
|
content = ([TextContent(text=text)] if text else []) + list(parts)
|
||||||
|
message = InboundMessage(
|
||||||
|
channel_id="c1",
|
||||||
|
channel_type="echo",
|
||||||
|
channel_session_id="s1",
|
||||||
|
channel_subject=ChannelSubject(subject_id="u1"),
|
||||||
|
content=content,
|
||||||
|
)
|
||||||
|
return ChannelMessage.of(message, session_key="s1")
|
||||||
|
|
||||||
|
|
||||||
|
def _drive(
|
||||||
|
db: Path,
|
||||||
|
payloads: list[ChannelMessage],
|
||||||
|
channel: EchoChannel,
|
||||||
|
*,
|
||||||
|
home: Path,
|
||||||
|
) -> list[RunStatus]:
|
||||||
|
"""跑一遍 AgentReply(调用方先 monkeypatch run_task / prepare_inbound)。
|
||||||
|
|
||||||
|
``home`` 指向临时目录,避免把测试媒体写进真实的 ``~/.bot042/data``。
|
||||||
|
"""
|
||||||
|
|
||||||
|
async def _run() -> list[RunStatus]:
|
||||||
|
bus = EventBus()
|
||||||
|
bus.bind_loop()
|
||||||
|
hub = RpcHub(bus)
|
||||||
|
manager = ChannelManager()
|
||||||
|
await manager.add_channel(channel)
|
||||||
|
await manager.start()
|
||||||
|
_ = hub.serve(RPC_SEND, manager.rpc_send)
|
||||||
|
|
||||||
|
services = ChannelServices(
|
||||||
|
channels=manager,
|
||||||
|
settings=Settings(home=home),
|
||||||
|
db=db,
|
||||||
|
hub=hub,
|
||||||
|
)
|
||||||
|
runner = TaskRunner(bus, deps=services)
|
||||||
|
runner.register(AgentReply)
|
||||||
|
_ = runner.add_task(AgentReply.name, EventTrigger(event="channel.inbound"))
|
||||||
|
await runner.start()
|
||||||
|
|
||||||
|
for payload in payloads:
|
||||||
|
bus.emit("channel.inbound", payload)
|
||||||
|
await runner.wait_idle(timeout=2)
|
||||||
|
await bus.wait_for_complete()
|
||||||
|
statuses = [run.status for run in runner.runs(AgentReply.name)]
|
||||||
|
await runner.stop()
|
||||||
|
hub.close()
|
||||||
|
await manager.stop()
|
||||||
|
return statuses
|
||||||
|
|
||||||
|
return asyncio.run(_run())
|
||||||
|
|
||||||
|
|
||||||
|
def test_agent_reply_feeds_inbound_image_to_agent(
|
||||||
|
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
|
||||||
|
) -> None:
|
||||||
|
"""文本 + 图片:两者都进上下文,图片走 image_url 且不入库字节。"""
|
||||||
|
seen: list[Any] = []
|
||||||
|
|
||||||
|
async def fake_run_task(
|
||||||
|
task, settings=None, history=None, *, cache=None, steering=None, outbox=None
|
||||||
|
):
|
||||||
|
seen.append(task)
|
||||||
|
return {"messages": [{"role": "assistant", "content": "图里是一只猫"}]}
|
||||||
|
|
||||||
|
monkeypatch.setattr(builtin, "run_task", fake_run_task)
|
||||||
|
channel = EchoChannel()
|
||||||
|
statuses = _drive(
|
||||||
|
tmp_path / "bot.db",
|
||||||
|
[_media_envelope(_image_part(), text="这是什么")],
|
||||||
|
channel,
|
||||||
|
home=tmp_path,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert statuses == [RunStatus.SUCCEEDED]
|
||||||
|
blocks = seen[0]
|
||||||
|
assert isinstance(blocks, list)
|
||||||
|
assert [b["type"] for b in blocks] == ["text", "image_url"]
|
||||||
|
assert blocks[0]["text"] == "这是什么"
|
||||||
|
assert blocks[1]["image_url"]["url"].startswith("data:image/png;base64,")
|
||||||
|
|
||||||
|
rows = asyncio.run(repo.list_messages(tmp_path / "bot.db", session_id="s1"))
|
||||||
|
assert rows[0]["role"] == "user"
|
||||||
|
assert "[附件]" in rows[0]["content"]
|
||||||
|
assert "base64" not in rows[0]["content"] # 历史只存清单,不存字节
|
||||||
|
# 落盘在 home 下的会话目录里
|
||||||
|
assert list((tmp_path / "data" / "inbound").rglob("*.png"))
|
||||||
|
# 回复没有媒体 → 仍走纯文本
|
||||||
|
assert channel.sent == ["图里是一只猫"]
|
||||||
|
assert channel.contents == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_agent_reply_media_only_inbound_is_multimodal(
|
||||||
|
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
|
||||||
|
) -> None:
|
||||||
|
"""只有图、没有正文:不能被当成空消息,也不能退化成纯文本把图丢掉。"""
|
||||||
|
seen: list[Any] = []
|
||||||
|
|
||||||
|
async def fake_run_task(
|
||||||
|
task, settings=None, history=None, *, cache=None, steering=None, outbox=None
|
||||||
|
):
|
||||||
|
seen.append(task)
|
||||||
|
return {"messages": [{"role": "assistant", "content": "收到"}]}
|
||||||
|
|
||||||
|
monkeypatch.setattr(builtin, "run_task", fake_run_task)
|
||||||
|
channel = EchoChannel()
|
||||||
|
statuses = _drive(
|
||||||
|
tmp_path / "bot.db", [_media_envelope(_image_part())], channel, home=tmp_path
|
||||||
|
)
|
||||||
|
|
||||||
|
assert statuses == [RunStatus.SUCCEEDED]
|
||||||
|
assert [b["type"] for b in seen[0]] == ["image_url"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_agent_reply_reports_rejected_attachments(
|
||||||
|
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
|
||||||
|
) -> None:
|
||||||
|
"""附件全部没接住:说清原因,不再笼统地当成「空消息」。"""
|
||||||
|
|
||||||
|
async def fake_prepare(
|
||||||
|
message, *, root, limits=None, policy=None, headers=None, transform=None
|
||||||
|
):
|
||||||
|
return PreparedInbound(text="", notes=["图片未接收:超过大小上限"])
|
||||||
|
|
||||||
|
async def boom(*args, **kwargs): # pragma: no cover - 不应被调用
|
||||||
|
raise AssertionError("附件全部失败时不应执行任务")
|
||||||
|
|
||||||
|
monkeypatch.setattr(builtin, "prepare_inbound", fake_prepare)
|
||||||
|
monkeypatch.setattr(builtin, "run_task", boom)
|
||||||
|
channel = EchoChannel()
|
||||||
|
statuses = _drive(
|
||||||
|
tmp_path / "bot.db", [_media_envelope(_image_part())], channel, home=tmp_path
|
||||||
|
)
|
||||||
|
|
||||||
|
assert statuses == [RunStatus.SUCCEEDED]
|
||||||
|
assert channel.sent == ["附件未能接收:图片未接收:超过大小上限"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_agent_reply_delivers_agent_media_in_one_content_message(
|
||||||
|
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
|
||||||
|
) -> None:
|
||||||
|
"""智能体用工具发的图:与正文合成一条富内容,不额外再发一条纯文本。"""
|
||||||
|
picture = tmp_path / "cat.png"
|
||||||
|
picture.write_bytes(PNG_HEAD)
|
||||||
|
|
||||||
|
async def fake_run_task(
|
||||||
|
task, settings=None, history=None, *, cache=None, steering=None, outbox=None
|
||||||
|
):
|
||||||
|
assert outbox is not None
|
||||||
|
outbox.push(ImageContent(local_path=str(picture), mime_type="image/png"))
|
||||||
|
return {"messages": [{"role": "assistant", "content": "图来了"}]}
|
||||||
|
|
||||||
|
monkeypatch.setattr(builtin, "run_task", fake_run_task)
|
||||||
|
channel = EchoChannel()
|
||||||
|
statuses = _drive(tmp_path / "bot.db", [_envelope("画只猫")], channel, home=tmp_path)
|
||||||
|
|
||||||
|
assert statuses == [RunStatus.SUCCEEDED]
|
||||||
|
assert channel.sent == [] # 没有再发一条纯文本
|
||||||
|
assert len(channel.contents) == 1
|
||||||
|
parts = channel.contents[0]
|
||||||
|
assert isinstance(parts[0], TextContent) and parts[0].text == "图来了"
|
||||||
|
assert isinstance(parts[1], ImageContent)
|
||||||
|
assert parts[1].local_path == str(picture)
|
||||||
|
|
||||||
|
|
||||||
|
def test_agent_reply_media_only_reply_skips_empty_placeholder(
|
||||||
|
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
|
||||||
|
) -> None:
|
||||||
|
"""只发了媒体、没写正文:不要补一句「(无输出)」。"""
|
||||||
|
picture = tmp_path / "cat.png"
|
||||||
|
picture.write_bytes(PNG_HEAD)
|
||||||
|
|
||||||
|
async def fake_run_task(
|
||||||
|
task, settings=None, history=None, *, cache=None, steering=None, outbox=None
|
||||||
|
):
|
||||||
|
assert outbox is not None
|
||||||
|
outbox.push(ImageContent(local_path=str(picture), mime_type="image/png"))
|
||||||
|
return {"messages": [{"role": "assistant", "content": ""}]}
|
||||||
|
|
||||||
|
monkeypatch.setattr(builtin, "run_task", fake_run_task)
|
||||||
|
channel = EchoChannel()
|
||||||
|
statuses = _drive(tmp_path / "bot.db", [_envelope("发张图")], channel, home=tmp_path)
|
||||||
|
|
||||||
|
assert statuses == [RunStatus.SUCCEEDED]
|
||||||
|
assert channel.sent == []
|
||||||
|
assert [type(part) for part in channel.contents[0]] == [ImageContent]
|
||||||
|
|
||||||
|
|
||||||
|
def test_media_policy_falls_back_to_defaults(tmp_path: Path) -> None:
|
||||||
|
"""拿不到渠道实例时按默认上限跑,不能因为查不到渠道就跳过校验。"""
|
||||||
|
services = ChannelServices(
|
||||||
|
channels=ChannelManager(), settings=Settings(home=tmp_path), db=tmp_path / "bot.db"
|
||||||
|
)
|
||||||
|
limits, headers = services.media_policy("no-such-channel")
|
||||||
|
assert headers == {}
|
||||||
|
assert limits.image > 0 and limits.file > 0
|
||||||
@@ -135,7 +135,7 @@ def test_run_service_end_to_end(
|
|||||||
RecordingChannel.instances.clear()
|
RecordingChannel.instances.clear()
|
||||||
|
|
||||||
async def fake_run_task(
|
async def fake_run_task(
|
||||||
task, settings=None, history=None, *, cache=None, steering=None
|
task, settings=None, history=None, *, cache=None, steering=None, outbox=None
|
||||||
):
|
):
|
||||||
assert task == "hello"
|
assert task == "hello"
|
||||||
return {"messages": [{"role": "assistant", "content": "答:" + task}]}
|
return {"messages": [{"role": "assistant", "content": "答:" + task}]}
|
||||||
@@ -189,7 +189,7 @@ def test_run_service_passes_reply_context(
|
|||||||
RecordingChannel.instances.clear()
|
RecordingChannel.instances.clear()
|
||||||
|
|
||||||
async def fake_run_task(
|
async def fake_run_task(
|
||||||
task, settings=None, history=None, *, cache=None, steering=None
|
task, settings=None, history=None, *, cache=None, steering=None, outbox=None
|
||||||
):
|
):
|
||||||
return {"messages": [{"role": "assistant", "content": "ok"}]}
|
return {"messages": [{"role": "assistant", "content": "ok"}]}
|
||||||
|
|
||||||
|
|||||||
+83
-4
@@ -16,6 +16,7 @@ from bot042.channels.models import (
|
|||||||
ChannelConfig,
|
ChannelConfig,
|
||||||
ChannelMessage,
|
ChannelMessage,
|
||||||
ChannelSubject,
|
ChannelSubject,
|
||||||
|
ImageContent,
|
||||||
InboundMessage,
|
InboundMessage,
|
||||||
OutboundStream,
|
OutboundStream,
|
||||||
TextContent,
|
TextContent,
|
||||||
@@ -24,7 +25,13 @@ from bot042.channels.registry import register_channel
|
|||||||
from bot042.config import Settings
|
from bot042.config import Settings
|
||||||
from bot042.db import repo
|
from bot042.db import repo
|
||||||
from bot042.flows import builtin
|
from bot042.flows import builtin
|
||||||
from bot042.flows.builtin import BUDGET_REPLY, RPC_STREAM, AgentReply, ChannelServices
|
from bot042.flows.builtin import (
|
||||||
|
BUDGET_REPLY,
|
||||||
|
RPC_SEND,
|
||||||
|
RPC_STREAM,
|
||||||
|
AgentReply,
|
||||||
|
ChannelServices,
|
||||||
|
)
|
||||||
from bot042.flows.bus import EventBus
|
from bot042.flows.bus import EventBus
|
||||||
from bot042.flows.rpc import RpcHub
|
from bot042.flows.rpc import RpcHub
|
||||||
from bot042.flows.stream import ReplyStream
|
from bot042.flows.stream import ReplyStream
|
||||||
@@ -125,6 +132,8 @@ class StreamChannel(BaseChannel):
|
|||||||
) -> None:
|
) -> None:
|
||||||
super().__init__(processor, config=config, channel_id=channel_id, **kwargs)
|
super().__init__(processor, config=config, channel_id=channel_id, **kwargs)
|
||||||
self.frames: list[tuple[str, bool]] = []
|
self.frames: list[tuple[str, bool]] = []
|
||||||
|
self.contents: list[list] = []
|
||||||
|
"""每次富内容发送的分片组合(流式通道只送文本,媒体走这里)。"""
|
||||||
StreamChannel.instances.append(self)
|
StreamChannel.instances.append(self)
|
||||||
|
|
||||||
async def start(self) -> None:
|
async def start(self) -> None:
|
||||||
@@ -155,7 +164,7 @@ class StreamChannel(BaseChannel):
|
|||||||
self.frames.append((text, True))
|
self.frames.append((text, True))
|
||||||
|
|
||||||
async def _send_content(self, subject: ChannelSubject, parts: list) -> None:
|
async def _send_content(self, subject: ChannelSubject, parts: list) -> None:
|
||||||
pass
|
self.contents.append(list(parts))
|
||||||
|
|
||||||
async def _send_media(self, subject: ChannelSubject, media: Any) -> None:
|
async def _send_media(self, subject: ChannelSubject, media: Any) -> None:
|
||||||
pass
|
pass
|
||||||
@@ -170,7 +179,9 @@ def test_agent_reply_acks_then_streams(
|
|||||||
"""收到消息先 ack,随后逐步覆盖,最后以 finish 收尾。"""
|
"""收到消息先 ack,随后逐步覆盖,最后以 finish 收尾。"""
|
||||||
StreamChannel.instances.clear()
|
StreamChannel.instances.clear()
|
||||||
|
|
||||||
async def fake_stream(task, settings=None, history=None, *, cache=None, steering=None):
|
async def fake_stream(
|
||||||
|
task, settings=None, history=None, *, cache=None, steering=None, outbox=None
|
||||||
|
):
|
||||||
for text in ("你", "你好"):
|
for text in ("你", "你好"):
|
||||||
yield text
|
yield text
|
||||||
|
|
||||||
@@ -228,7 +239,9 @@ def test_agent_reply_streams_budget_notice_after_partial(
|
|||||||
"""步数用尽:说明补在已流出的内容后面,落库,且不再补发通用错误文案。"""
|
"""步数用尽:说明补在已流出的内容后面,落库,且不再补发通用错误文案。"""
|
||||||
StreamChannel.instances.clear()
|
StreamChannel.instances.clear()
|
||||||
|
|
||||||
async def exhausted(task, settings=None, history=None, *, cache=None, steering=None):
|
async def exhausted(
|
||||||
|
task, settings=None, history=None, *, cache=None, steering=None, outbox=None
|
||||||
|
):
|
||||||
yield "做到一半"
|
yield "做到一半"
|
||||||
raise StepBudgetExceeded(200)
|
raise StepBudgetExceeded(200)
|
||||||
|
|
||||||
@@ -284,6 +297,72 @@ def test_agent_reply_streams_budget_notice_after_partial(
|
|||||||
assert rows[-1]["content"] == final
|
assert rows[-1]["content"] == final
|
||||||
|
|
||||||
|
|
||||||
|
def test_agent_reply_sends_media_after_stream_finishes(
|
||||||
|
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
|
||||||
|
) -> None:
|
||||||
|
"""流式路径:文本走流式消息,媒体在文本定稿后单独补一条。"""
|
||||||
|
StreamChannel.instances.clear()
|
||||||
|
picture = tmp_path / "cat.png"
|
||||||
|
picture.write_bytes(b"\x89PNG\r\n\x1a\n" + b"\x00" * 32)
|
||||||
|
|
||||||
|
async def fake_stream(
|
||||||
|
task, settings=None, history=None, *, cache=None, steering=None, outbox=None
|
||||||
|
):
|
||||||
|
assert outbox is not None
|
||||||
|
outbox.push(ImageContent(local_path=str(picture), mime_type="image/png"))
|
||||||
|
yield "图"
|
||||||
|
yield "图来了"
|
||||||
|
|
||||||
|
monkeypatch.setattr(builtin, "stream_task", fake_stream)
|
||||||
|
|
||||||
|
async def _run() -> tuple[list[tuple[str, bool]], list[list], list[RunStatus]]:
|
||||||
|
bus = EventBus()
|
||||||
|
bus.bind_loop()
|
||||||
|
hub = RpcHub(bus)
|
||||||
|
manager = ChannelManager()
|
||||||
|
channel = StreamChannel(channel_id="s1")
|
||||||
|
await manager.add_channel(channel)
|
||||||
|
await manager.start()
|
||||||
|
_ = hub.serve(RPC_STREAM, manager.rpc_stream)
|
||||||
|
_ = hub.serve(RPC_SEND, manager.rpc_send)
|
||||||
|
|
||||||
|
services = ChannelServices(
|
||||||
|
channels=manager,
|
||||||
|
settings=Settings(home=tmp_path),
|
||||||
|
db=tmp_path / "bot.db",
|
||||||
|
hub=hub,
|
||||||
|
)
|
||||||
|
runner = TaskRunner(bus, deps=services)
|
||||||
|
runner.register(AgentReply)
|
||||||
|
_ = runner.add_task(AgentReply.name, EventTrigger(event="channel.inbound"))
|
||||||
|
await runner.start()
|
||||||
|
|
||||||
|
bus.emit(
|
||||||
|
"channel.inbound",
|
||||||
|
ChannelMessage.of(
|
||||||
|
InboundMessage(
|
||||||
|
channel_id="s1",
|
||||||
|
channel_type="stream-test",
|
||||||
|
channel_subject=ChannelSubject(subject_id="u1"),
|
||||||
|
content=[TextContent(text="画只猫")],
|
||||||
|
)
|
||||||
|
),
|
||||||
|
)
|
||||||
|
await asyncio.sleep(0.05)
|
||||||
|
await runner.wait_idle()
|
||||||
|
statuses = [run.status for run in runner.runs(AgentReply.name)]
|
||||||
|
await runner.stop()
|
||||||
|
hub.close()
|
||||||
|
await manager.stop()
|
||||||
|
return channel.frames, channel.contents, statuses
|
||||||
|
|
||||||
|
frames, contents, statuses = asyncio.run(_run())
|
||||||
|
assert frames[-1] == ("图来了", True) # 文本仍然走流式收尾
|
||||||
|
assert statuses == [RunStatus.SUCCEEDED]
|
||||||
|
assert len(contents) == 1 # 媒体单独一条,且只含媒体分片
|
||||||
|
assert [type(part) for part in contents[0]] == [ImageContent]
|
||||||
|
|
||||||
|
|
||||||
def test_outbound_stream_payload_defaults() -> None:
|
def test_outbound_stream_payload_defaults() -> None:
|
||||||
payload = OutboundStream(
|
payload = OutboundStream(
|
||||||
channel_id="c", subject_id="u", stream_id="s", text="x"
|
channel_id="c", subject_id="u", stream_id="s", text="x"
|
||||||
|
|||||||
Reference in new issue
Block a user