8 Commits
Author SHA1 Message Date
openbotandopenbot-coder dfd6fab3af fix: 补齐 v0.6.3 遗漏的版本号(__init__.py / uv.lock) (#4)
## 问题
v0.6.3 的 bump 提交只改了 pyproject.toml 一处,漏掉:
- src/pytdxdata/__init__.py 的 __version__
- uv.lock 的 pytdxdata 条目版本

后果(远端 main 已复现):
- CI 两个 job 全部失败:`uv sync --locked` 报
  "The lockfile at `uv.lock` needs to be updated, but `--locked` was provided"
- Docs / Release 同样失败
- v0.6.3 tag 已打,但 PyPI 上 0.6.3 仍是 404(发布没成功)

## 修复
补齐两处版本号,四处对齐:pyproject.toml / __init__.py / uv.lock / CHANGELOG.md

## 验证
- `uv sync --locked` 退出码 0
- `pytdxdata.__version__` = `importlib.metadata.version()` = 0.6.3

Co-authored-by: openbot-coder <openbot-coder@users.noreply.github.com>
2026-10-22 08:52:19 +08:00
openbot 0618062e54 chore: bump version 0.6.3 2026-10-18 16:12:51 +08:00
openbot 9c97fea71b fix(pool): ExPool acquire backpressure + ex_pool_cap (fixes #1)
- acquire(): replace 3x0.05s fast-fail with Condition-based queueing
  (default acquire_timeout=30s); timeout error carries pool state
  (idle/total/servers/per_server_cap)
- api: expose ex_pool_cap (default 2, aligned with std/mac pool caps);
  single-server scenario no longer silently capped at 1 concurrent
2026-10-18 16:10:15 +08:00
openbot-coder 2d62d82b51 feat: 新增3台EX扩展行情服务器(connect.cfg官方扩展市场,handshake=login),全量复测30/36可用 2026-10-16 13:53:19 +08:00
openbot-coder 7e72f70eb2 chore: 离线探针复测写回 servers.json(2026-10-16,可达25→27,117.34.114.13/.14 复活) 2026-10-16 12:51:12 +08:00
openbotandopenbot-coder b2f899df49 fix(models): 修复美股分钟K线 datetime 时区拼接错位,还原为北京时间 (#2)
通达信 EX 通道对美股返回的分钟级 K线把两个时区拼进同一个五元组:日期分量是
美东交易日(zipday),时分分量是北京时间(tminutes)。直拼出的 datetime 会把
同一交易日的下午半场排到上午半场之前——单日 390 根首末颠倒,且全部落在同一个
北京自然日里,导致排序 / count 切片 / 日期区间二分与过滤 / 增量水位线全部不可用。

在 SecurityBar.__post_init__ 统一归一化为北京时间(凌晨段日期 +1 天,含月末与
年末进位),EX 的两条 K线命令路径(_bars_ex 的 0x23FF 与二分所依赖的 MAC 0x122E)
因此共用同一口径,不会出现"取数按一套时间、二分按另一套时间"的错位。

新增两个只读属性:trading_date(归属交易日;美股一个交易日横跨两个北京自然日,
按交易日聚合必须用它,不能用 datetime.date())与 is_us。

实测(安装 wheel 0.6.2,非源码树):10-06 ~ 10-12 五个完整交易日各 390 根、
严格单调,首根 open / 末根 close 与日线逐位相同,成交量差 <= 1;A股 / 港股零回归。

- 版本 0.6.1 -> 0.6.2(pyproject / __init__ / uv.lock)
- tests/test_protocol.py 新增 9 项回归测试(分界点 / 月末年末进位 / 390 根整日单调性 / MAC 路径),全量 191 passed
- CHANGELOG / README / SKILL.md / docs(faq、llms.txt、api/models)同步口径说明

遗留(本版未含,待决策):EX 分时对美股返回的 MinuteBar.hour 是未取模的连续分钟数
(实测可达 27,即次日 03:xx),且 MinuteBar 无日期分量可承载跨日信息。

Co-authored-by: openbot-coder <openbot-coder@users.noreply.github.com>
2026-10-13 16:47:57 +08:00
openbot-coder 8193d560da Revert "fix(models): 修复美股分钟K线 datetime 时区拼接错位,还原为北京时间"
This reverts commit 5408903a4c.
2026-10-13 16:42:41 +08:00
openbot-coder 5408903a4c fix(models): 修复美股分钟K线 datetime 时区拼接错位,还原为北京时间
通达信 EX 通道对美股返回的分钟级 K线把两个时区拼进同一个五元组:日期分量是
美东交易日(zipday),时分分量是北京时间(tminutes)。直拼出的 datetime 会把
同一交易日的下午半场排到上午半场之前——单日 390 根首末颠倒,且全部落在同一个
北京自然日里,导致排序 / count 切片 / 日期区间二分与过滤 / 增量水位线全部不可用。

在 SecurityBar.__post_init__ 统一归一化为北京时间(凌晨段日期 +1 天,含月末与
年末进位),EX 的两条 K线命令路径(_bars_ex 的 0x23FF 与二分所依赖的 MAC 0x122E)
因此共用同一口径,不会出现"取数按一套时间、二分按另一套时间"的错位。

新增两个只读属性:trading_date(归属交易日;美股一个交易日横跨两个北京自然日,
按交易日聚合必须用它,不能用 datetime.date())与 is_us。

实测(安装 wheel 0.6.2,非源码树):10-06 ~ 10-12 五个完整交易日各 390 根、
严格单调,首根 open / 末根 close 与日线逐位相同,成交量差 <= 1;A股 / 港股零回归。

- 版本 0.6.1 -> 0.6.2(pyproject / __init__ / uv.lock)
- tests/test_protocol.py 新增 9 项回归测试(分界点 / 月末年末进位 / 390 根整日单调性 / MAC 路径),全量 191 passed
- CHANGELOG / README / SKILL.md / docs(faq、llms.txt、api/models)同步口径说明

遗留(本版未含,待决策):EX 分时对美股返回的 MinuteBar.hour 是未取模的连续分钟数
(实测可达 27,即次日 03:xx),且 MinuteBar 无日期分量可承载跨日信息。
2026-10-13 15:10:22 +08:00
15 changed files with 662 additions and 190 deletions

No files matched your search

+47
View File
@@ -2,6 +2,53 @@
本项目遵循 [语义化版本](https://semver.org/lang/zh-CN/)。
## [0.6.3] - 2026-10-18
- fix(pool): ExPool acquire 背压等待(Condition 排队,默认 30s 超时)替代 3x0.05s 快速失败;超时异常带池状态
- feat(api): 新增 ex_pool_cap 参数(默认 2),单 EX 服务器场景不再静默退化为 1 并发(fixes #1)
## [0.6.2] - 2026-10-13
### 修复
- **美股分钟级 K 线的 `datetime` 全部错位**(`get_bars("usAAPL", KlinePeriod.MIN_1)`)。
通达信 EX 通道返回的美股分钟线时间戳,把**两个时区拼进了同一个五元组**:日期分量是
**美东交易日**(协议里的 `zipday`),时分分量却是**北京时间**(`tminutes`)。直拼出的
`datetime` 会把同一交易日的**下午半场排到上午半场之前**——单日 390 根不仅首末颠倒,
而且全部落在同一个北京自然日里。于是任何依赖 `datetime` 的行为都是错的:排序、
`count` 切片("取最新 N 根")、日期区间二分与过滤、增量更新水位线。
| 观测项 | 修复前 | 修复后 |
|---|---|---|
| 单日 390 根的日期分布 | **全部**落在 10-12(hour 0~23 混在一起) | 10-12 的 21:31~23:59(149 根)+ 10-13 的 00:00~04:00(241 根) |
| 时间轴 | 上午半场被排到下午半场**之前** | 严格单调递增 |
| `count=30` 是否取到真正最新的 30 根 | 否(拿到上一交易日的 23:30~23:59) | 是(当日收盘段 03:31~04:00) |
| 每组首根 `open` / 末根 `close` 对日线 | 对不上 | **逐位相同**(10-07/08/09/12 四个交易日实测) |
现在 `SecurityBar.datetime` **统一为北京时间**:美股凌晨段(北京 hour < 12)的日期
自动 +1 天(含月末 / 年末进位)。A股 / 港股 / 期货原始即为北京时间,**零行为变化**。
新增两个只读属性:
| 属性 | 用途 |
|---|---|
| `SecurityBar.trading_date` | 该 bar 归属的交易日 `YYYY-MM-DD`。美股一个交易日横跨两个北京自然日,**按交易日聚合(分钟线合成日线、日内统计)必须用它,不能用 `datetime.date()`** |
| `SecurityBar.is_us` | 是否美股(EX 通道 `US_STOCK`) |
归一化在 `SecurityBar.__post_init__` 中实现,因此 **EX 的两条 K 线命令路径**
(`_bars_ex` 的 0x23FF 与日期区间二分所依赖的 MAC 0x122E)自动共用同一份口径,
不会出现"取数按一套时间、二分按另一套时间"的错位。
### 说明
- 分钟线缓存 TTL 为 **60 秒**,升级后旧缓存最多残留 1 分钟即自然失效,**无需手动清理**;
日线(`period >= DAY`)不受本缺陷影响。
- 本版只修 K 线。**同日发现的同类问题未包含在内**:EX 分时(`get_minutes`)对美股返回的
`MinuteBar.hour` 是**未取模的连续分钟数**(实测可达 27,即次日 03:xx),且 `MinuteBar`
无日期分量可承载跨日信息——修法涉及"是否新增字段 / 日期如何归属"的取舍,留待下一版决策。
- 新增 9 项针对性回归测试(分界点、月末与年末进位、390 根整日单调性、MAC 路径同样归一化)。
## [0.6.1] - 2026-09-21
### 变更
+4 -1
View File
@@ -104,7 +104,7 @@ curl -o ~/.workbuddy/skills/pytdxdata/SKILL.md --create-dirs \
```bash
uv sync # 建虚拟环境 + 注册 tdx 命令
uv run pytest tests/ # 182 个测试用例(协议 / 池 / 缓存 / 分页 / 全接口回归 / 文件解析 / CLI / 文档漂移 / 当日测速缓存)
uv run pytest tests/ # 191 个测试用例(协议 / 池 / 缓存 / 分页 / 全接口回归 / 文件解析 / CLI / 文档漂移 / 当日测速缓存)
uv sync --group docs # 文档站依赖(mkdocs-material + mkdocstrings)
uv run mkdocs serve # 本地预览文档站
```
@@ -114,6 +114,9 @@ uv run mkdocs serve # 本地预览文档站
- 行情来自通达信公开服务器,**15 分钟延时**;需实时数据请使用券商行情
- 标准通道 K 线 `count < 100` 部分服务器返回空 → 分页器已强制 `min_page=100` 规避
- 港股 / 美股期权在通达信协议内无数据
- **K 线时间戳统一为北京时间**(v0.6.2 起)。美股原始报文把「美东交易日日期」与「北京时间时分」
拼在一起,库已自动归一化,`bar.datetime` 可直接排序 / 过滤;但**美股一个交易日横跨两个北京
自然日**,按交易日聚合请用 `SecurityBar.trading_date`,不要用 `datetime.date()`
- 服务器可用性随时间变化。**当日首次启动会自动在后台测速并缓存**(对调用方无感,当天不再重测);
也可用 `python -m pytdxdata.server_probe` 离线重生成随包清单。
缓存路径与三个开关见 [安装与配置 → 当日自动测速](https://openbot-coder.github.io/pytdxdata/getting-started/#当日自动测速无感)
+2
View File
@@ -164,6 +164,7 @@ await td.get_bars("cffex:IFL0", KlinePeriod.DAY, count=700) # 期货合约
- **数据有 15 分钟延时**(通达信公开服务器特性)。要实时数据请用券商行情。
- **复权 K 线与 1 分钟 K 线走 MAC 通道**。`get_bars(..., adjust=Adjust.QFQ)` 自动切换。
- **K 线时间戳统一为北京时间**(v0.6.2 起)。美股原始报文里日期是美东交易日、时分是北京时间(两个时区拼在一起),库已自动归一化,`bar.datetime` 可直接排序 / 比较 / 过滤。但**美股一个交易日横跨两个北京自然日**,按交易日聚合(分钟线合成日线、日内统计)必须用 `bar.trading_date`。
- **`count` 很大时会自动多连接并发分页**,无需手动翻页。但标准通道 K 线 `count < 100` 部分服务器会返回空——内部已强制 `min_page=100` 规避。
- **`get_quotes` 一次 >80 只自动切批**;走 MAC 字段位通道(`fields=`)时上限为 80 只/次。
- **历史逐笔没有笔数字段**,只有当日逐笔的 `trade_count` 有效。
@@ -183,6 +184,7 @@ await td.get_bars("cffex:IFL0", KlinePeriod.DAY, count=700) # 期货合约
| 用 `get_block_info` 又自己拆二进制 | 白写解析器 | 用 `get_block_parsed`,已经解析好了 |
| 循环里逐个 `await get_bars` | 慢 | 用列表传多个标的,自动并发 |
| 港股/美股五档 bid/ask | 没有(协议不支持) | `fields` 字段位通道有 name 但无五档 |
| 美股分钟线用 `datetime.date()` 分组做日聚合 | 一个交易日被拆成两天(美股跨北京零点) | 用 `bar.trading_date`;`bar.datetime` 是归一化后的北京时间 |
| 依赖实时性做信号 | 数据是 15 分钟前的 | 换券商实时源 |
| 手动改包内 `servers.json` / 写死 IP | 容器只读、升级被覆盖、快节点捞不回来 | 日常靠内置的**当日测速缓存**自动选路;要强制指定就用 `PYPYTDX_DATA_{STANDARD,MAC,EX}_HOSTS` 环境变量 |
+16
View File
@@ -62,6 +62,22 @@ from pytdxdata.models import BoardInfo, MemberQuote, AuctionItem, CapitalFlow
| 结构 | 具名属性(`zongguben`、`jingzichan` ...) | `fields: list[float]`,原始字段数组 |
| 单位 | 万元 / 万股 | 原始 float,需自行按字段语义换算 |
!!! tip "`SecurityBar.datetime` 的时间戳口径(v0.6.2 起)"
时间统一为**北京时间**。A股 / 港股 / 期货的原始时间戳即为北京时间;美股的原始报文
混用了「美东交易日日期」与「北京时间时分」两个时区,库在构造时已自动归一化
(凌晨段日期 +1 天),因此 `bar.datetime` 可直接排序 / 比较 / 过滤。
但要注意**美股一个交易日横跨两个北京自然日**,按交易日聚合请用 `bar.trading_date`:
```python
bars = await td.get_bars("usAAPL", KlinePeriod.MIN_1, count=390)
bars[0].datetime # 2026-10-12 21:31:00(北京时间)
bars[-1].datetime # 2026-10-13 04:00:00(次日)
bars[0].trading_date # "2026-10-12" —— 归属的美东交易日
```
`bar.is_us` 可判定是否美股。详见 [常见问题 → 美股分钟 K 线](../faq.md)。
## 完整字段定义
以下由 `mkdocstrings` 从源码实时渲染(**以源码为准**):
+17
View File
@@ -99,6 +99,23 @@
2. **是否含未收盘的那根** —— 盘中最后一根是动态的
3. 分钟线注意**数据来源深度**不同
??? question "美股分钟 K 线的时间为什么跨了两个日期?"
**已在上游修好(v0.6.2)**。通达信 EX 通道对美股返回的原生时间戳把两个时区拼在一起:
日期分量是**美东交易日**、时分分量是**北京时间**。修复前,同一交易日的 390 根会全部
落在同一个北京自然日里,并且下午半场被排在上午半场**之前**。
现在 `SecurityBar.datetime` 已统一为**北京时间**(凌晨段日期自动 +1 天),可直接排序 /
过滤 / 做增量水位线。但要注意**美股一个交易日横跨两个北京自然日**:
```python
bars = await td.get_bars("usAAPL", KlinePeriod.MIN_1, count=390)
print(bars[0].datetime, bars[-1].datetime) # 2026-10-12 21:31:00 → 2026-10-13 04:00:00
print({b.trading_date for b in bars}) # {'2026-10-12'} —— 同一个交易日
```
按**交易日**聚合(分钟线合成日线、日内统计)请用 `b.trading_date`,不要用
`b.datetime.date()`。A股 / 港股 / 期货原始即为北京时间,不受影响。
??? question "缓存会不会返回过期数据?"
缓存按数据类型分级 TTL,不会返回超出 TTL 的数据:
+4
View File
@@ -93,6 +93,10 @@ tdx financial gpcw20260331.zip --limit 5
- **`KlinePeriod.MIN_1` 走 MAC 通道并单连接串行分页**,这是刻意设计:并发分页在多服务器
历史深度不一致时会产生非确定性数据空洞。不要试图"优化"成并发。
- **复权 K 线走 MAC 通道**,标准通道不提供复权数据。
- **K 线 `datetime` 统一为北京时间**(v0.6.2 起)。美股原始报文的日期分量是美东交易日、
时分分量是北京时间,库已自动归一化,`bar.datetime` 可直接排序 / 比较 / 过滤。
但**美股一个交易日横跨两个北京自然日**:按交易日聚合(分钟线合成日线、日内统计)
用 `bar.trading_date`,不要用 `bar.datetime.date()`。`bar.is_us` 判定美股。
- **标准通道部分服务器在 `count < 100` 时返回空**,分页器已强制 `min_page=100` 规避。
- **本地 Windows 上 `asyncio.new_event_loop()` 可能永久挂起**,根因是回环 `socket.socketpair()`
被本地代理/安全软件拦截。异步测试请在 Linux/WSL 运行。
+1 -1
View File
@@ -1,6 +1,6 @@
[project]
name = "pytdxdata"
version = "0.6.1"
version = "0.6.3"
description = "通达信(TongDaXin)行情数据客户端:动态连接池 + 双层TTL缓存 + 3连接并发分页,纯asyncio零运行时依赖,支持A股/港股/美股/期货/期权"
readme = "README.md"
requires-python = ">=3.11"
+1 -1
View File
@@ -16,7 +16,7 @@ from .models import (
XdxrRecord,
)
__version__ = "0.6.1"
__version__ = "0.6.3"
__all__ = [
"TdxData",
"Market",
+5
View File
@@ -233,11 +233,13 @@ class TdxData:
pool_max: int = 12,
mac_pool_cap: int = 4,
std_pool_cap: int = 4,
ex_pool_cap: int = 2,
) -> None:
self.port = port
self.timeout = timeout
self.default_adjust = default_adjust
self.mac_pool_cap = mac_pool_cap
self.ex_pool_cap = ex_pool_cap
self._std_servers = list(standard_servers or get_standard_hosts())
self._mac_servers = list(mac_servers or get_mac_hosts())
self._ex_servers = list(ex_servers or get_ex_hosts())
@@ -262,8 +264,11 @@ class TdxData:
probe_factory=lambda: MacKlineCmd(1, "600000", KlinePeriod.DAY, Adjust.NONE, 0, 100),
)
# EX扩展市场池(港股/期货/美股,端口7727;MAC EX login + pytdx EX setup 混合)
# 单服务器场景(测速缓存收缩后仅 1 台)下,有效并发 = 服务器数 × ex_pool_cap,
# 默认 cap=2 保证至少 2 并发;acquire 有 Condition 背压等待,不再快速失败。
self._ex_pool = ExPool(self._ex_servers, timeout=timeout,
min_size=min(pool_min, 2), max_size=min(pool_max, 6),
per_server_cap=ex_pool_cap,
handshake_map=EX_HANDSHAKES)
disk = None
if self._cache_dir is not None:
+48 -2
View File
@@ -3,7 +3,7 @@ from __future__ import annotations
import struct
from dataclasses import dataclass, field
from datetime import datetime
from datetime import date, datetime, timedelta
from enum import IntEnum
from typing import Any
@@ -92,9 +92,25 @@ class Adjust(IntEnum):
HFQ = 2 # 后复权
# 扩展市场中「时间戳跨时区拼接」的市场号:通达信 EX 通道对美股返回的分钟级 K线
# 里,日期字段是**美东交易日**、时分字段是**北京时间**,两个时区被拼进同一个
# 五元组,必须归一化(见 SecurityBar.__post_init__)。
_US_EX_MARKETS: frozenset[int] = frozenset({int(ExMarket.US_STOCK)})
# 归一化判据:北京时间 hour >= 12 的 bar 与美东交易日属同一自然日。美股常规时段
# 09:30~16:00 EDT = 北京 21:30~次日 04:00(冬令时 22:30~次日 05:00),夏令时/
# 冬令时下分界都稳定落在 12 点。
_US_DAY_SPLIT_HOUR = 12
@dataclass(slots=True)
class SecurityBar:
"""K线记录(价格单位:元)。跨市场统一模型(A股/港股/美股/期货共用)。"""
"""K线记录(价格单位:元)。跨市场统一模型(A股/港股/美股/期货共用)。
时间戳口径**统一为北京时间**:A股/港股/期货原始即为北京时间;美股原始是
「日期=美东交易日 + 时分=北京时间」的混合值,构造时已还原为真实北京时间
(见 ``__post_init__``)。该 bar 归属的交易日见 :attr:`trading_date`。
"""
market: int
code: str
open: float
@@ -110,10 +126,40 @@ class SecurityBar:
minute: int = 0
symbol: str = "" # 规范化标的串(sz000001 / hk00700),由 API 层填充
def __post_init__(self) -> None:
"""美股(EX 通道)时间戳归一化:还原成真实北京时间。
原始报文里 ``zipday`` 是**美东交易日**、``tminutes`` 是**北京时间分钟数**,
直拼出的 ``datetime`` 会把同一交易日的下午半场(北京 00:00~04:00)排到
上午半场(北京 21:30~23:59)**之前** —— 单日 390 根全部错位、首末颠倒。
这里把凌晨段(北京 hour < 12)的日期 +1 天,得到严格单调的北京时间序列。
"""
if self.market in _US_EX_MARKETS and self.hour < _US_DAY_SPLIT_HOUR:
d = date(self.year, self.month, self.day) + timedelta(days=1)
self.year, self.month, self.day = d.year, d.month, d.day
@property
def is_us(self) -> bool:
"""是否美股(EX 通道 ``US_STOCK``)。"""
return self.market in _US_EX_MARKETS
@property
def datetime(self) -> datetime:
"""bar 的时刻(北京时间)。"""
return datetime(self.year, self.month, self.day, self.hour, self.minute)
@property
def trading_date(self) -> str:
"""该 bar 归属的交易日,``YYYY-MM-DD``。
美股:``datetime`` 已还原为北京时间,凌晨段(北京 hour < 12)属于**前一天**
的美东交易日,故回退一天;A股/港股/期货的交易日即自然日。按交易日聚合
(如分钟线合成日线)时应当用本字段,而不是 ``datetime.date()``。
"""
if self.is_us and self.hour < _US_DAY_SPLIT_HOUR:
return (date(self.year, self.month, self.day) - timedelta(days=1)).isoformat()
return f"{self.year:04d}-{self.month:02d}-{self.day:02d}"
@dataclass(slots=True)
class IndexBar(SecurityBar):
+89 -40
View File
@@ -1,4 +1,11 @@
"""EX扩展市场连接池(简化版:固定连接数,借还+坏连接重建+login自动)"""
"""EX扩展市场连接池(固定连接数,借还+坏连接重建+login自动)。
与标准池 pool.py 的背压语义对齐:acquire 在无空闲连接且无法扩容时,
基于 asyncio.Condition 排队等待 release 唤醒(默认 acquire_timeout=30s),
而不是快速失败。per_server_cap 限制单台服务器连接数;单服务器场景下
有效并发 = servers × per_server_cap(默认 1),通过 TdxData(ex_pool_cap=N)
可对同一 EX 主机开多连接。
"""
from __future__ import annotations
import asyncio
@@ -9,6 +16,23 @@ from .ex_connection import ExTdxConnection
log = logging.getLogger(__name__)
DEFAULT_ACQUIRE_TIMEOUT = 30.0
class ExPoolTimeoutError(ConnectionError):
"""acquire 等待超时(携带池状态快照,便于排障)。"""
def __init__(self, pool: "ExPool", timeout: float) -> None:
idle = len(pool._idle)
total = pool._total
servers = len(pool.servers)
cap = pool.per_server_cap
super().__init__(
f"EX连接池获取超时(等了 {timeout:.1f}s):"
f"idle={idle} total={total} servers={servers} per_server_cap={cap} "
f"有效并发={servers * cap}"
)
class ExPool:
"""EX通道连接池(港股/期货,端口7727,连接时自动login)。"""
@@ -16,6 +40,7 @@ class ExPool:
def __init__(self, servers: list[str], *, timeout: float = 10.0,
min_size: int = 2, max_size: int = 6,
per_server_cap: int = 1,
acquire_timeout: float | None = None,
handshake: str = "login",
handshake_map: dict | None = None) -> None:
self.servers = servers
@@ -25,14 +50,17 @@ class ExPool:
self.min_size = min_size
self.max_size = max_size
self.per_server_cap = max(1, per_server_cap) # 一台服务器最多几个连接(默认1=单连接)
self.acquire_timeout = (
DEFAULT_ACQUIRE_TIMEOUT if acquire_timeout is None else acquire_timeout
)
self._idle: list[tuple[str, ExTdxConnection]] = []
self._server_count: dict[str, int] = {}
self._total = 0
self._closed = False
self._lock = asyncio.Lock()
self._cond = asyncio.Condition()
async def start(self) -> None:
async with self._lock:
async with self._cond:
for host in self.servers[: self.min_size]:
if self._server_count.get(host, 0) >= self.per_server_cap:
continue
@@ -47,7 +75,7 @@ class ExPool:
async def close(self) -> None:
self._closed = True
async with self._lock:
async with self._cond:
for _, conn in self._idle:
try:
await conn.close()
@@ -56,48 +84,61 @@ class ExPool:
self._idle.clear()
self._total = 0
self._server_count.clear()
self._cond.notify_all()
async def acquire(self) -> ExTdxConnection:
for _ in range(3):
stale_conn = None
new_conn = None
new_host = None
async with self._lock:
async def acquire(self, timeout: float | None = None) -> ExTdxConnection:
"""借出一条连接;无空闲且无法扩容时排队等待 release 唤醒。
timeout 默认取 self.acquire_timeout(30s);传 None 表示无限等待。
"""
if timeout is None:
timeout = self.acquire_timeout
deadline = None if timeout is None else time.monotonic() + timeout
async with self._cond:
while True:
# 1. 优先借用空闲连接(过期则丢弃,槽位在锁内回收)
if self._idle:
_, conn = self._idle.pop()
fresh = time.monotonic() - conn.last_used < 30.0
if conn.alive and fresh:
return conn
# 连接空闲过久(login会话可能失效)→ 丢弃重连
# 连接空闲过久(login会话可能失效)→ 丢弃,槽位让给扩容
self._total = max(0, self._total - 1)
self._server_count[conn.host] = max(0, self._server_count.get(conn.host, 0) - 1)
stale_conn = conn
# 扩容:在锁内预留槽位,锁外执行网络IO
elif self._total < self.max_size and self.servers:
self._schedule_close(conn)
# 2. 可扩容 → 锁内预留槽位,锁外建连
new_conn = None
new_host = None
if self._total < self.max_size and self.servers:
host = self._next_available_server()
if host:
if host is not None:
self._total += 1
self._server_count[host] = self._server_count.get(host, 0) + 1
new_conn = ExTdxConnection(host, 7727, self.timeout, self.handshake_map.get(host, self.handshake))
new_host = host
# 锁外:关闭过期连接、建立新连接
if stale_conn is not None:
if new_conn is not None:
# 锁外网络 IO:连接失败回滚槽位后继续等待
try:
await new_conn.connect()
return new_conn
except Exception as e:
log.warning("EX扩容失败 %s: %s", new_host, str(e)[:60])
async with self._cond:
self._total = max(0, self._total - 1)
self._server_count[new_host] = max(0, self._server_count.get(new_host, 0) - 1)
# 3. 池满:排队等待 release 唤醒(或超时)
now = time.monotonic()
if deadline is not None and now >= deadline:
raise ExPoolTimeoutError(self, timeout)
try:
await stale_conn.close()
except Exception:
pass
if new_conn is not None:
try:
await new_conn.connect()
return new_conn
except Exception as e:
log.warning("EX扩容失败 %s: %s", new_host, str(e)[:60])
# 连接失败,回滚预留槽位
async with self._lock:
self._total = max(0, self._total - 1)
self._server_count[new_host] = max(0, self._server_count.get(new_host, 0) - 1)
await asyncio.sleep(0.05)
raise ConnectionError("EX连接池无可用连接")
if deadline is None:
await self._cond.wait()
else:
await asyncio.wait_for(self._cond.wait(), timeout=deadline - now)
except asyncio.TimeoutError:
raise ExPoolTimeoutError(self, timeout)
def _next_available_server(self) -> str | None:
for h in self.servers:
@@ -105,15 +146,23 @@ class ExPool:
return h
return None
async def release(self, conn: ExTdxConnection, *, broken: bool = False) -> None:
if broken or not conn.alive:
async with self._lock:
self._total = max(0, self._total - 1)
self._server_count[conn.host] = max(0, self._server_count.get(conn.host, 0) - 1)
def _schedule_close(self, conn: ExTdxConnection) -> None:
async def _close():
try:
await conn.close()
except Exception:
pass
return
async with self._lock:
self._idle.append((conn.host, conn))
try:
asyncio.get_running_loop().create_task(_close())
except RuntimeError:
pass
async def release(self, conn: ExTdxConnection, *, broken: bool = False) -> None:
async with self._cond:
if broken or not conn.alive:
self._total = max(0, self._total - 1)
self._server_count[conn.host] = max(0, self._server_count.get(conn.host, 0) - 1)
self._schedule_close(conn)
else:
self._idle.append((conn.host, conn))
self._cond.notify_all()
+197 -140
View File
@@ -1,6 +1,6 @@
{
"version": "2026-09-21",
"note": "离线探针复测(2026-09-21 19:03): 可达=25/33, 入池fast_pool=10台",
"version": "2026-10-16",
"note": "离线探针复测(2026-10-16 13:46): 可达=30/36, 入池fast_pool=10台",
"servers": [
{
"ip": "139.9.52.158",
@@ -11,14 +11,14 @@
],
"handshake": null,
"status": "ok",
"latency": 0.22,
"checked_at": "2026-09-21 19:03:04",
"latency": 0.172,
"checked_at": "2026-10-16 13:46:17",
"source": "probe-2026-08-12",
"fast_pool": true,
"checked_date": "2026-09-21",
"checked_date": "2026-10-16",
"_per_proto": {
"standard": 0.226,
"mac": 0.22
"standard": 0.172,
"mac": 0.183
}
},
{
@@ -30,14 +30,14 @@
],
"handshake": null,
"status": "ok",
"latency": 0.366,
"checked_at": "2026-09-21 19:03:04",
"latency": 0.298,
"checked_at": "2026-10-16 13:46:18",
"source": "probe-2026-08-12",
"fast_pool": false,
"checked_date": "2026-09-21",
"fast_pool": true,
"checked_date": "2026-10-16",
"_per_proto": {
"standard": 0.396,
"mac": 0.366
"standard": 0.316,
"mac": 0.298
}
},
{
@@ -48,13 +48,13 @@
],
"handshake": null,
"status": "ok",
"latency": 0.257,
"checked_at": "2026-09-21 19:03:09",
"latency": 0.236,
"checked_at": "2026-10-16 13:46:23",
"source": "probe-2026-08-12",
"fast_pool": true,
"checked_date": "2026-09-21",
"checked_date": "2026-10-16",
"_per_proto": {
"mac": 0.257
"mac": 0.236
}
},
{
@@ -65,13 +65,13 @@
],
"handshake": null,
"status": "ok",
"latency": 0.248,
"checked_at": "2026-09-21 19:03:04",
"latency": 0.213,
"checked_at": "2026-10-16 13:46:17",
"source": "probe-2026-08-12",
"fast_pool": true,
"checked_date": "2026-09-21",
"checked_date": "2026-10-16",
"_per_proto": {
"mac": 0.248
"mac": 0.213
}
},
{
@@ -84,15 +84,15 @@
],
"handshake": "setup",
"status": "ok",
"latency": 0.221,
"checked_at": "2026-09-21 19:03:00",
"latency": 0.316,
"checked_at": "2026-10-16 13:46:13",
"source": "probe-2026-08-12",
"fast_pool": true,
"checked_date": "2026-09-21",
"fast_pool": false,
"checked_date": "2026-10-16",
"_per_proto": {
"standard": 0.399,
"mac": 0.414,
"ex": 0.221
"standard": 0.316,
"mac": 0.329,
"ex": 0.539
}
},
{
@@ -104,10 +104,10 @@
"handshake": null,
"status": "fail",
"latency": null,
"checked_at": "2026-09-21 19:03:20",
"checked_at": "2026-10-16 13:46:34",
"source": "probe-2026-08-12",
"fast_pool": false,
"checked_date": "2026-09-21",
"checked_date": "2026-10-16",
"_per_proto": {}
},
{
@@ -118,13 +118,13 @@
],
"handshake": null,
"status": "ok",
"latency": 0.363,
"checked_at": "2026-09-21 19:03:10",
"latency": 0.682,
"checked_at": "2026-10-16 13:46:24",
"source": "probe-2026-08-12",
"fast_pool": true,
"checked_date": "2026-09-21",
"fast_pool": false,
"checked_date": "2026-10-16",
"_per_proto": {
"mac": 0.363
"mac": 0.682
}
},
{
@@ -136,10 +136,10 @@
"handshake": null,
"status": "fail",
"latency": null,
"checked_at": "2026-09-21 19:03:20",
"checked_at": "2026-10-16 13:46:34",
"source": "probe-2026-08-12",
"fast_pool": false,
"checked_date": "2026-09-21",
"checked_date": "2026-10-16",
"_per_proto": {}
},
{
@@ -150,13 +150,13 @@
],
"handshake": null,
"status": "ok",
"latency": 0.344,
"checked_at": "2026-09-21 19:03:10",
"latency": 0.607,
"checked_at": "2026-10-16 13:46:23",
"source": "probe-2026-08-12",
"fast_pool": true,
"checked_date": "2026-09-21",
"fast_pool": false,
"checked_date": "2026-10-16",
"_per_proto": {
"mac": 0.344
"mac": 0.607
}
},
{
@@ -167,13 +167,13 @@
],
"handshake": null,
"status": "ok",
"latency": 0.404,
"checked_at": "2026-09-21 19:03:10",
"latency": 1.264,
"checked_at": "2026-10-16 13:46:24",
"source": "probe-2026-08-12",
"fast_pool": false,
"checked_date": "2026-09-21",
"checked_date": "2026-10-16",
"_per_proto": {
"mac": 0.404
"mac": 1.264
}
},
{
@@ -185,10 +185,10 @@
"handshake": null,
"status": "fail",
"latency": null,
"checked_at": "2026-09-21 19:03:24",
"checked_at": "2026-10-16 13:46:34",
"source": "probe-2026-08-12",
"fast_pool": false,
"checked_date": "2026-09-21",
"checked_date": "2026-10-16",
"_per_proto": {}
},
{
@@ -200,10 +200,10 @@
"handshake": null,
"status": "fail",
"latency": null,
"checked_at": "2026-09-21 19:03:21",
"checked_at": "2026-10-16 13:46:34",
"source": "probe-2026-08-12",
"fast_pool": false,
"checked_date": "2026-09-21",
"checked_date": "2026-10-16",
"_per_proto": {}
},
{
@@ -214,13 +214,13 @@
],
"handshake": null,
"status": "ok",
"latency": 0.376,
"checked_at": "2026-09-21 19:03:10",
"latency": 0.612,
"checked_at": "2026-10-16 13:46:23",
"source": "probe-2026-08-12",
"fast_pool": false,
"checked_date": "2026-09-21",
"checked_date": "2026-10-16",
"_per_proto": {
"mac": 0.376
"mac": 0.612
}
},
{
@@ -231,13 +231,13 @@
],
"handshake": null,
"status": "ok",
"latency": 0.377,
"checked_at": "2026-09-21 19:03:10",
"latency": 0.613,
"checked_at": "2026-10-16 13:46:23",
"source": "probe-2026-08-12",
"fast_pool": false,
"checked_date": "2026-09-21",
"checked_date": "2026-10-16",
"_per_proto": {
"mac": 0.377
"mac": 0.613
}
},
{
@@ -248,13 +248,13 @@
],
"handshake": null,
"status": "ok",
"latency": 0.401,
"checked_at": "2026-09-21 19:03:10",
"latency": 0.624,
"checked_at": "2026-10-16 13:46:23",
"source": "probe-2026-08-12",
"fast_pool": false,
"checked_date": "2026-09-21",
"checked_date": "2026-10-16",
"_per_proto": {
"mac": 0.401
"mac": 0.624
}
},
{
@@ -265,13 +265,13 @@
],
"handshake": null,
"status": "ok",
"latency": 0.42,
"checked_at": "2026-09-21 19:03:10",
"latency": 0.592,
"checked_at": "2026-10-16 13:46:23",
"source": "probe-2026-08-12",
"fast_pool": false,
"checked_date": "2026-09-21",
"checked_date": "2026-10-16",
"_per_proto": {
"mac": 0.42
"mac": 0.592
}
},
{
@@ -282,13 +282,13 @@
],
"handshake": null,
"status": "ok",
"latency": 0.401,
"checked_at": "2026-09-21 19:03:10",
"latency": 0.624,
"checked_at": "2026-10-16 13:46:23",
"source": "probe-2026-08-12",
"fast_pool": false,
"checked_date": "2026-09-21",
"checked_date": "2026-10-16",
"_per_proto": {
"mac": 0.401
"mac": 0.624
}
},
{
@@ -300,14 +300,14 @@
],
"handshake": null,
"status": "ok",
"latency": 0.485,
"checked_at": "2026-09-21 19:03:05",
"latency": 0.306,
"checked_at": "2026-10-16 13:46:18",
"source": "probe-2026-08-12",
"fast_pool": false,
"checked_date": "2026-09-21",
"checked_date": "2026-10-16",
"_per_proto": {
"standard": 0.485,
"mac": 0.605
"standard": 0.352,
"mac": 0.306
}
},
{
@@ -319,10 +319,10 @@
"handshake": null,
"status": "fail",
"latency": null,
"checked_at": "2026-09-21 19:03:04",
"checked_at": "2026-10-16 13:46:18",
"source": "probe-2026-08-12",
"fast_pool": false,
"checked_date": "2026-09-21",
"checked_date": "2026-10-16",
"_per_proto": {}
},
{
@@ -334,14 +334,14 @@
],
"handshake": null,
"status": "ok",
"latency": 0.444,
"checked_at": "2026-09-21 19:03:05",
"latency": 0.307,
"checked_at": "2026-10-16 13:46:18",
"source": "probe-2026-08-12",
"fast_pool": false,
"checked_date": "2026-09-21",
"checked_date": "2026-10-16",
"_per_proto": {
"standard": 0.444,
"mac": 0.903
"standard": 0.35,
"mac": 0.307
}
},
{
@@ -352,13 +352,16 @@
"mac"
],
"handshake": null,
"status": "fail",
"latency": null,
"checked_at": "2026-09-21 19:03:15",
"status": "ok",
"latency": 0.395,
"checked_at": "2026-10-16 13:46:18",
"source": "probe-2026-08-12",
"fast_pool": false,
"checked_date": "2026-09-21",
"_per_proto": {}
"checked_date": "2026-10-16",
"_per_proto": {
"standard": 0.395,
"mac": 0.675
}
},
{
"ip": "117.34.114.15",
@@ -369,14 +372,14 @@
],
"handshake": null,
"status": "ok",
"latency": 0.435,
"checked_at": "2026-09-21 19:03:05",
"latency": 0.345,
"checked_at": "2026-10-16 13:46:18",
"source": "probe-2026-08-12",
"fast_pool": false,
"checked_date": "2026-09-21",
"checked_date": "2026-10-16",
"_per_proto": {
"standard": 0.463,
"mac": 0.435
"standard": 0.345,
"mac": 0.726
}
},
{
@@ -387,13 +390,16 @@
"mac"
],
"handshake": null,
"status": "fail",
"latency": null,
"checked_at": "2026-09-21 19:03:14",
"status": "ok",
"latency": 0.326,
"checked_at": "2026-10-16 13:46:18",
"source": "probe-2026-08-12",
"fast_pool": false,
"checked_date": "2026-09-21",
"_per_proto": {}
"checked_date": "2026-10-16",
"_per_proto": {
"standard": 0.326,
"mac": 0.743
}
},
{
"ip": "117.34.114.27",
@@ -404,14 +410,14 @@
],
"handshake": null,
"status": "ok",
"latency": 0.482,
"checked_at": "2026-09-21 19:03:05",
"latency": 0.394,
"checked_at": "2026-10-16 13:46:18",
"source": "probe-2026-08-12",
"fast_pool": false,
"checked_date": "2026-09-21",
"checked_date": "2026-10-16",
"_per_proto": {
"standard": 0.482,
"mac": 0.579
"standard": 0.394,
"mac": 0.78
}
},
{
@@ -423,14 +429,14 @@
],
"handshake": null,
"status": "ok",
"latency": 0.609,
"checked_at": "2026-09-21 19:03:05",
"latency": 0.327,
"checked_at": "2026-10-16 13:46:18",
"source": "probe-2026-08-12",
"fast_pool": false,
"checked_date": "2026-09-21",
"checked_date": "2026-10-16",
"_per_proto": {
"standard": 0.946,
"mac": 0.609
"standard": 0.327,
"mac": 0.328
}
},
{
@@ -442,14 +448,14 @@
],
"handshake": null,
"status": "ok",
"latency": 0.46,
"checked_at": "2026-09-21 19:03:05",
"latency": 0.348,
"checked_at": "2026-10-16 13:46:18",
"source": "probe-2026-08-12",
"fast_pool": false,
"checked_date": "2026-09-21",
"checked_date": "2026-10-16",
"_per_proto": {
"standard": 0.46,
"mac": 0.521
"standard": 0.348,
"mac": 0.74
}
},
{
@@ -461,9 +467,9 @@
"handshake": null,
"status": "fail",
"latency": null,
"checked_at": "2026-09-21 19:03:24",
"checked_at": "2026-10-16 13:46:37",
"source": "probe-2026-08-12",
"checked_date": "2026-09-21",
"checked_date": "2026-10-16",
"fast_pool": false,
"_per_proto": {}
},
@@ -475,13 +481,13 @@
],
"handshake": null,
"status": "ok",
"latency": 0.271,
"checked_at": "2026-09-21 19:03:04",
"latency": 0.726,
"checked_at": "2026-10-16 13:46:18",
"source": "probe-2026-08-12",
"checked_date": "2026-09-21",
"fast_pool": true,
"checked_date": "2026-10-16",
"fast_pool": false,
"_per_proto": {
"mac": 0.271
"mac": 0.726
}
},
{
@@ -492,13 +498,13 @@
],
"handshake": null,
"status": "ok",
"latency": 0.375,
"checked_at": "2026-09-21 19:03:05",
"latency": 0.624,
"checked_at": "2026-10-16 13:46:18",
"source": "probe-2026-08-12",
"checked_date": "2026-09-21",
"checked_date": "2026-10-16",
"fast_pool": false,
"_per_proto": {
"mac": 0.375
"mac": 0.624
}
},
{
@@ -509,13 +515,13 @@
],
"handshake": null,
"status": "ok",
"latency": 0.424,
"checked_at": "2026-09-21 19:03:05",
"latency": 0.56,
"checked_at": "2026-10-16 13:46:30",
"source": "probe-2026-08-12",
"checked_date": "2026-09-21",
"checked_date": "2026-10-16",
"fast_pool": false,
"_per_proto": {
"mac": 0.424
"mac": 0.56
}
},
{
@@ -526,13 +532,13 @@
],
"handshake": "login",
"status": "ok",
"latency": 0.176,
"checked_at": "2026-09-21 19:03:08",
"latency": 0.097,
"checked_at": "2026-10-16 13:46:21",
"source": "probe-2026-08-12",
"checked_date": "2026-09-21",
"checked_date": "2026-10-16",
"fast_pool": true,
"_per_proto": {
"ex": 0.176
"ex": 0.097
}
},
{
@@ -543,13 +549,13 @@
],
"handshake": "login",
"status": "ok",
"latency": 0.148,
"checked_at": "2026-09-21 19:03:08",
"latency": 0.097,
"checked_at": "2026-10-16 13:46:21",
"source": "probe-2026-08-12",
"checked_date": "2026-09-21",
"checked_date": "2026-10-16",
"fast_pool": true,
"_per_proto": {
"ex": 0.148
"ex": 0.097
}
},
{
@@ -560,15 +566,66 @@
],
"handshake": "setup",
"status": "ok",
"latency": 0.166,
"checked_at": "2026-09-21 19:03:08",
"latency": 0.103,
"checked_at": "2026-10-16 13:46:21",
"source": "probe-2026-08-12",
"checked_date": "2026-09-21",
"checked_date": "2026-10-16",
"fast_pool": true,
"_per_proto": {
"ex": 0.166
"ex": 0.103
}
},
{
"ip": "112.74.214.43",
"name": "扩展市场深圳双线1",
"protocols": [
"ex"
],
"handshake": "login",
"status": "ok",
"latency": 0.095,
"checked_at": "2026-10-16 13:46:21",
"source": "connect.cfg",
"fast_pool": true,
"checked_date": "2026-10-16",
"_per_proto": {
"ex": 0.095
}
},
{
"ip": "120.25.218.6",
"name": "扩展市场深圳双线2",
"protocols": [
"ex"
],
"handshake": "login",
"status": "ok",
"latency": 0.101,
"checked_at": "2026-10-16 13:46:21",
"source": "connect.cfg",
"fast_pool": true,
"checked_date": "2026-10-16",
"_per_proto": {
"ex": 0.101
}
},
{
"ip": "47.102.108.214",
"name": "扩展市场上海双线2",
"protocols": [
"ex"
],
"handshake": "login",
"status": "ok",
"latency": 0.149,
"checked_at": "2026-10-16 13:46:22",
"source": "connect.cfg",
"fast_pool": true,
"checked_date": "2026-10-16",
"_per_proto": {
"ex": 0.149
}
}
],
"checked_date": "2026-09-21"
"checked_date": "2026-10-16"
}
+116
View File
@@ -0,0 +1,116 @@
"""ExPool 背压/并发语义回归(Issue #1:并发超限 0.15s 秒失败)。
用 mock 连接模拟:单服务器 + per_server_cap=1 时,
- 并发 acquire 超过有效并发 → 排队等待而不是快速失败;
- release 唤醒等待者;
- acquire 超时异常带池状态;
- ex_pool_cap 提升同主机并发。
"""
from __future__ import annotations
import asyncio
import time
import pytest
from pytdxdata.pool.ex_pool import ExPool, ExPoolTimeoutError
class MockConn:
"""Minimal stand-in for ExTdxConnection."""
last_created = 0
def __init__(self, host: str, port: int, timeout: float, handshake: str = "login"):
self.host = host
self.port = port
self.alive = True
self.last_used = time.monotonic()
self.closed = False
MockConn.last_created += 1
async def connect(self) -> None:
await asyncio.sleep(0) # instant "connect"
async def close(self) -> None:
self.closed = True
self.alive = False
@pytest.fixture
def patch_conn(monkeypatch):
monkeypatch.setattr("pytdxdata.pool.ex_pool.ExTdxConnection", MockConn)
MockConn.last_created = 0
async def test_acquire_queues_instead_of_fast_fail(patch_conn):
"""并发 > 有效并发时,后续 acquire 等待 release,而非 0.15s 抛错。"""
pool = ExPool(["e1"], min_size=1, max_size=6, per_server_cap=1)
await pool.start()
c1 = await pool.acquire() # 唯一连接被占用
t0 = time.monotonic()
async def holder():
await asyncio.sleep(0.5) # 持有 0.5s 后归还
await pool.release(c1)
asyncio.create_task(holder())
c2 = await pool.acquire() # 应在 ~0.5s 后拿到,而不是 0.15s 抛错
waited = time.monotonic() - t0
assert waited >= 0.4, f"acquire 没有等待释放(仅 {waited:.2f}s)"
assert c2 is c1 # 归还的是同一条连接
async def test_acquire_timeout_carries_pool_state(patch_conn):
"""超时异常信息带池状态(idle/total/servers/cap)。"""
pool = ExPool(["e1"], min_size=1, max_size=6, per_server_cap=1, acquire_timeout=0.3)
await pool.start()
c1 = await pool.acquire()
with pytest.raises(ExPoolTimeoutError) as ei:
await pool.acquire()
msg = str(ei.value)
assert "total=1" in msg and "per_server_cap=1" in msg and "servers=1" in msg
await pool.release(c1)
async def test_ex_pool_cap_allows_parallel_conns_same_host(patch_conn):
"""per_server_cap=2 时单主机可并发持有 2 条连接。"""
pool = ExPool(["e1"], min_size=1, max_size=6, per_server_cap=2)
await pool.start()
got = await asyncio.gather(pool.acquire(), pool.acquire())
assert len(got) == 2 and got[0] is not got[1]
assert MockConn.last_created == 2
for c in got:
await pool.release(c)
async def test_concurrent_acquire_all_succeed(patch_conn):
"""Issue 场景:单服务器 + cap=2,6 个并发任务应全部拿到连接(排队)而非部分失败。"""
pool = ExPool(["e1"], min_size=2, max_size=6, per_server_cap=2)
await pool.start()
async def task(i):
c = await pool.acquire()
await asyncio.sleep(0.1) # 模拟一次 1m 分页拉取
await pool.release(c)
return i
results = await asyncio.gather(*[task(i) for i in range(6)])
assert sorted(results) == list(range(6)) # 全部成功,零失败
async def test_release_notifies_waiters(patch_conn):
"""broken 归还(连接死亡)也要唤醒等待者(槽位回收后可扩容)。"""
pool = ExPool(["e1"], min_size=1, max_size=1, per_server_cap=1)
await pool.start()
c1 = await pool.acquire()
c1.alive = False
async def broken_release():
await asyncio.sleep(0.2)
await pool.release(c1, broken=True)
asyncio.create_task(broken_release())
c2 = await pool.acquire() # 槽位被回收→扩容新建
assert c2 is not c1
await pool.release(c2)
+114 -4
View File
@@ -2,17 +2,20 @@
from __future__ import annotations
import struct
from collections import Counter
from datetime import date, datetime
import pytest
from pytdxdata.protocol.frame import FrameHeader, decompress_body, parse_header
from pytdxdata.protocol.price import get_price, put_price, decode_ohlc_diffs
from pytdxdata.protocol.volume import decode_volume
from pytdxdata.models import Adjust, KlinePeriod, SecurityBar
from pytdxdata.protocol.commands.ex_proto import GetExInstrumentBarsCmd
from pytdxdata.protocol.commands.kline import GetSecurityBarsCmd
from pytdxdata.protocol.commands.mac import MacKlineCmd
from pytdxdata.protocol.commands.minute import GetHistoryMinuteTimeDataCmd
from pytdxdata.protocol.commands.transaction import MacTransactionCmd
from pytdxdata.protocol.frame import FrameHeader, decompress_body, parse_header
from pytdxdata.protocol.price import decode_ohlc_diffs, get_price, put_price
from pytdxdata.protocol.volume import decode_volume
# ---------- 变长有符号整数 ----------
@@ -176,3 +179,110 @@ def test_mac_transaction_parse():
assert recs[0].trade_count == 3
assert recs[0].bs_flag == 1
assert recs[1].trade_count == 1
# ---------- 跨时区时间戳归一化(美股 EX 通道) ----------
#
# 通达信 EX 通道对美股返回的时间戳把两个时区拼在一起:日期字段是**美东交易日**、
# 时分字段是**北京时间**。直拼出的 datetime 会把同一交易日的下午半场(北京
# 00:00~04:00)排到上午半场(北京 21:30~23:59)之前,单日 390 根全部错位。
def _bar(market: int, y: int, mo: int, d: int, h: int, mi: int) -> SecurityBar:
return SecurityBar(market=market, code="X", open=1.0, high=1.0, low=1.0,
close=1.0, vol=1.0, amount=1.0,
year=y, month=mo, day=d, hour=h, minute=mi)
def test_us_evening_segment_keeps_date():
# 北京 21:31 ~ 23:59 与美东交易日属同一自然日,日期不动
b = _bar(74, 2026, 10, 12, 21, 31)
assert b.datetime == datetime(2026, 10, 12, 21, 31)
assert b.trading_date == "2026-10-12"
def test_us_midnight_segment_rolls_to_next_day():
# 北京次日 00:00 ~ 04:00 实际是美东 12:00~16:00,日期要 +1 天
b = _bar(74, 2026, 10, 12, 4, 0)
assert b.datetime == datetime(2026, 10, 13, 4, 0)
assert (b.hour, b.minute) == (4, 0) # 时分不参与归一化
assert b.trading_date == "2026-10-12" # 仍归属美东 10-12 交易日
def test_us_split_hour_boundary():
assert _bar(74, 2026, 10, 12, 11, 59).day == 13
assert _bar(74, 2026, 10, 12, 12, 0).day == 12
def test_us_month_and_year_rollover():
assert _bar(74, 2026, 10, 31, 4, 0).datetime == datetime(2026, 11, 1, 4, 0)
assert _bar(74, 2026, 12, 31, 4, 0).datetime == datetime(2027, 1, 1, 4, 0)
def test_cn_and_hk_bars_untouched():
cn = _bar(1, 2026, 10, 12, 9, 31) # A股/标准通道
assert (cn.year, cn.month, cn.day) == (2026, 10, 12)
assert cn.trading_date == "2026-10-12"
hk = _bar(31, 2026, 10, 12, 4, 0) # 港股同为 UTC+8,凌晨段不应 +1
assert (hk.year, hk.month, hk.day) == (2026, 10, 12)
assert hk.trading_date == "2026-10-12"
def test_us_full_session_is_monotonic_and_single_trading_day():
# 实测口径:北京 21:31~23:59(149 根)+ 次日 00:00~04:00(241 根)= 390 根
mins = list(range(21 * 60 + 31, 24 * 60)) + list(range(4 * 60 + 1))
assert len(mins) == 390
bars = [_bar(74, 2026, 10, 12, m // 60, m % 60) for m in mins]
bars.sort(key=lambda b: b.datetime)
dts = [b.datetime for b in bars]
assert dts == sorted(dts) # 归一化后严格单调
assert dts[0] == datetime(2026, 10, 12, 21, 31)
assert dts[-1] == datetime(2026, 10, 13, 4, 0)
# 全部归属于同一个美东交易日
assert {b.trading_date for b in bars} == {"2026-10-12"}
# 但横跨两个北京自然日(这正是不能用 datetime.date() 聚合的原因)
by_day = Counter(b.datetime.date() for b in bars)
assert by_day[date(2026, 10, 12)] == 149
assert by_day[date(2026, 10, 13)] == 241
# ---------- MAC 通道 K线(0x122E)同样归一化 ----------
def _mac_bars_body(records: list[tuple[int, int]]) -> bytes:
"""构造 0x122E 响应体:33B头(count@24) + 每条36B(4B ymd + 4B time秒 + 7float)。"""
body = bytearray(33 + len(records) * 36)
struct.pack_into("<HBHI", body, 24, 7, 1, len(records), 0)
for i, (ymd, tsec) in enumerate(records):
struct.pack_into("<II7f", body, 33 + i * 36,
ymd, tsec, 10.0, 10.5, 9.5, 10.2, 0.0, 1000.0, 0.0)
return bytes(body)
def test_mac_kline_us_bar_normalized():
# _probe_bar(日期区间二分依赖它)对美股走的就是这条命令
cmd = MacKlineCmd(74, "AAPL", KlinePeriod.MIN_1, Adjust.NONE, 0, 5)
bars = cmd.parse_response(_mac_bars_body([(20261012, 4 * 3600)])) # 北京 04:00
assert len(bars) == 1
assert bars[0].datetime == datetime(2026, 10, 13, 4, 0)
assert bars[0].trading_date == "2026-10-12"
def test_mac_kline_hk_bar_untouched():
cmd = MacKlineCmd(31, "00700", KlinePeriod.MIN_1, Adjust.NONE, 0, 5)
bars = cmd.parse_response(_mac_bars_body([(20260729, 13 * 3600 + 40 * 60)]))
assert (bars[0].datetime) == datetime(2026, 7, 29, 13, 40)
assert bars[0].trading_date == "2026-07-29"
# ---------- 排序:EX 返回顺序按时间正序(切片取「最新 N 根」依赖它) ----------
def test_finish_bars_sorts_us_session_chronologically():
from pytdxdata.api import TdxData
from pytdxdata.symbols import parse_symbols
sym = parse_symbols("usAAPL")[0]
# 故意按错误的顺序给(凌晨段在前)
out = TdxData._finish_bars([_bar(74, 2026, 10, 12, 4, 0),
_bar(74, 2026, 10, 12, 21, 31)], sym)
assert [b.datetime for b in out] == [datetime(2026, 10, 12, 21, 31),
datetime(2026, 10, 13, 4, 0)]
assert {b.symbol for b in out} == {"usAAPL"}
Generated
+1 -1
View File
@@ -535,7 +535,7 @@ wheels = [
[[package]]
name = "pytdxdata"
version = "0.6.1"
version = "0.6.3"
source = { editable = "." }
[package.optional-dependencies]