9 Commits
Author SHA1 Message Date
openbotandopenbot-coder ca24521d2c chore: bump version 0.6.4 (#5)
Co-authored-by: openbot-coder <openbot-coder@users.noreply.github.com>
2026-10-22 09:11:50 +08:00
openbotandopenbot-coder 8344ed6e6b fix: 修复沪市指数价格少10倍 + 逐笔竞价委托污染(Σvol 虚高/跨主机不稳定) (#3)
核对 sh000016 上证50 与 50 只成分股的 tick 汇总时,发现三个口径
(快照/分时/逐笔)互不一致,追查到 5 个缺陷。

1. 沪市指数价格整体少 10 倍
   price_decimal_digits 把沪市 000xxx(上证/上证50/沪深300/科创50)与
   8xxxxx(板块指数)判为 3 位小数,实际协议是 2 位。
   上证指数 3713 -> 371、上证50 2747 -> 274。
   实证 6 只指数 + 2 只板块指数全部 ×10,而 ETF/股票/科创板/深市全类 ×1 正确。
   改法:沪市仅 ETF(5开头) 保留 3 位,其余 2 位。

2. get_ticks 把 bs_flag=8 竞价委托当成成交
   该记录是 09:15~09:25 / 14:57~15:00 竞价段的撮合委托快照,
   vol 是累计值(可被后续覆盖),不是成交增量。
   - Σvol 虚高:sh600000 含 b8 = 668,693 vs 快照 462,562(1.4456);
     剔 b8 = 462,634(1.000156,精确一致)
   - 跨主机不稳定:干净主机 0 条、脏主机 135 条 -> 同一 start 落到不同真实成交
   改法:分页后统一剔除 b8。

3. A股逐笔改单连接串行游标(_ticks_cn_serial)
   并发分页各页来自不同主机,而 0x122F 的 start 是「从最新往回数」的相对锚点,
   主机间 b8 收录差异导致页边界既重叠(冗余 116 条)又漏条(13:03 附近丢 59 条)。
   逐笔单日仅 5~6 页,串行开销可忽略。

4. trade_count 有符号解析(前序)、返回列表时间序(前序)保留。

- sh000016 三口径闭合:快照 vol/amount 比值 1.00000061/0.99999997;
  指数逐笔 Σvol = 分时 Σvol = 1,045,584,467(完全相等)
- 6 只标的 Σvol/快照 vol 全部回到 1.0001~1.0007
- 跨主机公共时间窗逐条一致(5/5 标的,条数差 0)
- 测试 194 -> 217 passed(+20 小数位、+3 逐笔回归)

- TransactionRecord 类注释 + get_ticks docstring 明示指数语义:
  vol 单位「百元」(成交额/100)、trade_count 为「均价×1000」而非笔数
- API_REFERENCE / llms.txt 同步

Co-authored-by: openbot-coder <openbot-coder@users.noreply.github.com>
2026-10-22 09:06:40 +08:00
openbotandopenbot-coder dfd6fab3af fix: 补齐 v0.6.3 遗漏的版本号(__init__.py / uv.lock) (#4)
## 问题
v0.6.3 的 bump 提交只改了 pyproject.toml 一处,漏掉:
- src/pytdxdata/__init__.py 的 __version__
- uv.lock 的 pytdxdata 条目版本

后果(远端 main 已复现):
- CI 两个 job 全部失败:`uv sync --locked` 报
  "The lockfile at `uv.lock` needs to be updated, but `--locked` was provided"
- Docs / Release 同样失败
- v0.6.3 tag 已打,但 PyPI 上 0.6.3 仍是 404(发布没成功)

## 修复
补齐两处版本号,四处对齐:pyproject.toml / __init__.py / uv.lock / CHANGELOG.md

## 验证
- `uv sync --locked` 退出码 0
- `pytdxdata.__version__` = `importlib.metadata.version()` = 0.6.3

Co-authored-by: openbot-coder <openbot-coder@users.noreply.github.com>
2026-10-22 08:52:19 +08:00
openbot 0618062e54 chore: bump version 0.6.3 2026-10-18 16:12:51 +08:00
openbot 9c97fea71b fix(pool): ExPool acquire backpressure + ex_pool_cap (fixes #1)
- acquire(): replace 3x0.05s fast-fail with Condition-based queueing
  (default acquire_timeout=30s); timeout error carries pool state
  (idle/total/servers/per_server_cap)
- api: expose ex_pool_cap (default 2, aligned with std/mac pool caps);
  single-server scenario no longer silently capped at 1 concurrent
2026-10-18 16:10:15 +08:00
openbot-coder 2d62d82b51 feat: 新增3台EX扩展行情服务器(connect.cfg官方扩展市场,handshake=login),全量复测30/36可用 2026-10-16 13:53:19 +08:00
openbot-coder 7e72f70eb2 chore: 离线探针复测写回 servers.json(2026-10-16,可达25→27,117.34.114.13/.14 复活) 2026-10-16 12:51:12 +08:00
openbotandopenbot-coder b2f899df49 fix(models): 修复美股分钟K线 datetime 时区拼接错位,还原为北京时间 (#2)
通达信 EX 通道对美股返回的分钟级 K线把两个时区拼进同一个五元组:日期分量是
美东交易日(zipday),时分分量是北京时间(tminutes)。直拼出的 datetime 会把
同一交易日的下午半场排到上午半场之前——单日 390 根首末颠倒,且全部落在同一个
北京自然日里,导致排序 / count 切片 / 日期区间二分与过滤 / 增量水位线全部不可用。

在 SecurityBar.__post_init__ 统一归一化为北京时间(凌晨段日期 +1 天,含月末与
年末进位),EX 的两条 K线命令路径(_bars_ex 的 0x23FF 与二分所依赖的 MAC 0x122E)
因此共用同一口径,不会出现"取数按一套时间、二分按另一套时间"的错位。

新增两个只读属性:trading_date(归属交易日;美股一个交易日横跨两个北京自然日,
按交易日聚合必须用它,不能用 datetime.date())与 is_us。

实测(安装 wheel 0.6.2,非源码树):10-06 ~ 10-12 五个完整交易日各 390 根、
严格单调,首根 open / 末根 close 与日线逐位相同,成交量差 <= 1;A股 / 港股零回归。

- 版本 0.6.1 -> 0.6.2(pyproject / __init__ / uv.lock)
- tests/test_protocol.py 新增 9 项回归测试(分界点 / 月末年末进位 / 390 根整日单调性 / MAC 路径),全量 191 passed
- CHANGELOG / README / SKILL.md / docs(faq、llms.txt、api/models)同步口径说明

遗留(本版未含,待决策):EX 分时对美股返回的 MinuteBar.hour 是未取模的连续分钟数
(实测可达 27,即次日 03:xx),且 MinuteBar 无日期分量可承载跨日信息。

Co-authored-by: openbot-coder <openbot-coder@users.noreply.github.com>
2026-10-13 16:47:57 +08:00
openbot-coder 8193d560da Revert "fix(models): 修复美股分钟K线 datetime 时区拼接错位,还原为北京时间"
This reverts commit 5408903a4c.
2026-10-13 16:42:41 +08:00
16 changed files with 900 additions and 214 deletions

No files matched your search

+86
View File
@@ -2,6 +2,92 @@
本项目遵循 [语义化版本](https://semver.org/lang/zh-CN/)。 本项目遵循 [语义化版本](https://semver.org/lang/zh-CN/)。
## [0.6.4] - 2026-10-22
### 修复
- **`get_ticks()` 返回列表不是时间序,每 `page_size` 处时间回跳一次**。
`0x122F`(A股逐笔)与 EX 逐笔通道的偏移量 `start` 都是**从最新一笔往回数**的:
实测 `sh600000`,`start=0` 返回 `14:07:48..15:20:16`(当天最新 1000 条),
`start=4000` 返回 `09:15:03..09:41:18`(当天最早)。而 `paginator` 按 `start`
升序拼接,于是拼出"新的块在前、旧的块在后"的倒序结构:
| | 修复前 | 修复后 |
|---|---|---|
| `sh600000` 时间轴 | 4 处逆序跳跃(index 999 / 1999 / 2999) | 严格升序 |
| `hk00700` 时间轴 | 21 处逆序跳跃(每 1800 条一次) | 严格升序 |
修复分两层:① `PageSpec.newest_first=True` 让 `paginator` 反页序拼装(页内本身升序,
只翻页序),这条能力保留给将来需要按"最新优先"锚点分页的命令;② A股逐笔最终改为
**单连接串行游标**(见下一条),从根上避开多主机锚点错位。
**`start` 参数语义同时明确**:它是"从最新往回数的偏移",不是"从最早往后数的偏移",
`docstring` 已补充。注意这修正的是**顺序**,数据本身无缺口、无重复(4304 条唯一时间戳)。
- **`TransactionRecord.trade_count` 在集合竞价段被读成 42 亿量级脏值**。
该字段在协议里是**有符号** `int32`,但解析格式串写的是无符号 `I`。集合竞价段
(`bs_flag=8`)真实值为负数,按 uint32 解出 `4294966422`:
```
原始 18 字节: 17820000 0ad71b41 d9000000 96fcffff 0800
修复前: trade_count = 4294966422
修复后: trade_count = -874 ← 0xfffffc96 的 int32 补码
```
实测 `sh600000` 全天 4304 条中有 **38 条**命中(占 0.88%),全部落在 09:15~09:25
集合竞价段;正常交易段(`bs_flag` 0/1/2/5)**无一条为负**。旧行为会让
`max()` / `mean()` 等统计直接失真(曾算出 `mean=37,920,247`)。
格式串由 `<IfIIH` 改为 `<IfIiH`。**注意负值语义尚未确证**(可能是撤单量/净委托),
目前只是如实保留符号,未做业务解释。
- **`get_quotes()` 沪市指数价格整体少 10 倍**。
`price_decimal_digits` 把沪市 `000`(如 000001/000016/000300/000688)与 `8`
开头(如板块指数 880001/880305)一律判成 3 位小数(除 1000),但协议原始整数
实际是 **2 位**(除 100)→ 上证指数 3713 被读成 371、上证50 2747 被读成 274。
实证(2026-10-21):沪市指数 6 只 + 板块 2 只全部 `分时中位价 / 快照价 = 10.0`,
而沪市 ETF(510300/588000/511010/501018)、股票(60/601)、科创板(688)、
深市指数/股票/创业板/ETF 全部 `= 1.0` 正确。修法是**把沪市 `000`/`8` 从 3 位改回 2 位**,
其余前缀(`5` ETF/基金、深市 `1` 基金转债)保持 3 位不变。回归测试覆盖 21 组代码。
- **`get_ticks()` 把集合竞价**委托**(`bs_flag=8`)当成成交计入,导致 Σvol 虚高且跨主机不稳定**。
`0x122F` 第 5 字段 `bs_flag=8` 的记录是 **09:15~09:25 / 14:57~15:00 竞价段的撮合委托
快照**,`vol` 是**累计值**(可被同段后续记录覆盖),根本不是成交增量。旧代码把它一并
返回,两个后果:
1. **Σvol 比真值虚高**:`sh600000` 全天 Σvol 含 b8 时 = 668,693,快照 vol = 462,562,
比值 1.4456;剔掉 b8 后 Σvol = 462,634,比值 **1.000156**(与快照精确一致)。
2. **跨主机不稳定**:干净主机不收录 b8(0 条)、脏主机收录 135 条。并发分页时各页
来自不同主机,同一 `start`(从最新往回数的相对锚点)在不同主机落到**不同的真实成交**
上 → 页边界既**重叠**(sh600000 冗余 116 条)又**漏条**(13:03 附近丢 59 条),
同一查询两次调用结果不同。
修复:① 分页拼装后**统一剔除 `bs_flag=8`**;② A股逐笔改用**单连接串行游标翻页**
(`_ticks_cn_serial`),彻底消除跨主机锚点错位。修复后单主机 1.0002、剔 b8 后跨主机
公共时间窗**逐条一致**。
- **文档化指数逐笔字段的量纲差异**(非 bug,是数据源固有语义)。
指数(如 `sh000016`)的 `vol` 单位是**百元**(成交额/100),`trade_count` 是**均价×1000**
(不是笔数),与个股完全不同。实测 `快照amount / Σtick.vol = 100.0002`、`tc/1000` 与分时
`avg_price` 差 < 1.1。已写进 `TransactionRecord` 类注释与 `get_ticks` docstring,
避免用户把指数 Σvol 直接和成分股成交量比(会误得 ~26 倍)。
### 说明
- 新增回归测试:价格小数位 ×20(19 组参数化 + 1 组去 NUL)、`get_ticks` 剔 b8 /
单连接串行 / 多页无重叠 ×3,测试总数 **194 → 217**。
## [0.6.3] - 2026-10-18
- fix(pool): ExPool acquire 背压等待(Condition 排队,默认 30s 超时)替代 3x0.05s 快速失败;超时异常带池状态
- feat(api): 新增 ex_pool_cap 参数(默认 2),单 EX 服务器场景不再静默退化为 1 并发(fixes #1)
## [0.6.2] - 2026-10-13 ## [0.6.2] - 2026-10-13
### 修复 ### 修复
+1 -1
View File
@@ -69,7 +69,7 @@ from pytdxdata import (
| 方法 | 返回 | 说明 | | 方法 | 返回 | 说明 |
|------|------|------| |------|------|------|
| `get_ticks(symbols, *, date=None, start=0, count=None, concurrency=6)` | `list[TransactionRecord]` | 跨市场逐笔成交 | | `get_ticks(symbols, *, date=None, start=0, count=None, concurrency=6)` | `list[TransactionRecord]` | 跨市场逐笔成交。返回按时间升序;`start` **从最新一笔往回数**;`vol`/`trade_count` 为约 3 秒切片的增量值(非累计)。**已剔除 `bs_flag=8` 竞价委托**(非成交,`vol` 为累计撮合量)。**指数**:`vol` 单位是百元(成交额/100)、`trade_count` 是均价×1000,均非股票语义 |
| `get_minutes(symbols, *, date=None, sampling=False, concurrency=6)` | `list[MinuteBar]` | 跨市场分时(sampling=True 取缩略采样:A股由 0x122D 派生,扩展市场 0x254D) | | `get_minutes(symbols, *, date=None, sampling=False, concurrency=6)` | `list[MinuteBar]` | 跨市场分时(sampling=True 取缩略采样:A股由 0x122D 派生,扩展市场 0x254D) |
#### 公司信息/文件/板块 #### 公司信息/文件/板块
+1 -1
View File
@@ -48,7 +48,7 @@ asyncio.run(main())
- **K 线**:`get_bars`(自动并发分页、支持复权与日期区间、多标的并发) - **K 线**:`get_bars`(自动并发分页、支持复权与日期区间、多标的并发)
- **技术指标**:`get_indicators`(K线 + MACD/KDJ/RSI/BOLL/MA/EMA) - **技术指标**:`get_indicators`(K线 + MACD/KDJ/RSI/BOLL/MA/EMA)
- **报价**:`get_quotes`(五档,>80 只自动切批)、`get_snapshot`(个股特征)、`get_market_stat` - **报价**:`get_quotes`(五档,>80 只自动切批)、`get_snapshot`(个股特征)、`get_market_stat`
- **逐笔 / 分时**:`get_ticks`(含 `trade_count`,支持历史日期)、`get_minutes`(`sampling=True` 取缩略采样) - **逐笔 / 分时**:`get_ticks`(含 `trade_count`,支持历史日期,返回按时间升序;`vol`/`trade_count` 为约 3 秒切片增量而非累计;已剔除 `bs_flag=8` 竞价委托;**指数**的 `vol` 是百元、`trade_count` 是均价×1000)、`get_minutes`(`sampling=True` 取缩略采样)
- **财务 / 除权**:`get_finance`(最新快照)、`get_xdxr`、`get_price_limits`(本地计算涨跌停) - **财务 / 除权**:`get_finance`(最新快照)、`get_xdxr`、`get_price_limits`(本地计算涨跌停)
- **F10**:`get_f10`(公司资料目录 + 正文) - **F10**:`get_f10`(公司资料目录 + 正文)
- **板块**:`get_boards`、`get_board_members`、`get_board_of`、`get_board_summary`、 - **板块**:`get_boards`、`get_board_members`、`get_board_of`、`get_board_summary`、
+1 -1
View File
@@ -1,6 +1,6 @@
[project] [project]
name = "pytdxdata" name = "pytdxdata"
version = "0.6.2" version = "0.6.4"
description = "通达信(TongDaXin)行情数据客户端:动态连接池 + 双层TTL缓存 + 3连接并发分页,纯asyncio零运行时依赖,支持A股/港股/美股/期货/期权" description = "通达信(TongDaXin)行情数据客户端:动态连接池 + 双层TTL缓存 + 3连接并发分页,纯asyncio零运行时依赖,支持A股/港股/美股/期货/期权"
readme = "README.md" readme = "README.md"
requires-python = ">=3.11" requires-python = ">=3.11"
+1 -1
View File
@@ -16,7 +16,7 @@ from .models import (
XdxrRecord, XdxrRecord,
) )
__version__ = "0.6.2" __version__ = "0.6.4"
__all__ = [ __all__ = [
"TdxData", "TdxData",
"Market", "Market",
+97 -22
View File
@@ -146,6 +146,8 @@ _EX_PAGE = 1000 # EX 商品列表单页
_TICK_PAGE = 1000 # MAC 逐笔单页 _TICK_PAGE = 1000 # MAC 逐笔单页
_EX_TICK_PAGE = 1800 # 港股逐笔单页 _EX_TICK_PAGE = 1800 # 港股逐笔单页
_ALL_PAGES_CAP = 50 # count=None(全量)时的翻页上限 _ALL_PAGES_CAP = 50 # count=None(全量)时的翻页上限
# 集合竞价**委托**记录标记(0x122F 第 5 字段)。不是成交,见 _ticks_one 注释。
_BS_AUCTION_ORDER = 8
# 港股系市场:逐笔走 EX 专用 16B 协议(0x23FC/0x2406),其余扩展市场走 MAC 0x122F # 港股系市场:逐笔走 EX 专用 16B 协议(0x23FC/0x2406),其余扩展市场走 MAC 0x122F
_HK_MARKETS = frozenset({27, 31, 48, 49, 71, 98}) _HK_MARKETS = frozenset({27, 31, 48, 49, 71, 98})
@@ -233,11 +235,13 @@ class TdxData:
pool_max: int = 12, pool_max: int = 12,
mac_pool_cap: int = 4, mac_pool_cap: int = 4,
std_pool_cap: int = 4, std_pool_cap: int = 4,
ex_pool_cap: int = 2,
) -> None: ) -> None:
self.port = port self.port = port
self.timeout = timeout self.timeout = timeout
self.default_adjust = default_adjust self.default_adjust = default_adjust
self.mac_pool_cap = mac_pool_cap self.mac_pool_cap = mac_pool_cap
self.ex_pool_cap = ex_pool_cap
self._std_servers = list(standard_servers or get_standard_hosts()) self._std_servers = list(standard_servers or get_standard_hosts())
self._mac_servers = list(mac_servers or get_mac_hosts()) self._mac_servers = list(mac_servers or get_mac_hosts())
self._ex_servers = list(ex_servers or get_ex_hosts()) self._ex_servers = list(ex_servers or get_ex_hosts())
@@ -262,8 +266,11 @@ class TdxData:
probe_factory=lambda: MacKlineCmd(1, "600000", KlinePeriod.DAY, Adjust.NONE, 0, 100), probe_factory=lambda: MacKlineCmd(1, "600000", KlinePeriod.DAY, Adjust.NONE, 0, 100),
) )
# EX扩展市场池(港股/期货/美股,端口7727;MAC EX login + pytdx EX setup 混合) # EX扩展市场池(港股/期货/美股,端口7727;MAC EX login + pytdx EX setup 混合)
# 单服务器场景(测速缓存收缩后仅 1 台)下,有效并发 = 服务器数 × ex_pool_cap,
# 默认 cap=2 保证至少 2 并发;acquire 有 Condition 背压等待,不再快速失败。
self._ex_pool = ExPool(self._ex_servers, timeout=timeout, self._ex_pool = ExPool(self._ex_servers, timeout=timeout,
min_size=min(pool_min, 2), max_size=min(pool_max, 6), min_size=min(pool_min, 2), max_size=min(pool_max, 6),
per_server_cap=ex_pool_cap,
handshake_map=EX_HANDSHAKES) handshake_map=EX_HANDSHAKES)
disk = None disk = None
if self._cache_dir is not None: if self._cache_dir is not None:
@@ -790,12 +797,26 @@ class TdxData:
参数: 参数:
date: ``None``=当日,``YYYYMMDD``=历史某日。 date: ``None``=当日,``YYYYMMDD``=历史某日。
start: 起始偏移。 start: 起始偏移,**从最新一笔往回数**(``0``=当天最后一条)。
``start=1000`` 表示跳过最近 1000 条、从更早的位置开始。
count: ``None``=尽量取全量(自动翻页,上限 50 页);给了就取这么多条。 count: ``None``=尽量取全量(自动翻页,上限 50 页);给了就取这么多条。
返回: 返回:
``list[TransactionRecord]``,按入参标的顺序拼接(A股含 ``trade_count``, ``list[TransactionRecord]``,按入参标的顺序拼接,**每条标的内部按时间升序**。
港股为 HH:MM 时间 + ``fields["zengcang"]`` 增仓)。 注意服务端是按约 3 秒聚合的切片,``vol``/``trade_count`` 是切片增量
而非当日累计。
A股含 ``trade_count``,港股为 HH:MM 时间 + ``fields["zengcang"]`` 增仓。
**集合竞价委托(``bs_flag=8``)不会出现在结果里**:它们不是成交,``vol``
是累计撮合量(会被后续记录覆盖),计入会让 Σvol 比真值虚高 20%~160%,
且不同主机收录的条数不同 → 同一查询两次调用结果不一致。
**指数(``sh000016`` 等)的两个字段语义与个股不同**:
- ``vol`` 单位是**百元**(成交额/100),不是手。
- ``trade_count`` 是**当时均价 × 1000**,不是笔数。
详见 :class:`~pytdxdata.models.TransactionRecord` 的类注释。
""" """
syms = parse_symbols(symbols) syms = parse_symbols(symbols)
sem = asyncio.Semaphore(max(1, concurrency)) sem = asyncio.Semaphore(max(1, concurrency))
@@ -816,18 +837,19 @@ class TdxData:
if sym.is_ex: if sym.is_ex:
rows = await self._ticks_ex(sym, date=date, start=start, count=count) rows = await self._ticks_ex(sym, date=date, start=start, count=count)
else: else:
def make_cmd(s: int, c: int): rows = await self._ticks_cn_serial(sym, date=date, start=start,
return MacTransactionCmd(sym.market, sym.code, date or 0, s, c) count=count)
# ★ 过滤集合竞价**委托**记录(bs_flag=8)。必须在逐页拉取**全部结束之后**做:
spec = PageSpec( # 分页靠 `len(rows) < page_size` 判终止,若在 parse_response 里过滤,
page_size=_TICK_PAGE, # 满页会被剥短成"短页" → 提前终止 → 丢数据。
total=None if count is None else start + count, # 实测(2026-10-21 sh600000):b8 是 09:15~09:25 / 14:57~15:00 竞价段的
min_page=100, # 撮合快照,vol 是**累计值**(可被后续记录覆盖),不是成交增量;
stop_when_short=True, # 计入后 Σvol 比真值虚高 20%~160%,且**不同主机的 b8 集合不同**
max_pages=_ALL_PAGES_CAP if count is None else None, # (干净主机 0 条、脏主机 135 条)→ 同一查询两次调用结果不一致。
start_offset=start, # 剔除后:Σvol 与快照 vol 精确相等(比值 1.0000),且跨主机逐条一致。
) rows = [r for r in rows if r.bs_flag != _BS_AUCTION_ORDER]
rows = await self._fetch_paginated_list("mac", make_cmd, spec) # 排序放在 pack 之前,让缓存直接存入正确顺序,避免每次命中都重排。
rows.sort(key=lambda r: r.time)
return pack_payload(rows) return pack_payload(rows)
records = unpack_payload(await self._cache.get_or_fetch(key, fetch)) records = unpack_payload(await self._cache.get_or_fetch(key, fetch))
@@ -835,6 +857,54 @@ class TdxData:
r.symbol = sym.text r.symbol = sym.text
return records return records
async def _ticks_cn_serial(self, sym: Symbol, *, date: int | None, start: int,
count: int | None) -> list[TransactionRecord]:
"""A股逐笔(0x122F)**单连接串行游标**翻页。
为什么不用并发分页器:``0x122F`` 的 ``start`` 是「从**最新一笔**往回数的偏移」,
是个**相对锚点**,不是稳定的绝对序号。而池里各主机对集合竞价委托(``bs_flag=8``)
的收录**不一致**(实测干净主机 0 条、脏主机 135 条),同一 ``start`` 在不同主机上
落到**不同的真实成交**上。并发分页 = 各页来自不同主机 → 页边界既**重叠**
(sh600000 冗余 116 条)又**漏条**(13:03 附近丢 59 条),Σvol 偏移 −1.2%。
逐笔单日仅 5~6 页(1000 条/页),串行开销可忽略、换来精确一致,
故这里固定**单连接**、按 ``start`` 递增(= 向更早方向)推进。
返回:按页序累加后再**整体翻转**,得到时间升序的完整列表。
"""
chunks: list[list[TransactionRecord]] = []
pos = start
remaining = count
pc = await self._mac_pool.acquire()
try:
for _ in range(_ALL_PAGES_CAP):
n = _TICK_PAGE if remaining is None else min(_TICK_PAGE, remaining)
if n <= 0:
break
rows = await pc.execute(
MacTransactionCmd(sym.market, sym.code, date or 0, pos, n))
if not rows:
break
chunks.append(list(rows))
pos += len(rows)
if remaining is not None:
remaining -= len(rows)
if remaining <= 0:
break
# 短页 = 已到最早一笔
if len(rows) < n:
break
except Exception:
await self._mac_pool.release(pc, broken=True)
raise
await self._mac_pool.release(pc)
out: list[TransactionRecord] = []
for chunk in reversed(chunks):
out.extend(chunk)
for r in out:
r.symbol = sym.text
return out
async def _ticks_ex(self, sym: Symbol, *, date: int | None, start: int, async def _ticks_ex(self, sym: Symbol, *, date: int | None, start: int,
count: int | None) -> list[TransactionRecord]: count: int | None) -> list[TransactionRecord]:
from .protocol.commands.ex_ext import ( from .protocol.commands.ex_ext import (
@@ -843,7 +913,7 @@ class TdxData:
) )
hk = sym.market in _HK_MARKETS hk = sym.market in _HK_MARKETS
page = _EX_TICK_PAGE if hk else _TICK_PAGE page = _EX_TICK_PAGE if hk else _TICK_PAGE
out: list[TransactionRecord] = [] chunks: list[list[TransactionRecord]] = []
pos = start pos = start
remaining = count remaining = count
for _ in range(_ALL_PAGES_CAP): for _ in range(_ALL_PAGES_CAP):
@@ -855,22 +925,27 @@ class TdxData:
if date else if date else
GetExTransactionDataCmd(sym.market, sym.code, pos, n)) GetExTransactionDataCmd(sym.market, sym.code, pos, n))
rows = await self._execute_ex(cmd) rows = await self._execute_ex(cmd)
out.extend(self._ex_tick(sym, r) for r in rows) chunk = [self._ex_tick(sym, r) for r in rows]
else: else:
cmd = MacTransactionCmd(sym.market, sym.code, date or 0, pos, n) cmd = MacTransactionCmd(sym.market, sym.code, date or 0, pos, n)
rows = await self._execute_ex(cmd) rows = await self._execute_ex(cmd)
for r in rows: for r in rows:
r.symbol = sym.text r.symbol = sym.text
out.extend(rows) chunk = list(rows)
if not rows: if not chunk:
break break
pos += len(rows) # EX 通道的 pos 同样是"从最新往回数",逐页倒序累加 → 最后整体翻转。
chunks.append(chunk)
pos += len(chunk)
if remaining is not None: if remaining is not None:
remaining -= len(rows) remaining -= len(chunk)
if remaining <= 0: if remaining <= 0:
break break
if len(rows) < n: if len(chunk) < n:
break break
out: list[TransactionRecord] = []
for chunk in reversed(chunks):
out.extend(chunk)
return out return out
@staticmethod @staticmethod
+24 -4
View File
@@ -228,14 +228,34 @@ class SecurityInfo:
@dataclass(slots=True) @dataclass(slots=True)
class TransactionRecord: class TransactionRecord:
"""逐笔成交(跨市场统一模型;A股 MAC 通道含 trade_count=成交笔数)""" """逐笔成交(跨市场统一模型;A股 MAC 通道含 trade_count=成交笔数)。
注意(实测 2026-10-21):
- 服务端是按**约 3 秒聚合**返回的切片,不是交易所原始逐笔成交。
- ``vol`` / ``trade_count`` 都是**该切片自身的增量**,不是当日累计值。
- ``trade_count`` 为**有符号**值:集合竞价段(``bs_flag=8``)会是负数
(实测 sh600000 09:15 段为 -874),不要对全表直接求 max/mean。
- 返回列表已按时间升序,但同秒多条时顺序不保证。
- ``bs_flag=8`` 的**竞价委托记录已被 ``get_ticks()`` 过滤掉**,不会出现在结果里。
**指数语义完全不同**(``is_cn_index`` 为真时,如 ``sh000016``):
- ``vol`` 单位是**百元**(= 成交额 / 100),不是「手」。实测
``快照amount / Σtick.vol = 100.0002``,且 ``Σtick.vol`` 与分时 Σvol **逐位相等**。
想和成分股成交量对比必须先换算,直接比会得到 ~26 倍的假比例。
- ``trade_count`` **不是笔数**,而是**当时均价 × 1000**(实测 tc/1000 与分时
``avg_price`` 的差 < 1.1)。指数没有「成交笔数」概念。
- ``bs_flag`` 语义不成立(实测返回 2,而模型只定义 0=卖 / 1=买)。
"""
market: int market: int
code: str code: str
time: str # HH:MM:SS(扩展市场为 HH:MM) time: str # HH:MM:SS(扩展市场为 HH:MM)
price: float price: float
vol: float # 手(MAC逐笔) vol: float # 股票=手;指数=百元(成交额/100),见类注释
trade_count: int # 成交笔数(MAC=逐笔记录字段;标准当日逐笔协议也有但需单独解析) trade_count: int # 股票=笔数增量;指数=均价×1000,见类注释
bs_flag: int # 1=主动买 0=主动卖 bs_flag: int # 1=主动买 0=主动卖 2=中性 5=盘后固定价(指数语义不成立)
symbol: str = "" symbol: str = ""
fields: dict[str, float] = field(default_factory=dict) # 扩展市场额外字段(增仓等) fields: dict[str, float] = field(default_factory=dict) # 扩展市场额外字段(增仓等)
+81 -32
View File
@@ -1,4 +1,11 @@
"""EX扩展市场连接池(简化版:固定连接数,借还+坏连接重建+login自动)""" """EX扩展市场连接池(固定连接数,借还+坏连接重建+login自动)。
与标准池 pool.py 的背压语义对齐:acquire 在无空闲连接且无法扩容时,
基于 asyncio.Condition 排队等待 release 唤醒(默认 acquire_timeout=30s),
而不是快速失败。per_server_cap 限制单台服务器连接数;单服务器场景下
有效并发 = servers × per_server_cap(默认 1),通过 TdxData(ex_pool_cap=N)
可对同一 EX 主机开多连接。
"""
from __future__ import annotations from __future__ import annotations
import asyncio import asyncio
@@ -9,6 +16,23 @@ from .ex_connection import ExTdxConnection
log = logging.getLogger(__name__) log = logging.getLogger(__name__)
DEFAULT_ACQUIRE_TIMEOUT = 30.0
class ExPoolTimeoutError(ConnectionError):
"""acquire 等待超时(携带池状态快照,便于排障)。"""
def __init__(self, pool: "ExPool", timeout: float) -> None:
idle = len(pool._idle)
total = pool._total
servers = len(pool.servers)
cap = pool.per_server_cap
super().__init__(
f"EX连接池获取超时(等了 {timeout:.1f}s):"
f"idle={idle} total={total} servers={servers} per_server_cap={cap} "
f"有效并发={servers * cap}"
)
class ExPool: class ExPool:
"""EX通道连接池(港股/期货,端口7727,连接时自动login)。""" """EX通道连接池(港股/期货,端口7727,连接时自动login)。"""
@@ -16,6 +40,7 @@ class ExPool:
def __init__(self, servers: list[str], *, timeout: float = 10.0, def __init__(self, servers: list[str], *, timeout: float = 10.0,
min_size: int = 2, max_size: int = 6, min_size: int = 2, max_size: int = 6,
per_server_cap: int = 1, per_server_cap: int = 1,
acquire_timeout: float | None = None,
handshake: str = "login", handshake: str = "login",
handshake_map: dict | None = None) -> None: handshake_map: dict | None = None) -> None:
self.servers = servers self.servers = servers
@@ -25,14 +50,17 @@ class ExPool:
self.min_size = min_size self.min_size = min_size
self.max_size = max_size self.max_size = max_size
self.per_server_cap = max(1, per_server_cap) # 一台服务器最多几个连接(默认1=单连接) self.per_server_cap = max(1, per_server_cap) # 一台服务器最多几个连接(默认1=单连接)
self.acquire_timeout = (
DEFAULT_ACQUIRE_TIMEOUT if acquire_timeout is None else acquire_timeout
)
self._idle: list[tuple[str, ExTdxConnection]] = [] self._idle: list[tuple[str, ExTdxConnection]] = []
self._server_count: dict[str, int] = {} self._server_count: dict[str, int] = {}
self._total = 0 self._total = 0
self._closed = False self._closed = False
self._lock = asyncio.Lock() self._cond = asyncio.Condition()
async def start(self) -> None: async def start(self) -> None:
async with self._lock: async with self._cond:
for host in self.servers[: self.min_size]: for host in self.servers[: self.min_size]:
if self._server_count.get(host, 0) >= self.per_server_cap: if self._server_count.get(host, 0) >= self.per_server_cap:
continue continue
@@ -47,7 +75,7 @@ class ExPool:
async def close(self) -> None: async def close(self) -> None:
self._closed = True self._closed = True
async with self._lock: async with self._cond:
for _, conn in self._idle: for _, conn in self._idle:
try: try:
await conn.close() await conn.close()
@@ -56,48 +84,61 @@ class ExPool:
self._idle.clear() self._idle.clear()
self._total = 0 self._total = 0
self._server_count.clear() self._server_count.clear()
self._cond.notify_all()
async def acquire(self) -> ExTdxConnection: async def acquire(self, timeout: float | None = None) -> ExTdxConnection:
for _ in range(3): """借出一条连接;无空闲且无法扩容时排队等待 release 唤醒。
stale_conn = None
new_conn = None timeout 默认取 self.acquire_timeout(30s);传 None 表示无限等待。
new_host = None """
async with self._lock: if timeout is None:
timeout = self.acquire_timeout
deadline = None if timeout is None else time.monotonic() + timeout
async with self._cond:
while True:
# 1. 优先借用空闲连接(过期则丢弃,槽位在锁内回收)
if self._idle: if self._idle:
_, conn = self._idle.pop() _, conn = self._idle.pop()
fresh = time.monotonic() - conn.last_used < 30.0 fresh = time.monotonic() - conn.last_used < 30.0
if conn.alive and fresh: if conn.alive and fresh:
return conn return conn
# 连接空闲过久(login会话可能失效)→ 丢弃重连 # 连接空闲过久(login会话可能失效)→ 丢弃,槽位让给扩容
self._total = max(0, self._total - 1) self._total = max(0, self._total - 1)
self._server_count[conn.host] = max(0, self._server_count.get(conn.host, 0) - 1) self._server_count[conn.host] = max(0, self._server_count.get(conn.host, 0) - 1)
stale_conn = conn self._schedule_close(conn)
# 扩容:在锁内预留槽位,锁外执行网络IO
elif self._total < self.max_size and self.servers: # 2. 可扩容 → 锁内预留槽位,锁外建连
new_conn = None
new_host = None
if self._total < self.max_size and self.servers:
host = self._next_available_server() host = self._next_available_server()
if host: if host is not None:
self._total += 1 self._total += 1
self._server_count[host] = self._server_count.get(host, 0) + 1 self._server_count[host] = self._server_count.get(host, 0) + 1
new_conn = ExTdxConnection(host, 7727, self.timeout, self.handshake_map.get(host, self.handshake)) new_conn = ExTdxConnection(host, 7727, self.timeout, self.handshake_map.get(host, self.handshake))
new_host = host new_host = host
# 锁外:关闭过期连接、建立新连接
if stale_conn is not None:
try:
await stale_conn.close()
except Exception:
pass
if new_conn is not None: if new_conn is not None:
# 锁外网络 IO:连接失败回滚槽位后继续等待
try: try:
await new_conn.connect() await new_conn.connect()
return new_conn return new_conn
except Exception as e: except Exception as e:
log.warning("EX扩容失败 %s: %s", new_host, str(e)[:60]) log.warning("EX扩容失败 %s: %s", new_host, str(e)[:60])
# 连接失败,回滚预留槽位 async with self._cond:
async with self._lock:
self._total = max(0, self._total - 1) self._total = max(0, self._total - 1)
self._server_count[new_host] = max(0, self._server_count.get(new_host, 0) - 1) self._server_count[new_host] = max(0, self._server_count.get(new_host, 0) - 1)
await asyncio.sleep(0.05)
raise ConnectionError("EX连接池无可用连接") # 3. 池满:排队等待 release 唤醒(或超时)
now = time.monotonic()
if deadline is not None and now >= deadline:
raise ExPoolTimeoutError(self, timeout)
try:
if deadline is None:
await self._cond.wait()
else:
await asyncio.wait_for(self._cond.wait(), timeout=deadline - now)
except asyncio.TimeoutError:
raise ExPoolTimeoutError(self, timeout)
def _next_available_server(self) -> str | None: def _next_available_server(self) -> str | None:
for h in self.servers: for h in self.servers:
@@ -105,15 +146,23 @@ class ExPool:
return h return h
return None return None
async def release(self, conn: ExTdxConnection, *, broken: bool = False) -> None: def _schedule_close(self, conn: ExTdxConnection) -> None:
if broken or not conn.alive: async def _close():
async with self._lock:
self._total = max(0, self._total - 1)
self._server_count[conn.host] = max(0, self._server_count.get(conn.host, 0) - 1)
try: try:
await conn.close() await conn.close()
except Exception: except Exception:
pass pass
return try:
async with self._lock: asyncio.get_running_loop().create_task(_close())
except RuntimeError:
pass
async def release(self, conn: ExTdxConnection, *, broken: bool = False) -> None:
async with self._cond:
if broken or not conn.alive:
self._total = max(0, self._total - 1)
self._server_count[conn.host] = max(0, self._server_count.get(conn.host, 0) - 1)
self._schedule_close(conn)
else:
self._idle.append((conn.host, conn)) self._idle.append((conn.host, conn))
self._cond.notify_all()
+13 -6
View File
@@ -13,16 +13,23 @@ MAX_QUOTES = 80
def price_decimal_digits(market: int, code: str) -> int: def price_decimal_digits(market: int, code: str) -> int:
"""推断报价有效小数位:股票2位(/100)、指数/ETF/基金/转债/国债3位(/1000)。""" """推断报价有效小数位:股票 2 位(/100)、ETF/基金 3 位(/1000)。
**沪市指数必须是 2 位**,此前把 ``000``/``8`` 一并判成 3 位,导致上证50、
上证指数、科创50 等**价格整体少 10 倍**。实证(2026-10-21):协议原始整数
与分时/逐笔价格比对,6 只沪市指数 + 2 只板块指数(880001/880305)全部是
``×10`` 错误,而 4 只沪市 ETF(510300/510500/588000/511010)、4 只沪市
股票、3 只科创板、3 只深市指数、深市股票/创业板/ETF 全部 ``×1`` 正确。
"""
code = (code or "").strip().rstrip("\x00") code = (code or "").strip().rstrip("\x00")
if market == 1: # SH if market == 1: # SH
if code.startswith(("5", "000", "8")): if code.startswith("5"):
return 3 return 3 # ETF/基金
return 2 return 2 # 指数(000xxx/880xxx)、股票(60xxxx/688xxx)
if market == 0: # SZ if market == 0: # SZ
if code.startswith("1"): if code.startswith("1"):
return 3 return 3 # 基金/债券
return 2 return 2 # 指数(399xxx)、股票(00xxxx/30xxxx)
return 2 return 2
@@ -111,8 +111,11 @@ class MacTransactionCmd(BaseCommand[list[TransactionRecord]]):
for i in range(count): for i in range(count):
offset = 39 + i * 18 offset = 39 + i * 18
try: try:
# trade_count 必须按**有符号** int32 解析:集合竞价段(bs_flag=8)
# 该字段实为负值,按 uint32 会读出 42 亿量级的脏数据。
# 实测(2026-10-21 sh600000):09:15:03 原始 96fcffff = -874。
time_sec, price, volume, trade_count, bs_flag = unpack_from( time_sec, price, volume, trade_count, bs_flag = unpack_from(
"<IfIIH", body, offset, "mac transaction item" "<IfIiH", body, offset, "mac transaction item"
) )
except ValueError: except ValueError:
break break
+7 -3
View File
@@ -27,6 +27,7 @@ class PageSpec:
ordered: bool = True # 按 start 升序拼接 ordered: bool = True # 按 start 升序拼接
min_page: int = 100 # 规避标准K线 count<100 空响应坑 min_page: int = 100 # 规避标准K线 count<100 空响应坑
start_offset: int = 0 # 起始偏移(从该偏移开始分页,避免拉取历史全量) start_offset: int = 0 # 起始偏移(从该偏移开始分页,避免拉取历史全量)
newest_first: bool = False # 服务端 start 从最新往回数(0x122F 逐笔即如此)
@dataclass(slots=True) @dataclass(slots=True)
@@ -94,7 +95,7 @@ class Paginator:
if s.pc is not None: if s.pc is not None:
pc, s.pc = s.pc, None pc, s.pc = s.pc, None
await self.pool.release(pc) await self.pool.release(pc)
return self._assemble(out, spec.ordered) return self._assemble(out, spec.ordered, spec.newest_first)
def _plan_starts(self, spec: PageSpec) -> list[tuple[int, int]]: def _plan_starts(self, spec: PageSpec) -> list[tuple[int, int]]:
"""切页:返回 [(start, count), ...]。""" """切页:返回 [(start, count), ...]。"""
@@ -158,9 +159,12 @@ class Paginator:
return None, False return None, False
return None, False return None, False
def _assemble(self, results: list[PageResult], ordered: bool) -> list[Any]: def _assemble(self, results: list[PageResult], ordered: bool,
newest_first: bool = False) -> list[Any]:
# newest_first=True:start=0 是最新一页,start 越大越旧 → 反向拼才对得上时间序。
# 单页内部服务端已按时间升序,所以只翻页序,不翻页内元素。
if ordered: if ordered:
results.sort(key=lambda r: r.start) results.sort(key=lambda r: r.start, reverse=newest_first)
out: list[Any] = [] out: list[Any] = []
for r in results: for r in results:
out.extend(r.items) out.extend(r.items)
+197 -140
View File
@@ -1,6 +1,6 @@
{ {
"version": "2026-09-21", "version": "2026-10-16",
"note": "离线探针复测(2026-09-21 19:03): 可达=25/33, 入池fast_pool=10台", "note": "离线探针复测(2026-10-16 13:46): 可达=30/36, 入池fast_pool=10台",
"servers": [ "servers": [
{ {
"ip": "139.9.52.158", "ip": "139.9.52.158",
@@ -11,14 +11,14 @@
], ],
"handshake": null, "handshake": null,
"status": "ok", "status": "ok",
"latency": 0.22, "latency": 0.172,
"checked_at": "2026-09-21 19:03:04", "checked_at": "2026-10-16 13:46:17",
"source": "probe-2026-08-12", "source": "probe-2026-08-12",
"fast_pool": true, "fast_pool": true,
"checked_date": "2026-09-21", "checked_date": "2026-10-16",
"_per_proto": { "_per_proto": {
"standard": 0.226, "standard": 0.172,
"mac": 0.22 "mac": 0.183
} }
}, },
{ {
@@ -30,14 +30,14 @@
], ],
"handshake": null, "handshake": null,
"status": "ok", "status": "ok",
"latency": 0.366, "latency": 0.298,
"checked_at": "2026-09-21 19:03:04", "checked_at": "2026-10-16 13:46:18",
"source": "probe-2026-08-12", "source": "probe-2026-08-12",
"fast_pool": false, "fast_pool": true,
"checked_date": "2026-09-21", "checked_date": "2026-10-16",
"_per_proto": { "_per_proto": {
"standard": 0.396, "standard": 0.316,
"mac": 0.366 "mac": 0.298
} }
}, },
{ {
@@ -48,13 +48,13 @@
], ],
"handshake": null, "handshake": null,
"status": "ok", "status": "ok",
"latency": 0.257, "latency": 0.236,
"checked_at": "2026-09-21 19:03:09", "checked_at": "2026-10-16 13:46:23",
"source": "probe-2026-08-12", "source": "probe-2026-08-12",
"fast_pool": true, "fast_pool": true,
"checked_date": "2026-09-21", "checked_date": "2026-10-16",
"_per_proto": { "_per_proto": {
"mac": 0.257 "mac": 0.236
} }
}, },
{ {
@@ -65,13 +65,13 @@
], ],
"handshake": null, "handshake": null,
"status": "ok", "status": "ok",
"latency": 0.248, "latency": 0.213,
"checked_at": "2026-09-21 19:03:04", "checked_at": "2026-10-16 13:46:17",
"source": "probe-2026-08-12", "source": "probe-2026-08-12",
"fast_pool": true, "fast_pool": true,
"checked_date": "2026-09-21", "checked_date": "2026-10-16",
"_per_proto": { "_per_proto": {
"mac": 0.248 "mac": 0.213
} }
}, },
{ {
@@ -84,15 +84,15 @@
], ],
"handshake": "setup", "handshake": "setup",
"status": "ok", "status": "ok",
"latency": 0.221, "latency": 0.316,
"checked_at": "2026-09-21 19:03:00", "checked_at": "2026-10-16 13:46:13",
"source": "probe-2026-08-12", "source": "probe-2026-08-12",
"fast_pool": true, "fast_pool": false,
"checked_date": "2026-09-21", "checked_date": "2026-10-16",
"_per_proto": { "_per_proto": {
"standard": 0.399, "standard": 0.316,
"mac": 0.414, "mac": 0.329,
"ex": 0.221 "ex": 0.539
} }
}, },
{ {
@@ -104,10 +104,10 @@
"handshake": null, "handshake": null,
"status": "fail", "status": "fail",
"latency": null, "latency": null,
"checked_at": "2026-09-21 19:03:20", "checked_at": "2026-10-16 13:46:34",
"source": "probe-2026-08-12", "source": "probe-2026-08-12",
"fast_pool": false, "fast_pool": false,
"checked_date": "2026-09-21", "checked_date": "2026-10-16",
"_per_proto": {} "_per_proto": {}
}, },
{ {
@@ -118,13 +118,13 @@
], ],
"handshake": null, "handshake": null,
"status": "ok", "status": "ok",
"latency": 0.363, "latency": 0.682,
"checked_at": "2026-09-21 19:03:10", "checked_at": "2026-10-16 13:46:24",
"source": "probe-2026-08-12", "source": "probe-2026-08-12",
"fast_pool": true, "fast_pool": false,
"checked_date": "2026-09-21", "checked_date": "2026-10-16",
"_per_proto": { "_per_proto": {
"mac": 0.363 "mac": 0.682
} }
}, },
{ {
@@ -136,10 +136,10 @@
"handshake": null, "handshake": null,
"status": "fail", "status": "fail",
"latency": null, "latency": null,
"checked_at": "2026-09-21 19:03:20", "checked_at": "2026-10-16 13:46:34",
"source": "probe-2026-08-12", "source": "probe-2026-08-12",
"fast_pool": false, "fast_pool": false,
"checked_date": "2026-09-21", "checked_date": "2026-10-16",
"_per_proto": {} "_per_proto": {}
}, },
{ {
@@ -150,13 +150,13 @@
], ],
"handshake": null, "handshake": null,
"status": "ok", "status": "ok",
"latency": 0.344, "latency": 0.607,
"checked_at": "2026-09-21 19:03:10", "checked_at": "2026-10-16 13:46:23",
"source": "probe-2026-08-12", "source": "probe-2026-08-12",
"fast_pool": true, "fast_pool": false,
"checked_date": "2026-09-21", "checked_date": "2026-10-16",
"_per_proto": { "_per_proto": {
"mac": 0.344 "mac": 0.607
} }
}, },
{ {
@@ -167,13 +167,13 @@
], ],
"handshake": null, "handshake": null,
"status": "ok", "status": "ok",
"latency": 0.404, "latency": 1.264,
"checked_at": "2026-09-21 19:03:10", "checked_at": "2026-10-16 13:46:24",
"source": "probe-2026-08-12", "source": "probe-2026-08-12",
"fast_pool": false, "fast_pool": false,
"checked_date": "2026-09-21", "checked_date": "2026-10-16",
"_per_proto": { "_per_proto": {
"mac": 0.404 "mac": 1.264
} }
}, },
{ {
@@ -185,10 +185,10 @@
"handshake": null, "handshake": null,
"status": "fail", "status": "fail",
"latency": null, "latency": null,
"checked_at": "2026-09-21 19:03:24", "checked_at": "2026-10-16 13:46:34",
"source": "probe-2026-08-12", "source": "probe-2026-08-12",
"fast_pool": false, "fast_pool": false,
"checked_date": "2026-09-21", "checked_date": "2026-10-16",
"_per_proto": {} "_per_proto": {}
}, },
{ {
@@ -200,10 +200,10 @@
"handshake": null, "handshake": null,
"status": "fail", "status": "fail",
"latency": null, "latency": null,
"checked_at": "2026-09-21 19:03:21", "checked_at": "2026-10-16 13:46:34",
"source": "probe-2026-08-12", "source": "probe-2026-08-12",
"fast_pool": false, "fast_pool": false,
"checked_date": "2026-09-21", "checked_date": "2026-10-16",
"_per_proto": {} "_per_proto": {}
}, },
{ {
@@ -214,13 +214,13 @@
], ],
"handshake": null, "handshake": null,
"status": "ok", "status": "ok",
"latency": 0.376, "latency": 0.612,
"checked_at": "2026-09-21 19:03:10", "checked_at": "2026-10-16 13:46:23",
"source": "probe-2026-08-12", "source": "probe-2026-08-12",
"fast_pool": false, "fast_pool": false,
"checked_date": "2026-09-21", "checked_date": "2026-10-16",
"_per_proto": { "_per_proto": {
"mac": 0.376 "mac": 0.612
} }
}, },
{ {
@@ -231,13 +231,13 @@
], ],
"handshake": null, "handshake": null,
"status": "ok", "status": "ok",
"latency": 0.377, "latency": 0.613,
"checked_at": "2026-09-21 19:03:10", "checked_at": "2026-10-16 13:46:23",
"source": "probe-2026-08-12", "source": "probe-2026-08-12",
"fast_pool": false, "fast_pool": false,
"checked_date": "2026-09-21", "checked_date": "2026-10-16",
"_per_proto": { "_per_proto": {
"mac": 0.377 "mac": 0.613
} }
}, },
{ {
@@ -248,13 +248,13 @@
], ],
"handshake": null, "handshake": null,
"status": "ok", "status": "ok",
"latency": 0.401, "latency": 0.624,
"checked_at": "2026-09-21 19:03:10", "checked_at": "2026-10-16 13:46:23",
"source": "probe-2026-08-12", "source": "probe-2026-08-12",
"fast_pool": false, "fast_pool": false,
"checked_date": "2026-09-21", "checked_date": "2026-10-16",
"_per_proto": { "_per_proto": {
"mac": 0.401 "mac": 0.624
} }
}, },
{ {
@@ -265,13 +265,13 @@
], ],
"handshake": null, "handshake": null,
"status": "ok", "status": "ok",
"latency": 0.42, "latency": 0.592,
"checked_at": "2026-09-21 19:03:10", "checked_at": "2026-10-16 13:46:23",
"source": "probe-2026-08-12", "source": "probe-2026-08-12",
"fast_pool": false, "fast_pool": false,
"checked_date": "2026-09-21", "checked_date": "2026-10-16",
"_per_proto": { "_per_proto": {
"mac": 0.42 "mac": 0.592
} }
}, },
{ {
@@ -282,13 +282,13 @@
], ],
"handshake": null, "handshake": null,
"status": "ok", "status": "ok",
"latency": 0.401, "latency": 0.624,
"checked_at": "2026-09-21 19:03:10", "checked_at": "2026-10-16 13:46:23",
"source": "probe-2026-08-12", "source": "probe-2026-08-12",
"fast_pool": false, "fast_pool": false,
"checked_date": "2026-09-21", "checked_date": "2026-10-16",
"_per_proto": { "_per_proto": {
"mac": 0.401 "mac": 0.624
} }
}, },
{ {
@@ -300,14 +300,14 @@
], ],
"handshake": null, "handshake": null,
"status": "ok", "status": "ok",
"latency": 0.485, "latency": 0.306,
"checked_at": "2026-09-21 19:03:05", "checked_at": "2026-10-16 13:46:18",
"source": "probe-2026-08-12", "source": "probe-2026-08-12",
"fast_pool": false, "fast_pool": false,
"checked_date": "2026-09-21", "checked_date": "2026-10-16",
"_per_proto": { "_per_proto": {
"standard": 0.485, "standard": 0.352,
"mac": 0.605 "mac": 0.306
} }
}, },
{ {
@@ -319,10 +319,10 @@
"handshake": null, "handshake": null,
"status": "fail", "status": "fail",
"latency": null, "latency": null,
"checked_at": "2026-09-21 19:03:04", "checked_at": "2026-10-16 13:46:18",
"source": "probe-2026-08-12", "source": "probe-2026-08-12",
"fast_pool": false, "fast_pool": false,
"checked_date": "2026-09-21", "checked_date": "2026-10-16",
"_per_proto": {} "_per_proto": {}
}, },
{ {
@@ -334,14 +334,14 @@
], ],
"handshake": null, "handshake": null,
"status": "ok", "status": "ok",
"latency": 0.444, "latency": 0.307,
"checked_at": "2026-09-21 19:03:05", "checked_at": "2026-10-16 13:46:18",
"source": "probe-2026-08-12", "source": "probe-2026-08-12",
"fast_pool": false, "fast_pool": false,
"checked_date": "2026-09-21", "checked_date": "2026-10-16",
"_per_proto": { "_per_proto": {
"standard": 0.444, "standard": 0.35,
"mac": 0.903 "mac": 0.307
} }
}, },
{ {
@@ -352,13 +352,16 @@
"mac" "mac"
], ],
"handshake": null, "handshake": null,
"status": "fail", "status": "ok",
"latency": null, "latency": 0.395,
"checked_at": "2026-09-21 19:03:15", "checked_at": "2026-10-16 13:46:18",
"source": "probe-2026-08-12", "source": "probe-2026-08-12",
"fast_pool": false, "fast_pool": false,
"checked_date": "2026-09-21", "checked_date": "2026-10-16",
"_per_proto": {} "_per_proto": {
"standard": 0.395,
"mac": 0.675
}
}, },
{ {
"ip": "117.34.114.15", "ip": "117.34.114.15",
@@ -369,14 +372,14 @@
], ],
"handshake": null, "handshake": null,
"status": "ok", "status": "ok",
"latency": 0.435, "latency": 0.345,
"checked_at": "2026-09-21 19:03:05", "checked_at": "2026-10-16 13:46:18",
"source": "probe-2026-08-12", "source": "probe-2026-08-12",
"fast_pool": false, "fast_pool": false,
"checked_date": "2026-09-21", "checked_date": "2026-10-16",
"_per_proto": { "_per_proto": {
"standard": 0.463, "standard": 0.345,
"mac": 0.435 "mac": 0.726
} }
}, },
{ {
@@ -387,13 +390,16 @@
"mac" "mac"
], ],
"handshake": null, "handshake": null,
"status": "fail", "status": "ok",
"latency": null, "latency": 0.326,
"checked_at": "2026-09-21 19:03:14", "checked_at": "2026-10-16 13:46:18",
"source": "probe-2026-08-12", "source": "probe-2026-08-12",
"fast_pool": false, "fast_pool": false,
"checked_date": "2026-09-21", "checked_date": "2026-10-16",
"_per_proto": {} "_per_proto": {
"standard": 0.326,
"mac": 0.743
}
}, },
{ {
"ip": "117.34.114.27", "ip": "117.34.114.27",
@@ -404,14 +410,14 @@
], ],
"handshake": null, "handshake": null,
"status": "ok", "status": "ok",
"latency": 0.482, "latency": 0.394,
"checked_at": "2026-09-21 19:03:05", "checked_at": "2026-10-16 13:46:18",
"source": "probe-2026-08-12", "source": "probe-2026-08-12",
"fast_pool": false, "fast_pool": false,
"checked_date": "2026-09-21", "checked_date": "2026-10-16",
"_per_proto": { "_per_proto": {
"standard": 0.482, "standard": 0.394,
"mac": 0.579 "mac": 0.78
} }
}, },
{ {
@@ -423,14 +429,14 @@
], ],
"handshake": null, "handshake": null,
"status": "ok", "status": "ok",
"latency": 0.609, "latency": 0.327,
"checked_at": "2026-09-21 19:03:05", "checked_at": "2026-10-16 13:46:18",
"source": "probe-2026-08-12", "source": "probe-2026-08-12",
"fast_pool": false, "fast_pool": false,
"checked_date": "2026-09-21", "checked_date": "2026-10-16",
"_per_proto": { "_per_proto": {
"standard": 0.946, "standard": 0.327,
"mac": 0.609 "mac": 0.328
} }
}, },
{ {
@@ -442,14 +448,14 @@
], ],
"handshake": null, "handshake": null,
"status": "ok", "status": "ok",
"latency": 0.46, "latency": 0.348,
"checked_at": "2026-09-21 19:03:05", "checked_at": "2026-10-16 13:46:18",
"source": "probe-2026-08-12", "source": "probe-2026-08-12",
"fast_pool": false, "fast_pool": false,
"checked_date": "2026-09-21", "checked_date": "2026-10-16",
"_per_proto": { "_per_proto": {
"standard": 0.46, "standard": 0.348,
"mac": 0.521 "mac": 0.74
} }
}, },
{ {
@@ -461,9 +467,9 @@
"handshake": null, "handshake": null,
"status": "fail", "status": "fail",
"latency": null, "latency": null,
"checked_at": "2026-09-21 19:03:24", "checked_at": "2026-10-16 13:46:37",
"source": "probe-2026-08-12", "source": "probe-2026-08-12",
"checked_date": "2026-09-21", "checked_date": "2026-10-16",
"fast_pool": false, "fast_pool": false,
"_per_proto": {} "_per_proto": {}
}, },
@@ -475,13 +481,13 @@
], ],
"handshake": null, "handshake": null,
"status": "ok", "status": "ok",
"latency": 0.271, "latency": 0.726,
"checked_at": "2026-09-21 19:03:04", "checked_at": "2026-10-16 13:46:18",
"source": "probe-2026-08-12", "source": "probe-2026-08-12",
"checked_date": "2026-09-21", "checked_date": "2026-10-16",
"fast_pool": true, "fast_pool": false,
"_per_proto": { "_per_proto": {
"mac": 0.271 "mac": 0.726
} }
}, },
{ {
@@ -492,13 +498,13 @@
], ],
"handshake": null, "handshake": null,
"status": "ok", "status": "ok",
"latency": 0.375, "latency": 0.624,
"checked_at": "2026-09-21 19:03:05", "checked_at": "2026-10-16 13:46:18",
"source": "probe-2026-08-12", "source": "probe-2026-08-12",
"checked_date": "2026-09-21", "checked_date": "2026-10-16",
"fast_pool": false, "fast_pool": false,
"_per_proto": { "_per_proto": {
"mac": 0.375 "mac": 0.624
} }
}, },
{ {
@@ -509,13 +515,13 @@
], ],
"handshake": null, "handshake": null,
"status": "ok", "status": "ok",
"latency": 0.424, "latency": 0.56,
"checked_at": "2026-09-21 19:03:05", "checked_at": "2026-10-16 13:46:30",
"source": "probe-2026-08-12", "source": "probe-2026-08-12",
"checked_date": "2026-09-21", "checked_date": "2026-10-16",
"fast_pool": false, "fast_pool": false,
"_per_proto": { "_per_proto": {
"mac": 0.424 "mac": 0.56
} }
}, },
{ {
@@ -526,13 +532,13 @@
], ],
"handshake": "login", "handshake": "login",
"status": "ok", "status": "ok",
"latency": 0.176, "latency": 0.097,
"checked_at": "2026-09-21 19:03:08", "checked_at": "2026-10-16 13:46:21",
"source": "probe-2026-08-12", "source": "probe-2026-08-12",
"checked_date": "2026-09-21", "checked_date": "2026-10-16",
"fast_pool": true, "fast_pool": true,
"_per_proto": { "_per_proto": {
"ex": 0.176 "ex": 0.097
} }
}, },
{ {
@@ -543,13 +549,13 @@
], ],
"handshake": "login", "handshake": "login",
"status": "ok", "status": "ok",
"latency": 0.148, "latency": 0.097,
"checked_at": "2026-09-21 19:03:08", "checked_at": "2026-10-16 13:46:21",
"source": "probe-2026-08-12", "source": "probe-2026-08-12",
"checked_date": "2026-09-21", "checked_date": "2026-10-16",
"fast_pool": true, "fast_pool": true,
"_per_proto": { "_per_proto": {
"ex": 0.148 "ex": 0.097
} }
}, },
{ {
@@ -560,15 +566,66 @@
], ],
"handshake": "setup", "handshake": "setup",
"status": "ok", "status": "ok",
"latency": 0.166, "latency": 0.103,
"checked_at": "2026-09-21 19:03:08", "checked_at": "2026-10-16 13:46:21",
"source": "probe-2026-08-12", "source": "probe-2026-08-12",
"checked_date": "2026-09-21", "checked_date": "2026-10-16",
"fast_pool": true, "fast_pool": true,
"_per_proto": { "_per_proto": {
"ex": 0.166 "ex": 0.103
}
},
{
"ip": "112.74.214.43",
"name": "扩展市场深圳双线1",
"protocols": [
"ex"
],
"handshake": "login",
"status": "ok",
"latency": 0.095,
"checked_at": "2026-10-16 13:46:21",
"source": "connect.cfg",
"fast_pool": true,
"checked_date": "2026-10-16",
"_per_proto": {
"ex": 0.095
}
},
{
"ip": "120.25.218.6",
"name": "扩展市场深圳双线2",
"protocols": [
"ex"
],
"handshake": "login",
"status": "ok",
"latency": 0.101,
"checked_at": "2026-10-16 13:46:21",
"source": "connect.cfg",
"fast_pool": true,
"checked_date": "2026-10-16",
"_per_proto": {
"ex": 0.101
}
},
{
"ip": "47.102.108.214",
"name": "扩展市场上海双线2",
"protocols": [
"ex"
],
"handshake": "login",
"status": "ok",
"latency": 0.149,
"checked_at": "2026-10-16 13:46:22",
"source": "connect.cfg",
"fast_pool": true,
"checked_date": "2026-10-16",
"_per_proto": {
"ex": 0.149
} }
} }
], ],
"checked_date": "2026-09-21" "checked_date": "2026-10-16"
} }
+211 -1
View File
@@ -8,7 +8,13 @@ import unittest.mock as mock
import pytest import pytest
from pytdxdata.api import TdxData from pytdxdata.api import TdxData
from pytdxdata.models import KlinePeriod, Market, MinuteBar, SecurityBar from pytdxdata.models import (
KlinePeriod,
Market,
MinuteBar,
SecurityBar,
TransactionRecord,
)
from pytdxdata.protocol.commands.mac_tick import MacTickChartCmd from pytdxdata.protocol.commands.mac_tick import MacTickChartCmd
from pytdxdata.scheduler.paginator import PageSpec, Paginator from pytdxdata.scheduler.paginator import PageSpec, Paginator
@@ -243,6 +249,70 @@ async def test_paginator_start_offset_default_zero():
assert result[0].close == 10.0 # 从 offset 0 开始 assert result[0].close == 10.0 # 从 offset 0 开始
# ---------- Paginator 逆序偏移(服务端 start 从最新往回数) ----------
class NewestFirstConn:
"""模拟 0x122F 真实行为:start=0 返回**最新**的一页,start 越大越旧。
实测依据(2026-10-21,sh600000):
start=0 -> 14:07:48 .. 15:20:16(当天最新 1000 条)
start=4000 -> 09:15:03 .. 09:41:18(当天最早)
每页内部按时间升序,页与页之间是倒序。
"""
def __init__(self, total: int, page_size: int):
self.total = total
self.page_size = page_size
self.executed: list[tuple[int, int]] = []
async def execute(self, cmd):
start, count = cmd
self.executed.append((start, count))
# 服务端语义:偏移 start 是从"最新"往回数,所以取倒数区间
end_from_oldest = self.total - start
begin_from_oldest = max(0, end_from_oldest - count)
return [_make_bar(i) for i in range(begin_from_oldest, end_from_oldest)]
@pytest.mark.asyncio
async def test_paginator_newest_first_offset_returns_chronological():
"""服务端 start 从最新往回数时,交付列表必须仍是时间升序(BUG 回归)。"""
conns = [NewestFirstConn(3000, 1000) for _ in range(3)]
pool = FakePool(conns)
pag = Paginator(pool, concurrency=3) # type: ignore[arg-type]
def factory(s, c):
return (s, c)
result = await pag.fetch_all(
factory, PageSpec(page_size=1000, total=3000, newest_first=True))
assert len(result) == 3000
# 时间升序:close 随 index 单调递增
closes = [b.close for b in result]
assert closes == sorted(closes), "返回列表必须按时间升序拼接"
assert result[0].close == pytest.approx(10.0)
assert result[-1].close == pytest.approx(10.0 + 2999 * 0.01)
@pytest.mark.asyncio
async def test_paginator_newest_first_respects_start_offset():
"""newest_first + start_offset:只取最近 N 条里的后段。"""
conns = [NewestFirstConn(3000, 1000) for _ in range(3)]
pool = FakePool(conns)
pag = Paginator(pool, concurrency=3) # type: ignore[arg-type]
def factory(s, c):
return (s, c)
result = await pag.fetch_all(
factory,
PageSpec(page_size=1000, total=1000, newest_first=True, start_offset=0))
assert len(result) == 1000
# 只取了最新的 1000 条 → 是 2000..2999 这一段
assert result[0].close == pytest.approx(10.0 + 2000 * 0.01)
assert result[-1].close == pytest.approx(10.0 + 2999 * 0.01)
# ---------- get_kline 日期过滤 ---------- # ---------- get_kline 日期过滤 ----------
class DateBarApiConn(FakeApiConn): class DateBarApiConn(FakeApiConn):
@@ -329,3 +399,143 @@ async def test_get_kline_since():
assert bars[-1].year == 2027 assert bars[-1].year == 2027
finally: finally:
await td.close() await td.close()
# ---------- A股逐笔:b8 过滤 + 单连接串行游标(回归) ----------
class TickApiConn(FakeApiConn):
"""模拟 0x122F:服务端 start 从**最新**往回数,且可注入 bs_flag=8 委托记录。"""
# 类级:[(time, price, vol, trade_count, bs_flag), ...],时间升序(最早在前)
rows: list[tuple] = []
calls: list[tuple[int, int]] = []
async def execute(self, cmd):
if "MacTransactionCmd" not in type(cmd).__name__:
return await super().execute(cmd)
TickApiConn.calls.append((cmd.start, cmd.count))
# 从最新往回数:start 偏移落在末尾,往前取 count 条
end = len(TickApiConn.rows) - cmd.start
begin = max(0, end - cmd.count)
chunk = TickApiConn.rows[begin:end]
return [TransactionRecord(market=1, code="600000", time=t, price=p,
vol=v, trade_count=tc, bs_flag=bs)
for (t, p, v, tc, bs) in chunk]
def _build_tick_rows(n_trade: int = 25, n_b8: int = 7) -> list[tuple]:
"""构造 n_trade 条正常成交 + n_b8 条竞价委托(穿插在中间)。"""
rows: list[tuple] = []
for i in range(n_trade):
rows.append((f"09:{30 + i // 60:02d}:{i % 60:02d}", 10.0 + i * 0.01,
100.0 + i, 3 + i, 1 if i % 2 else 0))
if i % 4 == 0 and len([r for r in rows if r[4] == 8]) < n_b8:
# 竞价委托:(time, price, vol, tc, bs=8),vol 是累计值、不计入成交
rows.append((f"09:15:{i:02d}", 10.0, 9999.0, -1, 8))
return rows
@pytest.mark.asyncio
async def test_get_ticks_filters_auction_orders():
"""bs_flag=8 是竞价**委托**快照,不是成交 → 必须从 get_ticks 结果中剔除。
实测(2026-10-21 sh600000):b8 的 vol 是累计撮合量(可被覆盖),
计入后 Σvol 比快照 vol 虚高 20%~160%,且不同主机的 b8 集合不同。
"""
TickApiConn.rows = _build_tick_rows(n_trade=25, n_b8=7)
TickApiConn.calls = []
assert sum(1 for r in TickApiConn.rows if r[4] == 8) == 7
with mock.patch("pytdxdata.api.get_standard_hosts", return_value=["s1"]), \
mock.patch("pytdxdata.api.get_mac_hosts", return_value=["m1"]), \
mock.patch("pytdxdata.pool.pool.TdxConnection",
side_effect=lambda h, p=7709, t=10.0: TickApiConn(h)):
td = TdxData(cache_dir=None, pool_min=1, pool_max=2)
await td.start()
try:
out = await td.get_ticks("sh600000")
finally:
await td.close()
assert out, "应返回成交记录"
assert all(r.bs_flag != 8 for r in out), "bs_flag=8 竞价委托必须被剔除"
assert len(out) == 25
assert sum(r.vol for r in out) == sum(100.0 + i for i in range(25))
@pytest.mark.asyncio
async def test_get_ticks_pages_on_single_connection():
"""翻页必须走**同一连接**:并发多主机会让页边界错位(重叠 + 漏条)。"""
TickApiConn.rows = _build_tick_rows(n_trade=25, n_b8=0)
TickApiConn.calls = []
with mock.patch("pytdxdata.api.get_standard_hosts", return_value=["s1"]), \
mock.patch("pytdxdata.api.get_mac_hosts", return_value=["m1", "m2", "m3"]), \
mock.patch("pytdxdata.pool.pool.TdxConnection",
side_effect=lambda h, p=7709, t=10.0: TickApiConn(h)):
td = TdxData(cache_dir=None, pool_min=1, pool_max=4)
await td.start()
try:
out = await td.get_ticks("sh600000")
finally:
await td.close()
# 单页装得下 → 只发一次请求
assert len(TickApiConn.calls) == 1
assert len(out) == 25
# 时间升序
times = [r.time for r in out]
assert times == sorted(times)
@pytest.mark.asyncio
async def test_get_ticks_multi_page_uses_one_connection_and_no_overlap():
"""多页(>_TICK_PAGE):同一连接、游标按 len(rows) 推进、页间不重叠不漏条。
回归:并发分页时各页来自不同主机,而 0x122F 的 start 是「从最新往回数」的
相对锚点 —— 各主机对 bs_flag=8 收录不同,同一 start 落到不同真实成交上,
结果页边界既重叠又漏条(实测 sh600000 冗余 116 条、13:03 附近丢 59 条)。
"""
from pytdxdata.api import _TICK_PAGE
n = _TICK_PAGE * 2 + 137 # 3 页,末页短页 → 触发 stop_when_short
TickApiConn.rows = [
(f"{9 + i // 3600:02d}:{i // 60 % 60:02d}:{i % 60:02d}",
10.0 + i * 0.001, 1.0 + i, 1 + i, 1)
for i in range(n)
]
TickApiConn.calls = []
with mock.patch("pytdxdata.api.get_standard_hosts", return_value=["s1"]), \
mock.patch("pytdxdata.api.get_mac_hosts", return_value=["m1", "m2", "m3"]), \
mock.patch("pytdxdata.pool.pool.TdxConnection",
side_effect=lambda h, p=7709, t=10.0: TickApiConn(h)):
td = TdxData(cache_dir=None, pool_min=1, pool_max=4)
await td.start()
try:
out = await td.get_ticks("sh600000")
finally:
await td.close()
# 恰好 3 页:0 / 1000 / 2000,游标按实际返回条数推进
assert TickApiConn.calls == [(0, _TICK_PAGE),
(_TICK_PAGE, _TICK_PAGE),
(_TICK_PAGE * 2, _TICK_PAGE)]
assert len(out) == n, f"必须无重叠、无漏条:期望 {n},实得 {len(out)}"
vols = [r.vol for r in out]
assert len(set(vols)) == n, "vol 全唯一(无重复记录)"
assert vols == sorted(vols), "时间升序"
# count 参数:游标逐页扣减
TickApiConn.calls = []
with mock.patch("pytdxdata.api.get_standard_hosts", return_value=["s1"]), \
mock.patch("pytdxdata.api.get_mac_hosts", return_value=["m1", "m2", "m3"]), \
mock.patch("pytdxdata.pool.pool.TdxConnection",
side_effect=lambda h, p=7709, t=10.0: TickApiConn(h)):
td = TdxData(cache_dir=None, pool_min=1, pool_max=4)
await td.start()
try:
out2 = await td.get_ticks("sh600000", count=_TICK_PAGE + 50)
finally:
await td.close()
assert len(out2) == _TICK_PAGE + 50
assert TickApiConn.calls[1][1] == 50, "末页只补足剩余 50 条"
+116
View File
@@ -0,0 +1,116 @@
"""ExPool 背压/并发语义回归(Issue #1:并发超限 0.15s 秒失败)。
用 mock 连接模拟:单服务器 + per_server_cap=1 时,
- 并发 acquire 超过有效并发 → 排队等待而不是快速失败;
- release 唤醒等待者;
- acquire 超时异常带池状态;
- ex_pool_cap 提升同主机并发。
"""
from __future__ import annotations
import asyncio
import time
import pytest
from pytdxdata.pool.ex_pool import ExPool, ExPoolTimeoutError
class MockConn:
"""Minimal stand-in for ExTdxConnection."""
last_created = 0
def __init__(self, host: str, port: int, timeout: float, handshake: str = "login"):
self.host = host
self.port = port
self.alive = True
self.last_used = time.monotonic()
self.closed = False
MockConn.last_created += 1
async def connect(self) -> None:
await asyncio.sleep(0) # instant "connect"
async def close(self) -> None:
self.closed = True
self.alive = False
@pytest.fixture
def patch_conn(monkeypatch):
monkeypatch.setattr("pytdxdata.pool.ex_pool.ExTdxConnection", MockConn)
MockConn.last_created = 0
async def test_acquire_queues_instead_of_fast_fail(patch_conn):
"""并发 > 有效并发时,后续 acquire 等待 release,而非 0.15s 抛错。"""
pool = ExPool(["e1"], min_size=1, max_size=6, per_server_cap=1)
await pool.start()
c1 = await pool.acquire() # 唯一连接被占用
t0 = time.monotonic()
async def holder():
await asyncio.sleep(0.5) # 持有 0.5s 后归还
await pool.release(c1)
asyncio.create_task(holder())
c2 = await pool.acquire() # 应在 ~0.5s 后拿到,而不是 0.15s 抛错
waited = time.monotonic() - t0
assert waited >= 0.4, f"acquire 没有等待释放(仅 {waited:.2f}s)"
assert c2 is c1 # 归还的是同一条连接
async def test_acquire_timeout_carries_pool_state(patch_conn):
"""超时异常信息带池状态(idle/total/servers/cap)。"""
pool = ExPool(["e1"], min_size=1, max_size=6, per_server_cap=1, acquire_timeout=0.3)
await pool.start()
c1 = await pool.acquire()
with pytest.raises(ExPoolTimeoutError) as ei:
await pool.acquire()
msg = str(ei.value)
assert "total=1" in msg and "per_server_cap=1" in msg and "servers=1" in msg
await pool.release(c1)
async def test_ex_pool_cap_allows_parallel_conns_same_host(patch_conn):
"""per_server_cap=2 时单主机可并发持有 2 条连接。"""
pool = ExPool(["e1"], min_size=1, max_size=6, per_server_cap=2)
await pool.start()
got = await asyncio.gather(pool.acquire(), pool.acquire())
assert len(got) == 2 and got[0] is not got[1]
assert MockConn.last_created == 2
for c in got:
await pool.release(c)
async def test_concurrent_acquire_all_succeed(patch_conn):
"""Issue 场景:单服务器 + cap=2,6 个并发任务应全部拿到连接(排队)而非部分失败。"""
pool = ExPool(["e1"], min_size=2, max_size=6, per_server_cap=2)
await pool.start()
async def task(i):
c = await pool.acquire()
await asyncio.sleep(0.1) # 模拟一次 1m 分页拉取
await pool.release(c)
return i
results = await asyncio.gather(*[task(i) for i in range(6)])
assert sorted(results) == list(range(6)) # 全部成功,零失败
async def test_release_notifies_waiters(patch_conn):
"""broken 归还(连接死亡)也要唤醒等待者(槽位回收后可扩容)。"""
pool = ExPool(["e1"], min_size=1, max_size=1, per_server_cap=1)
await pool.start()
c1 = await pool.acquire()
c1.alive = False
async def broken_release():
await asyncio.sleep(0.2)
await pool.release(c1, broken=True)
asyncio.create_task(broken_release())
c2 = await pool.acquire() # 槽位被回收→扩容新建
assert c2 is not c1
await pool.release(c2)
+59
View File
@@ -12,6 +12,7 @@ from pytdxdata.protocol.commands.ex_proto import GetExInstrumentBarsCmd
from pytdxdata.protocol.commands.kline import GetSecurityBarsCmd from pytdxdata.protocol.commands.kline import GetSecurityBarsCmd
from pytdxdata.protocol.commands.mac import MacKlineCmd from pytdxdata.protocol.commands.mac import MacKlineCmd
from pytdxdata.protocol.commands.minute import GetHistoryMinuteTimeDataCmd from pytdxdata.protocol.commands.minute import GetHistoryMinuteTimeDataCmd
from pytdxdata.protocol.commands.quotes import price_decimal_digits
from pytdxdata.protocol.commands.transaction import MacTransactionCmd from pytdxdata.protocol.commands.transaction import MacTransactionCmd
from pytdxdata.protocol.frame import FrameHeader, decompress_body, parse_header from pytdxdata.protocol.frame import FrameHeader, decompress_body, parse_header
from pytdxdata.protocol.price import decode_ohlc_diffs, get_price, put_price from pytdxdata.protocol.price import decode_ohlc_diffs, get_price, put_price
@@ -181,6 +182,30 @@ def test_mac_transaction_parse():
assert recs[1].trade_count == 1 assert recs[1].trade_count == 1
def test_mac_transaction_parse_negative_trade_count_auction():
"""集合竞价段 trade_count 是有符号 int32,负数不能被读成 42 亿(BUG 回归)。
实测(2026-10-21,sh600000 09:15:03):18 字节为
17820000 0ad71b41 d9000000 96fcffff 0800
最后四字节前的 96fcffff = int32 -874,旧解析按 uint32 得 4294966422。
"""
body = bytearray(39 + 2 * 18)
struct.pack_into("<H", body, 29, 2)
# 第1条:集合竞价,trade_count = -874
struct.pack_into("<IfIiH", body, 39, 33303, 9.74, 217, -874, 8)
# 第2条:正常交易,trade_count = 3
struct.pack_into("<IfIiH", body, 39 + 18, 34200, 9.75, 100, 3, 1)
cmd = MacTransactionCmd(1, "600000", 20261021, 0, 100)
recs = cmd.parse_response(bytes(body))
assert len(recs) == 2
assert recs[0].trade_count == -874, "负值必须保留符号,不能变成 4294966422"
assert recs[0].bs_flag == 8
assert recs[1].trade_count == 3
# 正常交易段的量级不受影响
assert max(r.trade_count for r in recs) == 3
# ---------- 跨时区时间戳归一化(美股 EX 通道) ---------- # ---------- 跨时区时间戳归一化(美股 EX 通道) ----------
# #
# 通达信 EX 通道对美股返回的时间戳把两个时区拼在一起:日期字段是**美东交易日**、 # 通达信 EX 通道对美股返回的时间戳把两个时区拼在一起:日期字段是**美东交易日**、
@@ -286,3 +311,37 @@ def test_finish_bars_sorts_us_session_chronologically():
assert [b.datetime for b in out] == [datetime(2026, 10, 12, 21, 31), assert [b.datetime for b in out] == [datetime(2026, 10, 12, 21, 31),
datetime(2026, 10, 13, 4, 0)] datetime(2026, 10, 13, 4, 0)]
assert {b.symbol for b in out} == {"usAAPL"} assert {b.symbol for b in out} == {"usAAPL"}
# ---------- 报价小数位:沪市指数必须 2 位(回归:曾整体少 10 倍) ----------
@pytest.mark.parametrize("market,code,expect", [
# 沪市指数:回归 bug —— 曾被判 3 位,导致价格少 10 倍
(1, "000001", 2), # 上证指数
(1, "000016", 2), # 上证50
(1, "000300", 2), # 沪深300
(1, "000688", 2), # 科创50
(1, "880001", 2), # 板块指数(8 开头,曾被判 3 位)
(1, "880305", 2),
# 沪市 ETF/基金:仍是 3 位
(1, "510300", 3),
(1, "588000", 3),
(1, "511010", 3),
(1, "501018", 3),
# 沪市股票:2 位
(1, "600000", 2),
(1, "601398", 2),
(1, "688111", 2), # 科创板
# 深市
(0, "399001", 2), # 深证成指
(0, "399006", 2), # 创业板指
(0, "000001", 2), # 平安银行(不是指数!)
(0, "300750", 2),
(0, "159915", 3), # 深市 ETF
(0, "128035", 3), # 深市转债
])
def test_price_decimal_digits(market, code, expect):
assert price_decimal_digits(market, code) == expect
def test_price_decimal_digits_strips_nul():
assert price_decimal_digits(1, "000016\x00\x00") == 2
Generated
+1 -1
View File
@@ -535,7 +535,7 @@ wheels = [
[[package]] [[package]]
name = "pytdxdata" name = "pytdxdata"
version = "0.6.2" version = "0.6.4"
source = { editable = "." } source = { editable = "." }
[package.optional-dependencies] [package.optional-dependencies]