Compare commits
7
Commits
v0.6.1
...
0618062e54
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
0618062e54 | ||
|
|
9c97fea71b | ||
|
|
2d62d82b51 | ||
|
|
7e72f70eb2 | ||
|
|
b2f899df49 | ||
|
|
8193d560da | ||
|
|
5408903a4c |
No files matched your search
@@ -2,6 +2,53 @@
|
|||||||
|
|
||||||
本项目遵循 [语义化版本](https://semver.org/lang/zh-CN/)。
|
本项目遵循 [语义化版本](https://semver.org/lang/zh-CN/)。
|
||||||
|
|
||||||
|
## [0.6.3] - 2026-10-18
|
||||||
|
|
||||||
|
- fix(pool): ExPool acquire 背压等待(Condition 排队,默认 30s 超时)替代 3x0.05s 快速失败;超时异常带池状态
|
||||||
|
- feat(api): 新增 ex_pool_cap 参数(默认 2),单 EX 服务器场景不再静默退化为 1 并发(fixes #1)
|
||||||
|
|
||||||
|
## [0.6.2] - 2026-10-13
|
||||||
|
|
||||||
|
### 修复
|
||||||
|
|
||||||
|
- **美股分钟级 K 线的 `datetime` 全部错位**(`get_bars("usAAPL", KlinePeriod.MIN_1)`)。
|
||||||
|
|
||||||
|
通达信 EX 通道返回的美股分钟线时间戳,把**两个时区拼进了同一个五元组**:日期分量是
|
||||||
|
**美东交易日**(协议里的 `zipday`),时分分量却是**北京时间**(`tminutes`)。直拼出的
|
||||||
|
`datetime` 会把同一交易日的**下午半场排到上午半场之前**——单日 390 根不仅首末颠倒,
|
||||||
|
而且全部落在同一个北京自然日里。于是任何依赖 `datetime` 的行为都是错的:排序、
|
||||||
|
`count` 切片("取最新 N 根")、日期区间二分与过滤、增量更新水位线。
|
||||||
|
|
||||||
|
| 观测项 | 修复前 | 修复后 |
|
||||||
|
|---|---|---|
|
||||||
|
| 单日 390 根的日期分布 | **全部**落在 10-12(hour 0~23 混在一起) | 10-12 的 21:31~23:59(149 根)+ 10-13 的 00:00~04:00(241 根) |
|
||||||
|
| 时间轴 | 上午半场被排到下午半场**之前** | 严格单调递增 |
|
||||||
|
| `count=30` 是否取到真正最新的 30 根 | 否(拿到上一交易日的 23:30~23:59) | 是(当日收盘段 03:31~04:00) |
|
||||||
|
| 每组首根 `open` / 末根 `close` 对日线 | 对不上 | **逐位相同**(10-07/08/09/12 四个交易日实测) |
|
||||||
|
|
||||||
|
现在 `SecurityBar.datetime` **统一为北京时间**:美股凌晨段(北京 hour < 12)的日期
|
||||||
|
自动 +1 天(含月末 / 年末进位)。A股 / 港股 / 期货原始即为北京时间,**零行为变化**。
|
||||||
|
|
||||||
|
新增两个只读属性:
|
||||||
|
|
||||||
|
| 属性 | 用途 |
|
||||||
|
|---|---|
|
||||||
|
| `SecurityBar.trading_date` | 该 bar 归属的交易日 `YYYY-MM-DD`。美股一个交易日横跨两个北京自然日,**按交易日聚合(分钟线合成日线、日内统计)必须用它,不能用 `datetime.date()`** |
|
||||||
|
| `SecurityBar.is_us` | 是否美股(EX 通道 `US_STOCK`) |
|
||||||
|
|
||||||
|
归一化在 `SecurityBar.__post_init__` 中实现,因此 **EX 的两条 K 线命令路径**
|
||||||
|
(`_bars_ex` 的 0x23FF 与日期区间二分所依赖的 MAC 0x122E)自动共用同一份口径,
|
||||||
|
不会出现"取数按一套时间、二分按另一套时间"的错位。
|
||||||
|
|
||||||
|
### 说明
|
||||||
|
|
||||||
|
- 分钟线缓存 TTL 为 **60 秒**,升级后旧缓存最多残留 1 分钟即自然失效,**无需手动清理**;
|
||||||
|
日线(`period >= DAY`)不受本缺陷影响。
|
||||||
|
- 本版只修 K 线。**同日发现的同类问题未包含在内**:EX 分时(`get_minutes`)对美股返回的
|
||||||
|
`MinuteBar.hour` 是**未取模的连续分钟数**(实测可达 27,即次日 03:xx),且 `MinuteBar`
|
||||||
|
无日期分量可承载跨日信息——修法涉及"是否新增字段 / 日期如何归属"的取舍,留待下一版决策。
|
||||||
|
- 新增 9 项针对性回归测试(分界点、月末与年末进位、390 根整日单调性、MAC 路径同样归一化)。
|
||||||
|
|
||||||
## [0.6.1] - 2026-09-21
|
## [0.6.1] - 2026-09-21
|
||||||
|
|
||||||
### 变更
|
### 变更
|
||||||
|
|||||||
@@ -104,7 +104,7 @@ curl -o ~/.workbuddy/skills/pytdxdata/SKILL.md --create-dirs \
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
uv sync # 建虚拟环境 + 注册 tdx 命令
|
uv sync # 建虚拟环境 + 注册 tdx 命令
|
||||||
uv run pytest tests/ # 182 个测试用例(协议 / 池 / 缓存 / 分页 / 全接口回归 / 文件解析 / CLI / 文档漂移 / 当日测速缓存)
|
uv run pytest tests/ # 191 个测试用例(协议 / 池 / 缓存 / 分页 / 全接口回归 / 文件解析 / CLI / 文档漂移 / 当日测速缓存)
|
||||||
uv sync --group docs # 文档站依赖(mkdocs-material + mkdocstrings)
|
uv sync --group docs # 文档站依赖(mkdocs-material + mkdocstrings)
|
||||||
uv run mkdocs serve # 本地预览文档站
|
uv run mkdocs serve # 本地预览文档站
|
||||||
```
|
```
|
||||||
@@ -114,6 +114,9 @@ uv run mkdocs serve # 本地预览文档站
|
|||||||
- 行情来自通达信公开服务器,**15 分钟延时**;需实时数据请使用券商行情
|
- 行情来自通达信公开服务器,**15 分钟延时**;需实时数据请使用券商行情
|
||||||
- 标准通道 K 线 `count < 100` 部分服务器返回空 → 分页器已强制 `min_page=100` 规避
|
- 标准通道 K 线 `count < 100` 部分服务器返回空 → 分页器已强制 `min_page=100` 规避
|
||||||
- 港股 / 美股期权在通达信协议内无数据
|
- 港股 / 美股期权在通达信协议内无数据
|
||||||
|
- **K 线时间戳统一为北京时间**(v0.6.2 起)。美股原始报文把「美东交易日日期」与「北京时间时分」
|
||||||
|
拼在一起,库已自动归一化,`bar.datetime` 可直接排序 / 过滤;但**美股一个交易日横跨两个北京
|
||||||
|
自然日**,按交易日聚合请用 `SecurityBar.trading_date`,不要用 `datetime.date()`
|
||||||
- 服务器可用性随时间变化。**当日首次启动会自动在后台测速并缓存**(对调用方无感,当天不再重测);
|
- 服务器可用性随时间变化。**当日首次启动会自动在后台测速并缓存**(对调用方无感,当天不再重测);
|
||||||
也可用 `python -m pytdxdata.server_probe` 离线重生成随包清单。
|
也可用 `python -m pytdxdata.server_probe` 离线重生成随包清单。
|
||||||
缓存路径与三个开关见 [安装与配置 → 当日自动测速](https://openbot-coder.github.io/pytdxdata/getting-started/#当日自动测速无感)
|
缓存路径与三个开关见 [安装与配置 → 当日自动测速](https://openbot-coder.github.io/pytdxdata/getting-started/#当日自动测速无感)
|
||||||
|
|||||||
@@ -164,6 +164,7 @@ await td.get_bars("cffex:IFL0", KlinePeriod.DAY, count=700) # 期货合约
|
|||||||
|
|
||||||
- **数据有 15 分钟延时**(通达信公开服务器特性)。要实时数据请用券商行情。
|
- **数据有 15 分钟延时**(通达信公开服务器特性)。要实时数据请用券商行情。
|
||||||
- **复权 K 线与 1 分钟 K 线走 MAC 通道**。`get_bars(..., adjust=Adjust.QFQ)` 自动切换。
|
- **复权 K 线与 1 分钟 K 线走 MAC 通道**。`get_bars(..., adjust=Adjust.QFQ)` 自动切换。
|
||||||
|
- **K 线时间戳统一为北京时间**(v0.6.2 起)。美股原始报文里日期是美东交易日、时分是北京时间(两个时区拼在一起),库已自动归一化,`bar.datetime` 可直接排序 / 比较 / 过滤。但**美股一个交易日横跨两个北京自然日**,按交易日聚合(分钟线合成日线、日内统计)必须用 `bar.trading_date`。
|
||||||
- **`count` 很大时会自动多连接并发分页**,无需手动翻页。但标准通道 K 线 `count < 100` 部分服务器会返回空——内部已强制 `min_page=100` 规避。
|
- **`count` 很大时会自动多连接并发分页**,无需手动翻页。但标准通道 K 线 `count < 100` 部分服务器会返回空——内部已强制 `min_page=100` 规避。
|
||||||
- **`get_quotes` 一次 >80 只自动切批**;走 MAC 字段位通道(`fields=`)时上限为 80 只/次。
|
- **`get_quotes` 一次 >80 只自动切批**;走 MAC 字段位通道(`fields=`)时上限为 80 只/次。
|
||||||
- **历史逐笔没有笔数字段**,只有当日逐笔的 `trade_count` 有效。
|
- **历史逐笔没有笔数字段**,只有当日逐笔的 `trade_count` 有效。
|
||||||
@@ -183,6 +184,7 @@ await td.get_bars("cffex:IFL0", KlinePeriod.DAY, count=700) # 期货合约
|
|||||||
| 用 `get_block_info` 又自己拆二进制 | 白写解析器 | 用 `get_block_parsed`,已经解析好了 |
|
| 用 `get_block_info` 又自己拆二进制 | 白写解析器 | 用 `get_block_parsed`,已经解析好了 |
|
||||||
| 循环里逐个 `await get_bars` | 慢 | 用列表传多个标的,自动并发 |
|
| 循环里逐个 `await get_bars` | 慢 | 用列表传多个标的,自动并发 |
|
||||||
| 港股/美股五档 bid/ask | 没有(协议不支持) | `fields` 字段位通道有 name 但无五档 |
|
| 港股/美股五档 bid/ask | 没有(协议不支持) | `fields` 字段位通道有 name 但无五档 |
|
||||||
|
| 美股分钟线用 `datetime.date()` 分组做日聚合 | 一个交易日被拆成两天(美股跨北京零点) | 用 `bar.trading_date`;`bar.datetime` 是归一化后的北京时间 |
|
||||||
| 依赖实时性做信号 | 数据是 15 分钟前的 | 换券商实时源 |
|
| 依赖实时性做信号 | 数据是 15 分钟前的 | 换券商实时源 |
|
||||||
| 手动改包内 `servers.json` / 写死 IP | 容器只读、升级被覆盖、快节点捞不回来 | 日常靠内置的**当日测速缓存**自动选路;要强制指定就用 `PYPYTDX_DATA_{STANDARD,MAC,EX}_HOSTS` 环境变量 |
|
| 手动改包内 `servers.json` / 写死 IP | 容器只读、升级被覆盖、快节点捞不回来 | 日常靠内置的**当日测速缓存**自动选路;要强制指定就用 `PYPYTDX_DATA_{STANDARD,MAC,EX}_HOSTS` 环境变量 |
|
||||||
|
|
||||||
|
|||||||
@@ -62,6 +62,22 @@ from pytdxdata.models import BoardInfo, MemberQuote, AuctionItem, CapitalFlow
|
|||||||
| 结构 | 具名属性(`zongguben`、`jingzichan` ...) | `fields: list[float]`,原始字段数组 |
|
| 结构 | 具名属性(`zongguben`、`jingzichan` ...) | `fields: list[float]`,原始字段数组 |
|
||||||
| 单位 | 万元 / 万股 | 原始 float,需自行按字段语义换算 |
|
| 单位 | 万元 / 万股 | 原始 float,需自行按字段语义换算 |
|
||||||
|
|
||||||
|
!!! tip "`SecurityBar.datetime` 的时间戳口径(v0.6.2 起)"
|
||||||
|
时间统一为**北京时间**。A股 / 港股 / 期货的原始时间戳即为北京时间;美股的原始报文
|
||||||
|
混用了「美东交易日日期」与「北京时间时分」两个时区,库在构造时已自动归一化
|
||||||
|
(凌晨段日期 +1 天),因此 `bar.datetime` 可直接排序 / 比较 / 过滤。
|
||||||
|
|
||||||
|
但要注意**美股一个交易日横跨两个北京自然日**,按交易日聚合请用 `bar.trading_date`:
|
||||||
|
|
||||||
|
```python
|
||||||
|
bars = await td.get_bars("usAAPL", KlinePeriod.MIN_1, count=390)
|
||||||
|
bars[0].datetime # 2026-10-12 21:31:00(北京时间)
|
||||||
|
bars[-1].datetime # 2026-10-13 04:00:00(次日)
|
||||||
|
bars[0].trading_date # "2026-10-12" —— 归属的美东交易日
|
||||||
|
```
|
||||||
|
|
||||||
|
`bar.is_us` 可判定是否美股。详见 [常见问题 → 美股分钟 K 线](../faq.md)。
|
||||||
|
|
||||||
## 完整字段定义
|
## 完整字段定义
|
||||||
|
|
||||||
以下由 `mkdocstrings` 从源码实时渲染(**以源码为准**):
|
以下由 `mkdocstrings` 从源码实时渲染(**以源码为准**):
|
||||||
|
|||||||
+17
@@ -99,6 +99,23 @@
|
|||||||
2. **是否含未收盘的那根** —— 盘中最后一根是动态的
|
2. **是否含未收盘的那根** —— 盘中最后一根是动态的
|
||||||
3. 分钟线注意**数据来源深度**不同
|
3. 分钟线注意**数据来源深度**不同
|
||||||
|
|
||||||
|
??? question "美股分钟 K 线的时间为什么跨了两个日期?"
|
||||||
|
**已在上游修好(v0.6.2)**。通达信 EX 通道对美股返回的原生时间戳把两个时区拼在一起:
|
||||||
|
日期分量是**美东交易日**、时分分量是**北京时间**。修复前,同一交易日的 390 根会全部
|
||||||
|
落在同一个北京自然日里,并且下午半场被排在上午半场**之前**。
|
||||||
|
|
||||||
|
现在 `SecurityBar.datetime` 已统一为**北京时间**(凌晨段日期自动 +1 天),可直接排序 /
|
||||||
|
过滤 / 做增量水位线。但要注意**美股一个交易日横跨两个北京自然日**:
|
||||||
|
|
||||||
|
```python
|
||||||
|
bars = await td.get_bars("usAAPL", KlinePeriod.MIN_1, count=390)
|
||||||
|
print(bars[0].datetime, bars[-1].datetime) # 2026-10-12 21:31:00 → 2026-10-13 04:00:00
|
||||||
|
print({b.trading_date for b in bars}) # {'2026-10-12'} —— 同一个交易日
|
||||||
|
```
|
||||||
|
|
||||||
|
按**交易日**聚合(分钟线合成日线、日内统计)请用 `b.trading_date`,不要用
|
||||||
|
`b.datetime.date()`。A股 / 港股 / 期货原始即为北京时间,不受影响。
|
||||||
|
|
||||||
??? question "缓存会不会返回过期数据?"
|
??? question "缓存会不会返回过期数据?"
|
||||||
缓存按数据类型分级 TTL,不会返回超出 TTL 的数据:
|
缓存按数据类型分级 TTL,不会返回超出 TTL 的数据:
|
||||||
|
|
||||||
|
|||||||
@@ -93,6 +93,10 @@ tdx financial gpcw20260331.zip --limit 5
|
|||||||
- **`KlinePeriod.MIN_1` 走 MAC 通道并单连接串行分页**,这是刻意设计:并发分页在多服务器
|
- **`KlinePeriod.MIN_1` 走 MAC 通道并单连接串行分页**,这是刻意设计:并发分页在多服务器
|
||||||
历史深度不一致时会产生非确定性数据空洞。不要试图"优化"成并发。
|
历史深度不一致时会产生非确定性数据空洞。不要试图"优化"成并发。
|
||||||
- **复权 K 线走 MAC 通道**,标准通道不提供复权数据。
|
- **复权 K 线走 MAC 通道**,标准通道不提供复权数据。
|
||||||
|
- **K 线 `datetime` 统一为北京时间**(v0.6.2 起)。美股原始报文的日期分量是美东交易日、
|
||||||
|
时分分量是北京时间,库已自动归一化,`bar.datetime` 可直接排序 / 比较 / 过滤。
|
||||||
|
但**美股一个交易日横跨两个北京自然日**:按交易日聚合(分钟线合成日线、日内统计)
|
||||||
|
用 `bar.trading_date`,不要用 `bar.datetime.date()`。`bar.is_us` 判定美股。
|
||||||
- **标准通道部分服务器在 `count < 100` 时返回空**,分页器已强制 `min_page=100` 规避。
|
- **标准通道部分服务器在 `count < 100` 时返回空**,分页器已强制 `min_page=100` 规避。
|
||||||
- **本地 Windows 上 `asyncio.new_event_loop()` 可能永久挂起**,根因是回环 `socket.socketpair()`
|
- **本地 Windows 上 `asyncio.new_event_loop()` 可能永久挂起**,根因是回环 `socket.socketpair()`
|
||||||
被本地代理/安全软件拦截。异步测试请在 Linux/WSL 运行。
|
被本地代理/安全软件拦截。异步测试请在 Linux/WSL 运行。
|
||||||
|
|||||||
+1
-1
@@ -1,6 +1,6 @@
|
|||||||
[project]
|
[project]
|
||||||
name = "pytdxdata"
|
name = "pytdxdata"
|
||||||
version = "0.6.1"
|
version = "0.6.3"
|
||||||
description = "通达信(TongDaXin)行情数据客户端:动态连接池 + 双层TTL缓存 + 3连接并发分页,纯asyncio零运行时依赖,支持A股/港股/美股/期货/期权"
|
description = "通达信(TongDaXin)行情数据客户端:动态连接池 + 双层TTL缓存 + 3连接并发分页,纯asyncio零运行时依赖,支持A股/港股/美股/期货/期权"
|
||||||
readme = "README.md"
|
readme = "README.md"
|
||||||
requires-python = ">=3.11"
|
requires-python = ">=3.11"
|
||||||
|
|||||||
@@ -16,7 +16,7 @@ from .models import (
|
|||||||
XdxrRecord,
|
XdxrRecord,
|
||||||
)
|
)
|
||||||
|
|
||||||
__version__ = "0.6.1"
|
__version__ = "0.6.2"
|
||||||
__all__ = [
|
__all__ = [
|
||||||
"TdxData",
|
"TdxData",
|
||||||
"Market",
|
"Market",
|
||||||
|
|||||||
@@ -233,11 +233,13 @@ class TdxData:
|
|||||||
pool_max: int = 12,
|
pool_max: int = 12,
|
||||||
mac_pool_cap: int = 4,
|
mac_pool_cap: int = 4,
|
||||||
std_pool_cap: int = 4,
|
std_pool_cap: int = 4,
|
||||||
|
ex_pool_cap: int = 2,
|
||||||
) -> None:
|
) -> None:
|
||||||
self.port = port
|
self.port = port
|
||||||
self.timeout = timeout
|
self.timeout = timeout
|
||||||
self.default_adjust = default_adjust
|
self.default_adjust = default_adjust
|
||||||
self.mac_pool_cap = mac_pool_cap
|
self.mac_pool_cap = mac_pool_cap
|
||||||
|
self.ex_pool_cap = ex_pool_cap
|
||||||
self._std_servers = list(standard_servers or get_standard_hosts())
|
self._std_servers = list(standard_servers or get_standard_hosts())
|
||||||
self._mac_servers = list(mac_servers or get_mac_hosts())
|
self._mac_servers = list(mac_servers or get_mac_hosts())
|
||||||
self._ex_servers = list(ex_servers or get_ex_hosts())
|
self._ex_servers = list(ex_servers or get_ex_hosts())
|
||||||
@@ -262,8 +264,11 @@ class TdxData:
|
|||||||
probe_factory=lambda: MacKlineCmd(1, "600000", KlinePeriod.DAY, Adjust.NONE, 0, 100),
|
probe_factory=lambda: MacKlineCmd(1, "600000", KlinePeriod.DAY, Adjust.NONE, 0, 100),
|
||||||
)
|
)
|
||||||
# EX扩展市场池(港股/期货/美股,端口7727;MAC EX login + pytdx EX setup 混合)
|
# EX扩展市场池(港股/期货/美股,端口7727;MAC EX login + pytdx EX setup 混合)
|
||||||
|
# 单服务器场景(测速缓存收缩后仅 1 台)下,有效并发 = 服务器数 × ex_pool_cap,
|
||||||
|
# 默认 cap=2 保证至少 2 并发;acquire 有 Condition 背压等待,不再快速失败。
|
||||||
self._ex_pool = ExPool(self._ex_servers, timeout=timeout,
|
self._ex_pool = ExPool(self._ex_servers, timeout=timeout,
|
||||||
min_size=min(pool_min, 2), max_size=min(pool_max, 6),
|
min_size=min(pool_min, 2), max_size=min(pool_max, 6),
|
||||||
|
per_server_cap=ex_pool_cap,
|
||||||
handshake_map=EX_HANDSHAKES)
|
handshake_map=EX_HANDSHAKES)
|
||||||
disk = None
|
disk = None
|
||||||
if self._cache_dir is not None:
|
if self._cache_dir is not None:
|
||||||
|
|||||||
@@ -3,7 +3,7 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import struct
|
import struct
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field
|
||||||
from datetime import datetime
|
from datetime import date, datetime, timedelta
|
||||||
from enum import IntEnum
|
from enum import IntEnum
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
@@ -92,9 +92,25 @@ class Adjust(IntEnum):
|
|||||||
HFQ = 2 # 后复权
|
HFQ = 2 # 后复权
|
||||||
|
|
||||||
|
|
||||||
|
# 扩展市场中「时间戳跨时区拼接」的市场号:通达信 EX 通道对美股返回的分钟级 K线
|
||||||
|
# 里,日期字段是**美东交易日**、时分字段是**北京时间**,两个时区被拼进同一个
|
||||||
|
# 五元组,必须归一化(见 SecurityBar.__post_init__)。
|
||||||
|
_US_EX_MARKETS: frozenset[int] = frozenset({int(ExMarket.US_STOCK)})
|
||||||
|
|
||||||
|
# 归一化判据:北京时间 hour >= 12 的 bar 与美东交易日属同一自然日。美股常规时段
|
||||||
|
# 09:30~16:00 EDT = 北京 21:30~次日 04:00(冬令时 22:30~次日 05:00),夏令时/
|
||||||
|
# 冬令时下分界都稳定落在 12 点。
|
||||||
|
_US_DAY_SPLIT_HOUR = 12
|
||||||
|
|
||||||
|
|
||||||
@dataclass(slots=True)
|
@dataclass(slots=True)
|
||||||
class SecurityBar:
|
class SecurityBar:
|
||||||
"""K线记录(价格单位:元)。跨市场统一模型(A股/港股/美股/期货共用)。"""
|
"""K线记录(价格单位:元)。跨市场统一模型(A股/港股/美股/期货共用)。
|
||||||
|
|
||||||
|
时间戳口径**统一为北京时间**:A股/港股/期货原始即为北京时间;美股原始是
|
||||||
|
「日期=美东交易日 + 时分=北京时间」的混合值,构造时已还原为真实北京时间
|
||||||
|
(见 ``__post_init__``)。该 bar 归属的交易日见 :attr:`trading_date`。
|
||||||
|
"""
|
||||||
market: int
|
market: int
|
||||||
code: str
|
code: str
|
||||||
open: float
|
open: float
|
||||||
@@ -110,10 +126,40 @@ class SecurityBar:
|
|||||||
minute: int = 0
|
minute: int = 0
|
||||||
symbol: str = "" # 规范化标的串(sz000001 / hk00700),由 API 层填充
|
symbol: str = "" # 规范化标的串(sz000001 / hk00700),由 API 层填充
|
||||||
|
|
||||||
|
def __post_init__(self) -> None:
|
||||||
|
"""美股(EX 通道)时间戳归一化:还原成真实北京时间。
|
||||||
|
|
||||||
|
原始报文里 ``zipday`` 是**美东交易日**、``tminutes`` 是**北京时间分钟数**,
|
||||||
|
直拼出的 ``datetime`` 会把同一交易日的下午半场(北京 00:00~04:00)排到
|
||||||
|
上午半场(北京 21:30~23:59)**之前** —— 单日 390 根全部错位、首末颠倒。
|
||||||
|
这里把凌晨段(北京 hour < 12)的日期 +1 天,得到严格单调的北京时间序列。
|
||||||
|
"""
|
||||||
|
if self.market in _US_EX_MARKETS and self.hour < _US_DAY_SPLIT_HOUR:
|
||||||
|
d = date(self.year, self.month, self.day) + timedelta(days=1)
|
||||||
|
self.year, self.month, self.day = d.year, d.month, d.day
|
||||||
|
|
||||||
|
@property
|
||||||
|
def is_us(self) -> bool:
|
||||||
|
"""是否美股(EX 通道 ``US_STOCK``)。"""
|
||||||
|
return self.market in _US_EX_MARKETS
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def datetime(self) -> datetime:
|
def datetime(self) -> datetime:
|
||||||
|
"""bar 的时刻(北京时间)。"""
|
||||||
return datetime(self.year, self.month, self.day, self.hour, self.minute)
|
return datetime(self.year, self.month, self.day, self.hour, self.minute)
|
||||||
|
|
||||||
|
@property
|
||||||
|
def trading_date(self) -> str:
|
||||||
|
"""该 bar 归属的交易日,``YYYY-MM-DD``。
|
||||||
|
|
||||||
|
美股:``datetime`` 已还原为北京时间,凌晨段(北京 hour < 12)属于**前一天**
|
||||||
|
的美东交易日,故回退一天;A股/港股/期货的交易日即自然日。按交易日聚合
|
||||||
|
(如分钟线合成日线)时应当用本字段,而不是 ``datetime.date()``。
|
||||||
|
"""
|
||||||
|
if self.is_us and self.hour < _US_DAY_SPLIT_HOUR:
|
||||||
|
return (date(self.year, self.month, self.day) - timedelta(days=1)).isoformat()
|
||||||
|
return f"{self.year:04d}-{self.month:02d}-{self.day:02d}"
|
||||||
|
|
||||||
|
|
||||||
@dataclass(slots=True)
|
@dataclass(slots=True)
|
||||||
class IndexBar(SecurityBar):
|
class IndexBar(SecurityBar):
|
||||||
|
|||||||
@@ -1,4 +1,11 @@
|
|||||||
"""EX扩展市场连接池(简化版:固定连接数,借还+坏连接重建+login自动)"""
|
"""EX扩展市场连接池(固定连接数,借还+坏连接重建+login自动)。
|
||||||
|
|
||||||
|
与标准池 pool.py 的背压语义对齐:acquire 在无空闲连接且无法扩容时,
|
||||||
|
基于 asyncio.Condition 排队等待 release 唤醒(默认 acquire_timeout=30s),
|
||||||
|
而不是快速失败。per_server_cap 限制单台服务器连接数;单服务器场景下
|
||||||
|
有效并发 = servers × per_server_cap(默认 1),通过 TdxData(ex_pool_cap=N)
|
||||||
|
可对同一 EX 主机开多连接。
|
||||||
|
"""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import asyncio
|
import asyncio
|
||||||
@@ -9,6 +16,23 @@ from .ex_connection import ExTdxConnection
|
|||||||
|
|
||||||
log = logging.getLogger(__name__)
|
log = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
DEFAULT_ACQUIRE_TIMEOUT = 30.0
|
||||||
|
|
||||||
|
|
||||||
|
class ExPoolTimeoutError(ConnectionError):
|
||||||
|
"""acquire 等待超时(携带池状态快照,便于排障)。"""
|
||||||
|
|
||||||
|
def __init__(self, pool: "ExPool", timeout: float) -> None:
|
||||||
|
idle = len(pool._idle)
|
||||||
|
total = pool._total
|
||||||
|
servers = len(pool.servers)
|
||||||
|
cap = pool.per_server_cap
|
||||||
|
super().__init__(
|
||||||
|
f"EX连接池获取超时(等了 {timeout:.1f}s):"
|
||||||
|
f"idle={idle} total={total} servers={servers} per_server_cap={cap} "
|
||||||
|
f"有效并发={servers * cap}"
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
class ExPool:
|
class ExPool:
|
||||||
"""EX通道连接池(港股/期货,端口7727,连接时自动login)。"""
|
"""EX通道连接池(港股/期货,端口7727,连接时自动login)。"""
|
||||||
@@ -16,6 +40,7 @@ class ExPool:
|
|||||||
def __init__(self, servers: list[str], *, timeout: float = 10.0,
|
def __init__(self, servers: list[str], *, timeout: float = 10.0,
|
||||||
min_size: int = 2, max_size: int = 6,
|
min_size: int = 2, max_size: int = 6,
|
||||||
per_server_cap: int = 1,
|
per_server_cap: int = 1,
|
||||||
|
acquire_timeout: float | None = None,
|
||||||
handshake: str = "login",
|
handshake: str = "login",
|
||||||
handshake_map: dict | None = None) -> None:
|
handshake_map: dict | None = None) -> None:
|
||||||
self.servers = servers
|
self.servers = servers
|
||||||
@@ -25,14 +50,17 @@ class ExPool:
|
|||||||
self.min_size = min_size
|
self.min_size = min_size
|
||||||
self.max_size = max_size
|
self.max_size = max_size
|
||||||
self.per_server_cap = max(1, per_server_cap) # 一台服务器最多几个连接(默认1=单连接)
|
self.per_server_cap = max(1, per_server_cap) # 一台服务器最多几个连接(默认1=单连接)
|
||||||
|
self.acquire_timeout = (
|
||||||
|
DEFAULT_ACQUIRE_TIMEOUT if acquire_timeout is None else acquire_timeout
|
||||||
|
)
|
||||||
self._idle: list[tuple[str, ExTdxConnection]] = []
|
self._idle: list[tuple[str, ExTdxConnection]] = []
|
||||||
self._server_count: dict[str, int] = {}
|
self._server_count: dict[str, int] = {}
|
||||||
self._total = 0
|
self._total = 0
|
||||||
self._closed = False
|
self._closed = False
|
||||||
self._lock = asyncio.Lock()
|
self._cond = asyncio.Condition()
|
||||||
|
|
||||||
async def start(self) -> None:
|
async def start(self) -> None:
|
||||||
async with self._lock:
|
async with self._cond:
|
||||||
for host in self.servers[: self.min_size]:
|
for host in self.servers[: self.min_size]:
|
||||||
if self._server_count.get(host, 0) >= self.per_server_cap:
|
if self._server_count.get(host, 0) >= self.per_server_cap:
|
||||||
continue
|
continue
|
||||||
@@ -47,7 +75,7 @@ class ExPool:
|
|||||||
|
|
||||||
async def close(self) -> None:
|
async def close(self) -> None:
|
||||||
self._closed = True
|
self._closed = True
|
||||||
async with self._lock:
|
async with self._cond:
|
||||||
for _, conn in self._idle:
|
for _, conn in self._idle:
|
||||||
try:
|
try:
|
||||||
await conn.close()
|
await conn.close()
|
||||||
@@ -56,48 +84,61 @@ class ExPool:
|
|||||||
self._idle.clear()
|
self._idle.clear()
|
||||||
self._total = 0
|
self._total = 0
|
||||||
self._server_count.clear()
|
self._server_count.clear()
|
||||||
|
self._cond.notify_all()
|
||||||
|
|
||||||
async def acquire(self) -> ExTdxConnection:
|
async def acquire(self, timeout: float | None = None) -> ExTdxConnection:
|
||||||
for _ in range(3):
|
"""借出一条连接;无空闲且无法扩容时排队等待 release 唤醒。
|
||||||
stale_conn = None
|
|
||||||
new_conn = None
|
timeout 默认取 self.acquire_timeout(30s);传 None 表示无限等待。
|
||||||
new_host = None
|
"""
|
||||||
async with self._lock:
|
if timeout is None:
|
||||||
|
timeout = self.acquire_timeout
|
||||||
|
deadline = None if timeout is None else time.monotonic() + timeout
|
||||||
|
async with self._cond:
|
||||||
|
while True:
|
||||||
|
# 1. 优先借用空闲连接(过期则丢弃,槽位在锁内回收)
|
||||||
if self._idle:
|
if self._idle:
|
||||||
_, conn = self._idle.pop()
|
_, conn = self._idle.pop()
|
||||||
fresh = time.monotonic() - conn.last_used < 30.0
|
fresh = time.monotonic() - conn.last_used < 30.0
|
||||||
if conn.alive and fresh:
|
if conn.alive and fresh:
|
||||||
return conn
|
return conn
|
||||||
# 连接空闲过久(login会话可能失效)→ 丢弃重连
|
# 连接空闲过久(login会话可能失效)→ 丢弃,槽位让给扩容
|
||||||
self._total = max(0, self._total - 1)
|
self._total = max(0, self._total - 1)
|
||||||
self._server_count[conn.host] = max(0, self._server_count.get(conn.host, 0) - 1)
|
self._server_count[conn.host] = max(0, self._server_count.get(conn.host, 0) - 1)
|
||||||
stale_conn = conn
|
self._schedule_close(conn)
|
||||||
# 扩容:在锁内预留槽位,锁外执行网络IO
|
|
||||||
elif self._total < self.max_size and self.servers:
|
# 2. 可扩容 → 锁内预留槽位,锁外建连
|
||||||
|
new_conn = None
|
||||||
|
new_host = None
|
||||||
|
if self._total < self.max_size and self.servers:
|
||||||
host = self._next_available_server()
|
host = self._next_available_server()
|
||||||
if host:
|
if host is not None:
|
||||||
self._total += 1
|
self._total += 1
|
||||||
self._server_count[host] = self._server_count.get(host, 0) + 1
|
self._server_count[host] = self._server_count.get(host, 0) + 1
|
||||||
new_conn = ExTdxConnection(host, 7727, self.timeout, self.handshake_map.get(host, self.handshake))
|
new_conn = ExTdxConnection(host, 7727, self.timeout, self.handshake_map.get(host, self.handshake))
|
||||||
new_host = host
|
new_host = host
|
||||||
# 锁外:关闭过期连接、建立新连接
|
|
||||||
if stale_conn is not None:
|
|
||||||
try:
|
|
||||||
await stale_conn.close()
|
|
||||||
except Exception:
|
|
||||||
pass
|
|
||||||
if new_conn is not None:
|
if new_conn is not None:
|
||||||
|
# 锁外网络 IO:连接失败回滚槽位后继续等待
|
||||||
try:
|
try:
|
||||||
await new_conn.connect()
|
await new_conn.connect()
|
||||||
return new_conn
|
return new_conn
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
log.warning("EX扩容失败 %s: %s", new_host, str(e)[:60])
|
log.warning("EX扩容失败 %s: %s", new_host, str(e)[:60])
|
||||||
# 连接失败,回滚预留槽位
|
async with self._cond:
|
||||||
async with self._lock:
|
|
||||||
self._total = max(0, self._total - 1)
|
self._total = max(0, self._total - 1)
|
||||||
self._server_count[new_host] = max(0, self._server_count.get(new_host, 0) - 1)
|
self._server_count[new_host] = max(0, self._server_count.get(new_host, 0) - 1)
|
||||||
await asyncio.sleep(0.05)
|
|
||||||
raise ConnectionError("EX连接池无可用连接")
|
# 3. 池满:排队等待 release 唤醒(或超时)
|
||||||
|
now = time.monotonic()
|
||||||
|
if deadline is not None and now >= deadline:
|
||||||
|
raise ExPoolTimeoutError(self, timeout)
|
||||||
|
try:
|
||||||
|
if deadline is None:
|
||||||
|
await self._cond.wait()
|
||||||
|
else:
|
||||||
|
await asyncio.wait_for(self._cond.wait(), timeout=deadline - now)
|
||||||
|
except asyncio.TimeoutError:
|
||||||
|
raise ExPoolTimeoutError(self, timeout)
|
||||||
|
|
||||||
def _next_available_server(self) -> str | None:
|
def _next_available_server(self) -> str | None:
|
||||||
for h in self.servers:
|
for h in self.servers:
|
||||||
@@ -105,15 +146,23 @@ class ExPool:
|
|||||||
return h
|
return h
|
||||||
return None
|
return None
|
||||||
|
|
||||||
async def release(self, conn: ExTdxConnection, *, broken: bool = False) -> None:
|
def _schedule_close(self, conn: ExTdxConnection) -> None:
|
||||||
if broken or not conn.alive:
|
async def _close():
|
||||||
async with self._lock:
|
|
||||||
self._total = max(0, self._total - 1)
|
|
||||||
self._server_count[conn.host] = max(0, self._server_count.get(conn.host, 0) - 1)
|
|
||||||
try:
|
try:
|
||||||
await conn.close()
|
await conn.close()
|
||||||
except Exception:
|
except Exception:
|
||||||
pass
|
pass
|
||||||
return
|
try:
|
||||||
async with self._lock:
|
asyncio.get_running_loop().create_task(_close())
|
||||||
|
except RuntimeError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
async def release(self, conn: ExTdxConnection, *, broken: bool = False) -> None:
|
||||||
|
async with self._cond:
|
||||||
|
if broken or not conn.alive:
|
||||||
|
self._total = max(0, self._total - 1)
|
||||||
|
self._server_count[conn.host] = max(0, self._server_count.get(conn.host, 0) - 1)
|
||||||
|
self._schedule_close(conn)
|
||||||
|
else:
|
||||||
self._idle.append((conn.host, conn))
|
self._idle.append((conn.host, conn))
|
||||||
|
self._cond.notify_all()
|
||||||
+197
-140
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"version": "2026-09-21",
|
"version": "2026-10-16",
|
||||||
"note": "离线探针复测(2026-09-21 19:03): 可达=25/33, 入池fast_pool=10台",
|
"note": "离线探针复测(2026-10-16 13:46): 可达=30/36, 入池fast_pool=10台",
|
||||||
"servers": [
|
"servers": [
|
||||||
{
|
{
|
||||||
"ip": "139.9.52.158",
|
"ip": "139.9.52.158",
|
||||||
@@ -11,14 +11,14 @@
|
|||||||
],
|
],
|
||||||
"handshake": null,
|
"handshake": null,
|
||||||
"status": "ok",
|
"status": "ok",
|
||||||
"latency": 0.22,
|
"latency": 0.172,
|
||||||
"checked_at": "2026-09-21 19:03:04",
|
"checked_at": "2026-10-16 13:46:17",
|
||||||
"source": "probe-2026-08-12",
|
"source": "probe-2026-08-12",
|
||||||
"fast_pool": true,
|
"fast_pool": true,
|
||||||
"checked_date": "2026-09-21",
|
"checked_date": "2026-10-16",
|
||||||
"_per_proto": {
|
"_per_proto": {
|
||||||
"standard": 0.226,
|
"standard": 0.172,
|
||||||
"mac": 0.22
|
"mac": 0.183
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -30,14 +30,14 @@
|
|||||||
],
|
],
|
||||||
"handshake": null,
|
"handshake": null,
|
||||||
"status": "ok",
|
"status": "ok",
|
||||||
"latency": 0.366,
|
"latency": 0.298,
|
||||||
"checked_at": "2026-09-21 19:03:04",
|
"checked_at": "2026-10-16 13:46:18",
|
||||||
"source": "probe-2026-08-12",
|
"source": "probe-2026-08-12",
|
||||||
"fast_pool": false,
|
"fast_pool": true,
|
||||||
"checked_date": "2026-09-21",
|
"checked_date": "2026-10-16",
|
||||||
"_per_proto": {
|
"_per_proto": {
|
||||||
"standard": 0.396,
|
"standard": 0.316,
|
||||||
"mac": 0.366
|
"mac": 0.298
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -48,13 +48,13 @@
|
|||||||
],
|
],
|
||||||
"handshake": null,
|
"handshake": null,
|
||||||
"status": "ok",
|
"status": "ok",
|
||||||
"latency": 0.257,
|
"latency": 0.236,
|
||||||
"checked_at": "2026-09-21 19:03:09",
|
"checked_at": "2026-10-16 13:46:23",
|
||||||
"source": "probe-2026-08-12",
|
"source": "probe-2026-08-12",
|
||||||
"fast_pool": true,
|
"fast_pool": true,
|
||||||
"checked_date": "2026-09-21",
|
"checked_date": "2026-10-16",
|
||||||
"_per_proto": {
|
"_per_proto": {
|
||||||
"mac": 0.257
|
"mac": 0.236
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -65,13 +65,13 @@
|
|||||||
],
|
],
|
||||||
"handshake": null,
|
"handshake": null,
|
||||||
"status": "ok",
|
"status": "ok",
|
||||||
"latency": 0.248,
|
"latency": 0.213,
|
||||||
"checked_at": "2026-09-21 19:03:04",
|
"checked_at": "2026-10-16 13:46:17",
|
||||||
"source": "probe-2026-08-12",
|
"source": "probe-2026-08-12",
|
||||||
"fast_pool": true,
|
"fast_pool": true,
|
||||||
"checked_date": "2026-09-21",
|
"checked_date": "2026-10-16",
|
||||||
"_per_proto": {
|
"_per_proto": {
|
||||||
"mac": 0.248
|
"mac": 0.213
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -84,15 +84,15 @@
|
|||||||
],
|
],
|
||||||
"handshake": "setup",
|
"handshake": "setup",
|
||||||
"status": "ok",
|
"status": "ok",
|
||||||
"latency": 0.221,
|
"latency": 0.316,
|
||||||
"checked_at": "2026-09-21 19:03:00",
|
"checked_at": "2026-10-16 13:46:13",
|
||||||
"source": "probe-2026-08-12",
|
"source": "probe-2026-08-12",
|
||||||
"fast_pool": true,
|
"fast_pool": false,
|
||||||
"checked_date": "2026-09-21",
|
"checked_date": "2026-10-16",
|
||||||
"_per_proto": {
|
"_per_proto": {
|
||||||
"standard": 0.399,
|
"standard": 0.316,
|
||||||
"mac": 0.414,
|
"mac": 0.329,
|
||||||
"ex": 0.221
|
"ex": 0.539
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -104,10 +104,10 @@
|
|||||||
"handshake": null,
|
"handshake": null,
|
||||||
"status": "fail",
|
"status": "fail",
|
||||||
"latency": null,
|
"latency": null,
|
||||||
"checked_at": "2026-09-21 19:03:20",
|
"checked_at": "2026-10-16 13:46:34",
|
||||||
"source": "probe-2026-08-12",
|
"source": "probe-2026-08-12",
|
||||||
"fast_pool": false,
|
"fast_pool": false,
|
||||||
"checked_date": "2026-09-21",
|
"checked_date": "2026-10-16",
|
||||||
"_per_proto": {}
|
"_per_proto": {}
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -118,13 +118,13 @@
|
|||||||
],
|
],
|
||||||
"handshake": null,
|
"handshake": null,
|
||||||
"status": "ok",
|
"status": "ok",
|
||||||
"latency": 0.363,
|
"latency": 0.682,
|
||||||
"checked_at": "2026-09-21 19:03:10",
|
"checked_at": "2026-10-16 13:46:24",
|
||||||
"source": "probe-2026-08-12",
|
"source": "probe-2026-08-12",
|
||||||
"fast_pool": true,
|
"fast_pool": false,
|
||||||
"checked_date": "2026-09-21",
|
"checked_date": "2026-10-16",
|
||||||
"_per_proto": {
|
"_per_proto": {
|
||||||
"mac": 0.363
|
"mac": 0.682
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -136,10 +136,10 @@
|
|||||||
"handshake": null,
|
"handshake": null,
|
||||||
"status": "fail",
|
"status": "fail",
|
||||||
"latency": null,
|
"latency": null,
|
||||||
"checked_at": "2026-09-21 19:03:20",
|
"checked_at": "2026-10-16 13:46:34",
|
||||||
"source": "probe-2026-08-12",
|
"source": "probe-2026-08-12",
|
||||||
"fast_pool": false,
|
"fast_pool": false,
|
||||||
"checked_date": "2026-09-21",
|
"checked_date": "2026-10-16",
|
||||||
"_per_proto": {}
|
"_per_proto": {}
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -150,13 +150,13 @@
|
|||||||
],
|
],
|
||||||
"handshake": null,
|
"handshake": null,
|
||||||
"status": "ok",
|
"status": "ok",
|
||||||
"latency": 0.344,
|
"latency": 0.607,
|
||||||
"checked_at": "2026-09-21 19:03:10",
|
"checked_at": "2026-10-16 13:46:23",
|
||||||
"source": "probe-2026-08-12",
|
"source": "probe-2026-08-12",
|
||||||
"fast_pool": true,
|
"fast_pool": false,
|
||||||
"checked_date": "2026-09-21",
|
"checked_date": "2026-10-16",
|
||||||
"_per_proto": {
|
"_per_proto": {
|
||||||
"mac": 0.344
|
"mac": 0.607
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -167,13 +167,13 @@
|
|||||||
],
|
],
|
||||||
"handshake": null,
|
"handshake": null,
|
||||||
"status": "ok",
|
"status": "ok",
|
||||||
"latency": 0.404,
|
"latency": 1.264,
|
||||||
"checked_at": "2026-09-21 19:03:10",
|
"checked_at": "2026-10-16 13:46:24",
|
||||||
"source": "probe-2026-08-12",
|
"source": "probe-2026-08-12",
|
||||||
"fast_pool": false,
|
"fast_pool": false,
|
||||||
"checked_date": "2026-09-21",
|
"checked_date": "2026-10-16",
|
||||||
"_per_proto": {
|
"_per_proto": {
|
||||||
"mac": 0.404
|
"mac": 1.264
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -185,10 +185,10 @@
|
|||||||
"handshake": null,
|
"handshake": null,
|
||||||
"status": "fail",
|
"status": "fail",
|
||||||
"latency": null,
|
"latency": null,
|
||||||
"checked_at": "2026-09-21 19:03:24",
|
"checked_at": "2026-10-16 13:46:34",
|
||||||
"source": "probe-2026-08-12",
|
"source": "probe-2026-08-12",
|
||||||
"fast_pool": false,
|
"fast_pool": false,
|
||||||
"checked_date": "2026-09-21",
|
"checked_date": "2026-10-16",
|
||||||
"_per_proto": {}
|
"_per_proto": {}
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -200,10 +200,10 @@
|
|||||||
"handshake": null,
|
"handshake": null,
|
||||||
"status": "fail",
|
"status": "fail",
|
||||||
"latency": null,
|
"latency": null,
|
||||||
"checked_at": "2026-09-21 19:03:21",
|
"checked_at": "2026-10-16 13:46:34",
|
||||||
"source": "probe-2026-08-12",
|
"source": "probe-2026-08-12",
|
||||||
"fast_pool": false,
|
"fast_pool": false,
|
||||||
"checked_date": "2026-09-21",
|
"checked_date": "2026-10-16",
|
||||||
"_per_proto": {}
|
"_per_proto": {}
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -214,13 +214,13 @@
|
|||||||
],
|
],
|
||||||
"handshake": null,
|
"handshake": null,
|
||||||
"status": "ok",
|
"status": "ok",
|
||||||
"latency": 0.376,
|
"latency": 0.612,
|
||||||
"checked_at": "2026-09-21 19:03:10",
|
"checked_at": "2026-10-16 13:46:23",
|
||||||
"source": "probe-2026-08-12",
|
"source": "probe-2026-08-12",
|
||||||
"fast_pool": false,
|
"fast_pool": false,
|
||||||
"checked_date": "2026-09-21",
|
"checked_date": "2026-10-16",
|
||||||
"_per_proto": {
|
"_per_proto": {
|
||||||
"mac": 0.376
|
"mac": 0.612
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -231,13 +231,13 @@
|
|||||||
],
|
],
|
||||||
"handshake": null,
|
"handshake": null,
|
||||||
"status": "ok",
|
"status": "ok",
|
||||||
"latency": 0.377,
|
"latency": 0.613,
|
||||||
"checked_at": "2026-09-21 19:03:10",
|
"checked_at": "2026-10-16 13:46:23",
|
||||||
"source": "probe-2026-08-12",
|
"source": "probe-2026-08-12",
|
||||||
"fast_pool": false,
|
"fast_pool": false,
|
||||||
"checked_date": "2026-09-21",
|
"checked_date": "2026-10-16",
|
||||||
"_per_proto": {
|
"_per_proto": {
|
||||||
"mac": 0.377
|
"mac": 0.613
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -248,13 +248,13 @@
|
|||||||
],
|
],
|
||||||
"handshake": null,
|
"handshake": null,
|
||||||
"status": "ok",
|
"status": "ok",
|
||||||
"latency": 0.401,
|
"latency": 0.624,
|
||||||
"checked_at": "2026-09-21 19:03:10",
|
"checked_at": "2026-10-16 13:46:23",
|
||||||
"source": "probe-2026-08-12",
|
"source": "probe-2026-08-12",
|
||||||
"fast_pool": false,
|
"fast_pool": false,
|
||||||
"checked_date": "2026-09-21",
|
"checked_date": "2026-10-16",
|
||||||
"_per_proto": {
|
"_per_proto": {
|
||||||
"mac": 0.401
|
"mac": 0.624
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -265,13 +265,13 @@
|
|||||||
],
|
],
|
||||||
"handshake": null,
|
"handshake": null,
|
||||||
"status": "ok",
|
"status": "ok",
|
||||||
"latency": 0.42,
|
"latency": 0.592,
|
||||||
"checked_at": "2026-09-21 19:03:10",
|
"checked_at": "2026-10-16 13:46:23",
|
||||||
"source": "probe-2026-08-12",
|
"source": "probe-2026-08-12",
|
||||||
"fast_pool": false,
|
"fast_pool": false,
|
||||||
"checked_date": "2026-09-21",
|
"checked_date": "2026-10-16",
|
||||||
"_per_proto": {
|
"_per_proto": {
|
||||||
"mac": 0.42
|
"mac": 0.592
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -282,13 +282,13 @@
|
|||||||
],
|
],
|
||||||
"handshake": null,
|
"handshake": null,
|
||||||
"status": "ok",
|
"status": "ok",
|
||||||
"latency": 0.401,
|
"latency": 0.624,
|
||||||
"checked_at": "2026-09-21 19:03:10",
|
"checked_at": "2026-10-16 13:46:23",
|
||||||
"source": "probe-2026-08-12",
|
"source": "probe-2026-08-12",
|
||||||
"fast_pool": false,
|
"fast_pool": false,
|
||||||
"checked_date": "2026-09-21",
|
"checked_date": "2026-10-16",
|
||||||
"_per_proto": {
|
"_per_proto": {
|
||||||
"mac": 0.401
|
"mac": 0.624
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -300,14 +300,14 @@
|
|||||||
],
|
],
|
||||||
"handshake": null,
|
"handshake": null,
|
||||||
"status": "ok",
|
"status": "ok",
|
||||||
"latency": 0.485,
|
"latency": 0.306,
|
||||||
"checked_at": "2026-09-21 19:03:05",
|
"checked_at": "2026-10-16 13:46:18",
|
||||||
"source": "probe-2026-08-12",
|
"source": "probe-2026-08-12",
|
||||||
"fast_pool": false,
|
"fast_pool": false,
|
||||||
"checked_date": "2026-09-21",
|
"checked_date": "2026-10-16",
|
||||||
"_per_proto": {
|
"_per_proto": {
|
||||||
"standard": 0.485,
|
"standard": 0.352,
|
||||||
"mac": 0.605
|
"mac": 0.306
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -319,10 +319,10 @@
|
|||||||
"handshake": null,
|
"handshake": null,
|
||||||
"status": "fail",
|
"status": "fail",
|
||||||
"latency": null,
|
"latency": null,
|
||||||
"checked_at": "2026-09-21 19:03:04",
|
"checked_at": "2026-10-16 13:46:18",
|
||||||
"source": "probe-2026-08-12",
|
"source": "probe-2026-08-12",
|
||||||
"fast_pool": false,
|
"fast_pool": false,
|
||||||
"checked_date": "2026-09-21",
|
"checked_date": "2026-10-16",
|
||||||
"_per_proto": {}
|
"_per_proto": {}
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -334,14 +334,14 @@
|
|||||||
],
|
],
|
||||||
"handshake": null,
|
"handshake": null,
|
||||||
"status": "ok",
|
"status": "ok",
|
||||||
"latency": 0.444,
|
"latency": 0.307,
|
||||||
"checked_at": "2026-09-21 19:03:05",
|
"checked_at": "2026-10-16 13:46:18",
|
||||||
"source": "probe-2026-08-12",
|
"source": "probe-2026-08-12",
|
||||||
"fast_pool": false,
|
"fast_pool": false,
|
||||||
"checked_date": "2026-09-21",
|
"checked_date": "2026-10-16",
|
||||||
"_per_proto": {
|
"_per_proto": {
|
||||||
"standard": 0.444,
|
"standard": 0.35,
|
||||||
"mac": 0.903
|
"mac": 0.307
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -352,13 +352,16 @@
|
|||||||
"mac"
|
"mac"
|
||||||
],
|
],
|
||||||
"handshake": null,
|
"handshake": null,
|
||||||
"status": "fail",
|
"status": "ok",
|
||||||
"latency": null,
|
"latency": 0.395,
|
||||||
"checked_at": "2026-09-21 19:03:15",
|
"checked_at": "2026-10-16 13:46:18",
|
||||||
"source": "probe-2026-08-12",
|
"source": "probe-2026-08-12",
|
||||||
"fast_pool": false,
|
"fast_pool": false,
|
||||||
"checked_date": "2026-09-21",
|
"checked_date": "2026-10-16",
|
||||||
"_per_proto": {}
|
"_per_proto": {
|
||||||
|
"standard": 0.395,
|
||||||
|
"mac": 0.675
|
||||||
|
}
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"ip": "117.34.114.15",
|
"ip": "117.34.114.15",
|
||||||
@@ -369,14 +372,14 @@
|
|||||||
],
|
],
|
||||||
"handshake": null,
|
"handshake": null,
|
||||||
"status": "ok",
|
"status": "ok",
|
||||||
"latency": 0.435,
|
"latency": 0.345,
|
||||||
"checked_at": "2026-09-21 19:03:05",
|
"checked_at": "2026-10-16 13:46:18",
|
||||||
"source": "probe-2026-08-12",
|
"source": "probe-2026-08-12",
|
||||||
"fast_pool": false,
|
"fast_pool": false,
|
||||||
"checked_date": "2026-09-21",
|
"checked_date": "2026-10-16",
|
||||||
"_per_proto": {
|
"_per_proto": {
|
||||||
"standard": 0.463,
|
"standard": 0.345,
|
||||||
"mac": 0.435
|
"mac": 0.726
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -387,13 +390,16 @@
|
|||||||
"mac"
|
"mac"
|
||||||
],
|
],
|
||||||
"handshake": null,
|
"handshake": null,
|
||||||
"status": "fail",
|
"status": "ok",
|
||||||
"latency": null,
|
"latency": 0.326,
|
||||||
"checked_at": "2026-09-21 19:03:14",
|
"checked_at": "2026-10-16 13:46:18",
|
||||||
"source": "probe-2026-08-12",
|
"source": "probe-2026-08-12",
|
||||||
"fast_pool": false,
|
"fast_pool": false,
|
||||||
"checked_date": "2026-09-21",
|
"checked_date": "2026-10-16",
|
||||||
"_per_proto": {}
|
"_per_proto": {
|
||||||
|
"standard": 0.326,
|
||||||
|
"mac": 0.743
|
||||||
|
}
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"ip": "117.34.114.27",
|
"ip": "117.34.114.27",
|
||||||
@@ -404,14 +410,14 @@
|
|||||||
],
|
],
|
||||||
"handshake": null,
|
"handshake": null,
|
||||||
"status": "ok",
|
"status": "ok",
|
||||||
"latency": 0.482,
|
"latency": 0.394,
|
||||||
"checked_at": "2026-09-21 19:03:05",
|
"checked_at": "2026-10-16 13:46:18",
|
||||||
"source": "probe-2026-08-12",
|
"source": "probe-2026-08-12",
|
||||||
"fast_pool": false,
|
"fast_pool": false,
|
||||||
"checked_date": "2026-09-21",
|
"checked_date": "2026-10-16",
|
||||||
"_per_proto": {
|
"_per_proto": {
|
||||||
"standard": 0.482,
|
"standard": 0.394,
|
||||||
"mac": 0.579
|
"mac": 0.78
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -423,14 +429,14 @@
|
|||||||
],
|
],
|
||||||
"handshake": null,
|
"handshake": null,
|
||||||
"status": "ok",
|
"status": "ok",
|
||||||
"latency": 0.609,
|
"latency": 0.327,
|
||||||
"checked_at": "2026-09-21 19:03:05",
|
"checked_at": "2026-10-16 13:46:18",
|
||||||
"source": "probe-2026-08-12",
|
"source": "probe-2026-08-12",
|
||||||
"fast_pool": false,
|
"fast_pool": false,
|
||||||
"checked_date": "2026-09-21",
|
"checked_date": "2026-10-16",
|
||||||
"_per_proto": {
|
"_per_proto": {
|
||||||
"standard": 0.946,
|
"standard": 0.327,
|
||||||
"mac": 0.609
|
"mac": 0.328
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -442,14 +448,14 @@
|
|||||||
],
|
],
|
||||||
"handshake": null,
|
"handshake": null,
|
||||||
"status": "ok",
|
"status": "ok",
|
||||||
"latency": 0.46,
|
"latency": 0.348,
|
||||||
"checked_at": "2026-09-21 19:03:05",
|
"checked_at": "2026-10-16 13:46:18",
|
||||||
"source": "probe-2026-08-12",
|
"source": "probe-2026-08-12",
|
||||||
"fast_pool": false,
|
"fast_pool": false,
|
||||||
"checked_date": "2026-09-21",
|
"checked_date": "2026-10-16",
|
||||||
"_per_proto": {
|
"_per_proto": {
|
||||||
"standard": 0.46,
|
"standard": 0.348,
|
||||||
"mac": 0.521
|
"mac": 0.74
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -461,9 +467,9 @@
|
|||||||
"handshake": null,
|
"handshake": null,
|
||||||
"status": "fail",
|
"status": "fail",
|
||||||
"latency": null,
|
"latency": null,
|
||||||
"checked_at": "2026-09-21 19:03:24",
|
"checked_at": "2026-10-16 13:46:37",
|
||||||
"source": "probe-2026-08-12",
|
"source": "probe-2026-08-12",
|
||||||
"checked_date": "2026-09-21",
|
"checked_date": "2026-10-16",
|
||||||
"fast_pool": false,
|
"fast_pool": false,
|
||||||
"_per_proto": {}
|
"_per_proto": {}
|
||||||
},
|
},
|
||||||
@@ -475,13 +481,13 @@
|
|||||||
],
|
],
|
||||||
"handshake": null,
|
"handshake": null,
|
||||||
"status": "ok",
|
"status": "ok",
|
||||||
"latency": 0.271,
|
"latency": 0.726,
|
||||||
"checked_at": "2026-09-21 19:03:04",
|
"checked_at": "2026-10-16 13:46:18",
|
||||||
"source": "probe-2026-08-12",
|
"source": "probe-2026-08-12",
|
||||||
"checked_date": "2026-09-21",
|
"checked_date": "2026-10-16",
|
||||||
"fast_pool": true,
|
"fast_pool": false,
|
||||||
"_per_proto": {
|
"_per_proto": {
|
||||||
"mac": 0.271
|
"mac": 0.726
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -492,13 +498,13 @@
|
|||||||
],
|
],
|
||||||
"handshake": null,
|
"handshake": null,
|
||||||
"status": "ok",
|
"status": "ok",
|
||||||
"latency": 0.375,
|
"latency": 0.624,
|
||||||
"checked_at": "2026-09-21 19:03:05",
|
"checked_at": "2026-10-16 13:46:18",
|
||||||
"source": "probe-2026-08-12",
|
"source": "probe-2026-08-12",
|
||||||
"checked_date": "2026-09-21",
|
"checked_date": "2026-10-16",
|
||||||
"fast_pool": false,
|
"fast_pool": false,
|
||||||
"_per_proto": {
|
"_per_proto": {
|
||||||
"mac": 0.375
|
"mac": 0.624
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -509,13 +515,13 @@
|
|||||||
],
|
],
|
||||||
"handshake": null,
|
"handshake": null,
|
||||||
"status": "ok",
|
"status": "ok",
|
||||||
"latency": 0.424,
|
"latency": 0.56,
|
||||||
"checked_at": "2026-09-21 19:03:05",
|
"checked_at": "2026-10-16 13:46:30",
|
||||||
"source": "probe-2026-08-12",
|
"source": "probe-2026-08-12",
|
||||||
"checked_date": "2026-09-21",
|
"checked_date": "2026-10-16",
|
||||||
"fast_pool": false,
|
"fast_pool": false,
|
||||||
"_per_proto": {
|
"_per_proto": {
|
||||||
"mac": 0.424
|
"mac": 0.56
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -526,13 +532,13 @@
|
|||||||
],
|
],
|
||||||
"handshake": "login",
|
"handshake": "login",
|
||||||
"status": "ok",
|
"status": "ok",
|
||||||
"latency": 0.176,
|
"latency": 0.097,
|
||||||
"checked_at": "2026-09-21 19:03:08",
|
"checked_at": "2026-10-16 13:46:21",
|
||||||
"source": "probe-2026-08-12",
|
"source": "probe-2026-08-12",
|
||||||
"checked_date": "2026-09-21",
|
"checked_date": "2026-10-16",
|
||||||
"fast_pool": true,
|
"fast_pool": true,
|
||||||
"_per_proto": {
|
"_per_proto": {
|
||||||
"ex": 0.176
|
"ex": 0.097
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -543,13 +549,13 @@
|
|||||||
],
|
],
|
||||||
"handshake": "login",
|
"handshake": "login",
|
||||||
"status": "ok",
|
"status": "ok",
|
||||||
"latency": 0.148,
|
"latency": 0.097,
|
||||||
"checked_at": "2026-09-21 19:03:08",
|
"checked_at": "2026-10-16 13:46:21",
|
||||||
"source": "probe-2026-08-12",
|
"source": "probe-2026-08-12",
|
||||||
"checked_date": "2026-09-21",
|
"checked_date": "2026-10-16",
|
||||||
"fast_pool": true,
|
"fast_pool": true,
|
||||||
"_per_proto": {
|
"_per_proto": {
|
||||||
"ex": 0.148
|
"ex": 0.097
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -560,15 +566,66 @@
|
|||||||
],
|
],
|
||||||
"handshake": "setup",
|
"handshake": "setup",
|
||||||
"status": "ok",
|
"status": "ok",
|
||||||
"latency": 0.166,
|
"latency": 0.103,
|
||||||
"checked_at": "2026-09-21 19:03:08",
|
"checked_at": "2026-10-16 13:46:21",
|
||||||
"source": "probe-2026-08-12",
|
"source": "probe-2026-08-12",
|
||||||
"checked_date": "2026-09-21",
|
"checked_date": "2026-10-16",
|
||||||
"fast_pool": true,
|
"fast_pool": true,
|
||||||
"_per_proto": {
|
"_per_proto": {
|
||||||
"ex": 0.166
|
"ex": 0.103
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"ip": "112.74.214.43",
|
||||||
|
"name": "扩展市场深圳双线1",
|
||||||
|
"protocols": [
|
||||||
|
"ex"
|
||||||
|
],
|
||||||
|
"handshake": "login",
|
||||||
|
"status": "ok",
|
||||||
|
"latency": 0.095,
|
||||||
|
"checked_at": "2026-10-16 13:46:21",
|
||||||
|
"source": "connect.cfg",
|
||||||
|
"fast_pool": true,
|
||||||
|
"checked_date": "2026-10-16",
|
||||||
|
"_per_proto": {
|
||||||
|
"ex": 0.095
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"ip": "120.25.218.6",
|
||||||
|
"name": "扩展市场深圳双线2",
|
||||||
|
"protocols": [
|
||||||
|
"ex"
|
||||||
|
],
|
||||||
|
"handshake": "login",
|
||||||
|
"status": "ok",
|
||||||
|
"latency": 0.101,
|
||||||
|
"checked_at": "2026-10-16 13:46:21",
|
||||||
|
"source": "connect.cfg",
|
||||||
|
"fast_pool": true,
|
||||||
|
"checked_date": "2026-10-16",
|
||||||
|
"_per_proto": {
|
||||||
|
"ex": 0.101
|
||||||
|
}
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"ip": "47.102.108.214",
|
||||||
|
"name": "扩展市场上海双线2",
|
||||||
|
"protocols": [
|
||||||
|
"ex"
|
||||||
|
],
|
||||||
|
"handshake": "login",
|
||||||
|
"status": "ok",
|
||||||
|
"latency": 0.149,
|
||||||
|
"checked_at": "2026-10-16 13:46:22",
|
||||||
|
"source": "connect.cfg",
|
||||||
|
"fast_pool": true,
|
||||||
|
"checked_date": "2026-10-16",
|
||||||
|
"_per_proto": {
|
||||||
|
"ex": 0.149
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
],
|
],
|
||||||
"checked_date": "2026-09-21"
|
"checked_date": "2026-10-16"
|
||||||
}
|
}
|
||||||
@@ -0,0 +1,116 @@
|
|||||||
|
"""ExPool 背压/并发语义回归(Issue #1:并发超限 0.15s 秒失败)。
|
||||||
|
|
||||||
|
用 mock 连接模拟:单服务器 + per_server_cap=1 时,
|
||||||
|
- 并发 acquire 超过有效并发 → 排队等待而不是快速失败;
|
||||||
|
- release 唤醒等待者;
|
||||||
|
- acquire 超时异常带池状态;
|
||||||
|
- ex_pool_cap 提升同主机并发。
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import time
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from pytdxdata.pool.ex_pool import ExPool, ExPoolTimeoutError
|
||||||
|
|
||||||
|
|
||||||
|
class MockConn:
|
||||||
|
"""Minimal stand-in for ExTdxConnection."""
|
||||||
|
|
||||||
|
last_created = 0
|
||||||
|
|
||||||
|
def __init__(self, host: str, port: int, timeout: float, handshake: str = "login"):
|
||||||
|
self.host = host
|
||||||
|
self.port = port
|
||||||
|
self.alive = True
|
||||||
|
self.last_used = time.monotonic()
|
||||||
|
self.closed = False
|
||||||
|
MockConn.last_created += 1
|
||||||
|
|
||||||
|
async def connect(self) -> None:
|
||||||
|
await asyncio.sleep(0) # instant "connect"
|
||||||
|
|
||||||
|
async def close(self) -> None:
|
||||||
|
self.closed = True
|
||||||
|
self.alive = False
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture
|
||||||
|
def patch_conn(monkeypatch):
|
||||||
|
monkeypatch.setattr("pytdxdata.pool.ex_pool.ExTdxConnection", MockConn)
|
||||||
|
MockConn.last_created = 0
|
||||||
|
|
||||||
|
|
||||||
|
async def test_acquire_queues_instead_of_fast_fail(patch_conn):
|
||||||
|
"""并发 > 有效并发时,后续 acquire 等待 release,而非 0.15s 抛错。"""
|
||||||
|
pool = ExPool(["e1"], min_size=1, max_size=6, per_server_cap=1)
|
||||||
|
await pool.start()
|
||||||
|
c1 = await pool.acquire() # 唯一连接被占用
|
||||||
|
t0 = time.monotonic()
|
||||||
|
|
||||||
|
async def holder():
|
||||||
|
await asyncio.sleep(0.5) # 持有 0.5s 后归还
|
||||||
|
await pool.release(c1)
|
||||||
|
|
||||||
|
asyncio.create_task(holder())
|
||||||
|
c2 = await pool.acquire() # 应在 ~0.5s 后拿到,而不是 0.15s 抛错
|
||||||
|
waited = time.monotonic() - t0
|
||||||
|
assert waited >= 0.4, f"acquire 没有等待释放(仅 {waited:.2f}s)"
|
||||||
|
assert c2 is c1 # 归还的是同一条连接
|
||||||
|
|
||||||
|
|
||||||
|
async def test_acquire_timeout_carries_pool_state(patch_conn):
|
||||||
|
"""超时异常信息带池状态(idle/total/servers/cap)。"""
|
||||||
|
pool = ExPool(["e1"], min_size=1, max_size=6, per_server_cap=1, acquire_timeout=0.3)
|
||||||
|
await pool.start()
|
||||||
|
c1 = await pool.acquire()
|
||||||
|
with pytest.raises(ExPoolTimeoutError) as ei:
|
||||||
|
await pool.acquire()
|
||||||
|
msg = str(ei.value)
|
||||||
|
assert "total=1" in msg and "per_server_cap=1" in msg and "servers=1" in msg
|
||||||
|
await pool.release(c1)
|
||||||
|
|
||||||
|
|
||||||
|
async def test_ex_pool_cap_allows_parallel_conns_same_host(patch_conn):
|
||||||
|
"""per_server_cap=2 时单主机可并发持有 2 条连接。"""
|
||||||
|
pool = ExPool(["e1"], min_size=1, max_size=6, per_server_cap=2)
|
||||||
|
await pool.start()
|
||||||
|
got = await asyncio.gather(pool.acquire(), pool.acquire())
|
||||||
|
assert len(got) == 2 and got[0] is not got[1]
|
||||||
|
assert MockConn.last_created == 2
|
||||||
|
for c in got:
|
||||||
|
await pool.release(c)
|
||||||
|
|
||||||
|
|
||||||
|
async def test_concurrent_acquire_all_succeed(patch_conn):
|
||||||
|
"""Issue 场景:单服务器 + cap=2,6 个并发任务应全部拿到连接(排队)而非部分失败。"""
|
||||||
|
pool = ExPool(["e1"], min_size=2, max_size=6, per_server_cap=2)
|
||||||
|
await pool.start()
|
||||||
|
|
||||||
|
async def task(i):
|
||||||
|
c = await pool.acquire()
|
||||||
|
await asyncio.sleep(0.1) # 模拟一次 1m 分页拉取
|
||||||
|
await pool.release(c)
|
||||||
|
return i
|
||||||
|
|
||||||
|
results = await asyncio.gather(*[task(i) for i in range(6)])
|
||||||
|
assert sorted(results) == list(range(6)) # 全部成功,零失败
|
||||||
|
|
||||||
|
|
||||||
|
async def test_release_notifies_waiters(patch_conn):
|
||||||
|
"""broken 归还(连接死亡)也要唤醒等待者(槽位回收后可扩容)。"""
|
||||||
|
pool = ExPool(["e1"], min_size=1, max_size=1, per_server_cap=1)
|
||||||
|
await pool.start()
|
||||||
|
c1 = await pool.acquire()
|
||||||
|
c1.alive = False
|
||||||
|
|
||||||
|
async def broken_release():
|
||||||
|
await asyncio.sleep(0.2)
|
||||||
|
await pool.release(c1, broken=True)
|
||||||
|
|
||||||
|
asyncio.create_task(broken_release())
|
||||||
|
c2 = await pool.acquire() # 槽位被回收→扩容新建
|
||||||
|
assert c2 is not c1
|
||||||
|
await pool.release(c2)
|
||||||
+114
-4
@@ -2,17 +2,20 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import struct
|
import struct
|
||||||
|
from collections import Counter
|
||||||
|
from datetime import date, datetime
|
||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
from pytdxdata.protocol.frame import FrameHeader, decompress_body, parse_header
|
from pytdxdata.models import Adjust, KlinePeriod, SecurityBar
|
||||||
from pytdxdata.protocol.price import get_price, put_price, decode_ohlc_diffs
|
|
||||||
from pytdxdata.protocol.volume import decode_volume
|
|
||||||
from pytdxdata.protocol.commands.ex_proto import GetExInstrumentBarsCmd
|
from pytdxdata.protocol.commands.ex_proto import GetExInstrumentBarsCmd
|
||||||
from pytdxdata.protocol.commands.kline import GetSecurityBarsCmd
|
from pytdxdata.protocol.commands.kline import GetSecurityBarsCmd
|
||||||
|
from pytdxdata.protocol.commands.mac import MacKlineCmd
|
||||||
from pytdxdata.protocol.commands.minute import GetHistoryMinuteTimeDataCmd
|
from pytdxdata.protocol.commands.minute import GetHistoryMinuteTimeDataCmd
|
||||||
from pytdxdata.protocol.commands.transaction import MacTransactionCmd
|
from pytdxdata.protocol.commands.transaction import MacTransactionCmd
|
||||||
|
from pytdxdata.protocol.frame import FrameHeader, decompress_body, parse_header
|
||||||
|
from pytdxdata.protocol.price import decode_ohlc_diffs, get_price, put_price
|
||||||
|
from pytdxdata.protocol.volume import decode_volume
|
||||||
|
|
||||||
# ---------- 变长有符号整数 ----------
|
# ---------- 变长有符号整数 ----------
|
||||||
|
|
||||||
@@ -176,3 +179,110 @@ def test_mac_transaction_parse():
|
|||||||
assert recs[0].trade_count == 3
|
assert recs[0].trade_count == 3
|
||||||
assert recs[0].bs_flag == 1
|
assert recs[0].bs_flag == 1
|
||||||
assert recs[1].trade_count == 1
|
assert recs[1].trade_count == 1
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- 跨时区时间戳归一化(美股 EX 通道) ----------
|
||||||
|
#
|
||||||
|
# 通达信 EX 通道对美股返回的时间戳把两个时区拼在一起:日期字段是**美东交易日**、
|
||||||
|
# 时分字段是**北京时间**。直拼出的 datetime 会把同一交易日的下午半场(北京
|
||||||
|
# 00:00~04:00)排到上午半场(北京 21:30~23:59)之前,单日 390 根全部错位。
|
||||||
|
|
||||||
|
def _bar(market: int, y: int, mo: int, d: int, h: int, mi: int) -> SecurityBar:
|
||||||
|
return SecurityBar(market=market, code="X", open=1.0, high=1.0, low=1.0,
|
||||||
|
close=1.0, vol=1.0, amount=1.0,
|
||||||
|
year=y, month=mo, day=d, hour=h, minute=mi)
|
||||||
|
|
||||||
|
|
||||||
|
def test_us_evening_segment_keeps_date():
|
||||||
|
# 北京 21:31 ~ 23:59 与美东交易日属同一自然日,日期不动
|
||||||
|
b = _bar(74, 2026, 10, 12, 21, 31)
|
||||||
|
assert b.datetime == datetime(2026, 10, 12, 21, 31)
|
||||||
|
assert b.trading_date == "2026-10-12"
|
||||||
|
|
||||||
|
|
||||||
|
def test_us_midnight_segment_rolls_to_next_day():
|
||||||
|
# 北京次日 00:00 ~ 04:00 实际是美东 12:00~16:00,日期要 +1 天
|
||||||
|
b = _bar(74, 2026, 10, 12, 4, 0)
|
||||||
|
assert b.datetime == datetime(2026, 10, 13, 4, 0)
|
||||||
|
assert (b.hour, b.minute) == (4, 0) # 时分不参与归一化
|
||||||
|
assert b.trading_date == "2026-10-12" # 仍归属美东 10-12 交易日
|
||||||
|
|
||||||
|
|
||||||
|
def test_us_split_hour_boundary():
|
||||||
|
assert _bar(74, 2026, 10, 12, 11, 59).day == 13
|
||||||
|
assert _bar(74, 2026, 10, 12, 12, 0).day == 12
|
||||||
|
|
||||||
|
|
||||||
|
def test_us_month_and_year_rollover():
|
||||||
|
assert _bar(74, 2026, 10, 31, 4, 0).datetime == datetime(2026, 11, 1, 4, 0)
|
||||||
|
assert _bar(74, 2026, 12, 31, 4, 0).datetime == datetime(2027, 1, 1, 4, 0)
|
||||||
|
|
||||||
|
|
||||||
|
def test_cn_and_hk_bars_untouched():
|
||||||
|
cn = _bar(1, 2026, 10, 12, 9, 31) # A股/标准通道
|
||||||
|
assert (cn.year, cn.month, cn.day) == (2026, 10, 12)
|
||||||
|
assert cn.trading_date == "2026-10-12"
|
||||||
|
hk = _bar(31, 2026, 10, 12, 4, 0) # 港股同为 UTC+8,凌晨段不应 +1
|
||||||
|
assert (hk.year, hk.month, hk.day) == (2026, 10, 12)
|
||||||
|
assert hk.trading_date == "2026-10-12"
|
||||||
|
|
||||||
|
|
||||||
|
def test_us_full_session_is_monotonic_and_single_trading_day():
|
||||||
|
# 实测口径:北京 21:31~23:59(149 根)+ 次日 00:00~04:00(241 根)= 390 根
|
||||||
|
mins = list(range(21 * 60 + 31, 24 * 60)) + list(range(4 * 60 + 1))
|
||||||
|
assert len(mins) == 390
|
||||||
|
bars = [_bar(74, 2026, 10, 12, m // 60, m % 60) for m in mins]
|
||||||
|
bars.sort(key=lambda b: b.datetime)
|
||||||
|
dts = [b.datetime for b in bars]
|
||||||
|
assert dts == sorted(dts) # 归一化后严格单调
|
||||||
|
assert dts[0] == datetime(2026, 10, 12, 21, 31)
|
||||||
|
assert dts[-1] == datetime(2026, 10, 13, 4, 0)
|
||||||
|
# 全部归属于同一个美东交易日
|
||||||
|
assert {b.trading_date for b in bars} == {"2026-10-12"}
|
||||||
|
# 但横跨两个北京自然日(这正是不能用 datetime.date() 聚合的原因)
|
||||||
|
by_day = Counter(b.datetime.date() for b in bars)
|
||||||
|
assert by_day[date(2026, 10, 12)] == 149
|
||||||
|
assert by_day[date(2026, 10, 13)] == 241
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- MAC 通道 K线(0x122E)同样归一化 ----------
|
||||||
|
|
||||||
|
def _mac_bars_body(records: list[tuple[int, int]]) -> bytes:
|
||||||
|
"""构造 0x122E 响应体:33B头(count@24) + 每条36B(4B ymd + 4B time秒 + 7float)。"""
|
||||||
|
body = bytearray(33 + len(records) * 36)
|
||||||
|
struct.pack_into("<HBHI", body, 24, 7, 1, len(records), 0)
|
||||||
|
for i, (ymd, tsec) in enumerate(records):
|
||||||
|
struct.pack_into("<II7f", body, 33 + i * 36,
|
||||||
|
ymd, tsec, 10.0, 10.5, 9.5, 10.2, 0.0, 1000.0, 0.0)
|
||||||
|
return bytes(body)
|
||||||
|
|
||||||
|
|
||||||
|
def test_mac_kline_us_bar_normalized():
|
||||||
|
# _probe_bar(日期区间二分依赖它)对美股走的就是这条命令
|
||||||
|
cmd = MacKlineCmd(74, "AAPL", KlinePeriod.MIN_1, Adjust.NONE, 0, 5)
|
||||||
|
bars = cmd.parse_response(_mac_bars_body([(20261012, 4 * 3600)])) # 北京 04:00
|
||||||
|
assert len(bars) == 1
|
||||||
|
assert bars[0].datetime == datetime(2026, 10, 13, 4, 0)
|
||||||
|
assert bars[0].trading_date == "2026-10-12"
|
||||||
|
|
||||||
|
|
||||||
|
def test_mac_kline_hk_bar_untouched():
|
||||||
|
cmd = MacKlineCmd(31, "00700", KlinePeriod.MIN_1, Adjust.NONE, 0, 5)
|
||||||
|
bars = cmd.parse_response(_mac_bars_body([(20260729, 13 * 3600 + 40 * 60)]))
|
||||||
|
assert (bars[0].datetime) == datetime(2026, 7, 29, 13, 40)
|
||||||
|
assert bars[0].trading_date == "2026-07-29"
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- 排序:EX 返回顺序按时间正序(切片取「最新 N 根」依赖它) ----------
|
||||||
|
|
||||||
|
def test_finish_bars_sorts_us_session_chronologically():
|
||||||
|
from pytdxdata.api import TdxData
|
||||||
|
from pytdxdata.symbols import parse_symbols
|
||||||
|
|
||||||
|
sym = parse_symbols("usAAPL")[0]
|
||||||
|
# 故意按错误的顺序给(凌晨段在前)
|
||||||
|
out = TdxData._finish_bars([_bar(74, 2026, 10, 12, 4, 0),
|
||||||
|
_bar(74, 2026, 10, 12, 21, 31)], sym)
|
||||||
|
assert [b.datetime for b in out] == [datetime(2026, 10, 12, 21, 31),
|
||||||
|
datetime(2026, 10, 13, 4, 0)]
|
||||||
|
assert {b.symbol for b in out} == {"usAAPL"}
|
||||||
Reference in new issue
Block a user