Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
04e0c57569 | ||
|
|
2cf3657f4b | ||
|
|
dc78d81be8 | ||
|
|
8a78156145 | ||
|
|
2c07400a3a | ||
|
|
76644919de | ||
|
|
b84a6c8a1b | ||
|
|
ca24521d2c | ||
|
|
8344ed6e6b | ||
|
|
dfd6fab3af | ||
|
|
0618062e54 | ||
|
|
9c97fea71b | ||
|
|
2d62d82b51 | ||
|
|
7e72f70eb2 | ||
|
|
b2f899df49 | ||
|
|
8193d560da | ||
|
|
5408903a4c | ||
|
|
8951ec2678 | ||
|
|
b78d0fabf5 | ||
|
|
d422385877 | ||
|
|
7e830c7579 | ||
|
|
c7c6c70333 | ||
|
|
18d9e9a006 | ||
|
|
3e0beb27a6 |
No files matched your search
+728
-1
@@ -2,6 +2,732 @@
|
||||
|
||||
本项目遵循 [语义化版本](https://semver.org/lang/zh-CN/)。
|
||||
|
||||
## [0.7.0] - 2026-10-28
|
||||
|
||||
### 新增
|
||||
|
||||
- **DataFrame 直出:`TdxData(df="polars")` / `TdxData(df="pandas")`**。
|
||||
|
||||
表格型接口不再只能返回 `list[dataclass]`,构造时声明一次后端即可直接拿到 DataFrame:
|
||||
|
||||
```python
|
||||
async with TdxData(df="polars") as td:
|
||||
df = await td.get_bars("sh600000", KlinePeriod.DAY, count=240)
|
||||
```
|
||||
|
||||
- 覆盖 **42 个**返回表格数据的方法(33 个 `list[dataclass]`、2 个 `list[dict]`、
|
||||
2 个批量 `dict[str, list[...]]`、`get_indicators` 与 `get_index_trade_stats(_multi)`),
|
||||
另含 deprecated 别名(`get_kline` / `get_transactions` / `get_goods_*` 等)。
|
||||
- **核心包仍然零依赖**:pandas / polars 是可选 extras ——
|
||||
`pip install "pytdxdata[polars]"` / `[pandas]` / `[dataframe]`。
|
||||
- `df=None`(默认)**行为与 0.6.x 完全一致**,且装饰器直接透传,零额外开销。
|
||||
- 取值非法抛 `ValueError` 并列出可用值;后端未安装抛 `FrameBackendError`(继承 `ImportError`,
|
||||
`e.name` 为缺的包名,消息里带安装命令)——**在构造时就失败**,不会拖到第一次查询。
|
||||
|
||||
- **列口径**(三条约定,一次定死):
|
||||
|
||||
| 约定 | 说明 |
|
||||
|---|---|
|
||||
| 时间列替换 | `SecurityBar`/`IndexBar`/`MinuteBar` 的 `year/month/day/hour/minute` 不再单列输出,统一给 `datetime`(polars `Datetime('us')` / pandas `datetime64[ns]`)与 `trading_date`;`XdxrRecord` 给 `date` |
|
||||
| 嵌套定型 | `SecurityQuote.bid`/`ask` 展开成 `bid1_price`…`ask5_vol` 共 10 列(缺档 `None`);`fields` 字典 JSON 化成字符串列(空字典给 `None`);`TdxBlock.codes` 保持列表 |
|
||||
| 容器展开 | `get_indicators` → bars + 各指标一列(长度不足补 `None`);`get_index_trade_stats(_multi)` → `minutes` 展开并补 `index`/`date` 常量列;`get_kline_batch`/`get_minute_batch` → 各标的纵向拼接成一张表 |
|
||||
|
||||
- **嵌套调用不二次转换**:`get_kline_batch` 这类方法内部还会调 `get_bars`,
|
||||
用 `ContextVar`(按 task 隔离)记录层次,内层一律返回原始 `list`,只有最外层那次才转 DataFrame。
|
||||
并发调用互不干扰。
|
||||
|
||||
### 性能
|
||||
|
||||
- **按日期区间取 K 线:右端贴最新时改走「倒推」,1 分钟 K 线 4.2s → 82ms(51 倍)**。
|
||||
|
||||
原先 `start_date` / `end_date` 一律走「探总根数 → 二分定位偏移」。1 分钟 K 线历史总量
|
||||
二十万根量级,等于 6 次翻倍探测 + 2 次方向探测 + **两次各 ~18 次**跳转,共 36+ 次串行往返。
|
||||
而「取今天」「增量到最新」这类占绝对多数的调用,右端本来就贴着最新,根本不需要二分。
|
||||
|
||||
| 场景(1 分钟,冷启动) | 优化前 | 优化后 |
|
||||
|---|---|---|
|
||||
| 单只「取今天」 | 4,208 ms | **82 ms** |
|
||||
| 单只「2 天前 ~ now」 | 2,211 ms | **75 ms** |
|
||||
| 单只「取最近 3 天」 | 2,000 ms+ | **239 ms** |
|
||||
| 单只「历史中段(5~3 天前)」 | 1,512 ms | 285 ms |
|
||||
| 全市场 5,225 只「取今天」 | — | 32.1 s(1,140,699 根,**精确当天口径**) |
|
||||
|
||||
做法:右端离最新不超过 `_BACKTRACK_GRACE`(10 天,覆盖长假)时,直接
|
||||
`start=0, count=N` 拿最新 N 根再裁剪 —— N 按时间跨度估算(`_BARS_PER_DAY`),
|
||||
一天内一次就命中,不够才翻倍。历史中段仍走二分,结果完全一致。
|
||||
|
||||
偏移方向是**通道级属性**(与具体标的无关),首次探测后缓存在实例上:
|
||||
全市场 5,225 只批量调用只付一次探测成本,不会每只探两遍。
|
||||
|
||||
- **多标的 K 线改「多连接 × 单连接流水线」抓取,全市场 1 分钟 K 线 43.6s → 5s 级
|
||||
(盘中增量 1.5s)**。
|
||||
|
||||
单只 K 线是「一请求一标的」,5,000 只就是 5,000 次往返。实测单连接串行 **33ms/只
|
||||
≈ 恰好一个 RTT** —— 瓶颈是**往返次数,不是带宽**(此前「单 IP 限流 1.5MB/s」的
|
||||
结论是浅流水下的假象:同一台服务器 16 连接 × 深度 8 实测能跑到 5.2 MB/s)。
|
||||
|
||||
做法:同一连接连续下发 `pipeline` 条请求(协议帧自描述长度且服务器**按序应答**,
|
||||
无需等前一个回来),再叠加多台服务器并行(每台 1 连接,尊重 `per_server_cap`)。
|
||||
|
||||
| 场景(全市场 5,225 只,1 分钟) | 优化前(逐只 `concurrency=6`) | 优化后 |
|
||||
|---|---|---|
|
||||
| 取当天全天(240 根) | 43.6 s | **5~8 s** |
|
||||
| 盘中 10:00(35 根) | 15 s+ | **~2 s** |
|
||||
| 盘中增量(最新 2 根) | 12 s+ | **~1.5 s** |
|
||||
|
||||
- 流水深度默认**自适应**(`pipeline=None`):按单请求字节数把在途量控制在 ~60KB,
|
||||
夹在 8~32 之间 —— 取 240 根用深度 8,只取最新 2 根用深度 32。
|
||||
实测过深反而更慢(24 台 depth=64 比 depth=8 慢一倍,服务器会攒够一批才回)。
|
||||
- 服务器**只取最快的 12 台**:池里 24 台全用反而被慢节点拖长尾
|
||||
(`count=2` 全市场:24 台 2.95s / 12 台 1.45s)。
|
||||
- **盘中只拉已走过的根数**:`start_date=今天` 时按交易时段折算(10:00 → 30 根,
|
||||
而不是 240 根),数据量直接差 8 倍。收盘后回到一整天的量级。
|
||||
- 标准通道(日线不复权)同样走批量:全市场日线 240 根 49~72s → **~10s**。
|
||||
- 流水线中途失败时该批**交回逐只重试**(流水线失败意味着响应流已错配、连接作废),
|
||||
连接全挂时队列里剩下的标的也一并回退 —— 不会静默少数据(新增测试覆盖)。
|
||||
- `pipeline=1` 可关闭批量路径,行为与旧版逐只路径一致。
|
||||
|
||||
- **修正「右端是否贴最新」的时间戳比较**:`0x122E` 返回的 `hour/minute` 是**该周期的
|
||||
结束时刻**,最新一根可能比 `datetime.now()` 略大(14:29 那根此刻还在走)。
|
||||
原先没有容差,`end_date=datetime.now()` 这类最自然的写法会被误判成「历史中段」而走慢路径。
|
||||
|
||||
### 修复
|
||||
|
||||
- **EX 通道 K 线的 `amount` 恒为 0.0,且末位字段(结算价)被解包丢弃**。
|
||||
|
||||
`0x23FF` 一条记录 32B = 4B 时间戳 + **28B 行情**,这 28B 只有**一个** 4B 槽位可复用,
|
||||
语义按市场分两类(全市场实测,用 `成交额 / (close × vol)` 逐条校验):
|
||||
|
||||
| 市场 | 槽位 5 | 槽位 6 | 槽位 7 | 校验 |
|
||||
|---|---|---|---|---|
|
||||
| 港股主板 31 / 港股基金 49 | **成交额**(f) | 成交量(手=100 股) | 未知 | 额/(c·vol) = **100.0** |
|
||||
| 港股创业板 48 / 新三板 44 / 美股 74 | **成交额**(f) | 成交量(股) | 0 | 额/(c·vol) ≈ **1.0** |
|
||||
| 期货 / 期权(7/8/9/47…) | 持仓量(I) | 成交量 | **结算价**(f) | 与 CFFEX 官网日结逐日相符 |
|
||||
|
||||
改动前:`amount` 被硬编码成 `0.0`、第 7 个 float 用 `_` 丢弃 —— 于是港股/美股/新三板
|
||||
**明明帧内有成交额却全是 0**,期货期权的结算价也拿不到(用户只能换源)。
|
||||
|
||||
- 新增 `SecurityBar.settle`(结算价,`float | None`)。仅 EX 期货/期权**日线**有值,
|
||||
分钟线该槽位恒 0 已归一化为 `None`;其它通道恒 `None`。
|
||||
- 权益类市场的 `amount` 回填真实成交额。⚠️ 港股主板/基金的 `vol` 单位是**手(=100 股)**,
|
||||
`amount/vol` 是均价的 **100 倍**。
|
||||
- 期货/期权的 `amount` **仍是 0.0** —— 帧内确实没有成交额,这是协议限制,已在
|
||||
`SecurityBar` 与文档中写清,避免使用者再按文档误判为调用方 bug。
|
||||
- 港股指数 27 / 中证指数 62 该槽位量级对不上(既非成交额也非结算价),保守保持 `0.0`。
|
||||
- `SecurityBar.__setstate__` 兜底旧磁盘缓存:缓存键没有 schema 版本,升级后读旧 pickle
|
||||
会缺 `settle` 槽位,默认反序列化不补默认值 → 直接 `AttributeError`,已容错。
|
||||
EX K 线缓存键额外加 `@v2` 标记,让 `amount` 恒 0 的旧缓存自然失效。
|
||||
|
||||
### 兼容性
|
||||
|
||||
- 无破坏性变更。不传 `df` 时所有方法签名、返回类型、行为与 0.6.9 完全相同。
|
||||
- 空结果返回**列名齐全的空表**(列由模型定义推出),dtype 由后端自行推断。
|
||||
- `SecurityBar` 新增可选字段 `settle`(默认 `None`):K 线 DataFrame 因此多一列 `settle`,
|
||||
位置参数构造的老代码不受影响。
|
||||
|
||||
### 升级
|
||||
|
||||
```bash
|
||||
pip install -U pytdxdata # 只要 dataclass
|
||||
pip install -U "pytdxdata[polars]" # 想直出 DataFrame
|
||||
```
|
||||
|
||||
- **磁盘缓存不需要手动清**:`SecurityBar` 的 `settle` 由 `__setstate__` 兜底补默认值,
|
||||
旧缓存照常可读;EX K 线缓存键已带 `@v2`,`amount` 恒 0 的旧记录会自动失效重取。
|
||||
- 新参数 `pipeline`(`get_bars`)默认自适应,**不传就是最优**;只有想显式退回旧的逐只路径时才传 `1`。
|
||||
|
||||
|
||||
### 文档
|
||||
|
||||
- 新增 [DataFrame 输出](https://openbot-coder.github.io/pytdxdata/dataframe/) 专页,
|
||||
`API_REFERENCE.md`(构造参数表补 `df` 与 `ex_pool_cap`、`pool_max` 默认值修正为 24)、
|
||||
`SKILL.md`、`llms.txt`、`README.md`、`mkdocs.yml` 导航同步。
|
||||
- 全站同步 0.7.0 口径:首页特性表与导航卡片、安装页(extras 与构造参数表补
|
||||
`df` / `std_pool_cap` / `ex_pool_cap`)、场景示例的「转成 pandas」改为推荐 `df=` 开关、
|
||||
数据模型页说明 dataclass 与 DataFrame 是同一份数据、README「已知限制」补 EX 成交额口径。
|
||||
|
||||
## [0.6.9] - 2026-10-23
|
||||
|
||||
### 修复
|
||||
|
||||
- **`get_universe` 冷启动 12~18s / 偶发失败:证券清单请求多发了 2 字节,污染连接**。
|
||||
|
||||
`GetSecurityListCmd` 的请求体是「12 字节命令头 + market(H) + start(H)」= **16 字节**,
|
||||
但实现里多塞了一个 `H`(写死 0),实际发出 **18 字节**。服务端只消费前 16 字节,
|
||||
剩下的 `00 00` 停在它的接收缓冲里,把**下一个请求**的开头吃掉 —— 于是同一条连接
|
||||
「第一次成功、第二次起永远无响应」。
|
||||
|
||||
分页器复用连接,等于每拉一页都要先失败重连,冷启动被拖到 12~18s,还经常直接抛
|
||||
`ConnectionError`。
|
||||
|
||||
| 场景 | 修复前 | 修复后 |
|
||||
|---|---|---|
|
||||
| `get_universe("sh")` 冷启动(27,534 条 / 28 页) | 12,267ms | **1,610ms** |
|
||||
| `get_universe("sz")` 冷启动(24,460 条 / 25 页) | 18,260ms | **571ms** |
|
||||
| `get_universe()` 全市场冷启动(146,848 条) | 19,085ms | **4,371ms** |
|
||||
| `get_universe()` 热缓存 | — | 669ms |
|
||||
| 同一连接连发 2 次清单(27 台主机) | **0/27 成功** | **27/27 成功** |
|
||||
|
||||
- **北交所清单不再卡 15s 后报错,改为立即返回空列表**。
|
||||
|
||||
`0x053E` 对 `market=2` 在实测的 **27 台标准主机上全部无响应**(26 台 6s 无回包、
|
||||
1 台立刻拒连),而同市场的 `0x053D` 数量接口正常(返回 389)。原实现会重试 2 次、
|
||||
每次 3s 超时,最后让 `get_universe()`(不带参数)整批失败。
|
||||
|
||||
现在该市场直接返回空列表并 `WARNING` 一次,**北交所标的本身不受影响**:
|
||||
`get_quotes("bj920992")` / `get_bars` / 逐笔全部正常(实测 11.79 与日线 close 一致)。
|
||||
|
||||
### 优化
|
||||
|
||||
- **`get_universe()` 的 EX 全量只取一次**:命中缓存也要反序列化 10 万条(~200ms),
|
||||
原先十几个 EX 市场各来一次,纯属白烧。
|
||||
- **EX 商品列表改并发翻页**(110 页,8 并发):全市场冷启动的一半时间花在这里。
|
||||
|
||||
### 文档
|
||||
|
||||
- `get_universe` docstring 写明北交所限制;`API_REFERENCE` / `llms.txt` / `SKILL.md` 同步。
|
||||
- 新增 `tests/test_universe.py` 7 项:请求长度锁定 16 字节、无尾部 padding、
|
||||
北交所返回空且**不碰连接池**、`_ex_all` 只取一次。
|
||||
|
||||
### 已知限制(非本次引入)
|
||||
|
||||
- EX 全量列表含**非证券条目**(如人员名录),`(market, code)` 在原始数据里就不唯一,
|
||||
且重复项出现在**同一页内部**。属数据源特性,未做去重(去重会丢真实品种)。
|
||||
|
||||
## [0.6.8] - 2026-10-23
|
||||
|
||||
### 修复
|
||||
|
||||
- **CLI 价格按品种小数位显示;`tdx minute` 时间列改回真实时钟**。
|
||||
|
||||
两处纯显示层缺陷,数据本身没问题(分时/逐笔数值见 0.6.7 的核对结论):
|
||||
|
||||
| 现象 | 原因 |
|
||||
|---|---|
|
||||
| `tdx transactions sh113708` 打印 `128.89` | 价格写死 `%8.2f`;可转债报价步长 0.001 元,末位被截掉 |
|
||||
| `tdx minute sh113708` 时间列 `00:00~03:59` | 按序号推算 `i // 60, i % 60`,与 bar 自带时间无关 |
|
||||
| `tdx quotes sh113708` 打印 `128.8920` | `price_decimal_digits` 是**解码位数**(原始整数 / 10⁴),第 4 位恒为 0 |
|
||||
|
||||
| 位置 | 变化 |
|
||||
|---|---|
|
||||
| `_price_fmt(dp)`(新) | 宽度随小数位走(`%8.2f` → `%9.3f`),不破坏表格对齐 |
|
||||
| `_display_dp(market, code)`(新) | 可转债(沪 `110/111/113/118`、深 `123/127/128`)由解码 4 位降到显示 3 位;其余沿用 `price_decimal_digits`(ETF 3 位、国债/回购 4 位、股票/指数 2 位) |
|
||||
| `cmd_kline` / `cmd_transactions` / `cmd_minute` / `_print_quote` | 统一改用上面的格式串 |
|
||||
| `cmd_minute` | 时间取 `MinuteBar.hour/minute`,不再用序号推算 |
|
||||
| `cmd_transactions` / `cmd_minute` | 改调 `get_ticks` / `get_minutes`,消掉 `get_transactions` / `get_minute` 的 DeprecationWarning |
|
||||
|
||||
新增回归测试 8 项(`tests/test_cli_format.py`)。
|
||||
|
||||
文档:`docs/cli.md` 补「价格小数位按品种适配」对照表与 `tdx minute` 时间列说明。
|
||||
|
||||
## [0.6.7] - 2026-10-22
|
||||
|
||||
### 性能
|
||||
|
||||
- **全市场 A股报价 3.5s → 0.33s:单连接流水线 + 工作窃取**(5,227 只 / 66 片实测)。
|
||||
|
||||
老实现 `get_quotes` 是**纯串行 for 循环**逐片发请求:协议硬上限 80 只/请求,
|
||||
全市场必须切 66 片,每片一次往返(≈60ms)→ 3.5s 全花在等网络上。
|
||||
|
||||
关键发现:通达信响应是**自描述长度帧**(16B 帧头 + body)且服务器**按序应答**,
|
||||
因此同一 TCP 连接上可以连续下发多条请求、只付一次往返。实测单连接流水 66 个请求
|
||||
只要 526ms(串行 7,372ms),响应顺序与代码完全对得上(用昨收价交叉比对 30 只,零错配)。
|
||||
|
||||
| 位置 | 变化 |
|
||||
|---|---|
|
||||
| `TdxConnection.execute_pipeline(cmds)`(新) | 一次 write 全部请求 → 按序读回全部响应。中途失败**强制断连**(响应流已错配,不可复用) |
|
||||
| `PooledConnection.execute_pipeline`(新) | 池化包装转发 |
|
||||
| `api._execute_std_pipeline`(新) | 借一条连接流水一整组分片;节点不支持流水线时自动降级为逐片 |
|
||||
| `get_quotes(..., pipeline=True, concurrency=None)` | 新参数;小批量(≤80 只)仍走原路径,零额外开销 |
|
||||
| `_quotes_std` | 改为**工作窃取**:分片放队列,连接做完一批回来再取,快的多干、慢的少干 |
|
||||
| `DynamicPool.warm(n)`(新) | **并发**预建连接。`acquire()` 建连在全局锁内一次一条,22 并发首次会串行握手 22 次 —— 全市场首拉因此多花 5~6s;预热后降到 0.78s |
|
||||
| `CacheManager.get_or_fetch_obj`(新) | 回源直接返回对象,省掉冷路径「pickle→立刻 unpickle」的一次白烧(5,227 条实测 ~40ms) |
|
||||
| `TdxData(pool_max=)` | 默认 12 → 24,让报价并发能到「每台 2 连接」(池仍是懒扩容,不影响其他接口) |
|
||||
|
||||
调优实测(5,227 只 A股,11 台服务器,p50/max):
|
||||
|
||||
| 并发 | 每台连接 | 流水深度 | p50 | max |
|
||||
|---|---|---|---|---|
|
||||
| 11 | 1 | 6 | 455ms | 464ms |
|
||||
| 12 | 1~2 | 3 | 457ms | 523ms |
|
||||
| **22** | **2** | **3** | **325ms** | **334ms** |
|
||||
| 33 | 3 | 2 | 328ms | 345ms |
|
||||
|
||||
⚠️ 剩余下限由公网决定:本机 → 服务器 TCP 握手 RTT 59~80ms,66 片最多只能压到
|
||||
「1 次往返」;网络抖动时段实测会到 460~600ms。
|
||||
|
||||
端到端实测(全默认参数 `TdxData()`,11 台服务器,5,227 只):
|
||||
|
||||
| 场景 | 耗时 |
|
||||
|---|---|
|
||||
| 首次调用(含并发建 22 条连接) | 783ms |
|
||||
| 稳态(第 2 次起) | **300~343ms** |
|
||||
| 命中 TTL 缓存 | 67ms |
|
||||
| 老版本对照(串行) | 5,996ms(首次)/ 3,460ms(稳态) |
|
||||
|
||||
### 修复
|
||||
|
||||
- **指数 / ETF / LOF 的 `trade_count` 不再伪装成成交笔数**。
|
||||
|
||||
通达信对**指数**和**多数场内基金**不提供笔数,但逐笔报文里那 4 字节仍塞了值:
|
||||
指数是「当时均价 × 1000」,ETF/LOF 是「价格 × 1000」,货币 ETF(`sh511990`)恒为 `0`。
|
||||
旧版直接把它当笔数,于是 `tdx transactions sh510310` 会打印出「3994笔」这种假数据
|
||||
(实测 2026-10-22,价 3.989,原字段 3994)。
|
||||
|
||||
| 位置 | 变化 |
|
||||
|---|---|
|
||||
| `TransactionRecord.trade_count_available`(新字段) | 该品种协议上是否带笔数;默认 `True` |
|
||||
| `symbols.has_order_count(market, code)`(新函数) | 按品种判定;**沪市 501x/502x/506x 实测有真实笔数**,不能按「5 开头=基金」一刀切 |
|
||||
| `transaction.py` 三个解析器 | 不可用时 `trade_count=0` + `trade_count_available=False`;历史通道(0x013001,无笔数字段)同样标记 |
|
||||
| `tdx transactions` CLI | 不可用时打印 `(-)`,不再打印假笔数 |
|
||||
|
||||
实测样本(2026-10-22,按前缀各取样):沪股票/科创/深股票/深可转债/沪逆回购 → 真实笔数;
|
||||
沪 `51/52/56/58`、深 `159/15x/16x/17x/18x`、所有指数 → 不可用。
|
||||
|
||||
### 文档
|
||||
|
||||
- **可转债的量纲口径补全**(数据本身无缺陷,但对外文档一片空白,很容易被误判成 bug)。
|
||||
|
||||
起因:核查可转债的分时(`0x122D`)与逐笔(`0x122F`)是否有类似 ETF/LOF 的笔数问题。
|
||||
结论是**价格 / 成交量 / 笔数三样全部正确**,顺带确认了 K 线标度也对;唯一的问题是
|
||||
`vol` 的单位没人写清楚 —— 直接拿 `amount / vol` 算均价会**错 10 倍**。
|
||||
|
||||
实测(2026-10-22,16 只 A股可转债当日 + 8 只历史日 `2026-10-21`):
|
||||
|
||||
| 校验项 | 结果 |
|
||||
|---|---|
|
||||
| 分时末价 / 快照现价 | **1.000000** |
|
||||
| Σ分时 vol / 日线 vol | **1.000000** |
|
||||
| Σ逐笔 vol / 日线 vol | **1.000000** |
|
||||
| 逐笔 VWAP vs `amount/(vol×10)` | 吻合(差 ≤0.002,是 amount 精度) |
|
||||
| 快照 `pre_close` vs 上一交易日日线 `close` | **16/16 精确相等** |
|
||||
| `trade_count` | 真实笔数(`sh111000` 3631 条切片 / 20443 笔) |
|
||||
|
||||
关键口径对照(这是文档缺失的部分):
|
||||
|
||||
| 品种 | 分时 / 逐笔 `vol` | 日线 `vol` | `Σ分时vol / 日线vol` | `amount / vol` |
|
||||
|---|---|---|---|---|
|
||||
| 股票 / ETF / LOF | 手(= 100 股) | 股 | **0.01** | 均价 ✓ |
|
||||
| 可转债 | 手(= 10 张) | **手** | **1.0** | 均价 **×10** ⚠️ |
|
||||
|
||||
| 位置 | 变化 |
|
||||
|---|---|
|
||||
| `SecurityBar` / `TransactionRecord` / `MinuteBar` | 量纲注释补可转债(`vol`=手;日线 `amount/vol` 需再 `/10`) |
|
||||
| `symbols.has_order_count` | 实测表补沪可转债 `110/111/113/118`、沪 EB `132`、深可转债 `123/127/128`,并说明它们走「排除法」兜底分支 |
|
||||
| `docs/API_REFERENCE.md` | 新增**跨品种量纲对照表** |
|
||||
| `docs/llms.txt` / `SKILL.md` | 同步同一口径 |
|
||||
|
||||
**代码逻辑未改**:可转债在 `has_order_count` 里本来就判为「有笔数」(不在基金/指数的
|
||||
排除前缀里),本次只是把实测依据补进注释与文档。
|
||||
|
||||
## [0.6.6] - 2026-10-22
|
||||
|
||||
### 改进
|
||||
|
||||
- **价格小数位改用协议自带的权威值,前缀规则降为兜底**。
|
||||
|
||||
0.6.5 修好债券/回购标度时,采用的是「按代码前缀推断」+「用全量清单做离线校验」。
|
||||
但同一份证券清单接口(`get_universe`,底层 `GetSecurityListCmd`)本来就**返回**
|
||||
`SecurityInfo.decimal_point`(外加自描述浮点的 `pre_close`)——权威值一直就在手上,
|
||||
只是没接进取数路径。本次把它接进来:
|
||||
|
||||
| 位置 | 说明 |
|
||||
|---|---|
|
||||
| `price_decimal_digits` | 查找顺序改为 **权威表 → 静态前缀规则**,未同步时行为与 0.6.5 完全一致 |
|
||||
| `decimal_sync`(新模块) | 每日后台同步一次 + 磁盘缓存(`%LOCALAPPDATA%\pytdxdata\decimal-points.json`) |
|
||||
| `TdxData.sync_decimal_points()` | 手动强制刷新(排障 / 立刻要权威表) |
|
||||
| `TdxData.decimal_point_status()` | 状态:`source` / `stock` / `mixed_prefixes` / `syncing` |
|
||||
|
||||
**对用户仍无感**:`start()` 命中当日缓存就直接装表(实测 60 ms、零网络),
|
||||
未命中则挂后台 task(实测首次 50 s,含沪 27,520 + 深 24,462 条清单),
|
||||
不阻塞 `start()`(`start()` 仍 1.5 s);`close()` 会取消未完成的同步。
|
||||
|
||||
**表结构刻意做小**:全量 51,982 只标的只产生 **179 个**「三位前缀 → 小数位」映射
|
||||
(沪 96 + 深 83),内存里就是 179 条。若将来某段代码混用两种小数位,该前缀自动
|
||||
退化为**逐代码表**,只存例外代码(`mixed_prefixes` 会 > 0,可据此发现),
|
||||
不会静默算错。清单拉取不完整时表会偏小 —— 缺的部分退回静态规则,不影响正确性。
|
||||
|
||||
为什么保留前缀规则:`get_quotes` 解析是纯同步、无网络的路径,解析那一刻可能还没
|
||||
同步过清单;离线 / 首次安装也必须能出价格。实测两者在全量清单上**零分歧**
|
||||
(51,982 条逐条比对,不一致 0 条),并存不会造成口径冲突。
|
||||
|
||||
开关:`PYPYTDX_DATA_DISABLE_DECIMAL_SYNC=1` 关闭后台同步(CI / 测试用)。
|
||||
|
||||
### 说明
|
||||
|
||||
- 新增回归测试 16 项(权威表优先级 / 混合前缀退化 / 缓存跨日失效 / 损坏容错 /
|
||||
后台任务取消),测试总数 **289 → 306**。
|
||||
|
||||
## [0.6.5] - 2026-10-22
|
||||
|
||||
### 新增
|
||||
|
||||
- **指数成分成交汇总:一条接口拿到「某指数下全部股票的成交量与成交笔数」**
|
||||
(`get_index_members` / `get_index_trade_stats` / `get_index_trade_stats_multi`)。
|
||||
|
||||
起因是一个绕不开的协议事实:**指数本身没有成交笔数**。指数逐笔(`sh000001`)的
|
||||
`trade_count` 字段实际是「当时均价 × 1000」(2026-10-21 已验证),`get_minutes` /
|
||||
`get_market_stat` / `SecurityBar` / `get_quotes` 的 160 个字段位里也都**没有**笔数
|
||||
(扫描 0x122C 全字段位确认,`0x8`/`0x9` 是外盘/内盘手数)。所以「指数笔数」唯一可行
|
||||
的口径是**把成分股的逐笔笔数累加**。
|
||||
|
||||
```python
|
||||
async with TdxData() as td:
|
||||
st = await td.get_index_trade_stats("000300") # 沪深300,约 10 s
|
||||
for hhmm, cum in st.cum_trade_count(): # 截止该分钟的累计笔数
|
||||
print(hhmm, cum)
|
||||
st.minutes[0] # MinuteTradeStat(time='09:25', volume=…, trade_count=…)
|
||||
st.total_volume # 手
|
||||
st.total_trade_count # 笔
|
||||
```
|
||||
|
||||
| 方法 | 作用 |
|
||||
|---|---|
|
||||
| `get_index_members(index, *, fields=None, count=100000)` | 指数成分股(含实时报价)。裸 6 位按**指数语义**解析:`000001`=上证指数,不是平安银行 |
|
||||
| `get_index_trade_stats(index, date=None, *, concurrency=24, batch_size=50, members=None, skip_errors=True, on_progress=None)` | 成分逐笔 → 每分钟成交量/笔数;`date` 支持历史 |
|
||||
| `get_index_trade_stats_multi(index, dates, *, ...)` | 多日批量(成分只请求一次),用于「近 N 日同时点均值」基准 |
|
||||
|
||||
**口径校验(2026-10-22 实测)**:``total_amount`` 是**成分股**合计。对「成分 = 指数
|
||||
覆盖面」的指数,它与指数快照 ``amount`` 完全相等 —— 说明成分清单与量能口径一致。
|
||||
|
||||
| 指数 | 成分 | 成分合计 | 指数快照 | 比值 | 耗时 |
|
||||
|---|---|---|---|---|---|
|
||||
| `sh000016` 上证50 | 50 | 661.38 亿 | 661.34 亿 | 1.0001 | 5.8 s |
|
||||
| `sh000300` 沪深300 | 300 | 2373.98 亿 | 2373.98 亿 | 1.0000 | 9.7 s |
|
||||
| `sh000001` 上证指数 | 2230 | 3907.00 亿 | 4111.56 亿 | 0.9503 | 35.6 s |
|
||||
|
||||
历史日期同样成立(2026-10-21 收盘数据):`sh000016` 1046.29 / 1045.59 亿(1.0007)、
|
||||
`sh000300` 3881.65 / 3879.30 亿(1.0006)。
|
||||
|
||||
单只股票级别的口径更硬:`Σ(逐笔价 × 手 × 100)` 与日线 `amount` 比值
|
||||
**1.0000~1.0004**(`sh600000` / `sz000001` / `sh601398` / `bj920002` / `bj920015`
|
||||
全部吻合),`Σvol` 与快照 `vol` 比值 1.0000。
|
||||
|
||||
CLI 同步新增:`tdx index-stats 000300 --top 10`。
|
||||
|
||||
**三个必须知道的限制**:
|
||||
|
||||
1. **深市 `399xxx` 的指数成交额是「全板块」口径**,不是样本股 —— 实测
|
||||
`sz399001` 快照 5081.40 亿 = 深市 2908 只股票合计 5081.01 亿,`sz399006`
|
||||
2568.60 亿 = 创业板全部股票合计。所以拿它跟 `total_amount` 比会得到 0.43 这种
|
||||
无意义比值,不是数据错。**`get_index_trade_stats` 返回的始终是成分口径。**
|
||||
2. **北交所(`899xxx`)没有笔数字段** —— 逐笔 `trade_count` 恒为 0,此时
|
||||
`trade_count_available` 为 `False`,只有成交量可用。
|
||||
3. **成分表比全市场滞后** —— 上证指数返回 2230 只,而 `get_universe("sh")` 里
|
||||
股票段共 2360 只,差 130 只(新上市尚未纳入服务端板块),这正是上表
|
||||
0.9503 的来源;`sh000016`/`sh000300` 这类固定样本指数不受影响。
|
||||
需要精确覆盖时可自行传 `members=[...]`。
|
||||
|
||||
### 修复
|
||||
|
||||
- **债券与回购价格全线偏差:沪市少 100 倍、深市少 10 倍**(`get_quotes`)。
|
||||
|
||||
`price_decimal_digits` 只覆盖了「股票 2 位 / 基金 3 位」两档,债券与回购没有对应
|
||||
分支,落到默认档位。它们实际是 **4 位小数**,于是被读大了两个数量级:
|
||||
|
||||
| 标的 | 真实价 | 修复前 | 偏差 |
|
||||
|---|---|---|---|
|
||||
| `sh019742` 国债 24特国01 | 110.2280 | 11022.8000 | ×100 |
|
||||
| `sh110077` 可转债 洪城转债 | 199.2790 | 19927.9000 | ×100 |
|
||||
| `sh204001` 回购 GC001 | 1.3850 | 138.5000 | ×100 |
|
||||
| `sh900901` B股 云赛B股 | 0.5670 | 5.6700 | ×10 |
|
||||
| `sz123107` 可转债 温氏转债 | 112.1890 | 1121.8900 | ×10 |
|
||||
| `sz131810` 回购 R-001 | 1.3800 | 13.8000 | ×10 |
|
||||
|
||||
沪市 `010/018/019/020/1xx/2xx` 与深市 `10x-14x/19x/50x/52x/56x` 全部改为 4 位;
|
||||
沪市 B 股 `900xxx` 另改为 3 位(与深市 `200xxx` 的 2 位不同,两市不一致)。
|
||||
股票 / ETF / 指数 / 板块的行为**不变**。
|
||||
|
||||
**验证方式(可作为该函数的验收标准)**:拿 `pre_close`(五档快照口径,走
|
||||
`price_decimal_digits`)除以上一交易日**日线收盘**(K线口径,走独立的 `/1000`
|
||||
路径),比值 1.0 即两口径一致。修复后 15 类标的全数为 1.0000,零偏差。
|
||||
|
||||
**权威来源已确认**:证券列表里协议**自带** `SecurityInfo.decimal_point` 字段。
|
||||
2026-10-22 拉取全量 **51,982 只**(沪 27,520 + 深 24,462)逐条比对,本函数与
|
||||
协议字段**完全一致、零例外**——因为全量清单里没有任何一个「三位前缀」出现混合
|
||||
小数位。`get_quotes` 解析时拿不到证券清单,故仍用前缀推断;需要权威值时读
|
||||
`SecurityInfo.decimal_point`。
|
||||
|
||||
- **美股(EX 通道)分时的 `hour` 是非法时钟值**(`get_minutes("usAAPL")`)。
|
||||
|
||||
EX 分时的 `minutes` 字段是**自开盘起的连续分钟数**,不是「当天的分钟数」。
|
||||
旧代码直接 `minutes // 60`,于是 390 条里 **240 条** 的 `hour` ≥ 24:
|
||||
|
||||
```
|
||||
修复前: 21:30, 21:31, ..., 23:59, 24:00, 24:01, ..., 27:59 ← hour=27 不存在
|
||||
修复后: 21:30, 21:31, ..., 23:59, 00:00, 00:01, ..., 03:59 ← 真实北京时间
|
||||
```
|
||||
|
||||
任何 `datetime(...)` 拼接都会因 `hour=27` 抛 `ValueError`,按小时聚合也会把
|
||||
凌晨段错误地归到同一个自然日。
|
||||
|
||||
**修法**:`MinuteBar.__post_init__` 把越界的 `hour` 拆成
|
||||
「`hour %= 24` + `day_offset += hour // 24`」,并新增两个只读属性:
|
||||
|
||||
| 属性 | 用途 |
|
||||
|---|---|
|
||||
| `MinuteBar.datetime` | 该分时点的时刻(北京时间,已含跨日偏移) |
|
||||
| `MinuteBar.trading_date` | 归属的交易日 `YYYY-MM-DD`。美股一个交易日横跨两个北京自然日(21:30~次日 04:00),**按交易日聚合必须用它**,与 `SecurityBar.trading_date` 口径一致 |
|
||||
|
||||
A股(合成时间 09:30~15:00)/ 港股(09:30~16:00)/ 期货的 `hour` 本就合法
|
||||
→ **零行为变化**,`day_offset` 恒为 0。
|
||||
|
||||
实测(2026-10-22):`usAAPL` / `usTSLA` / `usNVDA` 各 390 条,修复后
|
||||
`hour` 全在 0~23、时间轴严格单调、`trading_date` 收敛为单一交易日;
|
||||
`hk00700` 90 条无任何变化。
|
||||
|
||||
### 说明
|
||||
|
||||
- 新增回归测试:债券/回购/B股小数位 ×29、美股分时跨日 ×6、指数成分成交汇总 ×34、
|
||||
全接口 ×1,测试总数 **217 → 289**。
|
||||
|
||||
## [0.6.4] - 2026-10-22
|
||||
|
||||
### 修复
|
||||
|
||||
- **`get_ticks()` 返回列表不是时间序,每 `page_size` 处时间回跳一次**。
|
||||
|
||||
`0x122F`(A股逐笔)与 EX 逐笔通道的偏移量 `start` 都是**从最新一笔往回数**的:
|
||||
实测 `sh600000`,`start=0` 返回 `14:07:48..15:20:16`(当天最新 1000 条),
|
||||
`start=4000` 返回 `09:15:03..09:41:18`(当天最早)。而 `paginator` 按 `start`
|
||||
升序拼接,于是拼出"新的块在前、旧的块在后"的倒序结构:
|
||||
|
||||
| | 修复前 | 修复后 |
|
||||
|---|---|---|
|
||||
| `sh600000` 时间轴 | 4 处逆序跳跃(index 999 / 1999 / 2999) | 严格升序 |
|
||||
| `hk00700` 时间轴 | 21 处逆序跳跃(每 1800 条一次) | 严格升序 |
|
||||
|
||||
修复分两层:① `PageSpec.newest_first=True` 让 `paginator` 反页序拼装(页内本身升序,
|
||||
只翻页序),这条能力保留给将来需要按"最新优先"锚点分页的命令;② A股逐笔最终改为
|
||||
**单连接串行游标**(见下一条),从根上避开多主机锚点错位。
|
||||
|
||||
**`start` 参数语义同时明确**:它是"从最新往回数的偏移",不是"从最早往后数的偏移",
|
||||
`docstring` 已补充。注意这修正的是**顺序**,数据本身无缺口、无重复(4304 条唯一时间戳)。
|
||||
|
||||
- **`TransactionRecord.trade_count` 在集合竞价段被读成 42 亿量级脏值**。
|
||||
|
||||
该字段在协议里是**有符号** `int32`,但解析格式串写的是无符号 `I`。集合竞价段
|
||||
(`bs_flag=8`)真实值为负数,按 uint32 解出 `4294966422`:
|
||||
|
||||
```
|
||||
原始 18 字节: 17820000 0ad71b41 d9000000 96fcffff 0800
|
||||
修复前: trade_count = 4294966422
|
||||
修复后: trade_count = -874 ← 0xfffffc96 的 int32 补码
|
||||
```
|
||||
|
||||
实测 `sh600000` 全天 4304 条中有 **38 条**命中(占 0.88%),全部落在 09:15~09:25
|
||||
集合竞价段;正常交易段(`bs_flag` 0/1/2/5)**无一条为负**。旧行为会让
|
||||
`max()` / `mean()` 等统计直接失真(曾算出 `mean=37,920,247`)。
|
||||
|
||||
格式串由 `<IfIIH` 改为 `<IfIiH`。**注意负值语义尚未确证**(可能是撤单量/净委托),
|
||||
目前只是如实保留符号,未做业务解释。
|
||||
|
||||
- **`get_quotes()` 沪市指数价格整体少 10 倍**。
|
||||
|
||||
`price_decimal_digits` 把沪市 `000`(如 000001/000016/000300/000688)与 `8`
|
||||
开头(如板块指数 880001/880305)一律判成 3 位小数(除 1000),但协议原始整数
|
||||
实际是 **2 位**(除 100)→ 上证指数 3713 被读成 371、上证50 2747 被读成 274。
|
||||
|
||||
实证(2026-10-21):沪市指数 6 只 + 板块 2 只全部 `分时中位价 / 快照价 = 10.0`,
|
||||
而沪市 ETF(510300/588000/511010/501018)、股票(60/601)、科创板(688)、
|
||||
深市指数/股票/创业板/ETF 全部 `= 1.0` 正确。修法是**把沪市 `000`/`8` 从 3 位改回 2 位**,
|
||||
其余前缀(`5` ETF/基金、深市 `1` 基金转债)保持 3 位不变。回归测试覆盖 21 组代码。
|
||||
|
||||
- **`get_ticks()` 把集合竞价**委托**(`bs_flag=8`)当成成交计入,导致 Σvol 虚高且跨主机不稳定**。
|
||||
|
||||
`0x122F` 第 5 字段 `bs_flag=8` 的记录是 **09:15~09:25 / 14:57~15:00 竞价段的撮合委托
|
||||
快照**,`vol` 是**累计值**(可被同段后续记录覆盖),根本不是成交增量。旧代码把它一并
|
||||
返回,两个后果:
|
||||
|
||||
1. **Σvol 比真值虚高**:`sh600000` 全天 Σvol 含 b8 时 = 668,693,快照 vol = 462,562,
|
||||
比值 1.4456;剔掉 b8 后 Σvol = 462,634,比值 **1.000156**(与快照精确一致)。
|
||||
2. **跨主机不稳定**:干净主机不收录 b8(0 条)、脏主机收录 135 条。并发分页时各页
|
||||
来自不同主机,同一 `start`(从最新往回数的相对锚点)在不同主机落到**不同的真实成交**
|
||||
上 → 页边界既**重叠**(sh600000 冗余 116 条)又**漏条**(13:03 附近丢 59 条),
|
||||
同一查询两次调用结果不同。
|
||||
|
||||
修复:① 分页拼装后**统一剔除 `bs_flag=8`**;② A股逐笔改用**单连接串行游标翻页**
|
||||
(`_ticks_cn_serial`),彻底消除跨主机锚点错位。修复后单主机 1.0002、剔 b8 后跨主机
|
||||
公共时间窗**逐条一致**。
|
||||
|
||||
- **文档化指数逐笔字段的量纲差异**(非 bug,是数据源固有语义)。
|
||||
|
||||
指数(如 `sh000016`)的 `vol` 单位是**百元**(成交额/100),`trade_count` 是**均价×1000**
|
||||
(不是笔数),与个股完全不同。实测 `快照amount / Σtick.vol = 100.0002`、`tc/1000` 与分时
|
||||
`avg_price` 差 < 1.1。已写进 `TransactionRecord` 类注释与 `get_ticks` docstring,
|
||||
避免用户把指数 Σvol 直接和成分股成交量比(会误得 ~26 倍)。
|
||||
|
||||
### 说明
|
||||
|
||||
- 新增回归测试:价格小数位 ×20(19 组参数化 + 1 组去 NUL)、`get_ticks` 剔 b8 /
|
||||
单连接串行 / 多页无重叠 ×3,测试总数 **194 → 217**。
|
||||
|
||||
## [0.6.3] - 2026-10-18
|
||||
|
||||
- fix(pool): ExPool acquire 背压等待(Condition 排队,默认 30s 超时)替代 3x0.05s 快速失败;超时异常带池状态
|
||||
- feat(api): 新增 ex_pool_cap 参数(默认 2),单 EX 服务器场景不再静默退化为 1 并发(fixes #1)
|
||||
|
||||
## [0.6.2] - 2026-10-13
|
||||
|
||||
### 修复
|
||||
|
||||
- **美股分钟级 K 线的 `datetime` 全部错位**(`get_bars("usAAPL", KlinePeriod.MIN_1)`)。
|
||||
|
||||
通达信 EX 通道返回的美股分钟线时间戳,把**两个时区拼进了同一个五元组**:日期分量是
|
||||
**美东交易日**(协议里的 `zipday`),时分分量却是**北京时间**(`tminutes`)。直拼出的
|
||||
`datetime` 会把同一交易日的**下午半场排到上午半场之前**——单日 390 根不仅首末颠倒,
|
||||
而且全部落在同一个北京自然日里。于是任何依赖 `datetime` 的行为都是错的:排序、
|
||||
`count` 切片("取最新 N 根")、日期区间二分与过滤、增量更新水位线。
|
||||
|
||||
| 观测项 | 修复前 | 修复后 |
|
||||
|---|---|---|
|
||||
| 单日 390 根的日期分布 | **全部**落在 10-12(hour 0~23 混在一起) | 10-12 的 21:31~23:59(149 根)+ 10-13 的 00:00~04:00(241 根) |
|
||||
| 时间轴 | 上午半场被排到下午半场**之前** | 严格单调递增 |
|
||||
| `count=30` 是否取到真正最新的 30 根 | 否(拿到上一交易日的 23:30~23:59) | 是(当日收盘段 03:31~04:00) |
|
||||
| 每组首根 `open` / 末根 `close` 对日线 | 对不上 | **逐位相同**(10-07/08/09/12 四个交易日实测) |
|
||||
|
||||
现在 `SecurityBar.datetime` **统一为北京时间**:美股凌晨段(北京 hour < 12)的日期
|
||||
自动 +1 天(含月末 / 年末进位)。A股 / 港股 / 期货原始即为北京时间,**零行为变化**。
|
||||
|
||||
新增两个只读属性:
|
||||
|
||||
| 属性 | 用途 |
|
||||
|---|---|
|
||||
| `SecurityBar.trading_date` | 该 bar 归属的交易日 `YYYY-MM-DD`。美股一个交易日横跨两个北京自然日,**按交易日聚合(分钟线合成日线、日内统计)必须用它,不能用 `datetime.date()`** |
|
||||
| `SecurityBar.is_us` | 是否美股(EX 通道 `US_STOCK`) |
|
||||
|
||||
归一化在 `SecurityBar.__post_init__` 中实现,因此 **EX 的两条 K 线命令路径**
|
||||
(`_bars_ex` 的 0x23FF 与日期区间二分所依赖的 MAC 0x122E)自动共用同一份口径,
|
||||
不会出现"取数按一套时间、二分按另一套时间"的错位。
|
||||
|
||||
### 说明
|
||||
|
||||
- 分钟线缓存 TTL 为 **60 秒**,升级后旧缓存最多残留 1 分钟即自然失效,**无需手动清理**;
|
||||
日线(`period >= DAY`)不受本缺陷影响。
|
||||
- 本版只修 K 线。**同日发现的同类问题未包含在内**:EX 分时(`get_minutes`)对美股返回的
|
||||
`MinuteBar.hour` 是**未取模的连续分钟数**(实测可达 27,即次日 03:xx),且 `MinuteBar`
|
||||
无日期分量可承载跨日信息——修法涉及"是否新增字段 / 日期如何归属"的取舍,留待下一版决策。
|
||||
- 新增 9 项针对性回归测试(分界点、月末与年末进位、390 根整日单调性、MAC 路径同样归一化)。
|
||||
|
||||
## [0.6.1] - 2026-09-21
|
||||
|
||||
### 变更
|
||||
|
||||
- **重写「当日首次启动测速缓存」,改为对用户完全无感**。原实现在 v0.5.0 已随包发布,但存在
|
||||
三个会让它**静默失效**的缺陷,本次一并修正:
|
||||
|
||||
| 缺陷 | 原实现 | 现在 |
|
||||
|---|---|---|
|
||||
| 候选集自指 | 连接池只把自己**已过滤后的成员**交给探针 → 只重测了 9 台,另外 20+ 台候选(含快节点)永远测不到 | 探针内部取**全部候选**(包内清单 ∪ 代码常量 ∪ env) |
|
||||
| 写回安装目录 | 结果写回 `src/pytdxdata/servers.json` → 容器 / 系统级安装只读、多用户互相覆盖 | 改写入**用户级缓存目录**,包内清单运行时**绝不被改写** |
|
||||
| 探测太弱 | 只做握手(`TdxConnection.test`),测不出「握手正常但 K 线空响应」的坏节点 | 发**真实业务命令**并要求返回非空(标准 / MAC / EX 各一) |
|
||||
| EX 池漏测 | EX 的 4 台从未被探测 | EX 一并纳入(7727 + 各自握手模式) |
|
||||
|
||||
- 测速结果按**实测延迟升序**被 `config.get_standard_hosts()` 等采用;当天已有缓存则**不再刷新**。
|
||||
后台执行,不阻塞 `start()`、不打印、失败静默降级。
|
||||
|
||||
- 本版基于 **0.6.0 的统一对外接口**构建(`get_bars` + 字符串标的);同时把 0.6.0
|
||||
全接口回归测试里对旧探针入口(`pool.get_probe_cache`)的替身改为新入口(`pool.schedule_probe`),
|
||||
保证单测不触发真实网络测速。
|
||||
|
||||
### 修复
|
||||
|
||||
- **A股分时缩略采样 100% 失败**(`get_minutes(sampling=True)` / `get_chart_sampling()`):
|
||||
旧实现走 `MacChartSamplingCmd`(**0x254D**),但该 opcode 属于 **EX 协议**,7709 标准 /
|
||||
MAC 主机对它一律不响应——实测 3 台主机各 10s 超时,叠加连接池 3 次重试约 24s 才抛错。
|
||||
现改为**由分时(0x122D)数据派生**价格点(分时本身就是 240 点,语义一致),
|
||||
实测 4 个 A股 场景均在 0.1s 内返回 240 点;EX 通道仍用 0x254D(84 点)。
|
||||
|
||||
- **历史专业财报整条链路不可用**(`get_financial_records_parsed()`):
|
||||
`tdxfin/gpcw.txt` 索引里写的是裸 `gpcwYYYYMMDD.zip`,而服务器**实际只提供未压缩的**
|
||||
`tdxfin/gpcwYYYYMMDD.dat`。旧实现把索引名直接丢给 `zipfile` → 必然 `BadZipFile`;
|
||||
废弃别名 `get_financial_records()` 更危险,会**静默返回 0~1 字节空壳**而不报错。
|
||||
现按候选名(`tdxfin/*.zip` → `tdxfin/*.dat` → 原名)逐个尝试并校验数据形态,
|
||||
zip 与未压缩 dat 都支持。实测 `gpcw20260630.zip` → **5568 条**、
|
||||
`gpcw20260331.zip` → **5547 条**(样本 000001,584 个 float 字段),
|
||||
别名返回 **13,068,116 字节**真实数据;尚未生成的占位条目
|
||||
(如 `gpcw20260930.zip`,filesize=164)优雅返回空列表而非报错。
|
||||
|
||||
### 新增
|
||||
|
||||
- `codec.financial` 新增 `looks_like_gpcw_dat()` / `extract_gpcw_dat()`:
|
||||
财报字节的形态判定与「zip / 未压缩 dat」统一提取(纯标准库,可脱网单测)
|
||||
- 环境变量 `PYPYTDX_DATA_CACHE_DIR`(覆盖测速缓存目录)、
|
||||
`PYPYTDX_DATA_DISABLE_PROBE`(关闭测速,CI / 测试用)
|
||||
- `tests/test_probe_cache.py`(37 项):当日缓存不刷新、跨日重测、
|
||||
原子写入且不留 tmp、**不改写包内 `servers.json`(逐字节校验)**、
|
||||
单飞并发、非法主机过滤、`DISABLE_PROBE`、池 `start()` 不阻塞 / `close()` 取消后台探针
|
||||
|
||||
### 文档
|
||||
|
||||
- [安装与配置](https://openbot-coder.github.io/pytdxdata/getting-started/#当日自动测速无感):
|
||||
新增「当日自动测速(无感)」小节——缓存路径、优先级链、环境变量一览
|
||||
- [通道与数据源](https://openbot-coder.github.io/pytdxdata/internals/routing/#daily-probe):
|
||||
新增「运行时:当日测速缓存」小节,逐条说明必须踩对的实现要点与反例
|
||||
|
||||
## [0.6.0] - 2026-09-16
|
||||
|
||||
### 新增
|
||||
|
||||
- **统一对外接口**:一个数据类型 = 一个方法,跨市场自动选路。
|
||||
标的统用字符串写法(`sz000001` / `hk00700` / `usAAPL` / `cffex:IFL0`),
|
||||
A股/港股/美股/期货/期权混合查询一次返回。旧的 51 个按通道/市场分裂的接口中,
|
||||
33 个保留为 deprecated 别名(运行时发出 `DeprecationWarning`,**1.0 移除**),
|
||||
另 18 个(`get_quotes` / `get_xdxr` / `get_finance` / `get_auction` / `get_capital_flow` /
|
||||
`get_market_stat` / `get_unusual` / `get_server_info` / `get_kline_offset` /
|
||||
`get_board_members` / `get_board_summary` / `get_board_ranking` / `get_board_change_ranking` /
|
||||
`get_block_parsed` / `get_industry_map` / `get_financial_file_infos` /
|
||||
`get_financial_records_parsed` / `get_price_limits`)同名升级为统一接口。
|
||||
|
||||
| 数据类型 | 新接口 | 合并掉的旧接口 |
|
||||
|----------|--------|----------------|
|
||||
| 标的清单 | `get_universe(market)` | get_security_count + get_security_list + get_security_list_all + get_goods_count + get_goods_list |
|
||||
| 报价 | `get_quotes(symbols, *, fields)` | get_quotes + get_stock_quotes + get_goods_quotes + get_goods_quotes_list |
|
||||
| K线 | `get_bars(symbols, period, ...)` | get_kline + get_index_kline + get_kline_batch + get_kline_since |
|
||||
| 指标 | `get_indicators(symbols, indicators, ...)` | get_stock_kline_with_indicators |
|
||||
| 逐笔 | `get_ticks(symbols, *, date)` | get_transactions + get_goods_transaction + get_goods_transaction_all |
|
||||
| 分时 | `get_minutes(symbols, *, date, sampling)` | get_minute + get_minute_batch + get_chart_sampling + get_goods_chart_sampling + get_goods_tick_chart |
|
||||
| F10 | `get_f10(symbol, section)` | get_company_info_category + get_company_info_content |
|
||||
| 文件 | `get_file(name)` | get_block_info + get_report_file + get_file_meta + download_file |
|
||||
| 财务文件 | `get_financial_file_infos()` / `get_financial_records_parsed(fn)` | get_financial_file_list + get_financial_records |
|
||||
| 板块列表 | `get_boards(kind)` | get_board_list |
|
||||
| 板块成员 | `get_board_members(board)` → 归一 `SecurityQuote` | get_board_members + get_stock_quotes_list(分类报价) |
|
||||
| 个股板块 | `get_board_of(symbols)` | get_belong_board |
|
||||
| 快照 | `get_snapshot(symbols)` | get_symbol_info |
|
||||
|
||||
> `TdxData` 公开方法 63 个 = **28 个统一接口** + 33 个 deprecated 别名 + `start`/`close`。
|
||||
> 统一接口一律只接受字符串标的;同名旧方法不再兼容 `(market, code)` 元组写法。
|
||||
|
||||
- **库层标的解析** `pytdxdata.symbols`(原仅 CLI 内有):`parse_symbol` / `format_symbol` /
|
||||
`parse_symbols` / `market_targets` / `is_cn_index`;裸 6 位数字视为沪市指数。
|
||||
- **归一数据模型**:`SecurityBar` / `SecurityQuote` / `TransactionRecord` / `MinuteBar` /
|
||||
`SecurityInfo` 等均新增 `symbol`(规范化标的串)/ `name` / `fields`(原始字段位兜底),
|
||||
老构造写法不受影响(新字段有默认值)。
|
||||
- `IndicatorSet` dataclass(`get_indicators` 返回单元:`bars + indicators` 组合)。
|
||||
- `CacheKey.quote_key(..., tag=)` 支持按取数口径(标准五档 / MAC 字段位)隔离缓存。
|
||||
|
||||
### 修复
|
||||
|
||||
- **分页器连接泄漏**:`Paginator._worker` 换连接后旧连接不释放、池子被耗干
|
||||
(`_Slot` 包装 + 归还时按槽释放)。
|
||||
- **日期区间 K线返回 0 条**:标准通道 `start` 偏移方向是递减(0=最新),旧的二分
|
||||
假设递增。改为运行时探测方向再二分(`_offset_descending` / `_bisect_offset`)。
|
||||
- **池子全冷却死等**:所有服务器进冷却后 `acquire` 排队 30s 超时。
|
||||
补 `ignore_cooldown=True` 兜底,避免无人持有连接时永久等待。
|
||||
- `HealthEngine.COOLDOWN_SECONDS` 保持 120s(旧的 FAIL_STREAK_COOLDOWN=1 意味着
|
||||
任意一次通信失败即冷却,实测下正常服务器不应被冷却,保留该行为)。
|
||||
|
||||
### 变更
|
||||
|
||||
- **`docs/api/index.md` 接口总览**重新组织:统一接口(推荐)优先于旧接口列表。
|
||||
- `scripts/check_docs.py`:`docs/api/index.md` 从 `MUST_LIST_ALL` 移到
|
||||
`MUST_NOT_BE_STALE`(只列推荐的统一接口,旧方法不强制出现在总览页)。
|
||||
- CLI `tdx quotes` 改为统一 `td.get_quotes(symbols)`(跨市场混合报价)。
|
||||
- CLI `tdx kline` 改为统一 `td.get_bars(sym, period)`。
|
||||
- 测试 127 个用例全绿(新增 codec mock `get_file` 适配 + check_docs 新方法列表 +
|
||||
`tests/test_all_interfaces.py` 全接口回归:mock 连接逐个调用全部 63 个公开方法,
|
||||
并以「覆盖哨兵」断言无方法遗漏)。
|
||||
|
||||
## [0.5.0] - 2026-09-15
|
||||
|
||||
### 新增
|
||||
@@ -97,7 +823,8 @@
|
||||
|
||||
- 初始版本:动态连接池 + 双层 TTL 缓存 + 3 连接并发分页 + 双通道(标准 / MAC)+ 扩展市场(港股 / 美股 / 期货 / 期权)
|
||||
|
||||
[Unreleased]: https://github.com/openbot-coder/pytdxdata/compare/v0.5.0...HEAD
|
||||
[Unreleased]: https://github.com/openbot-coder/pytdxdata/compare/v0.6.0...HEAD
|
||||
[0.6.0]: https://github.com/openbot-coder/pytdxdata/compare/v0.5.0...v0.6.0
|
||||
[0.5.0]: https://github.com/openbot-coder/pytdxdata/compare/v0.4.0...v0.5.0
|
||||
[0.4.0]: https://github.com/openbot-coder/pytdxdata/compare/v0.3.2...v0.4.0
|
||||
[0.3.2]: https://github.com/openbot-coder/pytdxdata/compare/v0.3.1...v0.3.2
|
||||
|
||||
@@ -29,32 +29,35 @@ pip install "pytdxdata[uvloop]"
|
||||
```python
|
||||
import asyncio
|
||||
from pytdxdata import TdxData
|
||||
from pytdxdata.models import Market, KlinePeriod, Adjust
|
||||
from pytdxdata.models import KlinePeriod, Adjust
|
||||
|
||||
async def main():
|
||||
async with TdxData() as td: # 自动建连接池 + 缓存
|
||||
# 日K线(count 超过单页大小会自动多连接并发分页)
|
||||
bars = await td.get_kline(Market.SH, "600000", KlinePeriod.DAY, count=240)
|
||||
bars = await td.get_bars(["sz000001"], KlinePeriod.DAY, count=240)
|
||||
# 前复权(自动走 MAC 通道)
|
||||
qfq = await td.get_kline(Market.SZ, "300308", KlinePeriod.DAY,
|
||||
qfq = await td.get_bars(["sz300308"], KlinePeriod.DAY,
|
||||
count=500, adjust=Adjust.QFQ)
|
||||
# 五档报价(传 (market, code) 元组列表,>80 只自动切批)
|
||||
quotes = await td.get_quotes([(Market.SH, "600000"), (Market.SZ, "000001")])
|
||||
# 跨市场报价:A股+港股+美股一次拿全
|
||||
quotes = await td.get_quotes(["sh600000", "hk00700", "usAAPL"])
|
||||
# 逐笔成交(MAC 通道,支持历史日期)
|
||||
ticks = await td.get_transactions(Market.SZ, "000001", date=20260811)
|
||||
ticks = await td.get_ticks(["sz000001"], date=20260811)
|
||||
print(bars[-1].close, quotes[0].price, len(ticks))
|
||||
|
||||
asyncio.run(main())
|
||||
```
|
||||
|
||||
返回的是标准库 `dataclass` 组成的 `list`(库本身不依赖 pandas)。要 DataFrame 一行转:
|
||||
默认返回的是标准库 `dataclass` 组成的 `list`(核心库不依赖 pandas)。需要 DataFrame 就换个后端构造:
|
||||
|
||||
```python
|
||||
from dataclasses import asdict
|
||||
import pandas as pd
|
||||
df = pd.DataFrame([asdict(b) for b in bars])
|
||||
async with TdxData(df="polars") as td: # 或 df="pandas";需 pip install "pytdxdata[polars]"
|
||||
df = await td.get_bars("sh600000", KlinePeriod.DAY, count=240)
|
||||
# 列:market code open high low close vol amount datetime trading_date symbol settle
|
||||
```
|
||||
|
||||
时间字段会被**合并成一列 `datetime`**(不再有 year/month/day/hour/minute),五档报价展开成
|
||||
`bid1_price`…`ask5_vol`,详见 [DataFrame 输出](https://openbot-coder.github.io/pytdxdata/dataframe/)。
|
||||
|
||||
## 特性
|
||||
|
||||
| 特性 | 说明 |
|
||||
@@ -63,9 +66,10 @@ df = pd.DataFrame([asdict(b) for b in bars])
|
||||
| 🗄️ 双层 TTL 缓存 | 内存 LRU + 磁盘 SQLite(WAL),TTL 分级;single-flight 并发同键去重;缓存**已解析记录**(重复请求 0 网络 0 解析) |
|
||||
| ⚡ 并发分页 | 单请求拆 N 页(显式 start 偏移 → 无状态可并行)并行拉取、按序拼接、失败换连接重试 |
|
||||
| 🔀 双通道整合 | 标准通道(经典行情)+ MAC 通道(逐笔 / 复权 / 1 分钟 K 线),按命令自动选路 |
|
||||
| 📦 零重型依赖 | 纯 asyncio + stdlib(socket / zlib / struct / sqlite3) |
|
||||
| 📦 零重型依赖 | 纯 asyncio + stdlib(socket / zlib / struct / sqlite3),`dependencies` 为空 |
|
||||
| 🐼 DataFrame 直出 | `TdxData(df="polars")` / `df="pandas"` 让 42 个表格型接口直接返回 DataFrame;时间列自动合并成 `datetime`,五档报价自动展开。后端是可选 extras,核心包仍零依赖 |
|
||||
| 📊 全市场覆盖 | A股、港股、美股、期货、期权(46 个扩展市场) |
|
||||
| 📄 服务器文件解析 | 板块 `block_*.dat` / 行业 `tdxhy.cfg` / 财报 `gpcw*.zip` **下载即解析**成 dataclass |
|
||||
| 📄 服务器文件解析 | 板块 `block_*.dat` / 行业 `tdxhy.cfg` / 财报 `gpcw*` **下载即解析**成 dataclass(索引名与服务器实际文件名不一致时自动纠正) |
|
||||
| 🤖 AI 助手友好 | 内置 [`SKILL.md`](https://github.com/openbot-coder/pytdxdata/blob/main/SKILL.md) 与 [`llms.txt`](https://openbot-coder.github.io/pytdxdata/llms.txt) |
|
||||
|
||||
## 文档导航
|
||||
@@ -78,7 +82,8 @@ df = pd.DataFrame([asdict(b) for b in bars])
|
||||
| 搞清枚举该填几 | [枚举速查](https://openbot-coder.github.io/pytdxdata/api/enums/) |
|
||||
| 看协议二进制格式、三池路由、缓存策略 | [深入原理](https://openbot-coder.github.io/pytdxdata/internals/architecture/) |
|
||||
| 用命令行 | [命令行工具](https://openbot-coder.github.io/pytdxdata/cli/) |
|
||||
| 完整接口清单(51 个数据方法) | [`docs/API_REFERENCE.md`](https://openbot-coder.github.io/pytdxdata/API_REFERENCE/) |
|
||||
| 完整接口清单(28 个统一方法 + 33 个废弃别名) | [`docs/API_REFERENCE.md`](https://openbot-coder.github.io/pytdxdata/API_REFERENCE/) |
|
||||
| 重构审查(数据一致性 / 遗漏对账) | [接口重构审查](https://openbot-coder.github.io/pytdxdata/refactor-audit/) |
|
||||
| 版本变更 | [`CHANGELOG.md`](https://github.com/openbot-coder/pytdxdata/blob/main/CHANGELOG.md) |
|
||||
|
||||
> 本 README 只做**入口层**。深度内容一律在文档站,避免两处维护造成漂移。
|
||||
@@ -86,7 +91,7 @@ df = pd.DataFrame([asdict(b) for b in bars])
|
||||
## 给 AI 助手用
|
||||
|
||||
仓库根的 [`SKILL.md`](https://github.com/openbot-coder/pytdxdata/blob/main/SKILL.md) 是一份可直接加载的技能:三条必守约定、`KlinePeriod` 等枚举取值表、
|
||||
「想做什么 → 用哪个方法」决策表、以及 9 条真实陷阱。AI 编程助手读完就能一次写对,不必翻源码。
|
||||
「想做什么 → 用哪个方法」决策表、以及 11 条真实陷阱。AI 编程助手读完就能一次写对,不必翻源码。
|
||||
|
||||
```bash
|
||||
# 丢进你所用 AI 助手的 skills 目录即可
|
||||
@@ -103,7 +108,7 @@ curl -o ~/.workbuddy/skills/pytdxdata/SKILL.md --create-dirs \
|
||||
|
||||
```bash
|
||||
uv sync # 建虚拟环境 + 注册 tdx 命令
|
||||
uv run pytest tests/ # 109 个测试用例(协议 / 池 / 缓存 / 分页 / API / 文件解析 / CLI / 文档漂移)
|
||||
uv run pytest tests/ # 191 个测试用例(协议 / 池 / 缓存 / 分页 / 全接口回归 / 文件解析 / CLI / 文档漂移 / 当日测速缓存)
|
||||
uv sync --group docs # 文档站依赖(mkdocs-material + mkdocstrings)
|
||||
uv run mkdocs serve # 本地预览文档站
|
||||
```
|
||||
@@ -113,7 +118,16 @@ uv run mkdocs serve # 本地预览文档站
|
||||
- 行情来自通达信公开服务器,**15 分钟延时**;需实时数据请使用券商行情
|
||||
- 标准通道 K 线 `count < 100` 部分服务器返回空 → 分页器已强制 `min_page=100` 规避
|
||||
- 港股 / 美股期权在通达信协议内无数据
|
||||
- 服务器可用性随时间变化,可用 `python -m pytdxdata.server_probe` 重新探测
|
||||
- **EX 通道(港股 / 美股 / 期货 / 期权)的 K 线没有统一成交额**:`0x23FF` 帧内只有一个可复用槽位,
|
||||
权益类市场(港股主板/基金/创业板、新三板、美股)该槽位是成交额,已回填;**期货 / 期权该槽位是持仓量,
|
||||
所以它们的 `SecurityBar.amount` 恒为 `0.0`**(协议限制,不是 bug)。期货/期权的结算价在 `SecurityBar.settle`。
|
||||
⚠️ 港股主板/基金的 `vol` 单位是「手」(=100 股),`amount/vol` 是均价的 100 倍
|
||||
- **K 线时间戳统一为北京时间**(v0.6.2 起)。美股原始报文把「美东交易日日期」与「北京时间时分」
|
||||
拼在一起,库已自动归一化,`bar.datetime` 可直接排序 / 过滤;但**美股一个交易日横跨两个北京
|
||||
自然日**,按交易日聚合请用 `SecurityBar.trading_date`,不要用 `datetime.date()`
|
||||
- 服务器可用性随时间变化。**当日首次启动会自动在后台测速并缓存**(对调用方无感,当天不再重测);
|
||||
也可用 `python -m pytdxdata.server_probe` 离线重生成随包清单。
|
||||
缓存路径与三个开关见 [安装与配置 → 当日自动测速](https://openbot-coder.github.io/pytdxdata/getting-started/#当日自动测速无感)
|
||||
|
||||
## 免责声明
|
||||
|
||||
|
||||
@@ -18,20 +18,24 @@ description: 用 pytdxdata 拉取通达信行情数据——A股(沪/深/北)、
|
||||
from pytdxdata import TdxData
|
||||
|
||||
async with TdxData() as td: # 自动 start(探测服务器 + 预建连接 + 建缓存)
|
||||
bars = await td.get_kline(...)
|
||||
bars = await td.get_bars(["sz000001"], KlinePeriod.DAY, count=240)
|
||||
# 退出即 close
|
||||
```
|
||||
|
||||
不要直接 `await TdxData().get_kline(...)`——没 `start()` 就没有连接池。
|
||||
不要直接 `await TdxData().get_bars(...)`——没 `start()` 就没有连接池。
|
||||
|
||||
**2. `market` 传的是整数枚举值,不是字符串**
|
||||
**2. 标的写法是字符串,不是元组**
|
||||
|
||||
这是最高频的错误来源。**不要传 `"sh"` / `"SH"` / `"1"`**,要用枚举(它就是 `IntEnum`,所以传 `1` 也能跑,但可读性差):
|
||||
这是 v0.6 的最重要变化。**不要传 `(Market.SH, "600000")` 元组**,用字符串:
|
||||
|
||||
```python
|
||||
from pytdxdata.models import Market, KlinePeriod, Adjust
|
||||
# 旧写法(deprecated,会发出 DeprecationWarning):
|
||||
bars = await td.get_kline(Market.SH, "600000", KlinePeriod.DAY, count=240)
|
||||
|
||||
await td.get_kline(Market.SH, "600000", KlinePeriod.DAY, count=240)
|
||||
# 新写法(推荐):
|
||||
bars = await td.get_bars(["sz000001"], KlinePeriod.DAY, count=240)
|
||||
# 港股/美股/期货也一样:
|
||||
bars = await td.get_bars(["hk00700", "usAAPL", "cffex:IFL0"], KlinePeriod.DAY, count=240)
|
||||
```
|
||||
|
||||
**3. 返回 `list[dataclass]`,不是 DataFrame**
|
||||
@@ -48,6 +52,7 @@ df = pd.DataFrame([asdict(b) for b in bars])
|
||||
|
||||
```bash
|
||||
pip install pytdxdata # 或 uv add pytdxdata
|
||||
pip install "pytdxdata[polars]" # 需要 DataFrame 输出时才装(可选 extras)
|
||||
```
|
||||
|
||||
## 可运行模板
|
||||
@@ -55,34 +60,42 @@ pip install pytdxdata # 或 uv add pytdxdata
|
||||
```python
|
||||
import asyncio
|
||||
from pytdxdata import TdxData
|
||||
from pytdxdata.models import Market, KlinePeriod, Adjust
|
||||
from pytdxdata.models import KlinePeriod, Adjust
|
||||
|
||||
async def main():
|
||||
async with TdxData() as td:
|
||||
# 日K线,240 根(count 超过单页大小会自动多连接并发分页)
|
||||
bars = await td.get_kline(Market.SH, "600000", KlinePeriod.DAY, count=240)
|
||||
bars = await td.get_bars(["sz000001"], KlinePeriod.DAY, count=240)
|
||||
print(bars[-1].close, bars[-1].datetime)
|
||||
|
||||
# 前复权日线(复权自动走 MAC 通道)
|
||||
qfq = await td.get_kline(Market.SZ, "300308", KlinePeriod.DAY,
|
||||
qfq = await td.get_bars(["sz300308"], KlinePeriod.DAY,
|
||||
count=500, adjust=Adjust.QFQ)
|
||||
|
||||
# 五档报价:入参是 (market, code) 元组列表,一次可传多只(>80 只自动切批)
|
||||
quotes = await td.get_quotes([(Market.SH, "600000"), (Market.SZ, "000001")])
|
||||
print(quotes[0].price, quotes[0].bid[0], quotes[0].ask[0])
|
||||
# 跨市场报价:A股+港股+美股一次拿全
|
||||
quotes = await td.get_quotes(["sz000001", "hk00700", "usAAPL"])
|
||||
print(quotes[0].price, quotes[1].name, quotes[2].amount)
|
||||
|
||||
# 逐笔成交(MAC 通道,支持历史日期,含 trade_count 成交笔数)
|
||||
ticks = await td.get_transactions(Market.SZ, "000001", date=20260811)
|
||||
print(sum(r.trade_count for r in ticks)) # 当日总成交笔数
|
||||
ticks = await td.get_ticks(["sz000001"], date=20260811)
|
||||
print(sum(r.trade_count for r in ticks))
|
||||
|
||||
# 分时(MAC 通道,今日或历史)
|
||||
minute = await td.get_minute(Market.SH, "600000")
|
||||
minutes = await td.get_minutes(["sz000001"])
|
||||
|
||||
# 批量 K 线:多股票自动并发,池子会分配到不同服务器
|
||||
batch = await td.get_kline_batch(
|
||||
[(Market.SH, "600000"), (Market.SZ, "000001")],
|
||||
# 批量 K 线:≥16 只自动走「多连接 × 单连接流水线」,池子把标的摊到多台服务器
|
||||
batch = await td.get_bars(
|
||||
["sz000001", "sh600000"],
|
||||
KlinePeriod.DAY, count=240,
|
||||
) # -> {"600000": [...], "000001": [...]}
|
||||
) # 平铺返回,每条带 .symbol 字段
|
||||
|
||||
# 全市场盘中实时:只拉已经走完的根数(10:00 → 30 根),5,200 只 1~2 秒
|
||||
from datetime import datetime
|
||||
live = await td.get_bars(
|
||||
[s.symbol for s in await td.get_universe("sh")],
|
||||
KlinePeriod.MIN_1, start_date=datetime.now(),
|
||||
)
|
||||
# pipeline=1 可关闭批量路径,退回逐只并发(结果一致,只是慢)
|
||||
|
||||
asyncio.run(main())
|
||||
```
|
||||
@@ -121,38 +134,40 @@ asyncio.run(main())
|
||||
|
||||
**`ExMarket`** — 扩展市场(港股 / 美股 / 期货 / 期权,共 46 个成员,走独立 EX 通道)
|
||||
|
||||
常用几个:`HK_MAIN_BOARD=31`、`US_STOCK=74`、`CFFEX_FUTURES=47`、`SH_STOCK_OPTION=8`、`SZ_STOCK_OPTION=9`。
|
||||
完整表见 <https://openbot-coder.github.io/pytdxdata/api/enums/>。
|
||||
v0.6 起统一接口用**字符串前缀**指定扩展市场,一般不再需要 `ExMarket`:
|
||||
|
||||
```python
|
||||
from pytdxdata.models import ExMarket
|
||||
|
||||
await td.get_kline(ExMarket.HK_MAIN_BOARD, "00700", KlinePeriod.DAY, count=700) # 港股,5 位代码
|
||||
await td.get_kline(ExMarket.US_STOCK, "AAPL", KlinePeriod.DAY, count=700) # 美股,ticker
|
||||
await td.get_kline(ExMarket.CFFEX_FUTURES, "IFL0", KlinePeriod.DAY, count=700) # 期货合约
|
||||
await td.get_bars("hk00700", KlinePeriod.DAY, count=700) # 港股(5 位代码)
|
||||
await td.get_bars("usAAPL", KlinePeriod.DAY, count=700) # 美股(ticker)
|
||||
await td.get_bars("cffex:IFL0", KlinePeriod.DAY, count=700) # 期货合约
|
||||
```
|
||||
|
||||
常用前缀:`hk` / `us` / `cffex` / `shf` / `dl` / `zz` / `gz`;完整表见 <https://openbot-coder.github.io/pytdxdata/api/enums/>。
|
||||
|
||||
## 想做什么 → 用哪个方法
|
||||
|
||||
| 需求 | 方法 |
|
||||
|---|---|
|
||||
| 日/周/月/分钟 K 线 | `get_kline(market, code, period, *, count, adjust)` |
|
||||
| 增量更新(从某天往后) | `get_kline_since(market, code, period, since, adjust)` |
|
||||
| 一次拉多只股票 K 线 | `get_kline_batch(stocks, period, *, count, adjust, concurrency)` |
|
||||
| 指数 K 线 | `get_index_kline(market, code, period, *, count)` |
|
||||
| K 线 + MACD/KDJ/RSI/BOLL | `get_stock_kline_with_indicators(market, code, indicators, ...)` |
|
||||
| 实时五档报价 | `get_quotes(stocks)` |
|
||||
| 逐笔成交 | `get_transactions(market, code, *, date)` |
|
||||
| 分时图 | `get_minute(market, code, *, date)` |
|
||||
| 全市场股票列表 | `get_security_list_all(market)`(缓存 1 天) |
|
||||
| 除权除息 / 财务快照 | `get_xdxr(...)` / `get_finance(...)` |
|
||||
| 板块列表 / 成分股 / 涨跌排行 | `get_board_list` / `get_board_members` / `get_board_ranking` |
|
||||
| 个股所属板块 | `get_belong_board(market, code)` |
|
||||
| 全市场标的清单(A股+港股+期货…;**北交所返回空**) | `get_universe(market=None)` |
|
||||
| 日/周/月/分钟 K 线(任意市场) | `get_bars(symbols, period, *, count, adjust, start_date, end_date)` |
|
||||
| K 线 + MACD/KDJ/RSI/BOLL | `get_indicators(symbols, indicators, ...)` |
|
||||
| 实时五档报价(A股)/ 基础报价(港股/美股/期货) | `get_quotes(symbols, *, fields=None, pipeline=True)` |
|
||||
| 逐笔成交(任意市场) | `get_ticks(symbols, *, date)` |
|
||||
| 分时图 / 缩略采样 | `get_minutes(symbols, *, date, sampling=True)` |
|
||||
| 全市场股票列表(缓存 1 天) | `get_universe("sz")` / `get_universe("sh")` |
|
||||
| ~~北交所全量代码~~ | 拿不到:清单命令对 market=2 无响应(27 台全测)。北交所标的仍可直接查 `bj920992` |
|
||||
| 除权除息 / 财务快照 | `get_xdxr(symbols)` / `get_finance(symbols)` |
|
||||
| F10 公司资料 | `get_f10(symbol, section)` |
|
||||
| 板块列表 / 成分股 / 涨跌排行 | `get_boards` / `get_board_members` / `get_board_ranking` |
|
||||
| **指数的成交量 + 成交笔数**(每分钟曲线) | `get_index_trade_stats(index, date)` → `.cum_trade_count()` |
|
||||
| **指数成分股** | `get_index_members(index)`(裸 `000001` 按指数语义解析) |
|
||||
| 个股所属板块 | `get_board_of(symbols)` |
|
||||
| 集合竞价 / 异动 / 资金流 | `get_auction` / `get_unusual` / `get_capital_flow` |
|
||||
| 个股特征快照 | `get_snapshot(symbols)` |
|
||||
| **板块成分文件**(离线快照) | `get_block_parsed("block_gn.dat")` |
|
||||
| **行业分类**(通达信 + 申万) | `get_industry_map()` |
|
||||
| **历史专业财报**(gpcw 全字段) | `get_financial_file_infos()` + `get_financial_records_parsed(fn)` |
|
||||
| 涨跌停价 | `get_price_limits(market, code, name, pre_close, listed_days)`(本地算,无网络) |
|
||||
| 涨跌停价 | `get_price_limits(symbol, name, pre_close, listed_days)`(本地算,无网络) |
|
||||
| 全市场涨跌家数/成交额 | `get_market_stat()` |
|
||||
|
||||
## 通道与数据特性(影响你能拿到什么)
|
||||
@@ -160,25 +175,66 @@ await td.get_kline(ExMarket.CFFEX_FUTURES, "IFL0", KlinePeriod.DAY, count=700)
|
||||
库内部有**标准通道**和 **MAC 通道**两条链路,会**按命令自动选路**,你不用管。但有几件事必须知道:
|
||||
|
||||
- **数据有 15 分钟延时**(通达信公开服务器特性)。要实时数据请用券商行情。
|
||||
- **复权 K 线与 1 分钟 K 线走 MAC 通道**。`get_kline(..., adjust=Adjust.QFQ)` 自动切换。
|
||||
- **复权 K 线与 1 分钟 K 线走 MAC 通道**。`get_bars(..., adjust=Adjust.QFQ)` 自动切换。
|
||||
- **K 线时间戳统一为北京时间**(v0.6.2 起)。美股原始报文里日期是美东交易日、时分是北京时间(两个时区拼在一起),库已自动归一化,`bar.datetime` 可直接排序 / 比较 / 过滤。但**美股一个交易日横跨两个北京自然日**,按交易日聚合(分钟线合成日线、日内统计)必须用 `bar.trading_date`。
|
||||
- **`count` 很大时会自动多连接并发分页**,无需手动翻页。但标准通道 K 线 `count < 100` 部分服务器会返回空——内部已强制 `min_page=100` 规避。
|
||||
- **`get_quotes` 一次 >80 只自动切批**;而 MAC 的 `get_stock_quotes` 上限就是 80 只/次。
|
||||
- **历史逐笔没有笔数字段**,只有当日逐笔的 `trade_count` 有效。
|
||||
- **`get_quotes` 一次 >80 只自动切批**;走 MAC 字段位通道(`fields=`)时上限为 80 只/次。
|
||||
- **全市场报价用默认参数即可,别自己写循环**。`get_quotes(全部A股)` 内部会切 66 片并走**单连接流水线**(一条连接连发多个分片、只付一次往返)+ 工作窃取调度,5,227 只实测 **p50 325ms**(老版本串行 3.5s)。自己 for 循环逐只/逐片调会把流水线优势全部丢掉。
|
||||
- 想更保守可 `concurrency=11`(每台 1 连接),代价 p50 457ms。
|
||||
- 个别服务器不支持流水线时自动降级逐片,结果不受影响。
|
||||
- **可转债(110/111/113/118/123/127/128)三样都是真的**:价格、`vol`(手=10 张)、`trade_count`(真实笔数,3631 条切片 / 20443 笔量级)全部可用,`Σvol` 与日线 `vol` **1:1 相等**。
|
||||
⚠️ 唯一要小心的是 **日线 `amount / vol` 不等于均价,而是均价的 10 倍**(`vol` 单位是手=10 张)。算可转债均价用 `amount / vol / 10`,或直接用 `Σ(price×vol) / Σvol`——后者跨品种都成立,最稳。
|
||||
对照:股票/ETF 的 `Σ逐笔vol / 日线vol = 0.01`(逐笔手、日线股),而 `amount/vol` 恰好就是均价。
|
||||
- **指数本身没有成交笔数**。`get_ticks("sh000001")` 的 `trade_count` 是「当时均价×1000」、`vol` 单位是百元(成交额/100)。要笔数只能用 `get_index_trade_stats(index)`(成分股逐笔累加,2230 只约 36-60 s)。北交所成分无笔数字段,该方法会返回 `trade_count_available=False`。
|
||||
- **ETF / LOF 也没有成交笔数**。`get_ticks("sh510310")` 的 `trade_count` 是「价格×1000」(货币 ETF 恒为 0)。判定用 `symbols.has_order_count(market, code)`,读记录时看 `TransactionRecord.trade_count_available`——为 `False` 时 `trade_count` 已被置 0,不要求和/求均值。注意**沪市 `501x/502x/506x` 实测有真实笔数**,不能按「5 开头=基金」一刀切。
|
||||
- **历史逐笔有笔数**(MAC `0x122F` 支持历史日期,实测可回溯最近 5 个交易日;更早返回空)。
|
||||
- **小数位表不用手动管,但可以手动刷**。`start()` 会检查当日缓存并后台同步一次(`SecurityInfo.decimal_point` 权威值);要强制刷新用 `await td.sync_decimal_points()`,看同步状态用 `await td.decimal_point_status()`。`PYPYTDX_DATA_DISABLE_DECIMAL_SYNC=1` 可关闭后台同步。
|
||||
- 板块文件(`block_*.dat`)/ 行业(`tdxhy.cfg`)/ 财报(`gpcw*.zip`)是**服务器上的静态文件**,解析后是普通 dataclass,**落 JSON 后完全脱网可用**——适合做离线快照。
|
||||
|
||||
## DataFrame 输出(v0.7+)
|
||||
|
||||
表格型方法可以直出 DataFrame,**唯一开关是构造参数 `df`**,不要在调用处传 `df=True`(没有这个参数):
|
||||
|
||||
```python
|
||||
async with TdxData(df="polars") as td: # 或 df="pandas"
|
||||
df = await td.get_bars("sh600000", KlinePeriod.DAY, count=240)
|
||||
```
|
||||
|
||||
| 事项 | 说明 |
|
||||
|---|---|
|
||||
| 安装 | `pip install "pytdxdata[polars]"` / `[pandas]` / `[dataframe]`;核心包仍零依赖 |
|
||||
| `df=None`(默认) | 行为与 0.6.x 完全一致,返回 `list[dataclass]` |
|
||||
| 报错 | `df` 取值非法 → `ValueError`;包没装 → `FrameBackendError`(启动即失败,不拖到第一次查询) |
|
||||
| 时间列 | `year/month/day/hour/minute` **被 `datetime` 替换**,另带 `trading_date`;`XdxrRecord` 给 `date` |
|
||||
| 嵌套 | `bid`/`ask` 展开成 `bid1_price`…`ask5_vol` 共 10 列;`fields` 字典 JSON 化成字符串列;`codes` 保持列表 |
|
||||
| 特殊容器 | `get_indicators` → bars+指标合并;`get_index_trade_stats(_multi)` → `minutes` 展开 + `index`/`date` 列;`get_kline_batch` / `get_minute_batch` → 纵向拼接成一张表 |
|
||||
| 不受影响 | `bytes` / `int` / `dict` / 单对象 / `tuple` / `list[float]` 类返回原样(`get_file`、`get_security_count`、`get_server_info`、`get_price_limits` 等) |
|
||||
| 空结果 | 列名齐全的空表;dtype 只由后端推断 |
|
||||
|
||||
完整口径见 [DataFrame 输出](https://openbot-coder.github.io/pytdxdata/dataframe/)。
|
||||
|
||||
## 陷阱清单
|
||||
|
||||
| 坑 | 后果 | 正确做法 |
|
||||
|---|---|---|
|
||||
| `market` 传 `"sh"` 字符串 | 直接报错或路由到错误市场 | 用 `Market.SH` 枚举 |
|
||||
| 以为 `KlinePeriod.DAY == 0` | 拿到的是 5 分钟线 | `DAY=4`,对照上面的表 |
|
||||
| 统一方法(`get_quotes`/`get_bars`…)传 `(Market.SH, "600000")` 元组或市场号 | v0.6 起直接报错 | 用字符串 `"sh600000"` |
|
||||
| 旧别名(`get_kline`/`get_transactions`…)传 `(market, code)` | 能跑,但发出 `DeprecationWarning` | 迁移到统一方法 + 字符串 |
|
||||
| 忘了 `async with` / `start()` | 无连接池,运行时报错 | 一律 `async with TdxData() as td:` |
|
||||
| `KlinePeriod.DAY == 0` | 拿到的是 5 分钟线 | `DAY=4`,对照枚举表 |
|
||||
| 等在返回值上找 `.to_dataframe()` | 不存在 | `pd.DataFrame([asdict(x) for x in bars])` |
|
||||
| 把 `close` 当成分 | 数值差 100 倍 | 价格单位是**元**(float) |
|
||||
| `SH 000001` 当成平安银行 | 拿到的是上证指数 | **代码不唯一**:`(Market.SH, "000001")`=上证指数,`(Market.SZ, "000001")`=平安银行 |
|
||||
| `sh000001` 当成平安银行 | 拿到的是上证指数 | **代码不唯一**:`sh000001`=上证指数,`sz000001`=平安银行 |
|
||||
| 用 `get_block_info` 又自己拆二进制 | 白写解析器 | 用 `get_block_parsed`,已经解析好了 |
|
||||
| 循环里逐个 `await get_kline` | 慢 | 用 `get_kline_batch` 让它并发 |
|
||||
| 循环里逐个 `await get_bars` | 慢 | 用列表传多个标的,自动并发 |
|
||||
| 港股/美股五档 bid/ask | 没有(协议不支持) | `fields` 字段位通道有 name 但无五档 |
|
||||
| 美股分钟线用 `datetime.date()` 分组做日聚合 | 一个交易日被拆成两天(美股跨北京零点) | 用 `bar.trading_date`;`bar.datetime` 是归一化后的北京时间 |
|
||||
| 拿指数逐笔的 `trade_count` 当笔数 | 拿到的是「均价×1000」,数值荒谬 | 用 `get_index_trade_stats(index)` 由成分累加 |
|
||||
| 拿 ETF/LOF 逐笔的 `trade_count` 当笔数 | 拿到的是「价格×1000」(`sh510310` 显示 3994 笔) | 看 `trade_count_available`,或 `symbols.has_order_count()` |
|
||||
| 拿深市 `399xxx` 的 `amount` 校验成分合计 | 比值 0.43,误以为数据错 | 深市指数 amount 是**全板块**口径(`sz399001`=深市全部 5081 亿),不是样本股 |
|
||||
| 在期货/期权 K 线上取 `amount` | 恒 0.0,误以为解析坏了 | EX `0x23FF` 帧内**没有成交额**(协议限制);结算价看 `bar.settle`,成交额改用 `get_quotes` |
|
||||
| 用港股主板的 `amount / vol` 当均价 | 差 **100 倍** | 港股主板/基金的 `vol` 单位是**手(=100 股)**,要 `amount/vol/100`;港股创业板/美股才是股 |
|
||||
| 依赖实时性做信号 | 数据是 15 分钟前的 | 换券商实时源 |
|
||||
| 手动改包内 `servers.json` / 写死 IP | 容器只读、升级被覆盖、快节点捞不回来 | 日常靠内置的**当日测速缓存**自动选路;要强制指定就用 `PYPYTDX_DATA_{STANDARD,MAC,EX}_HOSTS` 环境变量 |
|
||||
|
||||
## 深入参考
|
||||
|
||||
|
||||
+139
-32
@@ -1,6 +1,6 @@
|
||||
# pytdxdata 对外接口清单
|
||||
|
||||
> 以 `src/pytdxdata` 源码为准整理(v0.5.0)。分四类:Python API / 数据模型 / 辅助模块 / CLI。
|
||||
> 以 `src/pytdxdata` 源码为准整理(v0.7.0)。分四类:Python API / 数据模型 / 辅助模块 / CLI。
|
||||
>
|
||||
> 方法与代码的一致性由 `scripts/check_docs.py` 在 CI 中校验。
|
||||
|
||||
@@ -28,8 +28,9 @@ from pytdxdata import (
|
||||
| `cache_dir` | None | 磁盘缓存目录;None=仅内存缓存 |
|
||||
| `default_adjust` | `Adjust.NONE` | 默认复权方式 |
|
||||
| `max_inflight` | 64 | single-flight 并发上限 |
|
||||
| `pool_min` / `pool_max` | 3 / 12 | 动态池上下限 |
|
||||
| `mac_pool_cap` | 4 | 单台 MAC 服务器连接上限 |
|
||||
| `pool_min` / `pool_max` | 3 / 24 | 动态池上下限 |
|
||||
| `mac_pool_cap` / `std_pool_cap` / `ex_pool_cap` | 4 / 4 / 2 | 单台服务器连接上限 |
|
||||
| `df` | None | **DataFrame 后端**:`None`=返回 `list[dataclass]`(0.6.x 行为)、`"polars"`、`"pandas"`。需先装 extra:`pip install pytdxdata[polars]`,详见 [DataFrame 输出](dataframe.md) |
|
||||
|
||||
生命周期:
|
||||
|
||||
@@ -39,36 +40,88 @@ from pytdxdata import (
|
||||
| `async close()` | 关闭全部池与缓存 |
|
||||
| `async with TdxData() as td:` | 上下文管理器,自动 start/close |
|
||||
|
||||
### 2.1 K线(共 6 个)
|
||||
### 2.1 统一接口(v0.6+ 推荐){#unified}
|
||||
|
||||
> **DataFrame(v0.7+)**:本节的表格型方法在 `TdxData(df="polars")` / `df="pandas"` 时
|
||||
> 直接返回 DataFrame;`df=None`(默认)返回下表的 `list[...]`。列口径、时间列替换规则、
|
||||
> 不受影响的接口清单见 [DataFrame 输出](dataframe.md)。
|
||||
|
||||
以下方法在 v0.6 新增,标的一律用字符串(`sz000001` / `hk00700` / `usAAPL` / `cffex:IFL0`)。
|
||||
|
||||
#### 标的清单
|
||||
|
||||
| 方法 | 返回 | 说明 |
|
||||
|------|------|------|
|
||||
| `get_kline(market, code, period, *, start=0, count=800, adjust=None, start_date=None, end_date=None)` | `list[SecurityBar]` | 主接口;count>页自动并发分页;复权走 MAC;支持日期区间(二分定位) |
|
||||
| `get_universe(market=None)` | `list[SecurityInfo]` | 跨市场标的清单(A 股+港股+美股+期货/期权);每条带协议自带的 `decimal_point`(价格小数位)与 `pre_close`。**北交所清单协议不支持,返回空列表** |
|
||||
| `sync_decimal_points(markets=("sh","sz"))` | `dict` | 从证券清单同步**权威价格小数位表**(`SecurityInfo.decimal_point`),灌进 `price_decimal_digits` 的运行时表;`start()` 已自动做(当日缓存命中则零网络),此方法用于强制刷新/排障 |
|
||||
| `decimal_point_status()` | `dict` | 小数位表状态:`source`(`cache`/`universe`/空)、`stock`、`mixed_prefixes`、`syncing` |
|
||||
|
||||
#### 报价
|
||||
|
||||
| 方法 | 返回 | 说明 |
|
||||
|------|------|------|
|
||||
| `get_quotes(symbols, *, fields=None, concurrency=None, pipeline=True)` | `list[SecurityQuote]` | 跨市场混合报价(A 股标准五档;给了 fields 走 MAC/EX 字段位有 name)。超 80 只自动切批,`pipeline` 开启**单连接流水线**,全市场 5,200 只 ~0.33s |
|
||||
| `get_snapshot(symbols)` | `list[SymbolSnapshot]` | 个股特征快照(A 股) |
|
||||
| `get_auction(symbols)` | `list[AuctionItem]` | 集合竞价(A 股) |
|
||||
| `get_capital_flow(symbols)` | `list[CapitalFlow]` | 个股资金流向(A 股) |
|
||||
|
||||
#### K线
|
||||
|
||||
| 方法 | 返回 | 说明 |
|
||||
|------|------|------|
|
||||
| `get_bars(symbols, period=DAY, *, start=0, count=800, adjust=None, start_date=None, end_date=None, concurrency=6, pipeline=None)` | `list[SecurityBar]` | 跨市场 K 线(自动判指数、复权走 MAC、并发分页;日期区间贴最新走倒推、历史中段走二分)。**≥16 只自动走多连接流水线**:`pipeline` 为每条连接的在途请求数(默认按单请求字节自适应 8~32,传 `1` 关闭),全市场 1 分钟 K 线取当天 43.6s → 5s 级 |
|
||||
| `get_indicators(symbols, indicators, *, period=DAY, count=30, adjust=None, params=None)` | `list[IndicatorSet]` | K 线+技术指标(MACD/KDJ/RSI/BOLL/MA/EMA) |
|
||||
|
||||
#### 逐笔与分时
|
||||
|
||||
| 方法 | 返回 | 说明 |
|
||||
|------|------|------|
|
||||
| `get_ticks(symbols, *, date=None, start=0, count=None, concurrency=6)` | `list[TransactionRecord]` | 跨市场逐笔成交。返回按时间升序;`start` **从最新一笔往回数**;`vol`/`trade_count` 为约 3 秒切片的增量值(非累计)。**已剔除 `bs_flag=8` 竞价委托**(非成交,`vol` 为累计撮合量)。**指数**:`vol` 单位是百元(成交额/100)、`trade_count` 是均价×1000,均非股票语义;**可转债**:`vol` 是手(=10 张)、`trade_count` 是**真实笔数**,`Σvol` 与日线 `vol` **1:1** |
|
||||
| `get_minutes(symbols, *, date=None, sampling=False, concurrency=6)` | `list[MinuteBar]` | 跨市场分时(sampling=True 取缩略采样:A股由 0x122D 派生,扩展市场 0x254D) |
|
||||
|
||||
#### 公司信息/文件/板块
|
||||
|
||||
| 方法 | 返回 | 说明 |
|
||||
|------|------|------|
|
||||
| `get_f10(symbol, section=None, *, offset=0, length=65535)` | `list \| str` | F10 公司资料 |
|
||||
| `get_file(name)` | `bytes` | 服务器文件原始字节(自动选通道) |
|
||||
| `get_boards(kind="all", *, count=10000)` | `list[BoardInfo]` | 板块列表 |
|
||||
| `get_board_members(board, *, fields=None, count=100000)` | `list[SecurityQuote]` | 板块成分股报价(归一模型) |
|
||||
| `get_board_of(symbols)` | `list[BelongBoard]` | 个股所属板块 |
|
||||
| `get_index_members(index, *, fields=None, count=100000)` | `list[SecurityQuote]` | **指数成分股**报价。裸 6 位按**指数语义**解析(`000001`=上证指数,不是平安银行):`000/999/880`→沪、`399`→深、`899`→北交所 |
|
||||
| `get_index_trade_stats(index, date=None, *, concurrency=24, batch_size=50, members=None, skip_errors=True, on_progress=None)` | `IndexTradeStats` | **指数成分逐笔汇总**成每分钟成交量/成交笔数。协议里指数本身没有笔数(`trade_count` 是均价×1000),故由成分股逐笔累加;`.cumulative()` 取「截止该分钟累计」,`.cum_trade_count()` 直接给 `[(HH:MM, 累计笔数)]`。2230 只(上证)约 36-60 s |
|
||||
| `get_index_trade_stats_multi(index, dates, *, ...)` | `list[IndexTradeStats]` | 多日批量版(成分只请求一次),用于「近 N 日同时点累计均值」基准 |
|
||||
|
||||
### 2.2 K线(旧接口,共 6 个) {#legacy-kline}
|
||||
|
||||
| 方法 | 返回 | 说明 |
|
||||
|------|------|------|
|
||||
| `get_kline(market, code, period, *, start=0, count=800, adjust=None, start_date=None, end_date=None)` | `list[SecurityBar]` | 主接口;count>页自动并发分页;复权走 MAC;支持日期区间(贴最新倒推 / 中段二分) |
|
||||
| `get_kline_since(market, code, period, since, adjust=None)` | `list[SecurityBar]` | 增量便捷方法(≥since,含当天) |
|
||||
| `get_kline_batch(stocks, period, *, start=0, count=800, adjust=None, concurrency=6)` | `dict[str, list[SecurityBar]]` | 多股票并发,返回 {code: bars} |
|
||||
| `get_index_kline(market, code, period, *, start=0, count=800)` | `list[SecurityBar]` | 指数K线(含涨跌家数字段) |
|
||||
| `get_stock_kline_with_indicators(market, code, indicators, *, period=DAY, count=30, adjust=None, params=None)` | `dict` | K线+技术指标(MACD/KDJ/RSI/BOLL/MA/EMA) |
|
||||
| `get_kline_offset(offset=0, count=1)` | `tuple[int, int]` | K线偏移信息(MAC) |
|
||||
|
||||
### 2.2 报价(共 4 个)
|
||||
### 2.3 报价(旧接口,共 4 个) {#legacy-quote}
|
||||
|
||||
| 方法 | 返回 | 说明 |
|
||||
|------|------|------|
|
||||
| `get_quotes(stocks)` | `list[SecurityQuote]` | 五档报价;>80 只自动切批;TTL 5s |
|
||||
| `get_quotes(symbols, *, fields=None, concurrency=None, pipeline=True)` | `list[SecurityQuote]` | 五档报价;>80 只自动切批(流水线+工作窃取,5,200 只 ~0.33s);TTL 5s |
|
||||
| `get_stock_quotes(stocks, bits=None)` | `list[MemberQuote]` | MAC 自定义字段报价(≤80只/次) |
|
||||
| `get_stock_quotes_list(category, *, start=0, count=80, sort_type=0)` | `list[MemberQuote]` | 市场分类报价(0沪A/2深A/6全A/8科创/14创业) |
|
||||
| `get_market_stat()` | `MarketStat` | 全市场统计(涨跌家数/总额/市值/涨跌停数) |
|
||||
|
||||
### 2.3 逐笔成交 / 分时(共 4 个)
|
||||
### 2.4 逐笔成交 / 分时(旧接口,共 4 个) {#legacy-tick}
|
||||
|
||||
| 方法 | 返回 | 说明 |
|
||||
|------|------|------|
|
||||
| `get_transactions(market, code, *, date=None, start=0, count=2000)` | `list[TransactionRecord]` | 逐笔成交(MAC,含 trade_count),支持历史日期 |
|
||||
| `get_minute(market, code, *, date=None)` | `list[MinuteBar]` | 分时(MAC 0x122D,可历史) |
|
||||
| `get_minute_batch(stocks, *, date=None, concurrency=6)` | `dict[str, list[MinuteBar]]` | 批量分时 |
|
||||
| `get_chart_sampling(market, code)` | `list[float]` | 分时缩略采样(A股,MAC 0x254D) |
|
||||
| `get_chart_sampling(market, code)` | `list[float]` | 分时缩略采样(A股由 MAC **0x122D** 派生;0x254D 仅 EX 通道可用) |
|
||||
|
||||
### 2.4 证券列表(共 3 个)
|
||||
### 2.5 证券列表(旧接口,共 3 个) {#legacy-list}
|
||||
|
||||
| 方法 | 返回 | 说明 |
|
||||
|------|------|------|
|
||||
@@ -76,30 +129,30 @@ from pytdxdata import (
|
||||
| `get_security_list(market, *, start=0, count=None)` | `list[SecurityInfo]` | count=None=全部(先查总数防死循环) |
|
||||
| `get_security_list_all(market)` | `list[SecurityInfo]` | 全市场列表(高并发一次拉完,缓存 1 天) |
|
||||
|
||||
### 2.5 除权 / 财务 / F10 / 文件(共 15 个)
|
||||
### 2.6 除权 / 财务 / F10 / 文件(旧接口,共 15 个) {#file-methods}
|
||||
|
||||
| 方法 | 返回 | 说明 |
|
||||
|------|------|------|
|
||||
| `get_xdxr(market, code)` | `list[XdxrRecord]` | 除权除息历史(标准) |
|
||||
| `get_finance(market, code)` | `FinanceRecord \| None` | 财务快照(标准) |
|
||||
| `get_price_limits(market, code, name, pre_close, listed_days=9999)` | `tuple[float\|None, float\|None]` | 涨跌停价(本地计算) |
|
||||
| `get_xdxr(symbols)` | `list[XdxrRecord]` | 除权除息历史(标准) |
|
||||
| `get_finance(symbols)` | `list[FinanceRecord]` | 财务快照(标准;无数据的标的跳过) |
|
||||
| `get_price_limits(symbol, name=None, pre_close=None, listed_days=9999)` | `tuple[float\|None, float\|None]` | 涨跌停价(本地计算) |
|
||||
| `get_company_info_category(market, code)` | `list` | F10 目录(标准) |
|
||||
| `get_company_info_content(market, code, filename, offset=0, length=65535)` | `str` | F10 内容(分块) |
|
||||
| `get_block_info(filename)` | `bytes` | 板块文件**原始字节**(先查元数据再分块下载) |
|
||||
| `get_report_file(filename)` | `bytes` | 报告/财务文件**原始字节**(标准 0x06B9) |
|
||||
| `get_financial_file_list()` | `list[str]` | 财务文件列表**原始行**(tdxfin/gpcw.txt) |
|
||||
| `get_financial_records(filename)` | `bytes` | 财务文件内容(gpcw*.zip 原始字节) |
|
||||
| `get_financial_records(filename)` | `bytes` | 财报 `.dat` 原始字节(索引写 `.zip` 但服务器只有 `.dat` 时自动回退;取不到返回 `b""`) |
|
||||
| `get_file_meta(filename)` | `FileMeta` | 远程文件元信息(MAC 0x1215) |
|
||||
| `download_file(filename, filesize=0)` | `bytes` | 下载完整远程文件(MAC 0x1217,分块) |
|
||||
| **`get_block_parsed(filename)`** | `list[TdxBlock]` | 板块文件**下载+解析**为板块成分(category 按文件名推断:0行业/2概念/3风格) |
|
||||
| **`get_industry_map()`** | `dict[str, IndustryInfo]` | tdxhy.cfg **下载+解析**:`{代码: 通达信行业+申万行业}` |
|
||||
| **`get_financial_file_infos()`** | `list[FinancialFileInfo]` | gpcw.txt **下载+解析**:文件名/MD5/大小 |
|
||||
| **`get_financial_records_parsed(filename)`** | `list[FinancialRecord]` | gpcw*.zip **解压+解析**:每股原始 float 字段(报告期取自文件名,缺失回退头部) |
|
||||
| **`get_financial_records_parsed(filename)`** | `list[FinancialRecord]` | 财报**解析**:每股原始 float 字段。可直接传 `get_financial_file_infos()` 的 `filename`——自动纠正服务器实际文件名(`.zip`/`.dat` 两种形态都支持);报告期取自文件名,缺失回退头部 |
|
||||
|
||||
> 解析层独立于网络层:解析器位于 `pytdxdata.codec`(纯标准库 `struct`/`zipfile`),可脱网单测;
|
||||
> 原 `get_*` 返回 `bytes`/`list[str]` 的方法保持**向后兼容**,解析版为**新增**方法。
|
||||
|
||||
### 2.6 板块(共 6 个)
|
||||
### 2.7 板块(旧接口,共 6 个) {#legacy-board}
|
||||
|
||||
| 方法 | 返回 | 说明 |
|
||||
|------|------|------|
|
||||
@@ -110,17 +163,17 @@ from pytdxdata import (
|
||||
| `get_board_ranking(board_type=1, top_n=20)` | `list[dict]` | 板块涨跌幅排行 |
|
||||
| `get_board_change_ranking(board_type=1, days=20, top_n=20)` | `list[dict]` | 板块 N 日涨跌幅排行(板块指数K线) |
|
||||
|
||||
### 2.7 MAC 扩展:竞价 / 异动 / 快照 / 资金流 / 服务器(共 5 个)
|
||||
### 2.8 MAC 扩展:竞价 / 异动 / 快照 / 资金流 / 服务器(旧接口,共 5 个) {#legacy-mac}
|
||||
|
||||
| 方法 | 返回 | 说明 |
|
||||
|------|------|------|
|
||||
| `get_auction(market, code)` | `list[AuctionItem]` | 集合竞价 |
|
||||
| `get_unusual(market, start=0, count=600)` | `list[UnusualItem]` | 市场异动 |
|
||||
| `get_auction(symbols)` | `list[AuctionItem]` | 集合竞价 |
|
||||
| `get_unusual(market="all", *, start=0, count=600)` | `list[UnusualItem]` | 市场异动 |
|
||||
| `get_symbol_info(market, code)` | `SymbolSnapshot` | 个股特征快照 |
|
||||
| `get_capital_flow(market, code)` | `CapitalFlow` | 个股资金流向 |
|
||||
| `get_capital_flow(symbols)` | `list[CapitalFlow]` | 个股资金流向 |
|
||||
| `get_server_info()` | `ServerInfo` | 服务器交易时段信息 |
|
||||
|
||||
### 2.8 EX 扩展市场(港股/美股/期货/期权,共 8 个)
|
||||
### 2.9 EX 扩展市场(旧接口,港股/美股/期货/期权,共 8 个) {#legacy-ex}
|
||||
|
||||
| 方法 | 返回 | 说明 |
|
||||
|------|------|------|
|
||||
@@ -133,7 +186,8 @@ from pytdxdata import (
|
||||
| `get_goods_transaction(market, code, *, ymd=0, start=0, count=1800)` | `list[dict]` | 扩展市场逐笔(港股 0x23FC/0x2406) |
|
||||
| `get_goods_transaction_all(market, code, ymd=0)` | `list[dict]` | 港股全量逐笔(自动翻页,≤50页/9万条) |
|
||||
|
||||
**合计:`TdxData` 公开异步方法 51 个**(另有 `_execute_standard/_execute_mac/_execute_ex` 等内部执行方法)。
|
||||
**合计:`TdxData` 公开异步方法 63 个** = 28 个统一接口 + 33 个 deprecated 别名 + `start`/`close`
|
||||
(另有 `_execute_standard` / `_execute_mac` / `_execute_ex` 等内部执行方法)。
|
||||
|
||||
## 三、数据模型(`pytdxdata.models`)
|
||||
|
||||
@@ -150,12 +204,12 @@ from pytdxdata import (
|
||||
|
||||
| 模型 | 关键字段 |
|
||||
|------|---------|
|
||||
| `SecurityBar` | market/code/open/high/low/close/vol/amount/year/month/day/hour/minute + `datetime` 属性 |
|
||||
| `SecurityBar` | market/code/open/high/low/close/vol/amount/year/month/day/hour/minute/symbol/**settle** + `datetime` 属性(`settle` 结算价仅 EX 期货/期权日线有值,其余为 `None`) |
|
||||
| `IndexBar` | 继承 `SecurityBar`(指数K线) |
|
||||
| `SecurityQuote` | price/pre_close/OHLC/vol/amount/s_vol/b_vol/`bid[5]`/`ask[5]`/server_time/trading_status/cur_vol/rise_speed/limit_up/limit_down/decimal_point/open_amount |
|
||||
| `SecurityInfo` | market/code/name/volunit/decimal_point/pre_close |
|
||||
| `TransactionRecord` | market/code/time/price/vol/`trade_count`/bs_flag |
|
||||
| `MinuteBar` | price/vol/avg_price/hour/minute |
|
||||
| `MinuteBar` | price/vol/avg_price/hour/minute/day_offset + `datetime`/`trading_date` 只读属性(hour 恒为合法时钟值 0~23;美股一个交易日跨两个北京自然日,按交易日聚合用 `trading_date`) |
|
||||
| `XdxrRecord` | 分红送转/配股/缩股/行权 + 股本变动 + `date` 属性 |
|
||||
| `FinanceRecord` | 股本/资产/负债/营收/利润等 36 个字段(万元/万股) |
|
||||
| `BoardInfo` | 板块 + 领涨股信息 |
|
||||
@@ -174,6 +228,31 @@ from pytdxdata import (
|
||||
|
||||
> 注:`__init__` 仅导出 `Market/ExMarket/KlinePeriod/Adjust` 与 6 个核心模型;`BoardInfo/MemberQuote/AuctionItem` 等需 `from pytdxdata.models import ...`。
|
||||
|
||||
**量纲对照(跨品种,2026-10-22 实测)**
|
||||
|
||||
同名字段在不同品种上单位不同。最典型的坑是用 `amount / vol` 算均价——它在可转债上会**错 10 倍**:
|
||||
|
||||
| 品种 | 分时 / 逐笔 `vol` | 日线 `SecurityBar.vol` | `Σ分时vol / 日线vol` | 日线 `amount / vol` |
|
||||
|---|---|---|---|---|
|
||||
| 股票 / ETF / LOF | 手(= 100 股) | 股 | **0.01** | 均价 ✓ |
|
||||
| 可转债(110/111/113/118/123/127/128) | 手(= 10 张) | **手** | **1.0** | 均价 **×10** ⚠️ |
|
||||
| 指数 | 百元(成交额/100) | 手 | — | 无意义(`amount` 是全板块口径) |
|
||||
| 港股主板 31 / 港股基金 49 | — | **手(= 100 股)** | — | 均价 **×100** ⚠️ |
|
||||
| 港股创业板 48 / 新三板 44 / 美股 74 | — | 股 | — | 均价 ✓ |
|
||||
| 期货 / 期权(EX 全市场) | — | 手/张 | — | **`amount` 恒 0.0**(帧内无成交额,见下) |
|
||||
|
||||
算均价:股票 `amount / vol`;可转债 `amount / vol / 10`;港股主板/基金 `amount / vol / 100`;跨品种统一用 `Σ(price×vol) / Σvol`。
|
||||
|
||||
> ⚠️ **EX 通道(0x23FF)的成交额不是全市场都有的**。一条记录 32B = 4B 时间 + 28B 行情,
|
||||
> 28B 里只有**一个** 4B 槽位可复用,语义按市场分两类:
|
||||
> 权益类(港股主板/创业板/基金、新三板、美股)该槽位是**成交额**,`amount` 正常;
|
||||
> 期货期权类该槽位是**结算价**(体现在 `SecurityBar.settle`,日线才有值),
|
||||
> `amount` 恒 **0.0** —— 这是协议限制,不是解析漏读,要成交额请改用 `get_quotes` 或其它源。
|
||||
> 港股指数(27)/ 中证指数(62)该槽位语义未明,同样保守返回 `0.0`。
|
||||
|
||||
> 可转债的价格标度、分时、逐笔已全量校验(16 只当日 + 8 只历史日):分时末价 / 快照现价、`Σvol / 日线vol`、逐笔 VWAP vs `amount/(vol×10)` 比值均为 **1.000000**。
|
||||
|
||||
|
||||
## 四、辅助模块
|
||||
|
||||
**`pytdxdata.indicator`**(技术指标,纯函数)
|
||||
@@ -188,17 +267,36 @@ from pytdxdata import (
|
||||
| `boll(values, period=20, mult=2.0)` | 布林带 |
|
||||
| `compute_indicators(high, low, close, indicators, params=None, tail=None)` | 统一入口(被 `get_stock_kline_with_indicators` 调用) |
|
||||
|
||||
**`pytdxdata.config`**(服务器清单)
|
||||
**`pytdxdata.config`**(服务器清单与选路)
|
||||
|
||||
| 函数 | 说明 |
|
||||
主机来源优先级链(各协议独立):`PYPYTDX_DATA_*_HOSTS` 环境变量 → **当日测速缓存** →
|
||||
包内清单 `fast_pool` → 包内清单 → 代码内置常量。
|
||||
|
||||
| 函数 / 常量 | 说明 |
|
||||
|------|------|
|
||||
| `get_standard_hosts()` | 标准通道服务器列表 |
|
||||
| `get_standard_hosts()` | 标准通道服务器列表(按上述优先级链解析) |
|
||||
| `get_mac_hosts()` | MAC 通道服务器列表 |
|
||||
| `get_ex_hosts()` | EX 扩展市场服务器列表 |
|
||||
| `get_ex_handshakes()` | EX 握手方式映射 |
|
||||
| `all_probe_candidates()` | 全量候选主机 `{standard, mac, ex}`(清单 ∪ 常量 ∪ env,**不按状态过滤**) |
|
||||
| `is_valid_host(host)` | 主机名合法性校验(挡 mock 脏数据 / 非法条目) |
|
||||
| `probe_cache_dir()` / `probe_cache_file()` | 用户级当日测速缓存目录 / 文件路径 |
|
||||
| `load_probe_cache(today=None, path=None)` | 读当日测速缓存;缺失 / 损坏 / 非今日 / 版本不符 → `None` |
|
||||
| `probe_disabled()` | 是否已用 `PYPYTDX_DATA_DISABLE_PROBE` 关闭测速 |
|
||||
| `PROBE_CACHE_VERSION` | 测速缓存 payload 版本号 |
|
||||
|
||||
**`pytdxdata.server_probe`**(服务器探测,可 `python -m pytdxdata.server_probe` 运行)
|
||||
`load_servers()` / `save_servers(cfg)` / `SERVERS_FILE`
|
||||
**`pytdxdata.pool.probe_cache`**(当日测速:后台执行、对用户无感)
|
||||
|
||||
| 符号 | 说明 |
|
||||
|------|------|
|
||||
| `ProbeCache` | 进程内协调器;`ensure_probe()` 为幂等入口(单飞 + 原子写) |
|
||||
| `schedule_probe(hosts=None)` | 连接池用:挂后台 task 并返回该 task;无事件循环时返回 `None` |
|
||||
| `get_probe_cache()` | 取进程级单例 |
|
||||
|
||||
**`pytdxdata.server_probe`**(离线服务器探测,`python -m pytdxdata.server_probe`)
|
||||
`load_servers()` / `save_servers(cfg)` / `SERVERS_FILE` / `probe_one(server, timeout, discover)`
|
||||
CLI 开关:`--discover`(三种协议全测并据实修正 `protocols`,同时剔除脏记录)/
|
||||
`--group standard|mac|ex` / `--refresh`(只重测 `status=ok` 的)/ `--timeout`
|
||||
|
||||
## 五、CLI(`tdx` 命令,来自 `python -m pytdxdata.cli`)
|
||||
|
||||
@@ -220,7 +318,7 @@ from pytdxdata import (
|
||||
| `tdx block [filename] [--limit 30] [--codes]` | 板块文件 下载+解析(block_*.dat → 成分股) |
|
||||
| `tdx industry [codes...] [--limit 30] [--search 词]` | 行业分类 下载+解析(tdxhy.cfg) |
|
||||
| `tdx financial-list [--limit 30]` | 历史专业财报文件索引(gpcw.txt) |
|
||||
| `tdx financial <filename> [--code X] [--limit 5] [--fields 4]` | 历史专业财报记录 下载+解析(gpcw*.zip) |
|
||||
| `tdx financial <filename> [--code X] [--limit 5] [--fields 4]` | 历史专业财报记录 下载+解析(文件名自动纠正,见 `get_financial_records_parsed`) |
|
||||
| `tdx server-info` | 服务器交易时段 |
|
||||
| `tdx market-stat` | 市场统计 |
|
||||
|
||||
@@ -237,4 +335,13 @@ from pytdxdata import (
|
||||
| 逐笔成交 / 分时 | MAC |
|
||||
| 列表 / 数量 / 除权 / 财务 | 标准 |
|
||||
| 板块 / 竞价 / 异动 / 快照 / 资金流 / 服务器 / 分类报价 | MAC |
|
||||
| 港股 / 美股 / 期货 / 期权 | EX(端口 7727) |
|
||||
| 港股 / 美股 / 期货 / 期权 | EX(端口 7727,`0x23FF`) |
|
||||
|
||||
> EX 通道的 K 线(`0x23FF`)字段口径与 A 股不同,见「量纲对照」下的注:
|
||||
> 权益类市场(港股/美股/新三板)有 `amount`;期货期权有 `settle`(结算价)但**没有** `amount`。
|
||||
|
||||
**多标的 K 线(`get_bars` 传 ≥16 只)额外走「批量流水线」**:按上表选池后,
|
||||
多条连接并行、每条连接一次下发 `pipeline` 条请求(默认自适应 8~32),把 N 次往返
|
||||
折叠成 `N/pipeline` 次。只用实测最快的 12 台服务器(池里 24 台全用会被慢节点拖长尾)。
|
||||
`start_date=今天` 时只拉已走过的根数(10:00 → 30 根),盘中实时刷新因此只要 1~2 秒。
|
||||
流水线失败时该批自动交回逐只重试,不会丢数据。
|
||||
+10
-9
@@ -6,7 +6,7 @@ pytdxdata 的枚举取值**完全没有直觉**(`KlinePeriod.DAY` 是 `4` 不
|
||||
| 文件 | 面向 | 内容 |
|
||||
|---|---|---|
|
||||
| [`llms.txt`](https://openbot-coder.github.io/pytdxdata/llms.txt) | AI 爬虫 / 问答型模型 | 心智模型、能力分组、关键陷阱,自动随站点发布 |
|
||||
| [`SKILL.md`](https://github.com/openbot-coder/pytdxdata/blob/main/SKILL.md) | AI **编程**助手 | 必守约定、枚举取值表、可运行模板、任务→方法决策表、9 条陷阱 |
|
||||
| [`SKILL.md`](https://github.com/openbot-coder/pytdxdata/blob/main/SKILL.md) | AI **编程**助手 | 必守约定、枚举取值表、可运行模板、任务→方法决策表、11 条陷阱 |
|
||||
|
||||
`llms.txt` 解决「这个库能干什么」,`SKILL.md` 解决「代码该怎么写对」。
|
||||
|
||||
@@ -25,12 +25,12 @@ curl -o ~/.workbuddy/skills/pytdxdata/SKILL.md \
|
||||
|
||||
## 里面有什么
|
||||
|
||||
- **三条必守约定** —— 唯一入口是 `TdxData` 且必须 `async with`;`market` 是整数枚举而不是字符串;
|
||||
返回值是 `list[dataclass]` 而不是 DataFrame
|
||||
- **三条必守约定** —— 唯一入口是 `TdxData` 且必须 `async with`;标的是字符串(如 `sh600000`)
|
||||
而不是 `(market, code)` 元组;默认返回 `list[dataclass]`(要 DataFrame 得用 `TdxData(df="polars")` 显式声明后端)
|
||||
- **枚举取值表** —— `Market` / `KlinePeriod` / `Adjust` / `ExMarket` 的准确取值
|
||||
- **可运行模板** —— K 线、复权、报价、逐笔、分时、批量,全部是能直接跑的代码
|
||||
- **「想做什么 → 用哪个方法」决策表** —— 21 个常见需求的直接映射
|
||||
- **9 条真实陷阱** —— 每条都注明后果和正确做法(如 `SH 000001` 是上证指数、`SZ 000001` 才是平安银行)
|
||||
- **「想做什么 → 用哪个方法」决策表** —— 18 个常见需求的直接映射
|
||||
- **11 条真实陷阱** —— 每条都注明后果和正确做法(如 `sh000001` 是上证指数、`sz000001` 才是平安银行)
|
||||
|
||||
## 为什么值得让 AI 读
|
||||
|
||||
@@ -38,10 +38,11 @@ curl -o ~/.workbuddy/skills/pytdxdata/SKILL.md \
|
||||
这些都不是「读一眼签名就能猜到」的信息。不加载技能,AI 通常会写出:
|
||||
|
||||
```python
|
||||
# ❌ 三种典型错误
|
||||
await td.get_kline("sh", "600000", 0, count=240) # market 传字符串;0 是 5 分钟线不是日线
|
||||
td.get_kline(1, "600000", KlinePeriod.DAY) # 忘了 await,且没在 async with 内
|
||||
df = bars.to_dataframe() # 返回值是 list[dataclass],没有这个方法
|
||||
# ❌ 几种典型错误
|
||||
await td.get_bars("sh600000", 4, count=240) # 4 是数字字面量;应用 KlinePeriod.DAY
|
||||
await td.get_bars((Market.SH, "600000")) # 统一接口只接受字符串标的,不接受元组
|
||||
td.get_bars("sh600000") # 忘了 await,且没在 async with 内
|
||||
df = bars.to_dataframe() # 返回值是 list[dataclass],没有这个方法
|
||||
```
|
||||
|
||||
装载技能后这类错误基本消失。
|
||||
|
||||
+1
-1
@@ -26,7 +26,7 @@ blocks = parse_block_dat(open("block_gn.dat", "rb").read(), "block_gn.dat")
|
||||
如果不想自己管下载,直接用封装好的方法(内部就是调这些解析器):
|
||||
|
||||
`get_block_parsed()` / `get_industry_map()` / `get_financial_file_infos()` / `get_financial_records_parsed()`
|
||||
— 见 [接口总览 → 除权 / 财务 / F10 / 文件](index.md#file-methods)。
|
||||
— 见 [完整接口清单 → 除权 / 财务 / F10 / 文件](../API_REFERENCE.md#file-methods)。
|
||||
|
||||
## 关键细节
|
||||
|
||||
|
||||
+11
-7
@@ -18,8 +18,8 @@ from pytdxdata.models import Adjust, ExMarket, KlinePeriod, Market
|
||||
| `Market.BJ` | `2` | 北京(北交所) |
|
||||
|
||||
!!! warning "代码不唯一"
|
||||
**市场 + 代码**才唯一。`(SH, "000001")` 是上证指数,`(SZ, "000001")` 是平安银行。
|
||||
所有接口的第一个参数都是 `market`,就是为了消歧。
|
||||
**市场 + 代码**才唯一。`sh000001` 是上证指数,`sz000001` 是平安银行。
|
||||
统一接口用**字符串标的**把两者一起表达(前缀即市场),避免只写 6 位数字时的歧义。
|
||||
|
||||
## KlinePeriod — K 线周期
|
||||
|
||||
@@ -85,14 +85,18 @@ KlinePeriod.DAY.is_minute # False
|
||||
| `ExMarket.US_STOCK` | `74` | **美股(最常用)** |
|
||||
|
||||
```python
|
||||
from pytdxdata.models import ExMarket, KlinePeriod
|
||||
from pytdxdata.models import KlinePeriod
|
||||
|
||||
await td.get_kline(ExMarket.HK_MAIN_BOARD, "00700", KlinePeriod.DAY, count=700) # 腾讯
|
||||
await td.get_kline(ExMarket.US_STOCK, "AAPL", KlinePeriod.DAY, count=700) # 苹果
|
||||
await td.get_kline(ExMarket.CFFEX_FUTURES, "IFL0", KlinePeriod.DAY, count=700) # 股指期货
|
||||
await td.get_kline(ExMarket.SH_STOCK_OPTION, "10010971", KlinePeriod.DAY, count=700)
|
||||
# 统一接口用字符串前缀指定扩展市场(推荐):
|
||||
await td.get_bars("hk00700", KlinePeriod.DAY, count=700) # 腾讯
|
||||
await td.get_bars("usAAPL", KlinePeriod.DAY, count=700) # 苹果
|
||||
await td.get_bars("cffex:IFL0", KlinePeriod.DAY, count=700) # 股指期货
|
||||
await td.get_bars("sh_stock_option:10010971", KlinePeriod.DAY, count=700) # 沪股票期权
|
||||
```
|
||||
|
||||
> 上表的 `ExMarket` 枚举值用于查阅与内部协议;统一接口不再直接传枚举,改用字符串前缀
|
||||
> (短前缀 `hk` / `us` / `cffex` …,无短别名的成员用枚举名小写 + 冒号,如 `sh_stock_option:10010971`)。
|
||||
|
||||
### 完整成员列表
|
||||
|
||||
以下由 `mkdocstrings` 从源码实时渲染,**以源码为准**:
|
||||
|
||||
+54
-69
@@ -1,8 +1,9 @@
|
||||
# 接口总览
|
||||
|
||||
对外唯一入口是 **`TdxData`** 类,共 **51 个数据方法**
|
||||
(另有 `start()` / `close()` 两个生命周期方法,以及 `__aenter__` / `__aexit__`)。
|
||||
所有方法都可以直接 `await`,返回的是标准库 `dataclass` 组成的 `list`,没有自定义容器类型。
|
||||
对外唯一入口是 **`TdxData`** 类,所有方法都可直接 `await`,返回标准库 `dataclass` 组成的 `list`。
|
||||
|
||||
**v0.6+** 推荐使用**统一接口**(一个数据类型 = 一个方法,跨市场自动选路);
|
||||
旧的按通道/市场分裂的接口仍可调用,但会发出 `DeprecationWarning`,**1.0 移除**。
|
||||
|
||||
```python
|
||||
from pytdxdata import TdxData
|
||||
@@ -21,100 +22,84 @@ from pytdxdata import TdxData
|
||||
| `async close()` | 关闭全部连接池与缓存 |
|
||||
| `async with TdxData() as td:` | 上下文管理器,自动 start / close |
|
||||
|
||||
## K 线(6)
|
||||
---
|
||||
|
||||
## 统一接口(v0.6+ 推荐){#unified}
|
||||
|
||||
标的一律用**字符串写法**(`sz000001` / `hk00700` / `usAAPL` / `cffex:IFL0`,
|
||||
裸 6 位数字 = 沪市指数)。跨市场按通道自动分组并发,返回值统一带 `symbol` 字段。
|
||||
|
||||
### 标的清单
|
||||
|
||||
| 方法 | 返回 | 说明 |
|
||||
|------|------|------|
|
||||
| `get_kline(market, code, period, *, start, count, adjust, start_date, end_date)` | `list[SecurityBar]` | 主接口。`count > 页大小` 自动并发分页;复权走 MAC;支持日期区间(内部二分定位) |
|
||||
| `get_kline_since(market, code, period, since, adjust)` | `list[SecurityBar]` | 增量便捷方法(≥`since`,含当天) |
|
||||
| `get_kline_batch(stocks, period, *, start, count, adjust, concurrency)` | `dict[str, list[SecurityBar]]` | 多股票并发,返回 `{code: bars}` |
|
||||
| `get_index_kline(market, code, period, *, start, count)` | `list[SecurityBar]` | 指数 K 线(多涨跌家数字段已处理) |
|
||||
| `get_stock_kline_with_indicators(market, code, indicators, ...)` | `dict` | K 线 + 技术指标(MACD / KDJ / RSI / BOLL / MA / EMA) |
|
||||
| `get_kline_offset(offset, count)` | `tuple[int, int]` | K 线偏移信息(MAC) |
|
||||
| `get_universe(market=None)` | `list[SecurityInfo]` | 跨市场标的清单(A 股 + 港股 + 美股 + 期货/期权),EX 通道结果一次拉全后内存过滤 |
|
||||
|
||||
## 报价(4)
|
||||
### 报价
|
||||
|
||||
| 方法 | 返回 | 说明 |
|
||||
|------|------|------|
|
||||
| `get_quotes(stocks)` | `list[SecurityQuote]` | 五档报价;`>80` 只自动切批;TTL 5s |
|
||||
| `get_stock_quotes(stocks, bits)` | `list[MemberQuote]` | MAC 自定义字段报价(≤80 只/次) |
|
||||
| `get_stock_quotes_list(category, *, start, count, sort_type)` | `list[MemberQuote]` | 市场分类报价列表(0沪A / 6全A / 8科创 / 14创业) |
|
||||
| `get_quotes(symbols, *, fields=None)` | `list[SecurityQuote]` | 跨市场混合报价(A 股标准通道有五档 bid/ask + 涨跌停价;给了 `fields` 走 MAC/EX 字段位通道有 name) |
|
||||
| `get_snapshot(symbols)` | `list[SymbolSnapshot]` | 个股特征快照(当日/换手/活跃度,A 股) |
|
||||
| `get_auction(symbols)` | `list[AuctionItem]` | 集合竞价(A 股) |
|
||||
| `get_capital_flow(symbols)` | `list[CapitalFlow]` | 个股资金流向(A 股) |
|
||||
| `get_market_stat()` | `MarketStat` | 全市场统计(涨跌家数 / 总额 / 市值 / 涨跌停数) |
|
||||
|
||||
## 逐笔与分时(4)
|
||||
### K 线
|
||||
|
||||
| 方法 | 返回 | 说明 |
|
||||
|------|------|------|
|
||||
| `get_transactions(market, code, *, date, start, count)` | `list[TransactionRecord]` | 逐笔成交(MAC,**含 `trade_count` 成交笔数**),支持历史回溯 |
|
||||
| `get_minute(market, code, *, date)` | `list[MinuteBar]` | 分时(今日 / 历史,MAC `0x122D`) |
|
||||
| `get_minute_batch(stocks, *, date, concurrency)` | `dict[str, list[MinuteBar]]` | 批量分时 |
|
||||
| `get_chart_sampling(market, code)` | `list[float]` | 分时缩略采样(A 股走 MAC `0x254D`) |
|
||||
| `get_bars(symbols, period, *, start, count, adjust, start_date, end_date, concurrency)` | `list[SecurityBar]` | 跨市场 K 线(自动判指数、复权走 MAC、并发分页、日期区间二分定位,支持混合 A 股+港股+期货) |
|
||||
| `get_indicators(symbols, indicators, *, period, count, adjust, params)` | `list[IndicatorSet]` | K 线 + 技术指标(MACD / KDJ / RSI / BOLL / MA / EMA),返回 `bars + indicators` 组合 |
|
||||
|
||||
## 证券列表(3)
|
||||
### 逐笔与分时
|
||||
|
||||
| 方法 | 返回 | 说明 |
|
||||
|------|------|------|
|
||||
| `get_security_count(market)` | `int` | 市场证券数量 |
|
||||
| `get_security_list(market, *, start, count)` | `list[SecurityInfo]` | `count=None` = 全部(先查总数防死循环) |
|
||||
| `get_security_list_all(market)` | `list[SecurityInfo]` | 全市场列表(高并发一次拉完,缓存 1 天) |
|
||||
| `get_ticks(symbols, *, date, start, count, concurrency)` | `list[TransactionRecord]` | 跨市场逐笔成交(A 股 MAC `0x122F` 含 `trade_count`;港股 EX `0x23FC` / `0x2406`;其他扩展市场 MAC) |
|
||||
| `get_minutes(symbols, *, date, sampling, concurrency)` | `list[MinuteBar]` | 跨市场分时(A 股 MAC `0x122D` 支持历史;`sampling=True` 取缩略采样约 240 点) |
|
||||
|
||||
## 除权 / 财务 / F10 / 文件(15) {#file-methods}
|
||||
### 公司信息 / 除权 / 财务 / 文件
|
||||
|
||||
| 方法 | 返回 | 说明 |
|
||||
|------|------|------|
|
||||
| `get_xdxr(market, code)` | `list[XdxrRecord]` | 除权除息历史(标准) |
|
||||
| `get_finance(market, code)` | `FinanceRecord \| None` | 财务快照(标准) |
|
||||
| `get_price_limits(market, code, name, pre_close, listed_days)` | `tuple[float \| None, float \| None]` | 涨跌停价(**本地计算**,无网络) |
|
||||
| `get_company_info_category(market, code)` | `list` | F10 目录 |
|
||||
| `get_company_info_content(market, code, filename, offset, length)` | `str` | F10 内容(分块) |
|
||||
| `get_block_info(filename)` | `bytes` | 板块文件**原始字节** |
|
||||
| `get_report_file(filename)` | `bytes` | 报告 / 财务文件**原始字节** |
|
||||
| `get_financial_file_list()` | `list[str]` | 财务文件列表**原始行** |
|
||||
| `get_financial_records(filename)` | `bytes` | 财务文件内容**原始字节** |
|
||||
| `get_file_meta(filename)` | `FileMeta` | 远程文件元信息(MAC `0x1215`) |
|
||||
| `download_file(filename, filesize)` | `bytes` | 下载完整远程文件(MAC `0x1217`,分块) |
|
||||
| **`get_block_parsed(filename)`** | `list[TdxBlock]` | 板块文件**下载 + 解析**为成分股 |
|
||||
| **`get_industry_map()`** | `dict[str, IndustryInfo]` | `tdxhy.cfg` **下载 + 解析**:通达信 + 申万行业 |
|
||||
| **`get_financial_file_infos()`** | `list[FinancialFileInfo]` | `gpcw.txt` **下载 + 解析**:文件名 / MD5 / 大小 |
|
||||
| **`get_financial_records_parsed(filename)`** | `list[FinancialRecord]` | `gpcw*.zip` **解压 + 解析**:每股原始字段 |
|
||||
| `get_f10(symbol, section=None, *, offset, length)` | `list \| str` | F10 公司资料(`section=None` 返回目录,给了返回正文) |
|
||||
| `get_xdxr(symbols)` | `list[XdxrRecord]` | 除权除息历史(A 股) |
|
||||
| `get_finance(symbols)` | `list[FinanceRecord]` | 财务快照(A 股,返回列表,无数据的标的跳过) |
|
||||
| `get_price_limits(symbol, name, pre_close, listed_days)` | `tuple[float\|None, float\|None]` | 涨跌停价(本地计算,无网络) |
|
||||
| `get_file(name)` | `bytes` | 服务器文件原始字节(`block_*.dat` / `tdxhy.cfg` / `tdxfin/*` / MAC 远程文件) |
|
||||
| `get_block_parsed(filename)` | `list[TdxBlock]` | 板块文件**下载 + 解析**为成分股 |
|
||||
| `get_industry_map()` | `dict[str, IndustryInfo]` | `tdxhy.cfg` **下载 + 解析** |
|
||||
| `get_financial_file_infos()` | `list[FinancialFileInfo]` | `gpcw.txt` **下载 + 解析** |
|
||||
| `get_financial_records_parsed(filename)` | `list[FinancialRecord]` | `gpcw*.zip` **解压 + 解析** |
|
||||
|
||||
!!! note "原始字节通道保留"
|
||||
返回 `bytes` 的那 4 个方法**保持向后兼容**,解析版是**新增**方法而不是替换。
|
||||
解析器在 `pytdxdata.codec`,纯标准库,可以脱网单独复用(见 [文件解析器](codec.md))。
|
||||
|
||||
## 板块(6)
|
||||
### 市场 / 板块
|
||||
|
||||
| 方法 | 返回 | 说明 |
|
||||
|------|------|------|
|
||||
| `get_board_list(board_type, count)` | `list[BoardInfo]` | 板块列表(0全部 / 1行业 / 2概念) |
|
||||
| `get_board_members(board_symbol, count, sort_type)` | `list[MemberQuote]` | 板块成分股报价(如 `881001`) |
|
||||
| `get_belong_board(market, code)` | `list[BelongBoard]` | 个股所属板块 |
|
||||
| `get_board_summary(board_symbol)` | `dict` | 成分数 / 成交额 / 主力净流入 / 涨跌家数 |
|
||||
| `get_board_ranking(board_type, top_n)` | `list[dict]` | 板块涨跌幅排行 |
|
||||
| `get_board_change_ranking(board_type, days, top_n)` | `list[dict]` | 板块 N 日涨跌幅排行 |
|
||||
| `get_unusual(market="all", *, start, count)` | `list[UnusualItem]` | 市场异动(A 股,`"all"` = 深 + 沪) |
|
||||
| `get_server_info()` | `ServerInfo` | 服务器交易时段信息 |
|
||||
| `get_kline_offset(offset, count)` | `tuple[int, int]` | K 线偏移信息(MAC) |
|
||||
| `get_boards(kind="all", *, count)` | `list[BoardInfo]` | 板块列表(`"all"` / `"industry"` / `"concept"` / `"style"`) |
|
||||
| `get_board_members(board, *, fields, count)` | `list[SecurityQuote]` | 板块成分股报价(返回归一报价模型) |
|
||||
| `get_board_of(symbols)` | `list[BelongBoard]` | 个股所属板块(A 股) |
|
||||
| `get_board_summary(board)` | `dict` | 板块汇总(成分数 / 成交额 / 主力净流入 / 涨跌家数) |
|
||||
| `get_board_ranking(kind="industry", *, top_n)` | `list[dict]` | 板块当日涨跌幅排行 |
|
||||
| `get_board_change_ranking(kind="industry", *, days, top_n)` | `list[dict]` | 板块 N 日涨跌幅排行 |
|
||||
|
||||
## 竞价 / 异动 / 快照 / 资金流 / 服务器(5)
|
||||
---
|
||||
|
||||
| 方法 | 返回 | 说明 |
|
||||
|------|------|------|
|
||||
| `get_auction(market, code)` | `list[AuctionItem]` | 集合竞价 |
|
||||
| `get_unusual(market, start, count)` | `list[UnusualItem]` | 市场异动 |
|
||||
| `get_symbol_info(market, code)` | `SymbolSnapshot` | 个股特征快照 |
|
||||
| `get_capital_flow(market, code)` | `CapitalFlow` | 个股资金流向(主力 / 散户 + 5 档) |
|
||||
| `get_server_info()` | `ServerInfo` | 服务器交易时段 |
|
||||
## 旧接口(deprecated,1.0 移除){#legacy}
|
||||
|
||||
## 扩展市场 EX:港股 / 美股 / 期货 / 期权(8)
|
||||
旧接口按通道/市场分裂(33 个 deprecated 别名),保留向后兼容但仍可用。
|
||||
完整旧方法列表见 `docs/API_REFERENCE.md`。
|
||||
|
||||
| 方法 | 返回 | 说明 |
|
||||
|------|------|------|
|
||||
| `get_goods_list(market, start, count)` | `list[dict]` | 扩展市场商品列表(`0x23F5`,二分定位市场) |
|
||||
| `get_goods_count(market)` | `int` | 商品数量(`None` = 全市场) |
|
||||
| `get_goods_quotes(stocks, bits)` | `list[MemberQuote]` | 扩展市场报价(`0x122B`) |
|
||||
| `get_goods_quotes_list(market, count)` | `list[MemberQuote]` | 商品列表 + 报价组合 |
|
||||
| `get_goods_chart_sampling(market, code)` | `list[float]` | 扩展市场分时采样(`0x254D`) |
|
||||
| `get_goods_tick_chart(market, code, ymd)` | `list[dict]` | 扩展市场分时 tick |
|
||||
| `get_goods_transaction(market, code, *, ymd, start, count)` | `list[dict]` | 扩展市场逐笔(港股 `0x23FC` / `0x2406`) |
|
||||
| `get_goods_transaction_all(market, code, ymd)` | `list[dict]` | 港股全量逐笔(自动翻页,≤50 页 / 9 万条) |
|
||||
!!! warning "迁移建议"
|
||||
各通道独立方法(如 `get_kline` / `get_transactions`)在 v0.6 内仍可调用,且仍接受
|
||||
`(market, code)` 写法,但会发出 `DeprecationWarning`。
|
||||
统一接口(`get_quotes` / `get_bars` / `get_ticks` / `get_xdxr` / `get_finance` /
|
||||
`get_auction` / `get_capital_flow` 等)一律只接受字符串标的(如 `"sz000001"`)。
|
||||
迁移对照见 [更新日志](../changelog.md)。
|
||||
|
||||
## 下一步
|
||||
|
||||
|
||||
+37
-13
@@ -2,8 +2,16 @@
|
||||
|
||||
全部数据模型位于 `pytdxdata.models`,统一为 `@dataclass(slots=True)`。
|
||||
|
||||
选择 dataclass 而不是 pandas DataFrame 是刻意的:**零依赖 + 零转换开销**,
|
||||
需要 DataFrame 时你自己转(一行的事,见 [场景示例 → 转成 pandas](../cookbook.md#to-pandas))。
|
||||
默认(`df=None`)所有接口返回这些 dataclass —— **零依赖 + 零转换开销**。
|
||||
不想要 dataclass 就直接开 DataFrame 出口,不用自己 `asdict` 再建表:
|
||||
|
||||
```python
|
||||
async with TdxData(df="polars") as td: # 或 df="pandas"
|
||||
df = await td.get_bars("sh600000", KlinePeriod.DAY, count=240)
|
||||
```
|
||||
|
||||
列口径(时间列合并、五档展开、`fields` JSON 化)见 [DataFrame 输出](../dataframe.md);
|
||||
两者是同一份数据,只是容器不同,**量纲与语义完全一致**。
|
||||
|
||||
## 导入方式
|
||||
|
||||
@@ -28,25 +36,25 @@ from pytdxdata.models import BoardInfo, MemberQuote, AuctionItem, CapitalFlow
|
||||
|
||||
| 模型 | 代表什么 | 从哪来 |
|
||||
|---|---|---|
|
||||
| `SecurityBar` | 一根 K 线 | `get_kline` / `get_kline_batch` |
|
||||
| `IndexBar` | 指数 K 线(继承 `SecurityBar`) | `get_index_kline` |
|
||||
| `SecurityQuote` | 五档报价快照 | `get_quotes` |
|
||||
| `SecurityInfo` | 证券列表条目(代码 / 名称 / 前收) | `get_security_list` |
|
||||
| `TransactionRecord` | 一笔成交 | `get_transactions` |
|
||||
| `MinuteBar` | 一分钟分时点 | `get_minute` |
|
||||
| `SecurityBar` | 一根 K 线 | `get_bars` / `get_indicators` |
|
||||
| `IndexBar` | 指数 K 线(继承 `SecurityBar`) | `get_bars` |
|
||||
| `SecurityQuote` | 五档报价快照 | `get_quotes` / `get_board_members` |
|
||||
| `SecurityInfo` | 证券列表条目(代码 / 名称 / 前收) | `get_universe` |
|
||||
| `TransactionRecord` | 一笔成交 | `get_ticks` |
|
||||
| `MinuteBar` | 一分钟分时点 | `get_minutes` |
|
||||
| `XdxrRecord` | 一次除权除息事件 | `get_xdxr` |
|
||||
| `FinanceRecord` | 财务**快照**(最新一期,36 字段) | `get_finance` |
|
||||
| `FinancialRecord` | 历史专业财报(`gpcw*.zip`,多报告期) | `get_financial_records_parsed` |
|
||||
| `FinancialFileInfo` | 财报文件索引条目 | `get_financial_file_infos` |
|
||||
| `TdxBlock` | 一个板块及其成分股 | `get_block_parsed` |
|
||||
| `IndustryInfo` | 一只股票的行业归属 | `get_industry_map` |
|
||||
| `BoardInfo` | 板块列表条目(含领涨股) | `get_board_list` |
|
||||
| `MemberQuote` | 成分股报价(自定义字段) | `get_board_members` / `get_stock_quotes` |
|
||||
| `BelongBoard` | 个股所属板块 | `get_belong_board` |
|
||||
| `BoardInfo` | 板块列表条目(含领涨股) | `get_boards` |
|
||||
| `MemberQuote` | 成分股报价(MAC 原始字段) | 旧 `get_stock_quotes` / `get_stock_quotes_list` |
|
||||
| `BelongBoard` | 个股所属板块 | `get_board_of` |
|
||||
| `AuctionItem` | 集合竞价撮合点 | `get_auction` |
|
||||
| `UnusualItem` | 一笔市场异动 | `get_unusual` |
|
||||
| `SymbolSnapshot` | 个股特征快照 | `get_symbol_info` |
|
||||
| `GoodsItem` | 扩展市场商品 | `get_goods_list` |
|
||||
| `SymbolSnapshot` | 个股特征快照 | `get_snapshot` |
|
||||
| `GoodsItem` | 扩展市场商品 | 旧 `get_goods_list` |
|
||||
| `ServerInfo` | 服务器交易时段 | `get_server_info` |
|
||||
| `CapitalFlow` | 个股资金流向 | `get_capital_flow` |
|
||||
| `MarketStat` | 全市场统计 | `get_market_stat` |
|
||||
@@ -62,6 +70,22 @@ from pytdxdata.models import BoardInfo, MemberQuote, AuctionItem, CapitalFlow
|
||||
| 结构 | 具名属性(`zongguben`、`jingzichan` ...) | `fields: list[float]`,原始字段数组 |
|
||||
| 单位 | 万元 / 万股 | 原始 float,需自行按字段语义换算 |
|
||||
|
||||
!!! tip "`SecurityBar.datetime` 的时间戳口径(v0.6.2 起)"
|
||||
时间统一为**北京时间**。A股 / 港股 / 期货的原始时间戳即为北京时间;美股的原始报文
|
||||
混用了「美东交易日日期」与「北京时间时分」两个时区,库在构造时已自动归一化
|
||||
(凌晨段日期 +1 天),因此 `bar.datetime` 可直接排序 / 比较 / 过滤。
|
||||
|
||||
但要注意**美股一个交易日横跨两个北京自然日**,按交易日聚合请用 `bar.trading_date`:
|
||||
|
||||
```python
|
||||
bars = await td.get_bars("usAAPL", KlinePeriod.MIN_1, count=390)
|
||||
bars[0].datetime # 2026-10-12 21:31:00(北京时间)
|
||||
bars[-1].datetime # 2026-10-13 04:00:00(次日)
|
||||
bars[0].trading_date # "2026-10-12" —— 归属的美东交易日
|
||||
```
|
||||
|
||||
`bar.is_us` 可判定是否美股。详见 [常见问题 → 美股分钟 K 线](../faq.md)。
|
||||
|
||||
## 完整字段定义
|
||||
|
||||
以下由 `mkdocstrings` 从源码实时渲染(**以源码为准**):
|
||||
|
||||
+2
-2
@@ -8,11 +8,11 @@
|
||||
```python
|
||||
import asyncio
|
||||
from pytdxdata import TdxData
|
||||
from pytdxdata.models import KlinePeriod, Market
|
||||
from pytdxdata.models import KlinePeriod
|
||||
|
||||
async def main():
|
||||
async with TdxData() as td:
|
||||
bars = await td.get_kline(Market.SH, "600000", KlinePeriod.DAY, count=240)
|
||||
bars = await td.get_bars("sh600000", KlinePeriod.DAY, count=240)
|
||||
print(bars[-1])
|
||||
|
||||
asyncio.run(main())
|
||||
|
||||
+12
-1
@@ -56,6 +56,17 @@ tdx minute sz000001 # 今日分时
|
||||
tdx minute sz000001 --date 20260811 # 历史分时
|
||||
```
|
||||
|
||||
价格列的小数位**按品种自动适配**,不用自己换算:
|
||||
|
||||
| 品种 | 显示小数位 | 示例 |
|
||||
|---|---|---|
|
||||
| 股票 / 指数 | 2 | `12.34` |
|
||||
| ETF / LOF / 沪市 B 股 / 可转债 | 3 | `128.892`(可转债报价步长 0.001 元) |
|
||||
| 债券 / 回购 | 4 | `100.0100` |
|
||||
|
||||
> 可转债的**解码位数**是 4 位(原始整数 / 10⁴),但第 4 位恒为 0,所以显示按 3 位。
|
||||
> `tdx minute` 的时间列是该 bar 的**真实时钟**(`09:31` / `14:57`),不是序号推算。
|
||||
|
||||
## 板块
|
||||
|
||||
```bash
|
||||
@@ -97,7 +108,7 @@ tdx industry --limit 30 # 只看前 30 条
|
||||
|
||||
# 历史专业财报
|
||||
tdx financial-list # 索引:文件名 / MD5 / 大小(gpcw.txt)
|
||||
tdx financial gpcw20260331.zip --limit 5 # 下载 + 解压 + 解析
|
||||
tdx financial gpcw20260331.zip --limit 5 # 下载 + 解析(服务器实际给 .dat 时自动回退)
|
||||
tdx financial gpcw20260331.zip --code 600000 --fields 8 # 只取指定股票、打印更多字段
|
||||
```
|
||||
|
||||
|
||||
@@ -94,8 +94,8 @@ git push origin feat/xxx
|
||||
3. 提交并 push 到 `main`
|
||||
4. 打标签并推送:
|
||||
```bash
|
||||
git tag v0.5.0
|
||||
git push origin v0.5.0
|
||||
git tag v0.6.0
|
||||
git push origin v0.6.0
|
||||
```
|
||||
|
||||
推 tag 会自动触发 GitHub Actions:**test → build(含版本号一致性校验)→ OIDC 免密钥发布到 PyPI**。
|
||||
@@ -105,12 +105,12 @@ git push origin feat/xxx
|
||||
同名 tag **不能直接重推**(push 无变化,workflow 不会触发)。先删再推:
|
||||
|
||||
```bash
|
||||
git push --delete origin v0.5.0
|
||||
git tag -d v0.5.0 && git tag v0.5.0
|
||||
git push origin v0.5.0
|
||||
git push --delete origin v0.6.0
|
||||
git tag -d v0.6.0 && git tag v0.6.0
|
||||
git push origin v0.6.0
|
||||
```
|
||||
|
||||
或直接手动触发:`gh workflow run release.yml --ref v0.5.0`
|
||||
或直接手动触发:`gh workflow run release.yml --ref v0.6.0`
|
||||
|
||||
文档站由 `.github/workflows/docs.yml` 在 push `main` 时自动部署到
|
||||
<https://openbot-coder.github.io/pytdxdata/>。
|
||||
+57
-36
@@ -6,33 +6,32 @@
|
||||
import asyncio
|
||||
from pathlib import Path
|
||||
from pytdxdata import TdxData
|
||||
from pytdxdata.models import Adjust, ExMarket, KlinePeriod, Market
|
||||
from pytdxdata.models import Adjust, KlinePeriod
|
||||
```
|
||||
|
||||
## 全市场日线批量落库
|
||||
|
||||
连接池会把不同股票分配到不同服务器,这是 `*_batch` 存在的意义。
|
||||
传多标的时,连接池会把不同股票分配到不同服务器——这正是它比逐个 `await` 快的原因。
|
||||
|
||||
```python
|
||||
async def dump_all_daily():
|
||||
async with TdxData(cache_dir=Path("./tdx_cache")) as td:
|
||||
codes = await td.get_security_list_all(Market.SZ) # 缓存 1 天
|
||||
symbols = [(Market.SZ, s.code) for s in codes]
|
||||
infos = await td.get_universe("sz") # 缓存 1 天
|
||||
symbols = [i.symbol for i in infos] # ["sz000001", ...]
|
||||
|
||||
bars = await td.get_kline_batch(
|
||||
bars = await td.get_bars(
|
||||
symbols,
|
||||
KlinePeriod.DAY,
|
||||
count=800,
|
||||
adjust=Adjust.QFQ,
|
||||
concurrency=12,
|
||||
)
|
||||
print(f"{len(bars)} / {len(symbols)} 只拿到数据")
|
||||
got = {b.symbol for b in bars}
|
||||
print(f"{len(got)} / {len(symbols)} 只拿到数据")
|
||||
|
||||
for code, rows in bars.items():
|
||||
if not rows:
|
||||
continue
|
||||
# 落库 / 写 parquet 都行,这里只演示
|
||||
print(code, len(rows), rows[-1].datetime, rows[-1].close)
|
||||
# bars 是扁平 list[SecurityBar](按标的/时间排序),落库 / 写 parquet 都行
|
||||
if bars:
|
||||
print(bars[-1].symbol, bars[-1].datetime, bars[-1].close)
|
||||
```
|
||||
|
||||
!!! tip "分页与并发的取舍"
|
||||
@@ -41,12 +40,12 @@ async def dump_all_daily():
|
||||
|
||||
## 分钟线增量更新
|
||||
|
||||
只拉「上次之后」的部分,用 `get_kline_since` 省掉全量重拉。
|
||||
只拉「上次之后」的部分,用 `get_bars(start_date=...)` 省掉全量重拉。
|
||||
|
||||
```python
|
||||
async def update_minute(code: str, last_dt):
|
||||
async def update_minute(symbol: str, last_dt):
|
||||
async with TdxData(cache_dir=Path("./tdx_cache")) as td:
|
||||
rows = await td.get_kline_since(Market.SZ, code, KlinePeriod.MIN_1, since=last_dt)
|
||||
rows = await td.get_bars(symbol, KlinePeriod.MIN_1, start_date=last_dt)
|
||||
return [r for r in rows if r.datetime > last_dt]
|
||||
```
|
||||
|
||||
@@ -75,7 +74,7 @@ async def poll_quotes(symbols, interval=5.0):
|
||||
async def sector_rotation():
|
||||
async with TdxData() as td:
|
||||
# 近 5 日概念板块涨幅排行
|
||||
top = await td.get_board_change_ranking(board_type=2, days=5, top_n=10)
|
||||
top = await td.get_board_change_ranking(kind="concept", days=5, top_n=10)
|
||||
for row in top:
|
||||
print(row)
|
||||
|
||||
@@ -92,18 +91,18 @@ async def sector_rotation():
|
||||
## 个股所属板块(反向查询)
|
||||
|
||||
```python
|
||||
async def belong(code: str):
|
||||
async def belong(symbol: str):
|
||||
async with TdxData() as td:
|
||||
for b in await td.get_belong_board(Market.SH, code):
|
||||
for b in await td.get_board_of(symbol):
|
||||
print(b)
|
||||
```
|
||||
|
||||
## 逐笔复盘
|
||||
|
||||
```python
|
||||
async def review_transactions(code: str, ymd: int):
|
||||
async def review_transactions(symbol: str, ymd: int):
|
||||
async with TdxData() as td:
|
||||
rows = await td.get_transactions(Market.SZ, code, date=ymd, count=2000)
|
||||
rows = await td.get_ticks(symbol, date=ymd, count=2000)
|
||||
total = sum(r.trade_count or 0 for r in rows)
|
||||
buy = sum(r.vol for r in rows if r.bs_flag == 0) # 主动买
|
||||
sell = sum(r.vol for r in rows if r.bs_flag == 1) # 主动卖
|
||||
@@ -113,15 +112,26 @@ async def review_transactions(code: str, ymd: int):
|
||||
## 财务因子快照
|
||||
|
||||
```python
|
||||
async def finance_snapshot(codes):
|
||||
async def finance_snapshot(symbols):
|
||||
async with TdxData() as td:
|
||||
for mkt, code in codes:
|
||||
fin = await td.get_finance(mkt, code) # 标准通道,最新快照
|
||||
if fin:
|
||||
print(code, fin)
|
||||
for fin in await td.get_finance(symbols): # 标准通道,最新快照
|
||||
print(fin.symbol, fin)
|
||||
```
|
||||
|
||||
历史专业财报(`gpcw*.zip`,含多报告期)走文件接口:
|
||||
历史专业财报(`gpcw*`,含多报告期)走文件接口。
|
||||
|
||||
!!! warning "索引里的文件名与服务器实际不一致"
|
||||
`tdxfin/gpcw.txt` 索引里写的是 `gpcwYYYYMMDD.zip`,但服务器实际只提供**未压缩的**
|
||||
`tdxfin/gpcwYYYYMMDD.dat` —— 这些差异已由 `get_financial_records_parsed()` 内部处理,
|
||||
直接把索引里的 `filename` 传进去即可。
|
||||
|
||||
另外索引里**含有尚未生成的占位条目**(`filesize` 只有百来字节),取它们会得到空列表。
|
||||
想拿最新一期可用数据,先按体积过滤:
|
||||
|
||||
```python
|
||||
usable = [fi for fi in infos if fi.filesize > 500_000] # 排除占位条目
|
||||
records = await td.get_financial_records_parsed(usable[0].filename)
|
||||
```
|
||||
|
||||
```python
|
||||
async def history_financials(code: str):
|
||||
@@ -173,27 +183,38 @@ async def snapshot_blocks():
|
||||
```python
|
||||
async def ex_markets():
|
||||
async with TdxData() as td:
|
||||
hk = await td.get_kline(ExMarket.HK_MAIN_BOARD, "00700", KlinePeriod.DAY, count=700)
|
||||
us = await td.get_kline(ExMarket.US_STOCK, "AAPL", KlinePeriod.DAY, count=700)
|
||||
fut = await td.get_kline(ExMarket.CFFEX_FUTURES, "IFL0", KlinePeriod.DAY, count=700)
|
||||
opt = await td.get_kline(ExMarket.SH_STOCK_OPTION, "10010971", KlinePeriod.DAY, count=700)
|
||||
hk = await td.get_bars("hk00700", KlinePeriod.DAY, count=700)
|
||||
us = await td.get_bars("usAAPL", KlinePeriod.DAY, count=700)
|
||||
fut = await td.get_bars("cffex:IFL0", KlinePeriod.DAY, count=700)
|
||||
opt = await td.get_bars("sh_stock_option:10010971", KlinePeriod.DAY, count=700)
|
||||
print(len(hk), len(us), len(fut), len(opt))
|
||||
|
||||
# 港股全量逐笔(自动翻页,≤50 页 / 9 万条)
|
||||
ticks = await td.get_goods_transaction_all(ExMarket.HK_MAIN_BOARD, "00700")
|
||||
ticks = await td.get_ticks("hk00700", count=None)
|
||||
print(len(ticks))
|
||||
```
|
||||
|
||||
## 转成 pandas {#to-pandas}
|
||||
|
||||
包本身不带 pandas 依赖,但转换是一行的事:
|
||||
**0.7.0 起不用自己转了** —— 构造时声明一次后端,表格型接口直接返回 DataFrame:
|
||||
|
||||
```python
|
||||
async with TdxData(df="pandas") as td: # 或 df="polars"
|
||||
df = await td.get_bars("sh600000", KlinePeriod.DAY, count=800)
|
||||
df = df.set_index("datetime") # 时间列已经合并好了
|
||||
```
|
||||
|
||||
需要 `pip install "pytdxdata[pandas]"`(或 `[polars]` / `[dataframe]`),细节见
|
||||
[DataFrame 输出](dataframe.md)。
|
||||
|
||||
如果只想对**某一处**手动转、不想受影响全局,仍可用 dataclass 自己拼:
|
||||
|
||||
```python
|
||||
import pandas as pd
|
||||
|
||||
async def to_dataframe(code: str):
|
||||
async def to_dataframe(symbol: str):
|
||||
async with TdxData() as td:
|
||||
bars = await td.get_kline(Market.SH, code, KlinePeriod.DAY, count=800)
|
||||
bars = await td.get_bars(symbol, KlinePeriod.DAY, count=800)
|
||||
df = pd.DataFrame([{
|
||||
"dt": b.datetime, "open": b.open, "high": b.high,
|
||||
"low": b.low, "close": b.close, "vol": b.vol, "amount": b.amount,
|
||||
@@ -219,10 +240,10 @@ async def lifespan(app: FastAPI):
|
||||
|
||||
app = FastAPI(lifespan=lifespan)
|
||||
|
||||
@app.get("/kline/{code}")
|
||||
async def kline(code: str, request: Request):
|
||||
@app.get("/kline/{symbol}")
|
||||
async def kline(symbol: str, request: Request):
|
||||
td = request.app.state.td
|
||||
bars = await td.get_kline(Market.SH, code, KlinePeriod.DAY, count=30)
|
||||
bars = await td.get_bars(symbol, KlinePeriod.DAY, count=30)
|
||||
return [{"dt": str(b.datetime), "close": b.close} for b in bars]
|
||||
```
|
||||
|
||||
|
||||
@@ -0,0 +1,122 @@
|
||||
# DataFrame 输出(pandas / polars)
|
||||
|
||||
0.7.0 起,`TdxData` 可以**直接把表格型接口的返回值变成 DataFrame**,不用再自己 `[asdict(x) for x in rows]`。
|
||||
|
||||
设计上只有一条开关,就是构造参数 `df`:
|
||||
|
||||
```python
|
||||
from pytdxdata import TdxData, KlinePeriod
|
||||
|
||||
# 默认:df=None,行为和 0.6.x 完全一致,返回 list[dataclass]
|
||||
async with TdxData() as td:
|
||||
bars = await td.get_bars("sh600000", KlinePeriod.DAY, count=240) # list[SecurityBar]
|
||||
|
||||
# 要 DataFrame 就在构造时声明一次后端
|
||||
async with TdxData(df="polars") as td:
|
||||
df = await td.get_bars("sh600000", KlinePeriod.DAY, count=240) # polars.DataFrame
|
||||
|
||||
async with TdxData(df="pandas") as td:
|
||||
df = await td.get_bars("sh600000", KlinePeriod.DAY, count=240) # pandas.DataFrame
|
||||
```
|
||||
|
||||
## 安装
|
||||
|
||||
pandas / polars 都是**可选依赖**——`pytdxdata` 的 `dependencies` 仍然是空的,不想用 DataFrame 的人一点额外开销都不用付。
|
||||
|
||||
```bash
|
||||
pip install "pytdxdata[polars]" # 只要 polars
|
||||
pip install "pytdxdata[pandas]" # 只要 pandas
|
||||
pip install "pytdxdata[dataframe]" # 两个都要
|
||||
```
|
||||
|
||||
没装就 `TdxData(df="polars")`,会在**构造时**直接报错,而不是等到第一次查询:
|
||||
|
||||
```
|
||||
FrameBackendError: 未安装 polars,无法输出 DataFrame。请执行:pip install pytdxdata[polars]
|
||||
```
|
||||
|
||||
`df` 取值非法(比如 `df="numpy"`)抛 `ValueError`,并列出可用值。
|
||||
|
||||
## 三个口径约定
|
||||
|
||||
### 1. 时间列:`datetime` 替换拆开的年月日时分
|
||||
|
||||
原始模型为了解析性能把时间拆成 `year/month/day/hour/minute`。DataFrame 里再这么摆,用的人第一件事就是自己拼时间戳——所以直接给一列 `datetime`:
|
||||
|
||||
| 模型 | 原始字段 | DataFrame 里的列 |
|
||||
|---|---|---|
|
||||
| `SecurityBar` / `IndexBar` | `year month day hour minute` | `datetime`(+ `trading_date`) |
|
||||
| `MinuteBar` | `date hour minute day_offset` | `datetime`(+ `trading_date`) |
|
||||
| `XdxrRecord` | `year month day` | `date` |
|
||||
|
||||
- `datetime` 的 dtype:polars 是 `Datetime('us')`,pandas 是 `datetime64[ns]`。
|
||||
- `trading_date` 是 `YYYY-MM-DD` 字符串,**按交易日聚合要用它**:美股一个交易日横跨两个北京自然日(21:30→次日 04:00),凌晨段的 `datetime` 属于次日,但 `trading_date` 归属前一交易日。
|
||||
- 除权记录的 `date` 是「日」不是「时刻」,值为当日 00:00,仍用 datetime 类型以便排序比较。
|
||||
- `TransactionRecord.time` 只有 `HH:MM:SS`、**协议里没有日期**,所以逐笔不生成 `datetime`,`time` 原样保留。
|
||||
|
||||
### 2. 嵌套字段:能定型的定型,不能定型的 JSON 化
|
||||
|
||||
| 字段 | DataFrame 里的形态 |
|
||||
|---|---|
|
||||
| `SecurityQuote.bid` / `ask`(五档) | 展开成 `bid1_price` `bid1_vol` … `ask5_price` `ask5_vol` 共 **10 列**,缺档为 `None` |
|
||||
| `fields`(A股字段位 / EX 额外字段) | **JSON 字符串**列;空字典给 `None`(不会看到一列 `"{}"`) |
|
||||
| `TdxBlock.codes`(成分股列表) | 保持列表(polars 为 `List(Str)`,pandas 为 object),需要时 `.explode()` |
|
||||
|
||||
`fields` 之所以不展开:键集随市场和通道变化,展开会让列数失控。
|
||||
|
||||
### 3. 特殊容器会被展开成一张表
|
||||
|
||||
| 接口 | 原始返回 | DataFrame 形态 |
|
||||
|---|---|---|
|
||||
| `get_indicators` | `list[IndicatorSet]` | bars 列 + 每个指标一列,按长度对齐(不足补 `None`),`symbol` 取自 `IndicatorSet.symbol` |
|
||||
| `get_index_trade_stats` | `IndexTradeStats` | `minutes` 展开成行,并补 `index` / `date` 两个常量列 |
|
||||
| `get_index_trade_stats_multi` | `list[IndexTradeStats]` | 同上,多个指数纵向拼接 |
|
||||
| `get_kline_batch` / `get_minute_batch` | `dict[标的, list[...]]` | 各标的分块纵向拼接成一张表(`symbol`/`code` 列本来就在行里) |
|
||||
| `get_board_ranking` / `get_board_change_ranking` | `list[dict]` | 按首次出现顺序取键并集作为列名 |
|
||||
|
||||
## 哪些接口不受影响
|
||||
|
||||
`df` 只作用于**表格型返回**。下面这些原样返回,不转 DataFrame:
|
||||
|
||||
| 返回类型 | 接口 |
|
||||
|---|---|
|
||||
| `bytes` | `get_file` `get_block_info` `get_report_file` `download_file` `get_financial_records` |
|
||||
| `int` | `get_security_count` `get_goods_count` |
|
||||
| `dict` | `decimal_point_status` `get_board_summary` `get_industry_map` |
|
||||
| 单个对象 | `get_server_info` `get_market_stat` `get_symbol_info` |
|
||||
| `tuple` | `get_price_limits` `get_kline_offset` |
|
||||
| `list[float]` / `list[str]` | `get_chart_sampling` `get_goods_chart_sampling` `get_financial_file_list` |
|
||||
|
||||
理由是单行表没有意义,而 `bytes`/`int` 根本不是表格数据。
|
||||
|
||||
## 空结果
|
||||
|
||||
查不到数据时返回的是**列名齐全的空 DataFrame**(列由模型定义推出,不依赖是否拿到行):
|
||||
|
||||
```python
|
||||
async with TdxData(df="polars") as td:
|
||||
df = await td.get_xdxr("sz000001") # 假设当天无除权
|
||||
df.height, df.columns # 0, ['market', 'code', 'date', ...]
|
||||
```
|
||||
|
||||
⚠️ 空表只保证**列名**正确,dtype 由后端自行推断(polars 可能全为 `Null`,pandas 为 object)。
|
||||
|
||||
## 语义提醒
|
||||
|
||||
DataFrame 只是换了个容器,**下面这些口径不会因为转成 DataFrame 而变化**:
|
||||
|
||||
- 股票/ETF 的 `vol` 是「手」,日线是「股」;可转债 `vol` 手=10 张,`amount/vol` 是均价 **×10**;指数逐笔 `vol` 是百元。
|
||||
- 指数和 ETF/LOF 的 `trade_count` 不是笔数(见 `TransactionRecord.trade_count_available`)。
|
||||
- 深市 `399xxx` 的 `amount` 是全板块口径,不要拿去校验成分合计。
|
||||
- K 线的 `settle`(结算价)列只有 **EX 通道的期货/期权日线**有值,其余行全为 `None`;
|
||||
同一批 EX 标的中,期货/期权的 `amount` 恒 `0.0`(`0x23FF` 帧内没有成交额),
|
||||
港股主板/基金的 `vol` 单位是「手」(`amount/vol` 是均价的 100 倍)。
|
||||
|
||||
细节见 [数据模型](api/models.md) 与 [常见问题](faq.md)。
|
||||
|
||||
## 性能
|
||||
|
||||
转换是**列式采集**:按模型预先算好列计划,一次遍历填满所有列,不做逐行 `asdict()`。
|
||||
5,227 只 A 股报价转 polars 约 20~40ms,相对一次全市场查询(约 300ms)可以忽略。
|
||||
|
||||
`df=None` 时**完全不经过这层**(装饰器直接透传),零额外开销。
|
||||
+50
-9
@@ -19,7 +19,8 @@
|
||||
| SH | `000001` | 上证指数 |
|
||||
| SZ | `000001` | 平安银行 |
|
||||
|
||||
代码不唯一,**市场 + 代码**才唯一。这也是为什么所有接口的第一个参数都是 `market`。
|
||||
代码不唯一,**市场 + 代码**才唯一。统一接口用**字符串标的**把两者一起表达
|
||||
(`sh000001` / `sz000001`),避免只写 6 位数字时的歧义。
|
||||
|
||||
??? question "为什么报价格式有时候是 2 位小数,有时候是 3 位?"
|
||||
价格小数位按**品种**推断:股票 2 位,指数 / ETF / 可转债 3 位。
|
||||
@@ -28,17 +29,40 @@
|
||||
## 连接与网络
|
||||
|
||||
??? question "连不上服务器 / 全部超时怎么办?"
|
||||
服务器可用性会随时间变化(下线、换 IP、封禁)。按顺序排查:
|
||||
服务器可用性会随时间变化(下线、换 IP、封禁)。本包**当日首次启动会自动后台测速**并把
|
||||
可达主机按延迟排序缓存,多数情况下无需手工干预。按顺序排查:
|
||||
|
||||
1. 重新探测:`python -m pytdxdata.server_probe`
|
||||
2. 手工指定清单绕过探测结果:
|
||||
```python
|
||||
async with TdxData(standard_servers=[("1.2.3.4", 7709)]) as td: ...
|
||||
1. **删掉当日测速缓存**,让下次启动重新测速——缓存里可能存着今天刚挂掉的节点
|
||||
(路径见 [安装与配置](getting-started.md#当日自动测速无感))。
|
||||
2. 手工指定主机。优先级最高,**直接跳过测速与缓存**:
|
||||
```bash
|
||||
PYPYTDX_DATA_STANDARD_HOSTS=1.2.3.4,5.6.7.8 python your_script.py
|
||||
```
|
||||
3. 检查出口网络策略——标准 / MAC 端口 **7709**、EX 端口 **7727** 需要放通。
|
||||
4. 若在用代理软件,注意有些本地代理会劫持回环 TCP,导致 asyncio 建事件循环都卡住
|
||||
|
||||
或在代码里传入(**只填主机名**,端口由 `port=` 统一指定):
|
||||
```python
|
||||
async with TdxData(standard_servers=["1.2.3.4"]) as td: ...
|
||||
```
|
||||
3. 离线重生成随包清单(该脚本会真实拉一段 K 线,剔除「握手正常但返回空」的坏节点):
|
||||
```bash
|
||||
python -m pytdxdata.server_probe
|
||||
```
|
||||
4. 检查出口网络策略——标准 / MAC 端口 **7709**、EX 端口 **7727** 需要放通。
|
||||
5. 若在用代理软件,注意有些本地代理会劫持回环 TCP,导致 asyncio 建事件循环都卡住
|
||||
(见下方「在 Windows 上测试卡死」)。
|
||||
|
||||
??? question "启动时会变慢吗?会不会在后台偷偷联网?"
|
||||
会有一次**当日首次启动测速**,但完全在后台,对调用方无感:
|
||||
|
||||
- **不阻塞** `start()`、**不打印**任何东西、失败**静默降级**(绝不抛给调用方)
|
||||
- 结果只缓存到用户级目录(**不写安装目录**),**当天后续启动直接复用、不再重测**
|
||||
- 覆盖**全部候选**主机(含暂时被挤下去的),所以「上次慢、这次快」的节点能回来
|
||||
|
||||
三个开关:`PYPYTDX_DATA_DISABLE_PROBE=1` 关掉测速、
|
||||
`PYPYTDX_DATA_CACHE_DIR` 改缓存位置、
|
||||
`PYPYTDX_DATA_{STANDARD,MAC,EX}_HOSTS` 强制指定主机。
|
||||
详见 [安装与配置 → 当日自动测速](getting-started.md#当日自动测速无感)。
|
||||
|
||||
??? question "公司网络 / 代理环境下失败"
|
||||
通达信协议是裸 TCP,不走 HTTP 代理。如果环境强制走代理,需要把 `7709` / `7727`
|
||||
配成直连(bypass)。云服务器上常见问题是安全组只放通了 80/443。
|
||||
@@ -49,7 +73,7 @@
|
||||
| 场景 | 建议 |
|
||||
|---|---|
|
||||
| 偶发查询、CLI | 默认即可 |
|
||||
| 批量落库(`*_batch`) | `pool_max=16`、`concurrency=8~16` |
|
||||
| 批量落库(`get_bars` 传多标的) | `pool_max=16`、`concurrency=8~16` |
|
||||
| Web 服务多并发 | `pool_max=24`,并确保单实例复用 |
|
||||
|
||||
再大只是排队,不会更快——瓶颈是对端服务器的连接上限。
|
||||
@@ -75,6 +99,23 @@
|
||||
2. **是否含未收盘的那根** —— 盘中最后一根是动态的
|
||||
3. 分钟线注意**数据来源深度**不同
|
||||
|
||||
??? question "美股分钟 K 线的时间为什么跨了两个日期?"
|
||||
**已在上游修好(v0.6.2)**。通达信 EX 通道对美股返回的原生时间戳把两个时区拼在一起:
|
||||
日期分量是**美东交易日**、时分分量是**北京时间**。修复前,同一交易日的 390 根会全部
|
||||
落在同一个北京自然日里,并且下午半场被排在上午半场**之前**。
|
||||
|
||||
现在 `SecurityBar.datetime` 已统一为**北京时间**(凌晨段日期自动 +1 天),可直接排序 /
|
||||
过滤 / 做增量水位线。但要注意**美股一个交易日横跨两个北京自然日**:
|
||||
|
||||
```python
|
||||
bars = await td.get_bars("usAAPL", KlinePeriod.MIN_1, count=390)
|
||||
print(bars[0].datetime, bars[-1].datetime) # 2026-10-12 21:31:00 → 2026-10-13 04:00:00
|
||||
print({b.trading_date for b in bars}) # {'2026-10-12'} —— 同一个交易日
|
||||
```
|
||||
|
||||
按**交易日**聚合(分钟线合成日线、日内统计)请用 `b.trading_date`,不要用
|
||||
`b.datetime.date()`。A股 / 港股 / 期货原始即为北京时间,不受影响。
|
||||
|
||||
??? question "缓存会不会返回过期数据?"
|
||||
缓存按数据类型分级 TTL,不会返回超出 TTL 的数据:
|
||||
|
||||
|
||||
+66
-3
@@ -35,6 +35,23 @@
|
||||
|
||||
安装后即可使用 `tdx` 命令行工具(见 [CLI 手册](cli.md))。
|
||||
|
||||
### 可选:DataFrame 后端(0.7.0+)
|
||||
|
||||
想让表格型接口**直接返回 DataFrame**,需要额外装一个后端。核心包依旧零依赖,不装就完全不受影响:
|
||||
|
||||
```bash
|
||||
pip install "pytdxdata[polars]" # 只要 polars
|
||||
pip install "pytdxdata[pandas]" # 只要 pandas
|
||||
pip install "pytdxdata[dataframe]" # 两个都要
|
||||
```
|
||||
|
||||
装上之后在构造时声明一次即可,详见 [DataFrame 输出](dataframe.md):
|
||||
|
||||
```python
|
||||
async with TdxData(df="polars") as td: # 或 df="pandas"
|
||||
df = await td.get_bars("sh600000", KlinePeriod.DAY, count=240)
|
||||
```
|
||||
|
||||
### 可选:uvloop 加速
|
||||
|
||||
Linux / macOS 上可选用 `uvloop` 替换默认事件循环(Windows 不支持):
|
||||
@@ -63,8 +80,11 @@ asyncio.run(main())
|
||||
| `cache_dir` | `None` | 磁盘缓存目录;**`None` = 仅内存缓存**(进程退出即失效) |
|
||||
| `default_adjust` | `Adjust.NONE` | 默认复权方式,可被单次调用覆盖 |
|
||||
| `max_inflight` | `64` | single-flight 并发上限(同键请求合并) |
|
||||
| `pool_min` / `pool_max` | `3` / `12` | 动态连接池上下限 |
|
||||
| `pool_min` / `pool_max` | `3` / `24` | 动态连接池上下限 |
|
||||
| `mac_pool_cap` | `4` | 单台 MAC 服务器连接数上限 |
|
||||
| `std_pool_cap` | `4` | 单台标准服务器连接数上限 |
|
||||
| `ex_pool_cap` | `2` | 单台 EX 服务器连接数上限(EX 主机较少且较慢,压太高收益为负) |
|
||||
| `df` | `None` | 表格型接口的返回形态:`None` = `list[dataclass]`(与 0.6.x 一致);`"polars"` / `"pandas"` = 直接返回 DataFrame。见 [DataFrame 输出](dataframe.md) |
|
||||
|
||||
### 典型配置
|
||||
|
||||
@@ -96,12 +116,16 @@ asyncio.run(main())
|
||||
|
||||
```python
|
||||
async with TdxData(
|
||||
standard_servers=[("1.2.3.4", 7709), ("5.6.7.8", 7709)],
|
||||
mac_servers=[("9.10.11.12", 7709)],
|
||||
standard_servers=["1.2.3.4", "5.6.7.8"],
|
||||
mac_servers=["9.10.11.12"],
|
||||
port=7709,
|
||||
) as td:
|
||||
...
|
||||
```
|
||||
|
||||
主机元素**只填主机名**,端口由构造参数 `port=` 统一指定(不写在主机元素里)。
|
||||
等价的免代码写法是环境变量 `PYPYTDX_DATA_STANDARD_HOSTS` / `…_MAC_HOSTS`(逗号分隔)。
|
||||
|
||||
## 服务器清单
|
||||
|
||||
内置清单随包发布(`src/pytdxdata/servers.json`),由探测脚本生成:
|
||||
@@ -121,6 +145,45 @@ from pytdxdata.config import get_standard_hosts, get_mac_hosts, get_ex_hosts
|
||||
print(len(get_standard_hosts()), len(get_mac_hosts()), len(get_ex_hosts()))
|
||||
```
|
||||
|
||||
### 当日自动测速(无感)
|
||||
|
||||
除随包发布的静态清单外,`TdxData` 还在**当日首次启动**时做一次**后台全量测速**:
|
||||
|
||||
- 覆盖**全部候选**(包内清单 ∪ 代码内置常量 ∪ 环境变量覆盖),而不是只测当前池里的少数节点
|
||||
——否则「上次慢、这次快」的节点永远捞不回来
|
||||
- 用**真实 K 线命令**判定可达(标准 / MAC / EX 各发一条业务命令并要求非空),
|
||||
能剔除「握手正常但返回空数据」的坏节点
|
||||
- 结果按**实测延迟升序**写入**用户级缓存**,当天后续启动直接复用、**不再重测**
|
||||
- 全程在后台进行:**不打印、不阻塞 `start()`、失败静默降级**,对调用方完全无感
|
||||
|
||||
缓存位置(可用 `PYPYTDX_DATA_CACHE_DIR` 覆盖):
|
||||
|
||||
| 平台 | 路径 |
|
||||
|---|---|
|
||||
| Windows | `%LOCALAPPDATA%\pytdxdata\hosts-today.json` |
|
||||
| macOS | `~/Library/Caches/pytdxdata/hosts-today.json` |
|
||||
| Linux / 其他 | `$XDG_CACHE_HOME/pytdxdata/hosts-today.json`(默认 `~/.cache/…`) |
|
||||
|
||||
主机选择的优先级(各协议独立):
|
||||
|
||||
1. `PYPYTDX_DATA_STANDARD_HOSTS` / `…_MAC_HOSTS` / `…_EX_HOSTS` 环境变量(手工指定,最高优先)
|
||||
2. **当日测速缓存**中该协议实测可达的主机(按延迟升序)
|
||||
3. 包内清单里 `status=ok` 的 `fast_pool` 主机
|
||||
4. 包内清单里 `status=ok` 的主机
|
||||
5. 代码内置常量
|
||||
|
||||
!!! note "缓存绝不写回安装目录"
|
||||
测速结果只写入上面的**用户级缓存目录**。`site-packages` 在容器 / 系统级安装 / `pipx`
|
||||
下可能只读,多用户共用一个安装时还会互相覆盖,因此包内 `servers.json` 不会被运行时改写。
|
||||
|
||||
环境变量一览:
|
||||
|
||||
| 变量 | 作用 |
|
||||
|---|---|
|
||||
| `PYPYTDX_DATA_STANDARD_HOSTS` / `…_MAC_HOSTS` / `…_EX_HOSTS` | 逗号分隔,手工指定主机(优先级最高,直接跳过测速缓存) |
|
||||
| `PYPYTDX_DATA_CACHE_DIR` | 覆盖当日测速缓存目录 |
|
||||
| `PYPYTDX_DATA_DISABLE_PROBE=1` | 关闭当日测速与缓存(CI / 测试用,保证确定性) |
|
||||
|
||||
## 生命周期
|
||||
|
||||
`TdxData` 必须在 `async with` 内使用,或手动 `await td.start()` / `await td.close()`:
|
||||
|
||||
+18
-6
@@ -20,7 +20,7 @@ from pytdxdata.models import KlinePeriod
|
||||
|
||||
async def main():
|
||||
async with TdxData() as td: # 自动探测服务器 + 建池 + 建缓存
|
||||
bars = await td.get_kline(1, "600000", KlinePeriod.DAY, count=2400)
|
||||
bars = await td.get_bars("sh600000", KlinePeriod.DAY, count=2400)
|
||||
print(bars[-1].datetime, bars[-1].close)
|
||||
|
||||
asyncio.run(main())
|
||||
@@ -34,9 +34,10 @@ asyncio.run(main())
|
||||
| 🗄️ **双层 TTL 缓存** | 内存 LRU + 磁盘 SQLite(WAL);TTL 分级:报价 5s / 分钟K线 60s / 日线当天 / 复权K线 1天 / 历史逐笔永久;`single-flight` 同键并发去重;缓存「已解析记录」,重复请求 0 网络 0 解析 |
|
||||
| ⚡ **并发分页** | 把大请求拆成显式 `start` 偏移的页,多连接并行拉取、按序拼接、失败换连接重试 |
|
||||
| 🔀 **双通道整合** | 标准通道(经典行情)+ MAC 通道(逐笔 / 复权 / 1分钟K线),按命令自动选路 |
|
||||
| 📄 **服务器文件解析** | 板块 `block_*.dat` / 行业 `tdxhy.cfg` / 历史财报 `gpcw*.zip` **下载即解析**成 dataclass(纯 stdlib),无需自己拆二进制 |
|
||||
| 🚀 **批量下载** | `get_kline_batch` / `get_minute_batch`,多股票自动并发,连接池分配到不同服务器 |
|
||||
| 📦 **零重型依赖** | 返回 `list[dataclass]`,pandas / polars 由调用方按需转换 |
|
||||
| 📄 **服务器文件解析** | 板块 `block_*.dat` / 行业 `tdxhy.cfg` / 历史财报 `gpcw*` **下载即解析**成 dataclass(纯 stdlib),无需自己拆二进制;文件名与索引不一致时自动纠正 |
|
||||
| 🚀 **批量下载** | `get_bars` / `get_minutes` 传标的列表即多股票并发,连接池自动分配到不同服务器;K 线走「多连接 × 单连接流水线」,全市场 5,225 只 1 分钟线取当天 **5~8 秒**(盘中增量 1~2 秒) |
|
||||
| 🐼 **DataFrame 直出** | `TdxData(df="polars")` / `df="pandas"` 让 42 个表格型接口直接返回 DataFrame;时间列自动合并成 `datetime`,五档报价自动展开。后端是可选 extras,**核心包仍零依赖** |
|
||||
| 📦 **零重型依赖** | 默认返回 `list[dataclass]`,行为与 0.6.x 完全一致,转不转 DataFrame 由你决定 |
|
||||
|
||||
## 从这里开始
|
||||
|
||||
@@ -66,11 +67,19 @@ asyncio.run(main())
|
||||
|
||||
[:octicons-arrow-right-24: Cookbook](cookbook.md)
|
||||
|
||||
- **直接要 DataFrame**
|
||||
|
||||
---
|
||||
|
||||
`TdxData(df="polars")` 一行开关,42 个表格型接口直接出表,不用自己 `asdict`。
|
||||
|
||||
[:octicons-arrow-right-24: DataFrame 输出](dataframe.md)
|
||||
|
||||
- **查接口签名**
|
||||
|
||||
---
|
||||
|
||||
51 个数据方法(另有 `start` / `close`),从 docstring 实时生成,不会过期。
|
||||
35 个统一方法(+ 33 个 deprecated 别名),从 docstring 实时生成,不会过期。
|
||||
|
||||
[:octicons-arrow-right-24: API 参考](api/index.md)
|
||||
|
||||
@@ -104,7 +113,8 @@ asyncio.run(main())
|
||||
## 数据来源与限制
|
||||
|
||||
- 行情来自通达信**公开**行情服务器(无需账号),**15 分钟延时**;需实时数据请使用券商行情
|
||||
- 服务器清单由 `python -m pytdxdata.server_probe` 探测生成,可用性随时间变化
|
||||
- 服务器清单由 `python -m pytdxdata.server_probe` 探测生成;运行时**当日首次启动会自动在后台测速并缓存**
|
||||
(对调用方无感,当天不再重测),可用性随时间变化
|
||||
- 港股 / 美股 / 期货 / 期权走通达信 EX 通道(端口 7727),部分市场(美股期权、港股期权)协议内无数据
|
||||
|
||||
完整清单见 [常见问题](faq.md)。
|
||||
@@ -121,6 +131,7 @@ asyncio.run(main())
|
||||
- [安装与配置](getting-started.md)
|
||||
- [场景示例](cookbook.md)
|
||||
- [命令行工具](cli.md)
|
||||
- [DataFrame 输出](dataframe.md)
|
||||
|
||||
- **参考**
|
||||
- [接口总览](api/index.md)
|
||||
@@ -138,6 +149,7 @@ asyncio.run(main())
|
||||
- [AI 助手接入](ai.md)
|
||||
- [常见问题](faq.md)
|
||||
- [更新日志](changelog.md)
|
||||
- [接口重构审查](refactor-audit.md)
|
||||
- [参与贡献](contributing.md)
|
||||
|
||||
</div>
|
||||
@@ -6,7 +6,7 @@
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A["api.py — TdxData<br/>唯一对外入口,51 个公开方法"] --> B["router.py<br/>按命令选通道"]
|
||||
A["api.py — TdxData<br/>唯一对外入口,28 个统一方法 + 33 个废弃别名"] --> B["router.py<br/>按命令选通道"]
|
||||
B --> C["pool/ — 动态连接池<br/>标准池 / MAC 池 / EX 池"]
|
||||
B --> D["scheduler/ — 并发分页调度<br/>Paginator"]
|
||||
C --> E["protocol/ — 协议层<br/>帧 / 变长整数 / 价格 / 日期 / 命令"]
|
||||
@@ -21,8 +21,9 @@ flowchart TD
|
||||
```
|
||||
src/pytdxdata/
|
||||
├── api.py # 统一接口 TdxData(对外唯一入口)
|
||||
├── symbols.py # 标的解析:字符串 → Symbol(parse_symbol / format_symbol / market_targets)
|
||||
├── router.py # 标准 / MAC / EX 通道选路
|
||||
├── config.py # 服务器清单加载(servers.json)
|
||||
├── config.py # 服务器清单加载 + 选路优先级链 + 用户级测速缓存路径
|
||||
├── indicator.py # 技术指标(纯函数,无依赖)
|
||||
├── server_probe.py # 服务器探测,可 python -m 运行
|
||||
├── protocol/ # 协议层
|
||||
@@ -38,7 +39,7 @@ src/pytdxdata/
|
||||
│ ├── pool.py # 标准 / MAC 动态池
|
||||
│ ├── ex_pool.py # EX 池
|
||||
│ ├── health.py # 健康分与选路
|
||||
│ └── probe_cache.py # 探测结果缓存
|
||||
│ └── probe_cache.py # 当日全量测速 + 用户级缓存(后台、无感)
|
||||
├── scheduler/
|
||||
│ └── paginator.py # 并发分页调度(3 连接并行)
|
||||
├── cache/
|
||||
@@ -55,7 +56,7 @@ src/pytdxdata/
|
||||
└── financial.py # gpcw.txt / gpcw*.zip
|
||||
```
|
||||
|
||||
## 一次 `get_kline` 的完整路径
|
||||
## 一次 `get_bars` 的完整路径
|
||||
|
||||
```mermaid
|
||||
sequenceDiagram
|
||||
@@ -66,7 +67,7 @@ sequenceDiagram
|
||||
participant P as pool/
|
||||
participant S as scheduler/
|
||||
|
||||
U->>A: get_kline(SH, 600000, DAY, count=2400)
|
||||
U->>A: get_bars("sh600000", DAY, count=2400)
|
||||
A->>A: 规范化缓存键
|
||||
A->>C: 查缓存
|
||||
alt 命中
|
||||
@@ -99,7 +100,7 @@ sequenceDiagram
|
||||
|---|---|
|
||||
| **零运行时依赖** | 用 `socket` / `zlib` / `struct` / `sqlite3` / `asyncio` 全部搞定,避免依赖地狱和版本冲突 |
|
||||
| **返回 `list[dataclass]`** | 不让调用方被 pandas 绑架;需要 DataFrame 自己转一行 |
|
||||
| **API 向后兼容** | 新增能力用新方法名(`get_block_parsed` 而不是改 `get_block_info` 的返回类型),老代码永不失效 |
|
||||
| **API 向后兼容** | 一个数据类型收敛为一个跨市场统一方法;旧方法名保留为 deprecated 别名(1.0 移除),老代码在兼容窗口内仍可用 |
|
||||
|
||||
## 下一步
|
||||
|
||||
|
||||
@@ -44,7 +44,7 @@
|
||||
# 100 个协程同时请求同一只股票的同一段 K 线
|
||||
# → 只发 1 次网络请求
|
||||
results = await asyncio.gather(*[
|
||||
td.get_kline(Market.SH, "600000", KlinePeriod.DAY, count=800)
|
||||
td.get_bars("sh600000", KlinePeriod.DAY, count=800)
|
||||
for _ in range(100)
|
||||
])
|
||||
```
|
||||
@@ -81,11 +81,11 @@ results = await asyncio.gather(*[
|
||||
|
||||
EWMA(指数加权移动平均)延迟让选路对「最近变慢」敏感,而不是被历史均值拖住。
|
||||
|
||||
### 为什么 MAC 池更小
|
||||
### MAC 通道的单台连接上限
|
||||
|
||||
MAC 通道可用服务器只有数台(见 [通道与数据源](routing.md#server-groups)),
|
||||
所以 `mac_pool_cap` 默认只有 `4`——这是**对端限制**,不是本项目的选择。
|
||||
逐笔 / 分时的并发能力天然弱于日线,原因就在这里。
|
||||
`mac_pool_cap` 默认 `4`,与标准池的 `std_pool_cap` 一致——这是**对端单台连接限制**,
|
||||
不是本项目的选择。两个通道的可用台数见 [通道与数据源](routing.md#server-groups):
|
||||
实测可达台数会随服务器上下线变化,以**当日测速缓存**为准。
|
||||
|
||||
## 并发分页
|
||||
|
||||
@@ -110,4 +110,5 @@ count=2400 → 3 页 → 3 个连接并行 → 按序拼接
|
||||
## 下一步
|
||||
|
||||
- [架构总览](architecture.md) — 这些模块在整体里的位置
|
||||
- [通道与数据源](routing.md#daily-probe) — 服务器是怎么选出来的(当日测速缓存)
|
||||
- [常见问题](../faq.md) — 缓存与超时的排查
|
||||
+55
-10
@@ -20,7 +20,7 @@
|
||||
| 港股 / 美股 / 期货 / 期权 | **EX**(7727) | 完全不同的协议栈 |
|
||||
|
||||
!!! note "只请求,不改语义"
|
||||
路由是**内部实现细节**。同一个 `get_kline()` 调用,传不传 `adjust` 会走不同通道,
|
||||
路由是**内部实现细节**。同一个 `get_bars()` 调用,传不传 `adjust` 会走不同通道,
|
||||
但返回类型和字段完全一致——你不需要写 `if adjust:` 分支。
|
||||
|
||||
## 三组服务器 {#server-groups}
|
||||
@@ -29,10 +29,13 @@
|
||||
|
||||
| 池 | 数量级 | 用途 |
|
||||
|---|---|---|
|
||||
| **标准池** | 数十台 | 报价 / K线 / 列表 / 财务 |
|
||||
| **MAC 池** | 数台 | 逐笔 / 分时 / 复权 / 板块 / 资金流 |
|
||||
| **标准池** | 9 台 | 报价 / K线 / 列表 / 财务 |
|
||||
| **MAC 池** | 22 台 | 逐笔 / 分时 / 复权 / 板块 / 资金流 |
|
||||
| **EX 池** | 4 台 | 港股 / 美股 / 期货 / 期权 |
|
||||
|
||||
> 上表为 2026-09-21 离线探针(`python -m pytdxdata.server_probe --discover`)实测**可达**台数。
|
||||
> 服务器会下线、换 IP 或变更协议,**实际台数以当日测速缓存为准**(见下方「运行时:当日测速缓存」)。
|
||||
|
||||
EX 池内部还要再分两类,因为握手方式不同:
|
||||
|
||||
| 类型 | 数量 | 握手 |
|
||||
@@ -42,7 +45,7 @@ EX 池内部还要再分两类,因为握手方式不同:
|
||||
|
||||
这就是 `config.get_ex_handshakes()` 存在的原因——EX 的 4 台机器不是同一种握手。
|
||||
|
||||
MAC 池远小于标准池,这也解释了为什么逐笔 / 分时的并发能力天然弱于日线。
|
||||
注意 MAC 池实测**并不比标准池小**(22 台 vs 9 台)。早期版本曾按「MAC 可用节点很少」来解释逐笔 / 分时并发弱于日线,实测已推翻该前提——两者的单台连接上限(`mac_pool_cap` / `std_pool_cap`)默认相同,都是 `4`。
|
||||
|
||||
## 为什么这些服务器会被「探测」
|
||||
|
||||
@@ -51,12 +54,56 @@ MAC 池远小于标准池,这也解释了为什么逐笔 / 分时的并发能
|
||||
`server_probe` 的探测逻辑**不是只看 TCP 连通性**,而是真实请求一段 K 线并校验内容:
|
||||
|
||||
```bash
|
||||
python -m pytdxdata.server_probe
|
||||
python -m pytdxdata.server_probe # 按申报协议全量探测
|
||||
python -m pytdxdata.server_probe --discover # 三种协议全测,并据实修正 protocols
|
||||
python -m pytdxdata.server_probe --group ex # 只测 EX
|
||||
python -m pytdxdata.server_probe --refresh # 只重测当前 status=ok 的
|
||||
```
|
||||
|
||||
普通「只 ping 握手」的探测会把这类坏节点放进来,症状是**偶发的空返回**——
|
||||
比直接连不上更难查。所以探测必须校验数据。
|
||||
|
||||
写回字段:`status` / `latency` / `checked_at` / `checked_date` / `fast_pool`,
|
||||
以及顶层 `version` / `checked_date` / `note`。其中 `fast_pool` 与运行时缓存里的 `fast`
|
||||
用同一条规则(整体延迟前 10 名,或延迟 < 200ms),用于**还没有当日缓存时**的冷启动排序。
|
||||
|
||||
### 为什么需要 `--discover`
|
||||
|
||||
「申报协议」会过期。实测(2026-09-21)有一批服务器的 **`0x010C` 标准 K 线变成空响应
|
||||
(返回 2 字节空壳),但同端口同主机的 `0x122E` MAC 帧正常**——它们从 `standard` 退化成了
|
||||
`mac`-only。只测申报协议的话,这类退化永远发现不了,标准池会一直含着一批死节点。
|
||||
|
||||
`--discover` 对每台主机把三种协议都测一遍,并据实测结果重写 `protocols`;
|
||||
配套两个必须处理的细节:
|
||||
|
||||
- **失败必须重试一次**再判定不支持。单次抖动会误删协议,而 `protocols` 被收窄后不会自动恢复。
|
||||
- **EX 握手模式**申报可能错,探测失败时换另一种(`login` ⇄ `setup`)再试。
|
||||
- 无任何协议可达时**不动 `protocols`**,只把 `status` 置 `fail`——保留"它申报过什么"的信息。
|
||||
- 顺带剔除 `ip` 非法的脏记录(历史遗留如 `"s1"`)。
|
||||
|
||||
## 运行时:当日测速缓存 {#daily-probe}
|
||||
|
||||
`python -m pytdxdata.server_probe` 是**离线**重生成随包清单的工具;运行时还有一条**在线**的
|
||||
轻量路径——`TdxData.start()` 会异步触发一次全量测速,把结果缓存到**用户级目录**
|
||||
(`…/pytdxdata/hosts-today.json`),当天后续启动直接复用,不再重测。
|
||||
|
||||
设计与几个必须踩对的点:
|
||||
|
||||
| 关注点 | 正确做法 | 反例(会静默出错) |
|
||||
|---|---|---|
|
||||
| 候选集 | 取**全部候选**(清单 ∪ 常量 ∪ env) | 只把「当前池成员」丢给探针 → 上次慢的节点永远捞不回来 |
|
||||
| 可达判定 | 发**真实 K 线命令**并要求非空 | 只握手 / 只读报文头 → 测不出「空响应」坏节点 |
|
||||
| 写盘位置 | 用户级缓存目录 | 写回包内 `servers.json` → 容器只读、多用户互相覆盖 |
|
||||
| 对调用方 | 后台 task + debug 日志 + 绝不抛异常 | 在 `start()` 里 `await` 或打印 → 启动变慢、污染用户输出 |
|
||||
| 并发 | 进程内单例 + `asyncio.Lock` | 三个池各测一遍 → 三倍流量 |
|
||||
| 写入 | 临时文件 + `os.replace` 原子替换 | 直接覆写 → 并发进程读到半截 JSON |
|
||||
|
||||
全部不可达时**不写缓存**——否则「今天全挂了」会被缓存一整天,明天也不重试。
|
||||
|
||||
缓存存在时,`config.get_standard_hosts()` 等按**实测延迟升序**返回该协议的可达主机;
|
||||
缓存缺失 / 跨日 / 版本不符 / 已损坏时,才回落到包内清单。开关见
|
||||
[安装与配置 → 环境变量](../getting-started.md#当日自动测速无感)。
|
||||
|
||||
## EX 通道细节
|
||||
|
||||
- 端口固定 **7727**,与标准 / MAC 的 7709 不同
|
||||
@@ -66,13 +113,11 @@ python -m pytdxdata.server_probe
|
||||
- 美股期权 / 港股期权**协议内无数据**
|
||||
|
||||
```python
|
||||
from pytdxdata.models import ExMarket, KlinePeriod
|
||||
|
||||
await td.get_kline(ExMarket.HK_MAIN_BOARD, "00700", KlinePeriod.DAY, count=700)
|
||||
await td.get_kline(ExMarket.US_STOCK, "AAPL", KlinePeriod.DAY, count=700)
|
||||
await td.get_bars("hk00700", KlinePeriod.DAY, count=700)
|
||||
await td.get_bars("usAAPL", KlinePeriod.DAY, count=700)
|
||||
```
|
||||
|
||||
`ExMarket` 的完整成员见 [枚举速查](../api/enums.md#exmarket)。
|
||||
扩展市场用字符串前缀(`hk` / `us` / `cffex` …)指定;`ExMarket` 的完整成员见 [枚举速查](../api/enums.md#exmarket)。
|
||||
|
||||
## 下一步
|
||||
|
||||
|
||||
+69
-17
@@ -14,11 +14,11 @@ pip install pytdxdata # 或 uv add pytdxdata;要求 Python >= 3.11
|
||||
```python
|
||||
import asyncio
|
||||
from pytdxdata import TdxData
|
||||
from pytdxdata.models import KlinePeriod, Market
|
||||
from pytdxdata.models import KlinePeriod
|
||||
|
||||
async def main():
|
||||
async with TdxData() as td: # 自动探测服务器、建连接池、初始化缓存
|
||||
bars = await td.get_kline(Market.SH, "600000", KlinePeriod.DAY, count=2400)
|
||||
bars = await td.get_bars("sh600000", KlinePeriod.DAY, count=2400)
|
||||
print(bars[-1].datetime, bars[-1].close)
|
||||
|
||||
asyncio.run(main())
|
||||
@@ -26,10 +26,11 @@ asyncio.run(main())
|
||||
|
||||
## 心智模型(AI 必读)
|
||||
|
||||
1. **入口只有一个类**:`TdxData`。所有数据获取都是它的公开异步方法(51 个数据方法,
|
||||
另有 `start()` / `close()` 生命周期方法),必须在 `async with` 内使用。
|
||||
2. **标的需要「市场 + 代码」两个参数**。同一个 6 位代码在不同市场含义不同:
|
||||
`(SH, "000001")` 是上证指数,`(SZ, "000001")` 是平安银行。
|
||||
1. **入口只有一个类**:`TdxData`。所有数据获取都是它的公开异步方法(v0.6 起为 **28 个统一方法**,
|
||||
另有 33 个 deprecated 别名与 `start()` / `close()` 生命周期方法),必须在 `async with` 内使用。
|
||||
2. **标的用字符串**,形如 `sz000001` / `sh600000` / `hk00700` / `usAAPL` / `cffex:IFL0`。
|
||||
同一个 6 位代码在不同市场含义不同:`sh000001` 是上证指数,`sz000001` 是平安银行;
|
||||
裸 6 位数字(如 `000001`)视为沪市指数。统一接口不再接受 `(market, code)` 元组。
|
||||
3. **枚举取值没有直觉,必须用枚举成员,不要写数字字面量**:
|
||||
- `Market`: SZ=0 / SH=1 / BJ=2
|
||||
- `KlinePeriod`: MIN_5=0, MIN_15=1, MIN_30=2, MIN_60=3, **DAY=4**, WEEK=5, MONTH=6, **MIN_1=7**, MIN_3=8, YEAR=9, SEASON=10, YEAR_ALT=11
|
||||
@@ -39,19 +40,65 @@ asyncio.run(main())
|
||||
5. **实例要复用**:连接池与缓存都挂在 `TdxData` 实例上。长驻进程(Web 服务)应只创建一个实例,
|
||||
不要每请求新建,否则等于每请求重建连接池且缓存全部失效。
|
||||
|
||||
## 主要能力分组
|
||||
## 主要能力分组(统一接口)
|
||||
|
||||
- **K 线**:`get_kline`(主接口,自动并发分页、支持复权与日期区间)、`get_kline_batch`(多股票并发)、
|
||||
`get_kline_since`(增量)、`get_index_kline`、`get_stock_kline_with_indicators`
|
||||
- **报价**:`get_quotes`(五档,>80 只自动切批)、`get_stock_quotes_list`、`get_market_stat`
|
||||
- **逐笔 / 分时**:`get_transactions`(含 `trade_count` 成交笔数,支持历史日期)、`get_minute`、`get_minute_batch`
|
||||
- **证券列表**:`get_security_count`、`get_security_list`、`get_security_list_all`
|
||||
标的一律用字符串;以下均为 v0.6 统一接口,跨市场自动选路。
|
||||
|
||||
- **标的清单**:`get_universe`(A股 + 港股 + 美股 + 期货/期权)。⚠️ 北交所(`market="bj"`)清单**协议不支持**,返回空列表(`0x053E` 对 market=2 无响应,27 台服务器全测);但北交所**标的本身可用** —— 报价/K线/逐笔直接传 `bj920992` 这类代码正常
|
||||
- **K 线**:`get_bars`(自动并发分页、支持复权与日期区间、多标的并发)。传 ≥16 只时自动走
|
||||
**多连接流水线**(`pipeline` 默认自适应 8~32,传 `1` 关闭),全市场 5,200 只 1 分钟 K 线
|
||||
取当天 43.6s → 5s 级;`start_date=今天` 盘中只拉已走过的根数(10:00 → 30 根),实时刷新 1~2 秒
|
||||
- ⚠️ **EX 通道(港股/美股/期货/期权)的 `amount` 不是全有的**:`0x23FF` 帧内只有一个 4B 槽位可复用,
|
||||
权益类(港股主板 31/创业板 48/基金 49、新三板 44、美股 74)装**成交额**(`amount` 正常),
|
||||
期货期权装**结算价**(见 `SecurityBar.settle`,日线才有值)→ 它们 `amount` 恒 **0.0**,
|
||||
是协议限制而非解析漏读;港股主板/基金的 `vol` 单位是**手(=100 股)**,`amount/vol` 是均价的 100 倍
|
||||
- **技术指标**:`get_indicators`(K线 + MACD/KDJ/RSI/BOLL/MA/EMA)
|
||||
- **报价**:`get_quotes`(五档,>80 只自动切批;**单连接流水线**默认开启,全市场 5,200 只 A股 ~0.33s)、`get_snapshot`(个股特征)、`get_market_stat`
|
||||
- 价格标度:协议给的是整数,真实价 = `整数 / 10**小数位`。小数位由 `price_decimal_digits` 决定,
|
||||
优先用**权威表**(证券清单自带的 `SecurityInfo.decimal_point`,`start()` 每日后台同步一次,
|
||||
也可 `await td.sync_decimal_points()` 强制刷新),未同步时退回静态前缀规则。
|
||||
债券/回购是 **4 位**、沪市 B 股 3 位 —— 这里错过两次(整批价格差 100 倍/10 倍)。
|
||||
- **逐笔 / 分时**:`get_ticks`(含 `trade_count`,支持历史日期,返回按时间升序;`vol`/`trade_count` 为约 3 秒切片增量而非累计;已剔除 `bs_flag=8` 竞价委托;**指数**的 `vol` 是百元、`trade_count` 是均价×1000;**可转债**的 `vol` 是手=10 张、`trade_count` 是真实笔数,故日线 `amount/vol` 是均价的 **10 倍**,算均价要再 /10)、`get_minutes`(`sampling=True` 取缩略采样;`hour` 恒为合法时钟值 0~23,美股一个交易日跨两个北京自然日,按交易日聚合用 `MinuteBar.trading_date`、取时刻用 `MinuteBar.datetime`)
|
||||
- **财务 / 除权**:`get_finance`(最新快照)、`get_xdxr`、`get_price_limits`(本地计算涨跌停)
|
||||
- **板块**:`get_board_list`、`get_board_members`、`get_board_ranking`、`get_board_change_ranking`、`get_belong_board`
|
||||
- **服务器文件(下载 + 解析一步到位)**:`get_block_parsed`(板块成分)、`get_industry_map`(通达信+申万行业)、
|
||||
`get_financial_file_infos`、`get_financial_records_parsed`(历史专业财报)
|
||||
- **扩展市场**:`get_goods_list`、`get_goods_quotes`、`get_goods_transaction_all` 等,配合 `ExMarket` 使用
|
||||
- **批量**:`get_kline_batch` / `get_minute_batch`,多股票并发且连接池会分配到不同服务器
|
||||
- **F10**:`get_f10`(公司资料目录 + 正文)
|
||||
- **板块**:`get_boards`、`get_board_members`、`get_board_of`、`get_board_summary`、
|
||||
`get_board_ranking`、`get_board_change_ranking`
|
||||
- **服务器文件(下载 + 解析一步到位)**:`get_file`(原始字节)、`get_block_parsed`(板块成分)、
|
||||
`get_industry_map`(通达信+申万行业)、`get_financial_file_infos`、`get_financial_records_parsed`(历史专业财报)
|
||||
|
||||
> 旧的分通道/市场方法(`get_kline` / `get_transactions` / `get_goods_*` 等共 33 个)保留为
|
||||
> deprecated 别名,1.0 移除;完整对照见[接口重构审查](https://openbot-coder.github.io/pytdxdata/refactor-audit/)与 CHANGELOG。
|
||||
|
||||
## DataFrame 输出(v0.7+)
|
||||
|
||||
表格型接口可以直出 pandas / polars DataFrame,开关只有构造参数 `df`:
|
||||
|
||||
```python
|
||||
from pytdxdata import TdxData
|
||||
|
||||
async with TdxData(df="polars") as td: # 或 df="pandas";df=None(默认)= list[dataclass]
|
||||
df = await td.get_bars("sh600000", count=240)
|
||||
```
|
||||
|
||||
- 需先装 extra:`pip install "pytdxdata[polars]"` / `[pandas]` / `[dataframe]`。**核心包仍零依赖**。
|
||||
- `df=None` 时行为与 0.6.x 完全一致(装饰器直接透传,零开销);`df` 非法抛 `ValueError`,包没装抛 `FrameBackendError`(带安装命令)。
|
||||
- 列口径:**时间列替换**拆开的 `year/month/day/hour/minute` → `datetime`(+ `trading_date`);`bid`/`ask` 五档展开成 10 列;`fields` 字典 JSON 化成字符串列(空字典给 `None`);`TdxBlock.codes` 保持列表。
|
||||
- `get_indicators` → bars + 指标合并成一张表;`get_index_trade_stats(_multi)` → `minutes` 展开并补 `index`/`date` 常量列;`get_kline_batch`/`get_minute_batch` → 各标的纵向拼接成一张表。
|
||||
- **不受影响**:`bytes`/`int`/`dict`/单对象/`tuple`/`list[float]`/`list[str]` 类接口原样返回(如 `get_file`、`get_security_count`、`get_server_info`)。
|
||||
- 空结果返回**列名齐全的空表**,但 dtype 只由后端推断。
|
||||
|
||||
## 服务器与配置
|
||||
|
||||
服务器主机来源有一条优先级链(各协议独立):环境变量 → **当日测速缓存** → 包内清单
|
||||
`fast_pool` → 包内清单 → 代码内置常量。
|
||||
|
||||
- `TdxData.start()` 会在**当日首次启动**后台触发一次全量测速,用**真实 K 线命令**判定可达
|
||||
(不是只握手),结果按延迟升序写入**用户级缓存**(Windows `%LOCALAPPDATA%\pytdxdata`、
|
||||
macOS `~/Library/Caches/pytdxdata`、Linux `$XDG_CACHE_HOME/pytdxdata`),当天复用、不再重测。
|
||||
- 该操作对调用方**完全无感**:后台执行、不打印、不阻塞 `start()`、失败静默降级。
|
||||
运行时**绝不改写包内 `servers.json`**。
|
||||
- 环境变量:`PYPYTDX_DATA_STANDARD_HOSTS` / `…_MAC_HOSTS` / `…_EX_HOSTS`(逗号分隔,手工指定,最高优先)、
|
||||
`PYPYTDX_DATA_CACHE_DIR`(覆盖缓存目录)、`PYPYTDX_DATA_DISABLE_PROBE=1`(关闭测速)。
|
||||
|
||||
## CLI
|
||||
|
||||
@@ -74,6 +121,10 @@ tdx financial gpcw20260331.zip --limit 5
|
||||
- **`KlinePeriod.MIN_1` 走 MAC 通道并单连接串行分页**,这是刻意设计:并发分页在多服务器
|
||||
历史深度不一致时会产生非确定性数据空洞。不要试图"优化"成并发。
|
||||
- **复权 K 线走 MAC 通道**,标准通道不提供复权数据。
|
||||
- **K 线 `datetime` 统一为北京时间**(v0.6.2 起)。美股原始报文的日期分量是美东交易日、
|
||||
时分分量是北京时间,库已自动归一化,`bar.datetime` 可直接排序 / 比较 / 过滤。
|
||||
但**美股一个交易日横跨两个北京自然日**:按交易日聚合(分钟线合成日线、日内统计)
|
||||
用 `bar.trading_date`,不要用 `bar.datetime.date()`。`bar.is_us` 判定美股。
|
||||
- **标准通道部分服务器在 `count < 100` 时返回空**,分页器已强制 `min_page=100` 规避。
|
||||
- **本地 Windows 上 `asyncio.new_event_loop()` 可能永久挂起**,根因是回环 `socket.socketpair()`
|
||||
被本地代理/安全软件拦截。异步测试请在 Linux/WSL 运行。
|
||||
@@ -102,6 +153,7 @@ tdx financial gpcw20260331.zip --limit 5
|
||||
|
||||
- [SKILL.md](https://raw.githubusercontent.com/openbot-coder/pytdxdata/main/SKILL.md):面向 AI **编程**助手的技能文件(必守约定 / 枚举取值表 / 决策表 / 陷阱清单),比本文件更适合指导写代码
|
||||
- [完整接口清单](https://openbot-coder.github.io/pytdxdata/API_REFERENCE/):手工维护的全量清单,含参数默认值
|
||||
- [接口重构审查](https://openbot-coder.github.io/pytdxdata/refactor-audit/):统一接口 / 废弃别名全清单 + 数据一致性与遗漏对账
|
||||
- [更新日志](https://github.com/openbot-coder/pytdxdata/blob/main/CHANGELOG.md)
|
||||
- [GitHub 仓库](https://github.com/openbot-coder/pytdxdata)
|
||||
- [PyPI](https://pypi.org/project/pytdxdata/)
|
||||
@@ -0,0 +1,148 @@
|
||||
# 接口重构审查报告
|
||||
|
||||
> 范围:v0.6 统一接口重构(`TdxData` 对外方法 51 → 63)的**数据一致性**与**遗漏审查**。
|
||||
> 结论基于对旧实现(`git show HEAD:src/pytdxdata/api.py`)与新实现的逐字段 / 逐方法对照。
|
||||
|
||||
## 一、结论速览
|
||||
|
||||
| 维度 | 结论 |
|
||||
|------|------|
|
||||
| 数据一致性(旧接口 vs 统一接口) | ✅ 55 条断言全部通过,无字段差异 |
|
||||
| 方法丢失 | ✅ 无。旧 53 个公开方法 100% 保留 |
|
||||
| 文档覆盖 | ✅ `scripts/check_docs.py` 三条硬约束通过 |
|
||||
| 标的写法 | ✅ 统一接口一律只接受字符串,已无元组软兼容 |
|
||||
| 公开方法计数 | `28 统一 + 33 别名 + start/close = 63` |
|
||||
|
||||
---
|
||||
|
||||
## 二、数据一致性(旧接口 vs 新统一接口)
|
||||
|
||||
逐字段对照 24+ 组、**55 条断言全部通过**,覆盖:
|
||||
|
||||
- K 线(各周期 / 复权 / 日期区间)
|
||||
- 实时报价(五档 bid/ask、买卖量、涨跌停价)
|
||||
- 逐笔成交(含 `trade_count`)
|
||||
- 分时
|
||||
- 除权除息
|
||||
- 财务快照
|
||||
- F10(目录 + 正文)
|
||||
- 服务器文件(板块 / 行业 / 财报)
|
||||
- 板块(列表 / 成员 / 归属 / 汇总 / 排行)
|
||||
|
||||
唯一 1 条「失败」经核实为 **mock 假象**:`security_count` 的 mock 返回固定值 100,而
|
||||
`get_universe` 走真实分页逻辑,两者口径不同;**并非真实数据差异**,逐字段比较全部相等。
|
||||
|
||||
---
|
||||
|
||||
## 三、遗漏审查(方法级)
|
||||
|
||||
对旧 / 新 `api.py` 的公开方法集合做 AST 级 diff:
|
||||
|
||||
| 指标 | 数量 |
|
||||
|------|------|
|
||||
| 旧公开方法 | 53(51 数据 + `start` / `close`) |
|
||||
| 新公开方法 | 63(61 数据 + `start` / `close`) |
|
||||
| **旧有新无(丢失)** | **0** |
|
||||
| 新增 | 10(全部为统一入口) |
|
||||
|
||||
**数值闭合**:
|
||||
|
||||
- `28 统一 + 33 废弃别名 + start/close = 63`
|
||||
- `18 旧名升级 + 33 转别名 = 51 旧数据方法`
|
||||
- `51 旧 + 10 新入口 = 61 数据方法`
|
||||
|
||||
---
|
||||
|
||||
## 四、统一接口清单(28 个数据接口)
|
||||
|
||||
标的写法:`"sz000001"` / `"hk00700"` / `"usAAPL"` / `"cffex:IFL0"`;裸 6 位数字 = 沪市指数。
|
||||
|
||||
### 4.1 新入口(10,v0.6 新增)
|
||||
|
||||
| 方法 | 签名 | 返回 | 吸收的旧接口 |
|
||||
|------|------|------|--------------|
|
||||
| `get_universe` | `(market=None)` | `list[SecurityInfo]` | get_security_count / get_security_list / get_security_list_all / get_goods_list / get_goods_count |
|
||||
| `get_bars` | `(symbols, period=DAY, *, start=0, count=800, adjust=None, start_date=None, end_date=None, concurrency=6)` | `list[SecurityBar]` | get_kline / get_kline_since / get_kline_batch / get_index_kline |
|
||||
| `get_indicators` | `(symbols, indicators, *, period=DAY, count=30, adjust=None, params=None)` | `list[IndicatorSet]` | get_stock_kline_with_indicators |
|
||||
| `get_ticks` | `(symbols, *, date=None, start=0, count=None, concurrency=6)` | `list[TransactionRecord]` | get_transactions / get_goods_transaction / get_goods_transaction_all |
|
||||
| `get_minutes` | `(symbols, *, date=None, sampling=False, concurrency=6)` | `list[MinuteBar]` | get_minute / get_minute_batch / get_chart_sampling / get_goods_chart_sampling / get_goods_tick_chart |
|
||||
| `get_snapshot` | `(symbols)` | `list[SymbolSnapshot]` | get_symbol_info |
|
||||
| `get_f10` | `(symbol, section=None, *, offset=0, length=65535)` | `list \| str` | get_company_info_category / get_company_info_content |
|
||||
| `get_file` | `(name)` | `bytes` | get_block_info / get_report_file / get_file_meta / download_file / get_financial_file_list / get_financial_records |
|
||||
| `get_boards` | `(kind="all", *, count=10000)` | `list[BoardInfo]` | get_board_list |
|
||||
| `get_board_of` | `(symbols)` | `list[BelongBoard]` | get_belong_board |
|
||||
|
||||
### 4.2 旧名升级(18,同名泛化为统一接口)
|
||||
|
||||
| 方法 | 签名 | 返回 |
|
||||
|------|------|------|
|
||||
| `get_quotes` | `(symbols, *, fields=None)` | `list[SecurityQuote]` |
|
||||
| `get_auction` | `(symbols)` | `list[AuctionItem]` |
|
||||
| `get_capital_flow` | `(symbols)` | `list[CapitalFlow]` |
|
||||
| `get_xdxr` | `(symbols)` | `list[XdxrRecord]` |
|
||||
| `get_finance` | `(symbols)` | `list[FinanceRecord]` |
|
||||
| `get_price_limits` | `(symbol, name=None, pre_close=None, listed_days=9999)` | `tuple[float\|None, float\|None]` |
|
||||
| `get_market_stat` | `()` | `MarketStat` |
|
||||
| `get_unusual` | `(market="all", *, start=0, count=600)` | `list[UnusualItem]` |
|
||||
| `get_server_info` | `()` | `ServerInfo` |
|
||||
| `get_kline_offset` | `(offset=0, count=1)` | `tuple[int, int]` |
|
||||
| `get_board_members` | `(board, count=100000, sort_type=0, *, fields=None)` | `list[SecurityQuote]` |
|
||||
| `get_board_summary` | `(board)` | `dict` |
|
||||
| `get_board_ranking` | `(kind="industry", *, top_n=20)` | `list[dict]` |
|
||||
| `get_board_change_ranking` | `(kind="industry", *, days=20, top_n=20)` | `list[dict]` |
|
||||
| `get_block_parsed` | `(filename)` | `list[TdxBlock]` |
|
||||
| `get_industry_map` | `()` | `dict[str, IndustryInfo]` |
|
||||
| `get_financial_file_infos` | `()` | `list[FinancialFileInfo]` |
|
||||
| `get_financial_records_parsed` | `(filename)` | `list[FinancialRecord]` |
|
||||
|
||||
> 另有生命周期方法 `start()` / `close()`,合计 30 个非别名公开方法。
|
||||
|
||||
---
|
||||
|
||||
## 五、废弃别名清单(33 个,1.0 移除)
|
||||
|
||||
别名运行时发出 `DeprecationWarning`;**仍接受 `(market, code)` 写法**(这是别名与统一接口的关键差异)。
|
||||
|
||||
| 目标统一接口 | 数量 | 废弃别名 |
|
||||
|--------------|------|----------|
|
||||
| `get_bars` | 4 | `get_kline`、`get_kline_since`、`get_kline_batch`、`get_index_kline` |
|
||||
| `get_indicators` | 1 | `get_stock_kline_with_indicators` |
|
||||
| `get_quotes` | 3 | `get_stock_quotes`、`get_goods_quotes`、`get_goods_quotes_list` |
|
||||
| `get_board_members` | 1 | `get_stock_quotes_list` |
|
||||
| `get_ticks` | 3 | `get_transactions`、`get_goods_transaction`、`get_goods_transaction_all` |
|
||||
| `get_minutes` | 3 | `get_minute`、`get_minute_batch`、`get_goods_tick_chart` |
|
||||
| `get_minutes(sampling=True)` | 2 | `get_chart_sampling`、`get_goods_chart_sampling` |
|
||||
| `get_universe` | 5 | `get_security_count`、`get_security_list`、`get_security_list_all`、`get_goods_list`、`get_goods_count` |
|
||||
| `get_f10` | 2 | `get_company_info_category`、`get_company_info_content` |
|
||||
| `get_file` | 4 | `get_block_info`、`get_report_file`、`get_file_meta`、`download_file` |
|
||||
| `get_financial_file_infos` | 1 | `get_financial_file_list` |
|
||||
| `get_financial_records_parsed` | 1 | `get_financial_records` |
|
||||
| `get_boards` | 1 | `get_board_list` |
|
||||
| `get_board_of` | 1 | `get_belong_board` |
|
||||
| `get_snapshot` | 1 | `get_symbol_info` |
|
||||
| **合计** | **33** | |
|
||||
|
||||
---
|
||||
|
||||
## 六、标的写法统一(字符串化)
|
||||
|
||||
统一接口**只接受字符串标的**,已移除所有 `(market, code)` 元组软兼容:
|
||||
|
||||
- 删除模块级 `_as_symbols()`;`get_quotes` / `get_xdxr` / `get_finance` / `get_auction` /
|
||||
`get_capital_flow` 去掉 `code` 位置参数,统一走 `parse_symbols()`。
|
||||
- `get_price_limits` 去掉 `(market, code, …)` 旧参形式检测。
|
||||
- `get_unusual` 去掉整数市场号检测。
|
||||
- 调用方(CLI、示例)同步改为字符串;`get_finance` / `get_capital_flow` 的返回类型改为 `list`。
|
||||
|
||||
!!! note "保留项"
|
||||
`get_boards(kind)` 仍接受数字板块类别(`0/1/2/3`)。这是**板类别选择器**、非标的,
|
||||
文档已明示「也接受数字」,旧别名 `get_board_list(board_type=0)` 依赖它。
|
||||
|
||||
---
|
||||
|
||||
## 七、验证
|
||||
|
||||
- `pytest`:**127 passed**(审查当时的快照;当前基线为 164 passed)。其中 `tests/test_all_interfaces.py` 为全接口回归:mock 连接逐个
|
||||
调用 63 个公开方法,并以「覆盖哨兵」断言无遗漏(7 条告警为别名测试的预期 `DeprecationWarning`)。
|
||||
- `scripts/check_docs.py`:`MUST_LIST_ALL` + `MUST_NOT_BE_STALE` 全部 `[OK]`。
|
||||
- 签名核对:统一接口签名与返回类型与本文一致;全仓无 `_as_symbols` 及元组式调用残留。
|
||||
@@ -4,18 +4,19 @@
|
||||
python examples/quickstart.py
|
||||
|
||||
说明:数据来自通达信公开行情服务器(无需账号,15 分钟延时)。
|
||||
标的一律用字符串(``sh600000`` / ``sz000001`` / ``hk00700`` / ``usAAPL`` / ``cffex:IFL0``)。
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
|
||||
from pytdxdata import KlinePeriod, Market, TdxData
|
||||
from pytdxdata import TdxData
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
# TdxData 为异步上下文管理器:进入时建连,退出时释放
|
||||
async with TdxData() as td:
|
||||
# 1) 批量五档报价(可一次传多只,超过 80 只自动切批)
|
||||
quotes = await td.get_quotes([(Market.SH, "600000"), (Market.SZ, "000001")])
|
||||
quotes = await td.get_quotes(["sh600000", "sz000001"])
|
||||
print("== 报价 ==")
|
||||
for q in quotes:
|
||||
print(
|
||||
@@ -24,7 +25,7 @@ async def main() -> None:
|
||||
)
|
||||
|
||||
# 2) 日K线(最近 10 根)
|
||||
bars = await td.get_kline(Market.SH, "600000", KlinePeriod.DAY, count=10)
|
||||
bars = await td.get_bars("sh600000", count=10)
|
||||
print("\n== 日K(最近 3 根)==")
|
||||
for b in bars[-3:]:
|
||||
print(
|
||||
@@ -33,7 +34,7 @@ async def main() -> None:
|
||||
)
|
||||
|
||||
# 3) 当日分时
|
||||
minutes = await td.get_minute(Market.SH, "600000")
|
||||
minutes = await td.get_minutes("sh600000")
|
||||
latest = minutes[-1].price if minutes else "-"
|
||||
print(f"\n== 分时 ==\n共 {len(minutes)} 个点,最新价={latest}")
|
||||
|
||||
|
||||
@@ -47,6 +47,7 @@ nav:
|
||||
- 安装与配置: getting-started.md
|
||||
- 场景示例: cookbook.md
|
||||
- 命令行工具: cli.md
|
||||
- DataFrame 输出: dataframe.md
|
||||
- API 参考:
|
||||
- 接口总览: api/index.md
|
||||
- TdxData 方法: api/tdxdata.md
|
||||
@@ -64,6 +65,7 @@ nav:
|
||||
- 其他:
|
||||
- 常见问题: faq.md
|
||||
- 更新日志: changelog.md
|
||||
- 接口重构审查: refactor-audit.md
|
||||
- 参与贡献: contributing.md
|
||||
|
||||
plugins:
|
||||
|
||||
+7
-1
@@ -1,6 +1,6 @@
|
||||
[project]
|
||||
name = "pytdxdata"
|
||||
version = "0.5.0"
|
||||
version = "0.7.0"
|
||||
description = "通达信(TongDaXin)行情数据客户端:动态连接池 + 双层TTL缓存 + 3连接并发分页,纯asyncio零运行时依赖,支持A股/港股/美股/期货/期权"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.11"
|
||||
@@ -31,6 +31,9 @@ Documentation = "https://openbot-coder.github.io/pytdxdata/"
|
||||
|
||||
[project.optional-dependencies]
|
||||
uvloop = ["uvloop>=0.21"]
|
||||
pandas = ["pandas>=2.0"]
|
||||
polars = ["polars>=1.0"]
|
||||
dataframe = ["pandas>=2.0", "polars>=1.0"]
|
||||
dev = ["pytest>=8.0", "pytest-asyncio>=0.24", "ruff>=0.6"]
|
||||
|
||||
[project.scripts]
|
||||
@@ -52,6 +55,9 @@ testpaths = ["tests"]
|
||||
dev = [
|
||||
"pytest>=9.1.1",
|
||||
"pytest-asyncio>=1.4.0",
|
||||
# DataFrame 后端的双实现都要在本地/CI 里被真实覆盖,不能只在 extras 里声明
|
||||
"pandas>=2.0",
|
||||
"polars>=1.0",
|
||||
]
|
||||
docs = [
|
||||
"mkdocs-material>=9.5",
|
||||
|
||||
@@ -32,12 +32,12 @@ PKG_DIR = ROOT / "src" / "pytdxdata"
|
||||
|
||||
# 必须**列全**所有公开方法的文档(漏写 = 漂移)
|
||||
MUST_LIST_ALL = [
|
||||
ROOT / "docs" / "api" / "index.md",
|
||||
ROOT / "docs" / "API_REFERENCE.md",
|
||||
]
|
||||
|
||||
# 只需**不提到已删除的方法**的文档:
|
||||
# 这些是入口 / 教程 / 精选清单,本来就不要求列全,但绝不能引用已不存在的方法
|
||||
# 这些是入口 / 教程 / 精选清单 / 接口总览(只列推荐的统一接口,旧方法不强制列出),
|
||||
# 但绝不能引用已不存在的方法
|
||||
MUST_NOT_BE_STALE = [
|
||||
ROOT / "SKILL.md",
|
||||
ROOT / "README.md",
|
||||
@@ -48,6 +48,8 @@ MUST_NOT_BE_STALE = [
|
||||
ROOT / "docs" / "faq.md",
|
||||
ROOT / "docs" / "cli.md",
|
||||
ROOT / "docs" / "ai.md",
|
||||
ROOT / "docs" / "api" / "index.md",
|
||||
ROOT / "docs" / "refactor-audit.md",
|
||||
]
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,108 @@
|
||||
"""可复现核查:指数成分逐笔汇总的口径是否与官方日线一致。
|
||||
|
||||
用法::
|
||||
|
||||
python scripts/verify_index_stats.py # 当日
|
||||
python scripts/verify_index_stats.py --date 20261021 # 历史某日
|
||||
python scripts/verify_index_stats.py --indices 000001 000300
|
||||
|
||||
两段校验:
|
||||
|
||||
1. **单股级**:``Σ(逐笔价 × 手 × 100)`` 对**当日日线 amount**。比值 1.0000~1.0002
|
||||
说明逐笔没漏条、价格与单位都对(``bj`` 也行)。
|
||||
2. **指数级**:成分股合计成交额对**该指数的日线 amount**。相等说明成分清单与
|
||||
指数的量能口径一致;明显偏小则是成分表滞后(新上市未纳入)。
|
||||
|
||||
⚠️ **深市 `399xxx` 的指数 amount 是「全板块」口径**,不是样本股:实测
|
||||
`sz399001` 5081.40 亿 = 深市 2908 只股票合计,`sz399006` 2568.60 亿 = 创业板
|
||||
全部股票合计。所以这两个指数的比值天然远离 1(约 0.43),**不是数据错**。
|
||||
|
||||
为什么不用快照做基准:快照只有「此刻」,历史日期对不上。日线 amount 两者都能给。
|
||||
|
||||
注意 ``bj``(北交所)逐笔**没有笔数字段**(``trade_count`` 恒 0),脚本会打出
|
||||
``有笔数`` 列便于识别。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import asyncio
|
||||
from datetime import date as _date
|
||||
|
||||
|
||||
def _ymd(bar) -> int:
|
||||
return bar.year * 10000 + bar.month * 100 + bar.day
|
||||
|
||||
|
||||
async def _daily_amount(td, symbol: str, ymd: int) -> float | None:
|
||||
from pytdxdata import KlinePeriod
|
||||
|
||||
bars = await td.get_bars(symbol, KlinePeriod.DAY, count=10)
|
||||
for b in bars:
|
||||
if _ymd(b) == ymd:
|
||||
return b.amount
|
||||
return None
|
||||
|
||||
|
||||
async def check_symbols(td, symbols: list[str], ymd: int) -> None:
|
||||
print(f"## 单股级({ymd})\n")
|
||||
print("| 标的 | Σ(p×手×100) / 日线amount | 笔数 |")
|
||||
print("|---|---|---|")
|
||||
for sym in symbols:
|
||||
try:
|
||||
ticks = await td.get_ticks(sym, date=ymd)
|
||||
ref = await _daily_amount(td, sym, ymd)
|
||||
except Exception as exc: # noqa: BLE001
|
||||
print(f"| {sym} | ERR {type(exc).__name__} | - |")
|
||||
continue
|
||||
if not ticks:
|
||||
print(f"| {sym} | 无逐笔(非交易日?) | - |")
|
||||
continue
|
||||
sa = sum(t.price * t.vol * 100 for t in ticks)
|
||||
tc = sum(t.trade_count for t in ticks)
|
||||
ratio = sa / ref if ref else 0.0
|
||||
print(f"| {sym} | {ratio:.4f} | {tc:,} |")
|
||||
print()
|
||||
|
||||
|
||||
async def check_indices(td, indices: list[str], ymd: int, day: int | None) -> None:
|
||||
print(f"## 指数级({ymd})\n")
|
||||
print("| 指数 | 成分 | 有成交 | 有笔数 | 成分合计 | 日线amount | 比值 | 耗时 |")
|
||||
print("|---|---|---|---|---|---|---|---|")
|
||||
for idx in indices:
|
||||
try:
|
||||
st = await td.get_index_trade_stats(idx, day)
|
||||
ref = await _daily_amount(td, st.index, ymd)
|
||||
except Exception as exc: # noqa: BLE001
|
||||
print(f"| {idx} | ERR {type(exc).__name__}: {exc} | - | - | - | - | - | - |")
|
||||
continue
|
||||
ratio = st.total_amount / ref if ref else 0.0
|
||||
print(f"| {st.index} | {st.member_count} | {st.active_members} | "
|
||||
f"{st.trade_count_members} | {st.total_amount / 1e8:,.2f} 亿 | "
|
||||
f"{(ref or 0) / 1e8:,.2f} 亿 | {ratio:.4f} | {st.elapsed:.1f}s |")
|
||||
print()
|
||||
print("提示:`399xxx` 的比值天然远离 1 —— 深市指数的 amount 字段是全板块口径。")
|
||||
print()
|
||||
|
||||
|
||||
async def main() -> None:
|
||||
ap = argparse.ArgumentParser(description=__doc__)
|
||||
ap.add_argument("--symbols", nargs="*",
|
||||
default=["sh600000", "sz000001", "sh601398"])
|
||||
ap.add_argument("--indices", nargs="*", default=["000016", "000300"])
|
||||
ap.add_argument("--date", default=None, help="YYYYMMDD(缺省=当日)")
|
||||
ap.add_argument("--skip-symbols", action="store_true")
|
||||
args = ap.parse_args()
|
||||
|
||||
from pytdxdata import TdxData
|
||||
|
||||
ymd = int(args.date) if args.date else int(_date.today().strftime("%Y%m%d"))
|
||||
day = int(args.date) if args.date else None
|
||||
async with TdxData(cache_dir=None, timeout=10.0) as td:
|
||||
if not args.skip_symbols:
|
||||
await check_symbols(td, args.symbols, ymd)
|
||||
if args.indices:
|
||||
await check_indices(td, args.indices, ymd, day)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
@@ -7,16 +7,18 @@ from .models import (
|
||||
Adjust,
|
||||
ExMarket,
|
||||
FinanceRecord,
|
||||
IndexTradeStats,
|
||||
KlinePeriod,
|
||||
Market,
|
||||
MinuteBar,
|
||||
MinuteTradeStat,
|
||||
SecurityBar,
|
||||
SecurityQuote,
|
||||
TransactionRecord,
|
||||
XdxrRecord,
|
||||
)
|
||||
|
||||
__version__ = "0.5.0"
|
||||
__version__ = "0.7.0"
|
||||
__all__ = [
|
||||
"TdxData",
|
||||
"Market",
|
||||
@@ -27,6 +29,8 @@ __all__ = [
|
||||
"SecurityQuote",
|
||||
"TransactionRecord",
|
||||
"MinuteBar",
|
||||
"MinuteTradeStat",
|
||||
"IndexTradeStats",
|
||||
"XdxrRecord",
|
||||
"FinanceRecord",
|
||||
"__version__",
|
||||
|
||||
+2284
-933
File diff suppressed because it is too large.
Load diff
Vendored
+8
-3
@@ -40,7 +40,12 @@ class CacheKey:
|
||||
)
|
||||
|
||||
@classmethod
|
||||
def quote_key(cls, stocks: Sequence[tuple[int, str]]) -> "CacheKey":
|
||||
"""报价键:对股票集规范化排序,顺序无关可命中。"""
|
||||
def quote_key(cls, stocks: Sequence[tuple[int, str]],
|
||||
tag: str = "") -> "CacheKey":
|
||||
"""报价键:对股票集规范化排序,顺序无关可命中。
|
||||
|
||||
``tag`` 区分取数口径(标准五档 / MAC·EX 字段位),不同口径不互相命中。
|
||||
"""
|
||||
norm = sorted(f"{m}:{c}" for m, c in stocks)
|
||||
return cls(category="quote", extra=",".join(norm))
|
||||
extra = ",".join(norm)
|
||||
return cls(category="quote", extra=f"{tag}#{extra}" if tag else extra)
|
||||
Vendored
+32
@@ -90,6 +90,38 @@ class CacheManager:
|
||||
# 确保清理锁(即使 fetcher 异常也清理,避免无限增长)
|
||||
self._locks.pop(ks, None)
|
||||
|
||||
async def get_or_fetch_obj(
|
||||
self,
|
||||
key: CacheKey,
|
||||
fetcher: Callable[[], Awaitable[Any]],
|
||||
*,
|
||||
pack: Callable[[Any], bytes],
|
||||
unpack: Callable[[bytes], Any],
|
||||
) -> Any:
|
||||
"""同 :meth:`get_or_fetch`,但**回源时直接返回新取到的对象**。
|
||||
|
||||
用于「一次取回上千条记录」的场景(如全市场报价):常规路径对冷数据
|
||||
会先 ``pack`` 再立刻 ``unpack`` 一次,纯属往返浪费 —— 5,200 条报价实测
|
||||
白烧约 90ms。这里命中缓存才解序列化,回源则把对象原样返回,只在写缓存时打包。
|
||||
"""
|
||||
ks = key.to_str()
|
||||
cached = await self.get(key)
|
||||
if cached is not None:
|
||||
return unpack(cached)
|
||||
|
||||
lock = self._locks.setdefault(ks, asyncio.Lock())
|
||||
try:
|
||||
async with lock:
|
||||
cached = await self.get(key)
|
||||
if cached is not None:
|
||||
return unpack(cached)
|
||||
async with self._semaphore:
|
||||
obj = await fetcher()
|
||||
await self.set(key, pack(obj))
|
||||
return obj
|
||||
finally:
|
||||
self._locks.pop(ks, None)
|
||||
|
||||
async def invalidate(self, category: str, *, market: int | None = None,
|
||||
code: str | None = None) -> int:
|
||||
"""按前缀批量失效,返回失效条数。"""
|
||||
|
||||
+130
-67
@@ -8,6 +8,7 @@
|
||||
tdx transactions sz000001 --count 50 # 逐笔成交
|
||||
tdx minute sz000001 # 分时(今日/历史--date)
|
||||
tdx index 000001 --count 30 # 指数K线
|
||||
tdx index-stats 000300 --top 10 # 指数成分成交汇总(每分钟成交量/笔数)
|
||||
tdx board list [--type 1] # 板块列表(0全部/1行业/2概念)
|
||||
tdx board members 881001 --count 20 # 板块成分
|
||||
tdx board ranking [--top 10] # 板块涨跌幅排行
|
||||
@@ -37,6 +38,7 @@ from .api import TdxData
|
||||
from dataclasses import asdict
|
||||
|
||||
from .models import Adjust, ExMarket, KlinePeriod, Market
|
||||
from .protocol.commands.quotes import price_decimal_digits
|
||||
|
||||
PERIODS = {
|
||||
"min1": KlinePeriod.MIN_1, "min5": KlinePeriod.MIN_5, "min15": KlinePeriod.MIN_15,
|
||||
@@ -106,99 +108,117 @@ def _pct(price: float, pre: float) -> str:
|
||||
return "%+.2f%%" % ((price / pre - 1) * 100)
|
||||
|
||||
|
||||
def _price_fmt(dp: int) -> str:
|
||||
"""按品种小数位生成右对齐格式串。
|
||||
|
||||
股票 2 位、**可转债/ETF 3 位**、债券/回购 4 位。固定写 ``%8.2f`` 会把
|
||||
可转债的 128.891 打成 128.89(少一位有效报价)。
|
||||
"""
|
||||
dp = max(0, min(int(dp or 2), 4))
|
||||
return "%%%d.%df" % (6 + dp, dp)
|
||||
|
||||
|
||||
# 可转债后缀(解码 4 位,但交易所报价步长 0.001 元,第 4 位恒为 0)
|
||||
_CB_PREFIX = {1: ("110", "111", "113", "118"), 0: ("123", "127", "128")}
|
||||
|
||||
|
||||
def _display_dp(market: int, code: str) -> int:
|
||||
"""报价**显示**小数位。
|
||||
|
||||
注意 ``price_decimal_digits`` 给的是**解码位数**(原始整数 / 10**dp),
|
||||
可转债是 4 位;但真实报价是 0.001 元的步长,第 4 位恒为 0,
|
||||
按 4 位打印只会多一个 ``0``(``128.8920``)。
|
||||
"""
|
||||
dp = price_decimal_digits(market, code)
|
||||
if str(code).startswith(_CB_PREFIX.get(int(market), ())):
|
||||
return 3
|
||||
return dp
|
||||
|
||||
|
||||
def _dp_of(sym) -> int:
|
||||
"""Symbol -> 价格小数位(非 A 股/指数一律 2 位)。"""
|
||||
if getattr(sym, "kind", "") == "cn":
|
||||
return _display_dp(sym.market, sym.code)
|
||||
return 2
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ #
|
||||
# 各子命令
|
||||
# ------------------------------------------------------------------ #
|
||||
|
||||
async def cmd_quotes(td: TdxData, symbols: list[str]) -> None:
|
||||
cn, ex = [], []
|
||||
for sym in symbols:
|
||||
s = parse_symbol(sym)
|
||||
if s.kind == "cn":
|
||||
cn.append((Market(s.market), s.code))
|
||||
else:
|
||||
ex.append((s.market, s.code))
|
||||
for q in await td.get_quotes(cn):
|
||||
_print_cn_quote(q)
|
||||
if ex:
|
||||
from .protocol.commands.ex_proto import GetExInstrumentQuoteCmd
|
||||
for market, code in ex:
|
||||
q = await td._execute_ex(GetExInstrumentQuoteCmd(market, code))
|
||||
_print_ex_quote(market, code, q)
|
||||
quotes = await td.get_quotes(symbols)
|
||||
for q in quotes:
|
||||
_print_quote(q)
|
||||
|
||||
|
||||
def _print_cn_quote(q) -> None:
|
||||
def _print_quote(q) -> None:
|
||||
pct = (q.price / q.pre_close - 1) * 100 if q.pre_close else 0
|
||||
mkt = {31: "港股", 74: "美股"}.get(q.market, "")
|
||||
if q.market in (0, 1):
|
||||
dp = _display_dp(q.market, q.code)
|
||||
else:
|
||||
dp = getattr(q, "decimal_point", 2) or 2
|
||||
print("=" * 58)
|
||||
print(" %s %s" % (q.code, q.server_time or datetime.now().strftime("%H:%M:%S")))
|
||||
print(" 最新 %-10.2f 涨跌 %+8.2f (%+.2f%%)" % (q.price, q.price - q.pre_close, pct))
|
||||
print(" 今开 %-10.2f 最高 %.2f 最低 %.2f" % (q.open, q.high, q.low))
|
||||
print(" 昨收 %-10.2f 成交量 %s 成交额 %s" % (q.pre_close, _fmt_num(q.vol), _fmt_num(q.amount)))
|
||||
label = "%s %s" % (mkt, q.code) if mkt else "%s %s" % (q.code, q.server_time or datetime.now().strftime("%H:%M:%S"))
|
||||
print(" %s" % label)
|
||||
print(" 最新 %-10.*f 涨跌 %+8.*f (%+.2f%%)" % (dp, q.price, dp, q.price - q.pre_close, pct))
|
||||
print(" 今开 %-10.*f 最高 %.*f 最低 %.*f" % (dp, q.open, dp, q.high, dp, q.low))
|
||||
print(" 昨收 %-10.*f 成交量 %s 成交额 %s" % (dp, q.pre_close, _fmt_num(q.vol), _fmt_num(q.amount)))
|
||||
if q.bid and q.ask:
|
||||
print(" " + "-" * 56)
|
||||
pf = _price_fmt(dp)
|
||||
for i in range(max(len(q.bid), len(q.ask))):
|
||||
b = q.bid[i] if i < len(q.bid) else (0, 0)
|
||||
a = q.ask[i] if i < len(q.ask) else (0, 0)
|
||||
print(" 买%-1d %8.2f %10s | 卖%-1d %8.2f %10s" % (
|
||||
print((" 买%-1d " + pf + " %10s | 卖%-1d " + pf + " %10s") % (
|
||||
i + 1, b[0], _fmt_num(b[1]), i + 1, a[0], _fmt_num(a[1])))
|
||||
print()
|
||||
|
||||
|
||||
def _print_ex_quote(market: int, code: str, q: dict | None) -> None:
|
||||
if not q:
|
||||
print(" %s: 无报价数据" % code)
|
||||
return
|
||||
price, pre = q.get("price") or 0, q.get("pre_close") or 0
|
||||
mkt = {31: "港股", 74: "美股"}.get(market, "扩展")
|
||||
print("=" * 58)
|
||||
print(" %s %s" % (mkt, code))
|
||||
print(" 最新 %-10.2f 涨跌 %+8.2f (%s)" % (price, price - pre, _pct(price, pre)))
|
||||
print(" 今开 %-10.2f 最高 %.2f 最低 %.2f" % (
|
||||
q.get("open") or 0, q.get("high") or 0, q.get("low") or 0))
|
||||
print(" 昨收 %-10.2f 总量 %s" % (pre, _fmt_num(q.get("zongliang") or 0)))
|
||||
print()
|
||||
|
||||
|
||||
async def cmd_kline(td: TdxData, sym: str, period: KlinePeriod, count: int,
|
||||
start: int, adjust: Adjust) -> None:
|
||||
s = parse_symbol(sym)
|
||||
if s.kind == "cn":
|
||||
bars = await td.get_kline(Market(s.market), s.code, period, start=start,
|
||||
count=count, adjust=adjust)
|
||||
elif s.kind == "index":
|
||||
bars = await td.get_index_kline(Market(s.market), s.code, period, start=start, count=count)
|
||||
else:
|
||||
bars = await td.get_kline(ExMarket(s.market), s.code, period, start=start, count=count)
|
||||
pf = _price_fmt(_dp_of(s))
|
||||
bars = await td.get_bars(sym, period, start=start, count=count, adjust=adjust)
|
||||
print("%s K线 (%s, %d根):" % (sym, period.name, len(bars)))
|
||||
print(" %-12s %8s %8s %8s %8s %12s" % ("日期", "开盘", "最高", "最低", "收盘", "成交量"))
|
||||
for b in bars:
|
||||
dt = "%04d-%02d-%02d" % (b.year, b.month, b.day)
|
||||
if b.hour:
|
||||
dt += " %02d:%02d" % (b.hour, b.minute)
|
||||
print(" %-12s %8.2f %8.2f %8.2f %8.2f %12s" % (
|
||||
print((" %-12s " + " ".join([pf] * 4) + " %12s") % (
|
||||
dt, b.open, b.high, b.low, b.close, _fmt_num(b.vol)))
|
||||
|
||||
|
||||
async def cmd_transactions(td: TdxData, sym: str, count: int, date: int | None) -> None:
|
||||
s = parse_symbol(sym)
|
||||
tx = await td.get_transactions(Market(s.market), s.code, date=date, count=count)
|
||||
pf = _price_fmt(_dp_of(s))
|
||||
tx = await td.get_ticks(sym, date=date, count=count)
|
||||
print("%s 逐笔成交 (%d条):" % (sym, len(tx)))
|
||||
print(" 时间 价格 成交量 方向")
|
||||
for t in tx:
|
||||
bs = "买" if getattr(t, "bs_flag", 0) == 1 else "卖" if getattr(t, "bs_flag", 0) == 2 else "-"
|
||||
tc = getattr(t, "trade_count", 0)
|
||||
print(" %-10s %8.2f %10s %s%s" % (t.time, t.price, _fmt_num(t.vol), bs,
|
||||
" (%d笔)" % tc if tc else ""))
|
||||
if not getattr(t, "trade_count_available", True):
|
||||
tc = " (-)" # 指数/ETF/LOF:协议不提供笔数
|
||||
elif getattr(t, "trade_count", 0):
|
||||
tc = " (%d笔)" % t.trade_count
|
||||
else:
|
||||
tc = ""
|
||||
print((" %-10s " + pf + " %10s %s%s") % (
|
||||
t.time, t.price, _fmt_num(t.vol), bs, tc))
|
||||
|
||||
|
||||
async def cmd_minute(td: TdxData, sym: str, date: int | None) -> None:
|
||||
s = parse_symbol(sym)
|
||||
bars = await td.get_minute(Market(s.market), s.code, date=date)
|
||||
pf = _price_fmt(_dp_of(s))
|
||||
bars = await td.get_minutes(sym, date=date)
|
||||
print("%s 分时 (%d条):" % (sym, len(bars)))
|
||||
print(" 时间 价格 均价 成交量")
|
||||
for i, b in enumerate(bars):
|
||||
print(" %02d:%02d %8.2f %8.2f %10s" % (
|
||||
i // 60, i % 60, b.price, b.avg_price, _fmt_num(b.vol)))
|
||||
for b in bars:
|
||||
# 用 bar 自带时间:早期按序号推算会打成 00:00~03:59
|
||||
print((" %02d:%02d " + pf + " " + pf + " %10s") % (
|
||||
b.hour, b.minute, b.price, b.avg_price, _fmt_num(b.vol)))
|
||||
|
||||
|
||||
async def cmd_board(td: TdxData, args) -> None:
|
||||
@@ -235,6 +255,43 @@ async def cmd_board(td: TdxData, args) -> None:
|
||||
print(" %-12s %+8.2f%%" % (r.get("name", ""), r.get("change_pct", 0)))
|
||||
|
||||
|
||||
async def cmd_index_stats(td: TdxData, index: str, date: int | None,
|
||||
show_all: bool, top: int, concurrency: int) -> None:
|
||||
"""指数成分成交汇总:每分钟成交量 / 笔数(笔数=成分股逐笔累加)。"""
|
||||
total = 0
|
||||
|
||||
def on_progress(done: int, all_: int) -> None:
|
||||
nonlocal total
|
||||
total = all_
|
||||
print("\r 拉取成分逐笔 %d/%d ..." % (done, all_), end="", file=sys.stderr,
|
||||
flush=True)
|
||||
|
||||
st = await td.get_index_trade_stats(
|
||||
index, date, concurrency=concurrency, on_progress=on_progress)
|
||||
if total:
|
||||
print("\r" + " " * 40 + "\r", end="", file=sys.stderr, flush=True)
|
||||
|
||||
print("%s 成分成交汇总" % st.index)
|
||||
print(" 成分 %d 只 / 有成交 %d 只 / 有笔数 %d 只%s" % (
|
||||
st.member_count, st.active_members, st.trade_count_members,
|
||||
"" if st.trade_count_available else " ★该指数无笔数口径"))
|
||||
print(" 成交量 %s 手 成交笔数 %s 笔 成交额 %.2f 亿 耗时 %.1fs" % (
|
||||
_fmt_num(st.total_volume),
|
||||
_fmt_num(st.total_trade_count) if st.trade_count_available else "-",
|
||||
st.total_amount / 1e8, st.elapsed))
|
||||
if st.failed_members:
|
||||
print(" (%d 只成分逐笔拉取失败,已跳过)" % st.failed_members)
|
||||
print()
|
||||
print(" 时间 累计成交量 累计笔数")
|
||||
rows = st.cumulative()
|
||||
if not show_all:
|
||||
rows = rows[-top:]
|
||||
for m in rows:
|
||||
print(" %-6s %12s %12s" % (
|
||||
m.time, _fmt_num(m.volume),
|
||||
_fmt_num(m.trade_count) if st.trade_count_available else "-"))
|
||||
|
||||
|
||||
async def cmd_info(td: TdxData, sym: str) -> None:
|
||||
s = parse_symbol(sym)
|
||||
info = await td.get_symbol_info(Market(s.market), s.code)
|
||||
@@ -256,8 +313,7 @@ async def cmd_list(td: TdxData, market: str, count: int) -> None:
|
||||
|
||||
|
||||
async def cmd_auction(td: TdxData, sym: str) -> None:
|
||||
s = parse_symbol(sym)
|
||||
items = await td.get_auction(Market(s.market), s.code)
|
||||
items = await td.get_auction(sym)
|
||||
print("%s 集合竞价 (%d条):" % (sym, len(items)))
|
||||
for a in items:
|
||||
print(" %s 价格=%.3f 量=%d %s" % (
|
||||
@@ -266,7 +322,7 @@ async def cmd_auction(td: TdxData, sym: str) -> None:
|
||||
|
||||
|
||||
async def cmd_unusual(td: TdxData, market: str, count: int) -> None:
|
||||
items = await td.get_unusual(MARKET_NAMES[market], count=count)
|
||||
items = await td.get_unusual(market, count=count)
|
||||
print("市场异动 (%d条):" % len(items))
|
||||
for u in items:
|
||||
print(" %s %s %s" % (getattr(u, "code", ""), getattr(u, "name", ""),
|
||||
@@ -274,8 +330,7 @@ async def cmd_unusual(td: TdxData, market: str, count: int) -> None:
|
||||
|
||||
|
||||
async def cmd_xdxr(td: TdxData, sym: str) -> None:
|
||||
s = parse_symbol(sym)
|
||||
items = await td.get_xdxr(Market(s.market), s.code)
|
||||
items = await td.get_xdxr(sym)
|
||||
print("%s 除权除息 (%d条):" % (sym, len(items)))
|
||||
for x in items[:20]:
|
||||
print(" %s 送%g 配%g 派%g" % (
|
||||
@@ -284,23 +339,20 @@ async def cmd_xdxr(td: TdxData, sym: str) -> None:
|
||||
|
||||
|
||||
async def cmd_finance(td: TdxData, sym: str) -> None:
|
||||
s = parse_symbol(sym)
|
||||
f = await td.get_finance(Market(s.market), s.code)
|
||||
if not f:
|
||||
fins = await td.get_finance(sym)
|
||||
if not fins:
|
||||
print("无财务数据")
|
||||
return
|
||||
for k, v in asdict(f).items():
|
||||
for k, v in asdict(fins[0]).items():
|
||||
print(" %-24s %s" % (k, v))
|
||||
|
||||
|
||||
async def cmd_flow(td: TdxData, sym: str) -> None:
|
||||
s = parse_symbol(sym)
|
||||
f = await td.get_capital_flow(Market(s.market), s.code)
|
||||
if isinstance(f, dict):
|
||||
items = f.items()
|
||||
else:
|
||||
items = asdict(f).items()
|
||||
for k, v in items:
|
||||
flows = await td.get_capital_flow(sym)
|
||||
if not flows:
|
||||
print("无资金流向数据")
|
||||
return
|
||||
for k, v in asdict(flows[0]).items():
|
||||
print(" %-20s %s" % (k, v))
|
||||
|
||||
|
||||
@@ -426,6 +478,17 @@ def main() -> None:
|
||||
ix.set_defaults(func=lambda td, a: cmd_kline(
|
||||
td, a.code, KlinePeriod.DAY, a.count, 0, Adjust.NONE))
|
||||
|
||||
ist = add("index-stats", "指数成分成交汇总(每分钟成交量/笔数)")
|
||||
ist.add_argument("index", help="指数代码,如 000300 / 000001 / sh000016")
|
||||
ist.add_argument("--date", type=str, default=None, help="YYYYMMDD,缺省=当日")
|
||||
ist.add_argument("--top", type=int, default=10, help="只显示最后 N 分钟(默认10)")
|
||||
ist.add_argument("--all", dest="show_all", action="store_true",
|
||||
help="显示全部分钟")
|
||||
ist.add_argument("--concurrency", type=int, default=24, help="逐笔并发数")
|
||||
ist.set_defaults(func=lambda td, a: cmd_index_stats(
|
||||
td, a.index, int(a.date) if a.date else None,
|
||||
a.show_all, a.top, a.concurrency))
|
||||
|
||||
b = add("board", "板块")
|
||||
b.add_argument("action", choices=["list", "members", "ranking", "change"])
|
||||
b.add_argument("symbol", nargs="?", default="")
|
||||
|
||||
@@ -17,13 +17,19 @@
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import io
|
||||
import struct
|
||||
import zipfile
|
||||
|
||||
from ..models import FinancialFileInfo, FinancialRecord
|
||||
|
||||
_HEADER_FMT = "<1hI1H3L"
|
||||
_INDEX_FMT = "<6sBL"
|
||||
|
||||
_HEADER_SIZE = struct.calcsize(_HEADER_FMT)
|
||||
_INDEX_SIZE = struct.calcsize(_INDEX_FMT)
|
||||
_ZIP_MAGIC = b"PK\x03\x04"
|
||||
|
||||
|
||||
def parse_financial_file_list(data: bytes) -> list[FinancialFileInfo]:
|
||||
"""解析 tdxfin/gpcw.txt 内容(每行 ``filename,md5hash,filesize``)。"""
|
||||
@@ -46,8 +52,58 @@ def parse_financial_file_list(data: bytes) -> list[FinancialFileInfo]:
|
||||
return out
|
||||
|
||||
|
||||
def looks_like_gpcw_dat(data: bytes) -> bool:
|
||||
"""粗略判定字节流是否为 gpcw ``.dat``(校验 20 字节 header 的自洽性)。
|
||||
|
||||
用于把「真的取到了财报数据」与「服务器对不存在的文件名回的 0/1 字节空壳」区分开:
|
||||
空壳要么长度不足、要么 header 全零,一律判否。
|
||||
|
||||
Args:
|
||||
data: 待判定的字节流。
|
||||
|
||||
Returns:
|
||||
可疑时返回 True。
|
||||
"""
|
||||
if len(data) < _HEADER_SIZE:
|
||||
return False
|
||||
_unknown, _date, max_count, _u1, report_size, _u2 = struct.unpack(
|
||||
_HEADER_FMT, data[:_HEADER_SIZE]
|
||||
)
|
||||
if max_count <= 0 or report_size <= 0 or report_size % 4 != 0:
|
||||
return False
|
||||
# 索引区必须放得下 max_count 条 11 字节条目
|
||||
return _HEADER_SIZE + max_count * _INDEX_SIZE <= len(data)
|
||||
|
||||
|
||||
def extract_gpcw_dat(data: bytes) -> bytes:
|
||||
"""从「zip 包」或「未压缩 dat」中取出 ``.dat`` 字节;无法识别时返回 ``b""``。
|
||||
|
||||
服务器**实际只提供未压缩的** ``tdxfin/gpcwYYYYMMDD.dat``,而 ``tdxfin/gpcw.txt``
|
||||
索引里写的是 ``gpcwYYYYMMDD.zip``,因此两种形态都要能走通。
|
||||
|
||||
Args:
|
||||
data: zip 包字节或未压缩 ``.dat`` 字节。
|
||||
|
||||
Returns:
|
||||
内层 / 原始的 ``.dat`` 字节;不是有效财报数据时返回 ``b""``。
|
||||
"""
|
||||
if not data:
|
||||
return b""
|
||||
if data.startswith(_ZIP_MAGIC):
|
||||
try:
|
||||
with zipfile.ZipFile(io.BytesIO(data)) as zf:
|
||||
names = [n for n in zf.namelist() if n.lower().endswith(".dat")]
|
||||
if not names:
|
||||
return b""
|
||||
inner = zf.read(names[0])
|
||||
except (zipfile.BadZipFile, OSError, KeyError):
|
||||
return b""
|
||||
return inner if looks_like_gpcw_dat(inner) else b""
|
||||
return data if looks_like_gpcw_dat(data) else b""
|
||||
|
||||
|
||||
def parse_financial_dat(data: bytes, report_date: int = 0) -> list[FinancialRecord]:
|
||||
"""解析 gpcw*.zip 内的 .dat 二进制。
|
||||
"""解析 gpcw ``.dat`` 二进制(zip 内的或服务器直接给的未压缩形式都行)。
|
||||
|
||||
Args:
|
||||
data: .dat 文件完整字节。
|
||||
@@ -56,7 +112,7 @@ def parse_financial_dat(data: bytes, report_date: int = 0) -> list[FinancialReco
|
||||
Returns:
|
||||
FinancialRecord 列表;数据不足或字段非法时返回空列表。
|
||||
"""
|
||||
header_size = struct.calcsize(_HEADER_FMT)
|
||||
header_size = _HEADER_SIZE
|
||||
if len(data) < header_size:
|
||||
return []
|
||||
|
||||
@@ -72,7 +128,7 @@ def parse_financial_dat(data: bytes, report_date: int = 0) -> list[FinancialReco
|
||||
if num_fields <= 0:
|
||||
return []
|
||||
|
||||
index_size = struct.calcsize(_INDEX_FMT)
|
||||
index_size = _INDEX_SIZE
|
||||
report_fmt = f"<{num_fields}f"
|
||||
report_pack_size = struct.calcsize(report_fmt)
|
||||
|
||||
|
||||
+191
-13
@@ -1,12 +1,28 @@
|
||||
"""服务器清单与默认配置
|
||||
|
||||
服务器清单存于包内 servers.json,由 server_probe.py 导入时自动探测可用性与延时。
|
||||
config.py 加载时只采用 status="ok" 的服务器;环境变量可覆盖;代码内嵌清单为兜底。
|
||||
服务器清单存于包内 ``servers.json`` —— 它是**随包分发的基线清单,运行期只读**。
|
||||
每日首次启动的后台测速结果写入**用户级缓存目录**,运行期优先采用当日实测结果。
|
||||
|
||||
主机选取优先级(每个协议独立)::
|
||||
|
||||
环境变量 PYPYTDX_DATA_*_HOSTS > 当日测速缓存 > 包内清单 fast_pool
|
||||
> 包内清单 ok > 代码内嵌常量
|
||||
|
||||
相关环境变量:
|
||||
|
||||
- ``PYPYTDX_DATA_CACHE_DIR`` 覆盖用户级缓存目录(默认见 :func:`probe_cache_dir`)
|
||||
- ``PYPYTDX_DATA_DISABLE_PROBE`` 置真值时既不读当日缓存也不测速(CI / 测试用,保证确定性)
|
||||
|
||||
为什么不把测速结果写回包内 ``servers.json``:安装目录可能是只读的(容器 / 系统级
|
||||
site-packages / pipx),且多用户共用一份安装时彼此覆盖,运行期改写分发数据本身也不妥。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import ipaddress
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
from datetime import date
|
||||
from pathlib import Path
|
||||
|
||||
DEFAULT_PORT = 7709
|
||||
@@ -14,19 +30,30 @@ DEFAULT_TIMEOUT = 10.0
|
||||
|
||||
_SERVERS_FILE = Path(__file__).resolve().parent / "servers.json" # 包内清单(随包分发)
|
||||
|
||||
# 当日测速缓存(结构变更时递增,旧版本缓存自动失效)
|
||||
PROBE_CACHE_VERSION = 1
|
||||
_PROBE_CACHE_NAME = "hosts-today.json"
|
||||
_DECIMAL_CACHE_NAME = "decimal-points.json"
|
||||
|
||||
|
||||
# ---------- 包内清单(只读基线) ----------
|
||||
def _read_servers_json() -> dict:
|
||||
"""读取包内清单;文件缺失 / 内容损坏 / 顶层非对象 → 空 dict。"""
|
||||
try:
|
||||
cfg = json.loads(_SERVERS_FILE.read_text(encoding="utf-8"))
|
||||
except (OSError, ValueError):
|
||||
return {}
|
||||
return cfg if isinstance(cfg, dict) else {}
|
||||
|
||||
|
||||
def _load_ok_hosts(protocol: str, *, fast_only: bool = False) -> list[str]:
|
||||
"""从 servers.json 加载 status=ok 且支持指定协议的服务器。
|
||||
|
||||
fast_only=True 时只返回当日测速入池(fast_pool)的服务器(排名前N或延迟<阈值),
|
||||
fast_only=True 时只返回测速入池(fast_pool)的服务器(排名前N或延迟<阈值),
|
||||
用于动态连接池优先使用优质节点。
|
||||
"""
|
||||
try:
|
||||
cfg = json.loads(_SERVERS_FILE.read_text(encoding="utf-8"))
|
||||
except (OSError, ValueError):
|
||||
return []
|
||||
out = []
|
||||
for s in cfg.get("servers", []):
|
||||
for s in _read_servers_json().get("servers", []):
|
||||
if s.get("status") != "ok":
|
||||
continue
|
||||
if protocol not in s.get("protocols", []):
|
||||
@@ -38,13 +65,102 @@ def _load_ok_hosts(protocol: str, *, fast_only: bool = False) -> list[str]:
|
||||
|
||||
|
||||
def _load_ex_handshakes() -> dict[str, str]:
|
||||
try:
|
||||
cfg = json.loads(_SERVERS_FILE.read_text(encoding="utf-8"))
|
||||
except (OSError, ValueError):
|
||||
return {}
|
||||
return {s["ip"]: s.get("handshake", "setup") for s in cfg.get("servers", [])
|
||||
"""包内清单里 EX 服务器的握手模式。"""
|
||||
return {s["ip"]: s.get("handshake", "setup")
|
||||
for s in _read_servers_json().get("servers", [])
|
||||
if "ex" in s.get("protocols", []) and s.get("status") == "ok"}
|
||||
|
||||
|
||||
# ---------- 用户级当日测速缓存 ----------
|
||||
def probe_cache_dir() -> Path:
|
||||
"""用户级缓存目录(保证可写,绝不指向安装目录)。
|
||||
|
||||
``PYPYTDX_DATA_CACHE_DIR`` 可覆盖(测试指向临时目录)。按平台约定:
|
||||
|
||||
- Windows: ``%LOCALAPPDATA%\\pytdxdata``
|
||||
- macOS: ``~/Library/Caches/pytdxdata``
|
||||
- 其他: ``$XDG_CACHE_HOME/pytdxdata`` 或 ``~/.cache/pytdxdata``
|
||||
"""
|
||||
env = (os.environ.get("PYPYTDX_DATA_CACHE_DIR") or "").strip()
|
||||
if env:
|
||||
return Path(env).expanduser()
|
||||
if os.name == "nt":
|
||||
base = os.environ.get("LOCALAPPDATA") or os.environ.get("APPDATA")
|
||||
return (Path(base) if base else Path.home() / "AppData" / "Local") / "pytdxdata"
|
||||
if sys.platform == "darwin":
|
||||
return Path.home() / "Library" / "Caches" / "pytdxdata"
|
||||
base = os.environ.get("XDG_CACHE_HOME")
|
||||
return (Path(base) if base else Path.home() / ".cache") / "pytdxdata"
|
||||
|
||||
|
||||
def probe_cache_file() -> Path:
|
||||
"""当日测速缓存文件路径(每次调用重新解析,便于测试改环境变量后立即生效)。"""
|
||||
return probe_cache_dir() / _PROBE_CACHE_NAME
|
||||
|
||||
|
||||
def decimal_cache_file() -> Path:
|
||||
"""权威价格小数位表缓存路径(每次调用重新解析,便于测试改环境变量后生效)。"""
|
||||
return probe_cache_dir() / _DECIMAL_CACHE_NAME
|
||||
|
||||
|
||||
def decimal_sync_disabled() -> bool:
|
||||
"""是否已通过 ``PYPYTDX_DATA_DISABLE_DECIMAL_SYNC`` 关闭小数位后台同步。"""
|
||||
return os.environ.get("PYPYTDX_DATA_DISABLE_DECIMAL_SYNC", "").strip().lower() in {
|
||||
"1", "true", "yes", "on",
|
||||
}
|
||||
|
||||
|
||||
def probe_disabled() -> bool:
|
||||
"""是否已通过 ``PYPYTDX_DATA_DISABLE_PROBE`` 关闭当日测速缓存。"""
|
||||
return os.environ.get("PYPYTDX_DATA_DISABLE_PROBE", "").strip().lower() in {
|
||||
"1", "true", "yes", "on",
|
||||
}
|
||||
|
||||
|
||||
def load_probe_cache(today: str | None = None,
|
||||
path: str | Path | None = None) -> dict | None:
|
||||
"""读取**当日**测速缓存;缺失 / 损坏 / 非今日 / 版本不符 → ``None``。
|
||||
|
||||
返回非 None 即表示"今天已经测过速了",调用方应直接复用而不再刷新。
|
||||
``path`` 可显式指定缓存文件(默认 :func:`probe_cache_file`),便于注入与测试。
|
||||
"""
|
||||
if probe_disabled():
|
||||
return None
|
||||
target = Path(path) if path is not None else probe_cache_file()
|
||||
try:
|
||||
cfg = json.loads(target.read_text(encoding="utf-8"))
|
||||
except (OSError, ValueError):
|
||||
return None
|
||||
if not isinstance(cfg, dict) or cfg.get("version") != PROBE_CACHE_VERSION:
|
||||
return None
|
||||
if str(cfg.get("date", "")) != (today or date.today().isoformat()):
|
||||
return None
|
||||
results = cfg.get("results")
|
||||
if not isinstance(results, dict) or not results:
|
||||
return None
|
||||
return cfg
|
||||
|
||||
|
||||
def _load_cached_hosts(protocol: str) -> list[str]:
|
||||
"""当日测速缓存中该协议**实测可达**的主机,按延迟升序(越快越靠前)。"""
|
||||
cfg = load_probe_cache()
|
||||
if cfg is None:
|
||||
return []
|
||||
rows: list[tuple[float, str]] = []
|
||||
for ip, per_proto in cfg["results"].items():
|
||||
if not isinstance(per_proto, dict):
|
||||
continue
|
||||
lat = per_proto.get(protocol)
|
||||
if isinstance(lat, bool) or not isinstance(lat, (int, float)):
|
||||
continue
|
||||
if lat < 0:
|
||||
continue
|
||||
rows.append((float(lat), str(ip)))
|
||||
rows.sort(key=lambda kv: (kv[0], kv[1]))
|
||||
return [ip for _, ip in rows]
|
||||
|
||||
|
||||
# ---------- 代码内嵌兜底清单 ----------
|
||||
# 标准行情服务器 —— 真实K线探测验证可用(2026-08-12, GetSecurityBarsCmd 600000 日K)
|
||||
# 27台可用 = pytdxwrapper券商服务器(国泰君安/中信短线宝/华林/安信等) + 通达信官方主站
|
||||
# 来源①: pytdxwrapper config.py TDX_HOSTS(券商+官方, 51台探测27可用)
|
||||
@@ -91,10 +207,66 @@ EX_HANDSHAKES: dict[str, str] = {
|
||||
}
|
||||
|
||||
|
||||
# ---------- 测速候选枚举 ----------
|
||||
def is_valid_host(host: object) -> bool:
|
||||
"""是否是可用作服务器地址的合法主机(IPv4/IPv6 字面量,或含点的域名)。
|
||||
|
||||
用于挡住测试 mock 混进来的伪主机名(如 ``"s1"``)被写进缓存 / 清单。
|
||||
"""
|
||||
if not isinstance(host, str):
|
||||
return False
|
||||
h = host.strip()
|
||||
if not h or len(h) > 253 or any(c.isspace() for c in h):
|
||||
return False
|
||||
try:
|
||||
ipaddress.ip_address(h)
|
||||
return True
|
||||
except ValueError:
|
||||
pass
|
||||
return "." in h and all(
|
||||
part and part[0] != "-" and part[-1] != "-" and part.replace("-", "").isalnum()
|
||||
for part in h.split(".")
|
||||
)
|
||||
|
||||
|
||||
def all_probe_candidates() -> dict[str, list[str]]:
|
||||
"""[测速用] 返回 ``{协议: [候选主机]}`` 全量清单(已校验、去重、保序)。
|
||||
|
||||
来源 = 包内清单申报的协议 ∪ 代码内嵌常量 ∪ 环境变量覆盖。
|
||||
|
||||
与 ``get_*_hosts()`` 的关键区别:这里**不做 status / 延迟过滤**。
|
||||
测速必须能看到全部候选,否则"上次慢、这次快"的节点永远捞不回来,
|
||||
形成自指循环(只重测已在池里的少数节点,其余记录一直过期)。
|
||||
"""
|
||||
out: dict[str, list[str]] = {"standard": [], "mac": [], "ex": []}
|
||||
for s in _read_servers_json().get("servers", []):
|
||||
ip = s.get("ip")
|
||||
for protocol in s.get("protocols", []):
|
||||
if protocol in out:
|
||||
out[protocol].append(ip)
|
||||
out["standard"] += STANDARD_HOSTS
|
||||
out["mac"] += MAC_HOSTS
|
||||
out["ex"] += EX_HOSTS
|
||||
for protocol, var in (("standard", "PYPYTDX_DATA_STANDARD_HOSTS"),
|
||||
("mac", "PYPYTDX_DATA_MAC_HOSTS"),
|
||||
("ex", "PYPYTDX_DATA_EX_HOSTS")):
|
||||
env = os.environ.get(var)
|
||||
if env:
|
||||
out[protocol] += env.split(",")
|
||||
return {
|
||||
protocol: [h for h in dict.fromkeys(hosts) if is_valid_host(h)]
|
||||
for protocol, hosts in out.items()
|
||||
}
|
||||
|
||||
|
||||
# ---------- 对外取主机 ----------
|
||||
def get_standard_hosts() -> list[str]:
|
||||
env = os.environ.get("PYPYTDX_DATA_STANDARD_HOSTS")
|
||||
if env:
|
||||
return [h.strip() for h in env.split(",") if h.strip()]
|
||||
cached = _load_cached_hosts("standard")
|
||||
if cached:
|
||||
return cached
|
||||
fast = _load_ok_hosts("standard", fast_only=True)
|
||||
if fast:
|
||||
return fast
|
||||
@@ -106,6 +278,9 @@ def get_mac_hosts() -> list[str]:
|
||||
env = os.environ.get("PYPYTDX_DATA_MAC_HOSTS")
|
||||
if env:
|
||||
return [h.strip() for h in env.split(",") if h.strip()]
|
||||
cached = _load_cached_hosts("mac")
|
||||
if cached:
|
||||
return cached
|
||||
fast = _load_ok_hosts("mac", fast_only=True)
|
||||
if fast:
|
||||
return fast
|
||||
@@ -117,6 +292,9 @@ def get_ex_hosts() -> list[str]:
|
||||
env = os.environ.get("PYPYTDX_DATA_EX_HOSTS")
|
||||
if env:
|
||||
return [h.strip() for h in env.split(",") if h.strip()]
|
||||
cached = _load_cached_hosts("ex")
|
||||
if cached:
|
||||
return cached
|
||||
from_json = _load_ok_hosts("ex")
|
||||
return from_json or list(EX_HOSTS)
|
||||
|
||||
|
||||
@@ -0,0 +1,223 @@
|
||||
"""权威价格小数位表的同步与缓存(对用户无感)。
|
||||
|
||||
为什么需要它
|
||||
------------
|
||||
``get_quotes`` 解析报价时,协议给的是**整数**,真实价格 = ``整数 / 10**小数位``。
|
||||
小数位在协议里**不是**随报价一起下发的,只能靠推断(见
|
||||
:func:`pytdxdata.protocol.commands.quotes.price_decimal_digits`)。
|
||||
|
||||
但证券清单接口(``get_universe``,底层 ``GetSecurityListCmd``)**自带**这个字段:
|
||||
``SecurityInfo.decimal_point``,同时还带 ``pre_close``。这就是权威来源。本模块把它
|
||||
灌进 ``price_decimal_digits`` 使用的运行时表,使价格标度不再依赖硬编码前缀规则。
|
||||
|
||||
行为
|
||||
----
|
||||
- **当日首次启动**检查缓存文件(``decimal-points.json``):命中就直接装表(纯内存,
|
||||
无网络、约 10 ms);未命中则挂一个**后台 task** 拉证券清单并写缓存,
|
||||
当天不再重复。
|
||||
- 后台 task 不阻塞 ``start()``,不打印、不抛异常(取消除外),失败只落 debug 日志。
|
||||
- 未同步成功时 ``price_decimal_digits`` 静默退回静态前缀规则,行为与 0.6.5 一致。
|
||||
- 证券清单本身已被 ``cache/`` 层缓存,因此同步的实际网络开销只在**首次**
|
||||
(约 26 + 25 个分页请求)。
|
||||
|
||||
开关
|
||||
----
|
||||
- ``PYPYTDX_DATA_DISABLE_DECIMAL_SYNC=1`` 关闭后台同步(CI / 测试用,保证确定性)。
|
||||
- ``PYPYTDX_DATA_CACHE_DIR`` 覆盖缓存目录(测试指向临时目录)。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
from datetime import date
|
||||
from pathlib import Path
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from .config import decimal_cache_file, decimal_sync_disabled
|
||||
from .protocol.commands.quotes import (
|
||||
clear_decimal_map,
|
||||
decimal_map_source,
|
||||
decimal_map_stats,
|
||||
install_decimal_map,
|
||||
)
|
||||
|
||||
if TYPE_CHECKING: # pragma: no cover
|
||||
from .api import TdxData
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
#: 同步哪些市场(只对 A 股有意义:扩展市场报价本身不含标度歧义)
|
||||
DEFAULT_MARKETS: tuple[str, ...] = ("sh", "sz")
|
||||
|
||||
_CN_NAME = {"sh": 1, "sz": 0}
|
||||
|
||||
_state: dict[str, Any] = {"loaded": "", "task": None}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# 磁盘缓存
|
||||
# ---------------------------------------------------------------------- #
|
||||
|
||||
def cache_file() -> Path:
|
||||
"""缓存文件路径(每次重新解析,便于测试改环境变量)。"""
|
||||
return decimal_cache_file()
|
||||
|
||||
|
||||
def _read_cache(path: Path | None = None) -> dict[str, dict[str, int]] | None:
|
||||
"""读缓存 → ``{"1": {code: dp}}``;缺失 / 损坏 → ``None``。"""
|
||||
target = Path(path) if path is not None else cache_file()
|
||||
try:
|
||||
raw = json.loads(target.read_text(encoding="utf-8"))
|
||||
except (OSError, ValueError):
|
||||
return None
|
||||
if not isinstance(raw, dict):
|
||||
return None
|
||||
if raw.get("date") != date.today().isoformat():
|
||||
return None
|
||||
markets = raw.get("markets")
|
||||
if not isinstance(markets, dict) or not markets:
|
||||
return None
|
||||
out: dict[str, dict[str, int]] = {}
|
||||
for k, v in markets.items():
|
||||
if not isinstance(v, dict):
|
||||
continue
|
||||
try:
|
||||
out[str(int(k))] = {str(c): int(d) for c, d in v.items() if int(d) > 0}
|
||||
except (TypeError, ValueError):
|
||||
continue
|
||||
return out or None
|
||||
|
||||
|
||||
def load_cached(path: Path | None = None) -> int:
|
||||
"""把当日缓存装进权威表;成功返回覆盖条目数,否则 0。"""
|
||||
if decimal_sync_disabled():
|
||||
return 0
|
||||
markets = _read_cache(path)
|
||||
if not markets:
|
||||
return 0
|
||||
items = [(int(m), c, d) for m, codes in markets.items() for c, d in codes.items()]
|
||||
stats = install_decimal_map(items, source="cache")
|
||||
_state["loaded"] = date.today().isoformat()
|
||||
log.debug("小数位表已从缓存加载: %s", stats)
|
||||
return int(stats["stock"])
|
||||
|
||||
|
||||
def _write_cache(markets: dict[str, dict[str, int]], path: Path | None = None) -> None:
|
||||
"""原子写缓存(tmp + ``os.replace``),失败只落 debug 日志。"""
|
||||
target = Path(path) if path is not None else cache_file()
|
||||
payload = {"date": date.today().isoformat(), "markets": markets}
|
||||
tmp = target.with_suffix(".tmp")
|
||||
try:
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
tmp.write_text(
|
||||
json.dumps(payload, ensure_ascii=False, separators=(",", ":")),
|
||||
encoding="utf-8",
|
||||
)
|
||||
os.replace(tmp, target)
|
||||
except OSError as e: # pragma: no cover - 磁盘不可写等
|
||||
log.debug("小数位表缓存写入失败: %s", e)
|
||||
try:
|
||||
tmp.unlink(missing_ok=True)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# 同步
|
||||
# ---------------------------------------------------------------------- #
|
||||
|
||||
async def refresh(td: "TdxData", markets: tuple[str, ...] = DEFAULT_MARKETS,
|
||||
*, path: Path | None = None) -> dict[str, int]:
|
||||
"""拉证券清单 → 装表 + 写缓存。返回统计。
|
||||
|
||||
证券清单走 ``get_universe``(含 ``cache/`` 层缓存),只有首次真正发请求。
|
||||
"""
|
||||
tables: dict[str, dict[str, int]] = {}
|
||||
items: list[tuple[int, str, int]] = []
|
||||
for name in markets:
|
||||
m = _CN_NAME.get(name)
|
||||
if m is None:
|
||||
continue
|
||||
rows = await td.get_universe(name)
|
||||
table: dict[str, int] = {}
|
||||
for it in rows:
|
||||
dp = int(getattr(it, "decimal_point", 0) or 0)
|
||||
code = str(getattr(it, "code", "") or "").strip().rstrip("\x00")
|
||||
if dp <= 0 or not code:
|
||||
continue
|
||||
table[code] = dp
|
||||
items.append((m, code, dp))
|
||||
if table:
|
||||
tables[str(m)] = table
|
||||
|
||||
stats = install_decimal_map(items, source="universe")
|
||||
_state["loaded"] = date.today().isoformat()
|
||||
if tables:
|
||||
_write_cache(tables, path)
|
||||
log.debug("小数位表同步完成: %s", stats)
|
||||
return stats
|
||||
|
||||
|
||||
async def _run(td: "TdxData") -> dict[str, int]:
|
||||
try:
|
||||
return await refresh(td)
|
||||
except asyncio.CancelledError:
|
||||
raise
|
||||
except Exception as e: # pragma: no cover - 网络/解析异常都不该影响用户
|
||||
log.debug("小数位表后台同步失败: %s", e)
|
||||
return {}
|
||||
|
||||
|
||||
def schedule(td: "TdxData") -> asyncio.Task | None:
|
||||
"""[内部] 需要时把同步挂到后台并返回 Task(调用方持有 / 取消)。
|
||||
|
||||
- 已关闭(``PYPYTDX_DATA_DISABLE_DECIMAL_SYNC=1``)→ ``None``
|
||||
- 当日缓存命中 → 立即装表(同步、无网络),返回 ``None``
|
||||
- 否则创建后台 task;进程内单例,重复调用复用同一个 task
|
||||
"""
|
||||
if decimal_sync_disabled():
|
||||
return None
|
||||
if decimal_map_stats()["stock"] and _state["loaded"] == date.today().isoformat():
|
||||
return None
|
||||
if load_cached():
|
||||
return None
|
||||
task = _state.get("task")
|
||||
if task is not None and not task.done():
|
||||
return task
|
||||
task = asyncio.get_running_loop().create_task(_run(td), name="pytdxdata-decimal-sync")
|
||||
_state["task"] = task
|
||||
return task
|
||||
|
||||
|
||||
async def cancel(task: asyncio.Task | None) -> None:
|
||||
"""取消并等待后台同步 task(供 ``TdxData.close()`` 调用)。"""
|
||||
if task is None or task.done():
|
||||
return
|
||||
task.cancel()
|
||||
try:
|
||||
await task
|
||||
except (asyncio.CancelledError, Exception): # noqa: BLE001 - 关闭路径不抛
|
||||
pass
|
||||
|
||||
|
||||
def status() -> dict[str, Any]:
|
||||
"""当前状态快照(排障 / 测试用)。"""
|
||||
task = _state.get("task")
|
||||
return {
|
||||
"loaded": _state["loaded"],
|
||||
"source": decimal_map_source(),
|
||||
"syncing": bool(task is not None and not task.done()),
|
||||
**decimal_map_stats(),
|
||||
}
|
||||
|
||||
|
||||
def reset() -> None:
|
||||
"""清空内存状态(测试用;不动磁盘缓存)。"""
|
||||
task = _state.get("task")
|
||||
if task is not None and not task.done():
|
||||
task.cancel()
|
||||
_state["task"] = None
|
||||
_state["loaded"] = ""
|
||||
clear_decimal_map()
|
||||
@@ -0,0 +1,431 @@
|
||||
"""DataFrame 转换层(0.7.0):把模型列表转成 pandas / polars 的 DataFrame。
|
||||
|
||||
为什么单独一层而不是改各方法的返回:
|
||||
|
||||
- **零运行时依赖**是项目的硬约束(``dependencies = []``)。pandas / polars 只在
|
||||
真正用到 ``TdxData(df=...)`` 时才导入,没装就走不到这里。
|
||||
- 转换口径(哪些列留、时间怎么摆、嵌套字段怎么展开)只写一份,40 个返回表格型
|
||||
数据的方法共用,不会各自漂移。
|
||||
|
||||
三条口径约定(0.7.0 定稿):
|
||||
|
||||
1. **时间列替换原始列**:K线/分时的 ``year/month/day/hour/minute`` 不再单列输出,
|
||||
统一给一列 ``datetime``(pandas 为 ``datetime64[ns]``,polars 为 ``Datetime('us')``),
|
||||
另带 ``trading_date``(``YYYY-MM-DD`` 字符串,美股跨日口径见模型注释)。
|
||||
2. **嵌套字段定型**:``SecurityQuote.bid/ask`` 展开成 ``bid1_price``…``ask5_vol`` 共
|
||||
10 列(缺失为 ``None``);``fields`` 这类不定长字典序列化成 JSON 字符串列,
|
||||
空字典给 ``None`` 而不是 ``"{}"``;``TdxBlock.codes`` 保持列表。
|
||||
3. **嵌套调用不二次转换**:``get_kline_batch`` 这类方法内部还会调 ``get_bars``,
|
||||
用 :data:`_FRAME_DEPTH`(``ContextVar``,按 task 隔离)标记层次,内层一律返回
|
||||
原始 ``list``,只有最外层那次才转 DataFrame。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import functools
|
||||
import json
|
||||
import re
|
||||
from collections.abc import Callable, Sequence
|
||||
from contextvars import ContextVar
|
||||
from datetime import datetime as _dt
|
||||
from enum import IntEnum
|
||||
from typing import Any
|
||||
|
||||
from . import models as _models
|
||||
from .models import (
|
||||
IndexTradeStats,
|
||||
IndicatorSet,
|
||||
MinuteBar,
|
||||
MinuteTradeStat,
|
||||
SecurityBar,
|
||||
SecurityQuote,
|
||||
TransactionRecord,
|
||||
XdxrRecord,
|
||||
)
|
||||
|
||||
__all__ = ["BACKENDS", "FrameBackendError", "framed", "to_frame"]
|
||||
|
||||
BACKENDS: tuple[str, ...] = ("polars", "pandas")
|
||||
"""支持的后端名,也是 ``TdxData(df=...)`` 的合法取值。"""
|
||||
|
||||
|
||||
class FrameBackendError(ImportError):
|
||||
"""指定的 DataFrame 后端未安装。
|
||||
|
||||
继承 :class:`ImportError` 是刻意的:捕获姿势和缺包一致,
|
||||
且 ``e.name`` 就是缺的那个包名。
|
||||
"""
|
||||
|
||||
def __init__(self, backend: str) -> None:
|
||||
self.backend = backend
|
||||
super().__init__(
|
||||
f"未安装 {backend},无法输出 DataFrame。请执行:pip install pytdxdata[{backend}]",
|
||||
name=backend,
|
||||
)
|
||||
|
||||
|
||||
# 转换层次计数:>0 表示当前正处于某次公开方法的执行过程中(内层),此时不转换。
|
||||
_FRAME_DEPTH: ContextVar[int] = ContextVar("pytdxdata_frame_depth", default=0)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 后端
|
||||
|
||||
|
||||
class _Backend:
|
||||
"""后端适配:只封装「造表」和「拼表」两个动作。"""
|
||||
|
||||
name: str = ""
|
||||
|
||||
def frame(self, cols: dict[str, list]) -> Any: # pragma: no cover - 抽象
|
||||
raise NotImplementedError
|
||||
|
||||
def concat(self, frames: Sequence[Any]) -> Any: # pragma: no cover - 抽象
|
||||
raise NotImplementedError
|
||||
|
||||
def empty(self, cols: Sequence[str]) -> Any:
|
||||
return self.frame({name: [] for name in cols})
|
||||
|
||||
|
||||
class _PolarsBackend(_Backend):
|
||||
name = "polars"
|
||||
|
||||
def __init__(self) -> None:
|
||||
try:
|
||||
import polars as _pl
|
||||
except ImportError as exc: # pragma: no cover - 依赖环境
|
||||
raise FrameBackendError("polars") from exc
|
||||
self._pl = _pl
|
||||
|
||||
def frame(self, cols: dict[str, list]) -> Any:
|
||||
# 全空时必须显式给列名,否则 polars 会造出 0 列的表
|
||||
return self._pl.DataFrame(cols)
|
||||
|
||||
def concat(self, frames: Sequence[Any]) -> Any:
|
||||
frames = [f for f in frames if f is not None and f.height > 0]
|
||||
if not frames:
|
||||
return None
|
||||
if len(frames) == 1:
|
||||
return frames[0]
|
||||
# vertical_relaxed:批量接口各标的分块建表,dtype 可能略有差异(如 Null 列)
|
||||
return self._pl.concat(frames, how="vertical_relaxed")
|
||||
|
||||
|
||||
class _PandasBackend(_Backend):
|
||||
name = "pandas"
|
||||
|
||||
def __init__(self) -> None:
|
||||
try:
|
||||
import pandas as _pd
|
||||
except ImportError as exc: # pragma: no cover - 依赖环境
|
||||
raise FrameBackendError("pandas") from exc
|
||||
self._pd = _pd
|
||||
|
||||
def frame(self, cols: dict[str, list]) -> Any:
|
||||
return self._pd.DataFrame(cols)
|
||||
|
||||
def concat(self, frames: Sequence[Any]) -> Any:
|
||||
frames = [f for f in frames if f is not None and len(f) > 0]
|
||||
if not frames:
|
||||
return None
|
||||
if len(frames) == 1:
|
||||
return frames[0]
|
||||
return self._pd.concat(frames, ignore_index=True)
|
||||
|
||||
|
||||
_BACKEND_CACHE: dict[str, _Backend] = {}
|
||||
|
||||
|
||||
def get_backend(name: str) -> _Backend:
|
||||
"""取后端实例(进程内缓存,重复取不会重复 import)。
|
||||
|
||||
:raises ValueError: 名字不在 :data:`BACKENDS` 里。
|
||||
:raises FrameBackendError: 对应的包没装。
|
||||
"""
|
||||
if name not in BACKENDS:
|
||||
raise ValueError(f"未知的 DataFrame 后端 {name!r},可选:{', '.join(BACKENDS)}")
|
||||
be = _BACKEND_CACHE.get(name)
|
||||
if be is None:
|
||||
be = _PolarsBackend() if name == "polars" else _PandasBackend()
|
||||
_BACKEND_CACHE[name] = be
|
||||
return be
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 列计划
|
||||
|
||||
|
||||
def _as_int(value: Any) -> Any:
|
||||
"""``IntEnum`` 转纯 ``int``:polars 对 IntEnum 的类型推断不可靠,转掉最稳。"""
|
||||
if isinstance(value, IntEnum):
|
||||
return int(value)
|
||||
return value
|
||||
|
||||
|
||||
def _json_or_none(value: Any) -> str | None:
|
||||
"""不定长字典序列化成 JSON 列;空字典给 ``None``(别让用户看到一列 ``"{}"``)。"""
|
||||
if not value:
|
||||
return None
|
||||
return json.dumps(value, ensure_ascii=False, separators=(",", ":"))
|
||||
|
||||
|
||||
def _getter(name: str) -> Callable[[Any], Any]:
|
||||
def _get(row: Any) -> Any:
|
||||
return _as_int(getattr(row, name))
|
||||
|
||||
return _get
|
||||
|
||||
|
||||
def _json_getter(name: str) -> Callable[[Any], Any]:
|
||||
def _get(row: Any) -> Any:
|
||||
return _json_or_none(getattr(row, name))
|
||||
|
||||
return _get
|
||||
|
||||
|
||||
def _level_getter(side: str, index: int, pos: int) -> Callable[[Any], Any]:
|
||||
"""``bid``/``ask`` 第 ``index`` 档的价格(pos=0)或数量(pos=1)。"""
|
||||
|
||||
def _get(row: Any) -> Any:
|
||||
levels = getattr(row, side)
|
||||
if len(levels) <= index:
|
||||
return None
|
||||
return levels[index][pos]
|
||||
|
||||
return _get
|
||||
|
||||
|
||||
def _moment(row: SecurityBar | MinuteBar) -> _dt:
|
||||
return row.datetime
|
||||
|
||||
|
||||
def _trading_date(row: SecurityBar | MinuteBar) -> str:
|
||||
return row.trading_date
|
||||
|
||||
|
||||
def _xdxr_day(row: XdxrRecord) -> _dt:
|
||||
# 除权日是「日」不是「时刻」,但 datetime64 / Datetime 才是可排序、可比较的列类型。
|
||||
# 全项目时间口径都是「不带 tzinfo 的北京时间」,这里保持一致。
|
||||
return _dt(row.year, row.month, row.day) # noqa: DTZ001
|
||||
|
||||
|
||||
#: 模型 -> 需要丢弃的原始字段(时间拆解字段,由增强列替代)
|
||||
_DROPS: dict[type, tuple[str, ...]] = {
|
||||
SecurityBar: ("year", "month", "day", "hour", "minute"),
|
||||
MinuteBar: ("hour", "minute", "date", "day_offset"),
|
||||
XdxrRecord: ("year", "month", "day"),
|
||||
SecurityQuote: ("bid", "ask"),
|
||||
}
|
||||
|
||||
#: 模型 -> 在「第一个被丢弃字段的位置」插入的增强列
|
||||
_EXTRAS: dict[type, tuple[tuple[str, Callable[[Any], Any]], ...]] = {
|
||||
SecurityBar: (("datetime", _moment), ("trading_date", _trading_date)),
|
||||
MinuteBar: (("datetime", _moment), ("trading_date", _trading_date)),
|
||||
XdxrRecord: (("date", _xdxr_day),),
|
||||
SecurityQuote: tuple(
|
||||
(f"{side}{level + 1}_{part}", _level_getter(side, level, pos))
|
||||
for side in ("bid", "ask")
|
||||
for level in range(5)
|
||||
for pos, part in ((0, "price"), (1, "vol"))
|
||||
),
|
||||
}
|
||||
|
||||
#: 需要 JSON 化的字典字段(按类区分,避免误伤 ``FinancialRecord.fields: list[float]``)
|
||||
_JSON_FIELDS: dict[type, frozenset[str]] = {
|
||||
SecurityQuote: frozenset({"fields"}),
|
||||
TransactionRecord: frozenset({"fields"}),
|
||||
MinuteBar: frozenset({"fields"}),
|
||||
}
|
||||
|
||||
_PLAN_CACHE: dict[type, tuple[tuple[str, Callable[[Any], Any]], ...]] = {}
|
||||
|
||||
|
||||
def _plan_for(cls: type) -> tuple[tuple[str, Callable[[Any], Any]], ...]:
|
||||
"""构造(并缓存)某模型的输出列计划。
|
||||
|
||||
查找顺序按 MRO 走:``IndexBar`` 没有自己的计划,但它是 ``SecurityBar`` 的子类,
|
||||
直接复用父类计划即可(字段完全相同)。
|
||||
"""
|
||||
cached = _PLAN_CACHE.get(cls)
|
||||
if cached is not None:
|
||||
return cached
|
||||
|
||||
drops: tuple[str, ...] = ()
|
||||
extras: tuple[tuple[str, Callable[[Any], Any]], ...] = ()
|
||||
json_fields: frozenset[str] = frozenset()
|
||||
for klass in cls.__mro__:
|
||||
if not drops and klass in _DROPS:
|
||||
drops = _DROPS[klass]
|
||||
if not extras and klass in _EXTRAS:
|
||||
extras = _EXTRAS[klass]
|
||||
if not json_fields and klass in _JSON_FIELDS:
|
||||
json_fields = _JSON_FIELDS[klass]
|
||||
|
||||
import dataclasses
|
||||
|
||||
plan: list[tuple[str, Callable[[Any], Any]]] = []
|
||||
inserted = False
|
||||
for f in dataclasses.fields(cls): # type: ignore[arg-type]
|
||||
if f.name in drops:
|
||||
if not inserted:
|
||||
plan.extend(extras)
|
||||
inserted = True
|
||||
continue
|
||||
getter = _json_getter(f.name) if f.name in json_fields else _getter(f.name)
|
||||
plan.append((f.name, getter))
|
||||
if not inserted:
|
||||
# 没有丢弃字段的模型(如 SecurityInfo):增强列直接追加到末尾
|
||||
plan.extend(extras)
|
||||
|
||||
_PLAN_CACHE[cls] = tuple(plan)
|
||||
return _PLAN_CACHE[cls]
|
||||
|
||||
|
||||
def _columns(cls: type, rows: Sequence[Any]) -> dict[str, list]:
|
||||
"""列式采集:一次遍历填满所有列,比逐行 ``asdict()`` 快得多。"""
|
||||
return {name: [get(row) for row in rows] for name, get in _plan_for(cls)}
|
||||
|
||||
|
||||
def _columns_of_dicts(rows: Sequence[dict]) -> dict[str, list]:
|
||||
"""``list[dict]``(板块排行这类字段不固定的返回)按首次出现顺序并集列名。"""
|
||||
keys: list[str] = []
|
||||
seen: set[str] = set()
|
||||
for row in rows:
|
||||
for key in row:
|
||||
if key not in seen:
|
||||
seen.add(key)
|
||||
keys.append(key)
|
||||
return {key: [row.get(key) for row in rows] for key in keys}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 容器分发
|
||||
|
||||
|
||||
def _model_by_name(name: str | None) -> type | None:
|
||||
if not name:
|
||||
return None
|
||||
obj = getattr(_models, name, None)
|
||||
return obj if isinstance(obj, type) else None
|
||||
|
||||
|
||||
def _resolve_model(data: Any, model: str | None) -> type | None:
|
||||
"""推断元素类型:非空列表看首元素;空列表只能靠调用方给的类型提示。"""
|
||||
if isinstance(data, list) and data:
|
||||
return type(data[0])
|
||||
return _model_by_name(model)
|
||||
|
||||
|
||||
def _frames_of_rows(be: _Backend, cls: type | None, rows: Sequence[Any]) -> Any:
|
||||
if cls is not None and rows and hasattr(cls, "__dataclass_fields__"):
|
||||
return be.frame(_columns(cls, rows))
|
||||
if rows and isinstance(rows[0], dict):
|
||||
return be.frame(_columns_of_dicts(rows))
|
||||
return be.empty(())
|
||||
|
||||
|
||||
def _from_indicator_sets(be: _Backend, items: Sequence[IndicatorSet]) -> Any:
|
||||
frames = []
|
||||
for item in items:
|
||||
cols = _columns(SecurityBar, item.bars)
|
||||
size = len(item.bars)
|
||||
if item.symbol:
|
||||
cols["symbol"] = [item.symbol] * size
|
||||
for name, series in item.indicators.items():
|
||||
values = list(series[:size])
|
||||
if len(values) < size:
|
||||
values.extend([None] * (size - len(values)))
|
||||
cols[name] = values
|
||||
frames.append(be.frame(cols))
|
||||
merged = be.concat(frames)
|
||||
return merged if merged is not None else be.frame(_columns(SecurityBar, []))
|
||||
|
||||
|
||||
def _from_index_trade_stats(be: _Backend, items: Sequence[IndexTradeStats]) -> Any:
|
||||
frames = []
|
||||
for stat in items:
|
||||
cols = _columns(MinuteTradeStat, stat.minutes)
|
||||
size = len(stat.minutes)
|
||||
# 元信息补成常量列:一张表就能看出是哪个指数、哪一天
|
||||
cols = {"index": [stat.index] * size, "date": [stat.date] * size, **cols}
|
||||
frames.append(be.frame(cols))
|
||||
merged = be.concat(frames)
|
||||
return merged if merged is not None else be.frame(_columns(MinuteTradeStat, []))
|
||||
|
||||
|
||||
def to_frame(data: Any, *, backend: str = "polars", model: str | None = None) -> Any:
|
||||
"""把模型返回值转成 DataFrame。
|
||||
|
||||
:param data: ``list[dataclass]`` / ``list[dict]`` / ``dict[str, list[dataclass]]``
|
||||
/ ``list[IndicatorSet]`` / ``IndexTradeStats`` / ``list[IndexTradeStats]``。
|
||||
:param backend: ``"polars"`` 或 ``"pandas"``。
|
||||
:param model: 元素类型的名字(如 ``"SecurityBar"``);只在 ``data`` 为空、
|
||||
无法从元素推断类型时使用,由 :func:`framed` 从返回注解里自动填。
|
||||
"""
|
||||
be = get_backend(backend)
|
||||
|
||||
if isinstance(data, IndexTradeStats):
|
||||
return _from_index_trade_stats(be, [data])
|
||||
if isinstance(data, IndicatorSet):
|
||||
return _from_indicator_sets(be, [data])
|
||||
|
||||
if isinstance(data, dict):
|
||||
# 批量接口:{标的: [行]} → 拼成一张表(symbol/code 列本来就在行里)
|
||||
frames = []
|
||||
for rows in data.values():
|
||||
if isinstance(rows, list) and rows:
|
||||
frames.append(_frames_of_rows(be, type(rows[0]), rows))
|
||||
merged = be.concat(frames)
|
||||
if merged is not None:
|
||||
return merged
|
||||
cls = _model_by_name(model)
|
||||
return be.empty(tuple(name for name, _ in _plan_for(cls)) if cls else ())
|
||||
|
||||
if isinstance(data, list):
|
||||
if data and isinstance(data[0], IndexTradeStats):
|
||||
return _from_index_trade_stats(be, data)
|
||||
if data and isinstance(data[0], IndicatorSet):
|
||||
return _from_indicator_sets(be, data)
|
||||
cls = _resolve_model(data, model)
|
||||
if data and isinstance(data[0], dict):
|
||||
return _frames_of_rows(be, None, data)
|
||||
if cls is not None and data and hasattr(cls, "__dataclass_fields__"):
|
||||
return _frames_of_rows(be, cls, data)
|
||||
if data:
|
||||
# 元素既不是 dataclass 也不是 dict(如 list[float])——不该走到这里
|
||||
raise TypeError(f"无法转换为 DataFrame 的列表元素:{type(data[0])!r}")
|
||||
return be.frame({name: [] for name, _ in _plan_for(cls)}) if cls else be.empty(())
|
||||
|
||||
raise TypeError(f"无法转换为 DataFrame 的返回值类型:{type(data)!r}")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 接线
|
||||
|
||||
|
||||
_RETURN_MODEL = re.compile(r"list\[(\w+)\]")
|
||||
|
||||
|
||||
def framed(method: Callable[..., Any]) -> Callable[..., Any]:
|
||||
"""给 ``TdxData`` 的表格型方法套一层「按需转 DataFrame」。
|
||||
|
||||
- ``self._df_backend is None``(默认)时零开销,原样返回 ``list[dataclass]``。
|
||||
- 处于某次公开方法执行过程中(:data:`_FRAME_DEPTH` > 0)时不转换,
|
||||
保证批量/别名方法内部拿到的仍是原始 ``list``。
|
||||
"""
|
||||
model: str | None = None
|
||||
annotation = getattr(method, "__annotations__", {}).get("return")
|
||||
if isinstance(annotation, str):
|
||||
matched = _RETURN_MODEL.search(annotation)
|
||||
if matched:
|
||||
model = matched.group(1)
|
||||
|
||||
@functools.wraps(method)
|
||||
async def wrapper(self: Any, *args: Any, **kwargs: Any) -> Any:
|
||||
backend = getattr(self, "_df_backend", None)
|
||||
if backend is None or _FRAME_DEPTH.get() > 0:
|
||||
return await method(self, *args, **kwargs)
|
||||
|
||||
token = _FRAME_DEPTH.set(1)
|
||||
try:
|
||||
result = await method(self, *args, **kwargs)
|
||||
finally:
|
||||
_FRAME_DEPTH.reset(token)
|
||||
return to_frame(result, backend=backend, model=model)
|
||||
|
||||
return wrapper
|
||||
@@ -3,7 +3,7 @@ from __future__ import annotations
|
||||
|
||||
import struct
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import datetime
|
||||
from datetime import date, datetime, timedelta
|
||||
from enum import IntEnum
|
||||
from typing import Any
|
||||
|
||||
@@ -92,27 +92,104 @@ class Adjust(IntEnum):
|
||||
HFQ = 2 # 后复权
|
||||
|
||||
|
||||
# 扩展市场中「时间戳跨时区拼接」的市场号:通达信 EX 通道对美股返回的分钟级 K线
|
||||
# 里,日期字段是**美东交易日**、时分字段是**北京时间**,两个时区被拼进同一个
|
||||
# 五元组,必须归一化(见 SecurityBar.__post_init__)。
|
||||
_US_EX_MARKETS: frozenset[int] = frozenset({int(ExMarket.US_STOCK)})
|
||||
|
||||
# 归一化判据:北京时间 hour >= 12 的 bar 与美东交易日属同一自然日。美股常规时段
|
||||
# 09:30~16:00 EDT = 北京 21:30~次日 04:00(冬令时 22:30~次日 05:00),夏令时/
|
||||
# 冬令时下分界都稳定落在 12 点。
|
||||
_US_DAY_SPLIT_HOUR = 12
|
||||
|
||||
|
||||
# SecurityBar 上「后加的」可选字段:旧磁盘缓存反序列化时要补默认值(见 __setstate__)
|
||||
_BAR_LATE_FIELDS: tuple[str, ...] = ("settle",)
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class SecurityBar:
|
||||
"""K线记录(价格单位:元)"""
|
||||
"""K线记录(价格单位:元)。跨市场统一模型(A股/港股/美股/期货共用)。
|
||||
|
||||
时间戳口径**统一为北京时间**:A股/港股/期货原始即为北京时间;美股原始是
|
||||
「日期=美东交易日 + 时分=北京时间」的混合值,构造时已还原为真实北京时间
|
||||
(见 ``__post_init__``)。该 bar 归属的交易日见 :attr:`trading_date`。
|
||||
|
||||
``amount`` / ``settle`` 的可取性**按通道**:A股标准/MAC 通道两者都有;
|
||||
EX 通道(0x23FF)一个 32B 记录里只有**一个** 4B 槽位可复用,权益类市场用它
|
||||
装成交额、期货期权类用它装结算价,两者不可能同时拿到 —— 详见 :attr:`amount`。
|
||||
"""
|
||||
market: int
|
||||
code: str
|
||||
open: float
|
||||
high: float
|
||||
low: float
|
||||
close: float
|
||||
vol: float # 股票=股 / 转债=手(与vipdoc一致)
|
||||
amount: float # 元
|
||||
# vol 单位按品种:A股/ETF/美股=股;转债=手(1手=10张);⚠️港股主板/基金=手(1手=100股)
|
||||
vol: float
|
||||
# 元。⚠️ amount/vol 只在 A股上等于均价;转债还要再 /10(手=10张)、港股再 /100(手=100股)。
|
||||
# ⚠️ EX 通道的期货/期权(中金所、上期所、郑商所、大商所及各期权)协议帧内**没有成交额**,
|
||||
# 恒为 0.0 —— 要成交额请改用 EX 报价(``get_quotes``)或其它数据源。
|
||||
amount: float
|
||||
year: int
|
||||
month: int
|
||||
day: int
|
||||
hour: int = 0
|
||||
minute: int = 0
|
||||
symbol: str = "" # 规范化标的串(sz000001 / hk00700),由 API 层填充
|
||||
# 结算价。仅 EX 通道的**期货/期权**有(日线才有值,分钟线为 None);其余通道恒 None。
|
||||
settle: float | None = None
|
||||
|
||||
def __setstate__(self, state: Any) -> None:
|
||||
"""反序列化兜底:早期版本写下的磁盘缓存里没有后加的可选字段(如 ``settle``)。
|
||||
|
||||
``@dataclass(slots=True)`` 的默认反序列化只把 pickle 里存在的槽位塞回去,
|
||||
缺键的槽位**不会**被赋默认值 —— 直接读 ``bar.settle`` 会抛 ``AttributeError``。
|
||||
缓存键里没有 schema 版本号,所以这里必须容错,否则升级后读旧缓存即崩。
|
||||
"""
|
||||
if isinstance(state, tuple) and len(state) == 2: # slots 形式的 (dict, slots)
|
||||
data = dict(state[1] or {})
|
||||
else:
|
||||
data = dict(state or {})
|
||||
for name in _BAR_LATE_FIELDS:
|
||||
data.setdefault(name, None)
|
||||
for key, value in data.items():
|
||||
object.__setattr__(self, key, value)
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""美股(EX 通道)时间戳归一化:还原成真实北京时间。
|
||||
|
||||
原始报文里 ``zipday`` 是**美东交易日**、``tminutes`` 是**北京时间分钟数**,
|
||||
直拼出的 ``datetime`` 会把同一交易日的下午半场(北京 00:00~04:00)排到
|
||||
上午半场(北京 21:30~23:59)**之前** —— 单日 390 根全部错位、首末颠倒。
|
||||
这里把凌晨段(北京 hour < 12)的日期 +1 天,得到严格单调的北京时间序列。
|
||||
"""
|
||||
if self.market in _US_EX_MARKETS and self.hour < _US_DAY_SPLIT_HOUR:
|
||||
d = date(self.year, self.month, self.day) + timedelta(days=1)
|
||||
self.year, self.month, self.day = d.year, d.month, d.day
|
||||
|
||||
@property
|
||||
def is_us(self) -> bool:
|
||||
"""是否美股(EX 通道 ``US_STOCK``)。"""
|
||||
return self.market in _US_EX_MARKETS
|
||||
|
||||
@property
|
||||
def datetime(self) -> datetime:
|
||||
"""bar 的时刻(北京时间)。"""
|
||||
return datetime(self.year, self.month, self.day, self.hour, self.minute)
|
||||
|
||||
@property
|
||||
def trading_date(self) -> str:
|
||||
"""该 bar 归属的交易日,``YYYY-MM-DD``。
|
||||
|
||||
美股:``datetime`` 已还原为北京时间,凌晨段(北京 hour < 12)属于**前一天**
|
||||
的美东交易日,故回退一天;A股/港股/期货的交易日即自然日。按交易日聚合
|
||||
(如分钟线合成日线)时应当用本字段,而不是 ``datetime.date()``。
|
||||
"""
|
||||
if self.is_us and self.hour < _US_DAY_SPLIT_HOUR:
|
||||
return (date(self.year, self.month, self.day) - timedelta(days=1)).isoformat()
|
||||
return f"{self.year:04d}-{self.month:02d}-{self.day:02d}"
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class IndexBar(SecurityBar):
|
||||
@@ -120,9 +197,26 @@ class IndexBar(SecurityBar):
|
||||
pass
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class IndicatorSet:
|
||||
"""K线 + 技术指标组合(``TdxData.get_indicators`` 的返回单元)。
|
||||
|
||||
``indicators`` 里每个序列与 ``bars`` 等长、尾部对齐;预热期为 ``None`` 的指标
|
||||
(如 BOLL 前三档)保留 ``None``。
|
||||
"""
|
||||
symbol: str
|
||||
bars: list[SecurityBar] = field(default_factory=list)
|
||||
indicators: dict[str, list[float | None]] = field(default_factory=dict)
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class SecurityQuote:
|
||||
"""实时五档报价"""
|
||||
"""报价(跨市场统一模型)。
|
||||
|
||||
A股(标准通道):五档 ``bid``/``ask`` + 买卖量 + 涨跌停价齐全,``name`` 为空。
|
||||
A股(``fields=[...]`` 走 MAC 0x122B):``name`` 与 ``fields`` 齐全,五档为空。
|
||||
扩展市场(港股/美股/期货):``name`` + 基础价量在属性上,其余字段位在 ``fields``。
|
||||
"""
|
||||
market: int
|
||||
code: str
|
||||
price: float
|
||||
@@ -144,39 +238,144 @@ class SecurityQuote:
|
||||
limit_down: float = 0.0 # 跌停价
|
||||
decimal_point: int = 2 # 价格小数位
|
||||
open_amount: float = 0.0 # 开盘金额(个股)
|
||||
symbol: str = "" # 规范化标的串
|
||||
name: str = "" # 名称(仅 MAC/EX 通道有)
|
||||
fields: dict[str, float] = field(default_factory=dict) # 原始字段位("0x7" 等)
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class SecurityInfo:
|
||||
"""证券列表条目"""
|
||||
"""标的清单条目(跨市场统一模型:A股证券列表 + 扩展市场商品列表)"""
|
||||
market: int
|
||||
code: str
|
||||
name: str
|
||||
volunit: int
|
||||
decimal_point: int
|
||||
pre_close: float
|
||||
symbol: str = "" # 规范化标的串
|
||||
kind: str = "" # 分类描述(扩展市场的 category/desc)
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class TransactionRecord:
|
||||
"""逐笔成交(MAC通道含 trade_count=成交笔数)"""
|
||||
"""逐笔成交(跨市场统一模型;A股 MAC 通道含 trade_count=成交笔数)。
|
||||
|
||||
注意(实测 2026-10-21):
|
||||
|
||||
- 服务端是按**约 3 秒聚合**返回的切片,不是交易所原始逐笔成交。
|
||||
- ``vol`` / ``trade_count`` 都是**该切片自身的增量**,不是当日累计值。
|
||||
- ``trade_count`` 为**有符号**值:集合竞价段(``bs_flag=8``)会是负数
|
||||
(实测 sh600000 09:15 段为 -874),不要对全表直接求 max/mean。
|
||||
- 返回列表已按时间升序,但同秒多条时顺序不保证。
|
||||
- ``bs_flag=8`` 的**竞价委托记录已被 ``get_ticks()`` 过滤掉**,不会出现在结果里。
|
||||
|
||||
**指数语义完全不同**(``is_cn_index`` 为真时,如 ``sh000016``):
|
||||
|
||||
- ``vol`` 单位是**百元**(= 成交额 / 100),不是「手」。实测
|
||||
``快照amount / Σtick.vol = 100.0002``,且 ``Σtick.vol`` 与分时 Σvol **逐位相等**。
|
||||
想和成分股成交量对比必须先换算,直接比会得到 ~26 倍的假比例。
|
||||
- ``trade_count`` **不是笔数**,而是**当时均价 × 1000**(实测 tc/1000 与分时
|
||||
``avg_price`` 的差 < 1.1)。指数没有「成交笔数」概念。
|
||||
- ``bs_flag`` 语义不成立(实测返回 2,而模型只定义 0=卖 / 1=买)。
|
||||
|
||||
**场内基金同样没有笔数**(ETF / 多数 LOF,见
|
||||
:func:`~pytdxdata.symbols.has_order_count`):
|
||||
|
||||
- ``trade_count`` 是 ``价格 × 1000``(实测 ``sh510310`` 收盘 tc=3994,
|
||||
当时价 3.989 → 3989,偏差 σ≈2;货币 ETF ``sh511990`` 恒为 0)。
|
||||
- ``vol`` 仍是**手**,``bs_flag`` 正常。
|
||||
- 这类标的的 :attr:`trade_count_available` 为 ``False``,``trade_count`` 已被
|
||||
置 ``0``,不要拿去求和/求均值。
|
||||
|
||||
``trade_count_available=False`` 表示该品种**协议上就不带笔数**,
|
||||
与「本切片恰好 0 笔」不是一回事。
|
||||
|
||||
**可转债(110/111/113/118/123/127/128/132)三样都是真的**(实测 2026-10-22,
|
||||
16 只 A股可转债当日 + 8 只历史日全部自洽):
|
||||
|
||||
- ``vol`` 单位是**手**(1 手 = 10 张),与股票逐笔一致;``trade_count`` 是**真实成交笔数**
|
||||
(``sh111000`` 3631 条切片 / 20443 笔,可转债在
|
||||
:func:`~pytdxdata.symbols.has_order_count` 里判为**可用**)。
|
||||
- ``Σvol`` 与**日线** ``SecurityBar.vol`` **1:1 相等**(比值 1.000000)——
|
||||
这点和股票相反:股票逐笔是「手」、日线是「股」,Σ比值是 **0.01**。
|
||||
- ⚠️ 但**日线** ``amount / vol`` **不等于**均价,而是均价的 **10 倍**(``vol`` 的「手」= 10 张)。
|
||||
要算可转债均价须用 ``amount / vol / 10``;股票则是 ``amount / vol`` 直接就对。
|
||||
实测:``sz127025`` 逐笔 VWAP 105.9063 vs ``amount/vol`` 1059.0957,比值 10.0003。
|
||||
"""
|
||||
|
||||
market: int
|
||||
code: str
|
||||
time: str # HH:MM:SS
|
||||
time: str # HH:MM:SS(扩展市场为 HH:MM)
|
||||
price: float
|
||||
vol: float # 手(MAC逐笔)
|
||||
trade_count: int # 成交笔数(MAC=逐笔记录字段;标准当日逐笔协议也有但需单独解析)
|
||||
bs_flag: int # 1=主动买 0=主动卖
|
||||
vol: float # 股票/转债=手;指数=百元(成交额/100),见类注释
|
||||
trade_count: int # 股票=笔数增量;指数/基金=占位值,见类注释
|
||||
bs_flag: int # 1=主动买 0=主动卖 2=中性 5=盘后固定价(指数语义不成立)
|
||||
symbol: str = ""
|
||||
fields: dict[str, float] = field(default_factory=dict) # 扩展市场额外字段(增仓等)
|
||||
trade_count_available: bool = True # 指数/ETF/LOF 为 False(协议无笔数字段)
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class MinuteBar:
|
||||
"""分时数据"""
|
||||
"""分时(跨市场统一模型;A股 MAC 0x122D / 扩展市场 0x122D 同构)。
|
||||
|
||||
``hour``/``minute`` 是**真实时钟**(``0 <= hour < 24``),跨日的分时请用
|
||||
:attr:`datetime` / :attr:`trading_date`,不要自己拿 ``hour`` 拼日期。
|
||||
"""
|
||||
price: float
|
||||
vol: float
|
||||
vol: float # 手(股票/转债同);Σ分时vol 与当日日线 vol 同口径
|
||||
avg_price: float = 0.0 # 均价(协议第2字段)
|
||||
hour: int = 0
|
||||
minute: int = 0
|
||||
symbol: str = ""
|
||||
market: int = 0
|
||||
code: str = ""
|
||||
date: int = 0 # YYYYMMDD
|
||||
fields: dict[str, float] = field(default_factory=dict) # 扩展市场额外字段(动量等)
|
||||
day_offset: int = 0 # 相对 date 的自然日偏移:美股凌晨段为 +1
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
"""美股(EX 通道)分时时间归一化。
|
||||
|
||||
EX 分时的 ``minutes`` 字段是**自开盘起的连续分钟数**,不是「当天的分钟数」:
|
||||
实测 ``usAAPL`` 返回 ``21:30`` ~ ``27:59``(390 条里 240 条的 ``hour`` ≥ 24)。
|
||||
``27:59`` 实际是次日 ``03:59`` 北京时间。直接 ``minutes // 60`` 会得到
|
||||
``hour=27`` 这种**非法时钟值**,任何 ``datetime(...)`` 拼接都会抛异常,
|
||||
按小时聚合也会把凌晨段错误地归到同一天。
|
||||
|
||||
这里拆成「真实小时 + 日偏移」:``hour = (minutes // 60) % 24``,
|
||||
``day_offset = (minutes // 60) // 24``。A股/港股/期货的 ``hour`` 本就合法
|
||||
(A股合成时间 09:30~15:00、港股 09:30~16:00),**零行为变化**。
|
||||
"""
|
||||
if self.hour >= 24:
|
||||
self.day_offset += self.hour // 24
|
||||
self.hour %= 24
|
||||
|
||||
@property
|
||||
def datetime(self) -> datetime:
|
||||
"""该分时点的时刻(北京时间,已含跨日偏移)。"""
|
||||
base = date(self.date // 10000, self.date // 100 % 100, self.date % 100)
|
||||
d = base + timedelta(days=self.day_offset)
|
||||
return datetime(d.year, d.month, d.day, self.hour, self.minute)
|
||||
|
||||
@property
|
||||
def is_us(self) -> bool:
|
||||
"""是否美股(EX 通道 ``US_STOCK``)。"""
|
||||
return self.market in _US_EX_MARKETS
|
||||
|
||||
@property
|
||||
def trading_date(self) -> str:
|
||||
"""该分时点归属的交易日,``YYYY-MM-DD``。
|
||||
|
||||
美股:一个交易日横跨两个北京自然日(21:30~次日 04:00),凌晨段
|
||||
(北京 hour < 12)归属**前一天**的美东交易日,与
|
||||
:attr:`SecurityBar.trading_date` 口径一致。A股/港股/期货即 ``date`` 本身。
|
||||
按交易日聚合分时(合成日线、日内统计)**必须用它**。
|
||||
"""
|
||||
base = self.datetime
|
||||
if self.is_us and base.hour < _US_DAY_SPLIT_HOUR:
|
||||
base -= timedelta(days=1)
|
||||
return base.date().isoformat()
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
@@ -201,6 +400,7 @@ class XdxrRecord:
|
||||
qian_zongguben: float = 0.0
|
||||
panhou_liutong: float = 0.0
|
||||
hou_zongguben: float = 0.0
|
||||
symbol: str = ""
|
||||
|
||||
@property
|
||||
def date(self) -> str:
|
||||
@@ -248,6 +448,7 @@ class FinanceRecord:
|
||||
weifen_lirun: float = 0.0
|
||||
meigujing_zichan: float = 0.0
|
||||
reserve2: float = 0.0
|
||||
symbol: str = ""
|
||||
|
||||
|
||||
# ===== MAC 扩展命令模型 =====
|
||||
@@ -268,15 +469,21 @@ class BoardInfo:
|
||||
symbol_price: float = 0.0
|
||||
symbol_rise_speed: float = 0.0
|
||||
symbol_pre_close: float = 0.0
|
||||
symbol: str = "" # 规范化标的串(板块码,如 sh881001)
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class MemberQuote:
|
||||
"""板块成分股报价(MAC自定义字段)"""
|
||||
"""板块成分/自定义字段报价的**协议层原始记录**。
|
||||
|
||||
API 层会把它归一成 :class:`SecurityQuote`(``fields`` 原样保留、基础字段位
|
||||
填进属性),所以普通调用不会看到本类。
|
||||
"""
|
||||
market: int
|
||||
code: str
|
||||
name: str
|
||||
fields: dict[str, float] = field(default_factory=dict)
|
||||
symbol: str = ""
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
@@ -287,6 +494,7 @@ class BelongBoard:
|
||||
board_name: str
|
||||
close: float
|
||||
pre_close: float
|
||||
symbol: str = "" # 被查询的个股(非板块)
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
@@ -296,6 +504,7 @@ class AuctionItem:
|
||||
price: float
|
||||
matched: int
|
||||
unmatched: int
|
||||
symbol: str = ""
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
@@ -308,6 +517,7 @@ class UnusualItem:
|
||||
desc: str
|
||||
value: float
|
||||
time: str
|
||||
symbol: str = ""
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
@@ -328,6 +538,7 @@ class SymbolSnapshot:
|
||||
amount: float
|
||||
turnover: float
|
||||
avg_price: float
|
||||
symbol: str = ""
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
@@ -361,6 +572,7 @@ class CapitalFlow:
|
||||
sell5: float
|
||||
big5: float
|
||||
mid5: float
|
||||
symbol: str = ""
|
||||
|
||||
|
||||
# ===== 服务器文件解析模型(板块/行业/历史专业财报) =====
|
||||
@@ -405,3 +617,82 @@ class FinancialRecord:
|
||||
market: int # 0=深 1=沪
|
||||
report_date: int # 报告期 YYYYMMDD
|
||||
fields: list[float] = field(default_factory=list)
|
||||
|
||||
|
||||
# ===== 指数成分成交汇总(TdxData.get_index_trade_stats) =====
|
||||
|
||||
@dataclass(slots=True)
|
||||
class MinuteTradeStat:
|
||||
"""指数**全部成分股**汇总后的单分钟成交**增量**(不是累计)。
|
||||
|
||||
累计曲线用 :meth:`IndexTradeStats.cumulative`,不要自己再对同一字段反复累加。
|
||||
"""
|
||||
time: str # HH:MM
|
||||
volume: float # 成交量合计(手,1 手 = 100 股)
|
||||
trade_count: int # 成交笔数合计
|
||||
amount: float # 成交额估算(元)= Σ(逐笔价 × 手 × 100)
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
class IndexTradeStats:
|
||||
"""某个指数「全部成分股逐笔」汇总出的成交曲线。
|
||||
|
||||
为什么需要它:**通达信协议里指数本身没有成交笔数** —— 指数逐笔
|
||||
(``sh000001``)的 ``trade_count`` 字段实际是「当时均价 × 1000」(见
|
||||
:class:`TransactionRecord`)。所以「指数笔数」唯一可行的口径是
|
||||
**把成分股的逐笔笔数累加**,本类即该口径。
|
||||
|
||||
口径校验(2026-10-22 实测):
|
||||
|
||||
- ``total_amount`` 是**成分股**合计。对「成分恰好就是指数覆盖面」的指数,它与
|
||||
指数快照 ``amount`` **完全相等**:``sh000016`` 661.34 亿 = 661.34 亿、
|
||||
``sh000300`` 2373.98 亿 = 2373.98 亿(比值 1.0000)。
|
||||
- **但深市 ``399xxx`` 的指数成交额字段是「全板块」口径**,不是样本股:
|
||||
实测 ``sz399001`` 快照 5081.40 亿 = 深市 2908 只股票合计 5081.01 亿、
|
||||
``sz399006`` 2568.60 亿 = 创业板全部股票合计。所以拿深市指数 ``amount``
|
||||
与 ``total_amount`` 比会得到 ~0.43 这类无意义比值——不是数据错,是口径不同。
|
||||
- ``total_volume`` 单位是**手**(与指数逐笔的「百元」单位不同,勿混用)。
|
||||
- 北交所成分(``bj`` 前缀)**没有笔数字段**(实测 ``trade_count`` 恒为 0):
|
||||
此时 ``trade_count_members = 0``、:attr:`trade_count_available` 为 ``False``,
|
||||
只有成交量可用。``sh``/``sz`` 成分均正常返回笔数。
|
||||
|
||||
``minutes`` 只包含**实际有成交**的分钟(午休 11:30~13:00 之间无桶),
|
||||
画累计曲线时请按 ``time`` 自行补齐或直接连线。
|
||||
"""
|
||||
index: str # 规范化指数标的(如 sh000300)
|
||||
date: int | None # None=当日;否则 YYYYMMDD
|
||||
member_count: int # 指数成分数
|
||||
active_members: int # 实际有逐笔成交的成分数(停牌/无成交不计)
|
||||
trade_count_members: int # 返回了笔数的成分数(北交所为 0)
|
||||
minutes: list[MinuteTradeStat] = field(default_factory=list)
|
||||
total_volume: float = 0.0
|
||||
total_trade_count: int = 0
|
||||
total_amount: float = 0.0
|
||||
failed_members: int = 0 # 逐笔拉取失败的成分数(skip_errors=True 时)
|
||||
elapsed: float = 0.0 # 本次汇总耗时(秒)
|
||||
|
||||
@property
|
||||
def trade_count_available(self) -> bool:
|
||||
"""是否有笔数口径(北交所成分为 ``False``,此时 ``trade_count`` 全 0)。"""
|
||||
return self.trade_count_members > 0
|
||||
|
||||
def cumulative(self) -> list[MinuteTradeStat]:
|
||||
"""累计曲线:当日**截止该分钟**的累计量,时间升序。
|
||||
|
||||
这是「当日截止该分钟累计成交笔数」的直接数据源。
|
||||
"""
|
||||
out: list[MinuteTradeStat] = []
|
||||
vol = 0.0
|
||||
tc = 0
|
||||
amt = 0.0
|
||||
for m in self.minutes:
|
||||
vol += m.volume
|
||||
tc += m.trade_count
|
||||
amt += m.amount
|
||||
out.append(MinuteTradeStat(time=m.time, volume=vol,
|
||||
trade_count=tc, amount=amt))
|
||||
return out
|
||||
|
||||
def cum_trade_count(self) -> list[tuple[str, int]]:
|
||||
"""``[(HH:MM, 累计笔数)]``,画「截止该分钟累计笔数」曲线用。"""
|
||||
return [(m.time, m.trade_count) for m in self.cumulative()]
|
||||
@@ -3,6 +3,7 @@ from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import time
|
||||
from collections.abc import Sequence
|
||||
from typing import Generic, TypeVar
|
||||
|
||||
from ..protocol.commands.base import BaseCommand
|
||||
@@ -22,6 +23,7 @@ class TdxConnection(Generic[T]):
|
||||
self._reader: asyncio.StreamReader | None = None
|
||||
self._writer: asyncio.StreamWriter | None = None
|
||||
self._last_result: T | None = None
|
||||
self._pipeline_result: list[T] = []
|
||||
|
||||
@property
|
||||
def alive(self) -> bool:
|
||||
@@ -107,6 +109,58 @@ class TdxConnection(Generic[T]):
|
||||
body = decompress_body(header, raw_body)
|
||||
self._last_result: T = cmd.parse_response(body)
|
||||
|
||||
async def execute_pipeline(
|
||||
self, cmds: Sequence[BaseCommand[T]], *, timeout: float | None = None
|
||||
) -> list[T]:
|
||||
"""单连接**流水线**:一次性发出全部请求,再按序读取全部响应。
|
||||
|
||||
通达信协议是自描述长度帧(16 字节帧头 + body),且服务器**按序**应答,
|
||||
因此可以在同一 TCP 连接上连续下发多条请求、省掉 ``(N-1)`` 次往返。
|
||||
实测 66 个「80 只报价」请求:逐条 7,372ms → 流水线 526ms(14 倍)。
|
||||
|
||||
与 :meth:`execute` 的关键差异:``execute`` 逐条严格配对,失败可小范围重试;
|
||||
流水线一旦中途失败,**响应流已错配**,剩余响应无法再与请求对齐,
|
||||
故此处强制断开连接(上层应换连接重试)。
|
||||
|
||||
参数:
|
||||
cmds: 同一连接的请求序列(通常为同一类命令的不同分片)。
|
||||
timeout: 整体超时(默认 ``self.timeout * 2``,流水线耗时随条数增长)。
|
||||
|
||||
返回:
|
||||
与 ``cmds`` **等长且同序**的结果列表。
|
||||
"""
|
||||
if not cmds:
|
||||
return []
|
||||
if not self.alive:
|
||||
raise ConnectionError(f"连接未建立或已关闭: {self.host}")
|
||||
assert self._reader is not None and self._writer is not None
|
||||
budget = timeout if timeout is not None else self.timeout * 2
|
||||
try:
|
||||
try:
|
||||
await asyncio.wait_for(self._io_pipeline(cmds), budget)
|
||||
except asyncio.TimeoutError:
|
||||
self._force_close()
|
||||
raise ConnectionError(
|
||||
f"流水线超时 {self.host} ({budget:.1f}s, {len(cmds)} 条)") from None
|
||||
except (ConnectionError, OSError) as e:
|
||||
self._force_close()
|
||||
raise ConnectionError(f"流水线错误 {self.host}: {e}") from e
|
||||
except (ConnectionError, OSError):
|
||||
raise
|
||||
return self._pipeline_result
|
||||
|
||||
async def _io_pipeline(self, cmds: Sequence[BaseCommand[T]]) -> None:
|
||||
assert self._reader is not None and self._writer is not None
|
||||
self._writer.write(b"".join(c.build_request() for c in cmds))
|
||||
await self._writer.drain()
|
||||
results: list[T] = []
|
||||
for cmd in cmds:
|
||||
header_buf = await self._recv_exact(HEADER_SIZE)
|
||||
header = parse_header(header_buf)
|
||||
raw_body = await self._recv_exact(header.zipsize)
|
||||
results.append(cmd.parse_response(decompress_body(header, raw_body)))
|
||||
self._pipeline_result: list[T] = results
|
||||
|
||||
async def _send_setup(self) -> None:
|
||||
"""按序发送3条握手命令并丢弃响应。"""
|
||||
assert self._reader is not None and self._writer is not None
|
||||
|
||||
@@ -1,4 +1,11 @@
|
||||
"""EX扩展市场连接池(简化版:固定连接数,借还+坏连接重建+login自动)"""
|
||||
"""EX扩展市场连接池(固定连接数,借还+坏连接重建+login自动)。
|
||||
|
||||
与标准池 pool.py 的背压语义对齐:acquire 在无空闲连接且无法扩容时,
|
||||
基于 asyncio.Condition 排队等待 release 唤醒(默认 acquire_timeout=30s),
|
||||
而不是快速失败。per_server_cap 限制单台服务器连接数;单服务器场景下
|
||||
有效并发 = servers × per_server_cap(默认 1),通过 TdxData(ex_pool_cap=N)
|
||||
可对同一 EX 主机开多连接。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
@@ -9,6 +16,23 @@ from .ex_connection import ExTdxConnection
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
DEFAULT_ACQUIRE_TIMEOUT = 30.0
|
||||
|
||||
|
||||
class ExPoolTimeoutError(ConnectionError):
|
||||
"""acquire 等待超时(携带池状态快照,便于排障)。"""
|
||||
|
||||
def __init__(self, pool: "ExPool", timeout: float) -> None:
|
||||
idle = len(pool._idle)
|
||||
total = pool._total
|
||||
servers = len(pool.servers)
|
||||
cap = pool.per_server_cap
|
||||
super().__init__(
|
||||
f"EX连接池获取超时(等了 {timeout:.1f}s):"
|
||||
f"idle={idle} total={total} servers={servers} per_server_cap={cap} "
|
||||
f"有效并发={servers * cap}"
|
||||
)
|
||||
|
||||
|
||||
class ExPool:
|
||||
"""EX通道连接池(港股/期货,端口7727,连接时自动login)。"""
|
||||
@@ -16,6 +40,7 @@ class ExPool:
|
||||
def __init__(self, servers: list[str], *, timeout: float = 10.0,
|
||||
min_size: int = 2, max_size: int = 6,
|
||||
per_server_cap: int = 1,
|
||||
acquire_timeout: float | None = None,
|
||||
handshake: str = "login",
|
||||
handshake_map: dict | None = None) -> None:
|
||||
self.servers = servers
|
||||
@@ -25,14 +50,17 @@ class ExPool:
|
||||
self.min_size = min_size
|
||||
self.max_size = max_size
|
||||
self.per_server_cap = max(1, per_server_cap) # 一台服务器最多几个连接(默认1=单连接)
|
||||
self.acquire_timeout = (
|
||||
DEFAULT_ACQUIRE_TIMEOUT if acquire_timeout is None else acquire_timeout
|
||||
)
|
||||
self._idle: list[tuple[str, ExTdxConnection]] = []
|
||||
self._server_count: dict[str, int] = {}
|
||||
self._total = 0
|
||||
self._closed = False
|
||||
self._lock = asyncio.Lock()
|
||||
self._cond = asyncio.Condition()
|
||||
|
||||
async def start(self) -> None:
|
||||
async with self._lock:
|
||||
async with self._cond:
|
||||
for host in self.servers[: self.min_size]:
|
||||
if self._server_count.get(host, 0) >= self.per_server_cap:
|
||||
continue
|
||||
@@ -47,7 +75,7 @@ class ExPool:
|
||||
|
||||
async def close(self) -> None:
|
||||
self._closed = True
|
||||
async with self._lock:
|
||||
async with self._cond:
|
||||
for _, conn in self._idle:
|
||||
try:
|
||||
await conn.close()
|
||||
@@ -56,48 +84,61 @@ class ExPool:
|
||||
self._idle.clear()
|
||||
self._total = 0
|
||||
self._server_count.clear()
|
||||
self._cond.notify_all()
|
||||
|
||||
async def acquire(self) -> ExTdxConnection:
|
||||
for _ in range(3):
|
||||
stale_conn = None
|
||||
new_conn = None
|
||||
new_host = None
|
||||
async with self._lock:
|
||||
async def acquire(self, timeout: float | None = None) -> ExTdxConnection:
|
||||
"""借出一条连接;无空闲且无法扩容时排队等待 release 唤醒。
|
||||
|
||||
timeout 默认取 self.acquire_timeout(30s);传 None 表示无限等待。
|
||||
"""
|
||||
if timeout is None:
|
||||
timeout = self.acquire_timeout
|
||||
deadline = None if timeout is None else time.monotonic() + timeout
|
||||
async with self._cond:
|
||||
while True:
|
||||
# 1. 优先借用空闲连接(过期则丢弃,槽位在锁内回收)
|
||||
if self._idle:
|
||||
_, conn = self._idle.pop()
|
||||
fresh = time.monotonic() - conn.last_used < 30.0
|
||||
if conn.alive and fresh:
|
||||
return conn
|
||||
# 连接空闲过久(login会话可能失效)→ 丢弃重连
|
||||
# 连接空闲过久(login会话可能失效)→ 丢弃,槽位让给扩容
|
||||
self._total = max(0, self._total - 1)
|
||||
self._server_count[conn.host] = max(0, self._server_count.get(conn.host, 0) - 1)
|
||||
stale_conn = conn
|
||||
# 扩容:在锁内预留槽位,锁外执行网络IO
|
||||
elif self._total < self.max_size and self.servers:
|
||||
self._schedule_close(conn)
|
||||
|
||||
# 2. 可扩容 → 锁内预留槽位,锁外建连
|
||||
new_conn = None
|
||||
new_host = None
|
||||
if self._total < self.max_size and self.servers:
|
||||
host = self._next_available_server()
|
||||
if host:
|
||||
if host is not None:
|
||||
self._total += 1
|
||||
self._server_count[host] = self._server_count.get(host, 0) + 1
|
||||
new_conn = ExTdxConnection(host, 7727, self.timeout, self.handshake_map.get(host, self.handshake))
|
||||
new_host = host
|
||||
# 锁外:关闭过期连接、建立新连接
|
||||
if stale_conn is not None:
|
||||
if new_conn is not None:
|
||||
# 锁外网络 IO:连接失败回滚槽位后继续等待
|
||||
try:
|
||||
await new_conn.connect()
|
||||
return new_conn
|
||||
except Exception as e:
|
||||
log.warning("EX扩容失败 %s: %s", new_host, str(e)[:60])
|
||||
async with self._cond:
|
||||
self._total = max(0, self._total - 1)
|
||||
self._server_count[new_host] = max(0, self._server_count.get(new_host, 0) - 1)
|
||||
|
||||
# 3. 池满:排队等待 release 唤醒(或超时)
|
||||
now = time.monotonic()
|
||||
if deadline is not None and now >= deadline:
|
||||
raise ExPoolTimeoutError(self, timeout)
|
||||
try:
|
||||
await stale_conn.close()
|
||||
except Exception:
|
||||
pass
|
||||
if new_conn is not None:
|
||||
try:
|
||||
await new_conn.connect()
|
||||
return new_conn
|
||||
except Exception as e:
|
||||
log.warning("EX扩容失败 %s: %s", new_host, str(e)[:60])
|
||||
# 连接失败,回滚预留槽位
|
||||
async with self._lock:
|
||||
self._total = max(0, self._total - 1)
|
||||
self._server_count[new_host] = max(0, self._server_count.get(new_host, 0) - 1)
|
||||
await asyncio.sleep(0.05)
|
||||
raise ConnectionError("EX连接池无可用连接")
|
||||
if deadline is None:
|
||||
await self._cond.wait()
|
||||
else:
|
||||
await asyncio.wait_for(self._cond.wait(), timeout=deadline - now)
|
||||
except asyncio.TimeoutError:
|
||||
raise ExPoolTimeoutError(self, timeout)
|
||||
|
||||
def _next_available_server(self) -> str | None:
|
||||
for h in self.servers:
|
||||
@@ -105,15 +146,23 @@ class ExPool:
|
||||
return h
|
||||
return None
|
||||
|
||||
async def release(self, conn: ExTdxConnection, *, broken: bool = False) -> None:
|
||||
if broken or not conn.alive:
|
||||
async with self._lock:
|
||||
self._total = max(0, self._total - 1)
|
||||
self._server_count[conn.host] = max(0, self._server_count.get(conn.host, 0) - 1)
|
||||
def _schedule_close(self, conn: ExTdxConnection) -> None:
|
||||
async def _close():
|
||||
try:
|
||||
await conn.close()
|
||||
except Exception:
|
||||
pass
|
||||
return
|
||||
async with self._lock:
|
||||
self._idle.append((conn.host, conn))
|
||||
try:
|
||||
asyncio.get_running_loop().create_task(_close())
|
||||
except RuntimeError:
|
||||
pass
|
||||
|
||||
async def release(self, conn: ExTdxConnection, *, broken: bool = False) -> None:
|
||||
async with self._cond:
|
||||
if broken or not conn.alive:
|
||||
self._total = max(0, self._total - 1)
|
||||
self._server_count[conn.host] = max(0, self._server_count.get(conn.host, 0) - 1)
|
||||
self._schedule_close(conn)
|
||||
else:
|
||||
self._idle.append((conn.host, conn))
|
||||
self._cond.notify_all()
|
||||
+71
-12
@@ -6,8 +6,9 @@
|
||||
- 动态:支持运行时 add() 新服务器;支持后台全量测速入库(当日首次启动)。
|
||||
- 单连接约束(核心需求):per_server_cap 默认 1 —— 一台服务器只允许建立 1 个连接,
|
||||
不允许多开,避免对单一券商服务器造成并发压力被限流/封禁。
|
||||
- 后台测速(核心需求):当日首次启动时,在后台对全量候选服务器测速,按
|
||||
「排名前 FAST_POOL_TOP_N 或 延迟 < FAST_POOL_MAX_LATENCY」筛选入库,结果写回 servers.json。
|
||||
- 后台测速(核心需求):当日首次启动时,在后台对**全量候选**服务器测速(覆盖标准/MAC/EX
|
||||
三个协议,而不是只测当前池内节点),结果缓存到用户级缓存目录;当天已有缓存则跳过、不再刷新。
|
||||
全程不阻塞 start()、不打印、不抛错(对用户无感)。详见 pool/probe_cache.py。
|
||||
- 弹性:空闲连接超时后由 reaper 缩容回 min_size;支持 stats 统计。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
@@ -19,7 +20,7 @@ import time
|
||||
|
||||
from .connection import TdxConnection
|
||||
from .health import HealthEngine
|
||||
from .probe_cache import get_probe_cache
|
||||
from .probe_cache import schedule_probe
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
@@ -58,6 +59,11 @@ class PooledConnection:
|
||||
self.last_used = time.monotonic()
|
||||
return await self.conn.execute(cmd)
|
||||
|
||||
async def execute_pipeline(self, cmds, *, timeout: float | None = None):
|
||||
"""在同一条连接上流水线执行多条命令(见 TdxConnection.execute_pipeline)。"""
|
||||
self.last_used = time.monotonic()
|
||||
return await self.conn.execute_pipeline(cmds, timeout=timeout)
|
||||
|
||||
async def close(self) -> None:
|
||||
self.alive = False
|
||||
try:
|
||||
@@ -101,6 +107,7 @@ class DynamicPool:
|
||||
self._lock = asyncio.Lock()
|
||||
self._cond = asyncio.Condition(self._lock)
|
||||
self._reaper_task: asyncio.Task | None = None
|
||||
self._probe_task: asyncio.Task | None = None
|
||||
|
||||
# ---------- 生命周期 ----------
|
||||
async def start(self) -> None:
|
||||
@@ -108,13 +115,11 @@ class DynamicPool:
|
||||
return
|
||||
self._started = True
|
||||
|
||||
# 需求①:当日首次启动 → 后台全量测速入库(不阻塞主流程)
|
||||
# 需求①:当日首次启动 → 后台全量测速(不阻塞主流程;当天已有缓存则自动跳过)
|
||||
# 候选清单由 probe_cache 自行枚举全量主机(本池的 servers 只是其中一个子集,
|
||||
# 不能直接当作测速输入,否则只会重测已在池内的少数节点、其余记录永久过期)。
|
||||
if self.auto_probe:
|
||||
hosts = list(self.servers)
|
||||
asyncio.create_task(
|
||||
get_probe_cache().ensure_probe(hosts),
|
||||
name="pytdxdata-daily-probe",
|
||||
)
|
||||
self._probe_task = schedule_probe()
|
||||
|
||||
targets = self._pick_servers(self.min_size)
|
||||
for host in targets:
|
||||
@@ -175,6 +180,15 @@ class DynamicPool:
|
||||
if self._reaper_task is not None:
|
||||
self._reaper_task.cancel()
|
||||
self._reaper_task = None
|
||||
# 后台测速仍可能持有 socket;取消并等它退场,避免事件循环关闭时冒出
|
||||
# "Task was destroyed but it is pending"(那会破坏"无感")
|
||||
if self._probe_task is not None:
|
||||
self._probe_task.cancel()
|
||||
try:
|
||||
await self._probe_task
|
||||
except (asyncio.CancelledError, Exception):
|
||||
pass
|
||||
self._probe_task = None
|
||||
async with self._lock:
|
||||
for pc in list(self._conns):
|
||||
try:
|
||||
@@ -211,6 +225,10 @@ class DynamicPool:
|
||||
# 2. 可扩容(未达上限且有可用服务器)→ 新建连接
|
||||
if self._total < self.max_size:
|
||||
host = self._next_available_server()
|
||||
if host is None:
|
||||
# 兜底:全部服务器都在冷却中,而池子未满 —— 此时等 release
|
||||
# 永远等不到(没人持有),只能挑最健康的一台顶着冷却重试
|
||||
host = self._next_available_server(ignore_cooldown=True)
|
||||
if host:
|
||||
try:
|
||||
pc = await self._new_conn(host)
|
||||
@@ -244,21 +262,62 @@ class DynamicPool:
|
||||
break
|
||||
return out
|
||||
|
||||
def _next_available_server(self) -> str | None:
|
||||
def _next_available_server(self, *, ignore_cooldown: bool = False) -> str | None:
|
||||
ranked = self.health.rank(self.servers)
|
||||
# 1. 优先选未冷却且未达并发上限的服务器
|
||||
for h in ranked:
|
||||
if self.health.in_cooldown(h):
|
||||
if self.health.in_cooldown(h) and not ignore_cooldown:
|
||||
continue
|
||||
if self._server_count.get(h, 0) < self.per_server_cap:
|
||||
return h
|
||||
# 2. 兜底:仅从「未冷却」的已占用服务器里随机复用(避免反复打坏服务器)
|
||||
alive = [
|
||||
h for h in ranked
|
||||
if not self.health.in_cooldown(h) and self._server_count.get(h, 0) > 0
|
||||
if (ignore_cooldown or not self.health.in_cooldown(h))
|
||||
and self._server_count.get(h, 0) > 0
|
||||
]
|
||||
return random.choice(alive) if alive else None
|
||||
|
||||
async def warm(self, n: int) -> None:
|
||||
"""并发预建连接至 ``n`` 条(最多到 ``max_size``)。
|
||||
|
||||
:meth:`acquire` 建连时持有池锁,**一次只能建一条**——22 路并发首次
|
||||
调用会排队 22×握手时间(实测全市场首拉多花 5~6s)。大批量请求前先调
|
||||
本方法把连接并发建好,首拉只多付一次握手。
|
||||
|
||||
失败静默忽略:缺的连接后续由 ``acquire`` 按需补齐。
|
||||
"""
|
||||
if self._closed:
|
||||
return
|
||||
need = min(n, self.max_size) - self._total
|
||||
if need <= 0:
|
||||
return
|
||||
|
||||
# 用局部计数挑 host,避免并发建连时反复选中同一台
|
||||
counts = dict(self._server_count)
|
||||
ranked = self.health.rank(self.servers)
|
||||
hosts: list[str] = []
|
||||
for _ in range(need):
|
||||
host = next((h for h in ranked
|
||||
if not self.health.in_cooldown(h)
|
||||
and counts.get(h, 0) < self.per_server_cap), None)
|
||||
if host is None:
|
||||
break
|
||||
hosts.append(host)
|
||||
counts[host] = counts.get(host, 0) + 1
|
||||
if not hosts:
|
||||
return
|
||||
|
||||
results = await asyncio.gather(
|
||||
*(self._new_conn(h) for h in hosts), return_exceptions=True)
|
||||
async with self._lock:
|
||||
for pc in results:
|
||||
if isinstance(pc, PooledConnection):
|
||||
self._total += 1
|
||||
self.stats.total = self._total
|
||||
if not self._closed and pc.alive:
|
||||
self._idle.append(pc)
|
||||
|
||||
async def release(self, pc: PooledConnection, *, broken: bool = False) -> None:
|
||||
if self.stats.borrowed > 0:
|
||||
self.stats.borrowed -= 1
|
||||
|
||||
+272
-112
@@ -1,162 +1,322 @@
|
||||
"""当日首次启动的全量服务器测速与入库缓存。
|
||||
"""当日首次启动的全量服务器测速与缓存(对用户完全无感)。
|
||||
|
||||
设计要点:
|
||||
- 仅在"当日首次启动"时触发一次后台全量测速(按本地日期判断是否已探测)。
|
||||
- 测速采用真实握手连通(TdxConnection.test),比单纯 ping 更能反映通达信可用性。
|
||||
- 入库规则:延迟排名前 FAST_POOL_TOP_N(10) **或** 延迟 < FAST_POOL_MAX_LATENCY(0.2s) 的服务器
|
||||
才标记为 status="ok" 并写入 servers.json,其余降权(保留但不优先)。
|
||||
- 结果持久化到 servers.json,供 config / 连接池直接使用。
|
||||
行为
|
||||
----
|
||||
- **当日首次启动**触发一次后台全量测速;当天已有缓存则直接复用,**不再刷新**。
|
||||
- 测速覆盖**全部候选**(包内清单 ∪ 代码常量 ∪ env 覆盖,见
|
||||
:func:`pytdxdata.config.all_probe_candidates`),而不是只测当前已在池里的少数节点 ——
|
||||
否则"上次慢、这次快"的节点永远捞不回来。
|
||||
- 用**真实业务命令**判定可达(标准: GetSecurityBarsCmd / MAC: MacKlineCmd /
|
||||
EX: GetExInstrumentBarsCmd)并要求返回非空数据,而不是只做握手回显:
|
||||
存在"握手正常但 K线空响应"的坏节点(见 ``config.py`` 注释),只有真数据能测出来。
|
||||
- 结果写入**用户级缓存目录**(见 :func:`pytdxdata.config.probe_cache_file`),
|
||||
不写安装目录:site-packages 在容器 / 系统级安装 / pipx 下可能只读,
|
||||
多用户共用一份安装时还会互相覆盖。
|
||||
- 结果被 :func:`pytdxdata.config.get_standard_hosts` 等按**实测延迟升序**采用。
|
||||
|
||||
无感保证
|
||||
--------
|
||||
- 不打印任何东西;日志一律 ``debug`` 级(默认不输出)。
|
||||
- 不抛异常(取消除外);失败只落 debug 日志,下次启动自然重试。
|
||||
- 由连接池以后台 task 方式调用,不阻塞 ``start()`` 与任何用户请求。
|
||||
|
||||
一致性与并发
|
||||
------------
|
||||
- 进程内单例 + ``asyncio.Lock``:标准 / MAC / EX 三个池共用一次真正测速。
|
||||
- 原子写入(tmp + ``os.replace``):进程被杀或多进程并发都不会读到半截 JSON。
|
||||
- 全部不可达时**不写缓存**,避免把"全挂"状态缓存一整天。
|
||||
|
||||
开关
|
||||
----
|
||||
- ``PYPYTDX_DATA_DISABLE_PROBE=1`` 关闭(CI / 测试用,保证确定性)。
|
||||
- ``PYPYTDX_DATA_CACHE_DIR`` 覆盖缓存目录(测试指向临时目录)。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import time
|
||||
from collections import defaultdict
|
||||
from collections.abc import Awaitable, Callable
|
||||
from datetime import date
|
||||
from pathlib import Path
|
||||
|
||||
from ..config import _SERVERS_FILE
|
||||
from ..config import (
|
||||
PROBE_CACHE_VERSION,
|
||||
all_probe_candidates,
|
||||
get_ex_handshakes,
|
||||
is_valid_host,
|
||||
load_probe_cache,
|
||||
probe_cache_file,
|
||||
probe_disabled,
|
||||
)
|
||||
from ..models import Adjust, KlinePeriod
|
||||
from ..protocol.commands.ex_proto import GetExInstrumentBarsCmd
|
||||
from ..protocol.commands.kline import GetSecurityBarsCmd
|
||||
from ..protocol.commands.mac import MacKlineCmd
|
||||
from .connection import TdxConnection
|
||||
from .ex_connection import ExTdxConnection
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
# 入池阈值
|
||||
FAST_POOL_TOP_N = 10 # 延迟排名前 N
|
||||
FAST_POOL_MAX_LATENCY = 0.2 # 延迟 < 200ms(秒)也可入池
|
||||
# 入池阈值(仅用于缓存里的 fast 诊断标记;运行时按实测延迟整体排序,不做硬切)
|
||||
FAST_POOL_TOP_N = 10
|
||||
FAST_POOL_MAX_LATENCY = 0.2
|
||||
# 探针参数
|
||||
PROBE_PORT = 7709
|
||||
EX_PORT = 7727
|
||||
PROBE_TIMEOUT = 3.0
|
||||
PROBE_CONCURRENCY = 24
|
||||
PROBE_CONCURRENCY = 12
|
||||
PROBE_BUDGET = 60.0 # 整轮测速硬上限(秒),防止极端网络下长期占着后台
|
||||
|
||||
ProbeFn = Callable[[str, float], Awaitable[float | None]]
|
||||
|
||||
|
||||
# ---------- 三个协议各自的真实数据探针 ----------
|
||||
async def _probe_standard(host: str, timeout: float) -> float | None:
|
||||
"""标准通道 7709:取 600000 日K,非空才算可达。"""
|
||||
conn = TdxConnection(host, PROBE_PORT, timeout)
|
||||
t0 = time.monotonic()
|
||||
try:
|
||||
await conn.connect()
|
||||
bars = await conn.execute(GetSecurityBarsCmd(1, "600000", KlinePeriod.DAY, 0, 100))
|
||||
return time.monotonic() - t0 if bars else None
|
||||
except Exception:
|
||||
return None
|
||||
finally:
|
||||
await conn.close()
|
||||
|
||||
|
||||
async def _probe_mac(host: str, timeout: float) -> float | None:
|
||||
"""MAC 通道 7709:MAC 帧与标准帧不同,用 MacKlineCmd 取日K。"""
|
||||
conn = TdxConnection(host, PROBE_PORT, timeout)
|
||||
t0 = time.monotonic()
|
||||
try:
|
||||
await conn.connect()
|
||||
bars = await conn.execute(
|
||||
MacKlineCmd(1, "600000", KlinePeriod.DAY, Adjust.NONE, 0, 100)
|
||||
)
|
||||
return time.monotonic() - t0 if bars else None
|
||||
except Exception:
|
||||
return None
|
||||
finally:
|
||||
await conn.close()
|
||||
|
||||
|
||||
async def _probe_ex(host: str, timeout: float) -> float | None:
|
||||
"""EX 扩展市场 7727:取港股 00700,握手模式按主机取(login / setup)。"""
|
||||
conn = ExTdxConnection(host, EX_PORT, timeout, get_ex_handshakes().get(host, "setup"))
|
||||
t0 = time.monotonic()
|
||||
try:
|
||||
await conn.connect()
|
||||
bars = await conn.execute(GetExInstrumentBarsCmd(31, "00700", 0, 0, 5))
|
||||
return time.monotonic() - t0 if bars else None
|
||||
except Exception:
|
||||
return None
|
||||
finally:
|
||||
await conn.close()
|
||||
|
||||
|
||||
_DEFAULT_PROBERS: dict[str, ProbeFn] = {
|
||||
"standard": _probe_standard,
|
||||
"mac": _probe_mac,
|
||||
"ex": _probe_ex,
|
||||
}
|
||||
|
||||
|
||||
class ProbeCache:
|
||||
"""管理当日测速缓存与 servers.json 入库。
|
||||
"""当日全量测速的进程内协调器。
|
||||
|
||||
一台服务器不论协议(standard/mac/ex),测速结果共享:
|
||||
只要任一端口可达即视为该主机可用,并以最快端口延迟作为延迟基准。
|
||||
一台主机不论申报几个协议,都按协议分别探测;缓存里记录
|
||||
``{host: {protocol: 延迟或 None}}``,``None`` 表示该协议本次不可达。
|
||||
|
||||
参数(后 5 个主要供测试注入):
|
||||
cache_file: 显式指定缓存文件;``None`` 时按 config 的平台规则解析。
|
||||
probers: 协议 → 探针函数;默认使用真实网络探针。
|
||||
today_fn: 返回 ``YYYY-MM-DD``;测试可注入假日期来验证跨日重测。
|
||||
"""
|
||||
|
||||
def __init__(self, servers_file: Path | None = None) -> None:
|
||||
self._file = servers_file or _SERVERS_FILE
|
||||
def __init__(self, cache_file: Path | str | None = None, *,
|
||||
timeout: float = PROBE_TIMEOUT,
|
||||
concurrency: int = PROBE_CONCURRENCY,
|
||||
budget: float = PROBE_BUDGET,
|
||||
probers: dict[str, ProbeFn] | None = None,
|
||||
today_fn: Callable[[], str] | None = None) -> None:
|
||||
self._file = Path(cache_file) if cache_file is not None else None
|
||||
self._timeout = timeout
|
||||
self._concurrency = concurrency
|
||||
self._budget = budget
|
||||
self._probers = dict(probers or _DEFAULT_PROBERS)
|
||||
self._today_fn = today_fn or (lambda: date.today().isoformat())
|
||||
self._lock = asyncio.Lock()
|
||||
self._last_probe_date: str | None = None
|
||||
self._probed_today = False
|
||||
self._handled = False # 本进程是否已处理过今日测速(含"已有缓存,跳过")
|
||||
self._task: asyncio.Task | None = None
|
||||
|
||||
# ---------- 路径与状态 ----------
|
||||
def cache_file(self) -> Path:
|
||||
"""缓存文件路径(构造时显式传入的优先,否则按平台规则解析)。"""
|
||||
return self._file if self._file is not None else probe_cache_file()
|
||||
|
||||
def today(self) -> str:
|
||||
return self._today_fn()
|
||||
|
||||
def read_cache(self) -> dict | None:
|
||||
"""读取当日缓存;``None`` 表示"今天还没测过"。"""
|
||||
return load_probe_cache(self.today(), self.cache_file())
|
||||
|
||||
# ---- 公开 API ----
|
||||
def needs_probe_today(self) -> bool:
|
||||
"""是否有必要在今日执行一次全量测速(文件不存在 / 日期不符 / 无 ok 项)。"""
|
||||
if not self._file.exists():
|
||||
return True
|
||||
try:
|
||||
cfg = json.loads(self._file.read_text(encoding="utf-8"))
|
||||
except (OSError, ValueError):
|
||||
return True
|
||||
checked = str(cfg.get("checked_date", ""))
|
||||
today = date.today().isoformat()
|
||||
if checked != today:
|
||||
return True
|
||||
servers = cfg.get("servers", [])
|
||||
if not any(s.get("status") == "ok" for s in servers):
|
||||
return True
|
||||
return False
|
||||
"""今日是否仍需测速(缓存缺失 / 非今日 / 损坏 / 版本不符 / 无任何可用项)。"""
|
||||
if probe_disabled():
|
||||
return False
|
||||
return self.read_cache() is None
|
||||
|
||||
async def ensure_probe(self, hosts: list[str]) -> None:
|
||||
"""若今日尚未测速,则后台执行一次全量测速并入库(幂等)。"""
|
||||
def reset(self) -> None:
|
||||
"""清除"本进程已处理"标记(仅供测试复用单例)。"""
|
||||
self._handled = False
|
||||
|
||||
# ---------- 主入口 ----------
|
||||
async def ensure_probe(self, hosts: list[str] | None = None) -> None:
|
||||
"""幂等入口:当日尚未测速则测一次;当天已有缓存则立即返回。
|
||||
|
||||
- 不阻塞调用方:正常由连接池以后台 task 调用。
|
||||
- 绝不抛异常(``CancelledError`` 除外):测速 / 写盘失败只落 debug 日志。
|
||||
- ``hosts``:为兼容旧签名保留,作为**额外的标准通道候选**并入,
|
||||
且必须通过 :func:`~pytdxdata.config.is_valid_host` 校验(挡住 mock 脏数据)。
|
||||
"""
|
||||
async with self._lock:
|
||||
if self._probed_today:
|
||||
if self._handled:
|
||||
return
|
||||
self._handled = True # 先占位:并发调用方不必排队等网络
|
||||
if not self.needs_probe_today():
|
||||
self._probed_today = True
|
||||
log.debug("当日测速缓存已存在,跳过测速: %s", self.cache_file())
|
||||
return
|
||||
self._probed_today = True
|
||||
|
||||
# 在锁外执行(耗时网络 IO),允许并发进入的调用方共享本次结果
|
||||
try:
|
||||
await self._run_full_probe(hosts)
|
||||
except Exception as e: # 测速失败不能阻塞主流程
|
||||
log.warning("后台全量测速失败,沿用既有清单: %s", str(e)[:80])
|
||||
payload = await self._run_probe(hosts)
|
||||
if payload is not None:
|
||||
self._write(payload)
|
||||
except asyncio.CancelledError:
|
||||
raise
|
||||
except Exception as exc: # 测速绝不能影响主流程
|
||||
log.debug("后台测速失败,沿用既有清单: %s", str(exc)[:120])
|
||||
|
||||
# ---- 内部实现 ----
|
||||
async def _run_full_probe(self, hosts: list[str]) -> None:
|
||||
from .connection import TdxConnection
|
||||
# ---------- 内部实现 ----------
|
||||
async def _run_probe(self, extra_hosts: list[str] | None = None) -> dict | None:
|
||||
"""全量并发测速,返回待写入的缓存 payload;无任何可达主机时返回 None。"""
|
||||
candidates = all_probe_candidates()
|
||||
if extra_hosts:
|
||||
extra = [h.strip() for h in extra_hosts
|
||||
if isinstance(h, str) and is_valid_host(h)]
|
||||
if extra:
|
||||
candidates["standard"] = list(dict.fromkeys(candidates["standard"] + extra))
|
||||
jobs = [(proto, ip) for proto, ips in candidates.items() for ip in ips]
|
||||
if not jobs:
|
||||
log.debug("无测速候选,跳过")
|
||||
return None
|
||||
|
||||
log.info("开始当日全量服务器测速 (%d 台)...", len(hosts))
|
||||
sem = asyncio.Semaphore(PROBE_CONCURRENCY)
|
||||
results: dict[str, float | None] = {}
|
||||
log.debug("开始当日全量测速: %d 台主机 / %d 个协议探针",
|
||||
len({ip for _, ip in jobs}), len(jobs))
|
||||
sem = asyncio.Semaphore(self._concurrency)
|
||||
# {host: {protocol: 延迟或 None}}
|
||||
results: dict[str, dict[str, float | None]] = defaultdict(dict)
|
||||
|
||||
async def _probe_one(host: str) -> None:
|
||||
async def _one(protocol: str, host: str) -> None:
|
||||
async with sem:
|
||||
try:
|
||||
# 使用连接自带的 ping():建连+握手+发1条命令测往返延迟
|
||||
conn = TdxConnection(host, PROBE_PORT, PROBE_TIMEOUT)
|
||||
dt = await asyncio.wait_for(conn.ping(), timeout=PROBE_TIMEOUT + 1.0)
|
||||
results[host] = dt # 失败时为 None
|
||||
lat = await asyncio.wait_for(
|
||||
self._probers[protocol](host, self._timeout),
|
||||
timeout=self._timeout + 2.0,
|
||||
)
|
||||
except Exception:
|
||||
results[host] = None
|
||||
lat = None
|
||||
results[host][protocol] = lat
|
||||
|
||||
await asyncio.gather(*(_probe_one(h) for h in hosts))
|
||||
|
||||
ok_hosts = {h: lt for h, lt in results.items() if lt is not None}
|
||||
# 按延迟升序排名
|
||||
ranked = sorted(ok_hosts.items(), key=lambda kv: kv[1])
|
||||
fast = set()
|
||||
for i, (h, lt) in enumerate(ranked):
|
||||
if i < FAST_POOL_TOP_N or lt < FAST_POOL_MAX_LATENCY:
|
||||
fast.add(h)
|
||||
|
||||
log.info("测速完成: 可用 %d 台, 入池 %d 台 (前%d或<%dms)",
|
||||
len(ok_hosts), len(fast), FAST_POOL_TOP_N,
|
||||
int(FAST_POOL_MAX_LATENCY * 1000))
|
||||
self._write_results(results, fast)
|
||||
|
||||
def _write_results(self, results: dict[str, float | None], fast: set[str]) -> None:
|
||||
t0 = time.monotonic()
|
||||
try:
|
||||
cfg = json.loads(self._file.read_text(encoding="utf-8"))
|
||||
except (OSError, ValueError):
|
||||
cfg = {"servers": []}
|
||||
servers = cfg.get("servers", [])
|
||||
by_ip = {s.get("ip"): s for s in servers}
|
||||
await asyncio.wait_for(
|
||||
asyncio.gather(*(_one(p, ip) for p, ip in jobs)),
|
||||
timeout=self._budget,
|
||||
)
|
||||
except TimeoutError:
|
||||
# 超时:采用已完成的部分结果(部分可用也强于完全没有)
|
||||
log.debug("测速整体超过 %.0fs 预算,采用已完成的部分结果", self._budget)
|
||||
except Exception as exc:
|
||||
log.debug("测速过程异常: %s", str(exc)[:120])
|
||||
elapsed = time.monotonic() - t0
|
||||
|
||||
today = date.today().isoformat()
|
||||
now = time.strftime("%Y-%m-%d %H:%M:%S")
|
||||
checked_any = False
|
||||
for host, lt in results.items():
|
||||
entry = by_ip.get(host)
|
||||
if entry is None:
|
||||
entry = {"ip": host, "name": host, "protocols": ["standard"]}
|
||||
servers.append(entry)
|
||||
by_ip[host] = entry
|
||||
if lt is not None:
|
||||
checked_any = True
|
||||
entry["status"] = "ok" if host in fast else "slow"
|
||||
entry["latency"] = round(lt, 3)
|
||||
entry["fast_pool"] = host in fast
|
||||
else:
|
||||
# 探测不可达:不降级已有 ok(避免误杀),仅标记本次不可达
|
||||
entry.setdefault("status", "unknown")
|
||||
entry["latency"] = None
|
||||
entry["fast_pool"] = False
|
||||
entry["checked_date"] = today
|
||||
entry["checked_at"] = now
|
||||
responsive = {
|
||||
ip: {p: lat for p, lat in per.items() if lat is not None}
|
||||
for ip, per in results.items()
|
||||
}
|
||||
responsive = {ip: per for ip, per in responsive.items() if per}
|
||||
if not responsive:
|
||||
log.debug("测速结束: 无任何可达主机,不写缓存(下次启动重试)")
|
||||
return None
|
||||
|
||||
# 若有 ok 服务器写入可入池日期;否则保留今日以免反复重试
|
||||
cfg["checked_date"] = today
|
||||
cfg["note"] = (
|
||||
f"当日全量测速入库({today}): 入池={len(fast)}台 "
|
||||
f"(排名前{FAST_POOL_TOP_N}或延迟<{int(FAST_POOL_MAX_LATENCY*1000)}ms)"
|
||||
def _ordered(protocol: str) -> list[str]:
|
||||
rows = [(per[protocol], ip) for ip, per in responsive.items()
|
||||
if protocol in per]
|
||||
rows.sort(key=lambda kv: (kv[0], kv[1]))
|
||||
return [ip for _, ip in rows]
|
||||
|
||||
hosts_by_proto = {p: _ordered(p) for p in ("standard", "mac", "ex")}
|
||||
overall = sorted(
|
||||
(min(per.values()), ip) for ip, per in responsive.items()
|
||||
)
|
||||
if not checked_any:
|
||||
# 全部不可达时不写 checked_date,下次启动重试
|
||||
cfg.pop("checked_date", None)
|
||||
fast = {ip for i, (lat, ip) in enumerate(overall)
|
||||
if i < FAST_POOL_TOP_N or lat < FAST_POOL_MAX_LATENCY}
|
||||
|
||||
self._file.write_text(
|
||||
json.dumps(cfg, ensure_ascii=False, indent=2), encoding="utf-8"
|
||||
)
|
||||
log.info("测速结果已写入 %s", self._file)
|
||||
log.debug("测速完成: 候选 %d 台, 可达 %d 台, fast %d 台, 耗时 %.2fs",
|
||||
len({ip for _, ip in jobs}), len(responsive), len(fast), elapsed)
|
||||
return {
|
||||
"version": PROBE_CACHE_VERSION,
|
||||
"date": self.today(),
|
||||
"probed_at": time.strftime("%Y-%m-%d %H:%M:%S"),
|
||||
"elapsed": round(elapsed, 3),
|
||||
# 每个协议的完整探测结果(含 None = 本次不可达),便于事后诊断
|
||||
"results": {ip: dict(per) for ip, per in results.items() if per},
|
||||
# 按延迟升序的可达主机(运行时即按此顺序取用)
|
||||
"hosts": hosts_by_proto,
|
||||
# 诊断用:符合"前N名或<200ms"的可达主机
|
||||
"fast": {p: [h for h in ips if h in fast]
|
||||
for p, ips in hosts_by_proto.items()},
|
||||
}
|
||||
|
||||
def _write(self, payload: dict) -> None:
|
||||
"""原子写入:先写临时文件再 ``os.replace``,避免读到半截 JSON。"""
|
||||
target = self.cache_file()
|
||||
tmp = target.with_name(f"{target.name}.tmp{os.getpid()}")
|
||||
try:
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
tmp.write_text(json.dumps(payload, ensure_ascii=False, indent=2),
|
||||
encoding="utf-8")
|
||||
os.replace(tmp, target)
|
||||
log.debug("当日测速结果已缓存: %s", target)
|
||||
except OSError as exc:
|
||||
log.debug("测速结果写入失败(忽略): %s", str(exc)[:120])
|
||||
try:
|
||||
tmp.unlink(missing_ok=True)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
# 进程级单例(跨池共享,确保当日仅测一次)
|
||||
# 进程级单例(标准 / MAC / EX 三池共享,确保当日只真正测一次)
|
||||
_probe_cache = ProbeCache()
|
||||
|
||||
|
||||
def get_probe_cache() -> ProbeCache:
|
||||
return _probe_cache
|
||||
|
||||
|
||||
def schedule_probe(hosts: list[str] | None = None) -> asyncio.Task | None:
|
||||
"""[连接池用] 把当日测速挂到后台并返回 Task,供调用方持有 / 取消。
|
||||
|
||||
返回 ``None`` 表示当前没有运行中的事件循环(此时无法起后台任务)。
|
||||
"""
|
||||
try:
|
||||
return asyncio.create_task(
|
||||
get_probe_cache().ensure_probe(hosts), name="pytdxdata-daily-probe"
|
||||
)
|
||||
except RuntimeError:
|
||||
return None
|
||||
@@ -4,9 +4,24 @@ from __future__ import annotations
|
||||
import struct
|
||||
|
||||
from ..._binary import unpack_from
|
||||
from ...models import ExMarket
|
||||
from ..datetime_ import get_datetime
|
||||
from .base import BaseCommand
|
||||
|
||||
# 0x23FF 的「槽位 5 是成交额」的市场集合(其余市场该槽位是持仓量、槽位 7 是结算价)。
|
||||
# 实测依据(2026-10-28,日线,逐条校验 成交额/(close×vol)):
|
||||
# 31 港股主板 = 100.0(vol 单位手,1手=100股)、49 港股基金 = 100.3、
|
||||
# 48 港股创业板 ≈1.0、44 新三板 ≈0.99、74 美股 ≈1.0(vol 单位股)。
|
||||
# ⚠️ 27 港股指数 / 62 中证指数 该槽位既非成交额也非结算价(量级对不上),故**不在**表内,
|
||||
# 保持 amount=0.0 的原始行为,避免给出可疑数值。
|
||||
_EX_AMOUNT_MARKETS: frozenset[int] = frozenset({
|
||||
int(ExMarket.HK_MAIN_BOARD),
|
||||
int(ExMarket.HK_GEM),
|
||||
int(ExMarket.HK_FUND),
|
||||
int(ExMarket.NEEQ),
|
||||
int(ExMarket.US_STOCK),
|
||||
})
|
||||
|
||||
# pytdx EX setup 握手命令(完整84B: 12B帧头+72B body, 0x2454)—— 与MAC EX login同ID不同body
|
||||
EX_SETUP_CMD = bytes.fromhex("0101486500015200520054241f32c6e5d53dfb411f32c6e5d53dfb411f32c6e5d53dfb411f32c6e5d53dfb411f32c6e5d53dfb411f32c6e5d53dfb411f32c6e5d53dfb411f32c6e5d53dfb41cce16dffd5ba3fb8cbc57a054f7748ea")
|
||||
|
||||
@@ -57,7 +72,24 @@ class GetMarketsCmd(BaseCommand[list[dict]]):
|
||||
|
||||
|
||||
class GetExInstrumentBarsCmd(BaseCommand[list[dict]]):
|
||||
"""EX商品K线(0x23FF, 32B/条: 4B时间戳+28B行情)。"""
|
||||
"""EX商品K线(0x23FF, 32B/条: 4B时间戳+28B行情)。
|
||||
|
||||
后 28B 的字段语义**按市场分两类**(2026-10-28 全市场实测,见 ``_EX_AMOUNT_MARKETS``):
|
||||
|
||||
* 权益类(港股主板/创业板/基金、新三板、美股)::
|
||||
|
||||
open(4) high(4) low(4) close(4) 成交额(4f) 成交量(4I) 保留(4f)
|
||||
|
||||
成交额可用;校验 ``成交额 / (close × 成交量)`` = 100.0(港股主板/基金,
|
||||
``vol`` 单位是「手」=100 股)或 ≈1.0(港股创业板/新三板/美股,``vol`` 单位是股)。
|
||||
|
||||
* 期货期权类(中金所/上期所/郑商所/大商所及各期权)::
|
||||
|
||||
open(4) high(4) low(4) close(4) 持仓量(4I) 成交量(4I) 结算价(4f)
|
||||
|
||||
**协议帧内没有成交额**(``amount`` 恒 0.0,不是解析漏读);末位 float 是结算价,
|
||||
日线有值、分钟线为 0(``settle`` 归一化成 ``None``)。
|
||||
"""
|
||||
|
||||
def __init__(self, market: int, code: str, category: int = 0,
|
||||
start: int = 0, count: int = 700) -> None:
|
||||
@@ -76,6 +108,7 @@ class GetExInstrumentBarsCmd(BaseCommand[list[dict]]):
|
||||
if len(body) < 20:
|
||||
return []
|
||||
(ret_count,) = unpack_from("<H", body, 18, "ex bars count")
|
||||
amount_market = self.market in _EX_AMOUNT_MARKETS
|
||||
items: list[dict] = []
|
||||
off = 20
|
||||
for _ in range(ret_count):
|
||||
@@ -83,14 +116,23 @@ class GetExInstrumentBarsCmd(BaseCommand[list[dict]]):
|
||||
break
|
||||
# 时间4字节按category分格式:分钟级=2B zipday+2B tminutes;日线及以上=YYYYMMDD
|
||||
year, month, day, hour, minute, _ = get_datetime(self.category, body, off)
|
||||
open_, high, low, close, position, trade, _ = struct.unpack(
|
||||
"<ffffIIf", body[off + 4: off + 32])
|
||||
items.append({
|
||||
raw = body[off + 4: off + 32]
|
||||
open_, high, low, close = struct.unpack("<ffff", raw[:16])
|
||||
item = {
|
||||
"year": year, "month": month, "day": day,
|
||||
"hour": hour, "minute": minute,
|
||||
"open": open_, "high": high, "low": low, "close": close,
|
||||
"position": position, "trade": trade,
|
||||
})
|
||||
}
|
||||
if amount_market:
|
||||
# 权益类:槽位 5 = 成交额(float),槽位 7 语义未知(保留不用)
|
||||
amount, trade, _reserved = struct.unpack("<fIf", raw[16:28])
|
||||
item.update(amount=amount, trade=trade, position=0, settle=None)
|
||||
else:
|
||||
# 期货/期权:槽位 5 = 持仓量,槽位 7 = 结算价(分钟线为 0 → None)
|
||||
position, trade, settle = struct.unpack("<IIf", raw[16:28])
|
||||
item.update(amount=0.0, trade=trade, position=position,
|
||||
settle=settle or None)
|
||||
items.append(item)
|
||||
off += 32
|
||||
return items
|
||||
|
||||
|
||||
@@ -12,20 +12,155 @@ from .base import BaseCommand
|
||||
MAX_QUOTES = 80
|
||||
|
||||
|
||||
def price_decimal_digits(market: int, code: str) -> int:
|
||||
"""推断报价有效小数位:股票2位(/100)、指数/ETF/基金/转债/国债3位(/1000)。"""
|
||||
code = (code or "").strip().rstrip("\x00")
|
||||
if market == 1: # SH
|
||||
if code.startswith(("5", "000", "8")):
|
||||
return 3
|
||||
return 2
|
||||
if market == 0: # SZ
|
||||
if code.startswith("1"):
|
||||
return 3
|
||||
return 2
|
||||
# ====================================================================== #
|
||||
# 权威小数位表(由证券清单 SecurityInfo.decimal_point 灌入)
|
||||
# ====================================================================== #
|
||||
# 查找顺序:**权威表 → 静态前缀规则**。权威表来自 ``get_universe`` 返回的
|
||||
# ``SecurityInfo.decimal_point``(协议自带,见 protocol/commands/security_list.py),
|
||||
# 由 ``pytdxdata.decimal_sync`` 在后台每日同步一次;未同步时静默退回静态规则,
|
||||
# 行为与 0.6.5 完全一致。
|
||||
_DECIMAL_PREFIX: dict[tuple[int, str], int] = {}
|
||||
_DECIMAL_CODE: dict[tuple[int, str], int] = {}
|
||||
_DECIMAL_MIXED: set[tuple[int, str]] = set()
|
||||
_DECIMAL_SOURCE: str = ""
|
||||
|
||||
|
||||
def install_decimal_map(
|
||||
items: "list[tuple[int, str, int]] | tuple[tuple[int, str, int], ...]",
|
||||
*,
|
||||
source: str = "",
|
||||
) -> dict[str, int]:
|
||||
"""用证券清单灌入权威小数位表。
|
||||
|
||||
``items`` 为 ``(market, code, decimal_point)`` 序列(直接来自
|
||||
``SecurityInfo``)。按**三位代码前缀**归并成一个极小的表(全量 5 万只标的
|
||||
只产生数百个前缀);若某个前缀出现**不同**小数位,则该前缀退化为逐代码查表,
|
||||
只把这些例外代码存进 ``_DECIMAL_CODE``——这样即使将来交易所新增代码段,
|
||||
也不会因为前缀规则过时而算错价格。
|
||||
|
||||
返回统计:``{"prefixes", "codes", "mixed_prefixes", "stock"}``。
|
||||
"""
|
||||
global _DECIMAL_SOURCE
|
||||
votes: dict[tuple[int, str], dict[int, int]] = {}
|
||||
per_code: dict[tuple[int, str], int] = {}
|
||||
for market, code, dp in items:
|
||||
c = str(code or "").strip().rstrip("\x00")
|
||||
if len(c) < 3:
|
||||
continue
|
||||
m = int(market)
|
||||
if m not in (0, 1):
|
||||
continue
|
||||
d = int(dp)
|
||||
if d <= 0:
|
||||
continue
|
||||
per_code[(m, c)] = d
|
||||
key = (m, c[:3])
|
||||
bucket = votes.setdefault(key, {})
|
||||
bucket[d] = bucket.get(d, 0) + 1
|
||||
|
||||
_DECIMAL_PREFIX.clear()
|
||||
_DECIMAL_CODE.clear()
|
||||
_DECIMAL_MIXED.clear()
|
||||
for key, bucket in votes.items():
|
||||
if len(bucket) == 1:
|
||||
_DECIMAL_PREFIX[key] = next(iter(bucket))
|
||||
else:
|
||||
_DECIMAL_MIXED.add(key)
|
||||
for (m, c), d in per_code.items():
|
||||
if (m, c[:3]) in _DECIMAL_MIXED:
|
||||
_DECIMAL_CODE[(m, c)] = d
|
||||
_DECIMAL_SOURCE = source or "universe"
|
||||
return decimal_map_stats()
|
||||
|
||||
|
||||
def clear_decimal_map() -> None:
|
||||
"""清空权威表(回到纯静态前缀规则;测试与排障用)。"""
|
||||
global _DECIMAL_SOURCE
|
||||
_DECIMAL_PREFIX.clear()
|
||||
_DECIMAL_CODE.clear()
|
||||
_DECIMAL_MIXED.clear()
|
||||
_DECIMAL_SOURCE = ""
|
||||
|
||||
|
||||
def decimal_map_stats() -> dict[str, int]:
|
||||
"""权威表状态:已覆盖前缀数 / 例外代码数 / 混合前缀数 / 总条目数。"""
|
||||
return {
|
||||
"prefixes": len(_DECIMAL_PREFIX),
|
||||
"codes": len(_DECIMAL_CODE),
|
||||
"mixed_prefixes": len(_DECIMAL_MIXED),
|
||||
"stock": len(_DECIMAL_PREFIX) + len(_DECIMAL_CODE),
|
||||
}
|
||||
|
||||
|
||||
def decimal_map_source() -> str:
|
||||
"""权威表来源标记(``universe`` / ``cache`` / ``""``=未加载)。"""
|
||||
return _DECIMAL_SOURCE
|
||||
|
||||
|
||||
def _decimal_digits_static(market: int, code: str) -> int:
|
||||
"""静态前缀规则(**兜底**,权威表缺失时使用)。"""
|
||||
if market == 1: # 沪市
|
||||
if code.startswith("5"):
|
||||
return 3 # 基金/ETF/LOF(50/51/52/56/58 系)
|
||||
if code.startswith("900"):
|
||||
return 3 # B股
|
||||
if code.startswith(("000", "36", "88", "99", "7", "6")):
|
||||
return 2 # 指数/板块/优先股/配售/股票(60/601/603/605/688)
|
||||
return 4 # 债券(01x/1xx/2xx)、回购(204xxx)、ABS
|
||||
if market == 0: # 深市
|
||||
if code.startswith(("080", "158", "159", "160", "161", "162", "163",
|
||||
"164", "165", "166", "167", "168", "169",
|
||||
"180", "181")):
|
||||
return 3 # 基金/ETF/LOF/配债
|
||||
if code.startswith(("000", "001", "002", "003", "070", "140",
|
||||
"200", "201", "300", "301", "302", "395", "399")):
|
||||
return 2 # 股票/指数/B股/优先股
|
||||
return 4 # 债券(10x-14x/19x)、回购(131xxx)、ABS(50x/52x/56x)
|
||||
return 2
|
||||
|
||||
|
||||
def price_decimal_digits(market: int, code: str) -> int:
|
||||
"""报价小数位:**协议原始整数除以 ``10**该值``** 才是真实价格。
|
||||
|
||||
查找顺序:
|
||||
|
||||
1. **权威表**(:func:`install_decimal_map` 灌入)—— 来自证券清单里协议自带的
|
||||
``SecurityInfo.decimal_point``,由 ``pytdxdata.decimal_sync`` 每日后台同步一次。
|
||||
2. **静态前缀规则**(:func:`_decimal_digits_static`)—— 权威表未就绪时兜底。
|
||||
四档:股票/指数/板块 ``2``、基金/ETF/B股 ``3``、债券/回购 ``4``。
|
||||
|
||||
为什么还要静态规则:``get_quotes`` 解析是**纯同步、无网络**的路径,解析那一刻
|
||||
可能还没同步过证券清单;而现网首次安装/离线环境也必须能算出价格。
|
||||
|
||||
权威表与静态规则的一致性:2026-10-22 拉取全量 51,982 只(沪 27,520 + 深 24,462)
|
||||
逐条比对,静态规则与 ``decimal_point`` **一一吻合、零例外**——因为全量清单里
|
||||
没有任何一个三位前缀出现混合小数位。因此两者并存不会产生口径冲突。
|
||||
|
||||
历史修正(两次都是同一类错误:把某段前缀想当然地归错档):
|
||||
|
||||
- **2026-10-21**:沪市 ``000``/``8`` 被判成 3 位 → 上证指数/上证50/科创50/
|
||||
板块指数**价格少 10 倍**(3713 读成 371)。它们实际是 2 位。
|
||||
- **2026-10-22**:债券与回购全线偏差 —— 沪市 ``010/018/019/020/1xx/2xx``
|
||||
与深市 ``10x/11x/12x/13x/14x/19x/50x/52x/56x`` 实际是 **4 位**,原先
|
||||
沪市落到 2 位(**少 100 倍**)、深市落到 3 位(**少 10 倍**)。实测例:
|
||||
``sh019742`` 110.228 读成 11022.8、``sh204001`` 1.385 读成 138.5、
|
||||
``sz131810`` 1.38 读成 13.8。同时沪市 B 股 ``900xxx`` 是 3 位(原先 2 位)。
|
||||
|
||||
实证方法:拿 ``pre_close``(快照口径)除以上一交易日**日线收盘**(K线口径,
|
||||
走 ``/1000`` 独立路径),比值即标度误差 —— 1.0 正确、10 少 10 倍、100 少 100 倍。
|
||||
"""
|
||||
code = (code or "").strip().rstrip("\x00")
|
||||
m = int(market)
|
||||
if m in (0, 1):
|
||||
hit = _DECIMAL_PREFIX.get((m, code[:3]))
|
||||
if hit is not None:
|
||||
return hit
|
||||
hit = _DECIMAL_CODE.get((m, code))
|
||||
if hit is not None:
|
||||
return hit
|
||||
return _decimal_digits_static(m, code)
|
||||
|
||||
|
||||
def format_server_time(raw: int) -> str:
|
||||
"""服务器时间编码:小时 + 百万分之一小时。14999212 -> 14:59:57.163"""
|
||||
hours, fractional_hour = divmod(raw, 1_000_000)
|
||||
|
||||
@@ -36,8 +36,12 @@ class GetSecurityListCmd(BaseCommand[list[SecurityInfo]]):
|
||||
self.start = start
|
||||
|
||||
def build_request(self) -> bytes:
|
||||
# 12 字节头 + market(H) + start(H) = **16 字节**。
|
||||
# ⚠️ 曾经多塞一个 H(18 字节):服务端只消费前 16 字节,剩下的 00 00
|
||||
# 留在它的接收缓冲里,会把**下一个请求**的开头吃掉 → 该连接的后续请求
|
||||
# 永远得不到响应(表现为「第一次成功、第二次起全部通信超时」)。
|
||||
header = bytes.fromhex("0c0118640101060006005004".replace(" ", ""))
|
||||
return header + struct.pack("<HHH", self.market, self.start, 0)
|
||||
return header + struct.pack("<HH", self.market, self.start)
|
||||
|
||||
def parse_response(self, body: bytes) -> list[SecurityInfo]:
|
||||
(num,) = unpack_from("<H", body, 0, "security list header")
|
||||
|
||||
@@ -5,6 +5,7 @@ import struct
|
||||
|
||||
from ..._binary import unpack_from
|
||||
from ...models import TransactionRecord
|
||||
from ...symbols import has_order_count
|
||||
from ..datetime_ import get_time
|
||||
from ..mac_frame import build_mac_request
|
||||
from ..price import get_price
|
||||
@@ -32,6 +33,8 @@ class GetTransactionDataCmd(BaseCommand[list[TransactionRecord]]):
|
||||
pos = 2
|
||||
last_price = 0
|
||||
records: list[TransactionRecord] = []
|
||||
code = self.code.decode("utf-8")
|
||||
has_tc = has_order_count(self.market, code)
|
||||
for _ in range(num):
|
||||
hour, minute, pos = get_time(body, pos)
|
||||
price_diff, pos = get_price(body, pos)
|
||||
@@ -41,10 +44,12 @@ class GetTransactionDataCmd(BaseCommand[list[TransactionRecord]]):
|
||||
_unknown, pos = get_price(body, pos)
|
||||
last_price += price_diff
|
||||
records.append(TransactionRecord(
|
||||
market=self.market, code=self.code.decode("utf-8"),
|
||||
market=self.market, code=code,
|
||||
time=f"{hour:02d}:{minute:02d}:00",
|
||||
price=last_price / 100.0, vol=float(vol),
|
||||
trade_count=int(num_orders), bs_flag=bs_flag,
|
||||
# 指数/ETF/LOF 该字段是「价格×1000」占位,不是笔数
|
||||
trade_count=int(num_orders) if has_tc else 0,
|
||||
bs_flag=bs_flag, trade_count_available=has_tc,
|
||||
))
|
||||
return records
|
||||
|
||||
@@ -68,6 +73,7 @@ class GetHistoryTransactionDataCmd(BaseCommand[list[TransactionRecord]]):
|
||||
pos = 6 # 2(num) + 4(skip)
|
||||
last_price = 0
|
||||
records: list[TransactionRecord] = []
|
||||
code = self.code.decode("utf-8")
|
||||
for _ in range(num):
|
||||
hour, minute, pos = get_time(body, pos)
|
||||
price_diff, pos = get_price(body, pos)
|
||||
@@ -76,10 +82,11 @@ class GetHistoryTransactionDataCmd(BaseCommand[list[TransactionRecord]]):
|
||||
_unknown, pos = get_price(body, pos)
|
||||
last_price += price_diff
|
||||
records.append(TransactionRecord(
|
||||
market=self.market, code=self.code.decode("utf-8"),
|
||||
market=self.market, code=code,
|
||||
time=f"{hour:02d}:{minute:02d}:00",
|
||||
price=last_price / 100.0, vol=float(vol),
|
||||
trade_count=0, bs_flag=bs_flag, # 历史无笔数
|
||||
trade_count=0, bs_flag=bs_flag,
|
||||
trade_count_available=False, # 历史通道整条链路都没有笔数字段
|
||||
))
|
||||
return records
|
||||
|
||||
@@ -108,11 +115,15 @@ class MacTransactionCmd(BaseCommand[list[TransactionRecord]]):
|
||||
def parse_response(self, body: bytes) -> list[TransactionRecord]:
|
||||
(count,) = unpack_from("<H", body, 29, "mac transaction count")
|
||||
records: list[TransactionRecord] = []
|
||||
has_tc = has_order_count(self.market, self.code)
|
||||
for i in range(count):
|
||||
offset = 39 + i * 18
|
||||
try:
|
||||
# trade_count 必须按**有符号** int32 解析:集合竞价段(bs_flag=8)
|
||||
# 该字段实为负值,按 uint32 会读出 42 亿量级的脏数据。
|
||||
# 实测(2026-10-21 sh600000):09:15:03 原始 96fcffff = -874。
|
||||
time_sec, price, volume, trade_count, bs_flag = unpack_from(
|
||||
"<IfIIH", body, offset, "mac transaction item"
|
||||
"<IfIiH", body, offset, "mac transaction item"
|
||||
)
|
||||
except ValueError:
|
||||
break
|
||||
@@ -121,6 +132,8 @@ class MacTransactionCmd(BaseCommand[list[TransactionRecord]]):
|
||||
market=self.market, code=self.code,
|
||||
time=f"{h:02d}:{m:02d}:{s:02d}",
|
||||
price=price, vol=float(volume),
|
||||
trade_count=int(trade_count), bs_flag=bs_flag,
|
||||
# 指数/ETF/LOF 该字段是「价格×1000」占位,不是笔数
|
||||
trade_count=int(trade_count) if has_tc else 0,
|
||||
bs_flag=bs_flag, trade_count_available=has_tc,
|
||||
))
|
||||
return records
|
||||
@@ -27,6 +27,7 @@ class PageSpec:
|
||||
ordered: bool = True # 按 start 升序拼接
|
||||
min_page: int = 100 # 规避标准K线 count<100 空响应坑
|
||||
start_offset: int = 0 # 起始偏移(从该偏移开始分页,避免拉取历史全量)
|
||||
newest_first: bool = False # 服务端 start 从最新往回数(0x122F 逐笔即如此)
|
||||
|
||||
|
||||
@dataclass(slots=True)
|
||||
@@ -36,6 +37,15 @@ class PageResult:
|
||||
items: list[Any]
|
||||
|
||||
|
||||
class _Slot:
|
||||
"""worker 当前持有的连接槽:失败换连时原地更新,保证最终能精确归还。"""
|
||||
|
||||
__slots__ = ("pc",)
|
||||
|
||||
def __init__(self, pc: PooledConnection) -> None:
|
||||
self.pc: PooledConnection | None = pc
|
||||
|
||||
|
||||
class Paginator:
|
||||
"""用至多 concurrency 个连接并行拉取分页数据。"""
|
||||
|
||||
@@ -64,23 +74,28 @@ class Paginator:
|
||||
conns = await self.pool.acquire_many(eff_conc)
|
||||
if not conns:
|
||||
raise PoolTimeoutError("连接池获取超时(无可用连接)")
|
||||
# 每个 worker 一个连接槽:中途换连会更新槽内引用,
|
||||
# 归还时按槽归还(否则换掉的新连接会漏还,把池耗干)
|
||||
slots = [_Slot(pc) for pc in conns]
|
||||
try:
|
||||
stop = asyncio.Event()
|
||||
out: list[PageResult] = []
|
||||
out_lock = asyncio.Lock()
|
||||
k = len(conns)
|
||||
k = len(slots)
|
||||
workers = [
|
||||
asyncio.create_task(
|
||||
self._worker(wid, conns[wid % k], jobs[wid::k], cmd_factory,
|
||||
self._worker(slots[wid], jobs[wid::k], cmd_factory,
|
||||
stop, out, out_lock, spec)
|
||||
)
|
||||
for wid in range(k)
|
||||
]
|
||||
await asyncio.gather(*workers)
|
||||
finally:
|
||||
for pc in conns:
|
||||
await self.pool.release(pc)
|
||||
return self._assemble(out, spec.ordered)
|
||||
for s in slots:
|
||||
if s.pc is not None:
|
||||
pc, s.pc = s.pc, None
|
||||
await self.pool.release(pc)
|
||||
return self._assemble(out, spec.ordered, spec.newest_first)
|
||||
|
||||
def _plan_starts(self, spec: PageSpec) -> list[tuple[int, int]]:
|
||||
"""切页:返回 [(start, count), ...]。"""
|
||||
@@ -102,53 +117,54 @@ class Paginator:
|
||||
return jobs
|
||||
|
||||
async def _worker(
|
||||
self, wid: int, conn: PooledConnection, jobs: list[tuple[int, int]],
|
||||
self, slot: _Slot, jobs: list[tuple[int, int]],
|
||||
cmd_factory: Callable[[int, int], Any], stop: asyncio.Event,
|
||||
out: list[PageResult], out_lock: asyncio.Lock, spec: PageSpec,
|
||||
) -> None:
|
||||
current = conn
|
||||
for start, count in jobs:
|
||||
if stop.is_set():
|
||||
if stop.is_set() or slot.pc is None:
|
||||
return
|
||||
items, current = await self._fetch_one(
|
||||
current, cmd_factory(start, count), count)
|
||||
if items is None:
|
||||
continue # 该页失败(strict=False 时留空)
|
||||
items, ok = await self._fetch_one(slot, cmd_factory(start, count), count)
|
||||
if not ok:
|
||||
return # 换连接失败 → 本 worker 退出(该页留空)
|
||||
async with out_lock:
|
||||
out.append(PageResult(start=start, count=count, items=items))
|
||||
if spec.stop_when_short and len(items) < count:
|
||||
stop.set()
|
||||
return
|
||||
|
||||
async def _fetch_one(self, conn: PooledConnection, cmd, page_size: int):
|
||||
async def _fetch_one(self, slot: _Slot, cmd, page_size: int):
|
||||
"""单页拉取,失败换连接重试。
|
||||
|
||||
返回 (items, current_conn) — current_conn 始终指向当前可用连接,
|
||||
worker 据此更新自己的连接引用,避免跨页使用已关闭连接。
|
||||
返回 ``(items, ok)``。失败时连接一律 ``broken=True`` 归还并把槽清空,
|
||||
调用方据此退出,不会留下悬空引用。
|
||||
"""
|
||||
current = conn
|
||||
for attempt in range(self.retries + 1):
|
||||
try:
|
||||
result = await current.execute(cmd)
|
||||
return (list(result) if result is not None else []), current
|
||||
result = await slot.pc.execute(cmd) # type: ignore[union-attr]
|
||||
return (list(result) if result is not None else []), True
|
||||
except Exception as e:
|
||||
if slot.pc is not None:
|
||||
await self.pool.release(slot.pc, broken=True)
|
||||
slot.pc = None
|
||||
if attempt >= self.retries:
|
||||
if self.strict:
|
||||
raise
|
||||
log.warning("分页拉取失败(strict=False留空): %s", e)
|
||||
return None, current
|
||||
# 换连接重试
|
||||
await self.pool.release(current, broken=True)
|
||||
return None, False
|
||||
try:
|
||||
current = await self.pool.acquire(timeout=3.0)
|
||||
slot.pc = await self.pool.acquire(timeout=3.0)
|
||||
except PoolTimeoutError:
|
||||
log.warning("分页重试获取连接失败,跳过该页: %s", e)
|
||||
return None, current
|
||||
return None, current
|
||||
return None, False
|
||||
return None, False
|
||||
|
||||
def _assemble(self, results: list[PageResult], ordered: bool) -> list[Any]:
|
||||
def _assemble(self, results: list[PageResult], ordered: bool,
|
||||
newest_first: bool = False) -> list[Any]:
|
||||
# newest_first=True:start=0 是最新一页,start 越大越旧 → 反向拼才对得上时间序。
|
||||
# 单页内部服务端已按时间升序,所以只翻页序,不翻页内元素。
|
||||
if ordered:
|
||||
results.sort(key=lambda r: r.start)
|
||||
results.sort(key=lambda r: r.start, reverse=newest_first)
|
||||
out: list[Any] = []
|
||||
for r in results:
|
||||
out.extend(r.items)
|
||||
|
||||
+113
-39
@@ -3,28 +3,43 @@
|
||||
读 servers.json → 并发探测每台可用性与延时(标准/MAC用7709、EX用7727)→ 写回状态。
|
||||
|
||||
用法:
|
||||
python -m pytdxdata.server_probe # 全量探测
|
||||
python -m pytdxdata.server_probe --group ex # 只测EX
|
||||
python -m pytdxdata.server_probe --refresh # 重新探测(status=ok的)
|
||||
python -m pytdxdata.server_probe # 按申报协议全量探测
|
||||
python -m pytdxdata.server_probe --discover # 额外测【每种协议】并据实修正 protocols
|
||||
python -m pytdxdata.server_probe --group ex # 只测EX
|
||||
python -m pytdxdata.server_probe --refresh # 只重测status=ok的
|
||||
|
||||
写回的字段
|
||||
----------
|
||||
``status`` / ``latency`` / ``checked_at`` / ``checked_date`` / ``fast_pool``
|
||||
以及顶层 ``checked_date`` / ``note``。其中 ``fast_pool`` 与运行时缓存里的 ``fast``
|
||||
用同一套规则(整体延迟前 10 名,或延迟 < 200ms),供"还没有当日缓存时"的冷启动排序。
|
||||
|
||||
``--discover`` 会顺带剔除 ip 非法的脏记录(如历史遗留的 ``"s1"``)。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import asyncio
|
||||
import json
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
from .config import is_valid_host
|
||||
from .models import Adjust, KlinePeriod
|
||||
from .pool.connection import TdxConnection
|
||||
from .pool.ex_connection import ExTdxConnection
|
||||
from .protocol.commands.ex_proto import GetExInstrumentBarsCmd
|
||||
from .protocol.commands.kline import GetSecurityBarsCmd
|
||||
from .protocol.commands.mac import MacKlineCmd
|
||||
from .protocol.commands.ex_proto import GetExInstrumentBarsCmd
|
||||
from .models import KlinePeriod, Adjust
|
||||
|
||||
SERVERS_FILE = Path(__file__).resolve().parent / "servers.json"
|
||||
|
||||
# 与 pool/probe_cache.py 保持同一套入池规则
|
||||
FAST_POOL_TOP_N = 10
|
||||
FAST_POOL_MAX_LATENCY = 0.2
|
||||
|
||||
ALL_PROTOCOLS = ("standard", "mac", "ex")
|
||||
|
||||
|
||||
def load_servers() -> dict:
|
||||
if not SERVERS_FILE.exists():
|
||||
@@ -41,7 +56,7 @@ async def probe_standard(ip: str, timeout: float = 5.0) -> tuple[bool, float]:
|
||||
t0 = time.monotonic()
|
||||
try:
|
||||
await conn.connect()
|
||||
bars = await conn.execute(GetSecurityBarsCmd(1, "600000", 5, 0, 100))
|
||||
bars = await conn.execute(GetSecurityBarsCmd(1, "600000", KlinePeriod.DAY, 0, 100))
|
||||
return (bool(bars), time.monotonic() - t0)
|
||||
except Exception:
|
||||
return (False, time.monotonic() - t0)
|
||||
@@ -75,25 +90,51 @@ async def probe_ex(ip: str, handshake: str, timeout: float = 5.0) -> tuple[bool,
|
||||
await conn.close()
|
||||
|
||||
|
||||
async def probe_one(server: dict, timeout: float) -> dict:
|
||||
async def probe_proto(proto: str, ip: str, handshake: str, timeout: float) -> tuple[bool, float]:
|
||||
"""按协议分派到具体探针。"""
|
||||
if proto == "ex":
|
||||
return await probe_ex(ip, handshake, timeout)
|
||||
if proto == "mac":
|
||||
return await probe_mac(ip, timeout)
|
||||
return await probe_standard(ip, timeout)
|
||||
|
||||
|
||||
async def probe_one(server: dict, timeout: float, discover: bool = False) -> dict:
|
||||
ip = server["ip"]
|
||||
protocols = server.get("protocols", [])
|
||||
best_lat = None
|
||||
ok = False
|
||||
# 逐协议探测(一台多协议: standard/mac同端口可复用连接——简化逐协议测)
|
||||
for proto in protocols:
|
||||
declared = server.get("protocols", [])
|
||||
handshake = server.get("handshake") or "setup"
|
||||
# discover: 三种协议全测(EX 先按申报握手,失败再试另一种);
|
||||
# 默认: 只测申报的协议(保持原行为,避免误改人工裁剪的清单)
|
||||
targets = list(ALL_PROTOCOLS) if discover else list(declared)
|
||||
|
||||
per: dict[str, float | None] = {}
|
||||
for proto in targets:
|
||||
if proto == "ex":
|
||||
r, lat = await probe_ex(ip, server.get("handshake", "setup"), timeout)
|
||||
elif proto == "mac":
|
||||
r, lat = await probe_mac(ip, timeout)
|
||||
ok, lat = await probe_ex(ip, handshake, timeout)
|
||||
if not ok and discover:
|
||||
# 握手模式申报错的节点:换一种再试(login <-> setup)
|
||||
alt = "setup" if handshake == "login" else "login"
|
||||
ok, lat = await probe_ex(ip, alt, timeout)
|
||||
else:
|
||||
r, lat = await probe_standard(ip, timeout)
|
||||
if r:
|
||||
ok = True
|
||||
best_lat = lat if best_lat is None else min(best_lat, lat)
|
||||
server["status"] = "ok" if ok else "fail"
|
||||
server["latency"] = round(best_lat, 3) if best_lat is not None else None
|
||||
server["checked_at"] = time.strftime("%Y-%m-%d %H:%M:%S")
|
||||
ok, lat = await probe_proto(proto, ip, handshake, timeout)
|
||||
if not ok and discover:
|
||||
# discover 会据结果**收窄 protocols**,单次抖动就会误删协议 → 失败重试一次
|
||||
ok, lat = await probe_proto(proto, ip, handshake, timeout)
|
||||
if ok:
|
||||
per[proto] = lat
|
||||
|
||||
if discover and per:
|
||||
# 据实测结果修正申报协议(保序:standard / mac / ex)
|
||||
server["protocols"] = [p for p in ALL_PROTOCOLS if p in per]
|
||||
|
||||
server["status"] = "ok" if per else "fail"
|
||||
server["latency"] = round(min(per.values()), 3) if per else None
|
||||
now = time.strftime("%Y-%m-%d %H:%M:%S")
|
||||
server["checked_at"] = now
|
||||
server["checked_date"] = now[:10]
|
||||
server.setdefault("fast_pool", False)
|
||||
# 供表格打印用,不落盘
|
||||
server["_per_proto"] = {p: (round(v, 3) if v is not None else None) for p, v in per.items()}
|
||||
return server
|
||||
|
||||
|
||||
@@ -102,10 +143,21 @@ async def main() -> None:
|
||||
ap.add_argument("--group", choices=["standard", "mac", "ex"], default=None)
|
||||
ap.add_argument("--timeout", type=float, default=5.0)
|
||||
ap.add_argument("--refresh", action="store_true", help="只重测status=ok的")
|
||||
ap.add_argument("--discover", action="store_true",
|
||||
help="三种协议全测并据实修正 protocols(同时剔除 ip 非法的脏记录)")
|
||||
args = ap.parse_args()
|
||||
|
||||
cfg = load_servers()
|
||||
servers = cfg["servers"]
|
||||
all_servers = cfg["servers"]
|
||||
|
||||
# --- 清理:ip 不合法的脏记录(历史遗留,如 "s1") ---
|
||||
junk = [s.get("ip") for s in all_servers if not is_valid_host(s.get("ip"))]
|
||||
if junk:
|
||||
cfg["servers"] = [s for s in all_servers if is_valid_host(s.get("ip"))]
|
||||
all_servers = cfg["servers"]
|
||||
print("已剔除 ip 非法记录: %s" % ", ".join(map(str, junk)))
|
||||
|
||||
servers = all_servers
|
||||
if args.group:
|
||||
servers = [s for s in servers if args.group in s.get("protocols", [])]
|
||||
if args.refresh:
|
||||
@@ -114,33 +166,55 @@ async def main() -> None:
|
||||
print("没有待探测的服务器")
|
||||
return
|
||||
|
||||
print("=" * 90)
|
||||
print("探测 %d 台服务器 (标准/MAC:7709, EX:7727)..." % len(servers))
|
||||
print("=" * 90)
|
||||
print("=" * 96)
|
||||
print("探测 %d 台服务器 (标准/MAC:7709, EX:7727)%s..." % (
|
||||
len(servers), " [discover: 全协议]" if args.discover else ""))
|
||||
print("=" * 96)
|
||||
t0 = time.time()
|
||||
results = await asyncio.gather(*[probe_one(s, args.timeout) for s in servers])
|
||||
results = await asyncio.gather(
|
||||
*[probe_one(s, args.timeout, args.discover) for s in servers]
|
||||
)
|
||||
elapsed = time.time() - t0
|
||||
|
||||
# 写回
|
||||
updated = {s["ip"]: s for s in results}
|
||||
# --- 入池标记:与运行时缓存同一规则(前N名 或 <200ms) ---
|
||||
ok_list = [s for s in results if s["status"] == "ok"]
|
||||
ok_list.sort(key=lambda s: s["latency"])
|
||||
fast_ips = {s["ip"] for i, s in enumerate(ok_list)
|
||||
if i < FAST_POOL_TOP_N or s["latency"] < FAST_POOL_MAX_LATENCY}
|
||||
for s in ok_list:
|
||||
s["fast_pool"] = s["ip"] in fast_ips
|
||||
fail_list = [s for s in results if s["status"] == "fail"]
|
||||
for s in fail_list:
|
||||
s["fast_pool"] = False
|
||||
|
||||
# --- 写回(剔除内部诊断字段) ---
|
||||
today = time.strftime("%Y-%m-%d")
|
||||
updated = {s["ip"]: {k: v for k, v in s.items() if not k.startswith("_")} for s in results}
|
||||
for s in cfg["servers"]:
|
||||
if s["ip"] in updated:
|
||||
s.update(updated[s["ip"]])
|
||||
cfg["checked_date"] = today
|
||||
cfg["version"] = today # 清单基线日期(无代码读取,仅标注本次复测时间)
|
||||
cfg["note"] = "离线探针复测(%s %s): 可达=%d/%d, 入池fast_pool=%d台" % (
|
||||
today, time.strftime("%H:%M"), len(ok_list), len(results), len(fast_ips))
|
||||
save_servers(cfg)
|
||||
|
||||
ok_list = [s for s in results if s["status"] == "ok"]
|
||||
ok_list.sort(key=lambda s: s["latency"])
|
||||
fail_list = [s for s in results if s["status"] == "fail"]
|
||||
print("\n可用: %d/%d (%.1fs)" % (len(ok_list), len(results), elapsed))
|
||||
print("-" * 90)
|
||||
print("\n可用: %d/%d (%.1fs) 入池: %d台" % (len(ok_list), len(results), elapsed, len(fast_ips)))
|
||||
print("-" * 96)
|
||||
for s in ok_list:
|
||||
print(" [OK] %-16s %-14s %-28s %.3fs %s" % (
|
||||
s["ip"], "/".join(s.get("protocols", [])), s["name"],
|
||||
s["latency"], s.get("handshake") or ""))
|
||||
per = s.get("_per_proto", {})
|
||||
detail = " ".join(
|
||||
"%s=%s" % (p, ("%.3fs" % v) if v is not None else "-")
|
||||
for p, v in per.items()
|
||||
) or "/".join(s.get("protocols", []))
|
||||
print(" [OK] %-16s %-24s %-24s %.3fs %s%s" % (
|
||||
s["ip"], "/".join(s.get("protocols", [])), s["name"], s["latency"],
|
||||
"★" if s["fast_pool"] else " ", detail))
|
||||
if fail_list:
|
||||
print("\n不可用: %d台" % len(fail_list))
|
||||
for s in fail_list:
|
||||
print(" [X] %-16s %s" % (s["ip"], "/".join(s.get("protocols", []))))
|
||||
print(" [X] %-16s %-24s %s" % (
|
||||
s["ip"], "/".join(s.get("protocols", [])), s["name"]))
|
||||
print("\n已写回 %s" % SERVERS_FILE)
|
||||
|
||||
|
||||
|
||||
+291
-148
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"version": "2026-08-12",
|
||||
"note": "当日全量测速入库(2026-09-08): 入池=9台 (排名前10或延迟<200ms)",
|
||||
"version": "2026-10-16",
|
||||
"note": "离线探针复测(2026-10-16 13:46): 可达=30/36, 入池fast_pool=10台",
|
||||
"servers": [
|
||||
{
|
||||
"ip": "139.9.52.158",
|
||||
@@ -11,11 +11,15 @@
|
||||
],
|
||||
"handshake": null,
|
||||
"status": "ok",
|
||||
"latency": null,
|
||||
"checked_at": "2026-08-21 11:23:14",
|
||||
"latency": 0.172,
|
||||
"checked_at": "2026-10-16 13:46:17",
|
||||
"source": "probe-2026-08-12",
|
||||
"fast_pool": false,
|
||||
"checked_date": "2026-08-21"
|
||||
"fast_pool": true,
|
||||
"checked_date": "2026-10-16",
|
||||
"_per_proto": {
|
||||
"standard": 0.172,
|
||||
"mac": 0.183
|
||||
}
|
||||
},
|
||||
{
|
||||
"ip": "103.251.85.94",
|
||||
@@ -25,42 +29,50 @@
|
||||
"mac"
|
||||
],
|
||||
"handshake": null,
|
||||
"status": "slow",
|
||||
"latency": 0.481,
|
||||
"checked_at": "2026-08-19 15:37:35",
|
||||
"status": "ok",
|
||||
"latency": 0.298,
|
||||
"checked_at": "2026-10-16 13:46:18",
|
||||
"source": "probe-2026-08-12",
|
||||
"fast_pool": false,
|
||||
"checked_date": "2026-08-19"
|
||||
"fast_pool": true,
|
||||
"checked_date": "2026-10-16",
|
||||
"_per_proto": {
|
||||
"standard": 0.316,
|
||||
"mac": 0.298
|
||||
}
|
||||
},
|
||||
{
|
||||
"ip": "58.67.221.146",
|
||||
"name": "中信短线宝21",
|
||||
"protocols": [
|
||||
"standard",
|
||||
"mac"
|
||||
],
|
||||
"handshake": null,
|
||||
"status": "ok",
|
||||
"latency": 0.254,
|
||||
"checked_at": "2026-09-08 10:05:12",
|
||||
"latency": 0.236,
|
||||
"checked_at": "2026-10-16 13:46:23",
|
||||
"source": "probe-2026-08-12",
|
||||
"fast_pool": true,
|
||||
"checked_date": "2026-09-08"
|
||||
"checked_date": "2026-10-16",
|
||||
"_per_proto": {
|
||||
"mac": 0.236
|
||||
}
|
||||
},
|
||||
{
|
||||
"ip": "59.36.5.11",
|
||||
"name": "安信",
|
||||
"protocols": [
|
||||
"standard",
|
||||
"mac"
|
||||
],
|
||||
"handshake": null,
|
||||
"status": "ok",
|
||||
"latency": 0.254,
|
||||
"checked_at": "2026-09-08 10:05:12",
|
||||
"latency": 0.213,
|
||||
"checked_at": "2026-10-16 13:46:17",
|
||||
"source": "probe-2026-08-12",
|
||||
"fast_pool": true,
|
||||
"checked_date": "2026-09-08"
|
||||
"checked_date": "2026-10-16",
|
||||
"_per_proto": {
|
||||
"mac": 0.213
|
||||
}
|
||||
},
|
||||
{
|
||||
"ip": "103.221.142.82",
|
||||
@@ -71,12 +83,17 @@
|
||||
"ex"
|
||||
],
|
||||
"handshake": "setup",
|
||||
"status": "slow",
|
||||
"latency": 0.487,
|
||||
"checked_at": "2026-08-19 15:37:35",
|
||||
"status": "ok",
|
||||
"latency": 0.316,
|
||||
"checked_at": "2026-10-16 13:46:13",
|
||||
"source": "probe-2026-08-12",
|
||||
"fast_pool": false,
|
||||
"checked_date": "2026-08-19"
|
||||
"checked_date": "2026-10-16",
|
||||
"_per_proto": {
|
||||
"standard": 0.316,
|
||||
"mac": 0.329,
|
||||
"ex": 0.539
|
||||
}
|
||||
},
|
||||
{
|
||||
"ip": "218.106.92.182",
|
||||
@@ -85,27 +102,30 @@
|
||||
"standard"
|
||||
],
|
||||
"handshake": null,
|
||||
"status": "ok",
|
||||
"latency": 0.371,
|
||||
"checked_at": "2026-09-08 10:05:12",
|
||||
"status": "fail",
|
||||
"latency": null,
|
||||
"checked_at": "2026-10-16 13:46:34",
|
||||
"source": "probe-2026-08-12",
|
||||
"fast_pool": true,
|
||||
"checked_date": "2026-09-08"
|
||||
"fast_pool": false,
|
||||
"checked_date": "2026-10-16",
|
||||
"_per_proto": {}
|
||||
},
|
||||
{
|
||||
"ip": "60.12.136.251",
|
||||
"name": "中信短线宝22",
|
||||
"protocols": [
|
||||
"standard",
|
||||
"mac"
|
||||
],
|
||||
"handshake": null,
|
||||
"status": "ok",
|
||||
"latency": 0.37,
|
||||
"checked_at": "2026-09-08 10:05:12",
|
||||
"latency": 0.682,
|
||||
"checked_at": "2026-10-16 13:46:24",
|
||||
"source": "probe-2026-08-12",
|
||||
"fast_pool": true,
|
||||
"checked_date": "2026-09-08"
|
||||
"fast_pool": false,
|
||||
"checked_date": "2026-10-16",
|
||||
"_per_proto": {
|
||||
"mac": 0.682
|
||||
}
|
||||
},
|
||||
{
|
||||
"ip": "218.106.92.183",
|
||||
@@ -114,42 +134,47 @@
|
||||
"standard"
|
||||
],
|
||||
"handshake": null,
|
||||
"status": "ok",
|
||||
"latency": 0.377,
|
||||
"checked_at": "2026-09-08 10:05:12",
|
||||
"status": "fail",
|
||||
"latency": null,
|
||||
"checked_at": "2026-10-16 13:46:34",
|
||||
"source": "probe-2026-08-12",
|
||||
"fast_pool": true,
|
||||
"checked_date": "2026-09-08"
|
||||
"fast_pool": false,
|
||||
"checked_date": "2026-10-16",
|
||||
"_per_proto": {}
|
||||
},
|
||||
{
|
||||
"ip": "60.12.136.250",
|
||||
"name": "中信短线宝26",
|
||||
"protocols": [
|
||||
"standard",
|
||||
"mac"
|
||||
],
|
||||
"handshake": null,
|
||||
"status": "ok",
|
||||
"latency": 0.373,
|
||||
"checked_at": "2026-09-08 10:05:12",
|
||||
"latency": 0.607,
|
||||
"checked_at": "2026-10-16 13:46:23",
|
||||
"source": "probe-2026-08-12",
|
||||
"fast_pool": true,
|
||||
"checked_date": "2026-09-08"
|
||||
"fast_pool": false,
|
||||
"checked_date": "2026-10-16",
|
||||
"_per_proto": {
|
||||
"mac": 0.607
|
||||
}
|
||||
},
|
||||
{
|
||||
"ip": "115.238.90.170",
|
||||
"name": "中信短线宝23",
|
||||
"protocols": [
|
||||
"standard",
|
||||
"mac"
|
||||
],
|
||||
"handshake": null,
|
||||
"status": "slow",
|
||||
"latency": 0.482,
|
||||
"checked_at": "2026-08-19 15:37:35",
|
||||
"status": "ok",
|
||||
"latency": 1.264,
|
||||
"checked_at": "2026-10-16 13:46:24",
|
||||
"source": "probe-2026-08-12",
|
||||
"fast_pool": false,
|
||||
"checked_date": "2026-08-19"
|
||||
"checked_date": "2026-10-16",
|
||||
"_per_proto": {
|
||||
"mac": 1.264
|
||||
}
|
||||
},
|
||||
{
|
||||
"ip": "220.178.55.86",
|
||||
@@ -158,12 +183,13 @@
|
||||
"standard"
|
||||
],
|
||||
"handshake": null,
|
||||
"status": "slow",
|
||||
"latency": 0.479,
|
||||
"checked_at": "2026-08-19 15:37:35",
|
||||
"status": "fail",
|
||||
"latency": null,
|
||||
"checked_at": "2026-10-16 13:46:34",
|
||||
"source": "probe-2026-08-12",
|
||||
"fast_pool": false,
|
||||
"checked_date": "2026-08-19"
|
||||
"checked_date": "2026-10-16",
|
||||
"_per_proto": {}
|
||||
},
|
||||
{
|
||||
"ip": "220.178.55.71",
|
||||
@@ -172,87 +198,98 @@
|
||||
"standard"
|
||||
],
|
||||
"handshake": null,
|
||||
"status": "slow",
|
||||
"latency": 0.47,
|
||||
"checked_at": "2026-08-19 15:37:35",
|
||||
"status": "fail",
|
||||
"latency": null,
|
||||
"checked_at": "2026-10-16 13:46:34",
|
||||
"source": "probe-2026-08-12",
|
||||
"fast_pool": false,
|
||||
"checked_date": "2026-08-19"
|
||||
"checked_date": "2026-10-16",
|
||||
"_per_proto": {}
|
||||
},
|
||||
{
|
||||
"ip": "180.153.18.170",
|
||||
"name": "中信短线宝1",
|
||||
"protocols": [
|
||||
"standard",
|
||||
"mac"
|
||||
],
|
||||
"handshake": null,
|
||||
"status": "slow",
|
||||
"latency": 0.438,
|
||||
"checked_at": "2026-08-19 15:37:35",
|
||||
"status": "ok",
|
||||
"latency": 0.612,
|
||||
"checked_at": "2026-10-16 13:46:23",
|
||||
"source": "probe-2026-08-12",
|
||||
"fast_pool": false,
|
||||
"checked_date": "2026-08-19"
|
||||
"checked_date": "2026-10-16",
|
||||
"_per_proto": {
|
||||
"mac": 0.612
|
||||
}
|
||||
},
|
||||
{
|
||||
"ip": "115.238.90.165",
|
||||
"name": "杭州J4",
|
||||
"protocols": [
|
||||
"standard",
|
||||
"mac"
|
||||
],
|
||||
"handshake": null,
|
||||
"status": "slow",
|
||||
"latency": 0.476,
|
||||
"checked_at": "2026-08-19 15:37:35",
|
||||
"status": "ok",
|
||||
"latency": 0.613,
|
||||
"checked_at": "2026-10-16 13:46:23",
|
||||
"source": "probe-2026-08-12",
|
||||
"fast_pool": false,
|
||||
"checked_date": "2026-08-19"
|
||||
"checked_date": "2026-10-16",
|
||||
"_per_proto": {
|
||||
"mac": 0.613
|
||||
}
|
||||
},
|
||||
{
|
||||
"ip": "60.191.117.167",
|
||||
"name": "杭州J1",
|
||||
"protocols": [
|
||||
"standard",
|
||||
"mac"
|
||||
],
|
||||
"handshake": null,
|
||||
"status": "ok",
|
||||
"latency": 0.362,
|
||||
"checked_at": "2026-09-08 10:05:12",
|
||||
"latency": 0.624,
|
||||
"checked_at": "2026-10-16 13:46:23",
|
||||
"source": "probe-2026-08-12",
|
||||
"fast_pool": true,
|
||||
"checked_date": "2026-09-08"
|
||||
"fast_pool": false,
|
||||
"checked_date": "2026-10-16",
|
||||
"_per_proto": {
|
||||
"mac": 0.624
|
||||
}
|
||||
},
|
||||
{
|
||||
"ip": "218.75.126.9",
|
||||
"name": "杭州J3",
|
||||
"protocols": [
|
||||
"standard",
|
||||
"mac"
|
||||
],
|
||||
"handshake": null,
|
||||
"status": "slow",
|
||||
"latency": 0.481,
|
||||
"checked_at": "2026-08-19 15:37:35",
|
||||
"status": "ok",
|
||||
"latency": 0.592,
|
||||
"checked_at": "2026-10-16 13:46:23",
|
||||
"source": "probe-2026-08-12",
|
||||
"fast_pool": false,
|
||||
"checked_date": "2026-08-19"
|
||||
"checked_date": "2026-10-16",
|
||||
"_per_proto": {
|
||||
"mac": 0.592
|
||||
}
|
||||
},
|
||||
{
|
||||
"ip": "115.238.56.198",
|
||||
"name": "杭州J2",
|
||||
"protocols": [
|
||||
"standard",
|
||||
"mac"
|
||||
],
|
||||
"handshake": null,
|
||||
"status": "ok",
|
||||
"latency": 0.362,
|
||||
"checked_at": "2026-09-08 10:05:12",
|
||||
"latency": 0.624,
|
||||
"checked_at": "2026-10-16 13:46:23",
|
||||
"source": "probe-2026-08-12",
|
||||
"fast_pool": true,
|
||||
"checked_date": "2026-09-08"
|
||||
"fast_pool": false,
|
||||
"checked_date": "2026-10-16",
|
||||
"_per_proto": {
|
||||
"mac": 0.624
|
||||
}
|
||||
},
|
||||
{
|
||||
"ip": "117.34.114.16",
|
||||
@@ -262,12 +299,16 @@
|
||||
"mac"
|
||||
],
|
||||
"handshake": null,
|
||||
"status": "slow",
|
||||
"latency": 1.575,
|
||||
"checked_at": "2026-08-19 15:37:35",
|
||||
"status": "ok",
|
||||
"latency": 0.306,
|
||||
"checked_at": "2026-10-16 13:46:18",
|
||||
"source": "probe-2026-08-12",
|
||||
"fast_pool": false,
|
||||
"checked_date": "2026-08-19"
|
||||
"checked_date": "2026-10-16",
|
||||
"_per_proto": {
|
||||
"standard": 0.352,
|
||||
"mac": 0.306
|
||||
}
|
||||
},
|
||||
{
|
||||
"ip": "159.75.55.232",
|
||||
@@ -276,26 +317,32 @@
|
||||
"standard"
|
||||
],
|
||||
"handshake": null,
|
||||
"status": "ok",
|
||||
"latency": 0.244,
|
||||
"checked_at": "2026-09-08 10:05:12",
|
||||
"status": "fail",
|
||||
"latency": null,
|
||||
"checked_at": "2026-10-16 13:46:18",
|
||||
"source": "probe-2026-08-12",
|
||||
"fast_pool": true,
|
||||
"checked_date": "2026-09-08"
|
||||
"fast_pool": false,
|
||||
"checked_date": "2026-10-16",
|
||||
"_per_proto": {}
|
||||
},
|
||||
{
|
||||
"ip": "117.34.114.17",
|
||||
"name": "国泰君安",
|
||||
"protocols": [
|
||||
"standard"
|
||||
"standard",
|
||||
"mac"
|
||||
],
|
||||
"handshake": null,
|
||||
"status": "slow",
|
||||
"latency": 1.573,
|
||||
"checked_at": "2026-08-19 15:37:35",
|
||||
"status": "ok",
|
||||
"latency": 0.307,
|
||||
"checked_at": "2026-10-16 13:46:18",
|
||||
"source": "probe-2026-08-12",
|
||||
"fast_pool": false,
|
||||
"checked_date": "2026-08-19"
|
||||
"checked_date": "2026-10-16",
|
||||
"_per_proto": {
|
||||
"standard": 0.35,
|
||||
"mac": 0.307
|
||||
}
|
||||
},
|
||||
{
|
||||
"ip": "117.34.114.14",
|
||||
@@ -305,12 +352,16 @@
|
||||
"mac"
|
||||
],
|
||||
"handshake": null,
|
||||
"status": "slow",
|
||||
"latency": 0.508,
|
||||
"checked_at": "2026-08-19 15:37:35",
|
||||
"status": "ok",
|
||||
"latency": 0.395,
|
||||
"checked_at": "2026-10-16 13:46:18",
|
||||
"source": "probe-2026-08-12",
|
||||
"fast_pool": false,
|
||||
"checked_date": "2026-08-19"
|
||||
"checked_date": "2026-10-16",
|
||||
"_per_proto": {
|
||||
"standard": 0.395,
|
||||
"mac": 0.675
|
||||
}
|
||||
},
|
||||
{
|
||||
"ip": "117.34.114.15",
|
||||
@@ -320,12 +371,16 @@
|
||||
"mac"
|
||||
],
|
||||
"handshake": null,
|
||||
"status": "slow",
|
||||
"latency": 0.636,
|
||||
"checked_at": "2026-08-19 15:37:35",
|
||||
"status": "ok",
|
||||
"latency": 0.345,
|
||||
"checked_at": "2026-10-16 13:46:18",
|
||||
"source": "probe-2026-08-12",
|
||||
"fast_pool": false,
|
||||
"checked_date": "2026-08-19"
|
||||
"checked_date": "2026-10-16",
|
||||
"_per_proto": {
|
||||
"standard": 0.345,
|
||||
"mac": 0.726
|
||||
}
|
||||
},
|
||||
{
|
||||
"ip": "117.34.114.13",
|
||||
@@ -335,12 +390,16 @@
|
||||
"mac"
|
||||
],
|
||||
"handshake": null,
|
||||
"status": "slow",
|
||||
"latency": 0.608,
|
||||
"checked_at": "2026-08-19 15:37:35",
|
||||
"status": "ok",
|
||||
"latency": 0.326,
|
||||
"checked_at": "2026-10-16 13:46:18",
|
||||
"source": "probe-2026-08-12",
|
||||
"fast_pool": false,
|
||||
"checked_date": "2026-08-19"
|
||||
"checked_date": "2026-10-16",
|
||||
"_per_proto": {
|
||||
"standard": 0.326,
|
||||
"mac": 0.743
|
||||
}
|
||||
},
|
||||
{
|
||||
"ip": "117.34.114.27",
|
||||
@@ -350,12 +409,16 @@
|
||||
"mac"
|
||||
],
|
||||
"handshake": null,
|
||||
"status": "slow",
|
||||
"latency": 1.138,
|
||||
"checked_at": "2026-08-19 15:37:35",
|
||||
"status": "ok",
|
||||
"latency": 0.394,
|
||||
"checked_at": "2026-10-16 13:46:18",
|
||||
"source": "probe-2026-08-12",
|
||||
"fast_pool": false,
|
||||
"checked_date": "2026-08-19"
|
||||
"checked_date": "2026-10-16",
|
||||
"_per_proto": {
|
||||
"standard": 0.394,
|
||||
"mac": 0.78
|
||||
}
|
||||
},
|
||||
{
|
||||
"ip": "117.34.114.18",
|
||||
@@ -365,12 +428,16 @@
|
||||
"mac"
|
||||
],
|
||||
"handshake": null,
|
||||
"status": "slow",
|
||||
"latency": 0.969,
|
||||
"checked_at": "2026-08-19 15:37:35",
|
||||
"status": "ok",
|
||||
"latency": 0.327,
|
||||
"checked_at": "2026-10-16 13:46:18",
|
||||
"source": "probe-2026-08-12",
|
||||
"fast_pool": false,
|
||||
"checked_date": "2026-08-19"
|
||||
"checked_date": "2026-10-16",
|
||||
"_per_proto": {
|
||||
"standard": 0.327,
|
||||
"mac": 0.328
|
||||
}
|
||||
},
|
||||
{
|
||||
"ip": "117.34.114.20",
|
||||
@@ -380,12 +447,16 @@
|
||||
"mac"
|
||||
],
|
||||
"handshake": null,
|
||||
"status": "slow",
|
||||
"latency": 0.526,
|
||||
"checked_at": "2026-08-19 15:37:35",
|
||||
"status": "ok",
|
||||
"latency": 0.348,
|
||||
"checked_at": "2026-10-16 13:46:18",
|
||||
"source": "probe-2026-08-12",
|
||||
"fast_pool": false,
|
||||
"checked_date": "2026-08-19"
|
||||
"checked_date": "2026-10-16",
|
||||
"_per_proto": {
|
||||
"standard": 0.348,
|
||||
"mac": 0.74
|
||||
}
|
||||
},
|
||||
{
|
||||
"ip": "117.34.114.30",
|
||||
@@ -396,8 +467,11 @@
|
||||
"handshake": null,
|
||||
"status": "fail",
|
||||
"latency": null,
|
||||
"checked_at": "2026-08-12 18:36:21",
|
||||
"source": "probe-2026-08-12"
|
||||
"checked_at": "2026-10-16 13:46:37",
|
||||
"source": "probe-2026-08-12",
|
||||
"checked_date": "2026-10-16",
|
||||
"fast_pool": false,
|
||||
"_per_proto": {}
|
||||
},
|
||||
{
|
||||
"ip": "121.37.207.165",
|
||||
@@ -407,9 +481,14 @@
|
||||
],
|
||||
"handshake": null,
|
||||
"status": "ok",
|
||||
"latency": 0.249,
|
||||
"checked_at": "2026-08-12 18:36:17",
|
||||
"source": "probe-2026-08-12"
|
||||
"latency": 0.726,
|
||||
"checked_at": "2026-10-16 13:46:18",
|
||||
"source": "probe-2026-08-12",
|
||||
"checked_date": "2026-10-16",
|
||||
"fast_pool": false,
|
||||
"_per_proto": {
|
||||
"mac": 0.726
|
||||
}
|
||||
},
|
||||
{
|
||||
"ip": "121.36.248.138",
|
||||
@@ -419,9 +498,14 @@
|
||||
],
|
||||
"handshake": null,
|
||||
"status": "ok",
|
||||
"latency": 0.436,
|
||||
"checked_at": "2026-08-12 18:36:17",
|
||||
"source": "probe-2026-08-12"
|
||||
"latency": 0.624,
|
||||
"checked_at": "2026-10-16 13:46:18",
|
||||
"source": "probe-2026-08-12",
|
||||
"checked_date": "2026-10-16",
|
||||
"fast_pool": false,
|
||||
"_per_proto": {
|
||||
"mac": 0.624
|
||||
}
|
||||
},
|
||||
{
|
||||
"ip": "123.60.47.136",
|
||||
@@ -431,9 +515,14 @@
|
||||
],
|
||||
"handshake": null,
|
||||
"status": "ok",
|
||||
"latency": 0.476,
|
||||
"checked_at": "2026-08-12 18:36:17",
|
||||
"source": "probe-2026-08-12"
|
||||
"latency": 0.56,
|
||||
"checked_at": "2026-10-16 13:46:30",
|
||||
"source": "probe-2026-08-12",
|
||||
"checked_date": "2026-10-16",
|
||||
"fast_pool": false,
|
||||
"_per_proto": {
|
||||
"mac": 0.56
|
||||
}
|
||||
},
|
||||
{
|
||||
"ip": "116.205.135.205",
|
||||
@@ -443,9 +532,14 @@
|
||||
],
|
||||
"handshake": "login",
|
||||
"status": "ok",
|
||||
"latency": 0.192,
|
||||
"checked_at": "2026-08-12 18:36:17",
|
||||
"source": "probe-2026-08-12"
|
||||
"latency": 0.097,
|
||||
"checked_at": "2026-10-16 13:46:21",
|
||||
"source": "probe-2026-08-12",
|
||||
"checked_date": "2026-10-16",
|
||||
"fast_pool": true,
|
||||
"_per_proto": {
|
||||
"ex": 0.097
|
||||
}
|
||||
},
|
||||
{
|
||||
"ip": "121.37.232.167",
|
||||
@@ -455,9 +549,14 @@
|
||||
],
|
||||
"handshake": "login",
|
||||
"status": "ok",
|
||||
"latency": 0.194,
|
||||
"checked_at": "2026-08-12 18:36:17",
|
||||
"source": "probe-2026-08-12"
|
||||
"latency": 0.097,
|
||||
"checked_at": "2026-10-16 13:46:21",
|
||||
"source": "probe-2026-08-12",
|
||||
"checked_date": "2026-10-16",
|
||||
"fast_pool": true,
|
||||
"_per_proto": {
|
||||
"ex": 0.097
|
||||
}
|
||||
},
|
||||
{
|
||||
"ip": "116.205.143.214",
|
||||
@@ -467,22 +566,66 @@
|
||||
],
|
||||
"handshake": "setup",
|
||||
"status": "ok",
|
||||
"latency": 0.192,
|
||||
"checked_at": "2026-08-12 18:36:17",
|
||||
"source": "probe-2026-08-12"
|
||||
"latency": 0.103,
|
||||
"checked_at": "2026-10-16 13:46:21",
|
||||
"source": "probe-2026-08-12",
|
||||
"checked_date": "2026-10-16",
|
||||
"fast_pool": true,
|
||||
"_per_proto": {
|
||||
"ex": 0.103
|
||||
}
|
||||
},
|
||||
{
|
||||
"ip": "s1",
|
||||
"name": "s1",
|
||||
"ip": "112.74.214.43",
|
||||
"name": "扩展市场深圳双线1",
|
||||
"protocols": [
|
||||
"standard"
|
||||
"ex"
|
||||
],
|
||||
"status": "unknown",
|
||||
"latency": null,
|
||||
"fast_pool": false,
|
||||
"checked_date": "2026-08-19",
|
||||
"checked_at": "2026-08-19 15:34:45"
|
||||
"handshake": "login",
|
||||
"status": "ok",
|
||||
"latency": 0.095,
|
||||
"checked_at": "2026-10-16 13:46:21",
|
||||
"source": "connect.cfg",
|
||||
"fast_pool": true,
|
||||
"checked_date": "2026-10-16",
|
||||
"_per_proto": {
|
||||
"ex": 0.095
|
||||
}
|
||||
},
|
||||
{
|
||||
"ip": "120.25.218.6",
|
||||
"name": "扩展市场深圳双线2",
|
||||
"protocols": [
|
||||
"ex"
|
||||
],
|
||||
"handshake": "login",
|
||||
"status": "ok",
|
||||
"latency": 0.101,
|
||||
"checked_at": "2026-10-16 13:46:21",
|
||||
"source": "connect.cfg",
|
||||
"fast_pool": true,
|
||||
"checked_date": "2026-10-16",
|
||||
"_per_proto": {
|
||||
"ex": 0.101
|
||||
}
|
||||
},
|
||||
{
|
||||
"ip": "47.102.108.214",
|
||||
"name": "扩展市场上海双线2",
|
||||
"protocols": [
|
||||
"ex"
|
||||
],
|
||||
"handshake": "login",
|
||||
"status": "ok",
|
||||
"latency": 0.149,
|
||||
"checked_at": "2026-10-16 13:46:22",
|
||||
"source": "connect.cfg",
|
||||
"fast_pool": true,
|
||||
"checked_date": "2026-10-16",
|
||||
"_per_proto": {
|
||||
"ex": 0.149
|
||||
}
|
||||
}
|
||||
],
|
||||
"checked_date": "2026-09-08"
|
||||
"checked_date": "2026-10-16"
|
||||
}
|
||||
@@ -0,0 +1,290 @@
|
||||
"""标的代码解析:全库唯一的字符串口径。
|
||||
|
||||
统一写法(也是 CLI 的写法)::
|
||||
|
||||
sz000001 / sh600000 / bj920992 A股(6位代码,必须带市场前缀)
|
||||
hk00700 港股主板(5位)
|
||||
usAAPL 美股(ticker)
|
||||
cffex:IFL0 / zz:AP0 期货(交易所前缀 + 合约,可带冒号)
|
||||
000001 裸 6 位数字 = 沪市指数
|
||||
|
||||
**裸 6 位数字只当作沪市指数**:A股代码不唯一(``000001`` 既是上证指数也是平安银行),
|
||||
要股票请写全前缀 ``sh000001`` / ``sz000001``。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Sequence
|
||||
|
||||
from .models import ExMarket, Market
|
||||
|
||||
# 通道:cn=标准/MAC A股通道,ex=扩展市场通道(端口 7727)
|
||||
CN = "cn"
|
||||
EX = "ex"
|
||||
INDEX = "index"
|
||||
|
||||
_CN_ALIASES: dict[str, Market] = {
|
||||
"sz": Market.SZ,
|
||||
"sh": Market.SH,
|
||||
"bj": Market.BJ,
|
||||
}
|
||||
_CN_PREFIX: dict[int, str] = {int(m): k for k, m in _CN_ALIASES.items()}
|
||||
|
||||
# EX 常用别名(短前缀优先;其余成员用枚举名小写,如 hk_gem:08001)
|
||||
_EX_ALIASES: dict[str, ExMarket] = {
|
||||
"hk": ExMarket.HK_MAIN_BOARD,
|
||||
"us": ExMarket.US_STOCK,
|
||||
"zz": ExMarket.ZZ_FUTURES,
|
||||
"dl": ExMarket.DL_FUTURES,
|
||||
"shf": ExMarket.SH_FUTURES,
|
||||
"cffex": ExMarket.CFFEX_FUTURES,
|
||||
"gz": ExMarket.GZ_FUTURES,
|
||||
}
|
||||
_EX_PREFIX: dict[int, str] = {int(m): k for k, m in _EX_ALIASES.items()}
|
||||
|
||||
ALL = "all"
|
||||
|
||||
|
||||
def _ex_by_name(name: str) -> ExMarket | None:
|
||||
"""按枚举名匹配(hk_main_board / us_stock / ...),保证 format→parse 可往返。"""
|
||||
key = name.upper()
|
||||
for member in ExMarket:
|
||||
if member.name == key:
|
||||
return member
|
||||
return None
|
||||
|
||||
|
||||
def _lookup(text: str) -> tuple[int, str] | None:
|
||||
"""前缀 -> (市场号, 通道)。"""
|
||||
if text in _CN_ALIASES:
|
||||
return int(_CN_ALIASES[text]), CN
|
||||
if text in _EX_ALIASES:
|
||||
return int(_EX_ALIASES[text]), EX
|
||||
member = _ex_by_name(text)
|
||||
if member is not None:
|
||||
return int(member), EX
|
||||
return None
|
||||
|
||||
|
||||
def _hint() -> str:
|
||||
cn = "/".join(sorted(_CN_ALIASES))
|
||||
ex = "/".join(sorted(_EX_ALIASES))
|
||||
return f"A股: {cn};扩展市场: {ex}(详见 pytdxdata.models.ExMarket)"
|
||||
|
||||
|
||||
@dataclass(frozen=True, slots=True)
|
||||
class Symbol:
|
||||
"""解析后的标的。``kind``: cn=A股 / ex=扩展市场 / index=裸数字指数。"""
|
||||
|
||||
market: int
|
||||
code: str
|
||||
kind: str
|
||||
raw: str = ""
|
||||
|
||||
@property
|
||||
def is_ex(self) -> bool:
|
||||
return self.kind == EX
|
||||
|
||||
@property
|
||||
def is_index(self) -> bool:
|
||||
return self.kind == INDEX
|
||||
|
||||
@property
|
||||
def channel(self) -> str:
|
||||
"""路由通道:ex 走扩展市场池,其余走标准/MAC A股池。"""
|
||||
return EX if self.is_ex else "std"
|
||||
|
||||
@property
|
||||
def text(self) -> str:
|
||||
"""规范化写法,同时用作记录上的 ``.symbol``。"""
|
||||
if self.is_index:
|
||||
return self.code
|
||||
return format_symbol(self.market, self.code)
|
||||
|
||||
def as_pair(self) -> tuple[int, str]:
|
||||
"""(市场号, 代码) 元组(协议层用)。"""
|
||||
return self.market, self.code
|
||||
|
||||
def __str__(self) -> str:
|
||||
return self.text
|
||||
|
||||
|
||||
def _prefix_for(market: int) -> str | None:
|
||||
"""市场号 -> 短前缀;无短别名的扩展市场回退到枚举名小写。"""
|
||||
m = int(market)
|
||||
prefix = _CN_PREFIX.get(m) or _EX_PREFIX.get(m)
|
||||
if prefix is not None:
|
||||
return prefix
|
||||
try:
|
||||
return ExMarket(m).name.lower()
|
||||
except ValueError:
|
||||
return None
|
||||
|
||||
|
||||
def format_symbol(market: int, code: str) -> str:
|
||||
"""(市场号, 代码) -> 规范化字符串(``1, "600000"`` -> ``"sh600000"``)。"""
|
||||
code = str(code).strip()
|
||||
if not code:
|
||||
return ""
|
||||
prefix = _prefix_for(market)
|
||||
if prefix is None:
|
||||
return code
|
||||
# 2 字母前缀贴代码(hk00700),长前缀用冒号(cffex:IFL0 / hk_gem:08001)
|
||||
return f"{prefix}:{code}" if len(prefix) > 2 else f"{prefix}{code}"
|
||||
|
||||
|
||||
def market_name(market: int) -> str:
|
||||
"""市场号 -> 市场选择器字符串(``1`` -> ``"sh"``,``31`` -> ``"hk"``)。
|
||||
|
||||
用于把旧接口的市场号参数转成统一接口要的市场选择器。
|
||||
"""
|
||||
prefix = _prefix_for(market)
|
||||
if prefix is None:
|
||||
raise ValueError(f"未知市场号: {market}")
|
||||
return prefix
|
||||
|
||||
|
||||
def parse_symbol(text: str) -> Symbol:
|
||||
"""字符串 -> Symbol。无法识别时抛 ValueError 并列出可用前缀。"""
|
||||
raw = text
|
||||
sym = str(text).strip()
|
||||
if not sym:
|
||||
raise ValueError("标的为空")
|
||||
|
||||
if ":" in sym:
|
||||
head, code = sym.split(":", 1)
|
||||
head, code = head.strip().lower(), code.strip()
|
||||
hit = _lookup(head)
|
||||
if hit is None:
|
||||
raise ValueError(f"未知市场前缀: {head}({_hint()})")
|
||||
if not code:
|
||||
raise ValueError(f"缺少代码: {raw}")
|
||||
return Symbol(market=hit[0], code=code.upper(), kind=hit[1], raw=raw)
|
||||
|
||||
head, code = sym[:2].lower(), sym[2:]
|
||||
hit = _lookup(head)
|
||||
if hit is not None and code:
|
||||
return Symbol(market=hit[0], code=code.upper(), kind=hit[1], raw=raw)
|
||||
|
||||
if sym.isdigit() and len(sym) == 6:
|
||||
# 裸 6 位 = 沪市指数(A股代码不唯一,股票必须带前缀)
|
||||
return Symbol(market=int(Market.SH), code=sym, kind=INDEX, raw=raw)
|
||||
|
||||
raise ValueError(f"无法识别标的: {raw}({_hint()};裸6位数字=沪市指数)")
|
||||
|
||||
|
||||
def parse_symbols(symbols: str | Sequence[str] | None) -> list[Symbol]:
|
||||
"""接受单个字符串 / 字符串序列 / Symbol 序列,去重并保持输入顺序。"""
|
||||
if symbols is None:
|
||||
raise ValueError("symbols 不能为空")
|
||||
items: list = [symbols] if isinstance(symbols, str) else list(symbols)
|
||||
if not items:
|
||||
raise ValueError("symbols 不能为空")
|
||||
out: list[Symbol] = []
|
||||
seen: set[tuple[int, str]] = set()
|
||||
for item in items:
|
||||
sym = item if isinstance(item, Symbol) else parse_symbol(item)
|
||||
if sym.as_pair() in seen:
|
||||
continue
|
||||
seen.add(sym.as_pair())
|
||||
out.append(sym)
|
||||
return out
|
||||
|
||||
|
||||
def cn_markets() -> list[int]:
|
||||
"""A股市场号(深/沪/北)。"""
|
||||
return [int(m) for m in Market]
|
||||
|
||||
|
||||
def ex_markets() -> list[int]:
|
||||
"""扩展市场号,按枚举值升序。"""
|
||||
return sorted(int(m) for m in ExMarket)
|
||||
|
||||
|
||||
def market_targets(text: str | None) -> list[tuple[int, str]]:
|
||||
"""市场选择器 -> [(市场号, 通道)];``None`` / ``"all"`` = 全部市场。
|
||||
|
||||
市场号在两条通道里会撞车(``Market.SH=1`` 与 ``ExMarket.TEMP_STOCK=1``),
|
||||
所以必须带通道一起返回,调用方按通道分别请求。
|
||||
"""
|
||||
key = "" if text is None else str(text).strip().lower()
|
||||
if not key or key == ALL:
|
||||
return ([(m, CN) for m in cn_markets()]
|
||||
+ [(m, EX) for m in ex_markets()])
|
||||
hit = _lookup(key)
|
||||
if hit is None:
|
||||
raise ValueError(f"未知市场: {text}({_hint()};全部用 all)")
|
||||
return [hit]
|
||||
|
||||
|
||||
def parse_market(text: str | None) -> int | None:
|
||||
"""市场选择器 -> 单个市场号;``None`` / ``"all"`` 返回 None。
|
||||
|
||||
只用于「本来就是单市场」的接口(如市场异动)。需要跨市场时用
|
||||
:func:`market_targets`。
|
||||
"""
|
||||
targets = market_targets(text)
|
||||
return None if len(targets) > 1 else targets[0][0]
|
||||
|
||||
|
||||
def is_cn_index(market: int, code: str) -> bool:
|
||||
"""A股指数代码判定:88开头板块指数 / 沪市000 / 深市399。
|
||||
|
||||
深市股票 000001 不是指数,必须带市场判断。
|
||||
"""
|
||||
code = str(code)
|
||||
return (
|
||||
code.startswith("88")
|
||||
or (code.startswith("000") and int(market) == int(Market.SH))
|
||||
or (code.startswith("399") and int(market) == int(Market.SZ))
|
||||
)
|
||||
|
||||
|
||||
def has_order_count(market: int, code: str) -> bool:
|
||||
"""该标的的逐笔 ``trade_count`` 是否是**真实成交笔数**。
|
||||
|
||||
通达信对**指数**和**多数场内基金**不提供笔数,但那 4 字节里仍塞了值:
|
||||
|
||||
- 指数:``当时均价 × 1000``(见 :class:`~pytdxdata.models.TransactionRecord`)
|
||||
- ETF / LOF:``价格 × 1000``;货币 ETF(如 ``sh511990``)恒为 ``0``
|
||||
|
||||
直接当笔数用会得到 3994 这种荒谬值(实测 ``sh510310``)。
|
||||
|
||||
实测覆盖(2026-10-22,按代码前缀各取样 1~7 只):
|
||||
|
||||
================== ========================== ==========
|
||||
品种 前缀 笔数可用
|
||||
================== ========================== ==========
|
||||
沪股票/科创 600/601/603/605/688/689 是
|
||||
沪 LOF 501/502/506 是
|
||||
沪 ETF/LOF 51/52/56/58 **否**
|
||||
沪指数 000/88 否
|
||||
沪可转债 110/111/113/118 是
|
||||
沪 EB(可交换债) 132 是
|
||||
沪逆回购 204 是
|
||||
深股票 000/001/002/003/300/301 是
|
||||
深 ETF 159 **否**
|
||||
深 LOF/基金 15/16/17/18 **否**
|
||||
深指数 399 否
|
||||
深可转债 123/127/128 是
|
||||
================== ========================== ==========
|
||||
|
||||
``501x/502x/506x``(沪市 LOF)实测**有**真实笔数,所以不能用「沪市 5 开头
|
||||
一律当基金」的粗规则;必须按上面这组前缀精确判定。
|
||||
|
||||
可转债(沪 110/111/113/118/132、深 123/127/128)**走的是兜底分支**——
|
||||
它们既不在沪市 ``51/52/56/58`` 里,也不在深市 ``159/15/16/17/18`` 里,
|
||||
所以两个分支都落到「是」。这不是巧合而是设计:兜底按「**排除法**」认定
|
||||
基金/指数,其余一律当普通品种。实测验证(2026-10-22,16 只当日 + 8 只历史日):
|
||||
笔数合计与日线量级吻合(``sh111000`` 3631 条切片 / 20443 笔,
|
||||
``sz128136`` 1939 条 / 14593 笔),且 ``Σvol`` 与日线 ``vol`` 比值 **1.000000**。
|
||||
"""
|
||||
code = str(code)
|
||||
if is_cn_index(market, code):
|
||||
return False
|
||||
m = int(market)
|
||||
if m == int(Market.SH):
|
||||
return code[:2] not in ("51", "52", "56", "58")
|
||||
if m == int(Market.SZ):
|
||||
return not (code[:3] == "159" or code[:2] in ("15", "16", "17", "18"))
|
||||
return True
|
||||
@@ -0,0 +1,31 @@
|
||||
"""测试全局隔离。
|
||||
|
||||
默认关掉两个**后台网络行为**:
|
||||
- 权威小数位表同步:否则任何调用 ``TdxData.start()`` 的用例都会顺手发起证券
|
||||
清单请求(26 + 25 个分页),测试变慢且依赖网络。
|
||||
- 当日服务器测速:后台 task 会真的去连主机取K线,会污染「往返次数」这类
|
||||
断言(实测同一个用例的 ``calls`` 在 3/4 之间随机跳),也会拖慢整体。
|
||||
|
||||
需要验证这两个行为本身的用例(``test_decimal_sync.py`` / ``test_probe_cache.py``)
|
||||
自己 ``delenv`` 后再测。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
|
||||
import pytest
|
||||
|
||||
_ENV_KEYS = ("PYPYTDX_DATA_DISABLE_DECIMAL_SYNC", "PYPYTDX_DATA_DISABLE_PROBE")
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True, scope="session")
|
||||
def _disable_background_network():
|
||||
old = {k: os.environ.get(k) for k in _ENV_KEYS}
|
||||
for k in _ENV_KEYS:
|
||||
os.environ[k] = "1"
|
||||
yield
|
||||
for k, v in old.items():
|
||||
if v is None:
|
||||
os.environ.pop(k, None)
|
||||
else:
|
||||
os.environ[k] = v
|
||||
@@ -0,0 +1,712 @@
|
||||
"""全接口回归:mock 连接,逐个调用 ``TdxData`` 的每一个公开方法。
|
||||
|
||||
目标是「没有接口是死的」——每个公开方法至少被真实调用一次并返回预期形状。
|
||||
末尾的 ``test_every_public_method_was_exercised`` 用 AST 解析 ``api.py``,
|
||||
断言覆盖集合与全部公开方法**完全相等**:以后新增方法若没纳入本文件就会失败
|
||||
(与 ``scripts/check_docs.py`` 的 ``MUST_LIST_ALL`` 互为呼应)。
|
||||
|
||||
不连任何网络:标准 / MAC / EX 连接全部被 mock;探测缓存被短路。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import ast
|
||||
import datetime
|
||||
import io
|
||||
import pathlib
|
||||
import struct
|
||||
import time
|
||||
import zipfile
|
||||
from unittest import mock
|
||||
|
||||
import pytest
|
||||
|
||||
from pytdxdata.api import TdxData
|
||||
from pytdxdata.models import (
|
||||
AuctionItem,
|
||||
BelongBoard,
|
||||
BoardInfo,
|
||||
CapitalFlow,
|
||||
FinanceRecord,
|
||||
MemberQuote,
|
||||
MinuteBar,
|
||||
SecurityBar,
|
||||
SecurityInfo,
|
||||
SecurityQuote,
|
||||
ServerInfo,
|
||||
SymbolSnapshot,
|
||||
TransactionRecord,
|
||||
UnusualItem,
|
||||
XdxrRecord,
|
||||
)
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# 覆盖追踪
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
EXERCISED: set[str] = set()
|
||||
|
||||
|
||||
def _use(name: str) -> None:
|
||||
EXERCISED.add(name)
|
||||
|
||||
|
||||
def _public_methods() -> set[str]:
|
||||
"""AST 解析 api.py,取 TdxData 的全部公开方法名(与 check_docs 同口径)。"""
|
||||
src = pathlib.Path(__file__).resolve().parent.parent / "src" / "pytdxdata" / "api.py"
|
||||
tree = ast.parse(src.read_text(encoding="utf-8"))
|
||||
for node in ast.walk(tree):
|
||||
if isinstance(node, ast.ClassDef) and node.name == "TdxData":
|
||||
return {
|
||||
item.name
|
||||
for item in node.body
|
||||
if isinstance(item, ast.FunctionDef | ast.AsyncFunctionDef)
|
||||
and not item.name.startswith("_")
|
||||
}
|
||||
raise AssertionError("api.py 里找不到 TdxData")
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# mock 数据
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
_TOTAL_BARS = 2000
|
||||
_BASE_DATE = datetime.date(2026, 9, 16)
|
||||
_QUOTE_FIELDS = {
|
||||
"0x0": 9.8, "0x1": 9.9, "0x2": 10.2, "0x3": 9.7, "0x4": 10.0,
|
||||
"0x5": 1000.0, "0x6": 1.0, "0x7": 1_000_000.0, "0x38": 5000.0,
|
||||
}
|
||||
|
||||
|
||||
def _mk_bar(market: int, code: str, offset: int) -> SecurityBar:
|
||||
"""offset 越大越老(标准通道语义:offset 0 = 最新一根)。"""
|
||||
d = _BASE_DATE - datetime.timedelta(days=offset)
|
||||
return SecurityBar(
|
||||
market=market, code=code, open=10.0, high=10.5, low=9.5,
|
||||
close=10.0 + (offset % 5) * 0.1, vol=1000.0, amount=10000.0,
|
||||
year=d.year, month=d.month, day=d.day, hour=9, minute=30,
|
||||
)
|
||||
|
||||
|
||||
def _bars(start: int, count: int, market: int, code: str) -> list[SecurityBar]:
|
||||
hi = min(start + count, _TOTAL_BARS)
|
||||
return [_mk_bar(market, code, i) for i in range(start, hi)]
|
||||
|
||||
|
||||
def _quote(market: int, code: str) -> SecurityQuote:
|
||||
return SecurityQuote(
|
||||
market=market, code=code, price=10.0, pre_close=9.8, open=9.9,
|
||||
high=10.2, low=9.7, vol=1000.0, amount=1_000_000.0, s_vol=10.0,
|
||||
b_vol=12.0, bid=[9.99] * 5, ask=[10.01] * 5,
|
||||
)
|
||||
|
||||
|
||||
def _member(market: int, code: str) -> MemberQuote:
|
||||
return MemberQuote(market=market, code=code, name="测试",
|
||||
fields=dict(_QUOTE_FIELDS))
|
||||
|
||||
|
||||
class _FileMeta:
|
||||
"""对应 protocol.commands.mac_ext.FileMeta(避免直接依赖内部类)。"""
|
||||
|
||||
__slots__ = ("flag", "md5", "offset", "size")
|
||||
|
||||
def __init__(self, offset=0, size=4, flag=0, md5=""):
|
||||
self.offset, self.size, self.flag, self.md5 = offset, size, flag, md5
|
||||
|
||||
|
||||
class _BlockMeta:
|
||||
__slots__ = ("filename", "md5", "size")
|
||||
|
||||
def __init__(self, size=4, md5="", filename=""):
|
||||
self.size, self.md5, self.filename = size, md5, filename
|
||||
|
||||
|
||||
def _dispatch(cmd) -> object:
|
||||
"""按命令类名返回形状正确的数据。"""
|
||||
n = type(cmd).__name__
|
||||
market = getattr(cmd, "market", 1)
|
||||
code = getattr(cmd, "code", "600000")
|
||||
start = getattr(cmd, "start", 0)
|
||||
count = getattr(cmd, "count", 100) or 100
|
||||
|
||||
if n == "GetSecurityCountCmd":
|
||||
return 8
|
||||
if n == "GetSecurityListCmd":
|
||||
return [SecurityInfo(market=1, code="600000", name="浦发银行",
|
||||
volunit=100, decimal_point=2, pre_close=10.0)]
|
||||
# ---- K线 ----
|
||||
if n in ("GetSecurityBarsCmd", "GetIndexBarsCmd", "MacKlineCmd"):
|
||||
return _bars(start, count, market, code)
|
||||
if n == "GetExInstrumentBarsCmd":
|
||||
return [{"year": 2026, "month": 9, "day": 1, "hour": 9, "minute": 30,
|
||||
"open": 10.0, "high": 10.5, "low": 9.5, "close": 10.0,
|
||||
"trade": 1000}]
|
||||
# ---- 报价 ----
|
||||
if n == "GetSecurityQuotesCmd":
|
||||
return [_quote(m, c) for m, c in cmd.stocks]
|
||||
if n == "MacSymbolQuotesCmd":
|
||||
return [_member(m, c) for m, c in cmd.stocks]
|
||||
if n == "MacBoardMembersQuotesCmd":
|
||||
return [_member(1, "600000")]
|
||||
# ---- 逐笔 / 分时 ----
|
||||
if n == "MacTransactionCmd":
|
||||
return [TransactionRecord(market=market, code=code, time="09:30",
|
||||
price=10.0, vol=100.0, trade_count=1, bs_flag=0)]
|
||||
if n in ("GetExTransactionDataCmd", "GetExHistoryTransactionDataCmd"):
|
||||
return [{"minutes": 570, "price": 10.0, "vol": 100.0, "direction": 0,
|
||||
"zengcang": 1.0}]
|
||||
if n == "MacTickChartCmd":
|
||||
return [MinuteBar(price=p, vol=100.0, hour=9, minute=30 + i)
|
||||
for i, p in enumerate((10.0, 10.1, 10.2))]
|
||||
if n == "SymbolTickChartCmd":
|
||||
return [{"minutes": 570, "price": 10.0, "vol": 100.0, "avg": 10.0,
|
||||
"momentum": 0.1}]
|
||||
if n == "MacChartSamplingCmd":
|
||||
# 刻意与 MacTickChartCmd 的 mock 数据不同:A股由分时派生、EX 用 0x254D,
|
||||
# 靠返回值即可区分实际走了哪条命令(BUG-1 回归用)。
|
||||
return [99.0, 98.0]
|
||||
# ---- 公司信息 ----
|
||||
if n == "MacSymbolInfoCmd":
|
||||
return SymbolSnapshot(
|
||||
market=market, code=code, name="浦发银行", date=20260916, time=930,
|
||||
activity=1.0, pre_close=9.8, open=9.9, high=10.2, low=9.7,
|
||||
close=10.0, vol=1000.0, amount=1_000_000.0, turnover=1.0,
|
||||
avg_price=10.0)
|
||||
if n == "MacAuctionCmd":
|
||||
return [AuctionItem(time=93000, price=10.0, matched=100, unmatched=0)]
|
||||
if n == "MacCapitalFlowCmd":
|
||||
return CapitalFlow(market=market, code=code, main_buy=1.0, main_sell=1.0,
|
||||
retail_buy=1.0, retail_sell=1.0, buy5=1.0, sell5=1.0,
|
||||
big5=1.0, mid5=1.0)
|
||||
if n == "GetXdxrInfoCmd":
|
||||
return [XdxrRecord(market=market, code=code, year=2025, month=6, day=1,
|
||||
category=1, name="除权除息")]
|
||||
if n == "GetFinanceInfoCmd":
|
||||
return FinanceRecord(market=market, code=code)
|
||||
if n == "GetCompanyInfoCategoryCmd":
|
||||
return [{"name": "最新提示", "filename": "600000.txt", "length": 10}]
|
||||
if n == "GetCompanyInfoContentCmd":
|
||||
return "公司资料正文"
|
||||
# ---- 市场 / 板块 ----
|
||||
if n == "MacUnusualCmd":
|
||||
return [UnusualItem(market=1, code="600000", name="测试", type=1,
|
||||
desc="描述", value=1.0, time=930)]
|
||||
if n == "MacServerInfoCmd":
|
||||
return ServerInfo(date=20260916, last_trade_date=20260915,
|
||||
sessions=[(570, 690)])
|
||||
if n == "MacKlineOffsetCmd":
|
||||
return (_TOTAL_BARS, 100)
|
||||
if n == "MacBoardListCmd":
|
||||
return [BoardInfo(market=1, code="881001", name="银行", price=110.0,
|
||||
rise_speed=0.1, pre_close=100.0)]
|
||||
if n == "MacBelongBoardCmd":
|
||||
return [BelongBoard(market=market, board_code=20686, board_name="银行",
|
||||
close=110.0, pre_close=100.0)]
|
||||
# ---- 文件 ----
|
||||
if n == "MacFileListCmd":
|
||||
return _FileMeta(size=4)
|
||||
if n == "GetBlockInfoMetaCmd":
|
||||
return _BlockMeta(size=4)
|
||||
if n in ("GetBlockInfoCmd", "GetReportFileCmd", "MacFileDownloadCmd"):
|
||||
return b"abcd"
|
||||
# ---- EX 市场 ----
|
||||
if n == "GetExInstrumentCountCmd":
|
||||
return 2
|
||||
if n == "GetExInstrumentInfoCmd":
|
||||
return [{"market": 31, "code": "00700", "name": "腾讯控股", "desc": "港股"},
|
||||
{"market": 74, "code": "AAPL", "name": "苹果", "desc": "美股"}]
|
||||
return []
|
||||
|
||||
|
||||
class _MockConn:
|
||||
"""替代 TdxConnection / ExTdxConnection。"""
|
||||
|
||||
def __init__(self, host="h", port=7709, timeout=3.0, *args, **kwargs):
|
||||
self.host = host
|
||||
self.server = host
|
||||
self.port = port
|
||||
self.timeout = timeout
|
||||
self.alive = True
|
||||
self.last_used = time.monotonic()
|
||||
|
||||
async def connect(self) -> None:
|
||||
self.alive = True
|
||||
|
||||
async def close(self) -> None:
|
||||
self.alive = False
|
||||
|
||||
async def ping(self):
|
||||
return 0.01
|
||||
|
||||
async def execute(self, cmd):
|
||||
self.last_used = time.monotonic()
|
||||
return _dispatch(cmd)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# 离线文件样本(供 *_parsed / get_file 走通解析路径)
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
_HEADER = 384
|
||||
|
||||
|
||||
def _make_block_dat() -> bytes:
|
||||
out = bytearray(_HEADER)
|
||||
out += struct.pack("<H", 1)
|
||||
rec = bytearray()
|
||||
rec += "银行".encode("gbk")[:9].ljust(9, b"\x00")
|
||||
rec += struct.pack("<HH", 1, 1)
|
||||
area = bytearray(2800)
|
||||
area[0:7] = "000001".encode("ascii").ljust(7, b"\x00")
|
||||
rec += area
|
||||
out += rec
|
||||
return bytes(out)
|
||||
|
||||
|
||||
def _make_financial_zip() -> bytes:
|
||||
entry = struct.pack("<6sBL", b"600000", 1, 31)
|
||||
header = struct.pack("<1hI1H3L", 0, 20260331, 1, 0, 8, 0)
|
||||
dat = header + entry + struct.pack("<2f", 1.0, 2.0)
|
||||
buf = io.BytesIO()
|
||||
with zipfile.ZipFile(buf, "w") as zf:
|
||||
zf.writestr("gpcw.dat", dat)
|
||||
return buf.getvalue()
|
||||
|
||||
|
||||
def _make_financial_dat() -> bytes:
|
||||
"""未压缩 .dat(服务器实际提供的形式)。"""
|
||||
entry = struct.pack("<6sBL", b"600000", 1, 31)
|
||||
header = struct.pack("<1hI1H3L", 0, 20260331, 1, 0, 8, 0)
|
||||
return header + entry + struct.pack("<2f", 1.0, 2.0)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# fixture
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
@pytest.fixture
|
||||
async def td():
|
||||
with mock.patch("pytdxdata.api.get_standard_hosts", return_value=["s1"]), \
|
||||
mock.patch("pytdxdata.api.get_mac_hosts", return_value=["m1"]), \
|
||||
mock.patch("pytdxdata.api.get_ex_hosts", return_value=["e1"]), \
|
||||
mock.patch("pytdxdata.pool.pool.TdxConnection", new=_MockConn), \
|
||||
mock.patch("pytdxdata.pool.ex_pool.ExTdxConnection", new=_MockConn), \
|
||||
mock.patch("pytdxdata.pool.pool.schedule_probe", return_value=None):
|
||||
t = TdxData(cache_dir=None, pool_min=1, pool_max=2)
|
||||
_use("start")
|
||||
await t.start()
|
||||
try:
|
||||
yield t
|
||||
finally:
|
||||
_use("close")
|
||||
await t.close()
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# 一、标的清单 / 报价 / 快照
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
async def test_universe_and_quotes(td):
|
||||
for name in ("get_universe", "get_quotes"):
|
||||
_use(name)
|
||||
uni = await td.get_universe()
|
||||
assert isinstance(uni, list) and uni
|
||||
cn = await td.get_universe("sz")
|
||||
assert isinstance(cn, list)
|
||||
ex = await td.get_universe("hk")
|
||||
assert isinstance(ex, list)
|
||||
|
||||
q = await td.get_quotes(["sh600000", "sz000001"])
|
||||
assert len(q) == 2 and all(x.symbol for x in q)
|
||||
qf = await td.get_quotes(["sh600000"], fields=[0x00, 0x04])
|
||||
assert qf[0].name == "测试"
|
||||
qex = await td.get_quotes(["hk00700"])
|
||||
assert len(qex) == 1
|
||||
|
||||
|
||||
async def test_decimal_point_sync(td, tmp_path, monkeypatch):
|
||||
"""权威小数位表同步:把证券清单的 decimal_point 灌进价格标度表。"""
|
||||
for name in ("sync_decimal_points", "decimal_point_status"):
|
||||
_use(name)
|
||||
from pytdxdata.protocol.commands.quotes import clear_decimal_map, price_decimal_digits
|
||||
|
||||
# 缓存目录指向临时目录,避免把 mock 数据写进用户真实缓存
|
||||
monkeypatch.setenv("PYPYTDX_DATA_CACHE_DIR", str(tmp_path))
|
||||
before = price_decimal_digits(1, "600000")
|
||||
try:
|
||||
st = await td.sync_decimal_points()
|
||||
assert set(st) >= {"prefixes", "codes", "mixed_prefixes", "stock"}
|
||||
assert st["stock"] >= 0
|
||||
status = await td.decimal_point_status()
|
||||
assert isinstance(status, dict) and "source" in status
|
||||
finally:
|
||||
clear_decimal_map()
|
||||
# 清空后回到静态规则,不受本用例污染
|
||||
assert price_decimal_digits(1, "600000") == before
|
||||
|
||||
|
||||
async def test_snapshot_auction_flow(td):
|
||||
for name in ("get_snapshot", "get_auction", "get_capital_flow"):
|
||||
_use(name)
|
||||
snaps = await td.get_snapshot("sh600000")
|
||||
assert len(snaps) == 1 and snaps[0].name == "浦发银行"
|
||||
auc = await td.get_auction("sh600000")
|
||||
assert len(auc) == 1
|
||||
flow = await td.get_capital_flow("sh600000")
|
||||
assert len(flow) == 1 and flow[0].main_buy == 1.0
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# 二、K线 / 指标
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
async def test_bars(td):
|
||||
_use("get_bars")
|
||||
bars = await td.get_bars("sh600000", count=120)
|
||||
assert len(bars) == 120
|
||||
assert all(b.symbol == "sh600000" for b in bars)
|
||||
# 分页(超过单页)
|
||||
big = await td.get_bars("sh600000", count=2000, concurrency=2)
|
||||
assert len(big) == 2000
|
||||
# 复权走 MAC 通道
|
||||
qfq = await td.get_bars("sh600000", count=200, adjust=None) # NONE -> 标准
|
||||
assert len(qfq) == 200
|
||||
from pytdxdata.models import Adjust
|
||||
mac = await td.get_bars("sh600000", count=200, adjust=Adjust.QFQ)
|
||||
assert len(mac) == 200
|
||||
# 多标的并发
|
||||
multi = await td.get_bars(["sh600000", "sz000001"], count=60)
|
||||
assert len(multi) == 120
|
||||
# 日期区间(二分定位路径)
|
||||
dated = await td.get_bars(
|
||||
"sh600000", start_date=datetime.datetime(2026, 8, 1),
|
||||
end_date=datetime.datetime(2026, 8, 31))
|
||||
assert isinstance(dated, list)
|
||||
# 扩展市场 K线
|
||||
exb = await td.get_bars("hk00700", count=60)
|
||||
assert isinstance(exb, list)
|
||||
|
||||
|
||||
async def test_indicators(td):
|
||||
_use("get_indicators")
|
||||
sets = await td.get_indicators("sh600000", ["macd", "kdj", "rsi"], count=30)
|
||||
assert len(sets) == 1
|
||||
assert sets[0].symbol == "sh600000"
|
||||
assert sets[0].indicators # 指标分量名(DIF/DEA/K/D/J/RSI* 等)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# 三、逐笔 / 分时
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
async def test_ticks(td):
|
||||
_use("get_ticks")
|
||||
rows = await td.get_ticks("sh600000", date=20260811, count=10)
|
||||
assert isinstance(rows, list) and rows and all(r.symbol for r in rows)
|
||||
full = await td.get_ticks("sh600000", count=None)
|
||||
assert isinstance(full, list)
|
||||
hk = await td.get_ticks("hk00700", count=10)
|
||||
assert isinstance(hk, list)
|
||||
|
||||
|
||||
async def test_minutes(td):
|
||||
_use("get_minutes")
|
||||
rows = await td.get_minutes("sh600000")
|
||||
assert isinstance(rows, list) and rows
|
||||
hist = await td.get_minutes("sh600000", date=20260811)
|
||||
assert isinstance(hist, list)
|
||||
samp = await td.get_minutes("sh600000", sampling=True)
|
||||
assert [b.price for b in samp] == [10.0, 10.1, 10.2]
|
||||
exm = await td.get_minutes("hk00700")
|
||||
assert isinstance(exm, list)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# 四、公司信息 / 市场
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
async def test_company_info(td):
|
||||
for name in ("get_xdxr", "get_finance", "get_price_limits", "get_f10"):
|
||||
_use(name)
|
||||
x = await td.get_xdxr("sh600000")
|
||||
assert isinstance(x, list) and x
|
||||
fin = await td.get_finance("sh600000")
|
||||
assert isinstance(fin, list) and fin and fin[0].symbol == "sh600000"
|
||||
lo, hi = await td.get_price_limits("sh600000", name="浦发银行", pre_close=10.0)
|
||||
assert (lo, hi) == (lo, hi) # 本地计算,不连网
|
||||
cat = await td.get_f10("sh600000")
|
||||
assert isinstance(cat, list)
|
||||
body = await td.get_f10("sh600000", section="600000.txt")
|
||||
assert isinstance(body, str)
|
||||
|
||||
|
||||
async def test_market(td):
|
||||
for name in ("get_market_stat", "get_unusual", "get_server_info",
|
||||
"get_kline_offset"):
|
||||
_use(name)
|
||||
stat = await td.get_market_stat()
|
||||
assert stat.up_count == 100 # price 10.0 * 10
|
||||
un = await td.get_unusual("sh")
|
||||
assert isinstance(un, list) and un
|
||||
info = await td.get_server_info()
|
||||
assert info.date == 20260916
|
||||
off = await td.get_kline_offset()
|
||||
assert off == (_TOTAL_BARS, 100)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# 五、板块
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
async def test_boards(td):
|
||||
for name in ("get_boards", "get_board_members", "get_board_of",
|
||||
"get_board_summary", "get_board_ranking",
|
||||
"get_board_change_ranking"):
|
||||
_use(name)
|
||||
bl = await td.get_boards("all")
|
||||
assert isinstance(bl, list) and bl
|
||||
mem = await td.get_board_members("881001")
|
||||
assert isinstance(mem, list) and mem
|
||||
of = await td.get_board_of("sh600000")
|
||||
assert isinstance(of, list) and of
|
||||
summ = await td.get_board_summary("881001")
|
||||
assert set(summ) == {"member_count", "amount", "main_net_amount",
|
||||
"up_count", "down_count"}
|
||||
rank = await td.get_board_ranking("industry")
|
||||
assert isinstance(rank, list)
|
||||
chg = await td.get_board_change_ranking("industry", days=5)
|
||||
assert isinstance(chg, list)
|
||||
|
||||
|
||||
async def test_index_trade_stats(td):
|
||||
"""指数成分成交汇总:成分 → 逐笔 → 每分钟量/笔数。"""
|
||||
for name in ("get_index_members", "get_index_trade_stats",
|
||||
"get_index_trade_stats_multi"):
|
||||
_use(name)
|
||||
mem = await td.get_index_members("000016")
|
||||
assert isinstance(mem, list) and mem and mem[0].symbol
|
||||
st = await td.get_index_trade_stats("000016")
|
||||
assert st.index == "sh000016"
|
||||
assert st.member_count >= 1
|
||||
assert st.minutes and st.minutes[0].time == "09:30"
|
||||
assert st.total_trade_count >= 1
|
||||
assert st.cum_trade_count()[-1][1] == st.total_trade_count
|
||||
multi = await td.get_index_trade_stats_multi("000016", [20261020, None])
|
||||
assert len(multi) == 2
|
||||
assert multi[0].date == 20261020 and multi[1].date is None
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# 六、服务器文件
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
async def test_files(td):
|
||||
_use("get_file")
|
||||
raw = await td.get_file("block_gn.dat")
|
||||
assert raw == b"abcd"
|
||||
|
||||
|
||||
async def test_parsed_files(td):
|
||||
for name in ("get_block_parsed", "get_industry_map",
|
||||
"get_financial_file_infos", "get_financial_records_parsed"):
|
||||
_use(name)
|
||||
with mock.patch.object(td, "get_file",
|
||||
new=mock.AsyncMock(return_value=_make_block_dat())):
|
||||
blocks = await td.get_block_parsed("block_gn.dat")
|
||||
assert blocks and blocks[0].category == 2
|
||||
|
||||
with mock.patch.object(td, "get_file",
|
||||
new=mock.AsyncMock(
|
||||
return_value="0|000001|A01|||S01\n".encode("gbk"))):
|
||||
ind = await td.get_industry_map()
|
||||
assert ind["000001"].sw_industry == "S01"
|
||||
|
||||
with mock.patch.object(td, "get_file",
|
||||
new=mock.AsyncMock(
|
||||
return_value=b"gpcw20260331.zip,abc,1024\n")):
|
||||
infos = await td.get_financial_file_infos()
|
||||
assert infos[0].filesize == 1024
|
||||
|
||||
with mock.patch.object(td, "get_file",
|
||||
new=mock.AsyncMock(return_value=_make_financial_zip())):
|
||||
recs = await td.get_financial_records_parsed("tdxfin/gpcw20260331.zip")
|
||||
assert len(recs) == 1 and recs[0].report_date == 20260331
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# 六之二、BUG 回归(真实联网实测发现,见 tests 说明)
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
async def test_financial_dat_fallback(td):
|
||||
"""BUG-2 回归:索引写 ``.zip``,服务器只给未压缩 ``.dat`` 时必须能回退取到。
|
||||
|
||||
服务器对**不存在的**文件名不报错,而是回 0~1 字节空壳;旧实现会把空壳直接
|
||||
丢给 ``zipfile`` → ``BadZipFile``。现在要求按候选名回退到 ``tdxfin/*.dat``。
|
||||
"""
|
||||
calls: list[str] = []
|
||||
|
||||
async def fake_get_file(name: str) -> bytes:
|
||||
calls.append(name)
|
||||
if name == "tdxfin/gpcw20260331.zip":
|
||||
return b"" # 服务器没有 .zip
|
||||
if name == "gpcw20260331.zip":
|
||||
return b"\x00" # 裸名被路由到 MAC 通道 → 1 字节空壳
|
||||
if name == "tdxfin/gpcw20260331.dat":
|
||||
return _make_financial_dat()
|
||||
return b""
|
||||
|
||||
with mock.patch.object(td, "get_file", new=fake_get_file):
|
||||
recs = await td.get_financial_records_parsed("gpcw20260331.zip")
|
||||
|
||||
assert "tdxfin/gpcw20260331.dat" in calls, "必须回退尝试未压缩 .dat"
|
||||
assert len(recs) == 1 and recs[0].report_date == 20260331
|
||||
assert recs[0].code == "600000"
|
||||
|
||||
|
||||
async def test_financial_records_alias_not_silent(td):
|
||||
"""BUG-2 回归:废弃别名取不到有效财报数据时应返回 ``b""``,而非 0~1 字节空壳。"""
|
||||
import warnings
|
||||
|
||||
async def empty_get_file(name: str) -> bytes:
|
||||
return b"\x00"
|
||||
|
||||
with mock.patch.object(td, "get_file", new=empty_get_file), warnings.catch_warnings():
|
||||
warnings.simplefilter("ignore", DeprecationWarning)
|
||||
got = await td.get_financial_records("gpcw20260331.zip")
|
||||
assert got == b""
|
||||
|
||||
|
||||
async def test_sampling_derives_from_tick_chart(td):
|
||||
"""BUG-1 回归:A股 ``sampling=True`` 必须由分时(0x122D)派生。
|
||||
|
||||
``MacChartSamplingCmd``(0x254D) 是 **EX 协议**命令,7709 主机对它一律不响应
|
||||
(实测 3 台各 10s 超时,叠加连接池重试 ≈24s 才报错)。两条路径的 mock 数据刻意
|
||||
不同,以便从结果区分实际走的是哪条命令。
|
||||
"""
|
||||
# A股:得到分时派生值(10.0/10.1/10.2),而非 0x254D 的 99.0/98.0
|
||||
a_share = await td.get_minutes("sh600000", sampling=True)
|
||||
a_share_hist = await td.get_minutes("sh600000", date=20260811, sampling=True)
|
||||
assert [b.price for b in a_share] == [10.0, 10.1, 10.2]
|
||||
assert [b.price for b in a_share_hist] == [10.0, 10.1, 10.2]
|
||||
assert all(b.vol == 0.0 and b.avg_price == 0 for b in a_share), "采样点无量/均价"
|
||||
assert [b.fields["i"] for b in a_share] == [0.0, 1.0, 2.0], "点位序号在 fields['i']"
|
||||
|
||||
# EX:仍走 0x254D
|
||||
ex = await td.get_minutes("hk00700", sampling=True)
|
||||
assert [b.price for b in ex] == [99.0, 98.0]
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# 七、废弃别名(33 个)
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
async def test_aliases_kline(td):
|
||||
for name in ("get_kline", "get_kline_since", "get_kline_batch",
|
||||
"get_index_kline", "get_stock_kline_with_indicators"):
|
||||
_use(name)
|
||||
import warnings
|
||||
with warnings.catch_warnings():
|
||||
warnings.simplefilter("ignore", DeprecationWarning)
|
||||
assert len(await td.get_kline(1, "600000", 4, count=60)) == 60
|
||||
assert len(await td.get_kline_since(
|
||||
1, "600000", 4, datetime.datetime(2026, 8, 1))) >= 0
|
||||
batch = await td.get_kline_batch([(1, "600000"), (0, "000001")], 4, count=30)
|
||||
assert isinstance(batch, dict) and batch
|
||||
assert len(await td.get_index_kline(1, "000001", 4, count=30)) == 30
|
||||
withind = await td.get_stock_kline_with_indicators(1, "600000", ["macd"])
|
||||
assert "bars" in withind and "indicators" in withind
|
||||
|
||||
|
||||
async def test_aliases_quotes_ticks(td):
|
||||
for name in ("get_stock_quotes", "get_stock_quotes_list", "get_transactions",
|
||||
"get_minute", "get_minute_batch", "get_chart_sampling"):
|
||||
_use(name)
|
||||
import warnings
|
||||
with warnings.catch_warnings():
|
||||
warnings.simplefilter("ignore", DeprecationWarning)
|
||||
assert len(await td.get_stock_quotes([(1, "600000")])) == 1
|
||||
assert isinstance(await td.get_stock_quotes_list(1, count=10), list)
|
||||
assert isinstance(await td.get_transactions(1, "600000", count=10), list)
|
||||
assert isinstance(await td.get_minute(0, "000001"), list)
|
||||
mb = await td.get_minute_batch([(1, "600000")])
|
||||
assert isinstance(mb, dict)
|
||||
assert (await td.get_chart_sampling(1, "600000")) == [10.0, 10.1, 10.2]
|
||||
|
||||
|
||||
async def test_aliases_universe(td):
|
||||
for name in ("get_security_count", "get_security_list",
|
||||
"get_security_list_all"):
|
||||
_use(name)
|
||||
import warnings
|
||||
with warnings.catch_warnings():
|
||||
warnings.simplefilter("ignore", DeprecationWarning)
|
||||
assert await td.get_security_count(0) == 8
|
||||
assert isinstance(await td.get_security_list(0, count=1), list)
|
||||
assert isinstance(await td.get_security_list_all(0), list)
|
||||
|
||||
|
||||
async def test_aliases_files(td):
|
||||
for name in ("get_financial_file_list", "get_financial_records",
|
||||
"get_company_info_category", "get_company_info_content",
|
||||
"get_block_info", "get_report_file", "get_file_meta",
|
||||
"download_file"):
|
||||
_use(name)
|
||||
import warnings
|
||||
with warnings.catch_warnings():
|
||||
warnings.simplefilter("ignore", DeprecationWarning)
|
||||
assert await td.get_financial_file_list() == ["abcd"]
|
||||
with mock.patch.object(td, "get_file",
|
||||
new=mock.AsyncMock(
|
||||
return_value=_make_financial_dat())):
|
||||
assert await td.get_financial_records(
|
||||
"gpcw20260331.zip") == _make_financial_dat()
|
||||
assert isinstance(await td.get_company_info_category(1, "600000"), list)
|
||||
assert isinstance(await td.get_company_info_content(
|
||||
1, "600000", "f.txt"), str)
|
||||
assert await td.get_block_info("block_gn.dat") == b"abcd"
|
||||
assert await td.get_report_file("tdxhy.cfg") == b"abcd"
|
||||
assert (await td.get_file_meta("x.dat")).size == 4
|
||||
assert await td.download_file("x.dat") == b"abcd"
|
||||
|
||||
|
||||
async def test_aliases_boards(td):
|
||||
for name in ("get_board_list", "get_belong_board", "get_symbol_info"):
|
||||
_use(name)
|
||||
import warnings
|
||||
with warnings.catch_warnings():
|
||||
warnings.simplefilter("ignore", DeprecationWarning)
|
||||
assert isinstance(await td.get_board_list(0), list)
|
||||
assert isinstance(await td.get_belong_board(1, "600000"), list)
|
||||
snap = await td.get_symbol_info(1, "600000")
|
||||
assert snap is not None and snap.name == "浦发银行"
|
||||
|
||||
|
||||
async def test_aliases_goods(td):
|
||||
for name in ("get_goods_list", "get_goods_count", "get_goods_quotes",
|
||||
"get_goods_quotes_list", "get_goods_chart_sampling",
|
||||
"get_goods_tick_chart", "get_goods_transaction",
|
||||
"get_goods_transaction_all"):
|
||||
_use(name)
|
||||
import warnings
|
||||
with warnings.catch_warnings():
|
||||
warnings.simplefilter("ignore", DeprecationWarning)
|
||||
assert isinstance(await td.get_goods_list(31), list)
|
||||
assert await td.get_goods_count(31) == 1 # 只有 1 条 market=31
|
||||
assert isinstance(await td.get_goods_quotes([(31, "00700")]), list)
|
||||
assert isinstance(await td.get_goods_quotes_list(31), list)
|
||||
assert await td.get_goods_chart_sampling(31, "00700") == [99.0, 98.0]
|
||||
assert isinstance(await td.get_goods_tick_chart(31, "00700"), list)
|
||||
assert isinstance(await td.get_goods_transaction(31, "00700"), list)
|
||||
assert isinstance(await td.get_goods_transaction_all(31, "00700"), list)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# 八、覆盖完整性(必须放最后)
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
def test_every_public_method_was_exercised():
|
||||
expected = _public_methods()
|
||||
missing = sorted(expected - EXERCISED)
|
||||
assert not missing, f"以下公开方法未被回归测试覆盖: {missing}"
|
||||
extra = sorted(EXERCISED - expected)
|
||||
assert not extra, f"覆盖集合包含非公开方法: {extra}"
|
||||
+211
-1
@@ -8,7 +8,13 @@ import unittest.mock as mock
|
||||
import pytest
|
||||
|
||||
from pytdxdata.api import TdxData
|
||||
from pytdxdata.models import KlinePeriod, Market, MinuteBar, SecurityBar
|
||||
from pytdxdata.models import (
|
||||
KlinePeriod,
|
||||
Market,
|
||||
MinuteBar,
|
||||
SecurityBar,
|
||||
TransactionRecord,
|
||||
)
|
||||
from pytdxdata.protocol.commands.mac_tick import MacTickChartCmd
|
||||
from pytdxdata.scheduler.paginator import PageSpec, Paginator
|
||||
|
||||
@@ -243,6 +249,70 @@ async def test_paginator_start_offset_default_zero():
|
||||
assert result[0].close == 10.0 # 从 offset 0 开始
|
||||
|
||||
|
||||
# ---------- Paginator 逆序偏移(服务端 start 从最新往回数) ----------
|
||||
|
||||
class NewestFirstConn:
|
||||
"""模拟 0x122F 真实行为:start=0 返回**最新**的一页,start 越大越旧。
|
||||
|
||||
实测依据(2026-10-21,sh600000):
|
||||
start=0 -> 14:07:48 .. 15:20:16(当天最新 1000 条)
|
||||
start=4000 -> 09:15:03 .. 09:41:18(当天最早)
|
||||
每页内部按时间升序,页与页之间是倒序。
|
||||
"""
|
||||
|
||||
def __init__(self, total: int, page_size: int):
|
||||
self.total = total
|
||||
self.page_size = page_size
|
||||
self.executed: list[tuple[int, int]] = []
|
||||
|
||||
async def execute(self, cmd):
|
||||
start, count = cmd
|
||||
self.executed.append((start, count))
|
||||
# 服务端语义:偏移 start 是从"最新"往回数,所以取倒数区间
|
||||
end_from_oldest = self.total - start
|
||||
begin_from_oldest = max(0, end_from_oldest - count)
|
||||
return [_make_bar(i) for i in range(begin_from_oldest, end_from_oldest)]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_paginator_newest_first_offset_returns_chronological():
|
||||
"""服务端 start 从最新往回数时,交付列表必须仍是时间升序(BUG 回归)。"""
|
||||
conns = [NewestFirstConn(3000, 1000) for _ in range(3)]
|
||||
pool = FakePool(conns)
|
||||
pag = Paginator(pool, concurrency=3) # type: ignore[arg-type]
|
||||
|
||||
def factory(s, c):
|
||||
return (s, c)
|
||||
|
||||
result = await pag.fetch_all(
|
||||
factory, PageSpec(page_size=1000, total=3000, newest_first=True))
|
||||
assert len(result) == 3000
|
||||
# 时间升序:close 随 index 单调递增
|
||||
closes = [b.close for b in result]
|
||||
assert closes == sorted(closes), "返回列表必须按时间升序拼接"
|
||||
assert result[0].close == pytest.approx(10.0)
|
||||
assert result[-1].close == pytest.approx(10.0 + 2999 * 0.01)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_paginator_newest_first_respects_start_offset():
|
||||
"""newest_first + start_offset:只取最近 N 条里的后段。"""
|
||||
conns = [NewestFirstConn(3000, 1000) for _ in range(3)]
|
||||
pool = FakePool(conns)
|
||||
pag = Paginator(pool, concurrency=3) # type: ignore[arg-type]
|
||||
|
||||
def factory(s, c):
|
||||
return (s, c)
|
||||
|
||||
result = await pag.fetch_all(
|
||||
factory,
|
||||
PageSpec(page_size=1000, total=1000, newest_first=True, start_offset=0))
|
||||
assert len(result) == 1000
|
||||
# 只取了最新的 1000 条 → 是 2000..2999 这一段
|
||||
assert result[0].close == pytest.approx(10.0 + 2000 * 0.01)
|
||||
assert result[-1].close == pytest.approx(10.0 + 2999 * 0.01)
|
||||
|
||||
|
||||
# ---------- get_kline 日期过滤 ----------
|
||||
|
||||
class DateBarApiConn(FakeApiConn):
|
||||
@@ -329,3 +399,143 @@ async def test_get_kline_since():
|
||||
assert bars[-1].year == 2027
|
||||
finally:
|
||||
await td.close()
|
||||
|
||||
# ---------- A股逐笔:b8 过滤 + 单连接串行游标(回归) ----------
|
||||
|
||||
class TickApiConn(FakeApiConn):
|
||||
"""模拟 0x122F:服务端 start 从**最新**往回数,且可注入 bs_flag=8 委托记录。"""
|
||||
|
||||
# 类级:[(time, price, vol, trade_count, bs_flag), ...],时间升序(最早在前)
|
||||
rows: list[tuple] = []
|
||||
calls: list[tuple[int, int]] = []
|
||||
|
||||
async def execute(self, cmd):
|
||||
if "MacTransactionCmd" not in type(cmd).__name__:
|
||||
return await super().execute(cmd)
|
||||
TickApiConn.calls.append((cmd.start, cmd.count))
|
||||
# 从最新往回数:start 偏移落在末尾,往前取 count 条
|
||||
end = len(TickApiConn.rows) - cmd.start
|
||||
begin = max(0, end - cmd.count)
|
||||
chunk = TickApiConn.rows[begin:end]
|
||||
return [TransactionRecord(market=1, code="600000", time=t, price=p,
|
||||
vol=v, trade_count=tc, bs_flag=bs)
|
||||
for (t, p, v, tc, bs) in chunk]
|
||||
|
||||
|
||||
def _build_tick_rows(n_trade: int = 25, n_b8: int = 7) -> list[tuple]:
|
||||
"""构造 n_trade 条正常成交 + n_b8 条竞价委托(穿插在中间)。"""
|
||||
rows: list[tuple] = []
|
||||
for i in range(n_trade):
|
||||
rows.append((f"09:{30 + i // 60:02d}:{i % 60:02d}", 10.0 + i * 0.01,
|
||||
100.0 + i, 3 + i, 1 if i % 2 else 0))
|
||||
if i % 4 == 0 and len([r for r in rows if r[4] == 8]) < n_b8:
|
||||
# 竞价委托:(time, price, vol, tc, bs=8),vol 是累计值、不计入成交
|
||||
rows.append((f"09:15:{i:02d}", 10.0, 9999.0, -1, 8))
|
||||
return rows
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_ticks_filters_auction_orders():
|
||||
"""bs_flag=8 是竞价**委托**快照,不是成交 → 必须从 get_ticks 结果中剔除。
|
||||
|
||||
实测(2026-10-21 sh600000):b8 的 vol 是累计撮合量(可被覆盖),
|
||||
计入后 Σvol 比快照 vol 虚高 20%~160%,且不同主机的 b8 集合不同。
|
||||
"""
|
||||
TickApiConn.rows = _build_tick_rows(n_trade=25, n_b8=7)
|
||||
TickApiConn.calls = []
|
||||
assert sum(1 for r in TickApiConn.rows if r[4] == 8) == 7
|
||||
|
||||
with mock.patch("pytdxdata.api.get_standard_hosts", return_value=["s1"]), \
|
||||
mock.patch("pytdxdata.api.get_mac_hosts", return_value=["m1"]), \
|
||||
mock.patch("pytdxdata.pool.pool.TdxConnection",
|
||||
side_effect=lambda h, p=7709, t=10.0: TickApiConn(h)):
|
||||
td = TdxData(cache_dir=None, pool_min=1, pool_max=2)
|
||||
await td.start()
|
||||
try:
|
||||
out = await td.get_ticks("sh600000")
|
||||
finally:
|
||||
await td.close()
|
||||
|
||||
assert out, "应返回成交记录"
|
||||
assert all(r.bs_flag != 8 for r in out), "bs_flag=8 竞价委托必须被剔除"
|
||||
assert len(out) == 25
|
||||
assert sum(r.vol for r in out) == sum(100.0 + i for i in range(25))
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_ticks_pages_on_single_connection():
|
||||
"""翻页必须走**同一连接**:并发多主机会让页边界错位(重叠 + 漏条)。"""
|
||||
TickApiConn.rows = _build_tick_rows(n_trade=25, n_b8=0)
|
||||
TickApiConn.calls = []
|
||||
|
||||
with mock.patch("pytdxdata.api.get_standard_hosts", return_value=["s1"]), \
|
||||
mock.patch("pytdxdata.api.get_mac_hosts", return_value=["m1", "m2", "m3"]), \
|
||||
mock.patch("pytdxdata.pool.pool.TdxConnection",
|
||||
side_effect=lambda h, p=7709, t=10.0: TickApiConn(h)):
|
||||
td = TdxData(cache_dir=None, pool_min=1, pool_max=4)
|
||||
await td.start()
|
||||
try:
|
||||
out = await td.get_ticks("sh600000")
|
||||
finally:
|
||||
await td.close()
|
||||
|
||||
# 单页装得下 → 只发一次请求
|
||||
assert len(TickApiConn.calls) == 1
|
||||
assert len(out) == 25
|
||||
# 时间升序
|
||||
times = [r.time for r in out]
|
||||
assert times == sorted(times)
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_ticks_multi_page_uses_one_connection_and_no_overlap():
|
||||
"""多页(>_TICK_PAGE):同一连接、游标按 len(rows) 推进、页间不重叠不漏条。
|
||||
|
||||
回归:并发分页时各页来自不同主机,而 0x122F 的 start 是「从最新往回数」的
|
||||
相对锚点 —— 各主机对 bs_flag=8 收录不同,同一 start 落到不同真实成交上,
|
||||
结果页边界既重叠又漏条(实测 sh600000 冗余 116 条、13:03 附近丢 59 条)。
|
||||
"""
|
||||
from pytdxdata.api import _TICK_PAGE
|
||||
|
||||
n = _TICK_PAGE * 2 + 137 # 3 页,末页短页 → 触发 stop_when_short
|
||||
TickApiConn.rows = [
|
||||
(f"{9 + i // 3600:02d}:{i // 60 % 60:02d}:{i % 60:02d}",
|
||||
10.0 + i * 0.001, 1.0 + i, 1 + i, 1)
|
||||
for i in range(n)
|
||||
]
|
||||
TickApiConn.calls = []
|
||||
|
||||
with mock.patch("pytdxdata.api.get_standard_hosts", return_value=["s1"]), \
|
||||
mock.patch("pytdxdata.api.get_mac_hosts", return_value=["m1", "m2", "m3"]), \
|
||||
mock.patch("pytdxdata.pool.pool.TdxConnection",
|
||||
side_effect=lambda h, p=7709, t=10.0: TickApiConn(h)):
|
||||
td = TdxData(cache_dir=None, pool_min=1, pool_max=4)
|
||||
await td.start()
|
||||
try:
|
||||
out = await td.get_ticks("sh600000")
|
||||
finally:
|
||||
await td.close()
|
||||
|
||||
# 恰好 3 页:0 / 1000 / 2000,游标按实际返回条数推进
|
||||
assert TickApiConn.calls == [(0, _TICK_PAGE),
|
||||
(_TICK_PAGE, _TICK_PAGE),
|
||||
(_TICK_PAGE * 2, _TICK_PAGE)]
|
||||
assert len(out) == n, f"必须无重叠、无漏条:期望 {n},实得 {len(out)}"
|
||||
vols = [r.vol for r in out]
|
||||
assert len(set(vols)) == n, "vol 全唯一(无重复记录)"
|
||||
assert vols == sorted(vols), "时间升序"
|
||||
|
||||
# count 参数:游标逐页扣减
|
||||
TickApiConn.calls = []
|
||||
with mock.patch("pytdxdata.api.get_standard_hosts", return_value=["s1"]), \
|
||||
mock.patch("pytdxdata.api.get_mac_hosts", return_value=["m1", "m2", "m3"]), \
|
||||
mock.patch("pytdxdata.pool.pool.TdxConnection",
|
||||
side_effect=lambda h, p=7709, t=10.0: TickApiConn(h)):
|
||||
td = TdxData(cache_dir=None, pool_min=1, pool_max=4)
|
||||
await td.start()
|
||||
try:
|
||||
out2 = await td.get_ticks("sh600000", count=_TICK_PAGE + 50)
|
||||
finally:
|
||||
await td.close()
|
||||
assert len(out2) == _TICK_PAGE + 50
|
||||
assert TickApiConn.calls[1][1] == 50, "末页只补足剩余 50 条"
|
||||
|
||||
@@ -0,0 +1,234 @@
|
||||
"""K线批量流水线路径(多连接 × 单连接流水线)。
|
||||
|
||||
背景:单只K线一请求一标的,全市场 5,000 只就是 5,000 次往返。串行时每只固定付
|
||||
一个 RTT(实测 33ms),实测 24 台 × 8 流水能把全市场 1 分钟K线取当天从 43.6s
|
||||
压到 5s 级。这里用替身验证「路径选择」与「结果口径」,不依赖网络。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import datetime
|
||||
|
||||
import pytest
|
||||
|
||||
from pytdxdata import TdxData
|
||||
from pytdxdata.api import _auto_pipeline, _bars_for_range, _elapsed_trading_minutes
|
||||
from pytdxdata.models import KlinePeriod, SecurityBar
|
||||
|
||||
# 替身的时间锚点必须跟着**真实 now** 走:API 内部用 ``datetime.now()`` 判断「是不是今天」,
|
||||
# 用固定日期会在 CI(UTC 时区,与本地差 8 小时)上漂到别的时段分支,断言时绿时红。
|
||||
NEWEST = datetime.datetime.now().replace(microsecond=0) # noqa: DTZ001
|
||||
|
||||
|
||||
def _bar(when: datetime.datetime, close: float = 10.0) -> SecurityBar:
|
||||
return SecurityBar(
|
||||
market=1, code="600000", open=10.0, high=10.1, low=9.9, close=close,
|
||||
vol=1000, amount=10000, year=when.year, month=when.month, day=when.day,
|
||||
hour=when.hour, minute=when.minute)
|
||||
|
||||
|
||||
class _MacFakeConn:
|
||||
"""MAC 通道语义替身:**offset 0 = 最新一根**,偏移越大越老。"""
|
||||
|
||||
seen: list[tuple[str, int, int, bool]] = [] # (命令名, start, count, 是否流水线)
|
||||
fail_pipeline = False
|
||||
|
||||
def __init__(self, host="m1", port=7709, timeout=10.0):
|
||||
self.host = host
|
||||
self._alive = True
|
||||
|
||||
@property
|
||||
def alive(self) -> bool:
|
||||
return self._alive
|
||||
|
||||
async def connect(self) -> None:
|
||||
pass
|
||||
|
||||
async def close(self) -> None:
|
||||
self._alive = False
|
||||
|
||||
def _rows(self, cmd, *, record: bool = True):
|
||||
if record:
|
||||
type(self).seen.append(
|
||||
(type(cmd).__name__, getattr(cmd, "start", 0),
|
||||
getattr(cmd, "count", 0), False))
|
||||
if "MacKlineCmd" not in type(cmd).__name__:
|
||||
return []
|
||||
out = []
|
||||
for off in range(cmd.start, cmd.start + cmd.count):
|
||||
if off > 20_000:
|
||||
continue
|
||||
when = NEWEST - datetime.timedelta(minutes=off)
|
||||
out.append(_bar(when, 10.0 + (20_000 - off) * 0.001))
|
||||
return out
|
||||
|
||||
async def execute(self, cmd):
|
||||
return self._rows(cmd)
|
||||
|
||||
async def execute_pipeline(self, cmds, **kwargs):
|
||||
if type(self).fail_pipeline:
|
||||
raise ConnectionError("流水线炸了")
|
||||
for c in cmds:
|
||||
type(self).seen.append(
|
||||
(type(c).__name__, getattr(c, "start", 0), getattr(c, "count", 0), True))
|
||||
return [self._rows(c, record=False) for c in cmds]
|
||||
|
||||
|
||||
def _syms(n: int) -> list[str]:
|
||||
"""n 只沪市股票代码(互不相同,避免按 symbol 归并时互相覆盖)。"""
|
||||
return [f"sh60{i:04d}" for i in range(n)]
|
||||
|
||||
|
||||
async def _run(n: int, period=KlinePeriod.MIN_1, **kw):
|
||||
_MacFakeConn.seen = []
|
||||
_MacFakeConn.fail_pipeline = False
|
||||
with pytest.MonkeyPatch.context() as mp:
|
||||
mp.setattr("pytdxdata.api.get_mac_hosts", lambda: ["m1", "m2"])
|
||||
mp.setattr("pytdxdata.pool.pool.TdxConnection",
|
||||
lambda h, p=7709, t=10.0: _MacFakeConn(h))
|
||||
td = TdxData(cache_dir=None, pool_min=1, pool_max=4)
|
||||
await td.start()
|
||||
try:
|
||||
bars = await td.get_bars(_syms(n), period, **kw)
|
||||
finally:
|
||||
await td.close()
|
||||
return bars
|
||||
|
||||
|
||||
def _data_calls() -> list[tuple[str, int, int, bool]]:
|
||||
"""只看真正的数据请求(排除建连探针:count=100 那条)。"""
|
||||
return [c for c in _MacFakeConn.seen if c[2] != 100]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_bulk_path_uses_pipeline_for_many_symbols():
|
||||
bars = await _run(20, count=240)
|
||||
|
||||
data = _data_calls()
|
||||
assert data, "应该有数据请求"
|
||||
assert all(via_pipe for *_, via_pipe in data), "20 只必须走流水线"
|
||||
assert len(data) == 20
|
||||
assert len(bars) == 20 * 240
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_small_batch_falls_back_to_single():
|
||||
"""低于阈值走逐只:不经过流水线,结果不变。"""
|
||||
bars = await _run(5, count=240)
|
||||
|
||||
data = _data_calls()
|
||||
assert data and not any(via_pipe for *_, via_pipe in data)
|
||||
assert len(bars) == 5 * 240
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_pipeline_1_disables_bulk():
|
||||
bars = await _run(20, count=240, pipeline=1)
|
||||
|
||||
data = _data_calls()
|
||||
assert data and not any(via_pipe for *_, via_pipe in data)
|
||||
assert len(bars) == 20 * 240
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_result_order_follows_input_order():
|
||||
"""返回按入参标的顺序拼接(组内升序)。"""
|
||||
bars = await _run(20, count=30)
|
||||
|
||||
seen: list[str] = []
|
||||
for b in bars:
|
||||
if not seen or seen[-1] != b.symbol:
|
||||
seen.append(b.symbol)
|
||||
assert seen == _syms(20)
|
||||
# 组内按时间正序
|
||||
head = [b.datetime for b in bars if b.symbol == _syms(20)[0]]
|
||||
assert head == sorted(head)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_long_count_is_paged_inside_pipeline():
|
||||
"""count 超过单请求上限时按页切分,且页也走流水线(不是逐只串行翻页)。"""
|
||||
bars = await _run(20, count=1600, pipeline=8)
|
||||
|
||||
data = _data_calls()
|
||||
assert all(via_pipe for *_, via_pipe in data)
|
||||
assert len(data) == 20 * 3 # 700 + 700 + 200
|
||||
assert {c[2] for c in data} == {700, 200}
|
||||
assert len(bars) == 20 * 1600
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_pipeline_failure_falls_back_to_single_per_symbol():
|
||||
"""流水线失败 → 该批交回逐只重试,结果不丢。"""
|
||||
_MacFakeConn.seen = []
|
||||
_MacFakeConn.fail_pipeline = True
|
||||
with pytest.MonkeyPatch.context() as mp:
|
||||
mp.setattr("pytdxdata.api.get_mac_hosts", lambda: ["m1", "m2"])
|
||||
mp.setattr("pytdxdata.pool.pool.TdxConnection",
|
||||
lambda h, p=7709, t=10.0: _MacFakeConn(h))
|
||||
td = TdxData(cache_dir=None, pool_min=1, pool_max=4)
|
||||
await td.start()
|
||||
try:
|
||||
bars = await td.get_bars(_syms(20), KlinePeriod.MIN_1, count=240)
|
||||
finally:
|
||||
await td.close()
|
||||
assert len(bars) == 20 * 240
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_dated_range_bulk_trims_to_start_date():
|
||||
"""日期区间走批量时,区间左端仍然精确裁剪。"""
|
||||
start = NEWEST - datetime.timedelta(minutes=30)
|
||||
bars = await _run(20, start_date=start)
|
||||
|
||||
assert bars
|
||||
assert all(b.datetime >= start for b in bars)
|
||||
assert all(b.symbol in set(_syms(20)) for b in bars)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_dated_future_range_returns_empty_without_requests():
|
||||
future = NEWEST + datetime.timedelta(days=5)
|
||||
bars = await _run(20, start_date=future)
|
||||
|
||||
assert bars == []
|
||||
|
||||
|
||||
# ---------------- 纯函数 ----------------
|
||||
def test_elapsed_trading_minutes():
|
||||
def at(h, m):
|
||||
return datetime.datetime(2026, 10, 28, h, m) # noqa: DTZ001
|
||||
|
||||
assert _elapsed_trading_minutes(at(9, 0)) == 0
|
||||
assert _elapsed_trading_minutes(at(9, 30)) == 0
|
||||
assert _elapsed_trading_minutes(at(10, 0)) == 30
|
||||
assert _elapsed_trading_minutes(at(11, 45)) == 120 # 午休
|
||||
assert _elapsed_trading_minutes(at(13, 30)) == 150
|
||||
assert _elapsed_trading_minutes(at(15, 0)) == 240
|
||||
assert _elapsed_trading_minutes(at(20, 0)) == 240 # 收盘后
|
||||
|
||||
|
||||
def test_bars_for_range_shrinks_intraday():
|
||||
"""盘中只需已走完的根数,不是一整天 240 根。"""
|
||||
morning = datetime.datetime(2026, 10, 28, 10, 0) # noqa: DTZ001
|
||||
midnight = datetime.datetime(2026, 10, 28, 0, 0) # noqa: DTZ001
|
||||
# 10:00 走了 30 分钟 → 30 + 余量,远小于 240
|
||||
assert 30 <= _bars_for_range(KlinePeriod.MIN_1, midnight, morning) <= 40
|
||||
# 收盘后回到一整天的量级
|
||||
close = datetime.datetime(2026, 10, 28, 16, 0) # noqa: DTZ001
|
||||
assert _bars_for_range(KlinePeriod.MIN_1, midnight, close) >= 245
|
||||
|
||||
|
||||
def test_bars_for_range_past_days():
|
||||
now = datetime.datetime(2026, 10, 28, 16, 0) # noqa: DTZ001
|
||||
start = now - datetime.timedelta(days=5)
|
||||
n = _bars_for_range(KlinePeriod.DAY, start, now)
|
||||
assert 5 <= n <= 12 # 5 个自然日 × 1.6 余量
|
||||
|
||||
|
||||
def test_auto_pipeline():
|
||||
# 每根 32B:小 count 用深流水,大 count 用浅流水(在途字节数恒定)
|
||||
assert _auto_pipeline(2, None) == 32
|
||||
assert _auto_pipeline(240, None) == 8
|
||||
assert _auto_pipeline(240, 4) == 4 # 显式指定优先
|
||||
assert _auto_pipeline(2, 1) == 2 # 夹到最小值 2
|
||||
@@ -0,0 +1,125 @@
|
||||
"""CLI 价格小数位 / 分时时间列的显示回归。
|
||||
|
||||
覆盖两个曾经出错的地方:
|
||||
|
||||
1. ``tdx minute`` 的时间列按序号推算(``i // 60, i % 60``),240 根分时被打印成
|
||||
``00:00~03:59``,与真实交易时间无关。
|
||||
2. 价格固定写死 ``%8.2f``,可转债(报价步长 0.001 元)会被截成 2 位
|
||||
——``128.892`` 打成 ``128.89``。
|
||||
|
||||
注意 ``price_decimal_digits`` 返回的是**解码位数**(原始整数 / 10**dp),可转债
|
||||
是 4 位,但第 4 位恒为 0,显示层只该给 3 位(见 ``_display_dp``)。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from pytdxdata.cli import (
|
||||
_display_dp,
|
||||
_dp_of,
|
||||
_price_fmt,
|
||||
cmd_minute,
|
||||
cmd_transactions,
|
||||
parse_symbol,
|
||||
)
|
||||
from pytdxdata.models import MinuteBar, TransactionRecord
|
||||
|
||||
|
||||
class FakeTD:
|
||||
"""只实现 minute / transactions 需要的两个接口。"""
|
||||
|
||||
def __init__(self, minutes=None, ticks=None):
|
||||
self._minutes = minutes or []
|
||||
self._ticks = ticks or []
|
||||
|
||||
async def get_minutes(self, sym, date=None):
|
||||
return self._minutes
|
||||
|
||||
async def get_ticks(self, sym, date=None, count=None):
|
||||
return self._ticks
|
||||
|
||||
|
||||
def _bar(price, hour, minute, vol=100.0, avg=0.0):
|
||||
return MinuteBar(price=price, vol=vol, avg_price=avg or price,
|
||||
hour=hour, minute=minute, fields={})
|
||||
|
||||
|
||||
def _tick(price, vol, trade_count=1, available=True, time="14:59:54"):
|
||||
return TransactionRecord(market=1, code="113708", time=time, price=price, vol=vol,
|
||||
trade_count=trade_count, bs_flag=0, fields={},
|
||||
trade_count_available=available)
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# 小数位判定
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
def test_price_fmt_width_tracks_decimals():
|
||||
assert _price_fmt(2) == "%8.2f" # 股票:保持原宽度,不破坏对齐
|
||||
assert _price_fmt(3) == "%9.3f"
|
||||
assert _price_fmt(4) == "%10.4f"
|
||||
assert _price_fmt(None) == "%8.2f" # 缺省当 2 位
|
||||
|
||||
|
||||
def test_display_dp_by_instrument():
|
||||
assert _display_dp(1, "600000") == 2 # 沪股票
|
||||
assert _display_dp(0, "000001") == 2 # 深股票
|
||||
assert _display_dp(1, "000001") == 2 # 指数
|
||||
assert _display_dp(1, "113708") == 3 # 沪可转债(解码 4 位,显示 3 位)
|
||||
assert _display_dp(1, "110102") == 3
|
||||
assert _display_dp(0, "123235") == 3 # 深可转债
|
||||
assert _display_dp(0, "128136") == 3
|
||||
assert _display_dp(1, "510310") == 3 # ETF
|
||||
assert _display_dp(1, "019547") == 4 # 国债:4 位是真实报价精度
|
||||
|
||||
|
||||
def test_dp_of_uses_cn_only():
|
||||
assert _dp_of(parse_symbol("sh113708")) == 3
|
||||
assert _dp_of(parse_symbol("sh600000")) == 2
|
||||
assert _dp_of(parse_symbol("hk00700")) == 2 # 非 A 股固定 2 位
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# tdx minute
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
async def test_cmd_minute_prints_real_clock(capsys):
|
||||
bars = [_bar(129.271, 9, 30), _bar(129.270, 9, 31), _bar(128.892, 14, 59)]
|
||||
await cmd_minute(FakeTD(minutes=bars), "sh113708", None)
|
||||
out = capsys.readouterr().out
|
||||
assert "09:30" in out and "09:31" in out and "14:59" in out
|
||||
assert "00:00" not in out and "03:59" not in out
|
||||
|
||||
|
||||
async def test_cmd_minute_cb_keeps_three_decimals(capsys):
|
||||
await cmd_minute(FakeTD(minutes=[_bar(128.892, 14, 59)]), "sh113708", None)
|
||||
out = capsys.readouterr().out
|
||||
assert "128.892" in out
|
||||
assert "128.89 " not in out
|
||||
|
||||
|
||||
async def test_cmd_minute_stock_two_decimals(capsys):
|
||||
await cmd_minute(FakeTD(minutes=[_bar(9.62, 15, 0)]), "sh600000", None)
|
||||
out = capsys.readouterr().out
|
||||
assert "9.62" in out and "9.620" not in out
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# tdx transactions
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
async def test_cmd_transactions_cb_price_and_count(capsys):
|
||||
await cmd_transactions(FakeTD(ticks=[_tick(128.892, 3, trade_count=4)]),
|
||||
"sh113708", 50, None)
|
||||
out = capsys.readouterr().out
|
||||
assert "128.892" in out
|
||||
assert "(4笔)" in out
|
||||
|
||||
|
||||
async def test_cmd_transactions_etf_marks_missing_count(capsys):
|
||||
tick = TransactionRecord(market=1, code="510310", time="15:27:35", price=3.989,
|
||||
vol=15, trade_count=0, bs_flag=0, fields={},
|
||||
trade_count_available=False)
|
||||
await cmd_transactions(FakeTD(ticks=[tick]), "sh510310", 50, None)
|
||||
out = capsys.readouterr().out
|
||||
assert "3.989" in out # ETF 也是 3 位
|
||||
assert "(-)" in out # 不打印假笔数
|
||||
assert "笔)" not in out
|
||||
+147
-9
@@ -6,14 +6,19 @@ from __future__ import annotations
|
||||
|
||||
import io
|
||||
import struct
|
||||
import unittest.mock as mock
|
||||
import zipfile
|
||||
from unittest import mock
|
||||
|
||||
import pytest
|
||||
|
||||
from pytdxdata.api import TdxData
|
||||
from pytdxdata.codec.block import infer_block_category, parse_block_dat
|
||||
from pytdxdata.codec.financial import parse_financial_dat, parse_financial_file_list
|
||||
from pytdxdata.codec.financial import (
|
||||
extract_gpcw_dat,
|
||||
looks_like_gpcw_dat,
|
||||
parse_financial_dat,
|
||||
parse_financial_file_list,
|
||||
)
|
||||
from pytdxdata.codec.industry import parse_tdxhy_cfg
|
||||
from pytdxdata.models import FinancialFileInfo, FinancialRecord, IndustryInfo, TdxBlock
|
||||
|
||||
@@ -252,7 +257,7 @@ def _new_tdxdata() -> TdxData:
|
||||
async def test_api_get_block_parsed():
|
||||
td = _new_tdxdata()
|
||||
raw = _make_block_dat([("银行", ["000001"], 1, 1)])
|
||||
with mock.patch.object(td, "get_block_info", new=mock.AsyncMock(return_value=raw)):
|
||||
with mock.patch.object(td, "get_file", new=mock.AsyncMock(return_value=raw)):
|
||||
blocks = await td.get_block_parsed("block_gn.dat")
|
||||
assert len(blocks) == 1
|
||||
assert blocks[0].category == 2 # gn -> 概念
|
||||
@@ -263,7 +268,7 @@ async def test_api_get_industry_map():
|
||||
td = _new_tdxdata()
|
||||
raw = "0|000001|A01|||S01\n".encode("gbk")
|
||||
patched = mock.AsyncMock(return_value=raw)
|
||||
with mock.patch.object(td, "get_report_file", new=patched):
|
||||
with mock.patch.object(td, "get_file", new=patched):
|
||||
result = await td.get_industry_map()
|
||||
patched.assert_awaited_once_with("tdxhy.cfg")
|
||||
assert result["000001"].sw_industry == "S01"
|
||||
@@ -274,7 +279,7 @@ async def test_api_get_financial_file_infos():
|
||||
td = _new_tdxdata()
|
||||
raw = b"gpcw20260331.zip,abc,1024\n"
|
||||
patched = mock.AsyncMock(return_value=raw)
|
||||
with mock.patch.object(td, "get_report_file", new=patched):
|
||||
with mock.patch.object(td, "get_file", new=patched):
|
||||
infos = await td.get_financial_file_infos()
|
||||
patched.assert_awaited_once_with("tdxfin/gpcw.txt")
|
||||
assert infos[0].filesize == 1024
|
||||
@@ -284,7 +289,7 @@ async def test_api_get_financial_file_infos():
|
||||
async def test_api_get_financial_records_parsed():
|
||||
td = _new_tdxdata()
|
||||
zip_bytes = _make_financial_zip(_valid_two_record_dat())
|
||||
with mock.patch.object(td, "get_report_file",
|
||||
with mock.patch.object(td, "get_file",
|
||||
new=mock.AsyncMock(return_value=zip_bytes)):
|
||||
recs = await td.get_financial_records_parsed("tdxfin/gpcw20260331.zip")
|
||||
assert len(recs) == 2
|
||||
@@ -294,7 +299,7 @@ async def test_api_get_financial_records_parsed():
|
||||
@pytest.mark.asyncio
|
||||
async def test_api_get_financial_records_parsed_empty_zip_data():
|
||||
td = _new_tdxdata()
|
||||
with mock.patch.object(td, "get_report_file",
|
||||
with mock.patch.object(td, "get_file",
|
||||
new=mock.AsyncMock(return_value=b"")):
|
||||
assert await td.get_financial_records_parsed("tdxfin/gpcw20260331.zip") == []
|
||||
|
||||
@@ -303,7 +308,7 @@ async def test_api_get_financial_records_parsed_empty_zip_data():
|
||||
async def test_api_get_financial_records_parsed_no_dat_member():
|
||||
td = _new_tdxdata()
|
||||
zip_bytes = _make_financial_zip(b"whatever", inner_name="readme.txt")
|
||||
with mock.patch.object(td, "get_report_file",
|
||||
with mock.patch.object(td, "get_file",
|
||||
new=mock.AsyncMock(return_value=zip_bytes)):
|
||||
assert await td.get_financial_records_parsed("nodate.zip") == []
|
||||
|
||||
@@ -313,7 +318,140 @@ async def test_api_get_financial_records_parsed_no_date_in_name():
|
||||
"""文件名无 8 位日期 -> 回退用 .dat 头部报告期。"""
|
||||
td = _new_tdxdata()
|
||||
zip_bytes = _make_financial_zip(_valid_two_record_dat())
|
||||
with mock.patch.object(td, "get_report_file",
|
||||
with mock.patch.object(td, "get_file",
|
||||
new=mock.AsyncMock(return_value=zip_bytes)):
|
||||
recs = await td.get_financial_records_parsed("gpcw.zip")
|
||||
assert recs[0].report_date == 20260331
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# looks_like_gpcw_dat / extract_gpcw_dat(BUG-2 修复的判据层)
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
def test_looks_like_gpcw_dat_accepts_real_dat():
|
||||
assert looks_like_gpcw_dat(_valid_two_record_dat()) is True
|
||||
|
||||
|
||||
def test_looks_like_gpcw_dat_rejects_short_shell():
|
||||
"""服务器对不存在的文件名回的空壳(0~1 字节)必须判否。"""
|
||||
assert looks_like_gpcw_dat(b"") is False
|
||||
assert looks_like_gpcw_dat(b"\x00") is False
|
||||
assert looks_like_gpcw_dat(b"abcd") is False
|
||||
|
||||
|
||||
def test_looks_like_gpcw_dat_rejects_zero_header():
|
||||
"""长度够但 header 全零(max_count=0 / report_size=0)也要判否。"""
|
||||
assert looks_like_gpcw_dat(b"\x00" * 64) is False
|
||||
|
||||
|
||||
def test_looks_like_gpcw_dat_rejects_report_size_not_multiple_of_4():
|
||||
data = _make_dat_raw([(b"600000", 1, 31)], struct.pack("<2f", 1.0, 2.0),
|
||||
report_size=6)
|
||||
assert looks_like_gpcw_dat(data) is False
|
||||
|
||||
|
||||
def test_looks_like_gpcw_dat_rejects_truncated_index():
|
||||
"""header 声称的条目数超出实际长度 -> 判否。"""
|
||||
data = _make_dat_raw([(b"600000", 1, 31)], struct.pack("<2f", 1.0, 2.0),
|
||||
max_count=100)
|
||||
assert looks_like_gpcw_dat(data) is False
|
||||
|
||||
|
||||
def test_extract_gpcw_dat_from_uncompressed():
|
||||
dat = _valid_two_record_dat()
|
||||
assert extract_gpcw_dat(dat) == dat
|
||||
|
||||
|
||||
def test_extract_gpcw_dat_from_zip():
|
||||
dat = _valid_two_record_dat()
|
||||
assert extract_gpcw_dat(_make_financial_zip(dat)) == dat
|
||||
|
||||
|
||||
def test_extract_gpcw_dat_empty_and_garbage():
|
||||
assert extract_gpcw_dat(b"") == b""
|
||||
assert extract_gpcw_dat(b"\x00") == b""
|
||||
assert extract_gpcw_dat(b"not a financial file") == b""
|
||||
|
||||
|
||||
def test_extract_gpcw_dat_zip_without_dat_member():
|
||||
assert extract_gpcw_dat(_make_financial_zip(b"x", inner_name="readme.txt")) == b""
|
||||
|
||||
|
||||
def test_extract_gpcw_dat_zip_with_invalid_inner_dat():
|
||||
"""zip 里虽有 .dat,但内容不是有效财报数据 -> 空。"""
|
||||
assert extract_gpcw_dat(_make_financial_zip(b"\x00" * 4)) == b""
|
||||
|
||||
|
||||
def test_extract_gpcw_dat_broken_zip():
|
||||
"""以 PK 开头但不是合法 zip -> 空,不抛异常。"""
|
||||
assert extract_gpcw_dat(b"PK\x03\x04broken-not-a-zip") == b""
|
||||
|
||||
|
||||
# --------------------------------------------------------------------------- #
|
||||
# TdxData._fetch_financial_dat 候选回退(BUG-2 回归)
|
||||
# --------------------------------------------------------------------------- #
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_fetch_financial_dat_falls_back_to_uncompressed_dat():
|
||||
"""索引名 ``gpcw*.zip`` 取不到时,必须回退到服务器实际的 ``tdxfin/*.dat``。"""
|
||||
td = _new_tdxdata()
|
||||
dat = _valid_two_record_dat()
|
||||
seen: list[str] = []
|
||||
|
||||
async def fake(name: str) -> bytes:
|
||||
seen.append(name)
|
||||
if name == "tdxfin/gpcw20260331.dat":
|
||||
return dat
|
||||
if name == "gpcw20260331.zip":
|
||||
return b"\x00" # 裸名 -> MAC 路由 -> 1 字节空壳
|
||||
return b"" # 带前缀的 .zip 不存在
|
||||
|
||||
with mock.patch.object(td, "get_file", new=fake):
|
||||
got = await td._fetch_financial_dat("gpcw20260331.zip")
|
||||
|
||||
assert got == dat
|
||||
assert seen == ["tdxfin/gpcw20260331.zip", "tdxfin/gpcw20260331.dat"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_fetch_financial_dat_prefers_declared_zip_when_present():
|
||||
"""服务器若恢复提供 .zip,应优先用索引声明的那个名字。"""
|
||||
td = _new_tdxdata()
|
||||
dat = _valid_two_record_dat()
|
||||
seen: list[str] = []
|
||||
|
||||
async def fake(name: str) -> bytes:
|
||||
seen.append(name)
|
||||
return _make_financial_zip(dat)
|
||||
|
||||
with mock.patch.object(td, "get_file", new=fake):
|
||||
got = await td._fetch_financial_dat("tdxfin/gpcw20260331.zip")
|
||||
|
||||
assert got == dat
|
||||
assert seen == ["tdxfin/gpcw20260331.zip"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_fetch_financial_dat_skips_raising_candidate():
|
||||
"""某个候选取回时抛异常,不应中断,继续试后面的候选。"""
|
||||
td = _new_tdxdata()
|
||||
dat = _valid_two_record_dat()
|
||||
|
||||
async def fake(name: str) -> bytes:
|
||||
if name.endswith(".zip"):
|
||||
raise ConnectionError("模拟网络错误")
|
||||
return dat
|
||||
|
||||
with mock.patch.object(td, "get_file", new=fake):
|
||||
assert await td._fetch_financial_dat("gpcw20260331.zip") == dat
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_fetch_financial_dat_all_candidates_fail_returns_empty():
|
||||
td = _new_tdxdata()
|
||||
|
||||
async def fake(name: str) -> bytes:
|
||||
return b""
|
||||
|
||||
with mock.patch.object(td, "get_file", new=fake):
|
||||
assert await td._fetch_financial_dat("gpcw20260331.zip") == b""
|
||||
@@ -0,0 +1,233 @@
|
||||
"""权威价格小数位表(decimal_sync)单元测试。
|
||||
|
||||
覆盖点:
|
||||
- 权威表(来自 ``SecurityInfo.decimal_point``)**优先于**静态前缀规则
|
||||
- 某前缀出现混合小数位 → 自动退化为逐代码查表,不误伤同前缀其它代码
|
||||
- 清空权威表 → 回到静态规则(与 0.6.5 行为一致)
|
||||
- 磁盘缓存:写入 / 读取 / 跨日失效 / 损坏容错,均不回退到网络
|
||||
- ``PYPYTDX_DATA_DISABLE_DECIMAL_SYNC=1`` → 完全不动作
|
||||
- 证券清单里 ``decimal_point == 0`` 的脏条目被忽略
|
||||
- 后台 task 可取消,且 ``close()`` 后不再改写表
|
||||
(全部注入假 ``TdxData``,不连真实网络)
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
|
||||
from pytdxdata import decimal_sync
|
||||
from pytdxdata.protocol.commands.quotes import (
|
||||
clear_decimal_map,
|
||||
decimal_map_stats,
|
||||
install_decimal_map,
|
||||
price_decimal_digits,
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# 夹具
|
||||
# ---------------------------------------------------------------------- #
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _isolate(tmp_path, monkeypatch):
|
||||
"""每个用例独立缓存目录 + 干净的内存状态。"""
|
||||
monkeypatch.setenv("PYPYTDX_DATA_CACHE_DIR", str(tmp_path))
|
||||
monkeypatch.delenv("PYPYTDX_DATA_DISABLE_DECIMAL_SYNC", raising=False)
|
||||
decimal_sync.reset()
|
||||
yield
|
||||
decimal_sync.reset()
|
||||
|
||||
|
||||
def _info(code: str, dp: int) -> SimpleNamespace:
|
||||
return SimpleNamespace(code=code, decimal_point=dp, market=1)
|
||||
|
||||
|
||||
class FakeTd:
|
||||
"""只有 ``get_universe`` 的假客户端(记录调用次数,便于断言"没走网络")。"""
|
||||
|
||||
def __init__(self, tables: dict[str, list[tuple[str, int]]] | None = None) -> None:
|
||||
self.tables = tables or {}
|
||||
self.calls: list[str] = []
|
||||
|
||||
async def get_universe(self, market: str | None = None):
|
||||
self.calls.append(market or "all")
|
||||
return [_info(c, d) for c, d in self.tables.get(market, [])]
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# 权威表优先级
|
||||
# ---------------------------------------------------------------------- #
|
||||
|
||||
def test_authoritative_map_overrides_static_rule():
|
||||
# 静态规则说 sh019742 是 4 位;权威表说 3 位 → 必须听权威表的
|
||||
assert price_decimal_digits(1, "019742") == 4
|
||||
install_decimal_map([(1, "019742", 3)], source="test")
|
||||
assert price_decimal_digits(1, "019742") == 3
|
||||
|
||||
|
||||
def test_codes_outside_map_still_use_static_rule():
|
||||
install_decimal_map([(1, "019742", 3)], source="test")
|
||||
assert price_decimal_digits(1, "600000") == 2 # 股票
|
||||
assert price_decimal_digits(1, "510300") == 3 # ETF
|
||||
assert price_decimal_digits(0, "131810") == 4 # 深回购(表里没有)
|
||||
|
||||
|
||||
def test_mixed_prefix_falls_back_to_per_code_lookup():
|
||||
"""同一前缀出现两种小数位 → 该前缀不进前缀表,只把例外代码存进逐代码表。"""
|
||||
stats = install_decimal_map(
|
||||
[(1, "110001", 4), (1, "110002", 4), (1, "110003", 3)], source="test")
|
||||
assert stats["mixed_prefixes"] == 1
|
||||
assert stats["codes"] == 3 # 混合前缀下所有代码都进逐代码表
|
||||
assert price_decimal_digits(1, "110001") == 4
|
||||
assert price_decimal_digits(1, "110003") == 3
|
||||
|
||||
|
||||
def test_clear_map_restores_static_rules():
|
||||
install_decimal_map([(1, "019742", 3)], source="test")
|
||||
assert price_decimal_digits(1, "019742") == 3
|
||||
clear_decimal_map()
|
||||
assert decimal_map_stats()["stock"] == 0
|
||||
assert price_decimal_digits(1, "019742") == 4
|
||||
|
||||
|
||||
def test_reinstall_replaces_previous_map():
|
||||
install_decimal_map([(1, "019742", 3)], source="a")
|
||||
install_decimal_map([(1, "019742", 4)], source="b")
|
||||
assert price_decimal_digits(1, "019742") == 4
|
||||
assert decimal_map_stats()["prefixes"] == 1
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# 同步
|
||||
# ---------------------------------------------------------------------- #
|
||||
|
||||
def test_refresh_installs_and_writes_cache(tmp_path):
|
||||
td = FakeTd({"sh": [("019742", 4), ("600000", 2)],
|
||||
"sz": [("000001", 2), ("131810", 4)]})
|
||||
stats = asyncio.run(decimal_sync.refresh(td))
|
||||
|
||||
assert td.calls == ["sh", "sz"]
|
||||
assert stats["stock"] == 4
|
||||
assert price_decimal_digits(1, "600000") == 2
|
||||
assert price_decimal_digits(0, "131810") == 4
|
||||
|
||||
raw = json.loads((tmp_path / "decimal-points.json").read_text(encoding="utf-8"))
|
||||
assert raw["markets"]["1"]["019742"] == 4
|
||||
assert raw["markets"]["0"]["131810"] == 4
|
||||
|
||||
|
||||
def test_refresh_skips_zero_decimal_point():
|
||||
td = FakeTd({"sh": [("019742", 0), ("600000", 2)]})
|
||||
stats = asyncio.run(decimal_sync.refresh(td, ("sh",)))
|
||||
assert stats["stock"] == 1
|
||||
|
||||
|
||||
def test_refresh_ignores_unknown_market():
|
||||
td = FakeTd({"hk": [("00700", 3)]})
|
||||
stats = asyncio.run(decimal_sync.refresh(td, ("hk",)))
|
||||
assert stats["stock"] == 0
|
||||
assert not (decimal_sync.cache_file()).exists()
|
||||
|
||||
|
||||
def test_load_cached_roundtrip(tmp_path):
|
||||
td = FakeTd({"sh": [("019742", 4)]})
|
||||
asyncio.run(decimal_sync.refresh(td))
|
||||
decimal_sync.reset()
|
||||
assert decimal_map_stats()["stock"] == 0
|
||||
|
||||
assert decimal_sync.load_cached() == 1
|
||||
assert price_decimal_digits(1, "019742") == 4
|
||||
|
||||
|
||||
def test_cache_expires_across_days(tmp_path):
|
||||
td = FakeTd({"sh": [("019742", 4)]})
|
||||
asyncio.run(decimal_sync.refresh(td))
|
||||
decimal_sync.reset()
|
||||
|
||||
path = tmp_path / "decimal-points.json"
|
||||
payload = json.loads(path.read_text(encoding="utf-8"))
|
||||
payload["date"] = "2000-01-01"
|
||||
path.write_text(json.dumps(payload), encoding="utf-8")
|
||||
|
||||
assert decimal_sync.load_cached() == 0
|
||||
|
||||
|
||||
def test_corrupt_cache_is_ignored(tmp_path):
|
||||
path = tmp_path / "decimal-points.json"
|
||||
path.write_text("{not json", encoding="utf-8")
|
||||
assert decimal_sync.load_cached() == 0
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------- #
|
||||
# 调度
|
||||
# ---------------------------------------------------------------------- #
|
||||
|
||||
def test_schedule_disabled_by_env(monkeypatch):
|
||||
monkeypatch.setenv("PYPYTDX_DATA_DISABLE_DECIMAL_SYNC", "1")
|
||||
td = FakeTd({"sh": [("019742", 4)]})
|
||||
|
||||
async def run():
|
||||
task = decimal_sync.schedule(td)
|
||||
return task
|
||||
|
||||
assert asyncio.run(run()) is None
|
||||
assert td.calls == []
|
||||
|
||||
|
||||
def test_schedule_loads_cache_without_network(tmp_path):
|
||||
td = FakeTd({"sh": [("019742", 4)]})
|
||||
asyncio.run(decimal_sync.refresh(td))
|
||||
decimal_sync.reset()
|
||||
quiet = FakeTd({"sh": [("019742", 4)]})
|
||||
|
||||
async def run():
|
||||
return decimal_sync.schedule(quiet)
|
||||
|
||||
assert asyncio.run(run()) is None # 缓存命中 → 不起 task
|
||||
assert quiet.calls == [] # 也没走网络
|
||||
assert price_decimal_digits(1, "019742") == 4
|
||||
|
||||
|
||||
def test_schedule_runs_background_sync_without_cache():
|
||||
td = FakeTd({"sh": [("019742", 4)]})
|
||||
|
||||
async def run():
|
||||
task = decimal_sync.schedule(td)
|
||||
assert task is not None
|
||||
await task
|
||||
return decimal_sync.status()
|
||||
|
||||
st = asyncio.run(run())
|
||||
assert st["source"] == "universe"
|
||||
assert st["syncing"] is False
|
||||
assert td.calls == ["sh", "sz"] # 默认两市
|
||||
|
||||
|
||||
def test_cancel_stops_background_task():
|
||||
class SlowTd:
|
||||
def __init__(self) -> None:
|
||||
self.calls = 0
|
||||
|
||||
async def get_universe(self, market=None):
|
||||
self.calls += 1
|
||||
await asyncio.sleep(5)
|
||||
return []
|
||||
|
||||
td = SlowTd()
|
||||
|
||||
async def run():
|
||||
task = decimal_sync.schedule(td)
|
||||
await asyncio.sleep(0.01)
|
||||
await decimal_sync.cancel(task)
|
||||
return task.done() or task.cancelled()
|
||||
|
||||
assert asyncio.run(run()) is True
|
||||
|
||||
|
||||
def test_status_shape():
|
||||
st = decimal_sync.status()
|
||||
assert set(st) >= {"loaded", "source", "syncing", "prefixes", "codes",
|
||||
"mixed_prefixes", "stock"}
|
||||
@@ -0,0 +1,175 @@
|
||||
"""EX 0x23FF K线的字段语义:``amount`` / ``settle`` 按市场分别回填。
|
||||
|
||||
背景(2026-10-28 实测):0x23FF 每条 32B = 4B 时间戳 + **28B 行情**,这 28B 只有
|
||||
**一个** 4B 槽位可复用,语义随市场变化 ——
|
||||
|
||||
* 权益类(港股主板/创业板/基金、新三板、美股):该槽位是**成交额**,
|
||||
校验 ``成交额 / (close × vol)`` ≈ 100(港股主板/基金,vol 单位是手)或 ≈ 1(其余,vol 是股);
|
||||
* 期货期权类:该槽位是**结算价**,成交额在帧内根本不存在(恒 0.0)。
|
||||
|
||||
改之前 ``amount`` 被硬编码成 0.0、末位 float 直接丢弃,于是港股/美股拿不到成交额、
|
||||
期货期权拿不到结算价。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import pickle
|
||||
import struct
|
||||
|
||||
import pytest
|
||||
|
||||
from pytdxdata.api import TdxData
|
||||
from pytdxdata.models import SecurityBar
|
||||
from pytdxdata.protocol.commands.ex_proto import (
|
||||
_EX_AMOUNT_MARKETS,
|
||||
GetExInstrumentBarsCmd,
|
||||
)
|
||||
from pytdxdata.symbols import parse_symbols
|
||||
|
||||
DAY = 20261028
|
||||
|
||||
|
||||
def _f2u(value: float) -> int:
|
||||
"""float32 -> 其位模式的 uint32(用来拼「同一槽位两种解释」的报文)。"""
|
||||
return struct.unpack("<I", struct.pack("<f", value))[0]
|
||||
|
||||
|
||||
def _ex_body(rows: list[tuple[int, ...]], category: int = 4) -> bytes:
|
||||
"""拼 0x23FF 响应体:20B 头(count@18)+ N × 32B(时间4B + 7×4B)。"""
|
||||
body = bytearray(20 + 32 * len(rows))
|
||||
struct.pack_into("<H", body, 18, len(rows))
|
||||
for i, words in enumerate(rows):
|
||||
off = 20 + i * 32
|
||||
struct.pack_into("<I", body, off, DAY)
|
||||
struct.pack_into("<7I", body, off + 4, *words)
|
||||
return bytes(body)
|
||||
|
||||
|
||||
def _equity_words(close: float, amount: float, vol: int) -> tuple[int, ...]:
|
||||
"""权益类布局:OHLC + 成交额(float) + 成交量(uint) + 保留(float)。"""
|
||||
return (_f2u(close), _f2u(close), _f2u(close), _f2u(close),
|
||||
_f2u(amount), vol, _f2u(0.0))
|
||||
|
||||
|
||||
def _derivative_words(close: float, position: int, vol: int,
|
||||
settle: float) -> tuple[int, ...]:
|
||||
"""期货/期权布局:OHLC + 持仓量(uint) + 成交量(uint) + 结算价(float)。"""
|
||||
return (_f2u(close), _f2u(close), _f2u(close), _f2u(close),
|
||||
position, vol, _f2u(settle))
|
||||
|
||||
|
||||
# ---------- 解析层 ----------
|
||||
|
||||
@pytest.mark.parametrize("market", [31, 48, 49, 44, 74])
|
||||
def test_equity_markets_read_amount_from_fifth_slot(market: int) -> None:
|
||||
cmd = GetExInstrumentBarsCmd(market, "00700", 4, 0, 1)
|
||||
rows = cmd.parse_response(_ex_body([_equity_words(10.2, 2_040_000.0, 2000)]))
|
||||
|
||||
assert len(rows) == 1
|
||||
assert rows[0]["amount"] == pytest.approx(2_040_000.0)
|
||||
assert rows[0]["trade"] == 2000 # 成交量不受影响
|
||||
assert rows[0]["position"] == 0 # 权益类无持仓量概念
|
||||
assert rows[0]["settle"] is None
|
||||
|
||||
|
||||
@pytest.mark.parametrize("market", [7, 8, 9, 47])
|
||||
def test_derivative_markets_read_settle_from_seventh_slot(market: int) -> None:
|
||||
cmd = GetExInstrumentBarsCmd(market, "IFL0", 4, 0, 1)
|
||||
rows = cmd.parse_response(_ex_body([_derivative_words(4021.0, 49_772, 25_392, 4047.8)]))
|
||||
|
||||
assert rows[0]["amount"] == 0.0 # 帧内没有成交额,不是解析漏读
|
||||
assert rows[0]["position"] == 49_772
|
||||
assert rows[0]["trade"] == 25_392
|
||||
assert rows[0]["settle"] == pytest.approx(4047.8)
|
||||
|
||||
|
||||
def test_intraday_settle_zero_normalized_to_none() -> None:
|
||||
"""结算价是日终量:分钟线该槽位恒 0,归一化成 None 而不是 0.0。"""
|
||||
cmd = GetExInstrumentBarsCmd(47, "IFL0", 7, 0, 1)
|
||||
rows = cmd.parse_response(_ex_body([_derivative_words(4054.6, 49_490, 260, 0.0)], 7))
|
||||
|
||||
assert rows[0]["settle"] is None
|
||||
|
||||
|
||||
def test_index_markets_kept_out_of_amount_table() -> None:
|
||||
"""港股指数(27)/中证指数(62) 该槽位既非成交额也非结算价 —— 不能猜,保持 0.0。"""
|
||||
assert 27 not in _EX_AMOUNT_MARKETS
|
||||
assert 62 not in _EX_AMOUNT_MARKETS
|
||||
cmd = GetExInstrumentBarsCmd(27, "HSI", 4, 0, 1)
|
||||
rows = cmd.parse_response(_ex_body([_equity_words(23041.32, 178_348.0, 17_835)]))
|
||||
assert rows[0]["amount"] == 0.0
|
||||
|
||||
|
||||
# ---------- API 层透传 ----------
|
||||
|
||||
def _fake_api(rows_for: dict[int, list[dict]]) -> TdxData:
|
||||
td = TdxData(cache_dir=None, pool_min=1, pool_max=2)
|
||||
seen: list[tuple[int, str]] = []
|
||||
|
||||
async def fake(make_cmd, spec): # noqa: ANN001 — 与真实签名同形
|
||||
cmd = make_cmd(0, spec.page_size)
|
||||
seen.append((cmd.market, cmd.code))
|
||||
return rows_for[cmd.market]
|
||||
|
||||
td._fetch_paginated_ex = fake # type: ignore[method-assign]
|
||||
td._seen_ex = seen # type: ignore[attr-defined]
|
||||
return td
|
||||
|
||||
|
||||
def _row(market: int, code: str, **kw) -> dict:
|
||||
base = {"year": 2026, "month": 10, "day": 28, "hour": 15, "minute": 0,
|
||||
"open": 10.0, "high": 10.5, "low": 9.5, "close": 10.2,
|
||||
"amount": 0.0, "position": 0, "trade": 2000, "settle": None}
|
||||
base.update(kw)
|
||||
return base
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_bars_ex_backfills_amount_for_hk() -> None:
|
||||
td = _fake_api({31: [_row(31, "00700", amount=2_040_000.0)]})
|
||||
bars = await td.get_bars("hk00700", count=1)
|
||||
|
||||
assert len(bars) == 1
|
||||
assert bars[0].amount == pytest.approx(2_040_000.0)
|
||||
assert bars[0].vol == 2000
|
||||
assert bars[0].settle is None
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_bars_ex_backfills_settle_for_futures() -> None:
|
||||
td = _fake_api({47: [_row(47, "IFL0", position=49_772, trade=25_392, settle=4047.8)]})
|
||||
bars = await td.get_bars("cffex_futures:IFL0", count=1)
|
||||
|
||||
assert bars[0].settle == pytest.approx(4047.8)
|
||||
assert bars[0].amount == 0.0
|
||||
|
||||
|
||||
# ---------- 旧磁盘缓存兼容 ----------
|
||||
|
||||
def test_legacy_pickle_without_settle_still_loads() -> None:
|
||||
"""升级前写下的缓存没有 settle 槽位;默认反序列化不会补默认值,读它必崩。"""
|
||||
bar = SecurityBar(market=1, code="600000", open=1.0, high=2.0, low=0.5,
|
||||
close=1.5, vol=100.0, amount=150.0,
|
||||
year=2026, month=10, day=28, settle=12.3)
|
||||
_, slots = bar.__getstate__()
|
||||
legacy = dict(slots)
|
||||
legacy.pop("settle")
|
||||
|
||||
revived = SecurityBar.__new__(SecurityBar)
|
||||
revived.__setstate__((None, legacy))
|
||||
|
||||
assert revived.settle is None
|
||||
assert revived.close == 1.5
|
||||
|
||||
|
||||
def test_new_pickle_roundtrip_keeps_settle() -> None:
|
||||
bar = SecurityBar(market=47, code="IFL0", open=4021.0, high=4060.0, low=4010.0,
|
||||
close=4054.6, vol=25_618.0, amount=0.0,
|
||||
year=2026, month=10, day=28, settle=4052.6)
|
||||
|
||||
assert pickle.loads(pickle.dumps(bar)).settle == pytest.approx(4052.6)
|
||||
|
||||
|
||||
def test_ex_symbols_route_to_ex_path() -> None:
|
||||
"""守住前提:上面对 31/47 的断言确实走的是 EX 通道。"""
|
||||
assert parse_symbols(["hk00700"])[0].is_ex
|
||||
assert parse_symbols(["cffex_futures:IFL0"])[0].is_ex
|
||||
@@ -0,0 +1,116 @@
|
||||
"""ExPool 背压/并发语义回归(Issue #1:并发超限 0.15s 秒失败)。
|
||||
|
||||
用 mock 连接模拟:单服务器 + per_server_cap=1 时,
|
||||
- 并发 acquire 超过有效并发 → 排队等待而不是快速失败;
|
||||
- release 唤醒等待者;
|
||||
- acquire 超时异常带池状态;
|
||||
- ex_pool_cap 提升同主机并发。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import time
|
||||
|
||||
import pytest
|
||||
|
||||
from pytdxdata.pool.ex_pool import ExPool, ExPoolTimeoutError
|
||||
|
||||
|
||||
class MockConn:
|
||||
"""Minimal stand-in for ExTdxConnection."""
|
||||
|
||||
last_created = 0
|
||||
|
||||
def __init__(self, host: str, port: int, timeout: float, handshake: str = "login"):
|
||||
self.host = host
|
||||
self.port = port
|
||||
self.alive = True
|
||||
self.last_used = time.monotonic()
|
||||
self.closed = False
|
||||
MockConn.last_created += 1
|
||||
|
||||
async def connect(self) -> None:
|
||||
await asyncio.sleep(0) # instant "connect"
|
||||
|
||||
async def close(self) -> None:
|
||||
self.closed = True
|
||||
self.alive = False
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def patch_conn(monkeypatch):
|
||||
monkeypatch.setattr("pytdxdata.pool.ex_pool.ExTdxConnection", MockConn)
|
||||
MockConn.last_created = 0
|
||||
|
||||
|
||||
async def test_acquire_queues_instead_of_fast_fail(patch_conn):
|
||||
"""并发 > 有效并发时,后续 acquire 等待 release,而非 0.15s 抛错。"""
|
||||
pool = ExPool(["e1"], min_size=1, max_size=6, per_server_cap=1)
|
||||
await pool.start()
|
||||
c1 = await pool.acquire() # 唯一连接被占用
|
||||
t0 = time.monotonic()
|
||||
|
||||
async def holder():
|
||||
await asyncio.sleep(0.5) # 持有 0.5s 后归还
|
||||
await pool.release(c1)
|
||||
|
||||
asyncio.create_task(holder())
|
||||
c2 = await pool.acquire() # 应在 ~0.5s 后拿到,而不是 0.15s 抛错
|
||||
waited = time.monotonic() - t0
|
||||
assert waited >= 0.4, f"acquire 没有等待释放(仅 {waited:.2f}s)"
|
||||
assert c2 is c1 # 归还的是同一条连接
|
||||
|
||||
|
||||
async def test_acquire_timeout_carries_pool_state(patch_conn):
|
||||
"""超时异常信息带池状态(idle/total/servers/cap)。"""
|
||||
pool = ExPool(["e1"], min_size=1, max_size=6, per_server_cap=1, acquire_timeout=0.3)
|
||||
await pool.start()
|
||||
c1 = await pool.acquire()
|
||||
with pytest.raises(ExPoolTimeoutError) as ei:
|
||||
await pool.acquire()
|
||||
msg = str(ei.value)
|
||||
assert "total=1" in msg and "per_server_cap=1" in msg and "servers=1" in msg
|
||||
await pool.release(c1)
|
||||
|
||||
|
||||
async def test_ex_pool_cap_allows_parallel_conns_same_host(patch_conn):
|
||||
"""per_server_cap=2 时单主机可并发持有 2 条连接。"""
|
||||
pool = ExPool(["e1"], min_size=1, max_size=6, per_server_cap=2)
|
||||
await pool.start()
|
||||
got = await asyncio.gather(pool.acquire(), pool.acquire())
|
||||
assert len(got) == 2 and got[0] is not got[1]
|
||||
assert MockConn.last_created == 2
|
||||
for c in got:
|
||||
await pool.release(c)
|
||||
|
||||
|
||||
async def test_concurrent_acquire_all_succeed(patch_conn):
|
||||
"""Issue 场景:单服务器 + cap=2,6 个并发任务应全部拿到连接(排队)而非部分失败。"""
|
||||
pool = ExPool(["e1"], min_size=2, max_size=6, per_server_cap=2)
|
||||
await pool.start()
|
||||
|
||||
async def task(i):
|
||||
c = await pool.acquire()
|
||||
await asyncio.sleep(0.1) # 模拟一次 1m 分页拉取
|
||||
await pool.release(c)
|
||||
return i
|
||||
|
||||
results = await asyncio.gather(*[task(i) for i in range(6)])
|
||||
assert sorted(results) == list(range(6)) # 全部成功,零失败
|
||||
|
||||
|
||||
async def test_release_notifies_waiters(patch_conn):
|
||||
"""broken 归还(连接死亡)也要唤醒等待者(槽位回收后可扩容)。"""
|
||||
pool = ExPool(["e1"], min_size=1, max_size=1, per_server_cap=1)
|
||||
await pool.start()
|
||||
c1 = await pool.acquire()
|
||||
c1.alive = False
|
||||
|
||||
async def broken_release():
|
||||
await asyncio.sleep(0.2)
|
||||
await pool.release(c1, broken=True)
|
||||
|
||||
asyncio.create_task(broken_release())
|
||||
c2 = await pool.acquire() # 槽位被回收→扩容新建
|
||||
assert c2 is not c1
|
||||
await pool.release(c2)
|
||||
@@ -0,0 +1,325 @@
|
||||
"""DataFrame 转换层测试(0.7.0)。
|
||||
|
||||
覆盖两条独立路径:
|
||||
|
||||
1. **转换口径**:直接调 :func:`to_frame`,喂合成模型对象,断言列名/列序/取值/dtype。
|
||||
不碰网络,双后端各跑一遍。
|
||||
2. **接线**:``TdxData(df=...)`` 的取值校验、装饰器的「内层不二次转换」语义。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
from pytdxdata import TdxData
|
||||
from pytdxdata import frames as _frames
|
||||
from pytdxdata.frames import FrameBackendError, framed, get_backend, to_frame
|
||||
from pytdxdata.models import (
|
||||
IndexBar,
|
||||
IndexTradeStats,
|
||||
IndicatorSet,
|
||||
Market,
|
||||
MinuteBar,
|
||||
MinuteTradeStat,
|
||||
SecurityBar,
|
||||
SecurityQuote,
|
||||
TdxBlock,
|
||||
XdxrRecord,
|
||||
)
|
||||
|
||||
BACKENDS = ("polars", "pandas")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 工具
|
||||
|
||||
|
||||
def bar(**kw) -> SecurityBar:
|
||||
base = {
|
||||
"market": int(Market.SH), "code": "600000", "open": 10.0, "high": 11.0,
|
||||
"low": 9.5, "close": 10.5, "vol": 1000.0, "amount": 10500.0,
|
||||
"year": 2026, "month": 10, "day": 27, "hour": 0, "minute": 0,
|
||||
"symbol": "sh600000",
|
||||
}
|
||||
base.update(kw)
|
||||
return SecurityBar(**base)
|
||||
|
||||
|
||||
def rows(df, backend: str) -> int:
|
||||
return len(df) if backend == "pandas" else df.height
|
||||
|
||||
|
||||
def cols(df, backend: str) -> list[str]:
|
||||
return list(df.columns)
|
||||
|
||||
|
||||
def cell(df, backend: str, name: str, index: int = 0):
|
||||
if backend == "pandas":
|
||||
return df[name].iloc[index]
|
||||
# polars 2.x:Series[0] 会返回 Series 而不是标量,统一走 to_list()
|
||||
return df.get_column(name).to_list()[index]
|
||||
|
||||
|
||||
def missing(value) -> bool:
|
||||
"""pandas 的数值列会把 None 变成 NaN,两者都算「缺」。"""
|
||||
return value is None or (isinstance(value, float) and math.isnan(value))
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 后端
|
||||
|
||||
|
||||
def test_backend_instances_are_cached():
|
||||
assert get_backend("polars") is get_backend("polars")
|
||||
assert get_backend("pandas") is get_backend("pandas")
|
||||
|
||||
|
||||
def test_unknown_backend_raises_value_error():
|
||||
with pytest.raises(ValueError, match="未知的 DataFrame 后端"):
|
||||
get_backend("numpy")
|
||||
|
||||
|
||||
def test_missing_backend_raises_with_install_hint(monkeypatch):
|
||||
monkeypatch.setitem(sys.modules, "polars", None) # 让 import polars 抛 ImportError
|
||||
monkeypatch.delitem(_frames._BACKEND_CACHE, "polars", raising=False)
|
||||
with pytest.raises(FrameBackendError) as excinfo:
|
||||
get_backend("polars")
|
||||
assert isinstance(excinfo.value, ImportError)
|
||||
assert excinfo.value.name == "polars"
|
||||
assert "pip install pytdxdata[polars]" in str(excinfo.value)
|
||||
monkeypatch.delitem(_frames._BACKEND_CACHE, "polars", raising=False)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 转换口径
|
||||
|
||||
|
||||
@pytest.mark.parametrize("backend", BACKENDS)
|
||||
def test_bar_replaces_split_time_with_datetime(backend):
|
||||
df = to_frame([bar(), bar(minute=30)], backend=backend)
|
||||
assert cols(df, backend) == [
|
||||
"market", "code", "open", "high", "low", "close", "vol", "amount",
|
||||
"datetime", "trading_date", "symbol", "settle",
|
||||
]
|
||||
assert rows(df, backend) == 2
|
||||
assert cell(df, backend, "market") == 1
|
||||
assert cell(df, backend, "trading_date") == "2026-10-27"
|
||||
stamp = cell(df, backend, "datetime", 1)
|
||||
assert stamp.year == 2026 and stamp.hour == 0 and stamp.minute == 30
|
||||
|
||||
|
||||
@pytest.mark.parametrize("backend", BACKENDS)
|
||||
def test_int_enum_column_is_plain_int(backend):
|
||||
df = to_frame([bar(market=Market.SZ)], backend=backend)
|
||||
value = cell(df, backend, "market")
|
||||
assert value == 0
|
||||
assert not isinstance(value, Market)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("backend", BACKENDS)
|
||||
def test_empty_list_falls_back_to_annotation_hint(backend):
|
||||
df = to_frame([], backend=backend, model="SecurityBar")
|
||||
assert rows(df, backend) == 0
|
||||
assert cols(df, backend) == [
|
||||
"market", "code", "open", "high", "low", "close", "vol", "amount",
|
||||
"datetime", "trading_date", "symbol", "settle",
|
||||
]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("backend", BACKENDS)
|
||||
def test_quote_expands_bid_ask_levels(backend):
|
||||
quote = SecurityQuote(
|
||||
market=1, code="600000", price=10.5, pre_close=10.0, open=10.1, high=10.6,
|
||||
low=9.9, vol=100.0, amount=1000.0, s_vol=40.0, b_vol=60.0,
|
||||
bid=[(10.49, 100.0), (10.48, 200.0)], ask=[(10.51, 300.0)],
|
||||
symbol="sh600000", fields={"0x7": 1.5},
|
||||
)
|
||||
df = to_frame([quote], backend=backend)
|
||||
out = cols(df, backend)
|
||||
assert "bid1_price" in out and "ask5_vol" in out
|
||||
assert "bid" not in out and "ask" not in out
|
||||
assert cell(df, backend, "bid2_vol") == 200.0
|
||||
assert missing(cell(df, backend, "bid3_price")) # 只有两档,缺档给 None
|
||||
assert cell(df, backend, "fields") == '{"0x7":1.5}'
|
||||
|
||||
|
||||
@pytest.mark.parametrize("backend", BACKENDS)
|
||||
def test_empty_dict_field_becomes_none(backend):
|
||||
df = to_frame([SecurityQuote(
|
||||
market=1, code="600000", price=1.0, pre_close=1.0, open=1.0, high=1.0,
|
||||
low=1.0, vol=1.0, amount=1.0, s_vol=0.0, b_vol=0.0,
|
||||
)], backend=backend)
|
||||
assert missing(cell(df, backend, "fields"))
|
||||
|
||||
|
||||
@pytest.mark.parametrize("backend", BACKENDS)
|
||||
def test_minute_uses_real_clock(backend):
|
||||
late = MinuteBar(price=1.0, vol=10.0, avg_price=1.0, hour=27, minute=59,
|
||||
symbol="usAAPL", market=74, code="AAPL", date=20261026)
|
||||
df = to_frame([late], backend=backend)
|
||||
assert cols(df, backend) == [
|
||||
"price", "vol", "avg_price", "datetime", "trading_date",
|
||||
"symbol", "market", "code", "fields",
|
||||
]
|
||||
stamp = cell(df, backend, "datetime")
|
||||
assert (stamp.year, stamp.month, stamp.day, stamp.hour, stamp.minute) == (
|
||||
2026, 10, 27, 3, 59,
|
||||
)
|
||||
assert cell(df, backend, "trading_date") == "2026-10-26"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("backend", BACKENDS)
|
||||
def test_xdxr_date_column(backend):
|
||||
row = XdxrRecord(market=1, code="600000", year=2026, month=6, day=12,
|
||||
category=1, name="分红", fenhong=0.5, symbol="sh600000")
|
||||
df = to_frame([row], backend=backend)
|
||||
assert cols(df, backend)[:3] == ["market", "code", "date"]
|
||||
stamp = cell(df, backend, "date")
|
||||
assert (stamp.year, stamp.month, stamp.day) == (2026, 6, 12)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("backend", BACKENDS)
|
||||
def test_tdx_block_keeps_codes_list(backend):
|
||||
df = to_frame([TdxBlock(name="银行", category=0, count=2, codes=["600000", "600036"])],
|
||||
backend=backend)
|
||||
assert cols(df, backend) == ["name", "category", "count", "codes"]
|
||||
assert cell(df, backend, "codes") == ["600000", "600036"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("backend", BACKENDS)
|
||||
def test_indicator_set_merges_bars_and_indicators(backend):
|
||||
item = IndicatorSet(
|
||||
symbol="sh600000",
|
||||
bars=[bar(), bar(minute=1), bar(minute=2)],
|
||||
indicators={"ma5": [1.0, 2.0, 3.0], "boll": [None, None, 9.0]},
|
||||
)
|
||||
df = to_frame([item], backend=backend)
|
||||
assert rows(df, backend) == 3
|
||||
assert cell(df, backend, "symbol") == "sh600000"
|
||||
assert cell(df, backend, "ma5", 2) == 3.0
|
||||
assert missing(cell(df, backend, "boll"))
|
||||
|
||||
|
||||
@pytest.mark.parametrize("backend", BACKENDS)
|
||||
def test_indicator_series_shorter_than_bars_is_padded(backend):
|
||||
item = IndicatorSet(symbol="sh600000", bars=[bar(), bar(minute=1)],
|
||||
indicators={"ma5": [7.0]})
|
||||
df = to_frame([item], backend=backend)
|
||||
assert rows(df, backend) == 2
|
||||
assert cell(df, backend, "ma5") == 7.0
|
||||
assert missing(cell(df, backend, "ma5", 1))
|
||||
|
||||
|
||||
@pytest.mark.parametrize("backend", BACKENDS)
|
||||
def test_index_trade_stats_single_and_multi(backend):
|
||||
stat = IndexTradeStats(
|
||||
index="sh000300", date=20261027, member_count=300, active_members=299,
|
||||
trade_count_members=299,
|
||||
minutes=[MinuteTradeStat(time="09:31", volume=10.0, trade_count=5, amount=1000.0)],
|
||||
)
|
||||
single = to_frame(stat, backend=backend)
|
||||
assert cols(single, backend) == ["index", "date", "time", "volume", "trade_count", "amount"]
|
||||
assert cell(single, backend, "index") == "sh000300"
|
||||
|
||||
multi = to_frame([stat, stat], backend=backend)
|
||||
assert rows(multi, backend) == 2
|
||||
|
||||
|
||||
@pytest.mark.parametrize("backend", BACKENDS)
|
||||
def test_batch_dict_is_concatenated(backend):
|
||||
data = {
|
||||
"600000": [bar(code="600000"), bar(code="600000", minute=1)],
|
||||
"600036": [bar(code="600036", symbol="sh600036")],
|
||||
}
|
||||
df = to_frame(data, backend=backend)
|
||||
assert rows(df, backend) == 3
|
||||
assert cols(df, backend)[:2] == ["market", "code"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("backend", BACKENDS)
|
||||
def test_batch_dict_all_empty_returns_typed_empty(backend):
|
||||
df = to_frame({"600000": [], "600036": []}, backend=backend, model="SecurityBar")
|
||||
assert rows(df, backend) == 0
|
||||
assert "datetime" in cols(df, backend)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("backend", BACKENDS)
|
||||
def test_list_of_dicts_uses_key_union(backend):
|
||||
df = to_frame([{"name": "银行", "rise": 1.2}, {"name": "券商", "amount": 5.0}],
|
||||
backend=backend)
|
||||
assert cols(df, backend) == ["name", "rise", "amount"]
|
||||
assert missing(cell(df, backend, "amount", 0))
|
||||
|
||||
|
||||
def test_unsupported_payload_raises_type_error():
|
||||
with pytest.raises(TypeError, match="无法转换为 DataFrame 的返回值类型"):
|
||||
to_frame(42, backend="polars")
|
||||
with pytest.raises(TypeError, match="无法转换为 DataFrame 的列表元素"):
|
||||
to_frame([1.0, 2.0], backend="polars")
|
||||
|
||||
|
||||
def test_subclass_reuses_parent_plan():
|
||||
df = to_frame([IndexBar(market=1, code="000001", open=1.0, high=1.0, low=1.0,
|
||||
close=1.0, vol=1.0, amount=1.0, year=2026, month=1, day=5)],
|
||||
backend="polars")
|
||||
assert "datetime" in df.columns
|
||||
assert "year" not in df.columns
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 接线
|
||||
|
||||
|
||||
def test_tdxdata_default_keeps_legacy_behaviour():
|
||||
td = TdxData()
|
||||
assert td._df_backend is None
|
||||
|
||||
|
||||
def test_tdxdata_rejects_unknown_backend():
|
||||
with pytest.raises(ValueError, match="未知的 DataFrame 后端"):
|
||||
TdxData(df="numpy")
|
||||
|
||||
|
||||
def test_tdxdata_accepts_supported_backend():
|
||||
td = TdxData(df="polars")
|
||||
assert td._df_backend == "polars"
|
||||
|
||||
|
||||
class _Source:
|
||||
"""最小替身:只验装饰器语义,不碰网络。"""
|
||||
|
||||
def __init__(self, backend: str | None) -> None:
|
||||
self._df_backend = backend
|
||||
|
||||
@framed
|
||||
async def single(self) -> list[SecurityBar]:
|
||||
return [bar()]
|
||||
|
||||
@framed
|
||||
async def nested(self) -> dict[str, list[SecurityBar]]:
|
||||
got = await self.single() # 内层必须是原始 list
|
||||
assert isinstance(got, list), f"内层被提前转换成了 {type(got)!r}"
|
||||
return {"600000": got}
|
||||
|
||||
|
||||
async def test_decorator_passthrough_when_no_backend():
|
||||
src = _Source(None)
|
||||
assert isinstance(await src.single(), list)
|
||||
|
||||
|
||||
async def test_decorator_converts_outermost_only():
|
||||
src = _Source("polars")
|
||||
out = await src.nested()
|
||||
assert cols(out, "polars")[0] == "market"
|
||||
assert rows(out, "polars") == 1
|
||||
|
||||
|
||||
def test_model_name_resolution():
|
||||
# 注解是 dict[str, list[SecurityBar]] 时正则也能取到 SecurityBar;
|
||||
# 但 "dict" 这类内建名在 models 里找不到,必须返回 None 而不是内建类型
|
||||
assert _frames._model_by_name("dict") is None
|
||||
assert _frames._model_by_name("SecurityBar") is SecurityBar
|
||||
|
||||
|
||||
def test_datetime_dtype_is_datetime_like():
|
||||
pl = pytest.importorskip("polars")
|
||||
df = to_frame([bar()], backend="polars")
|
||||
assert isinstance(df.schema["datetime"], pl.Datetime)
|
||||
@@ -0,0 +1,303 @@
|
||||
"""指数成分成交汇总(get_index_members / get_index_trade_stats)单元测试。
|
||||
|
||||
全部离线:成分与逐笔都用实例属性打补丁,不建立任何真实连接。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from types import SimpleNamespace
|
||||
|
||||
import pytest
|
||||
|
||||
from pytdxdata import TdxData
|
||||
from pytdxdata.models import (
|
||||
IndexTradeStats,
|
||||
MemberQuote,
|
||||
MinuteTradeStat,
|
||||
TransactionRecord,
|
||||
)
|
||||
|
||||
# ---------- _index_pair:裸代码按指数语义解析 ----------
|
||||
|
||||
@pytest.mark.parametrize("text,expect", [
|
||||
("000001", (1, "000001")), # 上证指数(不是平安银行!)
|
||||
("sh000001", (1, "000001")),
|
||||
("000300", (1, "000300")),
|
||||
("sh000300", (1, "000300")),
|
||||
("000905", (1, "000905")),
|
||||
("999999", (1, "999999")),
|
||||
("880305", (1, "880305")),
|
||||
("399001", (0, "399001")), # 深证成指
|
||||
("sz399006", (0, "399006")),
|
||||
("899050", (2, "899050")), # 北证50
|
||||
("bj899050", (2, "899050")),
|
||||
("sh.000300", (1, "000300")), # 点号/冒号都容忍
|
||||
("sz:399001", (0, "399001")),
|
||||
])
|
||||
def test_index_pair_parses(text, expect):
|
||||
assert TdxData._index_pair(text) == expect
|
||||
|
||||
|
||||
@pytest.mark.parametrize("text", [
|
||||
"", "600000", "sz000001x", "sh12345", "12345", "abc", "zzz000001",
|
||||
])
|
||||
def test_index_pair_rejects(text):
|
||||
with pytest.raises(ValueError):
|
||||
TdxData._index_pair(text)
|
||||
|
||||
|
||||
def test_index_pair_rejects_non_index_cn_code():
|
||||
"""000001 之外的普通股票段(如 600xxx)不是指数,裸写必须报错。"""
|
||||
with pytest.raises(ValueError):
|
||||
TdxData._index_pair("600000")
|
||||
|
||||
|
||||
# ---------- 模型:累计曲线 ----------
|
||||
|
||||
def test_index_trade_stats_cumulative():
|
||||
st = IndexTradeStats(
|
||||
index="sh000300", date=None, member_count=2, active_members=2,
|
||||
trade_count_members=2,
|
||||
minutes=[
|
||||
MinuteTradeStat("09:30", 100.0, 10, 1000.0),
|
||||
MinuteTradeStat("09:31", 200.0, 20, 2000.0),
|
||||
MinuteTradeStat("09:32", 50.0, 5, 500.0),
|
||||
],
|
||||
total_volume=350.0, total_trade_count=35, total_amount=3500.0,
|
||||
)
|
||||
cum = st.cumulative()
|
||||
assert [c.time for c in cum] == ["09:30", "09:31", "09:32"]
|
||||
assert [c.volume for c in cum] == [100.0, 300.0, 350.0]
|
||||
assert [c.trade_count for c in cum] == [10, 30, 35]
|
||||
assert [c.amount for c in cum] == [1000.0, 3000.0, 3500.0]
|
||||
# 累计必须是新对象,不污染原始增量
|
||||
assert st.minutes[1].volume == 200.0
|
||||
assert st.cum_trade_count() == [("09:30", 10), ("09:31", 30), ("09:32", 35)]
|
||||
|
||||
|
||||
def test_index_trade_stats_trade_count_available():
|
||||
ok = IndexTradeStats(index="sh000300", date=None, member_count=1,
|
||||
active_members=1, trade_count_members=1)
|
||||
assert ok.trade_count_available is True
|
||||
bj = IndexTradeStats(index="bj899050", date=None, member_count=1,
|
||||
active_members=1, trade_count_members=0)
|
||||
assert bj.trade_count_available is False
|
||||
|
||||
|
||||
# ---------- get_index_members:分页终止 ----------
|
||||
|
||||
def _member(market: int, code: str, name: str = "") -> MemberQuote:
|
||||
return MemberQuote(market=market, code=code, name=name,
|
||||
fields={"0x0": 9.9, "0x5": 1000.0, "0x7": 99000.0})
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_index_members_pages_until_short_batch():
|
||||
"""短页(不足 80 条)即视为最后一页,不再多发一次空请求。"""
|
||||
calls: list[int] = []
|
||||
|
||||
async def fake_exec(cmd):
|
||||
calls.append(cmd.start)
|
||||
if cmd.start == 0:
|
||||
return [_member(1, "600000"), _member(1, "600004")]
|
||||
return []
|
||||
|
||||
td = TdxData(cache_dir=None)
|
||||
td._execute_mac = fake_exec
|
||||
members = await td.get_index_members("000016")
|
||||
assert [m.symbol for m in members] == ["sh600000", "sh600004"]
|
||||
assert calls == [0]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_index_members_continues_on_full_page():
|
||||
"""满页(80 条)后继续翻页,直到空页。"""
|
||||
calls: list[int] = []
|
||||
|
||||
async def fake_exec(cmd):
|
||||
calls.append(cmd.start)
|
||||
if cmd.start == 0:
|
||||
return [_member(1, "600000")] * cmd.page_size
|
||||
return []
|
||||
|
||||
td = TdxData(cache_dir=None)
|
||||
td._execute_mac = fake_exec
|
||||
members = await td.get_index_members("000016")
|
||||
assert len(members) == 80
|
||||
assert calls == [0, 80]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_index_members_respects_count_cap():
|
||||
async def fake_exec(cmd):
|
||||
# 永远返回满页,验证 count 能截断结果
|
||||
return [_member(1, "600000")] * cmd.page_size
|
||||
|
||||
td = TdxData(cache_dir=None)
|
||||
td._execute_mac = fake_exec
|
||||
members = await td.get_index_members("000016", count=5)
|
||||
assert len(members) == 5
|
||||
|
||||
|
||||
# ---------- get_index_trade_stats:按分钟聚合 ----------
|
||||
|
||||
def _tick(sym: str, hhmm: str, price: float, vol: float, tc: int) -> TransactionRecord:
|
||||
market, code = (1, sym[2:]) if sym.startswith("sh") else (0, sym[2:])
|
||||
return TransactionRecord(market=market, code=code, time=f"{hhmm}:00",
|
||||
price=price, vol=vol, trade_count=tc, bs_flag=1,
|
||||
symbol=sym)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_index_trade_stats_aggregates_minutes():
|
||||
async def fake_ticks(symbols, date=None, concurrency=6, **kw):
|
||||
out = []
|
||||
for s in symbols:
|
||||
if s == "sh600000":
|
||||
out += [_tick(s, "09:30", 10.0, 100.0, 10),
|
||||
_tick(s, "09:30", 10.0, 50.0, 5),
|
||||
_tick(s, "09:31", 11.0, 200.0, 20)]
|
||||
else:
|
||||
out += [_tick(s, "09:30", 5.0, 1000.0, 7)]
|
||||
return out
|
||||
|
||||
td = TdxData(cache_dir=None)
|
||||
td.get_ticks = fake_ticks
|
||||
st = await td.get_index_trade_stats(
|
||||
"000016", members=["sh600000", "sz000001"])
|
||||
|
||||
assert st.index == "sh000016"
|
||||
assert st.member_count == 2
|
||||
assert st.active_members == 2
|
||||
assert st.trade_count_members == 2
|
||||
assert st.trade_count_available is True
|
||||
assert st.failed_members == 0
|
||||
|
||||
m = {x.time: x for x in st.minutes}
|
||||
assert sorted(m) == ["09:30", "09:31"]
|
||||
assert m["09:30"].volume == 1150.0 # 100 + 50 + 1000
|
||||
assert m["09:30"].trade_count == 22 # 10 + 5 + 7
|
||||
assert m["09:31"].volume == 200.0
|
||||
# amount = Σ(price × 手 × 100)
|
||||
assert m["09:30"].amount == pytest.approx(
|
||||
10.0 * 100 * 100 + 10.0 * 50 * 100 + 5.0 * 1000 * 100)
|
||||
assert st.total_volume == 1350.0
|
||||
assert st.total_trade_count == 42
|
||||
assert st.total_amount == pytest.approx(
|
||||
10.0 * 100 * 100 + 10.0 * 50 * 100 + 5.0 * 1000 * 100 + 11.0 * 200 * 100)
|
||||
assert st.cum_trade_count() == [("09:30", 22), ("09:31", 42)]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_index_trade_stats_bj_has_no_trade_count():
|
||||
"""北交所成分返回 trade_count=0 → trade_count_available False。"""
|
||||
async def fake_ticks(symbols, date=None, concurrency=6, **kw):
|
||||
return [_tick("sh600000", "09:30", 10.0, 100.0, 0)]
|
||||
|
||||
td = TdxData(cache_dir=None)
|
||||
td.get_ticks = fake_ticks
|
||||
st = await td.get_index_trade_stats("899050", members=["sh600000"])
|
||||
assert st.trade_count_members == 0
|
||||
assert st.trade_count_available is False
|
||||
assert st.total_volume == 100.0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_index_trade_stats_retries_per_symbol_on_batch_failure():
|
||||
"""整批失败 → 退化为逐只重试,只有真正失败的计入 failed_members。"""
|
||||
async def fake_ticks(symbols, date=None, concurrency=6, **kw):
|
||||
single = isinstance(symbols, str)
|
||||
if not single:
|
||||
raise ConnectionError("批量炸了")
|
||||
if symbols == "sh600004":
|
||||
raise ConnectionError("这只也炸了")
|
||||
return [_tick(symbols, "09:30", 10.0, 100.0, 3)]
|
||||
|
||||
td = TdxData(cache_dir=None)
|
||||
td.get_ticks = fake_ticks
|
||||
st = await td.get_index_trade_stats(
|
||||
"000016", members=["sh600000", "sh600004", "sh600006"])
|
||||
assert st.failed_members == 1
|
||||
assert st.active_members == 2
|
||||
assert st.total_trade_count == 6
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_index_trade_stats_raises_when_not_skip_errors():
|
||||
async def fake_ticks(symbols, date=None, concurrency=6, **kw):
|
||||
raise ConnectionError("boom")
|
||||
|
||||
td = TdxData(cache_dir=None)
|
||||
td.get_ticks = fake_ticks
|
||||
with pytest.raises(ConnectionError):
|
||||
await td.get_index_trade_stats(
|
||||
"000016", members=["sh600000"], skip_errors=False)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_index_trade_stats_progress_callback():
|
||||
seen: list[tuple[int, int]] = []
|
||||
|
||||
async def fake_ticks(symbols, date=None, concurrency=6, **kw):
|
||||
return [_tick(s, "09:30", 10.0, 1.0, 1) for s in symbols]
|
||||
|
||||
td = TdxData(cache_dir=None)
|
||||
td.get_ticks = fake_ticks
|
||||
st = await td.get_index_trade_stats(
|
||||
"000016", members=[f"sh60000{i}" for i in range(5)],
|
||||
batch_size=2, on_progress=lambda d, t: seen.append((d, t)))
|
||||
assert seen == [(2, 5), (4, 5), (5, 5)]
|
||||
assert st.member_count == 5
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_index_trade_stats_fetches_members_when_absent():
|
||||
asked: list[str] = []
|
||||
|
||||
async def fake_members(code):
|
||||
asked.append(code)
|
||||
return [SimpleNamespace(symbol="sh600000")]
|
||||
|
||||
async def fake_ticks(symbols, date=None, concurrency=6, **kw):
|
||||
return [_tick(s, "09:30", 10.0, 1.0, 1) for s in symbols]
|
||||
|
||||
td = TdxData(cache_dir=None)
|
||||
td.get_index_members = fake_members
|
||||
td.get_ticks = fake_ticks
|
||||
await td.get_index_trade_stats("sh000300")
|
||||
assert asked == ["000300"]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_index_trade_stats_rejects_empty_members():
|
||||
td = TdxData(cache_dir=None)
|
||||
|
||||
async def fake_members(code):
|
||||
return []
|
||||
|
||||
td.get_index_members = fake_members
|
||||
with pytest.raises(ValueError, match="没有取到成分股"):
|
||||
await td.get_index_trade_stats("000300")
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_get_index_trade_stats_multi_reuses_members():
|
||||
member_calls: list[str] = []
|
||||
tick_dates: list[int | None] = []
|
||||
|
||||
async def fake_members(code):
|
||||
member_calls.append(code)
|
||||
return [SimpleNamespace(symbol="sh600000")]
|
||||
|
||||
async def fake_ticks(symbols, date=None, concurrency=6, **kw):
|
||||
tick_dates.append(date)
|
||||
return [_tick(s, "09:30", 10.0, 1.0, 2) for s in symbols]
|
||||
|
||||
td = TdxData(cache_dir=None)
|
||||
td.get_index_members = fake_members
|
||||
td.get_ticks = fake_ticks
|
||||
out = await td.get_index_trade_stats_multi(
|
||||
"000300", [20261020, 20261021, None])
|
||||
assert len(out) == 3
|
||||
assert member_calls == ["000300"] # 成分只取一次
|
||||
assert tick_dates == [20261020, 20261021, None]
|
||||
assert all(s.total_trade_count == 2 for s in out)
|
||||
@@ -0,0 +1,166 @@
|
||||
"""K线「按日期区间」取数的路径选择:倒推快路径 vs 二分慢路径。
|
||||
|
||||
背景:二分要先探总根数(翻倍 6 次)再跳 log2(N) 次,1 分钟 K 线约 36 次串行
|
||||
往返 = 4 秒级;而「取今天」「增量到最新」这类贴着右端的请求,倒推只要 1~3 次。
|
||||
"""
|
||||
|
||||
import datetime
|
||||
from unittest import mock
|
||||
|
||||
import pytest
|
||||
|
||||
from pytdxdata import TdxData
|
||||
from pytdxdata.models import KlinePeriod, SecurityBar
|
||||
|
||||
NEWEST = datetime.datetime(2026, 10, 28, 15, 0) # noqa: DTZ001 — 全项目用裸北京时间
|
||||
TOTAL_BARS = 5_000
|
||||
|
||||
|
||||
def _bar(when: datetime.datetime, close: float) -> SecurityBar:
|
||||
return SecurityBar(
|
||||
market=1, code="600000", open=10.0, high=10.1, low=9.9, close=close,
|
||||
vol=1000, amount=10000, year=when.year, month=when.month, day=when.day,
|
||||
hour=when.hour, minute=when.minute)
|
||||
|
||||
|
||||
class _FakeConn:
|
||||
"""标准通道语义替身:**offset 0 = 最新一根**,偏移越大越老。"""
|
||||
|
||||
calls = 0
|
||||
|
||||
def __init__(self, host="s1", port=7709, timeout=10.0):
|
||||
self.host = host
|
||||
self.alive_flag = True
|
||||
|
||||
@property
|
||||
def alive(self):
|
||||
return self.alive_flag
|
||||
|
||||
async def connect(self):
|
||||
pass
|
||||
|
||||
async def close(self):
|
||||
self.alive_flag = False
|
||||
|
||||
async def execute_pipeline(self, cmds, **kwargs):
|
||||
return [await self.execute(c) for c in cmds]
|
||||
|
||||
async def execute(self, cmd):
|
||||
type(self).calls += 1
|
||||
if "GetSecurityBarsCmd" not in type(cmd).__name__:
|
||||
return []
|
||||
rows = []
|
||||
for off in range(cmd.start, min(cmd.start + cmd.count, TOTAL_BARS)):
|
||||
rows.append(_bar(NEWEST - datetime.timedelta(days=off),
|
||||
10.0 + (TOTAL_BARS - off) * 0.01))
|
||||
return rows
|
||||
|
||||
|
||||
class _AscendingConn(_FakeConn):
|
||||
"""方向相反的替身(MAC/EX 语义):offset 0 = **最老**一根。"""
|
||||
|
||||
async def execute(self, cmd):
|
||||
type(self).calls += 1
|
||||
if "GetSecurityBarsCmd" not in type(cmd).__name__:
|
||||
return []
|
||||
rows = []
|
||||
base = datetime.datetime(2020, 1, 1) # noqa: DTZ001
|
||||
for off in range(cmd.start, min(cmd.start + cmd.count, TOTAL_BARS)):
|
||||
rows.append(_bar(base + datetime.timedelta(days=off), 10.0 + off * 0.01))
|
||||
return rows
|
||||
|
||||
|
||||
def _patched(conn_cls):
|
||||
return (
|
||||
mock.patch("pytdxdata.api.get_standard_hosts", return_value=["s1"]),
|
||||
mock.patch("pytdxdata.api.get_mac_hosts", return_value=["m1"]),
|
||||
mock.patch("pytdxdata.pool.pool.TdxConnection",
|
||||
side_effect=lambda h, p=7709, t=10.0: conn_cls(h)),
|
||||
)
|
||||
|
||||
|
||||
async def _fetch(conn_cls, **kwargs):
|
||||
conn_cls.calls = 0
|
||||
p1, p2, p3 = _patched(conn_cls)
|
||||
with p1, p2, p3:
|
||||
td = TdxData(cache_dir=None, pool_min=1, pool_max=2)
|
||||
await td.start()
|
||||
try:
|
||||
bars = await td.get_bars("sh600000", KlinePeriod.DAY, **kwargs)
|
||||
finally:
|
||||
await td.close()
|
||||
return bars, conn_cls.calls
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_recent_range_uses_backtrack():
|
||||
"""右端贴最新 → 倒推,往返次数是个位数(不是二分那几十次)。"""
|
||||
bars, calls = await _fetch(
|
||||
_FakeConn, start_date=NEWEST - datetime.timedelta(days=5), end_date=NEWEST)
|
||||
|
||||
# 返回按时间升序 → 偏移由老到新(5,4,...,0)
|
||||
assert [b.datetime.date() for b in bars] == [
|
||||
(NEWEST - datetime.timedelta(days=i)).date() for i in range(5, -1, -1)]
|
||||
assert calls <= 4, f"倒推路径应 1~3 次往返,实际 {calls}"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_end_before_newest_within_one_bar_still_backtracks():
|
||||
"""end 比最新一根略早(K线时间戳是周期结束时刻)仍算贴最新。"""
|
||||
end = NEWEST - datetime.timedelta(minutes=30)
|
||||
bars, calls = await _fetch(
|
||||
_FakeConn, start_date=NEWEST - datetime.timedelta(days=3), end_date=end)
|
||||
|
||||
# offset 0 那根(15:00)晚于 end,被裁掉
|
||||
assert bars[-1].datetime.date() == (NEWEST - datetime.timedelta(days=1)).date()
|
||||
assert calls <= 4
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_middle_range_falls_back_to_bisect():
|
||||
"""历史中段倒推够不着左边 → 回落二分,结果仍要正确。"""
|
||||
bars, calls = await _fetch(
|
||||
_FakeConn,
|
||||
start_date=NEWEST - datetime.timedelta(days=100),
|
||||
end_date=NEWEST - datetime.timedelta(days=90))
|
||||
|
||||
assert [b.datetime.date() for b in bars] == [
|
||||
(NEWEST - datetime.timedelta(days=i)).date() for i in range(100, 89, -1)]
|
||||
assert calls > 10, f"中段应走二分(几十次往返),实际只有 {calls}"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_future_range_returns_empty_quickly():
|
||||
# 用相对 now 的未来,别用固定日期:固定日期会随运行日漂进 _BACKTRACK_GRACE
|
||||
# (默认 1 天容差覆盖长假),倒推就成立、会多取一轮,断言随日期时绿时红。
|
||||
future = datetime.datetime.now() + datetime.timedelta(days=5) # noqa: DTZ001
|
||||
bars, calls = await _fetch(_FakeConn, start_date=future)
|
||||
|
||||
assert bars == []
|
||||
assert calls <= 3
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_ascending_channel_skips_backtrack():
|
||||
"""offset 0 是最老(MAC/EX 语义)时倒推不成立,必须走二分且结果正确。"""
|
||||
base = datetime.datetime(2020, 1, 1) # noqa: DTZ001
|
||||
bars, calls = await _fetch(
|
||||
_AscendingConn,
|
||||
start_date=base + datetime.timedelta(days=99),
|
||||
end_date=base + datetime.timedelta(days=109))
|
||||
|
||||
assert [b.datetime.date() for b in bars] == [
|
||||
(base + datetime.timedelta(days=i)).date() for i in range(99, 110)]
|
||||
assert calls > 10
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_backtrack_matches_plain_count_result():
|
||||
"""倒推裁剪结果 == 直接取一大段再裁剪(两条路径口径必须一致)。"""
|
||||
start = NEWEST - datetime.timedelta(days=7)
|
||||
by_date, _ = await _fetch(_FakeConn, start_date=start, end_date=NEWEST)
|
||||
whole, _ = await _fetch(_FakeConn, count=60)
|
||||
|
||||
expect = [b for b in whole if b.datetime >= start]
|
||||
assert [(b.datetime, b.close) for b in by_date] == \
|
||||
[(b.datetime, b.close) for b in expect]
|
||||
@@ -0,0 +1,236 @@
|
||||
"""单连接流水线(execute_pipeline)单元测试:不连真实网络。
|
||||
|
||||
关键防线:流水线把多条请求塞进同一条 TCP 流,一旦响应与请求错配就是
|
||||
「价格张冠李戴」级别的灾难,所以这里逐项锁死顺序、长度与失败语义。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import struct
|
||||
import unittest.mock as mock
|
||||
|
||||
import pytest
|
||||
|
||||
from pytdxdata.pool.connection import TdxConnection
|
||||
from pytdxdata.pool.pool import PooledConnection
|
||||
from pytdxdata.protocol.commands.base import BaseCommand
|
||||
|
||||
|
||||
class EchoCmd(BaseCommand[bytes]):
|
||||
"""请求体即 tag,响应体原样带回,便于断言顺序。"""
|
||||
|
||||
def __init__(self, tag: bytes) -> None:
|
||||
self.tag = tag
|
||||
|
||||
def build_request(self) -> bytes:
|
||||
return b"REQ:" + self.tag
|
||||
|
||||
def parse_response(self, body: bytes) -> bytes:
|
||||
return body
|
||||
|
||||
|
||||
class FakeQuote:
|
||||
"""最小报价替身:stamp() 会写 symbol,需要 market/code 是可 int/str 的真值。"""
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.market = 0
|
||||
self.code = "000001"
|
||||
self.symbol = ""
|
||||
|
||||
|
||||
def frame(body: bytes) -> bytes:
|
||||
"""构造一帧未压缩响应(zipsize == unzipsize)。"""
|
||||
return struct.pack("<IIIHH", 7654321, 0, 0x053E, len(body), len(body)) + body
|
||||
|
||||
|
||||
class FakeReader:
|
||||
def __init__(self, data: bytes) -> None:
|
||||
self.data = data
|
||||
self.pos = 0
|
||||
|
||||
async def read(self, n: int) -> bytes:
|
||||
chunk = self.data[self.pos:self.pos + n]
|
||||
self.pos += len(chunk)
|
||||
return chunk
|
||||
|
||||
|
||||
class FakeWriter:
|
||||
def __init__(self) -> None:
|
||||
self.written = b""
|
||||
self.closed = False
|
||||
self.transport = FakeQuote()
|
||||
|
||||
def write(self, data: bytes) -> None:
|
||||
self.written += data
|
||||
|
||||
async def drain(self) -> None:
|
||||
pass
|
||||
|
||||
def is_closing(self) -> bool:
|
||||
return self.closed
|
||||
|
||||
def close(self) -> None:
|
||||
self.closed = True
|
||||
|
||||
async def wait_closed(self) -> None:
|
||||
pass
|
||||
|
||||
|
||||
def make_conn(payload: bytes, host: str = "1.2.3.4") -> TdxConnection:
|
||||
conn = TdxConnection(host, 7709, 5.0)
|
||||
conn._reader = FakeReader(payload) # type: ignore[assignment]
|
||||
conn._writer = FakeWriter() # type: ignore[assignment]
|
||||
return conn
|
||||
|
||||
|
||||
# ---------- 基本语义 ----------
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_empty_cmds_returns_empty():
|
||||
conn = make_conn(b"")
|
||||
assert await conn.execute_pipeline([]) == []
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_responses_paired_in_order():
|
||||
bodies = [b"AAA", b"BBB", b"CCC"]
|
||||
conn = make_conn(b"".join(frame(b) for b in bodies))
|
||||
cmds = [EchoCmd(t) for t in (b"1", b"2", b"3")]
|
||||
got = await conn.execute_pipeline(cmds)
|
||||
assert got == bodies # 顺序必须与 cmds 严格一致
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_all_requests_sent_at_once():
|
||||
conn = make_conn(b"".join(frame(b"X") for _ in range(3)))
|
||||
cmds = [EchoCmd(t) for t in (b"a", b"b", b"c")]
|
||||
await conn.execute_pipeline(cmds)
|
||||
assert conn._writer.written == b"REQ:aREQ:bREQ:c"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_closed_connection_raises():
|
||||
conn = make_conn(b"")
|
||||
conn._writer = None
|
||||
with pytest.raises(ConnectionError):
|
||||
await conn.execute_pipeline([EchoCmd(b"1")])
|
||||
|
||||
|
||||
# ---------- 失败语义:响应流错配后必须废连接 ----------
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_truncated_stream_force_closes():
|
||||
# 只给第一帧完整的、第二帧只有半个头 → 读不满,抛错且连接作废
|
||||
conn = make_conn(frame(b"AAA") + b"\x00" * 5)
|
||||
with pytest.raises(ConnectionError):
|
||||
await conn.execute_pipeline([EchoCmd(b"1"), EchoCmd(b"2")])
|
||||
assert conn._writer is None # _force_close 已清空 writer
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_body_length_mismatch_propagates():
|
||||
bad = struct.pack("<IIIHH", 7654321, 0, 0x053E, 99, 99) + b"short"
|
||||
conn = make_conn(bad)
|
||||
# 帧头自称 99 字节但流里没有 → _recv_exact 读空,连接作废
|
||||
with pytest.raises(ConnectionError):
|
||||
await conn.execute_pipeline([EchoCmd(b"1")])
|
||||
assert conn._writer is None
|
||||
|
||||
|
||||
# ---------- 池包装转发 ----------
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_pooled_connection_forwards_pipeline():
|
||||
inner = mock.AsyncMock()
|
||||
inner.execute_pipeline = mock.AsyncMock(return_value=["r1", "r2"])
|
||||
pc = PooledConnection(inner, "host1")
|
||||
got = await pc.execute_pipeline([EchoCmd(b"1"), EchoCmd(b"2")])
|
||||
assert got == ["r1", "r2"]
|
||||
inner.execute_pipeline.assert_awaited_once()
|
||||
|
||||
|
||||
# ---------- api 层:切批是否走流水线 ----------
|
||||
|
||||
def _api_with_fake_pool():
|
||||
from pytdxdata.api import TdxData
|
||||
from pytdxdata.symbols import CN, Symbol
|
||||
|
||||
td = TdxData.__new__(TdxData) # 跳过 __init__(不建网络池)
|
||||
td._std_pool = FakeQuote()
|
||||
td._std_pool.max_size = 24
|
||||
td._std_pool.servers = [f"s{i}" for i in range(11)]
|
||||
td._std_pool.acquire = mock.AsyncMock()
|
||||
td._std_pool.release = mock.AsyncMock()
|
||||
td._std_pool.warm = mock.AsyncMock() # 预建连接是网络动作,测试里空转
|
||||
return td, Symbol, CN
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_quotes_std_uses_pipeline_for_multi_page():
|
||||
td, Symbol, CN = _api_with_fake_pool()
|
||||
syms = [Symbol(0, f"{i:06d}", CN) for i in range(1, 401)] # 5 片
|
||||
chunks = []
|
||||
|
||||
async def fake_pipeline(batch_group):
|
||||
chunks.append(len(batch_group))
|
||||
return [[FakeQuote()] * 80 for _ in batch_group]
|
||||
|
||||
td._execute_std_pipeline = fake_pipeline
|
||||
out = await td._quotes_std(syms, pipeline=True)
|
||||
assert len(out) == 5 * 80
|
||||
assert sum(chunks) == 5 # 每片恰好被处理一次,不多不少
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_quotes_std_single_page_skips_pipeline():
|
||||
td, Symbol, CN = _api_with_fake_pool()
|
||||
called = []
|
||||
td._execute_std_pipeline = mock.AsyncMock(return_value=[])
|
||||
|
||||
async def fake_std(cmd):
|
||||
called.append(1)
|
||||
return []
|
||||
|
||||
td._execute_standard = fake_std
|
||||
syms = [Symbol(0, f"{i:06d}", CN) for i in range(1, 51)] # 1 片
|
||||
await td._quotes_std(syms, pipeline=True)
|
||||
assert called == [1]
|
||||
td._execute_std_pipeline.assert_not_awaited()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_quotes_std_no_pipeline_flag_uses_plain_path():
|
||||
td, Symbol, CN = _api_with_fake_pool()
|
||||
|
||||
async def fake_std(cmd):
|
||||
return [FakeQuote()] * 80
|
||||
|
||||
td._execute_standard = fake_std
|
||||
td._execute_std_pipeline = mock.AsyncMock(return_value=[])
|
||||
syms = [Symbol(0, f"{i:06d}", CN) for i in range(1, 401)]
|
||||
out = await td._quotes_std(syms, pipeline=False)
|
||||
assert len(out) == 5 * 80
|
||||
td._execute_std_pipeline.assert_not_awaited()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_pipeline_failure_falls_back_to_per_page():
|
||||
"""节点不支持流水线时,降级逐片,结果条数不变。"""
|
||||
td, Symbol, CN = _api_with_fake_pool()
|
||||
|
||||
pool = td._std_pool
|
||||
pool.acquire = mock.AsyncMock(return_value=mock.Mock(
|
||||
execute_pipeline=mock.AsyncMock(side_effect=ConnectionError("boom"))))
|
||||
pool.release = mock.AsyncMock()
|
||||
|
||||
calls = []
|
||||
|
||||
async def fake_std(cmd):
|
||||
calls.append(1)
|
||||
return [FakeQuote()] * 80
|
||||
|
||||
td._execute_standard = fake_std
|
||||
syms = [Symbol(0, f"{i:06d}", CN) for i in range(1, 241)] # 3 片
|
||||
out = await td._execute_std_pipeline(
|
||||
[syms[i:i + 80] for i in range(0, len(syms), 80)])
|
||||
assert len(out) == 3 and len(calls) == 3
|
||||
assert pool.release.await_args.kwargs.get("broken") is True
|
||||
@@ -0,0 +1,365 @@
|
||||
"""当日测速缓存单元测试(注入假探针与假日期,不连真实网络)
|
||||
|
||||
覆盖点:
|
||||
- 当日首次启动测一次;当天已有缓存则**不刷新**(探针一次都不发起)
|
||||
- 跨日 / 缓存损坏 / 版本不符 → 重新测速
|
||||
- 结果按实测延迟升序被 config 采用
|
||||
- **绝不改写包内 servers.json**,缓存落在用户级缓存目录
|
||||
- 全部不可达 → 不写缓存(下次启动重试)
|
||||
- 并发调用只真正测一次(单飞)
|
||||
- 非法主机名(测试 mock 混入的 "s1" 等)被挡在测速之外
|
||||
- 连接池 start() 不阻塞、close() 会取消后台测速
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import time
|
||||
from datetime import date, timedelta
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from pytdxdata import config
|
||||
from pytdxdata.pool.probe_cache import ProbeCache
|
||||
|
||||
TODAY = date.today().isoformat()
|
||||
YESTERDAY = (date.today() - timedelta(days=1)).isoformat()
|
||||
PROTOS = ("standard", "mac", "ex")
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def isolated_cache(tmp_path, monkeypatch):
|
||||
"""把用户级缓存目录指到临时目录,并复位所有相关环境变量。"""
|
||||
monkeypatch.setenv("PYPYTDX_DATA_CACHE_DIR", str(tmp_path))
|
||||
monkeypatch.delenv("PYPYTDX_DATA_DISABLE_PROBE", raising=False)
|
||||
for var in ("PYPYTDX_DATA_STANDARD_HOSTS", "PYPYTDX_DATA_MAC_HOSTS",
|
||||
"PYPYTDX_DATA_EX_HOSTS"):
|
||||
monkeypatch.delenv(var, raising=False)
|
||||
return tmp_path
|
||||
|
||||
|
||||
def fake_probers(latencies: dict[str, dict[str, float | None]], calls: list):
|
||||
"""构造假探针;latencies = {协议: {主机: 延迟或 None}}。"""
|
||||
def _make(protocol: str):
|
||||
async def _probe(host: str, timeout: float) -> float | None:
|
||||
calls.append((protocol, host))
|
||||
return latencies.get(protocol, {}).get(host)
|
||||
return _probe
|
||||
return {p: _make(p) for p in PROTOS}
|
||||
|
||||
|
||||
def patch_candidates(monkeypatch, mapping: dict[str, list[str]]) -> None:
|
||||
"""把"全量候选"换成受控清单,避免依赖真实服务器清单。"""
|
||||
monkeypatch.setattr("pytdxdata.pool.probe_cache.all_probe_candidates",
|
||||
lambda: mapping)
|
||||
|
||||
|
||||
def write_cache(path: Path, day: str, results: dict, version: int = 1) -> None:
|
||||
path.write_text(json.dumps({"version": version, "date": day, "results": results}),
|
||||
encoding="utf-8")
|
||||
|
||||
|
||||
# ---------- 是否该测速 ----------
|
||||
|
||||
def test_needs_probe_when_no_cache(tmp_path):
|
||||
pc = ProbeCache(tmp_path / "c.json", today_fn=lambda: TODAY)
|
||||
assert pc.needs_probe_today() is True
|
||||
assert pc.read_cache() is None
|
||||
|
||||
|
||||
def test_needs_probe_false_for_today_cache(tmp_path):
|
||||
cache = tmp_path / "c.json"
|
||||
write_cache(cache, TODAY, {"1.1.1.1": {"standard": 0.05}})
|
||||
pc = ProbeCache(cache, today_fn=lambda: TODAY)
|
||||
assert pc.needs_probe_today() is False
|
||||
|
||||
|
||||
def test_stale_cache_needs_probe(tmp_path):
|
||||
cache = tmp_path / "c.json"
|
||||
write_cache(cache, YESTERDAY, {"1.1.1.1": {"standard": 0.05}})
|
||||
pc = ProbeCache(cache, today_fn=lambda: TODAY)
|
||||
assert pc.needs_probe_today() is True
|
||||
|
||||
|
||||
def test_version_mismatch_needs_probe(tmp_path):
|
||||
cache = tmp_path / "c.json"
|
||||
write_cache(cache, TODAY, {"1.1.1.1": {"standard": 0.05}}, version=999)
|
||||
pc = ProbeCache(cache, today_fn=lambda: TODAY)
|
||||
assert pc.needs_probe_today() is True
|
||||
|
||||
|
||||
@pytest.mark.parametrize("content", ["{not json", "[]", '{"version":1}', ""])
|
||||
def test_broken_cache_is_ignored_not_crash(tmp_path, content):
|
||||
cache = tmp_path / "c.json"
|
||||
cache.write_text(content, encoding="utf-8")
|
||||
pc = ProbeCache(cache, today_fn=lambda: TODAY)
|
||||
assert pc.read_cache() is None
|
||||
assert pc.needs_probe_today() is True
|
||||
|
||||
|
||||
# ---------- 核心行为:当天已有缓存就不刷新 ----------
|
||||
|
||||
async def test_today_cache_is_not_refreshed(tmp_path, monkeypatch):
|
||||
cache = tmp_path / "c.json"
|
||||
write_cache(cache, TODAY, {"1.1.1.1": {"standard": 0.05}})
|
||||
before = cache.read_text(encoding="utf-8")
|
||||
calls: list = []
|
||||
patch_candidates(monkeypatch, {"standard": ["1.1.1.1"], "mac": [], "ex": []})
|
||||
|
||||
pc = ProbeCache(cache, probers=fake_probers({}, calls), today_fn=lambda: TODAY)
|
||||
await pc.ensure_probe()
|
||||
|
||||
assert calls == [] # 未发起任何测速
|
||||
assert cache.read_text(encoding="utf-8") == before # 缓存内容原封不动
|
||||
|
||||
|
||||
async def test_stale_cache_is_refreshed(tmp_path, monkeypatch):
|
||||
cache = tmp_path / "c.json"
|
||||
write_cache(cache, YESTERDAY, {"9.9.9.9": {"standard": 9.0}})
|
||||
calls: list = []
|
||||
patch_candidates(monkeypatch, {"standard": ["1.1.1.1"], "mac": [], "ex": []})
|
||||
|
||||
pc = ProbeCache(cache,
|
||||
probers=fake_probers({"standard": {"1.1.1.1": 0.02}}, calls),
|
||||
today_fn=lambda: TODAY)
|
||||
await pc.ensure_probe()
|
||||
|
||||
assert calls == [("standard", "1.1.1.1")]
|
||||
data = json.loads(cache.read_text(encoding="utf-8"))
|
||||
assert data["date"] == TODAY and data["version"] == config.PROBE_CACHE_VERSION
|
||||
assert data["results"] == {"1.1.1.1": {"standard": 0.02}}
|
||||
assert data["hosts"]["standard"] == ["1.1.1.1"]
|
||||
assert "9.9.9.9" not in data["results"] # 旧结果被整体替换,不残留
|
||||
|
||||
|
||||
# ---------- 缓存写入 ----------
|
||||
|
||||
async def test_write_is_atomic_and_leaves_no_tmp(tmp_path, monkeypatch):
|
||||
cache = tmp_path / "sub" / "c.json" # 目录不存在也要能建
|
||||
patch_candidates(monkeypatch, {"standard": ["1.1.1.1"], "mac": [], "ex": []})
|
||||
pc = ProbeCache(cache,
|
||||
probers=fake_probers({"standard": {"1.1.1.1": 0.03}}, []),
|
||||
today_fn=lambda: TODAY)
|
||||
await pc.ensure_probe()
|
||||
|
||||
assert cache.exists()
|
||||
assert list(cache.parent.glob("*.tmp*")) == []
|
||||
assert json.loads(cache.read_text(encoding="utf-8"))["probed_at"]
|
||||
|
||||
|
||||
async def test_all_unreachable_writes_nothing(tmp_path, monkeypatch):
|
||||
cache = tmp_path / "c.json"
|
||||
patch_candidates(monkeypatch, {"standard": ["1.1.1.1"], "mac": [], "ex": []})
|
||||
pc = ProbeCache(cache, probers=fake_probers({}, []), today_fn=lambda: TODAY)
|
||||
await pc.ensure_probe()
|
||||
|
||||
assert not cache.exists() # 不把"全挂"缓存一整天
|
||||
assert pc.needs_probe_today() is True
|
||||
|
||||
|
||||
async def test_never_touches_package_servers_json(tmp_path, monkeypatch):
|
||||
"""最关键的一条:测速**绝不能**改写随包分发的 servers.json。"""
|
||||
servers_json = config._SERVERS_FILE
|
||||
before_bytes = servers_json.read_bytes()
|
||||
before_mtime = servers_json.stat().st_mtime_ns
|
||||
|
||||
patch_candidates(monkeypatch, {"standard": ["1.1.1.1"], "mac": [], "ex": []})
|
||||
pc = ProbeCache(probers=fake_probers({"standard": {"1.1.1.1": 0.04}}, []),
|
||||
today_fn=lambda: TODAY)
|
||||
await pc.ensure_probe()
|
||||
|
||||
assert servers_json.read_bytes() == before_bytes
|
||||
assert servers_json.stat().st_mtime_ns == before_mtime
|
||||
assert pc.cache_file() == tmp_path / "hosts-today.json" # 落在用户缓存目录
|
||||
assert Path(config.probe_cache_file()).parent == tmp_path
|
||||
|
||||
|
||||
async def test_results_are_recorded_per_protocol(tmp_path, monkeypatch):
|
||||
cache = tmp_path / "c.json"
|
||||
patch_candidates(monkeypatch, {
|
||||
"standard": ["1.1.1.1", "2.2.2.2"],
|
||||
"mac": ["1.1.1.1", "2.2.2.2"],
|
||||
"ex": ["3.3.3.3"],
|
||||
})
|
||||
lats = {"standard": {"1.1.1.1": 0.30, "2.2.2.2": 0.10},
|
||||
"mac": {"1.1.1.1": 0.20}, # 2.2.2.2 的 mac 故意不给 → 应记为 None
|
||||
"ex": {"3.3.3.3": 0.05}}
|
||||
pc = ProbeCache(cache, probers=fake_probers(lats, []), today_fn=lambda: TODAY)
|
||||
await pc.ensure_probe()
|
||||
|
||||
data = json.loads(cache.read_text(encoding="utf-8"))
|
||||
assert data["hosts"]["standard"] == ["2.2.2.2", "1.1.1.1"] # 按延迟升序
|
||||
assert data["hosts"]["mac"] == ["1.1.1.1"] # 不可达的 2.2.2.2 被排除
|
||||
assert data["hosts"]["ex"] == ["3.3.3.3"]
|
||||
# 2.2.2.2 的 mac 探针返回 None,也要如实记录(便于事后诊断)
|
||||
assert data["results"]["2.2.2.2"]["mac"] is None
|
||||
|
||||
|
||||
# ---------- 单飞 / 开关 / 脏数据 ----------
|
||||
|
||||
async def test_concurrent_ensure_probe_runs_once(tmp_path, monkeypatch):
|
||||
patch_candidates(monkeypatch, {"standard": ["1.1.1.1", "2.2.2.2"], "mac": [], "ex": []})
|
||||
calls: list = []
|
||||
pc = ProbeCache(tmp_path / "c.json",
|
||||
probers=fake_probers({"standard": {"1.1.1.1": 0.01,
|
||||
"2.2.2.2": 0.02}}, calls),
|
||||
today_fn=lambda: TODAY)
|
||||
await asyncio.gather(pc.ensure_probe(), pc.ensure_probe(), pc.ensure_probe())
|
||||
|
||||
assert sorted(calls) == [("standard", "1.1.1.1"), ("standard", "2.2.2.2")]
|
||||
|
||||
|
||||
async def test_disabled_env_skips_everything(tmp_path, monkeypatch):
|
||||
monkeypatch.setenv("PYPYTDX_DATA_DISABLE_PROBE", "1")
|
||||
calls: list = []
|
||||
patch_candidates(monkeypatch, {"standard": ["1.1.1.1"], "mac": [], "ex": []})
|
||||
pc = ProbeCache(tmp_path / "c.json",
|
||||
probers=fake_probers({"standard": {"1.1.1.1": 0.01}}, calls),
|
||||
today_fn=lambda: TODAY)
|
||||
|
||||
assert pc.needs_probe_today() is False
|
||||
await pc.ensure_probe()
|
||||
assert calls == []
|
||||
assert not (tmp_path / "c.json").exists()
|
||||
|
||||
|
||||
async def test_illegal_hosts_are_filtered_out(tmp_path, monkeypatch):
|
||||
"""测试 mock 常传入 "s1"/"m1" 这类伪主机名,必须挡在缓存之外。"""
|
||||
patch_candidates(monkeypatch, {"standard": [], "mac": [], "ex": []})
|
||||
calls: list = []
|
||||
pc = ProbeCache(tmp_path / "c.json",
|
||||
probers=fake_probers({"standard": {"1.1.1.1": 0.01}}, calls),
|
||||
today_fn=lambda: TODAY)
|
||||
await pc.ensure_probe(["s1", "m1", "1.1.1.1", ""])
|
||||
|
||||
assert calls == [("standard", "1.1.1.1")]
|
||||
data = json.loads((tmp_path / "c.json").read_text(encoding="utf-8"))
|
||||
assert "s1" not in data["results"] and "m1" not in data["results"]
|
||||
|
||||
|
||||
async def test_reset_allows_recheck(tmp_path, monkeypatch):
|
||||
patch_candidates(monkeypatch, {"standard": ["1.1.1.1"], "mac": [], "ex": []})
|
||||
calls: list = []
|
||||
pc = ProbeCache(tmp_path / "c.json",
|
||||
probers=fake_probers({"standard": {"1.1.1.1": 0.01}}, calls),
|
||||
today_fn=lambda: TODAY)
|
||||
await pc.ensure_probe()
|
||||
await pc.ensure_probe()
|
||||
assert len(calls) == 1 # 第二次被单飞挡掉
|
||||
|
||||
pc.reset()
|
||||
await pc.ensure_probe()
|
||||
assert len(calls) == 1 # 重置后当天已有缓存,仍不重测
|
||||
|
||||
|
||||
# ---------- config 侧:采用顺序与兜底 ----------
|
||||
|
||||
def test_config_getters_prefer_cache_and_order_by_latency(tmp_path, monkeypatch):
|
||||
monkeypatch.setenv("PYPYTDX_DATA_CACHE_DIR", str(tmp_path))
|
||||
(tmp_path / "hosts-today.json").write_text(json.dumps({
|
||||
"version": config.PROBE_CACHE_VERSION, "date": TODAY,
|
||||
"results": {
|
||||
"3.3.3.3": {"standard": 0.9},
|
||||
"1.1.1.1": {"standard": 0.05},
|
||||
"2.2.2.2": {"standard": 0.4, "mac": 0.1},
|
||||
"9.9.9.9": {"ex": 0.2},
|
||||
},
|
||||
}), encoding="utf-8")
|
||||
|
||||
assert config.get_standard_hosts() == ["1.1.1.1", "2.2.2.2", "3.3.3.3"]
|
||||
assert config.get_mac_hosts() == ["2.2.2.2"]
|
||||
assert config.get_ex_hosts() == ["9.9.9.9"]
|
||||
|
||||
|
||||
def test_config_getters_fall_back_without_cache(tmp_path, monkeypatch):
|
||||
monkeypatch.setenv("PYPYTDX_DATA_CACHE_DIR", str(tmp_path))
|
||||
assert config.get_standard_hosts() == config._load_ok_hosts("standard", fast_only=True)
|
||||
assert config.get_mac_hosts() == config._load_ok_hosts("mac", fast_only=True)
|
||||
assert config.get_ex_hosts() == config._load_ok_hosts("ex")
|
||||
|
||||
|
||||
def test_env_override_beats_cache(tmp_path, monkeypatch):
|
||||
monkeypatch.setenv("PYPYTDX_DATA_CACHE_DIR", str(tmp_path))
|
||||
(tmp_path / "hosts-today.json").write_text(json.dumps({
|
||||
"version": config.PROBE_CACHE_VERSION, "date": TODAY,
|
||||
"results": {"1.1.1.1": {"standard": 0.05}},
|
||||
}), encoding="utf-8")
|
||||
monkeypatch.setenv("PYPYTDX_DATA_STANDARD_HOSTS", "8.8.8.8")
|
||||
assert config.get_standard_hosts() == ["8.8.8.8"]
|
||||
|
||||
|
||||
def test_all_probe_candidates_covers_full_set_not_filtered():
|
||||
cand = config.all_probe_candidates()
|
||||
assert set(config.STANDARD_HOSTS) <= set(cand["standard"])
|
||||
assert set(config.MAC_HOSTS) <= set(cand["mac"])
|
||||
assert set(config.EX_HOSTS) <= set(cand["ex"])
|
||||
assert "117.34.114.17" in cand["standard"] # 未申报 MAC 的节点也要被测到
|
||||
assert "s1" not in cand["standard"] # 包内清单里的脏数据被挡
|
||||
|
||||
|
||||
@pytest.mark.parametrize(("host", "expected"), [
|
||||
("1.2.3.4", True),
|
||||
("117.34.114.17", True),
|
||||
("a.b", True),
|
||||
("good-host.com", True),
|
||||
("s1", False),
|
||||
("m1", False),
|
||||
("", False),
|
||||
(" ", False),
|
||||
("-bad.com", False),
|
||||
("bad-.com", False),
|
||||
("has space.com", False),
|
||||
(123, False),
|
||||
(None, False),
|
||||
])
|
||||
def test_is_valid_host(host, expected):
|
||||
assert config.is_valid_host(host) is expected
|
||||
|
||||
|
||||
# ---------- 连接池集成 ----------
|
||||
|
||||
async def test_pool_start_does_not_block_and_close_cancels_probe(monkeypatch):
|
||||
from pytdxdata.pool import pool as pool_mod
|
||||
|
||||
cancel_seen = asyncio.Event()
|
||||
created: list[asyncio.Task] = []
|
||||
|
||||
async def _hang(hosts=None):
|
||||
try:
|
||||
await asyncio.sleep(30) # 模拟耗时测速
|
||||
except asyncio.CancelledError:
|
||||
cancel_seen.set()
|
||||
raise
|
||||
|
||||
def _fake_schedule(hosts=None):
|
||||
task = asyncio.create_task(_hang())
|
||||
created.append(task)
|
||||
return task
|
||||
|
||||
monkeypatch.setattr(pool_mod, "schedule_probe", _fake_schedule)
|
||||
|
||||
p = pool_mod.DynamicPool([], min_size=1, max_size=1)
|
||||
t0 = time.monotonic()
|
||||
await p.start()
|
||||
assert time.monotonic() - t0 < 1.0 # start() 不等测速
|
||||
assert p._probe_task is created[0]
|
||||
|
||||
await asyncio.sleep(0) # 让测速任务真正跑起来再关闭
|
||||
await p.close()
|
||||
assert p._probe_task is None
|
||||
assert created[0].cancelled()
|
||||
assert cancel_seen.is_set()
|
||||
|
||||
|
||||
async def test_pool_auto_probe_false_schedules_nothing(monkeypatch):
|
||||
from pytdxdata.pool import pool as pool_mod
|
||||
|
||||
def _boom(hosts=None): # pragma: no cover - 不应被调用
|
||||
raise AssertionError("auto_probe=False 时不应调度测速")
|
||||
|
||||
monkeypatch.setattr(pool_mod, "schedule_probe", _boom)
|
||||
p = pool_mod.DynamicPool([], min_size=1, max_size=1, auto_probe=False)
|
||||
await p.start()
|
||||
assert p._probe_task is None
|
||||
await p.close()
|
||||
+326
-4
@@ -2,17 +2,21 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import struct
|
||||
from collections import Counter
|
||||
from datetime import date, datetime
|
||||
|
||||
import pytest
|
||||
|
||||
from pytdxdata.protocol.frame import FrameHeader, decompress_body, parse_header
|
||||
from pytdxdata.protocol.price import get_price, put_price, decode_ohlc_diffs
|
||||
from pytdxdata.protocol.volume import decode_volume
|
||||
from pytdxdata.models import Adjust, KlinePeriod, SecurityBar
|
||||
from pytdxdata.protocol.commands.ex_proto import GetExInstrumentBarsCmd
|
||||
from pytdxdata.protocol.commands.kline import GetSecurityBarsCmd
|
||||
from pytdxdata.protocol.commands.mac import MacKlineCmd
|
||||
from pytdxdata.protocol.commands.minute import GetHistoryMinuteTimeDataCmd
|
||||
from pytdxdata.protocol.commands.quotes import price_decimal_digits
|
||||
from pytdxdata.protocol.commands.transaction import MacTransactionCmd
|
||||
|
||||
from pytdxdata.protocol.frame import FrameHeader, decompress_body, parse_header
|
||||
from pytdxdata.protocol.price import decode_ohlc_diffs, get_price, put_price
|
||||
from pytdxdata.protocol.volume import decode_volume
|
||||
|
||||
# ---------- 变长有符号整数 ----------
|
||||
|
||||
@@ -176,3 +180,321 @@ def test_mac_transaction_parse():
|
||||
assert recs[0].trade_count == 3
|
||||
assert recs[0].bs_flag == 1
|
||||
assert recs[1].trade_count == 1
|
||||
|
||||
|
||||
def test_mac_transaction_parse_negative_trade_count_auction():
|
||||
"""集合竞价段 trade_count 是有符号 int32,负数不能被读成 42 亿(BUG 回归)。
|
||||
|
||||
实测(2026-10-21,sh600000 09:15:03):18 字节为
|
||||
17820000 0ad71b41 d9000000 96fcffff 0800
|
||||
最后四字节前的 96fcffff = int32 -874,旧解析按 uint32 得 4294966422。
|
||||
"""
|
||||
body = bytearray(39 + 2 * 18)
|
||||
struct.pack_into("<H", body, 29, 2)
|
||||
# 第1条:集合竞价,trade_count = -874
|
||||
struct.pack_into("<IfIiH", body, 39, 33303, 9.74, 217, -874, 8)
|
||||
# 第2条:正常交易,trade_count = 3
|
||||
struct.pack_into("<IfIiH", body, 39 + 18, 34200, 9.75, 100, 3, 1)
|
||||
|
||||
cmd = MacTransactionCmd(1, "600000", 20261021, 0, 100)
|
||||
recs = cmd.parse_response(bytes(body))
|
||||
assert len(recs) == 2
|
||||
assert recs[0].trade_count == -874, "负值必须保留符号,不能变成 4294966422"
|
||||
assert recs[0].bs_flag == 8
|
||||
assert recs[1].trade_count == 3
|
||||
# 正常交易段的量级不受影响
|
||||
assert max(r.trade_count for r in recs) == 3
|
||||
|
||||
|
||||
def test_mac_transaction_etf_trade_count_unavailable():
|
||||
"""ETF/LOF 的 trade_count 是「价格×1000」占位,必须标记为不可用。
|
||||
|
||||
实测(2026-10-22 sh510310 15:27:35):价 3.989、原字段 3994;
|
||||
旧行为把它当笔数打印成「3994笔」,是假的。
|
||||
"""
|
||||
body = bytearray(39 + 2 * 18)
|
||||
struct.pack_into("<H", body, 29, 2)
|
||||
struct.pack_into("<IfIiH", body, 39, 56000, 3.989, 15, 3994, 5)
|
||||
struct.pack_into("<IfIiH", body, 39 + 18, 56003, 3.989, 4, 3994, 5)
|
||||
|
||||
for code, market, expect in (("510310", 1, False), ("159915", 0, False),
|
||||
("600000", 1, True), ("501050", 1, True)):
|
||||
cmd = MacTransactionCmd(market, code, 20261022, 0, 100)
|
||||
recs = cmd.parse_response(bytes(body))
|
||||
assert recs[0].trade_count_available is expect, code
|
||||
if expect:
|
||||
assert recs[0].trade_count == 3994
|
||||
else:
|
||||
assert recs[0].trade_count == 0
|
||||
|
||||
|
||||
@pytest.mark.parametrize("market,code,expect", [
|
||||
# 股票 / 可转债 / 回购:有笔数
|
||||
(1, "600000", True), (1, "688111", True), (1, "689009", True),
|
||||
(0, "000001", True), (0, "300750", True), (0, "128136", True),
|
||||
(1, "204001", True),
|
||||
# 沪市 LOF 50x:实测有真实笔数,不能一刀切
|
||||
(1, "501050", True), (1, "501018", True), (1, "502000", True), (1, "506000", True),
|
||||
# 沪市 ETF/LOF 51/52/56/58:无笔数
|
||||
(1, "510310", False), (1, "511990", False), (1, "512880", False),
|
||||
(1, "513050", False), (1, "515000", False), (1, "518880", False),
|
||||
(1, "560010", False), (1, "562500", False), (1, "588000", False),
|
||||
# 深市基金:无笔数
|
||||
(0, "159915", False), (0, "159001", False),
|
||||
(0, "161725", False), (0, "160119", False), (0, "165520", False),
|
||||
# 指数:无笔数
|
||||
(1, "000001", False), (0, "399001", False), (1, "880305", False),
|
||||
])
|
||||
def test_has_order_count(market, code, expect):
|
||||
from pytdxdata.symbols import has_order_count
|
||||
assert has_order_count(market, code) is expect
|
||||
|
||||
|
||||
# ---------- 跨时区时间戳归一化(美股 EX 通道) ----------
|
||||
#
|
||||
# 通达信 EX 通道对美股返回的时间戳把两个时区拼在一起:日期字段是**美东交易日**、
|
||||
# 时分字段是**北京时间**。直拼出的 datetime 会把同一交易日的下午半场(北京
|
||||
# 00:00~04:00)排到上午半场(北京 21:30~23:59)之前,单日 390 根全部错位。
|
||||
|
||||
def _bar(market: int, y: int, mo: int, d: int, h: int, mi: int) -> SecurityBar:
|
||||
return SecurityBar(market=market, code="X", open=1.0, high=1.0, low=1.0,
|
||||
close=1.0, vol=1.0, amount=1.0,
|
||||
year=y, month=mo, day=d, hour=h, minute=mi)
|
||||
|
||||
|
||||
def test_us_evening_segment_keeps_date():
|
||||
# 北京 21:31 ~ 23:59 与美东交易日属同一自然日,日期不动
|
||||
b = _bar(74, 2026, 10, 12, 21, 31)
|
||||
assert b.datetime == datetime(2026, 10, 12, 21, 31)
|
||||
assert b.trading_date == "2026-10-12"
|
||||
|
||||
|
||||
def test_us_midnight_segment_rolls_to_next_day():
|
||||
# 北京次日 00:00 ~ 04:00 实际是美东 12:00~16:00,日期要 +1 天
|
||||
b = _bar(74, 2026, 10, 12, 4, 0)
|
||||
assert b.datetime == datetime(2026, 10, 13, 4, 0)
|
||||
assert (b.hour, b.minute) == (4, 0) # 时分不参与归一化
|
||||
assert b.trading_date == "2026-10-12" # 仍归属美东 10-12 交易日
|
||||
|
||||
|
||||
def test_us_split_hour_boundary():
|
||||
assert _bar(74, 2026, 10, 12, 11, 59).day == 13
|
||||
assert _bar(74, 2026, 10, 12, 12, 0).day == 12
|
||||
|
||||
|
||||
def test_us_month_and_year_rollover():
|
||||
assert _bar(74, 2026, 10, 31, 4, 0).datetime == datetime(2026, 11, 1, 4, 0)
|
||||
assert _bar(74, 2026, 12, 31, 4, 0).datetime == datetime(2027, 1, 1, 4, 0)
|
||||
|
||||
|
||||
def test_cn_and_hk_bars_untouched():
|
||||
cn = _bar(1, 2026, 10, 12, 9, 31) # A股/标准通道
|
||||
assert (cn.year, cn.month, cn.day) == (2026, 10, 12)
|
||||
assert cn.trading_date == "2026-10-12"
|
||||
hk = _bar(31, 2026, 10, 12, 4, 0) # 港股同为 UTC+8,凌晨段不应 +1
|
||||
assert (hk.year, hk.month, hk.day) == (2026, 10, 12)
|
||||
assert hk.trading_date == "2026-10-12"
|
||||
|
||||
|
||||
def test_us_full_session_is_monotonic_and_single_trading_day():
|
||||
# 实测口径:北京 21:31~23:59(149 根)+ 次日 00:00~04:00(241 根)= 390 根
|
||||
mins = list(range(21 * 60 + 31, 24 * 60)) + list(range(4 * 60 + 1))
|
||||
assert len(mins) == 390
|
||||
bars = [_bar(74, 2026, 10, 12, m // 60, m % 60) for m in mins]
|
||||
bars.sort(key=lambda b: b.datetime)
|
||||
dts = [b.datetime for b in bars]
|
||||
assert dts == sorted(dts) # 归一化后严格单调
|
||||
assert dts[0] == datetime(2026, 10, 12, 21, 31)
|
||||
assert dts[-1] == datetime(2026, 10, 13, 4, 0)
|
||||
# 全部归属于同一个美东交易日
|
||||
assert {b.trading_date for b in bars} == {"2026-10-12"}
|
||||
# 但横跨两个北京自然日(这正是不能用 datetime.date() 聚合的原因)
|
||||
by_day = Counter(b.datetime.date() for b in bars)
|
||||
assert by_day[date(2026, 10, 12)] == 149
|
||||
assert by_day[date(2026, 10, 13)] == 241
|
||||
|
||||
|
||||
# ---------- MAC 通道 K线(0x122E)同样归一化 ----------
|
||||
|
||||
def _mac_bars_body(records: list[tuple[int, int]]) -> bytes:
|
||||
"""构造 0x122E 响应体:33B头(count@24) + 每条36B(4B ymd + 4B time秒 + 7float)。"""
|
||||
body = bytearray(33 + len(records) * 36)
|
||||
struct.pack_into("<HBHI", body, 24, 7, 1, len(records), 0)
|
||||
for i, (ymd, tsec) in enumerate(records):
|
||||
struct.pack_into("<II7f", body, 33 + i * 36,
|
||||
ymd, tsec, 10.0, 10.5, 9.5, 10.2, 0.0, 1000.0, 0.0)
|
||||
return bytes(body)
|
||||
|
||||
|
||||
def test_mac_kline_us_bar_normalized():
|
||||
# _probe_bar(日期区间二分依赖它)对美股走的就是这条命令
|
||||
cmd = MacKlineCmd(74, "AAPL", KlinePeriod.MIN_1, Adjust.NONE, 0, 5)
|
||||
bars = cmd.parse_response(_mac_bars_body([(20261012, 4 * 3600)])) # 北京 04:00
|
||||
assert len(bars) == 1
|
||||
assert bars[0].datetime == datetime(2026, 10, 13, 4, 0)
|
||||
assert bars[0].trading_date == "2026-10-12"
|
||||
|
||||
|
||||
def test_mac_kline_hk_bar_untouched():
|
||||
cmd = MacKlineCmd(31, "00700", KlinePeriod.MIN_1, Adjust.NONE, 0, 5)
|
||||
bars = cmd.parse_response(_mac_bars_body([(20260729, 13 * 3600 + 40 * 60)]))
|
||||
assert (bars[0].datetime) == datetime(2026, 7, 29, 13, 40)
|
||||
assert bars[0].trading_date == "2026-07-29"
|
||||
|
||||
|
||||
# ---------- 排序:EX 返回顺序按时间正序(切片取「最新 N 根」依赖它) ----------
|
||||
|
||||
def test_finish_bars_sorts_us_session_chronologically():
|
||||
from pytdxdata.api import TdxData
|
||||
from pytdxdata.symbols import parse_symbols
|
||||
|
||||
sym = parse_symbols("usAAPL")[0]
|
||||
# 故意按错误的顺序给(凌晨段在前)
|
||||
out = TdxData._finish_bars([_bar(74, 2026, 10, 12, 4, 0),
|
||||
_bar(74, 2026, 10, 12, 21, 31)], sym)
|
||||
assert [b.datetime for b in out] == [datetime(2026, 10, 12, 21, 31),
|
||||
datetime(2026, 10, 13, 4, 0)]
|
||||
assert {b.symbol for b in out} == {"usAAPL"}
|
||||
|
||||
# ---------- 报价小数位:沪市指数必须 2 位(回归:曾整体少 10 倍) ----------
|
||||
|
||||
@pytest.mark.parametrize("market,code,expect", [
|
||||
# 沪市指数:回归 bug —— 曾被判 3 位,导致价格少 10 倍
|
||||
(1, "000001", 2), # 上证指数
|
||||
(1, "000016", 2), # 上证50
|
||||
(1, "000300", 2), # 沪深300
|
||||
(1, "000688", 2), # 科创50
|
||||
(1, "880001", 2), # 板块指数(8 开头,曾被判 3 位)
|
||||
(1, "880305", 2),
|
||||
# 沪市 ETF/基金:仍是 3 位
|
||||
(1, "510300", 3),
|
||||
(1, "588000", 3),
|
||||
(1, "511010", 3),
|
||||
(1, "501018", 3),
|
||||
# 沪市股票:2 位
|
||||
(1, "600000", 2),
|
||||
(1, "601398", 2),
|
||||
(1, "688111", 2), # 科创板
|
||||
# 深市
|
||||
(0, "399001", 2), # 深证成指
|
||||
(0, "399006", 2), # 创业板指
|
||||
(0, "000001", 2), # 平安银行(不是指数!)
|
||||
(0, "300750", 2),
|
||||
(0, "159915", 3), # 深市 ETF
|
||||
# 债券/回购:2026-10-22 修正(沪市曾少 100 倍、深市曾少 10 倍)
|
||||
(1, "019742", 4), # 沪市国债 24特国01(110.228,曾读成 11022.8)
|
||||
(1, "019547", 4), # 沪市国债
|
||||
(1, "010706", 4), # 沪市老国债 07国债06
|
||||
(1, "018014", 4), # 沪市 018 段
|
||||
(1, "020812", 4), # 沪市贴债
|
||||
(1, "110077", 4), # 沪市可转债 洪城转债(199.279,曾读成 19927.9)
|
||||
(1, "113616", 4), # 沪市可转债 韦尔转债
|
||||
(1, "118031", 4), # 沪市可转债
|
||||
(1, "111000", 4), # 沪市 111 段
|
||||
(1, "132024", 4), # 沪市可交换债
|
||||
(1, "184832", 4), # 沪市 184 段
|
||||
(1, "204001", 4), # 沪市国债逆回购 GC001(1.385,曾读成 138.5)
|
||||
(1, "204028", 4),
|
||||
(0, "123107", 4), # 深市可转债 温氏转债(112.189,曾读成 1121.89)
|
||||
(0, "128136", 4), # 深市可转债 立讯转债
|
||||
(0, "127025", 4), # 深市可转债
|
||||
(0, "131810", 4), # 深市国债逆回购 R-001(1.38,曾读成 13.8)
|
||||
(0, "148004", 4), # 深市 148 段
|
||||
(0, "112001", 4), # 深市公司债
|
||||
(0, "101917", 4), # 深市国债
|
||||
(0, "520261", 4), # 深市 520 段
|
||||
# 沪市 B 股:3 位(2026-10-22 修正,曾按 2 位少 10 倍)
|
||||
(1, "900901", 3), # 云赛B股
|
||||
(1, "900948", 3),
|
||||
# 深市 B 股:2 位(与沪市不同!)
|
||||
(0, "200011", 2), # 深物业B
|
||||
(0, "200488", 2),
|
||||
# 边界:999xxx 是「指数」,不能被 9 开头当成 B 股
|
||||
(1, "999999", 2), # 上证指数
|
||||
(1, "999998", 2), # A股指数
|
||||
])
|
||||
def test_price_decimal_digits(market, code, expect):
|
||||
assert price_decimal_digits(market, code) == expect
|
||||
|
||||
|
||||
def test_price_decimal_digits_strips_nul():
|
||||
assert price_decimal_digits(1, "000016\x00\x00") == 2
|
||||
|
||||
|
||||
# ---------- 美股分时跨日:hour 必须是合法时钟值(回归:曾出现 hour=27) ----------
|
||||
|
||||
def _us_minutes(n: int = 390) -> list:
|
||||
"""构造 usAAPL 的 390 条分时:从北京 21:30 起连续 n 分钟。"""
|
||||
from pytdxdata.models import MinuteBar
|
||||
|
||||
start = 21 * 60 + 30
|
||||
return [
|
||||
MinuteBar(price=100.0 + i * 0.01, vol=1.0,
|
||||
hour=(start + i) // 60, minute=(start + i) % 60,
|
||||
market=74, code="AAPL", date=20261022)
|
||||
for i in range(n)
|
||||
]
|
||||
|
||||
|
||||
def test_minute_bar_us_hour_is_valid_clock_value():
|
||||
"""EX 分时的 minutes 是「自开盘起的连续分钟数」,直接 //60 会得到 hour=27。"""
|
||||
bars = _us_minutes()
|
||||
assert all(0 <= b.hour < 24 for b in bars)
|
||||
# 归一化前的原始值确实越界(21:30 + 390min = 28:00)
|
||||
assert max(b.hour + b.day_offset * 24 for b in bars) > 23
|
||||
|
||||
|
||||
def test_minute_bar_us_day_offset_marks_next_day():
|
||||
bars = _us_minutes()
|
||||
assert bars[0].day_offset == 0 # 21:30 当晚
|
||||
assert bars[-1].day_offset == 1 # 03:59 次日
|
||||
assert (bars[-1].hour, bars[-1].minute) == (3, 59)
|
||||
|
||||
|
||||
def test_minute_bar_us_datetime_is_monotonic_across_midnight():
|
||||
bars = _us_minutes()
|
||||
dts = [b.datetime for b in bars]
|
||||
assert dts == sorted(dts)
|
||||
assert dts[0] == datetime(2026, 10, 22, 21, 30)
|
||||
assert dts[-1] == datetime(2026, 10, 23, 3, 59)
|
||||
|
||||
|
||||
def test_minute_bar_us_trading_date_spans_two_calendar_days():
|
||||
"""一个美股交易日的分时横跨两个北京自然日,trading_date 必须都是同一天。"""
|
||||
bars = _us_minutes()
|
||||
tds = {b.trading_date for b in bars}
|
||||
assert tds == {"2026-10-22"}
|
||||
# 而 datetime.date() 会给出两天 —— 这正是不能用它的原因
|
||||
assert len({b.datetime.date() for b in bars}) == 2
|
||||
|
||||
|
||||
def test_minute_bar_cn_unchanged():
|
||||
"""A股/港股分时 hour 本就合法,归一化必须零行为变化。"""
|
||||
from pytdxdata.models import MinuteBar
|
||||
|
||||
for hour, minute in ((9, 30), (11, 29), (13, 0), (14, 59)):
|
||||
b = MinuteBar(price=10.0, vol=1.0, hour=hour, minute=minute,
|
||||
market=1, code="600000", date=20261022)
|
||||
assert (b.hour, b.minute) == (hour, minute)
|
||||
assert b.day_offset == 0
|
||||
assert b.datetime == datetime(2026, 10, 22, hour, minute)
|
||||
assert b.trading_date == "2026-10-22"
|
||||
|
||||
|
||||
def test_minute_bar_cn_datetime_monotonic_full_session():
|
||||
"""A股全天 240 条(09:30~11:29 + 13:00~14:59)时间轴严格递增。"""
|
||||
from pytdxdata.models import MinuteBar
|
||||
|
||||
bars = []
|
||||
for idx in range(240):
|
||||
if idx < 120:
|
||||
hour, minute = 9 + (30 + idx) // 60, (30 + idx) % 60
|
||||
else:
|
||||
hour, minute = 13 + (idx - 120) // 60, (idx - 120) % 60
|
||||
bars.append(MinuteBar(price=10.0, vol=1.0, hour=hour, minute=minute,
|
||||
market=1, code="600000", date=20261022))
|
||||
dts = [b.datetime for b in bars]
|
||||
assert dts == sorted(dts)
|
||||
assert dts[0] == datetime(2026, 10, 22, 9, 30)
|
||||
assert dts[-1] == datetime(2026, 10, 22, 14, 59)
|
||||
assert {b.trading_date for b in bars} == {"2026-10-22"}
|
||||
@@ -0,0 +1,91 @@
|
||||
"""证券清单(0x053E)回归测试:请求长度 + 北交所降级。全部离线。
|
||||
|
||||
背景(2026-10-23 实测):`build_request` 曾经多塞一个 ``H``(18 字节),
|
||||
服务端只消费前 16 字节,剩下的 ``00 00`` 留在它的接收缓冲里,把**下一个请求**
|
||||
的开头吃掉 → 同一连接「第一次成功、第二次起全部通信超时」。
|
||||
`get_universe` 的分页器复用连接,于是 66/28 页全靠失败重连,冷启动 12~18s。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import struct
|
||||
import unittest.mock as mock
|
||||
|
||||
import pytest
|
||||
|
||||
from pytdxdata.models import Market
|
||||
from pytdxdata.protocol.commands import GetSecurityListCmd
|
||||
|
||||
_HEADER = bytes.fromhex("0c0118640101060006005004")
|
||||
|
||||
|
||||
def test_request_is_exactly_16_bytes():
|
||||
"""12 字节命令头 + market(H) + start(H),一个字节都不能多。"""
|
||||
req = GetSecurityListCmd(1, 0).build_request()
|
||||
assert len(req) == 16
|
||||
assert req == _HEADER + struct.pack("<HH", 1, 0)
|
||||
|
||||
|
||||
def test_request_encodes_market_and_start_little_endian():
|
||||
req = GetSecurityListCmd(0, 2000).build_request()
|
||||
assert req[:12] == _HEADER
|
||||
market, start = struct.unpack("<HH", req[12:])
|
||||
assert (market, start) == (0, 2000)
|
||||
|
||||
|
||||
def test_request_has_no_trailing_padding():
|
||||
"""尾部不能有 padding:多余的 0 会让服务端解析错位。"""
|
||||
for market in (0, 1, 2):
|
||||
for start in (0, 1000, 27000):
|
||||
req = GetSecurityListCmd(market, start).build_request()
|
||||
assert len(req) == 16, f"market={market} start={start} -> {len(req)} 字节"
|
||||
|
||||
|
||||
def test_parse_response_reads_all_records():
|
||||
"""29 字节/条,头部 2 字节条数。"""
|
||||
code, name = b"600000", "浦发银行".encode("gbk").ljust(8, b"\x00")
|
||||
rec = struct.pack("<6sH8s4sBI4s", code, 100, name, b"\x00" * 4, 2, 0, b"\x00" * 4)
|
||||
assert len(rec) == 29
|
||||
body = struct.pack("<H", 2) + rec + rec
|
||||
rows = GetSecurityListCmd(1, 0).parse_response(body)
|
||||
assert len(rows) == 2
|
||||
assert rows[0].code == "600000"
|
||||
assert rows[0].decimal_point == 2
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_bj_universe_returns_empty_without_network():
|
||||
"""北交所清单协议不支持:必须立刻返回空,且**不碰连接池**。"""
|
||||
from pytdxdata.api import TdxData
|
||||
|
||||
td = TdxData()
|
||||
td._std_pool.acquire = mock.AsyncMock(side_effect=AssertionError("不该联网"))
|
||||
td._std_pool.acquire_many = mock.AsyncMock(
|
||||
side_effect=AssertionError("不该联网"))
|
||||
rows = await td.get_universe("bj")
|
||||
assert rows == []
|
||||
td._std_pool.acquire.assert_not_called()
|
||||
td._std_pool.acquire_many.assert_not_called()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_bj_market_is_in_unsupported_set():
|
||||
from pytdxdata import api as api_mod
|
||||
|
||||
assert int(Market.BJ) in api_mod._LIST_UNSUPPORTED_MARKETS
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_ex_all_fetched_once_in_get_universe():
|
||||
"""`get_universe()` 跨多个 EX 市场时,EX 全量只能取一次。"""
|
||||
from pytdxdata.api import TdxData
|
||||
|
||||
td = TdxData()
|
||||
calls = {"n": 0}
|
||||
|
||||
async def fake_ex_all():
|
||||
calls["n"] += 1
|
||||
return []
|
||||
|
||||
td._ex_all = fake_ex_all
|
||||
await td.get_universe("all")
|
||||
assert calls["n"] == 1, f"_ex_all 被调用 {calls['n']} 次(应为 1)"
|
||||
Reference in new issue
Block a user