Compare commits
1
Commits
v0.7.0
..
feec86bf13
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
feec86bf13 |
No files matched your search
+7
-553
@@ -2,567 +2,21 @@
|
|||||||
|
|
||||||
本项目遵循 [语义化版本](https://semver.org/lang/zh-CN/)。
|
本项目遵循 [语义化版本](https://semver.org/lang/zh-CN/)。
|
||||||
|
|
||||||
## [0.7.0] - 2026-10-28
|
|
||||||
|
|
||||||
### 新增
|
|
||||||
|
|
||||||
- **DataFrame 直出:`TdxData(df="polars")` / `TdxData(df="pandas")`**。
|
|
||||||
|
|
||||||
表格型接口不再只能返回 `list[dataclass]`,构造时声明一次后端即可直接拿到 DataFrame:
|
|
||||||
|
|
||||||
```python
|
|
||||||
async with TdxData(df="polars") as td:
|
|
||||||
df = await td.get_bars("sh600000", KlinePeriod.DAY, count=240)
|
|
||||||
```
|
|
||||||
|
|
||||||
- 覆盖 **42 个**返回表格数据的方法(33 个 `list[dataclass]`、2 个 `list[dict]`、
|
|
||||||
2 个批量 `dict[str, list[...]]`、`get_indicators` 与 `get_index_trade_stats(_multi)`),
|
|
||||||
另含 deprecated 别名(`get_kline` / `get_transactions` / `get_goods_*` 等)。
|
|
||||||
- **核心包仍然零依赖**:pandas / polars 是可选 extras ——
|
|
||||||
`pip install "pytdxdata[polars]"` / `[pandas]` / `[dataframe]`。
|
|
||||||
- `df=None`(默认)**行为与 0.6.x 完全一致**,且装饰器直接透传,零额外开销。
|
|
||||||
- 取值非法抛 `ValueError` 并列出可用值;后端未安装抛 `FrameBackendError`(继承 `ImportError`,
|
|
||||||
`e.name` 为缺的包名,消息里带安装命令)——**在构造时就失败**,不会拖到第一次查询。
|
|
||||||
|
|
||||||
- **列口径**(三条约定,一次定死):
|
|
||||||
|
|
||||||
| 约定 | 说明 |
|
|
||||||
|---|---|
|
|
||||||
| 时间列替换 | `SecurityBar`/`IndexBar`/`MinuteBar` 的 `year/month/day/hour/minute` 不再单列输出,统一给 `datetime`(polars `Datetime('us')` / pandas `datetime64[ns]`)与 `trading_date`;`XdxrRecord` 给 `date` |
|
|
||||||
| 嵌套定型 | `SecurityQuote.bid`/`ask` 展开成 `bid1_price`…`ask5_vol` 共 10 列(缺档 `None`);`fields` 字典 JSON 化成字符串列(空字典给 `None`);`TdxBlock.codes` 保持列表 |
|
|
||||||
| 容器展开 | `get_indicators` → bars + 各指标一列(长度不足补 `None`);`get_index_trade_stats(_multi)` → `minutes` 展开并补 `index`/`date` 常量列;`get_kline_batch`/`get_minute_batch` → 各标的纵向拼接成一张表 |
|
|
||||||
|
|
||||||
- **嵌套调用不二次转换**:`get_kline_batch` 这类方法内部还会调 `get_bars`,
|
|
||||||
用 `ContextVar`(按 task 隔离)记录层次,内层一律返回原始 `list`,只有最外层那次才转 DataFrame。
|
|
||||||
并发调用互不干扰。
|
|
||||||
|
|
||||||
### 性能
|
|
||||||
|
|
||||||
- **按日期区间取 K 线:右端贴最新时改走「倒推」,1 分钟 K 线 4.2s → 82ms(51 倍)**。
|
|
||||||
|
|
||||||
原先 `start_date` / `end_date` 一律走「探总根数 → 二分定位偏移」。1 分钟 K 线历史总量
|
|
||||||
二十万根量级,等于 6 次翻倍探测 + 2 次方向探测 + **两次各 ~18 次**跳转,共 36+ 次串行往返。
|
|
||||||
而「取今天」「增量到最新」这类占绝对多数的调用,右端本来就贴着最新,根本不需要二分。
|
|
||||||
|
|
||||||
| 场景(1 分钟,冷启动) | 优化前 | 优化后 |
|
|
||||||
|---|---|---|
|
|
||||||
| 单只「取今天」 | 4,208 ms | **82 ms** |
|
|
||||||
| 单只「2 天前 ~ now」 | 2,211 ms | **75 ms** |
|
|
||||||
| 单只「取最近 3 天」 | 2,000 ms+ | **239 ms** |
|
|
||||||
| 单只「历史中段(5~3 天前)」 | 1,512 ms | 285 ms |
|
|
||||||
| 全市场 5,225 只「取今天」 | — | 32.1 s(1,140,699 根,**精确当天口径**) |
|
|
||||||
|
|
||||||
做法:右端离最新不超过 `_BACKTRACK_GRACE`(10 天,覆盖长假)时,直接
|
|
||||||
`start=0, count=N` 拿最新 N 根再裁剪 —— N 按时间跨度估算(`_BARS_PER_DAY`),
|
|
||||||
一天内一次就命中,不够才翻倍。历史中段仍走二分,结果完全一致。
|
|
||||||
|
|
||||||
偏移方向是**通道级属性**(与具体标的无关),首次探测后缓存在实例上:
|
|
||||||
全市场 5,225 只批量调用只付一次探测成本,不会每只探两遍。
|
|
||||||
|
|
||||||
- **多标的 K 线改「多连接 × 单连接流水线」抓取,全市场 1 分钟 K 线 43.6s → 5s 级
|
|
||||||
(盘中增量 1.5s)**。
|
|
||||||
|
|
||||||
单只 K 线是「一请求一标的」,5,000 只就是 5,000 次往返。实测单连接串行 **33ms/只
|
|
||||||
≈ 恰好一个 RTT** —— 瓶颈是**往返次数,不是带宽**(此前「单 IP 限流 1.5MB/s」的
|
|
||||||
结论是浅流水下的假象:同一台服务器 16 连接 × 深度 8 实测能跑到 5.2 MB/s)。
|
|
||||||
|
|
||||||
做法:同一连接连续下发 `pipeline` 条请求(协议帧自描述长度且服务器**按序应答**,
|
|
||||||
无需等前一个回来),再叠加多台服务器并行(每台 1 连接,尊重 `per_server_cap`)。
|
|
||||||
|
|
||||||
| 场景(全市场 5,225 只,1 分钟) | 优化前(逐只 `concurrency=6`) | 优化后 |
|
|
||||||
|---|---|---|
|
|
||||||
| 取当天全天(240 根) | 43.6 s | **5~8 s** |
|
|
||||||
| 盘中 10:00(35 根) | 15 s+ | **~2 s** |
|
|
||||||
| 盘中增量(最新 2 根) | 12 s+ | **~1.5 s** |
|
|
||||||
|
|
||||||
- 流水深度默认**自适应**(`pipeline=None`):按单请求字节数把在途量控制在 ~60KB,
|
|
||||||
夹在 8~32 之间 —— 取 240 根用深度 8,只取最新 2 根用深度 32。
|
|
||||||
实测过深反而更慢(24 台 depth=64 比 depth=8 慢一倍,服务器会攒够一批才回)。
|
|
||||||
- 服务器**只取最快的 12 台**:池里 24 台全用反而被慢节点拖长尾
|
|
||||||
(`count=2` 全市场:24 台 2.95s / 12 台 1.45s)。
|
|
||||||
- **盘中只拉已走过的根数**:`start_date=今天` 时按交易时段折算(10:00 → 30 根,
|
|
||||||
而不是 240 根),数据量直接差 8 倍。收盘后回到一整天的量级。
|
|
||||||
- 标准通道(日线不复权)同样走批量:全市场日线 240 根 49~72s → **~10s**。
|
|
||||||
- 流水线中途失败时该批**交回逐只重试**(流水线失败意味着响应流已错配、连接作废),
|
|
||||||
连接全挂时队列里剩下的标的也一并回退 —— 不会静默少数据(新增测试覆盖)。
|
|
||||||
- `pipeline=1` 可关闭批量路径,行为与旧版逐只路径一致。
|
|
||||||
|
|
||||||
- **修正「右端是否贴最新」的时间戳比较**:`0x122E` 返回的 `hour/minute` 是**该周期的
|
|
||||||
结束时刻**,最新一根可能比 `datetime.now()` 略大(14:29 那根此刻还在走)。
|
|
||||||
原先没有容差,`end_date=datetime.now()` 这类最自然的写法会被误判成「历史中段」而走慢路径。
|
|
||||||
|
|
||||||
### 修复
|
|
||||||
|
|
||||||
- **EX 通道 K 线的 `amount` 恒为 0.0,且末位字段(结算价)被解包丢弃**。
|
|
||||||
|
|
||||||
`0x23FF` 一条记录 32B = 4B 时间戳 + **28B 行情**,这 28B 只有**一个** 4B 槽位可复用,
|
|
||||||
语义按市场分两类(全市场实测,用 `成交额 / (close × vol)` 逐条校验):
|
|
||||||
|
|
||||||
| 市场 | 槽位 5 | 槽位 6 | 槽位 7 | 校验 |
|
|
||||||
|---|---|---|---|---|
|
|
||||||
| 港股主板 31 / 港股基金 49 | **成交额**(f) | 成交量(手=100 股) | 未知 | 额/(c·vol) = **100.0** |
|
|
||||||
| 港股创业板 48 / 新三板 44 / 美股 74 | **成交额**(f) | 成交量(股) | 0 | 额/(c·vol) ≈ **1.0** |
|
|
||||||
| 期货 / 期权(7/8/9/47…) | 持仓量(I) | 成交量 | **结算价**(f) | 与 CFFEX 官网日结逐日相符 |
|
|
||||||
|
|
||||||
改动前:`amount` 被硬编码成 `0.0`、第 7 个 float 用 `_` 丢弃 —— 于是港股/美股/新三板
|
|
||||||
**明明帧内有成交额却全是 0**,期货期权的结算价也拿不到(用户只能换源)。
|
|
||||||
|
|
||||||
- 新增 `SecurityBar.settle`(结算价,`float | None`)。仅 EX 期货/期权**日线**有值,
|
|
||||||
分钟线该槽位恒 0 已归一化为 `None`;其它通道恒 `None`。
|
|
||||||
- 权益类市场的 `amount` 回填真实成交额。⚠️ 港股主板/基金的 `vol` 单位是**手(=100 股)**,
|
|
||||||
`amount/vol` 是均价的 **100 倍**。
|
|
||||||
- 期货/期权的 `amount` **仍是 0.0** —— 帧内确实没有成交额,这是协议限制,已在
|
|
||||||
`SecurityBar` 与文档中写清,避免使用者再按文档误判为调用方 bug。
|
|
||||||
- 港股指数 27 / 中证指数 62 该槽位量级对不上(既非成交额也非结算价),保守保持 `0.0`。
|
|
||||||
- `SecurityBar.__setstate__` 兜底旧磁盘缓存:缓存键没有 schema 版本,升级后读旧 pickle
|
|
||||||
会缺 `settle` 槽位,默认反序列化不补默认值 → 直接 `AttributeError`,已容错。
|
|
||||||
EX K 线缓存键额外加 `@v2` 标记,让 `amount` 恒 0 的旧缓存自然失效。
|
|
||||||
|
|
||||||
### 兼容性
|
|
||||||
|
|
||||||
- 无破坏性变更。不传 `df` 时所有方法签名、返回类型、行为与 0.6.9 完全相同。
|
|
||||||
- 空结果返回**列名齐全的空表**(列由模型定义推出),dtype 由后端自行推断。
|
|
||||||
- `SecurityBar` 新增可选字段 `settle`(默认 `None`):K 线 DataFrame 因此多一列 `settle`,
|
|
||||||
位置参数构造的老代码不受影响。
|
|
||||||
|
|
||||||
### 升级
|
|
||||||
|
|
||||||
```bash
|
|
||||||
pip install -U pytdxdata # 只要 dataclass
|
|
||||||
pip install -U "pytdxdata[polars]" # 想直出 DataFrame
|
|
||||||
```
|
|
||||||
|
|
||||||
- **磁盘缓存不需要手动清**:`SecurityBar` 的 `settle` 由 `__setstate__` 兜底补默认值,
|
|
||||||
旧缓存照常可读;EX K 线缓存键已带 `@v2`,`amount` 恒 0 的旧记录会自动失效重取。
|
|
||||||
- 新参数 `pipeline`(`get_bars`)默认自适应,**不传就是最优**;只有想显式退回旧的逐只路径时才传 `1`。
|
|
||||||
|
|
||||||
|
|
||||||
### 文档
|
|
||||||
|
|
||||||
- 新增 [DataFrame 输出](https://openbot-coder.github.io/pytdxdata/dataframe/) 专页,
|
|
||||||
`API_REFERENCE.md`(构造参数表补 `df` 与 `ex_pool_cap`、`pool_max` 默认值修正为 24)、
|
|
||||||
`SKILL.md`、`llms.txt`、`README.md`、`mkdocs.yml` 导航同步。
|
|
||||||
- 全站同步 0.7.0 口径:首页特性表与导航卡片、安装页(extras 与构造参数表补
|
|
||||||
`df` / `std_pool_cap` / `ex_pool_cap`)、场景示例的「转成 pandas」改为推荐 `df=` 开关、
|
|
||||||
数据模型页说明 dataclass 与 DataFrame 是同一份数据、README「已知限制」补 EX 成交额口径。
|
|
||||||
|
|
||||||
## [0.6.9] - 2026-10-23
|
|
||||||
|
|
||||||
### 修复
|
|
||||||
|
|
||||||
- **`get_universe` 冷启动 12~18s / 偶发失败:证券清单请求多发了 2 字节,污染连接**。
|
|
||||||
|
|
||||||
`GetSecurityListCmd` 的请求体是「12 字节命令头 + market(H) + start(H)」= **16 字节**,
|
|
||||||
但实现里多塞了一个 `H`(写死 0),实际发出 **18 字节**。服务端只消费前 16 字节,
|
|
||||||
剩下的 `00 00` 停在它的接收缓冲里,把**下一个请求**的开头吃掉 —— 于是同一条连接
|
|
||||||
「第一次成功、第二次起永远无响应」。
|
|
||||||
|
|
||||||
分页器复用连接,等于每拉一页都要先失败重连,冷启动被拖到 12~18s,还经常直接抛
|
|
||||||
`ConnectionError`。
|
|
||||||
|
|
||||||
| 场景 | 修复前 | 修复后 |
|
|
||||||
|---|---|---|
|
|
||||||
| `get_universe("sh")` 冷启动(27,534 条 / 28 页) | 12,267ms | **1,610ms** |
|
|
||||||
| `get_universe("sz")` 冷启动(24,460 条 / 25 页) | 18,260ms | **571ms** |
|
|
||||||
| `get_universe()` 全市场冷启动(146,848 条) | 19,085ms | **4,371ms** |
|
|
||||||
| `get_universe()` 热缓存 | — | 669ms |
|
|
||||||
| 同一连接连发 2 次清单(27 台主机) | **0/27 成功** | **27/27 成功** |
|
|
||||||
|
|
||||||
- **北交所清单不再卡 15s 后报错,改为立即返回空列表**。
|
|
||||||
|
|
||||||
`0x053E` 对 `market=2` 在实测的 **27 台标准主机上全部无响应**(26 台 6s 无回包、
|
|
||||||
1 台立刻拒连),而同市场的 `0x053D` 数量接口正常(返回 389)。原实现会重试 2 次、
|
|
||||||
每次 3s 超时,最后让 `get_universe()`(不带参数)整批失败。
|
|
||||||
|
|
||||||
现在该市场直接返回空列表并 `WARNING` 一次,**北交所标的本身不受影响**:
|
|
||||||
`get_quotes("bj920992")` / `get_bars` / 逐笔全部正常(实测 11.79 与日线 close 一致)。
|
|
||||||
|
|
||||||
### 优化
|
|
||||||
|
|
||||||
- **`get_universe()` 的 EX 全量只取一次**:命中缓存也要反序列化 10 万条(~200ms),
|
|
||||||
原先十几个 EX 市场各来一次,纯属白烧。
|
|
||||||
- **EX 商品列表改并发翻页**(110 页,8 并发):全市场冷启动的一半时间花在这里。
|
|
||||||
|
|
||||||
### 文档
|
|
||||||
|
|
||||||
- `get_universe` docstring 写明北交所限制;`API_REFERENCE` / `llms.txt` / `SKILL.md` 同步。
|
|
||||||
- 新增 `tests/test_universe.py` 7 项:请求长度锁定 16 字节、无尾部 padding、
|
|
||||||
北交所返回空且**不碰连接池**、`_ex_all` 只取一次。
|
|
||||||
|
|
||||||
### 已知限制(非本次引入)
|
|
||||||
|
|
||||||
- EX 全量列表含**非证券条目**(如人员名录),`(market, code)` 在原始数据里就不唯一,
|
|
||||||
且重复项出现在**同一页内部**。属数据源特性,未做去重(去重会丢真实品种)。
|
|
||||||
|
|
||||||
## [0.6.8] - 2026-10-23
|
|
||||||
|
|
||||||
### 修复
|
|
||||||
|
|
||||||
- **CLI 价格按品种小数位显示;`tdx minute` 时间列改回真实时钟**。
|
|
||||||
|
|
||||||
两处纯显示层缺陷,数据本身没问题(分时/逐笔数值见 0.6.7 的核对结论):
|
|
||||||
|
|
||||||
| 现象 | 原因 |
|
|
||||||
|---|---|
|
|
||||||
| `tdx transactions sh113708` 打印 `128.89` | 价格写死 `%8.2f`;可转债报价步长 0.001 元,末位被截掉 |
|
|
||||||
| `tdx minute sh113708` 时间列 `00:00~03:59` | 按序号推算 `i // 60, i % 60`,与 bar 自带时间无关 |
|
|
||||||
| `tdx quotes sh113708` 打印 `128.8920` | `price_decimal_digits` 是**解码位数**(原始整数 / 10⁴),第 4 位恒为 0 |
|
|
||||||
|
|
||||||
| 位置 | 变化 |
|
|
||||||
|---|---|
|
|
||||||
| `_price_fmt(dp)`(新) | 宽度随小数位走(`%8.2f` → `%9.3f`),不破坏表格对齐 |
|
|
||||||
| `_display_dp(market, code)`(新) | 可转债(沪 `110/111/113/118`、深 `123/127/128`)由解码 4 位降到显示 3 位;其余沿用 `price_decimal_digits`(ETF 3 位、国债/回购 4 位、股票/指数 2 位) |
|
|
||||||
| `cmd_kline` / `cmd_transactions` / `cmd_minute` / `_print_quote` | 统一改用上面的格式串 |
|
|
||||||
| `cmd_minute` | 时间取 `MinuteBar.hour/minute`,不再用序号推算 |
|
|
||||||
| `cmd_transactions` / `cmd_minute` | 改调 `get_ticks` / `get_minutes`,消掉 `get_transactions` / `get_minute` 的 DeprecationWarning |
|
|
||||||
|
|
||||||
新增回归测试 8 项(`tests/test_cli_format.py`)。
|
|
||||||
|
|
||||||
文档:`docs/cli.md` 补「价格小数位按品种适配」对照表与 `tdx minute` 时间列说明。
|
|
||||||
|
|
||||||
## [0.6.7] - 2026-10-22
|
## [0.6.7] - 2026-10-22
|
||||||
|
|
||||||
### 性能
|
|
||||||
|
|
||||||
- **全市场 A股报价 3.5s → 0.33s:单连接流水线 + 工作窃取**(5,227 只 / 66 片实测)。
|
|
||||||
|
|
||||||
老实现 `get_quotes` 是**纯串行 for 循环**逐片发请求:协议硬上限 80 只/请求,
|
|
||||||
全市场必须切 66 片,每片一次往返(≈60ms)→ 3.5s 全花在等网络上。
|
|
||||||
|
|
||||||
关键发现:通达信响应是**自描述长度帧**(16B 帧头 + body)且服务器**按序应答**,
|
|
||||||
因此同一 TCP 连接上可以连续下发多条请求、只付一次往返。实测单连接流水 66 个请求
|
|
||||||
只要 526ms(串行 7,372ms),响应顺序与代码完全对得上(用昨收价交叉比对 30 只,零错配)。
|
|
||||||
|
|
||||||
| 位置 | 变化 |
|
|
||||||
|---|---|
|
|
||||||
| `TdxConnection.execute_pipeline(cmds)`(新) | 一次 write 全部请求 → 按序读回全部响应。中途失败**强制断连**(响应流已错配,不可复用) |
|
|
||||||
| `PooledConnection.execute_pipeline`(新) | 池化包装转发 |
|
|
||||||
| `api._execute_std_pipeline`(新) | 借一条连接流水一整组分片;节点不支持流水线时自动降级为逐片 |
|
|
||||||
| `get_quotes(..., pipeline=True, concurrency=None)` | 新参数;小批量(≤80 只)仍走原路径,零额外开销 |
|
|
||||||
| `_quotes_std` | 改为**工作窃取**:分片放队列,连接做完一批回来再取,快的多干、慢的少干 |
|
|
||||||
| `DynamicPool.warm(n)`(新) | **并发**预建连接。`acquire()` 建连在全局锁内一次一条,22 并发首次会串行握手 22 次 —— 全市场首拉因此多花 5~6s;预热后降到 0.78s |
|
|
||||||
| `CacheManager.get_or_fetch_obj`(新) | 回源直接返回对象,省掉冷路径「pickle→立刻 unpickle」的一次白烧(5,227 条实测 ~40ms) |
|
|
||||||
| `TdxData(pool_max=)` | 默认 12 → 24,让报价并发能到「每台 2 连接」(池仍是懒扩容,不影响其他接口) |
|
|
||||||
|
|
||||||
调优实测(5,227 只 A股,11 台服务器,p50/max):
|
|
||||||
|
|
||||||
| 并发 | 每台连接 | 流水深度 | p50 | max |
|
|
||||||
|---|---|---|---|---|
|
|
||||||
| 11 | 1 | 6 | 455ms | 464ms |
|
|
||||||
| 12 | 1~2 | 3 | 457ms | 523ms |
|
|
||||||
| **22** | **2** | **3** | **325ms** | **334ms** |
|
|
||||||
| 33 | 3 | 2 | 328ms | 345ms |
|
|
||||||
|
|
||||||
⚠️ 剩余下限由公网决定:本机 → 服务器 TCP 握手 RTT 59~80ms,66 片最多只能压到
|
|
||||||
「1 次往返」;网络抖动时段实测会到 460~600ms。
|
|
||||||
|
|
||||||
端到端实测(全默认参数 `TdxData()`,11 台服务器,5,227 只):
|
|
||||||
|
|
||||||
| 场景 | 耗时 |
|
|
||||||
|---|---|
|
|
||||||
| 首次调用(含并发建 22 条连接) | 783ms |
|
|
||||||
| 稳态(第 2 次起) | **300~343ms** |
|
|
||||||
| 命中 TTL 缓存 | 67ms |
|
|
||||||
| 老版本对照(串行) | 5,996ms(首次)/ 3,460ms(稳态) |
|
|
||||||
|
|
||||||
### 修复
|
|
||||||
|
|
||||||
- **指数 / ETF / LOF 的 `trade_count` 不再伪装成成交笔数**。
|
|
||||||
|
|
||||||
通达信对**指数**和**多数场内基金**不提供笔数,但逐笔报文里那 4 字节仍塞了值:
|
|
||||||
指数是「当时均价 × 1000」,ETF/LOF 是「价格 × 1000」,货币 ETF(`sh511990`)恒为 `0`。
|
|
||||||
旧版直接把它当笔数,于是 `tdx transactions sh510310` 会打印出「3994笔」这种假数据
|
|
||||||
(实测 2026-10-22,价 3.989,原字段 3994)。
|
|
||||||
|
|
||||||
| 位置 | 变化 |
|
|
||||||
|---|---|
|
|
||||||
| `TransactionRecord.trade_count_available`(新字段) | 该品种协议上是否带笔数;默认 `True` |
|
|
||||||
| `symbols.has_order_count(market, code)`(新函数) | 按品种判定;**沪市 501x/502x/506x 实测有真实笔数**,不能按「5 开头=基金」一刀切 |
|
|
||||||
| `transaction.py` 三个解析器 | 不可用时 `trade_count=0` + `trade_count_available=False`;历史通道(0x013001,无笔数字段)同样标记 |
|
|
||||||
| `tdx transactions` CLI | 不可用时打印 `(-)`,不再打印假笔数 |
|
|
||||||
|
|
||||||
实测样本(2026-10-22,按前缀各取样):沪股票/科创/深股票/深可转债/沪逆回购 → 真实笔数;
|
|
||||||
沪 `51/52/56/58`、深 `159/15x/16x/17x/18x`、所有指数 → 不可用。
|
|
||||||
|
|
||||||
### 文档
|
|
||||||
|
|
||||||
- **可转债的量纲口径补全**(数据本身无缺陷,但对外文档一片空白,很容易被误判成 bug)。
|
|
||||||
|
|
||||||
起因:核查可转债的分时(`0x122D`)与逐笔(`0x122F`)是否有类似 ETF/LOF 的笔数问题。
|
|
||||||
结论是**价格 / 成交量 / 笔数三样全部正确**,顺带确认了 K 线标度也对;唯一的问题是
|
|
||||||
`vol` 的单位没人写清楚 —— 直接拿 `amount / vol` 算均价会**错 10 倍**。
|
|
||||||
|
|
||||||
实测(2026-10-22,16 只 A股可转债当日 + 8 只历史日 `2026-10-21`):
|
|
||||||
|
|
||||||
| 校验项 | 结果 |
|
|
||||||
|---|---|
|
|
||||||
| 分时末价 / 快照现价 | **1.000000** |
|
|
||||||
| Σ分时 vol / 日线 vol | **1.000000** |
|
|
||||||
| Σ逐笔 vol / 日线 vol | **1.000000** |
|
|
||||||
| 逐笔 VWAP vs `amount/(vol×10)` | 吻合(差 ≤0.002,是 amount 精度) |
|
|
||||||
| 快照 `pre_close` vs 上一交易日日线 `close` | **16/16 精确相等** |
|
|
||||||
| `trade_count` | 真实笔数(`sh111000` 3631 条切片 / 20443 笔) |
|
|
||||||
|
|
||||||
关键口径对照(这是文档缺失的部分):
|
|
||||||
|
|
||||||
| 品种 | 分时 / 逐笔 `vol` | 日线 `vol` | `Σ分时vol / 日线vol` | `amount / vol` |
|
|
||||||
|---|---|---|---|---|
|
|
||||||
| 股票 / ETF / LOF | 手(= 100 股) | 股 | **0.01** | 均价 ✓ |
|
|
||||||
| 可转债 | 手(= 10 张) | **手** | **1.0** | 均价 **×10** ⚠️ |
|
|
||||||
|
|
||||||
| 位置 | 变化 |
|
|
||||||
|---|---|
|
|
||||||
| `SecurityBar` / `TransactionRecord` / `MinuteBar` | 量纲注释补可转债(`vol`=手;日线 `amount/vol` 需再 `/10`) |
|
|
||||||
| `symbols.has_order_count` | 实测表补沪可转债 `110/111/113/118`、沪 EB `132`、深可转债 `123/127/128`,并说明它们走「排除法」兜底分支 |
|
|
||||||
| `docs/API_REFERENCE.md` | 新增**跨品种量纲对照表** |
|
|
||||||
| `docs/llms.txt` / `SKILL.md` | 同步同一口径 |
|
|
||||||
|
|
||||||
**代码逻辑未改**:可转债在 `has_order_count` 里本来就判为「有笔数」(不在基金/指数的
|
|
||||||
排除前缀里),本次只是把实测依据补进注释与文档。
|
|
||||||
|
|
||||||
## [0.6.6] - 2026-10-22
|
|
||||||
|
|
||||||
### 改进
|
|
||||||
|
|
||||||
- **价格小数位改用协议自带的权威值,前缀规则降为兜底**。
|
|
||||||
|
|
||||||
0.6.5 修好债券/回购标度时,采用的是「按代码前缀推断」+「用全量清单做离线校验」。
|
|
||||||
但同一份证券清单接口(`get_universe`,底层 `GetSecurityListCmd`)本来就**返回**
|
|
||||||
`SecurityInfo.decimal_point`(外加自描述浮点的 `pre_close`)——权威值一直就在手上,
|
|
||||||
只是没接进取数路径。本次把它接进来:
|
|
||||||
|
|
||||||
| 位置 | 说明 |
|
|
||||||
|---|---|
|
|
||||||
| `price_decimal_digits` | 查找顺序改为 **权威表 → 静态前缀规则**,未同步时行为与 0.6.5 完全一致 |
|
|
||||||
| `decimal_sync`(新模块) | 每日后台同步一次 + 磁盘缓存(`%LOCALAPPDATA%\pytdxdata\decimal-points.json`) |
|
|
||||||
| `TdxData.sync_decimal_points()` | 手动强制刷新(排障 / 立刻要权威表) |
|
|
||||||
| `TdxData.decimal_point_status()` | 状态:`source` / `stock` / `mixed_prefixes` / `syncing` |
|
|
||||||
|
|
||||||
**对用户仍无感**:`start()` 命中当日缓存就直接装表(实测 60 ms、零网络),
|
|
||||||
未命中则挂后台 task(实测首次 50 s,含沪 27,520 + 深 24,462 条清单),
|
|
||||||
不阻塞 `start()`(`start()` 仍 1.5 s);`close()` 会取消未完成的同步。
|
|
||||||
|
|
||||||
**表结构刻意做小**:全量 51,982 只标的只产生 **179 个**「三位前缀 → 小数位」映射
|
|
||||||
(沪 96 + 深 83),内存里就是 179 条。若将来某段代码混用两种小数位,该前缀自动
|
|
||||||
退化为**逐代码表**,只存例外代码(`mixed_prefixes` 会 > 0,可据此发现),
|
|
||||||
不会静默算错。清单拉取不完整时表会偏小 —— 缺的部分退回静态规则,不影响正确性。
|
|
||||||
|
|
||||||
为什么保留前缀规则:`get_quotes` 解析是纯同步、无网络的路径,解析那一刻可能还没
|
|
||||||
同步过清单;离线 / 首次安装也必须能出价格。实测两者在全量清单上**零分歧**
|
|
||||||
(51,982 条逐条比对,不一致 0 条),并存不会造成口径冲突。
|
|
||||||
|
|
||||||
开关:`PYPYTDX_DATA_DISABLE_DECIMAL_SYNC=1` 关闭后台同步(CI / 测试用)。
|
|
||||||
|
|
||||||
### 说明
|
|
||||||
|
|
||||||
- 新增回归测试 16 项(权威表优先级 / 混合前缀退化 / 缓存跨日失效 / 损坏容错 /
|
|
||||||
后台任务取消),测试总数 **289 → 306**。
|
|
||||||
|
|
||||||
## [0.6.5] - 2026-10-22
|
|
||||||
|
|
||||||
### 新增
|
### 新增
|
||||||
|
- `get_index_members` / `get_index_trade_stats` / `get_index_trade_stats_multi`:指数成分股与逐笔聚合统计(分钟级增量;北交所成分无笔数自动剔除)
|
||||||
- **指数成分成交汇总:一条接口拿到「某指数下全部股票的成交量与成交笔数」**
|
- `sync_decimal_points` / `decimal_point_status`:权威价格小数位表运行时同步(报价标度不再依赖硬编码前缀规则),`start()` 自动装载
|
||||||
(`get_index_members` / `get_index_trade_stats` / `get_index_trade_stats_multi`)。
|
- `decimal_sync` 模块
|
||||||
|
|
||||||
起因是一个绕不开的协议事实:**指数本身没有成交笔数**。指数逐笔(`sh000001`)的
|
|
||||||
`trade_count` 字段实际是「当时均价 × 1000」(2026-10-21 已验证),`get_minutes` /
|
|
||||||
`get_market_stat` / `SecurityBar` / `get_quotes` 的 160 个字段位里也都**没有**笔数
|
|
||||||
(扫描 0x122C 全字段位确认,`0x8`/`0x9` 是外盘/内盘手数)。所以「指数笔数」唯一可行
|
|
||||||
的口径是**把成分股的逐笔笔数累加**。
|
|
||||||
|
|
||||||
```python
|
|
||||||
async with TdxData() as td:
|
|
||||||
st = await td.get_index_trade_stats("000300") # 沪深300,约 10 s
|
|
||||||
for hhmm, cum in st.cum_trade_count(): # 截止该分钟的累计笔数
|
|
||||||
print(hhmm, cum)
|
|
||||||
st.minutes[0] # MinuteTradeStat(time='09:25', volume=…, trade_count=…)
|
|
||||||
st.total_volume # 手
|
|
||||||
st.total_trade_count # 笔
|
|
||||||
```
|
|
||||||
|
|
||||||
| 方法 | 作用 |
|
|
||||||
|---|---|
|
|
||||||
| `get_index_members(index, *, fields=None, count=100000)` | 指数成分股(含实时报价)。裸 6 位按**指数语义**解析:`000001`=上证指数,不是平安银行 |
|
|
||||||
| `get_index_trade_stats(index, date=None, *, concurrency=24, batch_size=50, members=None, skip_errors=True, on_progress=None)` | 成分逐笔 → 每分钟成交量/笔数;`date` 支持历史 |
|
|
||||||
| `get_index_trade_stats_multi(index, dates, *, ...)` | 多日批量(成分只请求一次),用于「近 N 日同时点均值」基准 |
|
|
||||||
|
|
||||||
**口径校验(2026-10-22 实测)**:``total_amount`` 是**成分股**合计。对「成分 = 指数
|
|
||||||
覆盖面」的指数,它与指数快照 ``amount`` 完全相等 —— 说明成分清单与量能口径一致。
|
|
||||||
|
|
||||||
| 指数 | 成分 | 成分合计 | 指数快照 | 比值 | 耗时 |
|
|
||||||
|---|---|---|---|---|---|
|
|
||||||
| `sh000016` 上证50 | 50 | 661.38 亿 | 661.34 亿 | 1.0001 | 5.8 s |
|
|
||||||
| `sh000300` 沪深300 | 300 | 2373.98 亿 | 2373.98 亿 | 1.0000 | 9.7 s |
|
|
||||||
| `sh000001` 上证指数 | 2230 | 3907.00 亿 | 4111.56 亿 | 0.9503 | 35.6 s |
|
|
||||||
|
|
||||||
历史日期同样成立(2026-10-21 收盘数据):`sh000016` 1046.29 / 1045.59 亿(1.0007)、
|
|
||||||
`sh000300` 3881.65 / 3879.30 亿(1.0006)。
|
|
||||||
|
|
||||||
单只股票级别的口径更硬:`Σ(逐笔价 × 手 × 100)` 与日线 `amount` 比值
|
|
||||||
**1.0000~1.0004**(`sh600000` / `sz000001` / `sh601398` / `bj920002` / `bj920015`
|
|
||||||
全部吻合),`Σvol` 与快照 `vol` 比值 1.0000。
|
|
||||||
|
|
||||||
CLI 同步新增:`tdx index-stats 000300 --top 10`。
|
|
||||||
|
|
||||||
**三个必须知道的限制**:
|
|
||||||
|
|
||||||
1. **深市 `399xxx` 的指数成交额是「全板块」口径**,不是样本股 —— 实测
|
|
||||||
`sz399001` 快照 5081.40 亿 = 深市 2908 只股票合计 5081.01 亿,`sz399006`
|
|
||||||
2568.60 亿 = 创业板全部股票合计。所以拿它跟 `total_amount` 比会得到 0.43 这种
|
|
||||||
无意义比值,不是数据错。**`get_index_trade_stats` 返回的始终是成分口径。**
|
|
||||||
2. **北交所(`899xxx`)没有笔数字段** —— 逐笔 `trade_count` 恒为 0,此时
|
|
||||||
`trade_count_available` 为 `False`,只有成交量可用。
|
|
||||||
3. **成分表比全市场滞后** —— 上证指数返回 2230 只,而 `get_universe("sh")` 里
|
|
||||||
股票段共 2360 只,差 130 只(新上市尚未纳入服务端板块),这正是上表
|
|
||||||
0.9503 的来源;`sh000016`/`sh000300` 这类固定样本指数不受影响。
|
|
||||||
需要精确覆盖时可自行传 `members=[...]`。
|
|
||||||
|
|
||||||
### 修复
|
### 修复
|
||||||
|
- **指数/ETF/LOF 逐笔的 `trade_count` 语义修正**:协议字段在指数上是「均价×1000」、ETF/LOF 上是「价格×1000」,**不是笔数**;新增 `TransactionRecord.trade_count_available`(判定见 `symbols.has_order_count`),CLI 对无笔数口径标的显示 `(-)` 而非假笔数
|
||||||
- **债券与回购价格全线偏差:沪市少 100 倍、深市少 10 倍**(`get_quotes`)。
|
- 集合竞价委托(bs_flag=8)不再混入逐笔结果(会被覆盖、Σvol 虚高 20%~160%、不同主机条数不一致)
|
||||||
|
- 缓存管理器 / 连接池 / 分页调度若干健壮性修复
|
||||||
`price_decimal_digits` 只覆盖了「股票 2 位 / 基金 3 位」两档,债券与回购没有对应
|
|
||||||
分支,落到默认档位。它们实际是 **4 位小数**,于是被读大了两个数量级:
|
|
||||||
|
|
||||||
| 标的 | 真实价 | 修复前 | 偏差 |
|
|
||||||
|---|---|---|---|
|
|
||||||
| `sh019742` 国债 24特国01 | 110.2280 | 11022.8000 | ×100 |
|
|
||||||
| `sh110077` 可转债 洪城转债 | 199.2790 | 19927.9000 | ×100 |
|
|
||||||
| `sh204001` 回购 GC001 | 1.3850 | 138.5000 | ×100 |
|
|
||||||
| `sh900901` B股 云赛B股 | 0.5670 | 5.6700 | ×10 |
|
|
||||||
| `sz123107` 可转债 温氏转债 | 112.1890 | 1121.8900 | ×10 |
|
|
||||||
| `sz131810` 回购 R-001 | 1.3800 | 13.8000 | ×10 |
|
|
||||||
|
|
||||||
沪市 `010/018/019/020/1xx/2xx` 与深市 `10x-14x/19x/50x/52x/56x` 全部改为 4 位;
|
|
||||||
沪市 B 股 `900xxx` 另改为 3 位(与深市 `200xxx` 的 2 位不同,两市不一致)。
|
|
||||||
股票 / ETF / 指数 / 板块的行为**不变**。
|
|
||||||
|
|
||||||
**验证方式(可作为该函数的验收标准)**:拿 `pre_close`(五档快照口径,走
|
|
||||||
`price_decimal_digits`)除以上一交易日**日线收盘**(K线口径,走独立的 `/1000`
|
|
||||||
路径),比值 1.0 即两口径一致。修复后 15 类标的全数为 1.0000,零偏差。
|
|
||||||
|
|
||||||
**权威来源已确认**:证券列表里协议**自带** `SecurityInfo.decimal_point` 字段。
|
|
||||||
2026-10-22 拉取全量 **51,982 只**(沪 27,520 + 深 24,462)逐条比对,本函数与
|
|
||||||
协议字段**完全一致、零例外**——因为全量清单里没有任何一个「三位前缀」出现混合
|
|
||||||
小数位。`get_quotes` 解析时拿不到证券清单,故仍用前缀推断;需要权威值时读
|
|
||||||
`SecurityInfo.decimal_point`。
|
|
||||||
|
|
||||||
- **美股(EX 通道)分时的 `hour` 是非法时钟值**(`get_minutes("usAAPL")`)。
|
|
||||||
|
|
||||||
EX 分时的 `minutes` 字段是**自开盘起的连续分钟数**,不是「当天的分钟数」。
|
|
||||||
旧代码直接 `minutes // 60`,于是 390 条里 **240 条** 的 `hour` ≥ 24:
|
|
||||||
|
|
||||||
```
|
|
||||||
修复前: 21:30, 21:31, ..., 23:59, 24:00, 24:01, ..., 27:59 ← hour=27 不存在
|
|
||||||
修复后: 21:30, 21:31, ..., 23:59, 00:00, 00:01, ..., 03:59 ← 真实北京时间
|
|
||||||
```
|
|
||||||
|
|
||||||
任何 `datetime(...)` 拼接都会因 `hour=27` 抛 `ValueError`,按小时聚合也会把
|
|
||||||
凌晨段错误地归到同一个自然日。
|
|
||||||
|
|
||||||
**修法**:`MinuteBar.__post_init__` 把越界的 `hour` 拆成
|
|
||||||
「`hour %= 24` + `day_offset += hour // 24`」,并新增两个只读属性:
|
|
||||||
|
|
||||||
| 属性 | 用途 |
|
|
||||||
|---|---|
|
|
||||||
| `MinuteBar.datetime` | 该分时点的时刻(北京时间,已含跨日偏移) |
|
|
||||||
| `MinuteBar.trading_date` | 归属的交易日 `YYYY-MM-DD`。美股一个交易日横跨两个北京自然日(21:30~次日 04:00),**按交易日聚合必须用它**,与 `SecurityBar.trading_date` 口径一致 |
|
|
||||||
|
|
||||||
A股(合成时间 09:30~15:00)/ 港股(09:30~16:00)/ 期货的 `hour` 本就合法
|
|
||||||
→ **零行为变化**,`day_offset` 恒为 0。
|
|
||||||
|
|
||||||
实测(2026-10-22):`usAAPL` / `usTSLA` / `usNVDA` 各 390 条,修复后
|
|
||||||
`hour` 全在 0~23、时间轴严格单调、`trading_date` 收敛为单一交易日;
|
|
||||||
`hk00700` 90 条无任何变化。
|
|
||||||
|
|
||||||
### 说明
|
### 说明
|
||||||
|
- 0.6.4~0.6.6 为中间迭代,未单独发版记录;本次合并 PyPI sdist 与仓库,源码恢复双仓一致
|
||||||
|
|
||||||
- 新增回归测试:债券/回购/B股小数位 ×29、美股分时跨日 ×6、指数成分成交汇总 ×34、
|
|
||||||
全接口 ×1,测试总数 **217 → 289**。
|
|
||||||
|
|
||||||
## [0.6.4] - 2026-10-22
|
|
||||||
|
|
||||||
### 修复
|
|
||||||
|
|
||||||
- **`get_ticks()` 返回列表不是时间序,每 `page_size` 处时间回跳一次**。
|
|
||||||
|
|
||||||
`0x122F`(A股逐笔)与 EX 逐笔通道的偏移量 `start` 都是**从最新一笔往回数**的:
|
|
||||||
实测 `sh600000`,`start=0` 返回 `14:07:48..15:20:16`(当天最新 1000 条),
|
|
||||||
`start=4000` 返回 `09:15:03..09:41:18`(当天最早)。而 `paginator` 按 `start`
|
|
||||||
升序拼接,于是拼出"新的块在前、旧的块在后"的倒序结构:
|
|
||||||
|
|
||||||
| | 修复前 | 修复后 |
|
|
||||||
|---|---|---|
|
|
||||||
| `sh600000` 时间轴 | 4 处逆序跳跃(index 999 / 1999 / 2999) | 严格升序 |
|
|
||||||
| `hk00700` 时间轴 | 21 处逆序跳跃(每 1800 条一次) | 严格升序 |
|
|
||||||
|
|
||||||
修复分两层:① `PageSpec.newest_first=True` 让 `paginator` 反页序拼装(页内本身升序,
|
|
||||||
只翻页序),这条能力保留给将来需要按"最新优先"锚点分页的命令;② A股逐笔最终改为
|
|
||||||
**单连接串行游标**(见下一条),从根上避开多主机锚点错位。
|
|
||||||
|
|
||||||
**`start` 参数语义同时明确**:它是"从最新往回数的偏移",不是"从最早往后数的偏移",
|
|
||||||
`docstring` 已补充。注意这修正的是**顺序**,数据本身无缺口、无重复(4304 条唯一时间戳)。
|
|
||||||
|
|
||||||
- **`TransactionRecord.trade_count` 在集合竞价段被读成 42 亿量级脏值**。
|
|
||||||
|
|
||||||
该字段在协议里是**有符号** `int32`,但解析格式串写的是无符号 `I`。集合竞价段
|
|
||||||
(`bs_flag=8`)真实值为负数,按 uint32 解出 `4294966422`:
|
|
||||||
|
|
||||||
```
|
|
||||||
原始 18 字节: 17820000 0ad71b41 d9000000 96fcffff 0800
|
|
||||||
修复前: trade_count = 4294966422
|
|
||||||
修复后: trade_count = -874 ← 0xfffffc96 的 int32 补码
|
|
||||||
```
|
|
||||||
|
|
||||||
实测 `sh600000` 全天 4304 条中有 **38 条**命中(占 0.88%),全部落在 09:15~09:25
|
|
||||||
集合竞价段;正常交易段(`bs_flag` 0/1/2/5)**无一条为负**。旧行为会让
|
|
||||||
`max()` / `mean()` 等统计直接失真(曾算出 `mean=37,920,247`)。
|
|
||||||
|
|
||||||
格式串由 `<IfIIH` 改为 `<IfIiH`。**注意负值语义尚未确证**(可能是撤单量/净委托),
|
|
||||||
目前只是如实保留符号,未做业务解释。
|
|
||||||
|
|
||||||
- **`get_quotes()` 沪市指数价格整体少 10 倍**。
|
|
||||||
|
|
||||||
`price_decimal_digits` 把沪市 `000`(如 000001/000016/000300/000688)与 `8`
|
|
||||||
开头(如板块指数 880001/880305)一律判成 3 位小数(除 1000),但协议原始整数
|
|
||||||
实际是 **2 位**(除 100)→ 上证指数 3713 被读成 371、上证50 2747 被读成 274。
|
|
||||||
|
|
||||||
实证(2026-10-21):沪市指数 6 只 + 板块 2 只全部 `分时中位价 / 快照价 = 10.0`,
|
|
||||||
而沪市 ETF(510300/588000/511010/501018)、股票(60/601)、科创板(688)、
|
|
||||||
深市指数/股票/创业板/ETF 全部 `= 1.0` 正确。修法是**把沪市 `000`/`8` 从 3 位改回 2 位**,
|
|
||||||
其余前缀(`5` ETF/基金、深市 `1` 基金转债)保持 3 位不变。回归测试覆盖 21 组代码。
|
|
||||||
|
|
||||||
- **`get_ticks()` 把集合竞价**委托**(`bs_flag=8`)当成成交计入,导致 Σvol 虚高且跨主机不稳定**。
|
|
||||||
|
|
||||||
`0x122F` 第 5 字段 `bs_flag=8` 的记录是 **09:15~09:25 / 14:57~15:00 竞价段的撮合委托
|
|
||||||
快照**,`vol` 是**累计值**(可被同段后续记录覆盖),根本不是成交增量。旧代码把它一并
|
|
||||||
返回,两个后果:
|
|
||||||
|
|
||||||
1. **Σvol 比真值虚高**:`sh600000` 全天 Σvol 含 b8 时 = 668,693,快照 vol = 462,562,
|
|
||||||
比值 1.4456;剔掉 b8 后 Σvol = 462,634,比值 **1.000156**(与快照精确一致)。
|
|
||||||
2. **跨主机不稳定**:干净主机不收录 b8(0 条)、脏主机收录 135 条。并发分页时各页
|
|
||||||
来自不同主机,同一 `start`(从最新往回数的相对锚点)在不同主机落到**不同的真实成交**
|
|
||||||
上 → 页边界既**重叠**(sh600000 冗余 116 条)又**漏条**(13:03 附近丢 59 条),
|
|
||||||
同一查询两次调用结果不同。
|
|
||||||
|
|
||||||
修复:① 分页拼装后**统一剔除 `bs_flag=8`**;② A股逐笔改用**单连接串行游标翻页**
|
|
||||||
(`_ticks_cn_serial`),彻底消除跨主机锚点错位。修复后单主机 1.0002、剔 b8 后跨主机
|
|
||||||
公共时间窗**逐条一致**。
|
|
||||||
|
|
||||||
- **文档化指数逐笔字段的量纲差异**(非 bug,是数据源固有语义)。
|
|
||||||
|
|
||||||
指数(如 `sh000016`)的 `vol` 单位是**百元**(成交额/100),`trade_count` 是**均价×1000**
|
|
||||||
(不是笔数),与个股完全不同。实测 `快照amount / Σtick.vol = 100.0002`、`tc/1000` 与分时
|
|
||||||
`avg_price` 差 < 1.1。已写进 `TransactionRecord` 类注释与 `get_ticks` docstring,
|
|
||||||
避免用户把指数 Σvol 直接和成分股成交量比(会误得 ~26 倍)。
|
|
||||||
|
|
||||||
### 说明
|
|
||||||
|
|
||||||
- 新增回归测试:价格小数位 ×20(19 组参数化 + 1 组去 NUL)、`get_ticks` 剔 b8 /
|
|
||||||
单连接串行 / 多页无重叠 ×3,测试总数 **194 → 217**。
|
|
||||||
|
|
||||||
## [0.6.3] - 2026-10-18
|
|
||||||
|
|
||||||
- fix(pool): ExPool acquire 背压等待(Condition 排队,默认 30s 超时)替代 3x0.05s 快速失败;超时异常带池状态
|
- fix(pool): ExPool acquire 背压等待(Condition 排队,默认 30s 超时)替代 3x0.05s 快速失败;超时异常带池状态
|
||||||
- feat(api): 新增 ex_pool_cap 参数(默认 2),单 EX 服务器场景不再静默退化为 1 并发(fixes #1)
|
- feat(api): 新增 ex_pool_cap 参数(默认 2),单 EX 服务器场景不再静默退化为 1 并发(fixes #1)
|
||||||
|
|||||||
@@ -47,17 +47,14 @@ async def main():
|
|||||||
asyncio.run(main())
|
asyncio.run(main())
|
||||||
```
|
```
|
||||||
|
|
||||||
默认返回的是标准库 `dataclass` 组成的 `list`(核心库不依赖 pandas)。需要 DataFrame 就换个后端构造:
|
返回的是标准库 `dataclass` 组成的 `list`(库本身不依赖 pandas)。要 DataFrame 一行转:
|
||||||
|
|
||||||
```python
|
```python
|
||||||
async with TdxData(df="polars") as td: # 或 df="pandas";需 pip install "pytdxdata[polars]"
|
from dataclasses import asdict
|
||||||
df = await td.get_bars("sh600000", KlinePeriod.DAY, count=240)
|
import pandas as pd
|
||||||
# 列:market code open high low close vol amount datetime trading_date symbol settle
|
df = pd.DataFrame([asdict(b) for b in bars])
|
||||||
```
|
```
|
||||||
|
|
||||||
时间字段会被**合并成一列 `datetime`**(不再有 year/month/day/hour/minute),五档报价展开成
|
|
||||||
`bid1_price`…`ask5_vol`,详见 [DataFrame 输出](https://openbot-coder.github.io/pytdxdata/dataframe/)。
|
|
||||||
|
|
||||||
## 特性
|
## 特性
|
||||||
|
|
||||||
| 特性 | 说明 |
|
| 特性 | 说明 |
|
||||||
@@ -66,8 +63,7 @@ async with TdxData(df="polars") as td: # 或 df="pandas";需 pip install
|
|||||||
| 🗄️ 双层 TTL 缓存 | 内存 LRU + 磁盘 SQLite(WAL),TTL 分级;single-flight 并发同键去重;缓存**已解析记录**(重复请求 0 网络 0 解析) |
|
| 🗄️ 双层 TTL 缓存 | 内存 LRU + 磁盘 SQLite(WAL),TTL 分级;single-flight 并发同键去重;缓存**已解析记录**(重复请求 0 网络 0 解析) |
|
||||||
| ⚡ 并发分页 | 单请求拆 N 页(显式 start 偏移 → 无状态可并行)并行拉取、按序拼接、失败换连接重试 |
|
| ⚡ 并发分页 | 单请求拆 N 页(显式 start 偏移 → 无状态可并行)并行拉取、按序拼接、失败换连接重试 |
|
||||||
| 🔀 双通道整合 | 标准通道(经典行情)+ MAC 通道(逐笔 / 复权 / 1 分钟 K 线),按命令自动选路 |
|
| 🔀 双通道整合 | 标准通道(经典行情)+ MAC 通道(逐笔 / 复权 / 1 分钟 K 线),按命令自动选路 |
|
||||||
| 📦 零重型依赖 | 纯 asyncio + stdlib(socket / zlib / struct / sqlite3),`dependencies` 为空 |
|
| 📦 零重型依赖 | 纯 asyncio + stdlib(socket / zlib / struct / sqlite3) |
|
||||||
| 🐼 DataFrame 直出 | `TdxData(df="polars")` / `df="pandas"` 让 42 个表格型接口直接返回 DataFrame;时间列自动合并成 `datetime`,五档报价自动展开。后端是可选 extras,核心包仍零依赖 |
|
|
||||||
| 📊 全市场覆盖 | A股、港股、美股、期货、期权(46 个扩展市场) |
|
| 📊 全市场覆盖 | A股、港股、美股、期货、期权(46 个扩展市场) |
|
||||||
| 📄 服务器文件解析 | 板块 `block_*.dat` / 行业 `tdxhy.cfg` / 财报 `gpcw*` **下载即解析**成 dataclass(索引名与服务器实际文件名不一致时自动纠正) |
|
| 📄 服务器文件解析 | 板块 `block_*.dat` / 行业 `tdxhy.cfg` / 财报 `gpcw*` **下载即解析**成 dataclass(索引名与服务器实际文件名不一致时自动纠正) |
|
||||||
| 🤖 AI 助手友好 | 内置 [`SKILL.md`](https://github.com/openbot-coder/pytdxdata/blob/main/SKILL.md) 与 [`llms.txt`](https://openbot-coder.github.io/pytdxdata/llms.txt) |
|
| 🤖 AI 助手友好 | 内置 [`SKILL.md`](https://github.com/openbot-coder/pytdxdata/blob/main/SKILL.md) 与 [`llms.txt`](https://openbot-coder.github.io/pytdxdata/llms.txt) |
|
||||||
@@ -118,10 +114,6 @@ uv run mkdocs serve # 本地预览文档站
|
|||||||
- 行情来自通达信公开服务器,**15 分钟延时**;需实时数据请使用券商行情
|
- 行情来自通达信公开服务器,**15 分钟延时**;需实时数据请使用券商行情
|
||||||
- 标准通道 K 线 `count < 100` 部分服务器返回空 → 分页器已强制 `min_page=100` 规避
|
- 标准通道 K 线 `count < 100` 部分服务器返回空 → 分页器已强制 `min_page=100` 规避
|
||||||
- 港股 / 美股期权在通达信协议内无数据
|
- 港股 / 美股期权在通达信协议内无数据
|
||||||
- **EX 通道(港股 / 美股 / 期货 / 期权)的 K 线没有统一成交额**:`0x23FF` 帧内只有一个可复用槽位,
|
|
||||||
权益类市场(港股主板/基金/创业板、新三板、美股)该槽位是成交额,已回填;**期货 / 期权该槽位是持仓量,
|
|
||||||
所以它们的 `SecurityBar.amount` 恒为 `0.0`**(协议限制,不是 bug)。期货/期权的结算价在 `SecurityBar.settle`。
|
|
||||||
⚠️ 港股主板/基金的 `vol` 单位是「手」(=100 股),`amount/vol` 是均价的 100 倍
|
|
||||||
- **K 线时间戳统一为北京时间**(v0.6.2 起)。美股原始报文把「美东交易日日期」与「北京时间时分」
|
- **K 线时间戳统一为北京时间**(v0.6.2 起)。美股原始报文把「美东交易日日期」与「北京时间时分」
|
||||||
拼在一起,库已自动归一化,`bar.datetime` 可直接排序 / 过滤;但**美股一个交易日横跨两个北京
|
拼在一起,库已自动归一化,`bar.datetime` 可直接排序 / 过滤;但**美股一个交易日横跨两个北京
|
||||||
自然日**,按交易日聚合请用 `SecurityBar.trading_date`,不要用 `datetime.date()`
|
自然日**,按交易日聚合请用 `SecurityBar.trading_date`,不要用 `datetime.date()`
|
||||||
|
|||||||
@@ -52,7 +52,6 @@ df = pd.DataFrame([asdict(b) for b in bars])
|
|||||||
|
|
||||||
```bash
|
```bash
|
||||||
pip install pytdxdata # 或 uv add pytdxdata
|
pip install pytdxdata # 或 uv add pytdxdata
|
||||||
pip install "pytdxdata[polars]" # 需要 DataFrame 输出时才装(可选 extras)
|
|
||||||
```
|
```
|
||||||
|
|
||||||
## 可运行模板
|
## 可运行模板
|
||||||
@@ -83,20 +82,12 @@ async def main():
|
|||||||
# 分时(MAC 通道,今日或历史)
|
# 分时(MAC 通道,今日或历史)
|
||||||
minutes = await td.get_minutes(["sz000001"])
|
minutes = await td.get_minutes(["sz000001"])
|
||||||
|
|
||||||
# 批量 K 线:≥16 只自动走「多连接 × 单连接流水线」,池子把标的摊到多台服务器
|
# 批量 K 线:多股票自动并发,池子会分配到不同服务器
|
||||||
batch = await td.get_bars(
|
batch = await td.get_bars(
|
||||||
["sz000001", "sh600000"],
|
["sz000001", "sh600000"],
|
||||||
KlinePeriod.DAY, count=240,
|
KlinePeriod.DAY, count=240,
|
||||||
) # 平铺返回,每条带 .symbol 字段
|
) # 平铺返回,每条带 .symbol 字段
|
||||||
|
|
||||||
# 全市场盘中实时:只拉已经走完的根数(10:00 → 30 根),5,200 只 1~2 秒
|
|
||||||
from datetime import datetime
|
|
||||||
live = await td.get_bars(
|
|
||||||
[s.symbol for s in await td.get_universe("sh")],
|
|
||||||
KlinePeriod.MIN_1, start_date=datetime.now(),
|
|
||||||
)
|
|
||||||
# pipeline=1 可关闭批量路径,退回逐只并发(结果一致,只是慢)
|
|
||||||
|
|
||||||
asyncio.run(main())
|
asyncio.run(main())
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -148,19 +139,16 @@ await td.get_bars("cffex:IFL0", KlinePeriod.DAY, count=700) # 期货合约
|
|||||||
|
|
||||||
| 需求 | 方法 |
|
| 需求 | 方法 |
|
||||||
|---|---|
|
|---|---|
|
||||||
| 全市场标的清单(A股+港股+期货…;**北交所返回空**) | `get_universe(market=None)` |
|
| 全市场标的清单(A股+港股+期货…) | `get_universe(market=None)` |
|
||||||
| 日/周/月/分钟 K 线(任意市场) | `get_bars(symbols, period, *, count, adjust, start_date, end_date)` |
|
| 日/周/月/分钟 K 线(任意市场) | `get_bars(symbols, period, *, count, adjust, start_date, end_date)` |
|
||||||
| K 线 + MACD/KDJ/RSI/BOLL | `get_indicators(symbols, indicators, ...)` |
|
| K 线 + MACD/KDJ/RSI/BOLL | `get_indicators(symbols, indicators, ...)` |
|
||||||
| 实时五档报价(A股)/ 基础报价(港股/美股/期货) | `get_quotes(symbols, *, fields=None, pipeline=True)` |
|
| 实时五档报价(A股)/ 基础报价(港股/美股/期货) | `get_quotes(symbols, *, fields=None)` |
|
||||||
| 逐笔成交(任意市场) | `get_ticks(symbols, *, date)` |
|
| 逐笔成交(任意市场) | `get_ticks(symbols, *, date)` |
|
||||||
| 分时图 / 缩略采样 | `get_minutes(symbols, *, date, sampling=True)` |
|
| 分时图 / 缩略采样 | `get_minutes(symbols, *, date, sampling=True)` |
|
||||||
| 全市场股票列表(缓存 1 天) | `get_universe("sz")` / `get_universe("sh")` |
|
| 全市场股票列表(缓存 1 天) | `get_universe("sz")` / `get_universe("sh")` |
|
||||||
| ~~北交所全量代码~~ | 拿不到:清单命令对 market=2 无响应(27 台全测)。北交所标的仍可直接查 `bj920992` |
|
|
||||||
| 除权除息 / 财务快照 | `get_xdxr(symbols)` / `get_finance(symbols)` |
|
| 除权除息 / 财务快照 | `get_xdxr(symbols)` / `get_finance(symbols)` |
|
||||||
| F10 公司资料 | `get_f10(symbol, section)` |
|
| F10 公司资料 | `get_f10(symbol, section)` |
|
||||||
| 板块列表 / 成分股 / 涨跌排行 | `get_boards` / `get_board_members` / `get_board_ranking` |
|
| 板块列表 / 成分股 / 涨跌排行 | `get_boards` / `get_board_members` / `get_board_ranking` |
|
||||||
| **指数的成交量 + 成交笔数**(每分钟曲线) | `get_index_trade_stats(index, date)` → `.cum_trade_count()` |
|
|
||||||
| **指数成分股** | `get_index_members(index)`(裸 `000001` 按指数语义解析) |
|
|
||||||
| 个股所属板块 | `get_board_of(symbols)` |
|
| 个股所属板块 | `get_board_of(symbols)` |
|
||||||
| 集合竞价 / 异动 / 资金流 | `get_auction` / `get_unusual` / `get_capital_flow` |
|
| 集合竞价 / 异动 / 资金流 | `get_auction` / `get_unusual` / `get_capital_flow` |
|
||||||
| 个股特征快照 | `get_snapshot(symbols)` |
|
| 个股特征快照 | `get_snapshot(symbols)` |
|
||||||
@@ -179,40 +167,9 @@ await td.get_bars("cffex:IFL0", KlinePeriod.DAY, count=700) # 期货合约
|
|||||||
- **K 线时间戳统一为北京时间**(v0.6.2 起)。美股原始报文里日期是美东交易日、时分是北京时间(两个时区拼在一起),库已自动归一化,`bar.datetime` 可直接排序 / 比较 / 过滤。但**美股一个交易日横跨两个北京自然日**,按交易日聚合(分钟线合成日线、日内统计)必须用 `bar.trading_date`。
|
- **K 线时间戳统一为北京时间**(v0.6.2 起)。美股原始报文里日期是美东交易日、时分是北京时间(两个时区拼在一起),库已自动归一化,`bar.datetime` 可直接排序 / 比较 / 过滤。但**美股一个交易日横跨两个北京自然日**,按交易日聚合(分钟线合成日线、日内统计)必须用 `bar.trading_date`。
|
||||||
- **`count` 很大时会自动多连接并发分页**,无需手动翻页。但标准通道 K 线 `count < 100` 部分服务器会返回空——内部已强制 `min_page=100` 规避。
|
- **`count` 很大时会自动多连接并发分页**,无需手动翻页。但标准通道 K 线 `count < 100` 部分服务器会返回空——内部已强制 `min_page=100` 规避。
|
||||||
- **`get_quotes` 一次 >80 只自动切批**;走 MAC 字段位通道(`fields=`)时上限为 80 只/次。
|
- **`get_quotes` 一次 >80 只自动切批**;走 MAC 字段位通道(`fields=`)时上限为 80 只/次。
|
||||||
- **全市场报价用默认参数即可,别自己写循环**。`get_quotes(全部A股)` 内部会切 66 片并走**单连接流水线**(一条连接连发多个分片、只付一次往返)+ 工作窃取调度,5,227 只实测 **p50 325ms**(老版本串行 3.5s)。自己 for 循环逐只/逐片调会把流水线优势全部丢掉。
|
- **历史逐笔没有笔数字段**,只有当日逐笔的 `trade_count` 有效。
|
||||||
- 想更保守可 `concurrency=11`(每台 1 连接),代价 p50 457ms。
|
|
||||||
- 个别服务器不支持流水线时自动降级逐片,结果不受影响。
|
|
||||||
- **可转债(110/111/113/118/123/127/128)三样都是真的**:价格、`vol`(手=10 张)、`trade_count`(真实笔数,3631 条切片 / 20443 笔量级)全部可用,`Σvol` 与日线 `vol` **1:1 相等**。
|
|
||||||
⚠️ 唯一要小心的是 **日线 `amount / vol` 不等于均价,而是均价的 10 倍**(`vol` 单位是手=10 张)。算可转债均价用 `amount / vol / 10`,或直接用 `Σ(price×vol) / Σvol`——后者跨品种都成立,最稳。
|
|
||||||
对照:股票/ETF 的 `Σ逐笔vol / 日线vol = 0.01`(逐笔手、日线股),而 `amount/vol` 恰好就是均价。
|
|
||||||
- **指数本身没有成交笔数**。`get_ticks("sh000001")` 的 `trade_count` 是「当时均价×1000」、`vol` 单位是百元(成交额/100)。要笔数只能用 `get_index_trade_stats(index)`(成分股逐笔累加,2230 只约 36-60 s)。北交所成分无笔数字段,该方法会返回 `trade_count_available=False`。
|
|
||||||
- **ETF / LOF 也没有成交笔数**。`get_ticks("sh510310")` 的 `trade_count` 是「价格×1000」(货币 ETF 恒为 0)。判定用 `symbols.has_order_count(market, code)`,读记录时看 `TransactionRecord.trade_count_available`——为 `False` 时 `trade_count` 已被置 0,不要求和/求均值。注意**沪市 `501x/502x/506x` 实测有真实笔数**,不能按「5 开头=基金」一刀切。
|
|
||||||
- **历史逐笔有笔数**(MAC `0x122F` 支持历史日期,实测可回溯最近 5 个交易日;更早返回空)。
|
|
||||||
- **小数位表不用手动管,但可以手动刷**。`start()` 会检查当日缓存并后台同步一次(`SecurityInfo.decimal_point` 权威值);要强制刷新用 `await td.sync_decimal_points()`,看同步状态用 `await td.decimal_point_status()`。`PYPYTDX_DATA_DISABLE_DECIMAL_SYNC=1` 可关闭后台同步。
|
|
||||||
- 板块文件(`block_*.dat`)/ 行业(`tdxhy.cfg`)/ 财报(`gpcw*.zip`)是**服务器上的静态文件**,解析后是普通 dataclass,**落 JSON 后完全脱网可用**——适合做离线快照。
|
- 板块文件(`block_*.dat`)/ 行业(`tdxhy.cfg`)/ 财报(`gpcw*.zip`)是**服务器上的静态文件**,解析后是普通 dataclass,**落 JSON 后完全脱网可用**——适合做离线快照。
|
||||||
|
|
||||||
## DataFrame 输出(v0.7+)
|
|
||||||
|
|
||||||
表格型方法可以直出 DataFrame,**唯一开关是构造参数 `df`**,不要在调用处传 `df=True`(没有这个参数):
|
|
||||||
|
|
||||||
```python
|
|
||||||
async with TdxData(df="polars") as td: # 或 df="pandas"
|
|
||||||
df = await td.get_bars("sh600000", KlinePeriod.DAY, count=240)
|
|
||||||
```
|
|
||||||
|
|
||||||
| 事项 | 说明 |
|
|
||||||
|---|---|
|
|
||||||
| 安装 | `pip install "pytdxdata[polars]"` / `[pandas]` / `[dataframe]`;核心包仍零依赖 |
|
|
||||||
| `df=None`(默认) | 行为与 0.6.x 完全一致,返回 `list[dataclass]` |
|
|
||||||
| 报错 | `df` 取值非法 → `ValueError`;包没装 → `FrameBackendError`(启动即失败,不拖到第一次查询) |
|
|
||||||
| 时间列 | `year/month/day/hour/minute` **被 `datetime` 替换**,另带 `trading_date`;`XdxrRecord` 给 `date` |
|
|
||||||
| 嵌套 | `bid`/`ask` 展开成 `bid1_price`…`ask5_vol` 共 10 列;`fields` 字典 JSON 化成字符串列;`codes` 保持列表 |
|
|
||||||
| 特殊容器 | `get_indicators` → bars+指标合并;`get_index_trade_stats(_multi)` → `minutes` 展开 + `index`/`date` 列;`get_kline_batch` / `get_minute_batch` → 纵向拼接成一张表 |
|
|
||||||
| 不受影响 | `bytes` / `int` / `dict` / 单对象 / `tuple` / `list[float]` 类返回原样(`get_file`、`get_security_count`、`get_server_info`、`get_price_limits` 等) |
|
|
||||||
| 空结果 | 列名齐全的空表;dtype 只由后端推断 |
|
|
||||||
|
|
||||||
完整口径见 [DataFrame 输出](https://openbot-coder.github.io/pytdxdata/dataframe/)。
|
|
||||||
|
|
||||||
## 陷阱清单
|
## 陷阱清单
|
||||||
|
|
||||||
| 坑 | 后果 | 正确做法 |
|
| 坑 | 后果 | 正确做法 |
|
||||||
@@ -228,11 +185,6 @@ async with TdxData(df="polars") as td: # 或 df="pandas"
|
|||||||
| 循环里逐个 `await get_bars` | 慢 | 用列表传多个标的,自动并发 |
|
| 循环里逐个 `await get_bars` | 慢 | 用列表传多个标的,自动并发 |
|
||||||
| 港股/美股五档 bid/ask | 没有(协议不支持) | `fields` 字段位通道有 name 但无五档 |
|
| 港股/美股五档 bid/ask | 没有(协议不支持) | `fields` 字段位通道有 name 但无五档 |
|
||||||
| 美股分钟线用 `datetime.date()` 分组做日聚合 | 一个交易日被拆成两天(美股跨北京零点) | 用 `bar.trading_date`;`bar.datetime` 是归一化后的北京时间 |
|
| 美股分钟线用 `datetime.date()` 分组做日聚合 | 一个交易日被拆成两天(美股跨北京零点) | 用 `bar.trading_date`;`bar.datetime` 是归一化后的北京时间 |
|
||||||
| 拿指数逐笔的 `trade_count` 当笔数 | 拿到的是「均价×1000」,数值荒谬 | 用 `get_index_trade_stats(index)` 由成分累加 |
|
|
||||||
| 拿 ETF/LOF 逐笔的 `trade_count` 当笔数 | 拿到的是「价格×1000」(`sh510310` 显示 3994 笔) | 看 `trade_count_available`,或 `symbols.has_order_count()` |
|
|
||||||
| 拿深市 `399xxx` 的 `amount` 校验成分合计 | 比值 0.43,误以为数据错 | 深市指数 amount 是**全板块**口径(`sz399001`=深市全部 5081 亿),不是样本股 |
|
|
||||||
| 在期货/期权 K 线上取 `amount` | 恒 0.0,误以为解析坏了 | EX `0x23FF` 帧内**没有成交额**(协议限制);结算价看 `bar.settle`,成交额改用 `get_quotes` |
|
|
||||||
| 用港股主板的 `amount / vol` 当均价 | 差 **100 倍** | 港股主板/基金的 `vol` 单位是**手(=100 股)**,要 `amount/vol/100`;港股创业板/美股才是股 |
|
|
||||||
| 依赖实时性做信号 | 数据是 15 分钟前的 | 换券商实时源 |
|
| 依赖实时性做信号 | 数据是 15 分钟前的 | 换券商实时源 |
|
||||||
| 手动改包内 `servers.json` / 写死 IP | 容器只读、升级被覆盖、快节点捞不回来 | 日常靠内置的**当日测速缓存**自动选路;要强制指定就用 `PYPYTDX_DATA_{STANDARD,MAC,EX}_HOSTS` 环境变量 |
|
| 手动改包内 `servers.json` / 写死 IP | 容器只读、升级被覆盖、快节点捞不回来 | 日常靠内置的**当日测速缓存**自动选路;要强制指定就用 `PYPYTDX_DATA_{STANDARD,MAC,EX}_HOSTS` 环境变量 |
|
||||||
|
|
||||||
|
|||||||
+27
-56
@@ -1,6 +1,6 @@
|
|||||||
# pytdxdata 对外接口清单
|
# pytdxdata 对外接口清单
|
||||||
|
|
||||||
> 以 `src/pytdxdata` 源码为准整理(v0.7.0)。分四类:Python API / 数据模型 / 辅助模块 / CLI。
|
> 以 `src/pytdxdata` 源码为准整理(v0.6.0)。分四类:Python API / 数据模型 / 辅助模块 / CLI。
|
||||||
>
|
>
|
||||||
> 方法与代码的一致性由 `scripts/check_docs.py` 在 CI 中校验。
|
> 方法与代码的一致性由 `scripts/check_docs.py` 在 CI 中校验。
|
||||||
|
|
||||||
@@ -28,9 +28,8 @@ from pytdxdata import (
|
|||||||
| `cache_dir` | None | 磁盘缓存目录;None=仅内存缓存 |
|
| `cache_dir` | None | 磁盘缓存目录;None=仅内存缓存 |
|
||||||
| `default_adjust` | `Adjust.NONE` | 默认复权方式 |
|
| `default_adjust` | `Adjust.NONE` | 默认复权方式 |
|
||||||
| `max_inflight` | 64 | single-flight 并发上限 |
|
| `max_inflight` | 64 | single-flight 并发上限 |
|
||||||
| `pool_min` / `pool_max` | 3 / 24 | 动态池上下限 |
|
| `pool_min` / `pool_max` | 3 / 12 | 动态池上下限 |
|
||||||
| `mac_pool_cap` / `std_pool_cap` / `ex_pool_cap` | 4 / 4 / 2 | 单台服务器连接上限 |
|
| `mac_pool_cap` | 4 | 单台 MAC 服务器连接上限 |
|
||||||
| `df` | None | **DataFrame 后端**:`None`=返回 `list[dataclass]`(0.6.x 行为)、`"polars"`、`"pandas"`。需先装 extra:`pip install pytdxdata[polars]`,详见 [DataFrame 输出](dataframe.md) |
|
|
||||||
|
|
||||||
生命周期:
|
生命周期:
|
||||||
|
|
||||||
@@ -42,25 +41,19 @@ from pytdxdata import (
|
|||||||
|
|
||||||
### 2.1 统一接口(v0.6+ 推荐){#unified}
|
### 2.1 统一接口(v0.6+ 推荐){#unified}
|
||||||
|
|
||||||
> **DataFrame(v0.7+)**:本节的表格型方法在 `TdxData(df="polars")` / `df="pandas"` 时
|
|
||||||
> 直接返回 DataFrame;`df=None`(默认)返回下表的 `list[...]`。列口径、时间列替换规则、
|
|
||||||
> 不受影响的接口清单见 [DataFrame 输出](dataframe.md)。
|
|
||||||
|
|
||||||
以下方法在 v0.6 新增,标的一律用字符串(`sz000001` / `hk00700` / `usAAPL` / `cffex:IFL0`)。
|
以下方法在 v0.6 新增,标的一律用字符串(`sz000001` / `hk00700` / `usAAPL` / `cffex:IFL0`)。
|
||||||
|
|
||||||
#### 标的清单
|
#### 标的清单
|
||||||
|
|
||||||
| 方法 | 返回 | 说明 |
|
| 方法 | 返回 | 说明 |
|
||||||
|------|------|------|
|
|------|------|------|
|
||||||
| `get_universe(market=None)` | `list[SecurityInfo]` | 跨市场标的清单(A 股+港股+美股+期货/期权);每条带协议自带的 `decimal_point`(价格小数位)与 `pre_close`。**北交所清单协议不支持,返回空列表** |
|
| `get_universe(market=None)` | `list[SecurityInfo]` | 跨市场标的清单(A 股+港股+美股+期货/期权) |
|
||||||
| `sync_decimal_points(markets=("sh","sz"))` | `dict` | 从证券清单同步**权威价格小数位表**(`SecurityInfo.decimal_point`),灌进 `price_decimal_digits` 的运行时表;`start()` 已自动做(当日缓存命中则零网络),此方法用于强制刷新/排障 |
|
|
||||||
| `decimal_point_status()` | `dict` | 小数位表状态:`source`(`cache`/`universe`/空)、`stock`、`mixed_prefixes`、`syncing` |
|
|
||||||
|
|
||||||
#### 报价
|
#### 报价
|
||||||
|
|
||||||
| 方法 | 返回 | 说明 |
|
| 方法 | 返回 | 说明 |
|
||||||
|------|------|------|
|
|------|------|------|
|
||||||
| `get_quotes(symbols, *, fields=None, concurrency=None, pipeline=True)` | `list[SecurityQuote]` | 跨市场混合报价(A 股标准五档;给了 fields 走 MAC/EX 字段位有 name)。超 80 只自动切批,`pipeline` 开启**单连接流水线**,全市场 5,200 只 ~0.33s |
|
| `get_quotes(symbols, *, fields=None)` | `list[SecurityQuote]` | 跨市场混合报价(A 股标准五档;给了 fields 走 MAC/EX 字段位有 name) |
|
||||||
| `get_snapshot(symbols)` | `list[SymbolSnapshot]` | 个股特征快照(A 股) |
|
| `get_snapshot(symbols)` | `list[SymbolSnapshot]` | 个股特征快照(A 股) |
|
||||||
| `get_auction(symbols)` | `list[AuctionItem]` | 集合竞价(A 股) |
|
| `get_auction(symbols)` | `list[AuctionItem]` | 集合竞价(A 股) |
|
||||||
| `get_capital_flow(symbols)` | `list[CapitalFlow]` | 个股资金流向(A 股) |
|
| `get_capital_flow(symbols)` | `list[CapitalFlow]` | 个股资金流向(A 股) |
|
||||||
@@ -69,14 +62,14 @@ from pytdxdata import (
|
|||||||
|
|
||||||
| 方法 | 返回 | 说明 |
|
| 方法 | 返回 | 说明 |
|
||||||
|------|------|------|
|
|------|------|------|
|
||||||
| `get_bars(symbols, period=DAY, *, start=0, count=800, adjust=None, start_date=None, end_date=None, concurrency=6, pipeline=None)` | `list[SecurityBar]` | 跨市场 K 线(自动判指数、复权走 MAC、并发分页;日期区间贴最新走倒推、历史中段走二分)。**≥16 只自动走多连接流水线**:`pipeline` 为每条连接的在途请求数(默认按单请求字节自适应 8~32,传 `1` 关闭),全市场 1 分钟 K 线取当天 43.6s → 5s 级 |
|
| `get_bars(symbols, period=DAY, *, start=0, count=800, adjust=None, start_date=None, end_date=None, concurrency=6)` | `list[SecurityBar]` | 跨市场 K 线(自动判指数、复权走 MAC、并发分页、日期区间二分) |
|
||||||
| `get_indicators(symbols, indicators, *, period=DAY, count=30, adjust=None, params=None)` | `list[IndicatorSet]` | K 线+技术指标(MACD/KDJ/RSI/BOLL/MA/EMA) |
|
| `get_indicators(symbols, indicators, *, period=DAY, count=30, adjust=None, params=None)` | `list[IndicatorSet]` | K 线+技术指标(MACD/KDJ/RSI/BOLL/MA/EMA) |
|
||||||
|
|
||||||
#### 逐笔与分时
|
#### 逐笔与分时
|
||||||
|
|
||||||
| 方法 | 返回 | 说明 |
|
| 方法 | 返回 | 说明 |
|
||||||
|------|------|------|
|
|------|------|------|
|
||||||
| `get_ticks(symbols, *, date=None, start=0, count=None, concurrency=6)` | `list[TransactionRecord]` | 跨市场逐笔成交。返回按时间升序;`start` **从最新一笔往回数**;`vol`/`trade_count` 为约 3 秒切片的增量值(非累计)。**已剔除 `bs_flag=8` 竞价委托**(非成交,`vol` 为累计撮合量)。**指数**:`vol` 单位是百元(成交额/100)、`trade_count` 是均价×1000,均非股票语义;**可转债**:`vol` 是手(=10 张)、`trade_count` 是**真实笔数**,`Σvol` 与日线 `vol` **1:1** |
|
| `get_ticks(symbols, *, date=None, start=0, count=None, concurrency=6)` | `list[TransactionRecord]` | 跨市场逐笔成交 |
|
||||||
| `get_minutes(symbols, *, date=None, sampling=False, concurrency=6)` | `list[MinuteBar]` | 跨市场分时(sampling=True 取缩略采样:A股由 0x122D 派生,扩展市场 0x254D) |
|
| `get_minutes(symbols, *, date=None, sampling=False, concurrency=6)` | `list[MinuteBar]` | 跨市场分时(sampling=True 取缩略采样:A股由 0x122D 派生,扩展市场 0x254D) |
|
||||||
|
|
||||||
#### 公司信息/文件/板块
|
#### 公司信息/文件/板块
|
||||||
@@ -88,15 +81,27 @@ from pytdxdata import (
|
|||||||
| `get_boards(kind="all", *, count=10000)` | `list[BoardInfo]` | 板块列表 |
|
| `get_boards(kind="all", *, count=10000)` | `list[BoardInfo]` | 板块列表 |
|
||||||
| `get_board_members(board, *, fields=None, count=100000)` | `list[SecurityQuote]` | 板块成分股报价(归一模型) |
|
| `get_board_members(board, *, fields=None, count=100000)` | `list[SecurityQuote]` | 板块成分股报价(归一模型) |
|
||||||
| `get_board_of(symbols)` | `list[BelongBoard]` | 个股所属板块 |
|
| `get_board_of(symbols)` | `list[BelongBoard]` | 个股所属板块 |
|
||||||
| `get_index_members(index, *, fields=None, count=100000)` | `list[SecurityQuote]` | **指数成分股**报价。裸 6 位按**指数语义**解析(`000001`=上证指数,不是平安银行):`000/999/880`→沪、`399`→深、`899`→北交所 |
|
|
||||||
| `get_index_trade_stats(index, date=None, *, concurrency=24, batch_size=50, members=None, skip_errors=True, on_progress=None)` | `IndexTradeStats` | **指数成分逐笔汇总**成每分钟成交量/成交笔数。协议里指数本身没有笔数(`trade_count` 是均价×1000),故由成分股逐笔累加;`.cumulative()` 取「截止该分钟累计」,`.cum_trade_count()` 直接给 `[(HH:MM, 累计笔数)]`。2230 只(上证)约 36-60 s |
|
#### 指数成分与统计(v0.6.7)
|
||||||
| `get_index_trade_stats_multi(index, dates, *, ...)` | `list[IndexTradeStats]` | 多日批量版(成分只请求一次),用于「近 N 日同时点累计均值」基准 |
|
|
||||||
|
| 方法 | 返回 | 说明 |
|
||||||
|
|------|------|------|
|
||||||
|
| `get_index_members(index, *, count=100000, fields=None)` | `list[SecurityQuote]` | 指数成分股含实时报价(MAC 0x122C,指数码→板块码映射) |
|
||||||
|
| `get_index_trade_stats(index, *, date=None, members=None, batch_size=50, concurrency=24, skip_errors=True)` | `IndexTradeStats` | 指数逐笔聚合统计(分钟级增量 vol/trade_count/amount;北交所成分无笔数自动剔除) |
|
||||||
|
| `get_index_trade_stats_multi(index, dates, *, concurrency=24, batch_size=50, skip_errors=True, on_progress=None)` | `dict[int, IndexTradeStats]` | 多日批量版(按日期键) |
|
||||||
|
|
||||||
|
#### 价格小数位(v0.6.7)
|
||||||
|
|
||||||
|
| 方法 | 返回 | 说明 |
|
||||||
|
|------|------|------|
|
||||||
|
| `sync_decimal_points(markets=("sh","sz"))` | `dict[str, int]` | 从证券清单同步权威价格小数位表(`start()` 已自动处理,此为手动刷新口) |
|
||||||
|
| `decimal_point_status()` | `dict` | 小数位表当前状态(source/stock/syncing,排障用) |
|
||||||
|
|
||||||
### 2.2 K线(旧接口,共 6 个) {#legacy-kline}
|
### 2.2 K线(旧接口,共 6 个) {#legacy-kline}
|
||||||
|
|
||||||
| 方法 | 返回 | 说明 |
|
| 方法 | 返回 | 说明 |
|
||||||
|------|------|------|
|
|------|------|------|
|
||||||
| `get_kline(market, code, period, *, start=0, count=800, adjust=None, start_date=None, end_date=None)` | `list[SecurityBar]` | 主接口;count>页自动并发分页;复权走 MAC;支持日期区间(贴最新倒推 / 中段二分) |
|
| `get_kline(market, code, period, *, start=0, count=800, adjust=None, start_date=None, end_date=None)` | `list[SecurityBar]` | 主接口;count>页自动并发分页;复权走 MAC;支持日期区间(二分定位) |
|
||||||
| `get_kline_since(market, code, period, since, adjust=None)` | `list[SecurityBar]` | 增量便捷方法(≥since,含当天) |
|
| `get_kline_since(market, code, period, since, adjust=None)` | `list[SecurityBar]` | 增量便捷方法(≥since,含当天) |
|
||||||
| `get_kline_batch(stocks, period, *, start=0, count=800, adjust=None, concurrency=6)` | `dict[str, list[SecurityBar]]` | 多股票并发,返回 {code: bars} |
|
| `get_kline_batch(stocks, period, *, start=0, count=800, adjust=None, concurrency=6)` | `dict[str, list[SecurityBar]]` | 多股票并发,返回 {code: bars} |
|
||||||
| `get_index_kline(market, code, period, *, start=0, count=800)` | `list[SecurityBar]` | 指数K线(含涨跌家数字段) |
|
| `get_index_kline(market, code, period, *, start=0, count=800)` | `list[SecurityBar]` | 指数K线(含涨跌家数字段) |
|
||||||
@@ -107,7 +112,7 @@ from pytdxdata import (
|
|||||||
|
|
||||||
| 方法 | 返回 | 说明 |
|
| 方法 | 返回 | 说明 |
|
||||||
|------|------|------|
|
|------|------|------|
|
||||||
| `get_quotes(symbols, *, fields=None, concurrency=None, pipeline=True)` | `list[SecurityQuote]` | 五档报价;>80 只自动切批(流水线+工作窃取,5,200 只 ~0.33s);TTL 5s |
|
| `get_quotes(symbols, *, fields=None)` | `list[SecurityQuote]` | 五档报价;>80 只自动切批;TTL 5s |
|
||||||
| `get_stock_quotes(stocks, bits=None)` | `list[MemberQuote]` | MAC 自定义字段报价(≤80只/次) |
|
| `get_stock_quotes(stocks, bits=None)` | `list[MemberQuote]` | MAC 自定义字段报价(≤80只/次) |
|
||||||
| `get_stock_quotes_list(category, *, start=0, count=80, sort_type=0)` | `list[MemberQuote]` | 市场分类报价(0沪A/2深A/6全A/8科创/14创业) |
|
| `get_stock_quotes_list(category, *, start=0, count=80, sort_type=0)` | `list[MemberQuote]` | 市场分类报价(0沪A/2深A/6全A/8科创/14创业) |
|
||||||
| `get_market_stat()` | `MarketStat` | 全市场统计(涨跌家数/总额/市值/涨跌停数) |
|
| `get_market_stat()` | `MarketStat` | 全市场统计(涨跌家数/总额/市值/涨跌停数) |
|
||||||
@@ -204,12 +209,12 @@ from pytdxdata import (
|
|||||||
|
|
||||||
| 模型 | 关键字段 |
|
| 模型 | 关键字段 |
|
||||||
|------|---------|
|
|------|---------|
|
||||||
| `SecurityBar` | market/code/open/high/low/close/vol/amount/year/month/day/hour/minute/symbol/**settle** + `datetime` 属性(`settle` 结算价仅 EX 期货/期权日线有值,其余为 `None`) |
|
| `SecurityBar` | market/code/open/high/low/close/vol/amount/year/month/day/hour/minute + `datetime` 属性 |
|
||||||
| `IndexBar` | 继承 `SecurityBar`(指数K线) |
|
| `IndexBar` | 继承 `SecurityBar`(指数K线) |
|
||||||
| `SecurityQuote` | price/pre_close/OHLC/vol/amount/s_vol/b_vol/`bid[5]`/`ask[5]`/server_time/trading_status/cur_vol/rise_speed/limit_up/limit_down/decimal_point/open_amount |
|
| `SecurityQuote` | price/pre_close/OHLC/vol/amount/s_vol/b_vol/`bid[5]`/`ask[5]`/server_time/trading_status/cur_vol/rise_speed/limit_up/limit_down/decimal_point/open_amount |
|
||||||
| `SecurityInfo` | market/code/name/volunit/decimal_point/pre_close |
|
| `SecurityInfo` | market/code/name/volunit/decimal_point/pre_close |
|
||||||
| `TransactionRecord` | market/code/time/price/vol/`trade_count`/bs_flag |
|
| `TransactionRecord` | market/code/time/price/vol/`trade_count`/bs_flag |
|
||||||
| `MinuteBar` | price/vol/avg_price/hour/minute/day_offset + `datetime`/`trading_date` 只读属性(hour 恒为合法时钟值 0~23;美股一个交易日跨两个北京自然日,按交易日聚合用 `trading_date`) |
|
| `MinuteBar` | price/vol/avg_price/hour/minute |
|
||||||
| `XdxrRecord` | 分红送转/配股/缩股/行权 + 股本变动 + `date` 属性 |
|
| `XdxrRecord` | 分红送转/配股/缩股/行权 + 股本变动 + `date` 属性 |
|
||||||
| `FinanceRecord` | 股本/资产/负债/营收/利润等 36 个字段(万元/万股) |
|
| `FinanceRecord` | 股本/资产/负债/营收/利润等 36 个字段(万元/万股) |
|
||||||
| `BoardInfo` | 板块 + 领涨股信息 |
|
| `BoardInfo` | 板块 + 领涨股信息 |
|
||||||
@@ -228,31 +233,6 @@ from pytdxdata import (
|
|||||||
|
|
||||||
> 注:`__init__` 仅导出 `Market/ExMarket/KlinePeriod/Adjust` 与 6 个核心模型;`BoardInfo/MemberQuote/AuctionItem` 等需 `from pytdxdata.models import ...`。
|
> 注:`__init__` 仅导出 `Market/ExMarket/KlinePeriod/Adjust` 与 6 个核心模型;`BoardInfo/MemberQuote/AuctionItem` 等需 `from pytdxdata.models import ...`。
|
||||||
|
|
||||||
**量纲对照(跨品种,2026-10-22 实测)**
|
|
||||||
|
|
||||||
同名字段在不同品种上单位不同。最典型的坑是用 `amount / vol` 算均价——它在可转债上会**错 10 倍**:
|
|
||||||
|
|
||||||
| 品种 | 分时 / 逐笔 `vol` | 日线 `SecurityBar.vol` | `Σ分时vol / 日线vol` | 日线 `amount / vol` |
|
|
||||||
|---|---|---|---|---|
|
|
||||||
| 股票 / ETF / LOF | 手(= 100 股) | 股 | **0.01** | 均价 ✓ |
|
|
||||||
| 可转债(110/111/113/118/123/127/128) | 手(= 10 张) | **手** | **1.0** | 均价 **×10** ⚠️ |
|
|
||||||
| 指数 | 百元(成交额/100) | 手 | — | 无意义(`amount` 是全板块口径) |
|
|
||||||
| 港股主板 31 / 港股基金 49 | — | **手(= 100 股)** | — | 均价 **×100** ⚠️ |
|
|
||||||
| 港股创业板 48 / 新三板 44 / 美股 74 | — | 股 | — | 均价 ✓ |
|
|
||||||
| 期货 / 期权(EX 全市场) | — | 手/张 | — | **`amount` 恒 0.0**(帧内无成交额,见下) |
|
|
||||||
|
|
||||||
算均价:股票 `amount / vol`;可转债 `amount / vol / 10`;港股主板/基金 `amount / vol / 100`;跨品种统一用 `Σ(price×vol) / Σvol`。
|
|
||||||
|
|
||||||
> ⚠️ **EX 通道(0x23FF)的成交额不是全市场都有的**。一条记录 32B = 4B 时间 + 28B 行情,
|
|
||||||
> 28B 里只有**一个** 4B 槽位可复用,语义按市场分两类:
|
|
||||||
> 权益类(港股主板/创业板/基金、新三板、美股)该槽位是**成交额**,`amount` 正常;
|
|
||||||
> 期货期权类该槽位是**结算价**(体现在 `SecurityBar.settle`,日线才有值),
|
|
||||||
> `amount` 恒 **0.0** —— 这是协议限制,不是解析漏读,要成交额请改用 `get_quotes` 或其它源。
|
|
||||||
> 港股指数(27)/ 中证指数(62)该槽位语义未明,同样保守返回 `0.0`。
|
|
||||||
|
|
||||||
> 可转债的价格标度、分时、逐笔已全量校验(16 只当日 + 8 只历史日):分时末价 / 快照现价、`Σvol / 日线vol`、逐笔 VWAP vs `amount/(vol×10)` 比值均为 **1.000000**。
|
|
||||||
|
|
||||||
|
|
||||||
## 四、辅助模块
|
## 四、辅助模块
|
||||||
|
|
||||||
**`pytdxdata.indicator`**(技术指标,纯函数)
|
**`pytdxdata.indicator`**(技术指标,纯函数)
|
||||||
@@ -335,13 +315,4 @@ CLI 开关:`--discover`(三种协议全测并据实修正 `protocols`,同
|
|||||||
| 逐笔成交 / 分时 | MAC |
|
| 逐笔成交 / 分时 | MAC |
|
||||||
| 列表 / 数量 / 除权 / 财务 | 标准 |
|
| 列表 / 数量 / 除权 / 财务 | 标准 |
|
||||||
| 板块 / 竞价 / 异动 / 快照 / 资金流 / 服务器 / 分类报价 | MAC |
|
| 板块 / 竞价 / 异动 / 快照 / 资金流 / 服务器 / 分类报价 | MAC |
|
||||||
| 港股 / 美股 / 期货 / 期权 | EX(端口 7727,`0x23FF`) |
|
| 港股 / 美股 / 期货 / 期权 | EX(端口 7727) |
|
||||||
|
|
||||||
> EX 通道的 K 线(`0x23FF`)字段口径与 A 股不同,见「量纲对照」下的注:
|
|
||||||
> 权益类市场(港股/美股/新三板)有 `amount`;期货期权有 `settle`(结算价)但**没有** `amount`。
|
|
||||||
|
|
||||||
**多标的 K 线(`get_bars` 传 ≥16 只)额外走「批量流水线」**:按上表选池后,
|
|
||||||
多条连接并行、每条连接一次下发 `pipeline` 条请求(默认自适应 8~32),把 N 次往返
|
|
||||||
折叠成 `N/pipeline` 次。只用实测最快的 12 台服务器(池里 24 台全用会被慢节点拖长尾)。
|
|
||||||
`start_date=今天` 时只拉已走过的根数(10:00 → 30 根),盘中实时刷新因此只要 1~2 秒。
|
|
||||||
流水线失败时该批自动交回逐只重试,不会丢数据。
|
|
||||||
+1
-1
@@ -26,7 +26,7 @@ curl -o ~/.workbuddy/skills/pytdxdata/SKILL.md \
|
|||||||
## 里面有什么
|
## 里面有什么
|
||||||
|
|
||||||
- **三条必守约定** —— 唯一入口是 `TdxData` 且必须 `async with`;标的是字符串(如 `sh600000`)
|
- **三条必守约定** —— 唯一入口是 `TdxData` 且必须 `async with`;标的是字符串(如 `sh600000`)
|
||||||
而不是 `(market, code)` 元组;默认返回 `list[dataclass]`(要 DataFrame 得用 `TdxData(df="polars")` 显式声明后端)
|
而不是 `(market, code)` 元组;返回值是 `list[dataclass]` 而不是 DataFrame
|
||||||
- **枚举取值表** —— `Market` / `KlinePeriod` / `Adjust` / `ExMarket` 的准确取值
|
- **枚举取值表** —— `Market` / `KlinePeriod` / `Adjust` / `ExMarket` 的准确取值
|
||||||
- **可运行模板** —— K 线、复权、报价、逐笔、分时、批量,全部是能直接跑的代码
|
- **可运行模板** —— K 线、复权、报价、逐笔、分时、批量,全部是能直接跑的代码
|
||||||
- **「想做什么 → 用哪个方法」决策表** —— 18 个常见需求的直接映射
|
- **「想做什么 → 用哪个方法」决策表** —— 18 个常见需求的直接映射
|
||||||
|
|||||||
+2
-10
@@ -2,16 +2,8 @@
|
|||||||
|
|
||||||
全部数据模型位于 `pytdxdata.models`,统一为 `@dataclass(slots=True)`。
|
全部数据模型位于 `pytdxdata.models`,统一为 `@dataclass(slots=True)`。
|
||||||
|
|
||||||
默认(`df=None`)所有接口返回这些 dataclass —— **零依赖 + 零转换开销**。
|
选择 dataclass 而不是 pandas DataFrame 是刻意的:**零依赖 + 零转换开销**,
|
||||||
不想要 dataclass 就直接开 DataFrame 出口,不用自己 `asdict` 再建表:
|
需要 DataFrame 时你自己转(一行的事,见 [场景示例 → 转成 pandas](../cookbook.md#to-pandas))。
|
||||||
|
|
||||||
```python
|
|
||||||
async with TdxData(df="polars") as td: # 或 df="pandas"
|
|
||||||
df = await td.get_bars("sh600000", KlinePeriod.DAY, count=240)
|
|
||||||
```
|
|
||||||
|
|
||||||
列口径(时间列合并、五档展开、`fields` JSON 化)见 [DataFrame 输出](../dataframe.md);
|
|
||||||
两者是同一份数据,只是容器不同,**量纲与语义完全一致**。
|
|
||||||
|
|
||||||
## 导入方式
|
## 导入方式
|
||||||
|
|
||||||
|
|||||||
-11
@@ -56,17 +56,6 @@ tdx minute sz000001 # 今日分时
|
|||||||
tdx minute sz000001 --date 20260811 # 历史分时
|
tdx minute sz000001 --date 20260811 # 历史分时
|
||||||
```
|
```
|
||||||
|
|
||||||
价格列的小数位**按品种自动适配**,不用自己换算:
|
|
||||||
|
|
||||||
| 品种 | 显示小数位 | 示例 |
|
|
||||||
|---|---|---|
|
|
||||||
| 股票 / 指数 | 2 | `12.34` |
|
|
||||||
| ETF / LOF / 沪市 B 股 / 可转债 | 3 | `128.892`(可转债报价步长 0.001 元) |
|
|
||||||
| 债券 / 回购 | 4 | `100.0100` |
|
|
||||||
|
|
||||||
> 可转债的**解码位数**是 4 位(原始整数 / 10⁴),但第 4 位恒为 0,所以显示按 3 位。
|
|
||||||
> `tdx minute` 的时间列是该 bar 的**真实时钟**(`09:31` / `14:57`),不是序号推算。
|
|
||||||
|
|
||||||
## 板块
|
## 板块
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
|||||||
+1
-12
@@ -196,18 +196,7 @@ async def ex_markets():
|
|||||||
|
|
||||||
## 转成 pandas {#to-pandas}
|
## 转成 pandas {#to-pandas}
|
||||||
|
|
||||||
**0.7.0 起不用自己转了** —— 构造时声明一次后端,表格型接口直接返回 DataFrame:
|
包本身不带 pandas 依赖,但转换是一行的事:
|
||||||
|
|
||||||
```python
|
|
||||||
async with TdxData(df="pandas") as td: # 或 df="polars"
|
|
||||||
df = await td.get_bars("sh600000", KlinePeriod.DAY, count=800)
|
|
||||||
df = df.set_index("datetime") # 时间列已经合并好了
|
|
||||||
```
|
|
||||||
|
|
||||||
需要 `pip install "pytdxdata[pandas]"`(或 `[polars]` / `[dataframe]`),细节见
|
|
||||||
[DataFrame 输出](dataframe.md)。
|
|
||||||
|
|
||||||
如果只想对**某一处**手动转、不想受影响全局,仍可用 dataclass 自己拼:
|
|
||||||
|
|
||||||
```python
|
```python
|
||||||
import pandas as pd
|
import pandas as pd
|
||||||
|
|||||||
@@ -1,122 +0,0 @@
|
|||||||
# DataFrame 输出(pandas / polars)
|
|
||||||
|
|
||||||
0.7.0 起,`TdxData` 可以**直接把表格型接口的返回值变成 DataFrame**,不用再自己 `[asdict(x) for x in rows]`。
|
|
||||||
|
|
||||||
设计上只有一条开关,就是构造参数 `df`:
|
|
||||||
|
|
||||||
```python
|
|
||||||
from pytdxdata import TdxData, KlinePeriod
|
|
||||||
|
|
||||||
# 默认:df=None,行为和 0.6.x 完全一致,返回 list[dataclass]
|
|
||||||
async with TdxData() as td:
|
|
||||||
bars = await td.get_bars("sh600000", KlinePeriod.DAY, count=240) # list[SecurityBar]
|
|
||||||
|
|
||||||
# 要 DataFrame 就在构造时声明一次后端
|
|
||||||
async with TdxData(df="polars") as td:
|
|
||||||
df = await td.get_bars("sh600000", KlinePeriod.DAY, count=240) # polars.DataFrame
|
|
||||||
|
|
||||||
async with TdxData(df="pandas") as td:
|
|
||||||
df = await td.get_bars("sh600000", KlinePeriod.DAY, count=240) # pandas.DataFrame
|
|
||||||
```
|
|
||||||
|
|
||||||
## 安装
|
|
||||||
|
|
||||||
pandas / polars 都是**可选依赖**——`pytdxdata` 的 `dependencies` 仍然是空的,不想用 DataFrame 的人一点额外开销都不用付。
|
|
||||||
|
|
||||||
```bash
|
|
||||||
pip install "pytdxdata[polars]" # 只要 polars
|
|
||||||
pip install "pytdxdata[pandas]" # 只要 pandas
|
|
||||||
pip install "pytdxdata[dataframe]" # 两个都要
|
|
||||||
```
|
|
||||||
|
|
||||||
没装就 `TdxData(df="polars")`,会在**构造时**直接报错,而不是等到第一次查询:
|
|
||||||
|
|
||||||
```
|
|
||||||
FrameBackendError: 未安装 polars,无法输出 DataFrame。请执行:pip install pytdxdata[polars]
|
|
||||||
```
|
|
||||||
|
|
||||||
`df` 取值非法(比如 `df="numpy"`)抛 `ValueError`,并列出可用值。
|
|
||||||
|
|
||||||
## 三个口径约定
|
|
||||||
|
|
||||||
### 1. 时间列:`datetime` 替换拆开的年月日时分
|
|
||||||
|
|
||||||
原始模型为了解析性能把时间拆成 `year/month/day/hour/minute`。DataFrame 里再这么摆,用的人第一件事就是自己拼时间戳——所以直接给一列 `datetime`:
|
|
||||||
|
|
||||||
| 模型 | 原始字段 | DataFrame 里的列 |
|
|
||||||
|---|---|---|
|
|
||||||
| `SecurityBar` / `IndexBar` | `year month day hour minute` | `datetime`(+ `trading_date`) |
|
|
||||||
| `MinuteBar` | `date hour minute day_offset` | `datetime`(+ `trading_date`) |
|
|
||||||
| `XdxrRecord` | `year month day` | `date` |
|
|
||||||
|
|
||||||
- `datetime` 的 dtype:polars 是 `Datetime('us')`,pandas 是 `datetime64[ns]`。
|
|
||||||
- `trading_date` 是 `YYYY-MM-DD` 字符串,**按交易日聚合要用它**:美股一个交易日横跨两个北京自然日(21:30→次日 04:00),凌晨段的 `datetime` 属于次日,但 `trading_date` 归属前一交易日。
|
|
||||||
- 除权记录的 `date` 是「日」不是「时刻」,值为当日 00:00,仍用 datetime 类型以便排序比较。
|
|
||||||
- `TransactionRecord.time` 只有 `HH:MM:SS`、**协议里没有日期**,所以逐笔不生成 `datetime`,`time` 原样保留。
|
|
||||||
|
|
||||||
### 2. 嵌套字段:能定型的定型,不能定型的 JSON 化
|
|
||||||
|
|
||||||
| 字段 | DataFrame 里的形态 |
|
|
||||||
|---|---|
|
|
||||||
| `SecurityQuote.bid` / `ask`(五档) | 展开成 `bid1_price` `bid1_vol` … `ask5_price` `ask5_vol` 共 **10 列**,缺档为 `None` |
|
|
||||||
| `fields`(A股字段位 / EX 额外字段) | **JSON 字符串**列;空字典给 `None`(不会看到一列 `"{}"`) |
|
|
||||||
| `TdxBlock.codes`(成分股列表) | 保持列表(polars 为 `List(Str)`,pandas 为 object),需要时 `.explode()` |
|
|
||||||
|
|
||||||
`fields` 之所以不展开:键集随市场和通道变化,展开会让列数失控。
|
|
||||||
|
|
||||||
### 3. 特殊容器会被展开成一张表
|
|
||||||
|
|
||||||
| 接口 | 原始返回 | DataFrame 形态 |
|
|
||||||
|---|---|---|
|
|
||||||
| `get_indicators` | `list[IndicatorSet]` | bars 列 + 每个指标一列,按长度对齐(不足补 `None`),`symbol` 取自 `IndicatorSet.symbol` |
|
|
||||||
| `get_index_trade_stats` | `IndexTradeStats` | `minutes` 展开成行,并补 `index` / `date` 两个常量列 |
|
|
||||||
| `get_index_trade_stats_multi` | `list[IndexTradeStats]` | 同上,多个指数纵向拼接 |
|
|
||||||
| `get_kline_batch` / `get_minute_batch` | `dict[标的, list[...]]` | 各标的分块纵向拼接成一张表(`symbol`/`code` 列本来就在行里) |
|
|
||||||
| `get_board_ranking` / `get_board_change_ranking` | `list[dict]` | 按首次出现顺序取键并集作为列名 |
|
|
||||||
|
|
||||||
## 哪些接口不受影响
|
|
||||||
|
|
||||||
`df` 只作用于**表格型返回**。下面这些原样返回,不转 DataFrame:
|
|
||||||
|
|
||||||
| 返回类型 | 接口 |
|
|
||||||
|---|---|
|
|
||||||
| `bytes` | `get_file` `get_block_info` `get_report_file` `download_file` `get_financial_records` |
|
|
||||||
| `int` | `get_security_count` `get_goods_count` |
|
|
||||||
| `dict` | `decimal_point_status` `get_board_summary` `get_industry_map` |
|
|
||||||
| 单个对象 | `get_server_info` `get_market_stat` `get_symbol_info` |
|
|
||||||
| `tuple` | `get_price_limits` `get_kline_offset` |
|
|
||||||
| `list[float]` / `list[str]` | `get_chart_sampling` `get_goods_chart_sampling` `get_financial_file_list` |
|
|
||||||
|
|
||||||
理由是单行表没有意义,而 `bytes`/`int` 根本不是表格数据。
|
|
||||||
|
|
||||||
## 空结果
|
|
||||||
|
|
||||||
查不到数据时返回的是**列名齐全的空 DataFrame**(列由模型定义推出,不依赖是否拿到行):
|
|
||||||
|
|
||||||
```python
|
|
||||||
async with TdxData(df="polars") as td:
|
|
||||||
df = await td.get_xdxr("sz000001") # 假设当天无除权
|
|
||||||
df.height, df.columns # 0, ['market', 'code', 'date', ...]
|
|
||||||
```
|
|
||||||
|
|
||||||
⚠️ 空表只保证**列名**正确,dtype 由后端自行推断(polars 可能全为 `Null`,pandas 为 object)。
|
|
||||||
|
|
||||||
## 语义提醒
|
|
||||||
|
|
||||||
DataFrame 只是换了个容器,**下面这些口径不会因为转成 DataFrame 而变化**:
|
|
||||||
|
|
||||||
- 股票/ETF 的 `vol` 是「手」,日线是「股」;可转债 `vol` 手=10 张,`amount/vol` 是均价 **×10**;指数逐笔 `vol` 是百元。
|
|
||||||
- 指数和 ETF/LOF 的 `trade_count` 不是笔数(见 `TransactionRecord.trade_count_available`)。
|
|
||||||
- 深市 `399xxx` 的 `amount` 是全板块口径,不要拿去校验成分合计。
|
|
||||||
- K 线的 `settle`(结算价)列只有 **EX 通道的期货/期权日线**有值,其余行全为 `None`;
|
|
||||||
同一批 EX 标的中,期货/期权的 `amount` 恒 `0.0`(`0x23FF` 帧内没有成交额),
|
|
||||||
港股主板/基金的 `vol` 单位是「手」(`amount/vol` 是均价的 100 倍)。
|
|
||||||
|
|
||||||
细节见 [数据模型](api/models.md) 与 [常见问题](faq.md)。
|
|
||||||
|
|
||||||
## 性能
|
|
||||||
|
|
||||||
转换是**列式采集**:按模型预先算好列计划,一次遍历填满所有列,不做逐行 `asdict()`。
|
|
||||||
5,227 只 A 股报价转 polars 约 20~40ms,相对一次全市场查询(约 300ms)可以忽略。
|
|
||||||
|
|
||||||
`df=None` 时**完全不经过这层**(装饰器直接透传),零额外开销。
|
|
||||||
+1
-21
@@ -35,23 +35,6 @@
|
|||||||
|
|
||||||
安装后即可使用 `tdx` 命令行工具(见 [CLI 手册](cli.md))。
|
安装后即可使用 `tdx` 命令行工具(见 [CLI 手册](cli.md))。
|
||||||
|
|
||||||
### 可选:DataFrame 后端(0.7.0+)
|
|
||||||
|
|
||||||
想让表格型接口**直接返回 DataFrame**,需要额外装一个后端。核心包依旧零依赖,不装就完全不受影响:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
pip install "pytdxdata[polars]" # 只要 polars
|
|
||||||
pip install "pytdxdata[pandas]" # 只要 pandas
|
|
||||||
pip install "pytdxdata[dataframe]" # 两个都要
|
|
||||||
```
|
|
||||||
|
|
||||||
装上之后在构造时声明一次即可,详见 [DataFrame 输出](dataframe.md):
|
|
||||||
|
|
||||||
```python
|
|
||||||
async with TdxData(df="polars") as td: # 或 df="pandas"
|
|
||||||
df = await td.get_bars("sh600000", KlinePeriod.DAY, count=240)
|
|
||||||
```
|
|
||||||
|
|
||||||
### 可选:uvloop 加速
|
### 可选:uvloop 加速
|
||||||
|
|
||||||
Linux / macOS 上可选用 `uvloop` 替换默认事件循环(Windows 不支持):
|
Linux / macOS 上可选用 `uvloop` 替换默认事件循环(Windows 不支持):
|
||||||
@@ -80,11 +63,8 @@ asyncio.run(main())
|
|||||||
| `cache_dir` | `None` | 磁盘缓存目录;**`None` = 仅内存缓存**(进程退出即失效) |
|
| `cache_dir` | `None` | 磁盘缓存目录;**`None` = 仅内存缓存**(进程退出即失效) |
|
||||||
| `default_adjust` | `Adjust.NONE` | 默认复权方式,可被单次调用覆盖 |
|
| `default_adjust` | `Adjust.NONE` | 默认复权方式,可被单次调用覆盖 |
|
||||||
| `max_inflight` | `64` | single-flight 并发上限(同键请求合并) |
|
| `max_inflight` | `64` | single-flight 并发上限(同键请求合并) |
|
||||||
| `pool_min` / `pool_max` | `3` / `24` | 动态连接池上下限 |
|
| `pool_min` / `pool_max` | `3` / `12` | 动态连接池上下限 |
|
||||||
| `mac_pool_cap` | `4` | 单台 MAC 服务器连接数上限 |
|
| `mac_pool_cap` | `4` | 单台 MAC 服务器连接数上限 |
|
||||||
| `std_pool_cap` | `4` | 单台标准服务器连接数上限 |
|
|
||||||
| `ex_pool_cap` | `2` | 单台 EX 服务器连接数上限(EX 主机较少且较慢,压太高收益为负) |
|
|
||||||
| `df` | `None` | 表格型接口的返回形态:`None` = `list[dataclass]`(与 0.6.x 一致);`"polars"` / `"pandas"` = 直接返回 DataFrame。见 [DataFrame 输出](dataframe.md) |
|
|
||||||
|
|
||||||
### 典型配置
|
### 典型配置
|
||||||
|
|
||||||
|
|||||||
+3
-13
@@ -35,9 +35,8 @@ asyncio.run(main())
|
|||||||
| ⚡ **并发分页** | 把大请求拆成显式 `start` 偏移的页,多连接并行拉取、按序拼接、失败换连接重试 |
|
| ⚡ **并发分页** | 把大请求拆成显式 `start` 偏移的页,多连接并行拉取、按序拼接、失败换连接重试 |
|
||||||
| 🔀 **双通道整合** | 标准通道(经典行情)+ MAC 通道(逐笔 / 复权 / 1分钟K线),按命令自动选路 |
|
| 🔀 **双通道整合** | 标准通道(经典行情)+ MAC 通道(逐笔 / 复权 / 1分钟K线),按命令自动选路 |
|
||||||
| 📄 **服务器文件解析** | 板块 `block_*.dat` / 行业 `tdxhy.cfg` / 历史财报 `gpcw*` **下载即解析**成 dataclass(纯 stdlib),无需自己拆二进制;文件名与索引不一致时自动纠正 |
|
| 📄 **服务器文件解析** | 板块 `block_*.dat` / 行业 `tdxhy.cfg` / 历史财报 `gpcw*` **下载即解析**成 dataclass(纯 stdlib),无需自己拆二进制;文件名与索引不一致时自动纠正 |
|
||||||
| 🚀 **批量下载** | `get_bars` / `get_minutes` 传标的列表即多股票并发,连接池自动分配到不同服务器;K 线走「多连接 × 单连接流水线」,全市场 5,225 只 1 分钟线取当天 **5~8 秒**(盘中增量 1~2 秒) |
|
| 🚀 **批量下载** | `get_bars` / `get_minutes` 传标的列表即多股票并发,连接池自动分配到不同服务器 |
|
||||||
| 🐼 **DataFrame 直出** | `TdxData(df="polars")` / `df="pandas"` 让 42 个表格型接口直接返回 DataFrame;时间列自动合并成 `datetime`,五档报价自动展开。后端是可选 extras,**核心包仍零依赖** |
|
| 📦 **零重型依赖** | 返回 `list[dataclass]`,pandas / polars 由调用方按需转换 |
|
||||||
| 📦 **零重型依赖** | 默认返回 `list[dataclass]`,行为与 0.6.x 完全一致,转不转 DataFrame 由你决定 |
|
|
||||||
|
|
||||||
## 从这里开始
|
## 从这里开始
|
||||||
|
|
||||||
@@ -67,19 +66,11 @@ asyncio.run(main())
|
|||||||
|
|
||||||
[:octicons-arrow-right-24: Cookbook](cookbook.md)
|
[:octicons-arrow-right-24: Cookbook](cookbook.md)
|
||||||
|
|
||||||
- **直接要 DataFrame**
|
|
||||||
|
|
||||||
---
|
|
||||||
|
|
||||||
`TdxData(df="polars")` 一行开关,42 个表格型接口直接出表,不用自己 `asdict`。
|
|
||||||
|
|
||||||
[:octicons-arrow-right-24: DataFrame 输出](dataframe.md)
|
|
||||||
|
|
||||||
- **查接口签名**
|
- **查接口签名**
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
35 个统一方法(+ 33 个 deprecated 别名),从 docstring 实时生成,不会过期。
|
28 个统一方法(+ 33 个 deprecated 别名),从 docstring 实时生成,不会过期。
|
||||||
|
|
||||||
[:octicons-arrow-right-24: API 参考](api/index.md)
|
[:octicons-arrow-right-24: API 参考](api/index.md)
|
||||||
|
|
||||||
@@ -131,7 +122,6 @@ asyncio.run(main())
|
|||||||
- [安装与配置](getting-started.md)
|
- [安装与配置](getting-started.md)
|
||||||
- [场景示例](cookbook.md)
|
- [场景示例](cookbook.md)
|
||||||
- [命令行工具](cli.md)
|
- [命令行工具](cli.md)
|
||||||
- [DataFrame 输出](dataframe.md)
|
|
||||||
|
|
||||||
- **参考**
|
- **参考**
|
||||||
- [接口总览](api/index.md)
|
- [接口总览](api/index.md)
|
||||||
|
|||||||
+4
-32
@@ -44,21 +44,11 @@ asyncio.run(main())
|
|||||||
|
|
||||||
标的一律用字符串;以下均为 v0.6 统一接口,跨市场自动选路。
|
标的一律用字符串;以下均为 v0.6 统一接口,跨市场自动选路。
|
||||||
|
|
||||||
- **标的清单**:`get_universe`(A股 + 港股 + 美股 + 期货/期权)。⚠️ 北交所(`market="bj"`)清单**协议不支持**,返回空列表(`0x053E` 对 market=2 无响应,27 台服务器全测);但北交所**标的本身可用** —— 报价/K线/逐笔直接传 `bj920992` 这类代码正常
|
- **标的清单**:`get_universe`(A股 + 港股 + 美股 + 期货/期权)
|
||||||
- **K 线**:`get_bars`(自动并发分页、支持复权与日期区间、多标的并发)。传 ≥16 只时自动走
|
- **K 线**:`get_bars`(自动并发分页、支持复权与日期区间、多标的并发)
|
||||||
**多连接流水线**(`pipeline` 默认自适应 8~32,传 `1` 关闭),全市场 5,200 只 1 分钟 K 线
|
|
||||||
取当天 43.6s → 5s 级;`start_date=今天` 盘中只拉已走过的根数(10:00 → 30 根),实时刷新 1~2 秒
|
|
||||||
- ⚠️ **EX 通道(港股/美股/期货/期权)的 `amount` 不是全有的**:`0x23FF` 帧内只有一个 4B 槽位可复用,
|
|
||||||
权益类(港股主板 31/创业板 48/基金 49、新三板 44、美股 74)装**成交额**(`amount` 正常),
|
|
||||||
期货期权装**结算价**(见 `SecurityBar.settle`,日线才有值)→ 它们 `amount` 恒 **0.0**,
|
|
||||||
是协议限制而非解析漏读;港股主板/基金的 `vol` 单位是**手(=100 股)**,`amount/vol` 是均价的 100 倍
|
|
||||||
- **技术指标**:`get_indicators`(K线 + MACD/KDJ/RSI/BOLL/MA/EMA)
|
- **技术指标**:`get_indicators`(K线 + MACD/KDJ/RSI/BOLL/MA/EMA)
|
||||||
- **报价**:`get_quotes`(五档,>80 只自动切批;**单连接流水线**默认开启,全市场 5,200 只 A股 ~0.33s)、`get_snapshot`(个股特征)、`get_market_stat`
|
- **报价**:`get_quotes`(五档,>80 只自动切批)、`get_snapshot`(个股特征)、`get_market_stat`
|
||||||
- 价格标度:协议给的是整数,真实价 = `整数 / 10**小数位`。小数位由 `price_decimal_digits` 决定,
|
- **逐笔 / 分时**:`get_ticks`(含 `trade_count`,支持历史日期)、`get_minutes`(`sampling=True` 取缩略采样)
|
||||||
优先用**权威表**(证券清单自带的 `SecurityInfo.decimal_point`,`start()` 每日后台同步一次,
|
|
||||||
也可 `await td.sync_decimal_points()` 强制刷新),未同步时退回静态前缀规则。
|
|
||||||
债券/回购是 **4 位**、沪市 B 股 3 位 —— 这里错过两次(整批价格差 100 倍/10 倍)。
|
|
||||||
- **逐笔 / 分时**:`get_ticks`(含 `trade_count`,支持历史日期,返回按时间升序;`vol`/`trade_count` 为约 3 秒切片增量而非累计;已剔除 `bs_flag=8` 竞价委托;**指数**的 `vol` 是百元、`trade_count` 是均价×1000;**可转债**的 `vol` 是手=10 张、`trade_count` 是真实笔数,故日线 `amount/vol` 是均价的 **10 倍**,算均价要再 /10)、`get_minutes`(`sampling=True` 取缩略采样;`hour` 恒为合法时钟值 0~23,美股一个交易日跨两个北京自然日,按交易日聚合用 `MinuteBar.trading_date`、取时刻用 `MinuteBar.datetime`)
|
|
||||||
- **财务 / 除权**:`get_finance`(最新快照)、`get_xdxr`、`get_price_limits`(本地计算涨跌停)
|
- **财务 / 除权**:`get_finance`(最新快照)、`get_xdxr`、`get_price_limits`(本地计算涨跌停)
|
||||||
- **F10**:`get_f10`(公司资料目录 + 正文)
|
- **F10**:`get_f10`(公司资料目录 + 正文)
|
||||||
- **板块**:`get_boards`、`get_board_members`、`get_board_of`、`get_board_summary`、
|
- **板块**:`get_boards`、`get_board_members`、`get_board_of`、`get_board_summary`、
|
||||||
@@ -69,24 +59,6 @@ asyncio.run(main())
|
|||||||
> 旧的分通道/市场方法(`get_kline` / `get_transactions` / `get_goods_*` 等共 33 个)保留为
|
> 旧的分通道/市场方法(`get_kline` / `get_transactions` / `get_goods_*` 等共 33 个)保留为
|
||||||
> deprecated 别名,1.0 移除;完整对照见[接口重构审查](https://openbot-coder.github.io/pytdxdata/refactor-audit/)与 CHANGELOG。
|
> deprecated 别名,1.0 移除;完整对照见[接口重构审查](https://openbot-coder.github.io/pytdxdata/refactor-audit/)与 CHANGELOG。
|
||||||
|
|
||||||
## DataFrame 输出(v0.7+)
|
|
||||||
|
|
||||||
表格型接口可以直出 pandas / polars DataFrame,开关只有构造参数 `df`:
|
|
||||||
|
|
||||||
```python
|
|
||||||
from pytdxdata import TdxData
|
|
||||||
|
|
||||||
async with TdxData(df="polars") as td: # 或 df="pandas";df=None(默认)= list[dataclass]
|
|
||||||
df = await td.get_bars("sh600000", count=240)
|
|
||||||
```
|
|
||||||
|
|
||||||
- 需先装 extra:`pip install "pytdxdata[polars]"` / `[pandas]` / `[dataframe]`。**核心包仍零依赖**。
|
|
||||||
- `df=None` 时行为与 0.6.x 完全一致(装饰器直接透传,零开销);`df` 非法抛 `ValueError`,包没装抛 `FrameBackendError`(带安装命令)。
|
|
||||||
- 列口径:**时间列替换**拆开的 `year/month/day/hour/minute` → `datetime`(+ `trading_date`);`bid`/`ask` 五档展开成 10 列;`fields` 字典 JSON 化成字符串列(空字典给 `None`);`TdxBlock.codes` 保持列表。
|
|
||||||
- `get_indicators` → bars + 指标合并成一张表;`get_index_trade_stats(_multi)` → `minutes` 展开并补 `index`/`date` 常量列;`get_kline_batch`/`get_minute_batch` → 各标的纵向拼接成一张表。
|
|
||||||
- **不受影响**:`bytes`/`int`/`dict`/单对象/`tuple`/`list[float]`/`list[str]` 类接口原样返回(如 `get_file`、`get_security_count`、`get_server_info`)。
|
|
||||||
- 空结果返回**列名齐全的空表**,但 dtype 只由后端推断。
|
|
||||||
|
|
||||||
## 服务器与配置
|
## 服务器与配置
|
||||||
|
|
||||||
服务器主机来源有一条优先级链(各协议独立):环境变量 → **当日测速缓存** → 包内清单
|
服务器主机来源有一条优先级链(各协议独立):环境变量 → **当日测速缓存** → 包内清单
|
||||||
|
|||||||
@@ -47,7 +47,6 @@ nav:
|
|||||||
- 安装与配置: getting-started.md
|
- 安装与配置: getting-started.md
|
||||||
- 场景示例: cookbook.md
|
- 场景示例: cookbook.md
|
||||||
- 命令行工具: cli.md
|
- 命令行工具: cli.md
|
||||||
- DataFrame 输出: dataframe.md
|
|
||||||
- API 参考:
|
- API 参考:
|
||||||
- 接口总览: api/index.md
|
- 接口总览: api/index.md
|
||||||
- TdxData 方法: api/tdxdata.md
|
- TdxData 方法: api/tdxdata.md
|
||||||
|
|||||||
+23
-12
@@ -1,14 +1,24 @@
|
|||||||
[project]
|
[project]
|
||||||
name = "pytdxdata"
|
name = "pytdxdata"
|
||||||
version = "0.7.0"
|
version = "0.6.7"
|
||||||
description = "通达信(TongDaXin)行情数据客户端:动态连接池 + 双层TTL缓存 + 3连接并发分页,纯asyncio零运行时依赖,支持A股/港股/美股/期货/期权"
|
description = "通达信(TongDaXin)行情数据客户端:动态连接池 + 双层TTL缓存 + 3连接并发分页,纯asyncio零运行时依赖,支持A股/港股/美股/期货/期权"
|
||||||
readme = "README.md"
|
readme = "README.md"
|
||||||
requires-python = ">=3.11"
|
requires-python = ">=3.11"
|
||||||
license = "MIT"
|
license = "MIT"
|
||||||
license-files = ["LICENSE"]
|
license-files = ["LICENSE"]
|
||||||
keywords = ["tdx", "trading", "stock", "kline", "quote", "通达信", "A股", "港股", "美股", "期货", "asyncio", "quant"]
|
keywords = [
|
||||||
authors = [
|
"tdx",
|
||||||
{ name = "openbot-coder", email = "openbot-coder@users.noreply.github.com" }
|
"trading",
|
||||||
|
"stock",
|
||||||
|
"kline",
|
||||||
|
"quote",
|
||||||
|
"通达信",
|
||||||
|
"A股",
|
||||||
|
"港股",
|
||||||
|
"美股",
|
||||||
|
"期货",
|
||||||
|
"asyncio",
|
||||||
|
"quant",
|
||||||
]
|
]
|
||||||
classifiers = [
|
classifiers = [
|
||||||
"Development Status :: 4 - Beta",
|
"Development Status :: 4 - Beta",
|
||||||
@@ -24,6 +34,10 @@ classifiers = [
|
|||||||
]
|
]
|
||||||
dependencies = []
|
dependencies = []
|
||||||
|
|
||||||
|
[[project.authors]]
|
||||||
|
name = "openbot-coder"
|
||||||
|
email = "openbot-coder@users.noreply.github.com"
|
||||||
|
|
||||||
[project.urls]
|
[project.urls]
|
||||||
Homepage = "https://github.com/openbot-coder/pytdxdata"
|
Homepage = "https://github.com/openbot-coder/pytdxdata"
|
||||||
Repository = "https://github.com/openbot-coder/pytdxdata"
|
Repository = "https://github.com/openbot-coder/pytdxdata"
|
||||||
@@ -31,10 +45,11 @@ Documentation = "https://openbot-coder.github.io/pytdxdata/"
|
|||||||
|
|
||||||
[project.optional-dependencies]
|
[project.optional-dependencies]
|
||||||
uvloop = ["uvloop>=0.21"]
|
uvloop = ["uvloop>=0.21"]
|
||||||
pandas = ["pandas>=2.0"]
|
dev = [
|
||||||
polars = ["polars>=1.0"]
|
"pytest>=8.0",
|
||||||
dataframe = ["pandas>=2.0", "polars>=1.0"]
|
"pytest-asyncio>=0.24",
|
||||||
dev = ["pytest>=8.0", "pytest-asyncio>=0.24", "ruff>=0.6"]
|
"ruff>=0.6",
|
||||||
|
]
|
||||||
|
|
||||||
[project.scripts]
|
[project.scripts]
|
||||||
tdx = "pytdxdata.cli:main"
|
tdx = "pytdxdata.cli:main"
|
||||||
@@ -55,9 +70,6 @@ testpaths = ["tests"]
|
|||||||
dev = [
|
dev = [
|
||||||
"pytest>=9.1.1",
|
"pytest>=9.1.1",
|
||||||
"pytest-asyncio>=1.4.0",
|
"pytest-asyncio>=1.4.0",
|
||||||
# DataFrame 后端的双实现都要在本地/CI 里被真实覆盖,不能只在 extras 里声明
|
|
||||||
"pandas>=2.0",
|
|
||||||
"polars>=1.0",
|
|
||||||
]
|
]
|
||||||
docs = [
|
docs = [
|
||||||
"mkdocs-material>=9.5",
|
"mkdocs-material>=9.5",
|
||||||
@@ -65,4 +77,3 @@ docs = [
|
|||||||
"ruff>=0.6",
|
"ruff>=0.6",
|
||||||
"jieba>=0.42",
|
"jieba>=0.42",
|
||||||
]
|
]
|
||||||
|
|
||||||
@@ -1,108 +0,0 @@
|
|||||||
"""可复现核查:指数成分逐笔汇总的口径是否与官方日线一致。
|
|
||||||
|
|
||||||
用法::
|
|
||||||
|
|
||||||
python scripts/verify_index_stats.py # 当日
|
|
||||||
python scripts/verify_index_stats.py --date 20261021 # 历史某日
|
|
||||||
python scripts/verify_index_stats.py --indices 000001 000300
|
|
||||||
|
|
||||||
两段校验:
|
|
||||||
|
|
||||||
1. **单股级**:``Σ(逐笔价 × 手 × 100)`` 对**当日日线 amount**。比值 1.0000~1.0002
|
|
||||||
说明逐笔没漏条、价格与单位都对(``bj`` 也行)。
|
|
||||||
2. **指数级**:成分股合计成交额对**该指数的日线 amount**。相等说明成分清单与
|
|
||||||
指数的量能口径一致;明显偏小则是成分表滞后(新上市未纳入)。
|
|
||||||
|
|
||||||
⚠️ **深市 `399xxx` 的指数 amount 是「全板块」口径**,不是样本股:实测
|
|
||||||
`sz399001` 5081.40 亿 = 深市 2908 只股票合计,`sz399006` 2568.60 亿 = 创业板
|
|
||||||
全部股票合计。所以这两个指数的比值天然远离 1(约 0.43),**不是数据错**。
|
|
||||||
|
|
||||||
为什么不用快照做基准:快照只有「此刻」,历史日期对不上。日线 amount 两者都能给。
|
|
||||||
|
|
||||||
注意 ``bj``(北交所)逐笔**没有笔数字段**(``trade_count`` 恒 0),脚本会打出
|
|
||||||
``有笔数`` 列便于识别。
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import argparse
|
|
||||||
import asyncio
|
|
||||||
from datetime import date as _date
|
|
||||||
|
|
||||||
|
|
||||||
def _ymd(bar) -> int:
|
|
||||||
return bar.year * 10000 + bar.month * 100 + bar.day
|
|
||||||
|
|
||||||
|
|
||||||
async def _daily_amount(td, symbol: str, ymd: int) -> float | None:
|
|
||||||
from pytdxdata import KlinePeriod
|
|
||||||
|
|
||||||
bars = await td.get_bars(symbol, KlinePeriod.DAY, count=10)
|
|
||||||
for b in bars:
|
|
||||||
if _ymd(b) == ymd:
|
|
||||||
return b.amount
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
async def check_symbols(td, symbols: list[str], ymd: int) -> None:
|
|
||||||
print(f"## 单股级({ymd})\n")
|
|
||||||
print("| 标的 | Σ(p×手×100) / 日线amount | 笔数 |")
|
|
||||||
print("|---|---|---|")
|
|
||||||
for sym in symbols:
|
|
||||||
try:
|
|
||||||
ticks = await td.get_ticks(sym, date=ymd)
|
|
||||||
ref = await _daily_amount(td, sym, ymd)
|
|
||||||
except Exception as exc: # noqa: BLE001
|
|
||||||
print(f"| {sym} | ERR {type(exc).__name__} | - |")
|
|
||||||
continue
|
|
||||||
if not ticks:
|
|
||||||
print(f"| {sym} | 无逐笔(非交易日?) | - |")
|
|
||||||
continue
|
|
||||||
sa = sum(t.price * t.vol * 100 for t in ticks)
|
|
||||||
tc = sum(t.trade_count for t in ticks)
|
|
||||||
ratio = sa / ref if ref else 0.0
|
|
||||||
print(f"| {sym} | {ratio:.4f} | {tc:,} |")
|
|
||||||
print()
|
|
||||||
|
|
||||||
|
|
||||||
async def check_indices(td, indices: list[str], ymd: int, day: int | None) -> None:
|
|
||||||
print(f"## 指数级({ymd})\n")
|
|
||||||
print("| 指数 | 成分 | 有成交 | 有笔数 | 成分合计 | 日线amount | 比值 | 耗时 |")
|
|
||||||
print("|---|---|---|---|---|---|---|---|")
|
|
||||||
for idx in indices:
|
|
||||||
try:
|
|
||||||
st = await td.get_index_trade_stats(idx, day)
|
|
||||||
ref = await _daily_amount(td, st.index, ymd)
|
|
||||||
except Exception as exc: # noqa: BLE001
|
|
||||||
print(f"| {idx} | ERR {type(exc).__name__}: {exc} | - | - | - | - | - | - |")
|
|
||||||
continue
|
|
||||||
ratio = st.total_amount / ref if ref else 0.0
|
|
||||||
print(f"| {st.index} | {st.member_count} | {st.active_members} | "
|
|
||||||
f"{st.trade_count_members} | {st.total_amount / 1e8:,.2f} 亿 | "
|
|
||||||
f"{(ref or 0) / 1e8:,.2f} 亿 | {ratio:.4f} | {st.elapsed:.1f}s |")
|
|
||||||
print()
|
|
||||||
print("提示:`399xxx` 的比值天然远离 1 —— 深市指数的 amount 字段是全板块口径。")
|
|
||||||
print()
|
|
||||||
|
|
||||||
|
|
||||||
async def main() -> None:
|
|
||||||
ap = argparse.ArgumentParser(description=__doc__)
|
|
||||||
ap.add_argument("--symbols", nargs="*",
|
|
||||||
default=["sh600000", "sz000001", "sh601398"])
|
|
||||||
ap.add_argument("--indices", nargs="*", default=["000016", "000300"])
|
|
||||||
ap.add_argument("--date", default=None, help="YYYYMMDD(缺省=当日)")
|
|
||||||
ap.add_argument("--skip-symbols", action="store_true")
|
|
||||||
args = ap.parse_args()
|
|
||||||
|
|
||||||
from pytdxdata import TdxData
|
|
||||||
|
|
||||||
ymd = int(args.date) if args.date else int(_date.today().strftime("%Y%m%d"))
|
|
||||||
day = int(args.date) if args.date else None
|
|
||||||
async with TdxData(cache_dir=None, timeout=10.0) as td:
|
|
||||||
if not args.skip_symbols:
|
|
||||||
await check_symbols(td, args.symbols, ymd)
|
|
||||||
if args.indices:
|
|
||||||
await check_indices(td, args.indices, ymd, day)
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
asyncio.run(main())
|
|
||||||
@@ -18,7 +18,7 @@ from .models import (
|
|||||||
XdxrRecord,
|
XdxrRecord,
|
||||||
)
|
)
|
||||||
|
|
||||||
__version__ = "0.7.0"
|
__version__ = "0.6.7"
|
||||||
__all__ = [
|
__all__ = [
|
||||||
"TdxData",
|
"TdxData",
|
||||||
"Market",
|
"Market",
|
||||||
|
|||||||
+20
-450
@@ -27,7 +27,7 @@ import time
|
|||||||
import warnings
|
import warnings
|
||||||
from collections import deque
|
from collections import deque
|
||||||
from collections.abc import Callable
|
from collections.abc import Callable
|
||||||
from datetime import date as _date, datetime, timedelta
|
from datetime import date as _date, datetime
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Sequence
|
from typing import Sequence
|
||||||
|
|
||||||
@@ -38,7 +38,6 @@ from .cache.manager import CacheManager
|
|||||||
from .cache.memory import MemoryCache
|
from .cache.memory import MemoryCache
|
||||||
from .codec.payload import pack_payload, unpack_payload
|
from .codec.payload import pack_payload, unpack_payload
|
||||||
from .config import EX_HANDSHAKES, get_ex_hosts, get_mac_hosts, get_standard_hosts
|
from .config import EX_HANDSHAKES, get_ex_hosts, get_mac_hosts, get_standard_hosts
|
||||||
from .frames import framed, get_backend
|
|
||||||
from .models import (
|
from .models import (
|
||||||
Adjust,
|
Adjust,
|
||||||
AuctionItem,
|
AuctionItem,
|
||||||
@@ -146,51 +145,6 @@ _KLINE_MAX_BARS: dict[KlinePeriod, int] = {
|
|||||||
KlinePeriod.MONTH: 500,
|
KlinePeriod.MONTH: 500,
|
||||||
}
|
}
|
||||||
|
|
||||||
# 每交易日大约多少根(倒推时按时间跨度估要拉多少,估小了会多跑一轮)
|
|
||||||
_BARS_PER_DAY: dict[KlinePeriod, float] = {
|
|
||||||
KlinePeriod.MIN_1: 240,
|
|
||||||
KlinePeriod.MIN_5: 48,
|
|
||||||
KlinePeriod.MIN_15: 16,
|
|
||||||
KlinePeriod.MIN_30: 8,
|
|
||||||
KlinePeriod.MIN_60: 4,
|
|
||||||
KlinePeriod.DAY: 1,
|
|
||||||
KlinePeriod.WEEK: 0.2,
|
|
||||||
KlinePeriod.MONTH: 0.05,
|
|
||||||
}
|
|
||||||
_BACKTRACK_MIN_BARS = 240 # 倒推起始根数:一天 1 分钟正好 240,一次命中
|
|
||||||
_BACKTRACK_MAX_BARS = 20_000 # 单次别拉太多;超了回落到二分
|
|
||||||
_BACKTRACK_ROUNDS = 5 # 最多放大 5 轮(240→7680)
|
|
||||||
_BACKTRACK_GRACE = timedelta(days=10) # 右端离最新多远还算「贴着最新」(覆盖长假)
|
|
||||||
|
|
||||||
# 一根K线跨多长时间。K线时间戳标的是**该周期的结束时刻**,所以最新一根可能
|
|
||||||
# 比 datetime.now() 略大(14:29 那根此刻还在走)——判断「右端贴最新」要给这个容差。
|
|
||||||
_BAR_SPAN: dict[KlinePeriod, timedelta] = {
|
|
||||||
KlinePeriod.MIN_1: timedelta(minutes=1),
|
|
||||||
KlinePeriod.MIN_5: timedelta(minutes=5),
|
|
||||||
KlinePeriod.MIN_15: timedelta(minutes=15),
|
|
||||||
KlinePeriod.MIN_30: timedelta(minutes=30),
|
|
||||||
KlinePeriod.MIN_60: timedelta(hours=1),
|
|
||||||
KlinePeriod.DAY: timedelta(days=1),
|
|
||||||
KlinePeriod.WEEK: timedelta(days=7),
|
|
||||||
KlinePeriod.MONTH: timedelta(days=31),
|
|
||||||
}
|
|
||||||
|
|
||||||
# ---- K线批量流水线(get_bars 多标的路径)----
|
|
||||||
# 单只K线是「一请求一标的」,全市场 5,000 只就是 5,000 次往返。实测单连接
|
|
||||||
# 串行 33ms/只 ≈ 1 个 RTT,**瓶颈是往返不是带宽**;同连接连续下发 pipe 条请求
|
|
||||||
# (协议帧自描述 + 服务器按序应答)能把往返折叠掉。实测 24 台 × 8 流水:
|
|
||||||
# 全市场 1 分钟K线取当天 43.6s → 5.1s。
|
|
||||||
_BULK_MIN_SYMS = 16 # 低于此数量走逐只路径(流水线收益抵不过批量开销)
|
|
||||||
_BULK_PIPE_MIN = 8 # 自适应深度下限
|
|
||||||
_BULK_PIPE_MAX = 32 # 自适应深度上限(再深服务器会攒够一批才回,反而慢)
|
|
||||||
_BULK_MAX_COUNT = 700 # MAC 单请求返回上限;超过只能逐只分页
|
|
||||||
# 最多用几台服务器:池里 24 台全用反而更慢 —— 慢节点会成为长尾(最长的那台
|
|
||||||
# 拖住整轮),实测 count=2 全市场 24 台 2.95s / 只取最快 12 台 1.45s。
|
|
||||||
_BULK_MAX_CONNS = 12
|
|
||||||
# 交易时段(分钟):上午 9:30-11:30、下午 13:00-15:00,共 240 分钟
|
|
||||||
_AM_OPEN, _AM_CLOSE = 9 * 60 + 30, 11 * 60 + 30
|
|
||||||
_PM_OPEN, _PM_CLOSE = 13 * 60, 15 * 60
|
|
||||||
|
|
||||||
# 自定义字段报价(0x122B/0x122C)基础字段位:昨收/开/高/低/现价/量/量比/额
|
# 自定义字段报价(0x122B/0x122C)基础字段位:昨收/开/高/低/现价/量/量比/额
|
||||||
_QUOTE_BITS = [0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07]
|
_QUOTE_BITS = [0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07]
|
||||||
_QUOTE_PAGE = 80 # 0x122B/0x122C 单次上限 80 只
|
_QUOTE_PAGE = 80 # 0x122B/0x122C 单次上限 80 只
|
||||||
@@ -201,76 +155,11 @@ _EX_TICK_PAGE = 1800 # 港股逐笔单页
|
|||||||
_ALL_PAGES_CAP = 50 # count=None(全量)时的翻页上限
|
_ALL_PAGES_CAP = 50 # count=None(全量)时的翻页上限
|
||||||
# 集合竞价**委托**记录标记(0x122F 第 5 字段)。不是成交,见 _ticks_one 注释。
|
# 集合竞价**委托**记录标记(0x122F 第 5 字段)。不是成交,见 _ticks_one 注释。
|
||||||
_BS_AUCTION_ORDER = 8
|
_BS_AUCTION_ORDER = 8
|
||||||
# 证券清单(0x053E)拿不到的市场。实测 27 台标准主机对 market=2(北交所)**全部
|
|
||||||
# 不响应**(26 台 6s 无回包、1 台立刻拒连),而同 market 的 0x053D 数量接口正常
|
|
||||||
# (返回 389)。故清单直接返回空,避免每次调用白等数秒再抛 ConnectionError。
|
|
||||||
# 北交所**标的本身没问题**:报价/K线/逐笔用 ``bj920992`` 这类代码都正常。
|
|
||||||
_LIST_UNSUPPORTED_MARKETS = frozenset({int(Market.BJ)})
|
|
||||||
_list_unsupported_warned: set[int] = set() # 同一市场只 WARNING 一次
|
|
||||||
|
|
||||||
# 港股系市场:逐笔走 EX 专用 16B 协议(0x23FC/0x2406),其余扩展市场走 MAC 0x122F
|
# 港股系市场:逐笔走 EX 专用 16B 协议(0x23FC/0x2406),其余扩展市场走 MAC 0x122F
|
||||||
_HK_MARKETS = frozenset({27, 31, 48, 49, 71, 98})
|
_HK_MARKETS = frozenset({27, 31, 48, 49, 71, 98})
|
||||||
|
|
||||||
|
|
||||||
def _elapsed_trading_minutes(now: datetime) -> int:
|
|
||||||
"""今天到现在为止已过去的**交易分钟数**(0~240)。
|
|
||||||
|
|
||||||
只按钟点算,不区分节假日(非交易日时区间裁剪会把这些根全滤掉,不影响正确性);
|
|
||||||
盘中调用是常态,10:00 只需 30 根而不是 240 根 —— 数据量直接差 8 倍。
|
|
||||||
"""
|
|
||||||
m = now.hour * 60 + now.minute
|
|
||||||
if m < _AM_OPEN:
|
|
||||||
return 0
|
|
||||||
if m <= _AM_CLOSE:
|
|
||||||
return m - _AM_OPEN
|
|
||||||
if m < _PM_OPEN:
|
|
||||||
return 120 # 午休:上午已走完
|
|
||||||
if m <= _PM_CLOSE:
|
|
||||||
return 120 + (m - _PM_OPEN)
|
|
||||||
return 240 # 收盘后
|
|
||||||
|
|
||||||
|
|
||||||
def _bars_for_range(period: KlinePeriod, start_dt: datetime,
|
|
||||||
now: datetime) -> int:
|
|
||||||
"""估算「从 ``start_dt`` 到最新」需要拉多少根(倒推初值)。"""
|
|
||||||
per_day = _BARS_PER_DAY.get(period, 1)
|
|
||||||
if start_dt.date() == now.date():
|
|
||||||
# 当天:按已过交易分钟数折算,别把一整天都拉下来(10:00 只需 30 根而非 240)
|
|
||||||
frac = _elapsed_trading_minutes(now) / 240
|
|
||||||
return max(1, int(per_day * frac) + 5)
|
|
||||||
days = max(0.0, (now - start_dt).total_seconds() / 86400.0)
|
|
||||||
return max(1, int(days * per_day * 1.6))
|
|
||||||
|
|
||||||
|
|
||||||
def _auto_pipeline(count: int, pipeline: int | None) -> int:
|
|
||||||
"""流水深度:默认按「在途字节数 ≈60KB」自适应,夹在 8~32。
|
|
||||||
|
|
||||||
轮数 = 请求数 / (连接数 × 深度),深度越深往返折叠得越彻底;但请求太深时
|
|
||||||
服务器会把整批处理完才回,实测 24 台 depth=64 反而比 8 慢一倍(5.79s vs 2.95s)。
|
|
||||||
每根的应答固定 32 字节,所以「取 240 根」天然只用得着浅流水,
|
|
||||||
「只取最新 2 根」则要用满深流水。
|
|
||||||
"""
|
|
||||||
if pipeline is not None:
|
|
||||||
return max(2, int(pipeline))
|
|
||||||
per_req = max(1, count * 32 + 33)
|
|
||||||
return max(_BULK_PIPE_MIN, min(_BULK_PIPE_MAX, 60_000 // per_req))
|
|
||||||
|
|
||||||
|
|
||||||
def _norm_range(start_date, end_date) -> tuple[datetime | None, datetime | None]:
|
|
||||||
"""日期参数归一化:``date`` → 当日 00:00 / 23:59:59.999999。"""
|
|
||||||
start_dt = None
|
|
||||||
if start_date is not None:
|
|
||||||
start_dt = (start_date if isinstance(start_date, datetime)
|
|
||||||
else datetime.combine(start_date, datetime.min.time()))
|
|
||||||
end_dt = None
|
|
||||||
if end_date is not None:
|
|
||||||
end_dt = (end_date if isinstance(end_date, datetime)
|
|
||||||
else datetime.combine(end_date, datetime.max.time()))
|
|
||||||
if end_dt.time() == datetime.min.time():
|
|
||||||
end_dt = end_dt.replace(hour=23, minute=59, second=59, microsecond=999999)
|
|
||||||
return start_dt, end_dt
|
|
||||||
|
|
||||||
|
|
||||||
def _fields_tag(bits: Sequence[int] | None) -> str:
|
def _fields_tag(bits: Sequence[int] | None) -> str:
|
||||||
"""字段位集合 -> 缓存键片段。"""
|
"""字段位集合 -> 缓存键片段。"""
|
||||||
if not bits:
|
if not bits:
|
||||||
@@ -354,19 +243,7 @@ class TdxData:
|
|||||||
mac_pool_cap: int = 4,
|
mac_pool_cap: int = 4,
|
||||||
std_pool_cap: int = 4,
|
std_pool_cap: int = 4,
|
||||||
ex_pool_cap: int = 2,
|
ex_pool_cap: int = 2,
|
||||||
df: str | None = None,
|
|
||||||
) -> None:
|
) -> None:
|
||||||
# df=None(默认)保持 0.6.x 的行为:表格型方法一律返回 list[dataclass]。
|
|
||||||
# df="polars" / df="pandas" 时这些方法直接返回对应后端的 DataFrame。
|
|
||||||
# 这里早失败一次:名字非法抛 ValueError,包没装抛 FrameBackendError(带安装命令)。
|
|
||||||
if df is not None:
|
|
||||||
get_backend(df)
|
|
||||||
self._df_backend = df
|
|
||||||
|
|
||||||
# K线偏移方向是**通道级属性**(不是每只标的的属性),首次探测后实例内复用:
|
|
||||||
# 全市场 5,000 只各探 2 次往返 = 上万次请求,纯属浪费。
|
|
||||||
self._kline_dir_cache: dict[str, bool] = {}
|
|
||||||
|
|
||||||
self.port = port
|
self.port = port
|
||||||
self.timeout = timeout
|
self.timeout = timeout
|
||||||
self.default_adjust = default_adjust
|
self.default_adjust = default_adjust
|
||||||
@@ -443,7 +320,6 @@ class TdxData:
|
|||||||
# 一、标的清单
|
# 一、标的清单
|
||||||
# ================================================================== #
|
# ================================================================== #
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_universe(self, market: str | None = None) -> list[SecurityInfo]:
|
async def get_universe(self, market: str | None = None) -> list[SecurityInfo]:
|
||||||
"""标的清单(A股证券列表 + 扩展市场商品列表),一次覆盖所有市场。
|
"""标的清单(A股证券列表 + 扩展市场商品列表),一次覆盖所有市场。
|
||||||
|
|
||||||
@@ -455,22 +331,11 @@ class TdxData:
|
|||||||
返回:
|
返回:
|
||||||
``list[SecurityInfo]``,每条带 ``symbol`` 规范化标的串与 ``kind`` 分类描述。
|
``list[SecurityInfo]``,每条带 ``symbol`` 规范化标的串与 ``kind`` 分类描述。
|
||||||
EX 通道无昨收/单位,对应字段为 0。
|
EX 通道无昨收/单位,对应字段为 0。
|
||||||
|
|
||||||
注意:
|
|
||||||
**北交所(``"bj"``)清单返回空列表**:清单命令 0x053E 对 market=2
|
|
||||||
在所有实测服务器上都不响应(27 台全测),协议层根本拿不到。
|
|
||||||
但北交所**标的本身可用** —— 报价/K线/逐笔直接传 ``bj920992`` 这类代码
|
|
||||||
都正常,只是无法枚举全量代码。``market="all"`` 时该市场被跳过,不会报错。
|
|
||||||
"""
|
"""
|
||||||
out: list[SecurityInfo] = []
|
out: list[SecurityInfo] = []
|
||||||
ex_items: list[SecurityInfo] | None = None
|
|
||||||
for mkt, channel in market_targets(market):
|
for mkt, channel in market_targets(market):
|
||||||
if channel == "ex":
|
if channel == "ex":
|
||||||
# EX 全量只取一次:命中缓存也要反序列化 10 万条(~200ms),
|
out.extend(await self._universe_ex(mkt))
|
||||||
# 十几个 EX 市场各来一次纯属白烧。
|
|
||||||
if ex_items is None:
|
|
||||||
ex_items = await self._ex_all()
|
|
||||||
out.extend(it for it in ex_items if it.market == mkt)
|
|
||||||
else:
|
else:
|
||||||
out.extend(await self._universe_cn(mkt))
|
out.extend(await self._universe_cn(mkt))
|
||||||
return out
|
return out
|
||||||
@@ -509,14 +374,6 @@ class TdxData:
|
|||||||
return decimal_sync.status()
|
return decimal_sync.status()
|
||||||
|
|
||||||
async def _universe_cn(self, market: int) -> list[SecurityInfo]:
|
async def _universe_cn(self, market: int) -> list[SecurityInfo]:
|
||||||
if market in _LIST_UNSUPPORTED_MARKETS:
|
|
||||||
if market not in _list_unsupported_warned:
|
|
||||||
_list_unsupported_warned.add(market)
|
|
||||||
log.warning(
|
|
||||||
"市场 %s 的证券清单接口协议不支持(0x053E 无响应),返回空列表;"
|
|
||||||
"该市场标的的报价/K线/逐笔不受影响", market)
|
|
||||||
return []
|
|
||||||
|
|
||||||
key = CacheKey("list", market)
|
key = CacheKey("list", market)
|
||||||
|
|
||||||
async def fetch() -> bytes:
|
async def fetch() -> bytes:
|
||||||
@@ -545,18 +402,13 @@ class TdxData:
|
|||||||
|
|
||||||
async def fetch() -> bytes:
|
async def fetch() -> bytes:
|
||||||
total = int(await self._execute_ex(GetExInstrumentCountCmd()))
|
total = int(await self._execute_ex(GetExInstrumentCountCmd()))
|
||||||
starts = list(range(0, total, _EX_PAGE))
|
|
||||||
sem = asyncio.Semaphore(8) # 实测 8 页并发 292ms(串行 110 页 ~4s)
|
|
||||||
|
|
||||||
async def one(pos: int) -> list[dict]:
|
|
||||||
async with sem:
|
|
||||||
return await self._execute_ex(
|
|
||||||
GetExInstrumentInfoCmd(0, pos, _EX_PAGE))
|
|
||||||
|
|
||||||
items: list[SecurityInfo] = []
|
items: list[SecurityInfo] = []
|
||||||
for page in await asyncio.gather(*(one(s) for s in starts)):
|
pos = 0
|
||||||
|
while pos < total:
|
||||||
|
page = await self._execute_ex(
|
||||||
|
GetExInstrumentInfoCmd(0, pos, _EX_PAGE))
|
||||||
if not page:
|
if not page:
|
||||||
continue
|
break
|
||||||
for g in page:
|
for g in page:
|
||||||
mkt = int(g["market"])
|
mkt = int(g["market"])
|
||||||
code = str(g["code"]).strip()
|
code = str(g["code"]).strip()
|
||||||
@@ -570,6 +422,7 @@ class TdxData:
|
|||||||
symbol=format_symbol(mkt, code),
|
symbol=format_symbol(mkt, code),
|
||||||
kind=str(g.get("desc", "")).strip(),
|
kind=str(g.get("desc", "")).strip(),
|
||||||
))
|
))
|
||||||
|
pos += len(page)
|
||||||
return pack_payload(items)
|
return pack_payload(items)
|
||||||
|
|
||||||
return unpack_payload(await self._cache.get_or_fetch(key, fetch))
|
return unpack_payload(await self._cache.get_or_fetch(key, fetch))
|
||||||
@@ -590,7 +443,6 @@ class TdxData:
|
|||||||
# 二、报价
|
# 二、报价
|
||||||
# ================================================================== #
|
# ================================================================== #
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_quotes(self, symbols: str | Sequence[str], *,
|
async def get_quotes(self, symbols: str | Sequence[str], *,
|
||||||
fields: Sequence[int] | None = None,
|
fields: Sequence[int] | None = None,
|
||||||
concurrency: int | None = None,
|
concurrency: int | None = None,
|
||||||
@@ -768,7 +620,6 @@ class TdxData:
|
|||||||
await self._std_pool.release(pc)
|
await self._std_pool.release(pc)
|
||||||
return rows
|
return rows
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_snapshot(self, symbols: str | Sequence[str]) -> list[SymbolSnapshot]:
|
async def get_snapshot(self, symbols: str | Sequence[str]) -> list[SymbolSnapshot]:
|
||||||
"""个股特征快照(当日/换手/活跃度等,仅 A股)。"""
|
"""个股特征快照(当日/换手/活跃度等,仅 A股)。"""
|
||||||
syms = parse_symbols(symbols)
|
syms = parse_symbols(symbols)
|
||||||
@@ -788,7 +639,6 @@ class TdxData:
|
|||||||
out.append(rec)
|
out.append(rec)
|
||||||
return out
|
return out
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_auction(self, symbols: str | Sequence[str]) -> list[AuctionItem]:
|
async def get_auction(self, symbols: str | Sequence[str]) -> list[AuctionItem]:
|
||||||
"""集合竞价(仅 A股)。"""
|
"""集合竞价(仅 A股)。"""
|
||||||
syms = parse_symbols(symbols)
|
syms = parse_symbols(symbols)
|
||||||
@@ -801,7 +651,6 @@ class TdxData:
|
|||||||
out.extend(items)
|
out.extend(items)
|
||||||
return out
|
return out
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_capital_flow(self, symbols: str | Sequence[str]) -> list[CapitalFlow]:
|
async def get_capital_flow(self, symbols: str | Sequence[str]) -> list[CapitalFlow]:
|
||||||
"""个股资金流向(仅 A股)。"""
|
"""个股资金流向(仅 A股)。"""
|
||||||
syms = parse_symbols(symbols)
|
syms = parse_symbols(symbols)
|
||||||
@@ -818,7 +667,6 @@ class TdxData:
|
|||||||
# 三、K线
|
# 三、K线
|
||||||
# ================================================================== #
|
# ================================================================== #
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_bars(
|
async def get_bars(
|
||||||
self,
|
self,
|
||||||
symbols: str | Sequence[str],
|
symbols: str | Sequence[str],
|
||||||
@@ -830,7 +678,6 @@ class TdxData:
|
|||||||
start_date: datetime | None = None,
|
start_date: datetime | None = None,
|
||||||
end_date: datetime | None = None,
|
end_date: datetime | None = None,
|
||||||
concurrency: int = 6,
|
concurrency: int = 6,
|
||||||
pipeline: int | None = None,
|
|
||||||
) -> list[SecurityBar]:
|
) -> list[SecurityBar]:
|
||||||
"""K线(多标的并发,自动判指数、自动复权选路、自动跨市场)。
|
"""K线(多标的并发,自动判指数、自动复权选路、自动跨市场)。
|
||||||
|
|
||||||
@@ -842,30 +689,12 @@ class TdxData:
|
|||||||
adjust: 复权方式(默认用实例 ``default_adjust``);``QFQ``/``HFQ`` 自动走 MAC 通道。
|
adjust: 复权方式(默认用实例 ``default_adjust``);``QFQ``/``HFQ`` 自动走 MAC 通道。
|
||||||
start_date / end_date: 按日期区间取(与 start/count 互斥,优先日期过滤)。
|
start_date / end_date: 按日期区间取(与 start/count 互斥,优先日期过滤)。
|
||||||
只给 ``start_date`` 即增量更新语义(取该日期之后的全部K线)。
|
只给 ``start_date`` 即增量更新语义(取该日期之后的全部K线)。
|
||||||
concurrency: 逐只路径的并发数(仅在对小批量/标准通道生效时使用)。
|
|
||||||
pipeline: **批量路径**每条连接同时在途的请求数。``None``(默认)按单请求
|
|
||||||
字节数自适应(约 60KB 在途,夹在 8~32);传 ``1`` 关闭流水线走逐只。
|
|
||||||
批量路径把 N 次往返折叠成 ``N/pipeline`` 次 —— 全市场 5,000 只取当天
|
|
||||||
1 分钟K线从 43.6s 降到 5s 级。
|
|
||||||
|
|
||||||
返回:
|
返回:
|
||||||
``list[SecurityBar]``,按入参标的顺序拼接,每根带 ``symbol``;组内按时间正序。
|
``list[SecurityBar]``,按入参标的顺序拼接,每根带 ``symbol``;组内按时间正序。
|
||||||
|
|
||||||
说明:
|
|
||||||
批量路径只在 A股/指数(非扩展市场)且区间贴着最新时启用;历史中段
|
|
||||||
仍需逐只二分定位,会回落到逐只路径(结果一致,只是慢一些)。
|
|
||||||
"""
|
"""
|
||||||
period = KlinePeriod(int(period))
|
period = KlinePeriod(int(period))
|
||||||
syms = parse_symbols(symbols)
|
syms = parse_symbols(symbols)
|
||||||
|
|
||||||
# 批量路径:一次请求一标的,逐只走就是 N 次往返;多连接流水线能把往返压掉
|
|
||||||
if len(syms) >= _BULK_MIN_SYMS and pipeline != 1:
|
|
||||||
bulk = await self._bars_bulk(
|
|
||||||
syms, period, start=start, count=count, adjust=adjust,
|
|
||||||
start_date=start_date, end_date=end_date, pipeline=pipeline)
|
|
||||||
if bulk is not None:
|
|
||||||
return bulk
|
|
||||||
|
|
||||||
sem = asyncio.Semaphore(max(1, concurrency))
|
sem = asyncio.Semaphore(max(1, concurrency))
|
||||||
|
|
||||||
async def one(sym: Symbol) -> list[SecurityBar]:
|
async def one(sym: Symbol) -> list[SecurityBar]:
|
||||||
@@ -877,140 +706,6 @@ class TdxData:
|
|||||||
groups = await asyncio.gather(*(one(s) for s in syms))
|
groups = await asyncio.gather(*(one(s) for s in syms))
|
||||||
return [b for g in groups for b in g]
|
return [b for g in groups for b in g]
|
||||||
|
|
||||||
async def _bars_bulk(
|
|
||||||
self, syms: Sequence[Symbol], period: KlinePeriod, *,
|
|
||||||
start: int, count: int, adjust: Adjust | None,
|
|
||||||
start_date: datetime | None, end_date: datetime | None,
|
|
||||||
pipeline: int | None,
|
|
||||||
) -> list[SecurityBar] | None:
|
|
||||||
"""多标的批量取K线(多连接流水线)。返回 ``None`` = 不适用,调用方回退逐只。
|
|
||||||
|
|
||||||
不适用只有两种情况:含扩展市场标的、需要历史中段二分定位。
|
|
||||||
"""
|
|
||||||
if any(s.is_ex for s in syms):
|
|
||||||
return None
|
|
||||||
adj = self.default_adjust if adjust is None else adjust
|
|
||||||
pool, page = self._bulk_channel(period, adj)
|
|
||||||
|
|
||||||
def make_cmd(s: Symbol, o: int, c: int):
|
|
||||||
if pool is self._mac_pool:
|
|
||||||
return MacKlineCmd(s.market, s.code, period, adj, o, c)
|
|
||||||
if is_cn_index(s.market, s.code):
|
|
||||||
return GetIndexBarsCmd(s.market, s.code, period, o, c)
|
|
||||||
return GetSecurityBarsCmd(s.market, s.code, period, o, c)
|
|
||||||
|
|
||||||
if start_date is None and end_date is None:
|
|
||||||
pipe = _auto_pipeline(min(count, page), pipeline)
|
|
||||||
got = await self._bulk_bars(pool, make_cmd, syms, start, count, page, pipe,
|
|
||||||
period=period, adj=adj)
|
|
||||||
return [b for s in syms for b in (got.get(s.text) or [])]
|
|
||||||
|
|
||||||
start_dt, end_dt = _norm_range(start_date, end_date)
|
|
||||||
now = datetime.now()
|
|
||||||
if end_dt is not None and end_dt + _BACKTRACK_GRACE < now:
|
|
||||||
return None # 历史中段:得先探总根数再二分,批量帮不上
|
|
||||||
if start_dt is None:
|
|
||||||
need = _BACKTRACK_MIN_BARS
|
|
||||||
else:
|
|
||||||
if start_dt > now + _BAR_SPAN.get(period, timedelta(days=1)):
|
|
||||||
return [] # 整段区间在未来
|
|
||||||
need = _bars_for_range(period, start_dt, now)
|
|
||||||
# 偏移方向是通道级属性:探一次(结果按通道缓存,全市场只付一次代价)
|
|
||||||
if not await self._kline_newest_at_zero(syms[0], period, adj):
|
|
||||||
return None
|
|
||||||
|
|
||||||
pipe = _auto_pipeline(min(need, page), pipeline)
|
|
||||||
got = await self._bulk_bars(pool, make_cmd, syms, 0, need, page, pipe,
|
|
||||||
period=period, adj=adj)
|
|
||||||
out: list[SecurityBar] = []
|
|
||||||
for s in syms:
|
|
||||||
bars = got.get(s.text) or []
|
|
||||||
if start_dt is not None:
|
|
||||||
bars = [b for b in bars if b.datetime >= start_dt]
|
|
||||||
if end_dt is not None:
|
|
||||||
bars = [b for b in bars if b.datetime <= end_dt]
|
|
||||||
out.extend(bars)
|
|
||||||
return out
|
|
||||||
|
|
||||||
def _bulk_channel(self, period: KlinePeriod, adj: Adjust):
|
|
||||||
"""批量路径用哪个池 / 单请求最多几根。"""
|
|
||||||
if route_for("kline", adjust=adj, period=period) == "mac":
|
|
||||||
return self._mac_pool, _BULK_MAX_COUNT
|
|
||||||
return self._std_pool, KLINE_PAGE
|
|
||||||
|
|
||||||
async def _bulk_bars(
|
|
||||||
self, pool, make_cmd, syms: Sequence[Symbol],
|
|
||||||
start: int, count: int, page: int, pipeline: int, *,
|
|
||||||
period: KlinePeriod, adj: Adjust,
|
|
||||||
) -> dict[str, list[SecurityBar]]:
|
|
||||||
"""多连接 + 单连接流水线抓K线,按 ``symbol.text`` 归集。
|
|
||||||
|
|
||||||
为什么快:单只一请求,串行时候每只固定付一个 RTT(实测 33ms);同一连接
|
|
||||||
连续发 ``pipeline`` 条请求,服务器按序应答,于是 ``pipeline`` 只只共付
|
|
||||||
一个 RTT。再叠加多台服务器并行(每台 1 连接,尊重 per_server_cap),
|
|
||||||
实测全市场 1 分钟K线取当天:43.6s → 5s 级。
|
|
||||||
|
|
||||||
单请求装不下 ``start+count`` 时按 ``page`` 切页,多页与其他标的的请求
|
|
||||||
混在同一批里流水线发出去(不是逐只串行翻页)。
|
|
||||||
"""
|
|
||||||
plan: list[tuple[int, int]] = []
|
|
||||||
off, remain = start, start + count
|
|
||||||
while remain > 0:
|
|
||||||
take = min(page, remain)
|
|
||||||
plan.append((off, take))
|
|
||||||
off += take
|
|
||||||
remain -= take
|
|
||||||
|
|
||||||
tasks: deque[tuple[Symbol, int, int]] = deque(
|
|
||||||
(s, o, c) for s in syms for o, c in plan)
|
|
||||||
n_conn = min(len(tasks), pool.max_size, _BULK_MAX_CONNS,
|
|
||||||
max(2, -(-len(tasks) // max(1, pipeline * 4))))
|
|
||||||
await pool.warm(n_conn) # 并发建连;acquire 建连是串行的,先热好
|
|
||||||
held = await pool.acquire_many(n_conn)
|
|
||||||
if not held:
|
|
||||||
return {}
|
|
||||||
|
|
||||||
by_text = {s.text: s for s in syms}
|
|
||||||
acc: dict[str, list[SecurityBar]] = {}
|
|
||||||
retry: dict[str, Symbol] = {}
|
|
||||||
|
|
||||||
async def worker(pc) -> None:
|
|
||||||
broken = False
|
|
||||||
try:
|
|
||||||
while tasks:
|
|
||||||
batch: list[tuple[Symbol, int, int]] = []
|
|
||||||
while tasks and len(batch) < pipeline:
|
|
||||||
batch.append(tasks.popleft())
|
|
||||||
cmds = [make_cmd(s, o, c) for s, o, c in batch]
|
|
||||||
try:
|
|
||||||
res = await pc.execute_pipeline(cmds)
|
|
||||||
except Exception:
|
|
||||||
# 流水线中途失败 = 响应流已错配,连接作废、整批交回逐只重试
|
|
||||||
broken = True
|
|
||||||
for s, _, _ in batch:
|
|
||||||
retry[s.text] = s
|
|
||||||
return
|
|
||||||
for (s, _, _), bars in zip(batch, res):
|
|
||||||
acc.setdefault(s.text, []).extend(bars)
|
|
||||||
finally:
|
|
||||||
await pool.release(pc, broken=broken)
|
|
||||||
|
|
||||||
await asyncio.gather(*(worker(pc) for pc in held))
|
|
||||||
# 连接全挂时 worker 会提前 return,队列里剩下的标的没人取 —— 一并交回逐只,
|
|
||||||
# 否则会**静默少数据**(实测 20 只只回 16 只)。
|
|
||||||
while tasks:
|
|
||||||
s, _, _ = tasks.popleft()
|
|
||||||
retry.setdefault(s.text, s)
|
|
||||||
out = {k: self._finish_bars(v, by_text[k]) for k, v in acc.items()}
|
|
||||||
for text, s in retry.items():
|
|
||||||
try:
|
|
||||||
out[text] = await self._bars_one(
|
|
||||||
s, period, start=start, count=count, adjust=adj,
|
|
||||||
start_date=None, end_date=None)
|
|
||||||
except Exception:
|
|
||||||
out.setdefault(text, [])
|
|
||||||
return out
|
|
||||||
|
|
||||||
async def _bars_one(
|
async def _bars_one(
|
||||||
self, sym: Symbol, period: KlinePeriod, *,
|
self, sym: Symbol, period: KlinePeriod, *,
|
||||||
start: int, count: int, adjust: Adjust | None,
|
start: int, count: int, adjust: Adjust | None,
|
||||||
@@ -1056,10 +751,8 @@ class TdxData:
|
|||||||
from .protocol.commands.ex_proto import GetExInstrumentBarsCmd
|
from .protocol.commands.ex_proto import GetExInstrumentBarsCmd
|
||||||
cat = int(period)
|
cat = int(period)
|
||||||
fetch_count = max(700, start + count)
|
fetch_count = max(700, start + count)
|
||||||
# extra 带的 @v2 是缓存 schema 标记:v2 起 EX K线开始回填 amount/settle,
|
|
||||||
# 旧版本写下的缓存里 amount 恒 0.0,必须让它们自然失效。
|
|
||||||
key = CacheKey("kline", sym.market, sym.code, period.name, start,
|
key = CacheKey("kline", sym.market, sym.code, period.name, start,
|
||||||
fetch_count, f"{adj.name}@{cat}@v2")
|
fetch_count, f"{adj.name}@{cat}")
|
||||||
|
|
||||||
async def fetch() -> bytes:
|
async def fetch() -> bytes:
|
||||||
rows = await self._fetch_paginated_ex(
|
rows = await self._fetch_paginated_ex(
|
||||||
@@ -1070,10 +763,7 @@ class TdxData:
|
|||||||
year=d["year"], month=d["month"], day=d["day"],
|
year=d["year"], month=d["month"], day=d["day"],
|
||||||
hour=d.get("hour", 0), minute=d.get("minute", 0),
|
hour=d.get("hour", 0), minute=d.get("minute", 0),
|
||||||
open=d["open"], high=d["high"], low=d["low"], close=d["close"],
|
open=d["open"], high=d["high"], low=d["low"], close=d["close"],
|
||||||
vol=float(d.get("trade", 0) or 0),
|
vol=float(d.get("trade", 0) or 0), amount=0.0,
|
||||||
# 权益类(港股/美股/新三板)真有成交额;期货期权的帧里没有,恒 0.0
|
|
||||||
amount=float(d.get("amount", 0.0) or 0.0),
|
|
||||||
settle=d.get("settle"),
|
|
||||||
) for d in rows]
|
) for d in rows]
|
||||||
return pack_payload(bars)
|
return pack_payload(bars)
|
||||||
|
|
||||||
@@ -1096,23 +786,21 @@ class TdxData:
|
|||||||
start_date: datetime | None, end_date: datetime | None,
|
start_date: datetime | None, end_date: datetime | None,
|
||||||
adjust: Adjust | None,
|
adjust: Adjust | None,
|
||||||
) -> list[SecurityBar]:
|
) -> list[SecurityBar]:
|
||||||
"""按日期区间取K线:贴最新走倒推,历史中段才二分定位。
|
"""按日期区间取K线:先二分定位偏移,再只拉目标区间。
|
||||||
|
|
||||||
关键坑:**偏移方向各通道不一致**(实测标准通道 offset 0 是最新一根、
|
关键坑:**偏移方向各通道不一致**(实测标准通道 offset 0 是最新一根、
|
||||||
越大越老;MAC/EX 反过来),所以先探一次方向再二分,不做静态假设。
|
越大越老;MAC/EX 反过来),所以先探一次方向再二分,不做静态假设。
|
||||||
|
|
||||||
快慢两条路径(:meth:`_bars_backtrack` 优先):
|
|
||||||
倒推 = 直接 ``count=N`` 拿最新 N 根再裁剪,1~3 次往返;
|
|
||||||
二分 = 先探总根数再 log2 次跳转,长周期上是 36+ 次串行往返。
|
|
||||||
「取今天」「增量到最新」都贴着右端,占了实际使用的绝大多数。
|
|
||||||
"""
|
"""
|
||||||
# 归一化日期边界
|
# 归一化日期边界
|
||||||
start_dt, end_dt = _norm_range(start_date, end_date)
|
start_dt = (start_date if isinstance(start_date, datetime)
|
||||||
|
else datetime.combine(start_date, datetime.min.time())) \
|
||||||
# 快路径:区间右端贴着最新 → 从最新倒推,1~3 次往返
|
if start_date is not None else None
|
||||||
fast = await self._bars_backtrack(sym, period, adjust, start_dt, end_dt)
|
end_dt = (end_date if isinstance(end_date, datetime)
|
||||||
if fast is not None:
|
else datetime.combine(end_date, datetime.max.time())) \
|
||||||
return fast
|
if end_date is not None else None
|
||||||
|
# end_date 传入 datetime 但时间为 00:00 时视为"当天结束"
|
||||||
|
if end_dt is not None and end_dt.time() == datetime.min.time().replace(tzinfo=None):
|
||||||
|
end_dt = end_dt.replace(hour=23, minute=59, second=59, microsecond=999999)
|
||||||
|
|
||||||
est = _KLINE_MAX_BARS.get(period, 10_000)
|
est = _KLINE_MAX_BARS.get(period, 10_000)
|
||||||
total = await self._bar_total(sym, period, adjust, est)
|
total = await self._bar_total(sym, period, adjust, est)
|
||||||
@@ -1122,7 +810,6 @@ class TdxData:
|
|||||||
|
|
||||||
if desc:
|
if desc:
|
||||||
# offset 越大越老:[0,k) 全都 >= start_date,[j,∞) 全都 <= end_date
|
# offset 越大越老:[0,k) 全都 >= start_date,[j,∞) 全都 <= end_date
|
||||||
# 不做并行二分:两个协程同时抢连接会触发服务器侧限速,实测比串行更慢
|
|
||||||
k = total
|
k = total
|
||||||
if start_dt is not None:
|
if start_dt is not None:
|
||||||
k = await self._bisect_offset(
|
k = await self._bisect_offset(
|
||||||
@@ -1154,87 +841,6 @@ class TdxData:
|
|||||||
bars = [b for b in bars if b.datetime <= end_dt]
|
bars = [b for b in bars if b.datetime <= end_dt]
|
||||||
return bars
|
return bars
|
||||||
|
|
||||||
async def _bars_backtrack(
|
|
||||||
self, sym: Symbol, period: KlinePeriod, adjust: Adjust | None,
|
|
||||||
start_dt: datetime | None, end_dt: datetime | None,
|
|
||||||
) -> list[SecurityBar] | None:
|
|
||||||
"""从最新一根往前倒推目标区间(区间右端贴着最新时适用)。
|
|
||||||
|
|
||||||
返回 ``None`` 表示**不适用**(偏移方向相反 / 要的是历史中间段 /
|
|
||||||
区间太长放大不划算),调用方应回落到二分路径;返回列表表示已算完
|
|
||||||
(**空列表**也是有效结果,代表区间内无数据)。
|
|
||||||
|
|
||||||
为什么快:二分要先 ``_bar_total`` 翻倍探总根数(长周期 6 次)再跳
|
|
||||||
``log2(总根数)`` 次,1 分钟 K 线约 36 次串行往返 = 4 秒级;
|
|
||||||
倒推只是「拿最新 N 根」,一次就能覆盖一天。
|
|
||||||
|
|
||||||
**标准通道不探测方向**:``_bars_one`` 的 ``count`` 路径本来就靠
|
|
||||||
「start=0 即最新端」这个协议事实,这里沿用同一假定,省掉 2 次往返
|
|
||||||
——全市场 5,000 只时每只省 2 次,是几十秒的量级。
|
|
||||||
只有 MAC/EX 才真去探(它们 offset 方向相反)。
|
|
||||||
"""
|
|
||||||
adj = self.default_adjust if adjust is None else adjust
|
|
||||||
span = _BAR_SPAN.get(period, timedelta(days=1))
|
|
||||||
now = datetime.now()
|
|
||||||
|
|
||||||
if not await self._kline_newest_at_zero(sym, period, adj):
|
|
||||||
return None # offset 0 是最老端 → 倒推不成立,交给二分
|
|
||||||
|
|
||||||
if start_dt is not None and start_dt > now + span:
|
|
||||||
return [] # 整段区间都在未来
|
|
||||||
# 右端够不着最新(留 _BACKTRACK_GRACE 余量覆盖周末/长假)→ 走二分
|
|
||||||
if end_dt is not None and end_dt + _BACKTRACK_GRACE < now:
|
|
||||||
return None
|
|
||||||
ref = now
|
|
||||||
|
|
||||||
# 初值按时间跨度估根数:一天内 1 次就够,长区间也基本 1 次到位
|
|
||||||
count = _BACKTRACK_MIN_BARS
|
|
||||||
if start_dt is not None:
|
|
||||||
need = _bars_for_range(period, start_dt, ref)
|
|
||||||
# 当天(盘中)实际只要已走完的那几十根,别沿用一整天 240 根
|
|
||||||
count = max(need, 8) if start_dt.date() == ref.date() else max(count, need)
|
|
||||||
count = min(count, _BACKTRACK_MAX_BARS)
|
|
||||||
|
|
||||||
bars: list[SecurityBar] = []
|
|
||||||
for _ in range(_BACKTRACK_ROUNDS):
|
|
||||||
bars = await self.get_bars(sym.text, period, start=0, count=count,
|
|
||||||
adjust=adjust)
|
|
||||||
if not bars:
|
|
||||||
return []
|
|
||||||
first = bars[0].datetime
|
|
||||||
# 覆盖到 start 即止;跨到自己之前的那一天也算够(K线连续,中间不会缺)
|
|
||||||
if start_dt is None or first <= start_dt or first.date() < start_dt.date():
|
|
||||||
break
|
|
||||||
if count >= _BACKTRACK_MAX_BARS:
|
|
||||||
return None
|
|
||||||
count = min(count * 2, _BACKTRACK_MAX_BARS)
|
|
||||||
else:
|
|
||||||
return None # 放大到上限仍不够 → 交给二分
|
|
||||||
|
|
||||||
if start_dt is not None:
|
|
||||||
bars = [b for b in bars if b.datetime >= start_dt]
|
|
||||||
if end_dt is not None:
|
|
||||||
bars = [b for b in bars if b.datetime <= end_dt]
|
|
||||||
return bars
|
|
||||||
|
|
||||||
async def _kline_newest_at_zero(self, sym: Symbol, period: KlinePeriod,
|
|
||||||
adjust: Adjust) -> bool:
|
|
||||||
"""K 线的 ``offset=0`` 是否就是**最新**一根(True 才能倒推)。
|
|
||||||
|
|
||||||
偏移方向是通道级属性,与具体标的无关,首次探测后按通道名缓存在实例上。
|
|
||||||
"""
|
|
||||||
key = "ex" if sym.is_ex else route_for("kline", adjust=adjust, period=period)
|
|
||||||
cached = self._kline_dir_cache.get(key)
|
|
||||||
if cached is not None:
|
|
||||||
return cached
|
|
||||||
newest = await self._probe_bar(sym, period, adjust, 0)
|
|
||||||
if newest is None:
|
|
||||||
return False # 探不到就别猜,交给二分去处理空数据
|
|
||||||
nxt = await self._probe_bar(sym, period, adjust, 1)
|
|
||||||
desc = nxt is None or nxt.datetime < newest.datetime
|
|
||||||
self._kline_dir_cache[key] = desc
|
|
||||||
return desc
|
|
||||||
|
|
||||||
async def _bar_total(self, sym: Symbol, period: KlinePeriod,
|
async def _bar_total(self, sym: Symbol, period: KlinePeriod,
|
||||||
adjust: Adjust | None, est: int) -> int:
|
adjust: Adjust | None, est: int) -> int:
|
||||||
"""指数扩张找K线总根数上界(探测到越界为止)。"""
|
"""指数扩张找K线总根数上界(探测到越界为止)。"""
|
||||||
@@ -1302,7 +908,6 @@ class TdxData:
|
|||||||
return None
|
return None
|
||||||
return bars[0] if bars else None
|
return bars[0] if bars else None
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_indicators(
|
async def get_indicators(
|
||||||
self, symbols: str | Sequence[str], indicators: Sequence[str], *,
|
self, symbols: str | Sequence[str], indicators: Sequence[str], *,
|
||||||
period: KlinePeriod = KlinePeriod.DAY, count: int = 30,
|
period: KlinePeriod = KlinePeriod.DAY, count: int = 30,
|
||||||
@@ -1341,7 +946,6 @@ class TdxData:
|
|||||||
# 四、逐笔成交 / 分时
|
# 四、逐笔成交 / 分时
|
||||||
# ================================================================== #
|
# ================================================================== #
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_ticks(
|
async def get_ticks(
|
||||||
self, symbols: str | Sequence[str], *,
|
self, symbols: str | Sequence[str], *,
|
||||||
date: int | None = None, start: int = 0, count: int | None = None,
|
date: int | None = None, start: int = 0, count: int | None = None,
|
||||||
@@ -1523,7 +1127,6 @@ class TdxData:
|
|||||||
trade_count_available=False, # 扩展市场逐笔不带笔数字段
|
trade_count_available=False, # 扩展市场逐笔不带笔数字段
|
||||||
)
|
)
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_minutes(
|
async def get_minutes(
|
||||||
self, symbols: str | Sequence[str], *,
|
self, symbols: str | Sequence[str], *,
|
||||||
date: int | None = None, sampling: bool = False, concurrency: int = 6,
|
date: int | None = None, sampling: bool = False, concurrency: int = 6,
|
||||||
@@ -1604,7 +1207,6 @@ class TdxData:
|
|||||||
# 五、公司信息
|
# 五、公司信息
|
||||||
# ================================================================== #
|
# ================================================================== #
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_xdxr(self, symbols: str | Sequence[str]) -> list[XdxrRecord]:
|
async def get_xdxr(self, symbols: str | Sequence[str]) -> list[XdxrRecord]:
|
||||||
"""除权除息历史,仅 A股。标的用字符串(``"sz000001"``)。"""
|
"""除权除息历史,仅 A股。标的用字符串(``"sz000001"``)。"""
|
||||||
syms = parse_symbols(symbols)
|
syms = parse_symbols(symbols)
|
||||||
@@ -1623,7 +1225,6 @@ class TdxData:
|
|||||||
out.extend(recs)
|
out.extend(recs)
|
||||||
return out
|
return out
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_finance(self, symbols: str | Sequence[str]) -> list[FinanceRecord]:
|
async def get_finance(self, symbols: str | Sequence[str]) -> list[FinanceRecord]:
|
||||||
"""财务快照(仅 A股)。无数据的标的会被跳过(原单标的接口返回 None)。"""
|
"""财务快照(仅 A股)。无数据的标的会被跳过(原单标的接口返回 None)。"""
|
||||||
syms = parse_symbols(symbols)
|
syms = parse_symbols(symbols)
|
||||||
@@ -1694,7 +1295,6 @@ class TdxData:
|
|||||||
stat.limit_down_count = int(q6.open * 10)
|
stat.limit_down_count = int(q6.open * 10)
|
||||||
return stat
|
return stat
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_unusual(self, market: str = "all", *, start: int = 0,
|
async def get_unusual(self, market: str = "all", *, start: int = 0,
|
||||||
count: int = 600) -> list[UnusualItem]:
|
count: int = 600) -> list[UnusualItem]:
|
||||||
"""市场异动(仅 A股;``market="all"`` = 深市 + 沪市)。"""
|
"""市场异动(仅 A股;``market="all"`` = 深市 + 沪市)。"""
|
||||||
@@ -1730,7 +1330,6 @@ class TdxData:
|
|||||||
f"未知板块类别: {kind}(可选: {'/'.join(table)} 或数字)")
|
f"未知板块类别: {kind}(可选: {'/'.join(table)} 或数字)")
|
||||||
return table[key]
|
return table[key]
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_boards(self, kind: str = "all", *, count: int = 10000) -> list[BoardInfo]:
|
async def get_boards(self, kind: str = "all", *, count: int = 10000) -> list[BoardInfo]:
|
||||||
"""板块列表(``kind``: ``all``/``industry``/``concept``/``style``,也接受数字)。
|
"""板块列表(``kind``: ``all``/``industry``/``concept``/``style``,也接受数字)。
|
||||||
|
|
||||||
@@ -1752,7 +1351,6 @@ class TdxData:
|
|||||||
b.symbol = format_symbol(b.market, b.code)
|
b.symbol = format_symbol(b.market, b.code)
|
||||||
return items
|
return items
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_board_members(self, board: str, count: int = 100000,
|
async def get_board_members(self, board: str, count: int = 100000,
|
||||||
sort_type: int = 0, *,
|
sort_type: int = 0, *,
|
||||||
fields: Sequence[int] | None = None
|
fields: Sequence[int] | None = None
|
||||||
@@ -1778,7 +1376,6 @@ class TdxData:
|
|||||||
break
|
break
|
||||||
return items
|
return items
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_board_of(self, symbols: str | Sequence[str]) -> list[BelongBoard]:
|
async def get_board_of(self, symbols: str | Sequence[str]) -> list[BelongBoard]:
|
||||||
"""个股所属板块(仅 A股;``symbol`` 记录被查询的个股)。"""
|
"""个股所属板块(仅 A股;``symbol`` 记录被查询的个股)。"""
|
||||||
syms = parse_symbols(symbols)
|
syms = parse_symbols(symbols)
|
||||||
@@ -1807,13 +1404,11 @@ class TdxData:
|
|||||||
"down_count": down,
|
"down_count": down,
|
||||||
}
|
}
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_board_ranking(self, kind: str = "industry", *,
|
async def get_board_ranking(self, kind: str = "industry", *,
|
||||||
top_n: int = 20) -> list[dict]:
|
top_n: int = 20) -> list[dict]:
|
||||||
"""板块当日涨跌幅排行(基于板块报价的现价/昨收)。"""
|
"""板块当日涨跌幅排行(基于板块报价的现价/昨收)。"""
|
||||||
return self._rank_boards(await self.get_boards(kind, count=200), top_n)
|
return self._rank_boards(await self.get_boards(kind, count=200), top_n)
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_board_change_ranking(self, kind: str = "industry", *,
|
async def get_board_change_ranking(self, kind: str = "industry", *,
|
||||||
days: int = 20, top_n: int = 20) -> list[dict]:
|
days: int = 20, top_n: int = 20) -> list[dict]:
|
||||||
"""板块 N 日涨跌幅排行(基于板块指数K线)。"""
|
"""板块 N 日涨跌幅排行(基于板块指数K线)。"""
|
||||||
@@ -1898,7 +1493,6 @@ class TdxData:
|
|||||||
return 2, s
|
return 2, s
|
||||||
raise ValueError(f"无法判定指数市场: {index!r}(不是 A股指数代码段)")
|
raise ValueError(f"无法判定指数市场: {index!r}(不是 A股指数代码段)")
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_index_members(self, index: str, *, count: int = 100000,
|
async def get_index_members(self, index: str, *, count: int = 100000,
|
||||||
fields: Sequence[int] | None = None
|
fields: Sequence[int] | None = None
|
||||||
) -> list[SecurityQuote]:
|
) -> list[SecurityQuote]:
|
||||||
@@ -1947,7 +1541,6 @@ class TdxData:
|
|||||||
break
|
break
|
||||||
return items[:count]
|
return items[:count]
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_index_trade_stats(
|
async def get_index_trade_stats(
|
||||||
self,
|
self,
|
||||||
index: str,
|
index: str,
|
||||||
@@ -2072,7 +1665,6 @@ class TdxData:
|
|||||||
elapsed=time.perf_counter() - started,
|
elapsed=time.perf_counter() - started,
|
||||||
)
|
)
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_index_trade_stats_multi(
|
async def get_index_trade_stats_multi(
|
||||||
self,
|
self,
|
||||||
index: str,
|
index: str,
|
||||||
@@ -2153,7 +1745,6 @@ class TdxData:
|
|||||||
break
|
break
|
||||||
return bytes(data)
|
return bytes(data)
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_block_parsed(self, filename: str) -> list[TdxBlock]:
|
async def get_block_parsed(self, filename: str) -> list[TdxBlock]:
|
||||||
"""板块文件 -> ``list[TdxBlock]``(板块名/类别/成分股)。
|
"""板块文件 -> ``list[TdxBlock]``(板块名/类别/成分股)。
|
||||||
|
|
||||||
@@ -2168,13 +1759,11 @@ class TdxData:
|
|||||||
from .codec.industry import parse_tdxhy_cfg
|
from .codec.industry import parse_tdxhy_cfg
|
||||||
return parse_tdxhy_cfg(await self.get_file("tdxhy.cfg"))
|
return parse_tdxhy_cfg(await self.get_file("tdxhy.cfg"))
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_financial_file_infos(self) -> list[FinancialFileInfo]:
|
async def get_financial_file_infos(self) -> list[FinancialFileInfo]:
|
||||||
"""tdxfin/gpcw.txt -> ``list[FinancialFileInfo]``(文件名/MD5/大小)。"""
|
"""tdxfin/gpcw.txt -> ``list[FinancialFileInfo]``(文件名/MD5/大小)。"""
|
||||||
from .codec.financial import parse_financial_file_list
|
from .codec.financial import parse_financial_file_list
|
||||||
return parse_financial_file_list(await self.get_file("tdxfin/gpcw.txt"))
|
return parse_financial_file_list(await self.get_file("tdxfin/gpcw.txt"))
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_financial_records_parsed(self, filename: str) -> list[FinancialRecord]:
|
async def get_financial_records_parsed(self, filename: str) -> list[FinancialRecord]:
|
||||||
"""财报数据解析 -> ``list[FinancialRecord]``。
|
"""财报数据解析 -> ``list[FinancialRecord]``。
|
||||||
|
|
||||||
@@ -2368,7 +1957,6 @@ class TdxData:
|
|||||||
# 九、旧接口(deprecated 别名;入参沿用旧口径,1.0 移除)
|
# 九、旧接口(deprecated 别名;入参沿用旧口径,1.0 移除)
|
||||||
# ================================================================== #
|
# ================================================================== #
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_kline(self, market, code, period, *, start: int = 0,
|
async def get_kline(self, market, code, period, *, start: int = 0,
|
||||||
count: int = 800, adjust: Adjust | None = None,
|
count: int = 800, adjust: Adjust | None = None,
|
||||||
start_date: datetime | None = None,
|
start_date: datetime | None = None,
|
||||||
@@ -2379,7 +1967,6 @@ class TdxData:
|
|||||||
start=start, count=count, adjust=adjust,
|
start=start, count=count, adjust=adjust,
|
||||||
start_date=start_date, end_date=end_date)
|
start_date=start_date, end_date=end_date)
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_kline_since(self, market, code, period, since: datetime,
|
async def get_kline_since(self, market, code, period, since: datetime,
|
||||||
adjust: Adjust | None = None) -> list[SecurityBar]:
|
adjust: Adjust | None = None) -> list[SecurityBar]:
|
||||||
"""已废弃 -> :meth:`get_bars`(``start_date=since``)。"""
|
"""已废弃 -> :meth:`get_bars`(``start_date=since``)。"""
|
||||||
@@ -2387,7 +1974,6 @@ class TdxData:
|
|||||||
return await self.get_bars(format_symbol(int(market), code), period,
|
return await self.get_bars(format_symbol(int(market), code), period,
|
||||||
start_date=since, adjust=adjust)
|
start_date=since, adjust=adjust)
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_kline_batch(self, stocks, period, *, start: int = 0,
|
async def get_kline_batch(self, stocks, period, *, start: int = 0,
|
||||||
count: int = 800, adjust: Adjust | None = None,
|
count: int = 800, adjust: Adjust | None = None,
|
||||||
concurrency: int = 6) -> dict[str, list[SecurityBar]]:
|
concurrency: int = 6) -> dict[str, list[SecurityBar]]:
|
||||||
@@ -2401,7 +1987,6 @@ class TdxData:
|
|||||||
out.setdefault(b.code, []).append(b)
|
out.setdefault(b.code, []).append(b)
|
||||||
return out
|
return out
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_index_kline(self, market, code, period, *, start: int = 0,
|
async def get_index_kline(self, market, code, period, *, start: int = 0,
|
||||||
count: int = 800) -> list[SecurityBar]:
|
count: int = 800) -> list[SecurityBar]:
|
||||||
"""已废弃 -> :meth:`get_bars`(指数自动识别,无需单独接口)。"""
|
"""已废弃 -> :meth:`get_bars`(指数自动识别,无需单独接口)。"""
|
||||||
@@ -2422,14 +2007,12 @@ class TdxData:
|
|||||||
one = sets[0]
|
one = sets[0]
|
||||||
return {"bars": one.bars, "indicators": one.indicators}
|
return {"bars": one.bars, "indicators": one.indicators}
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_stock_quotes(self, stocks, bits=None) -> list[SecurityQuote]:
|
async def get_stock_quotes(self, stocks, bits=None) -> list[SecurityQuote]:
|
||||||
"""已废弃 -> :meth:`get_quotes`(``fields=bits``)。"""
|
"""已废弃 -> :meth:`get_quotes`(``fields=bits``)。"""
|
||||||
_deprecated("get_stock_quotes", "get_quotes")
|
_deprecated("get_stock_quotes", "get_quotes")
|
||||||
return await self.get_quotes([format_symbol(int(m), c) for m, c in stocks],
|
return await self.get_quotes([format_symbol(int(m), c) for m, c in stocks],
|
||||||
fields=bits)
|
fields=bits)
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_stock_quotes_list(self, category: int, *, start: int = 0,
|
async def get_stock_quotes_list(self, category: int, *, start: int = 0,
|
||||||
count: int = 80, sort_type: int = 0
|
count: int = 80, sort_type: int = 0
|
||||||
) -> list[SecurityQuote]:
|
) -> list[SecurityQuote]:
|
||||||
@@ -2438,7 +2021,6 @@ class TdxData:
|
|||||||
return await self._board_members_raw(category, start=start, count=count,
|
return await self._board_members_raw(category, start=start, count=count,
|
||||||
sort_type=sort_type)
|
sort_type=sort_type)
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_transactions(self, market, code, *, date: int | None = None,
|
async def get_transactions(self, market, code, *, date: int | None = None,
|
||||||
start: int = 0, count: int = 2000
|
start: int = 0, count: int = 2000
|
||||||
) -> list[TransactionRecord]:
|
) -> list[TransactionRecord]:
|
||||||
@@ -2447,14 +2029,12 @@ class TdxData:
|
|||||||
return await self.get_ticks(format_symbol(int(market), code), date=date,
|
return await self.get_ticks(format_symbol(int(market), code), date=date,
|
||||||
start=start, count=count)
|
start=start, count=count)
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_minute(self, market, code, *, date: int | None = None
|
async def get_minute(self, market, code, *, date: int | None = None
|
||||||
) -> list[MinuteBar]:
|
) -> list[MinuteBar]:
|
||||||
"""已废弃 -> :meth:`get_minutes`。"""
|
"""已废弃 -> :meth:`get_minutes`。"""
|
||||||
_deprecated("get_minute", "get_minutes")
|
_deprecated("get_minute", "get_minutes")
|
||||||
return await self.get_minutes(format_symbol(int(market), code), date=date)
|
return await self.get_minutes(format_symbol(int(market), code), date=date)
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_minute_batch(self, stocks, *, date: int | None = None,
|
async def get_minute_batch(self, stocks, *, date: int | None = None,
|
||||||
concurrency: int = 6) -> dict[str, list[MinuteBar]]:
|
concurrency: int = 6) -> dict[str, list[MinuteBar]]:
|
||||||
"""已废弃 -> :meth:`get_minutes`(返回值由 ``{code: bars}`` 变为平铺列表)。"""
|
"""已废弃 -> :meth:`get_minutes`(返回值由 ``{code: bars}`` 变为平铺列表)。"""
|
||||||
@@ -2477,7 +2057,6 @@ class TdxData:
|
|||||||
_deprecated("get_security_count", "get_universe")
|
_deprecated("get_security_count", "get_universe")
|
||||||
return await self._security_count(int(market))
|
return await self._security_count(int(market))
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_security_list(self, market, *, start: int = 0,
|
async def get_security_list(self, market, *, start: int = 0,
|
||||||
count: int | None = None) -> list[SecurityInfo]:
|
count: int | None = None) -> list[SecurityInfo]:
|
||||||
"""已废弃 -> :meth:`get_universe`(本地切片 ``[start:start+count]``)。"""
|
"""已废弃 -> :meth:`get_universe`(本地切片 ``[start:start+count]``)。"""
|
||||||
@@ -2485,7 +2064,6 @@ class TdxData:
|
|||||||
items = await self.get_universe(market_name(int(market)))
|
items = await self.get_universe(market_name(int(market)))
|
||||||
return items[start:] if count is None else items[start:start + count]
|
return items[start:] if count is None else items[start:start + count]
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_security_list_all(self, market) -> list[SecurityInfo]:
|
async def get_security_list_all(self, market) -> list[SecurityInfo]:
|
||||||
"""已废弃 -> :meth:`get_universe`。"""
|
"""已废弃 -> :meth:`get_universe`。"""
|
||||||
_deprecated("get_security_list_all", "get_universe")
|
_deprecated("get_security_list_all", "get_universe")
|
||||||
@@ -2539,14 +2117,12 @@ class TdxData:
|
|||||||
_deprecated("download_file", "get_file")
|
_deprecated("download_file", "get_file")
|
||||||
return await self.get_file(filename)
|
return await self.get_file(filename)
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_board_list(self, board_type: int | str = 0,
|
async def get_board_list(self, board_type: int | str = 0,
|
||||||
count: int = 10000) -> list[BoardInfo]:
|
count: int = 10000) -> list[BoardInfo]:
|
||||||
"""已废弃 -> :meth:`get_boards`。"""
|
"""已废弃 -> :meth:`get_boards`。"""
|
||||||
_deprecated("get_board_list", "get_boards")
|
_deprecated("get_board_list", "get_boards")
|
||||||
return await self.get_boards(board_type, count=count)
|
return await self.get_boards(board_type, count=count)
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_belong_board(self, market, code) -> list[BelongBoard]:
|
async def get_belong_board(self, market, code) -> list[BelongBoard]:
|
||||||
"""已废弃 -> :meth:`get_board_of`。"""
|
"""已废弃 -> :meth:`get_board_of`。"""
|
||||||
_deprecated("get_belong_board", "get_board_of")
|
_deprecated("get_belong_board", "get_board_of")
|
||||||
@@ -2558,7 +2134,6 @@ class TdxData:
|
|||||||
items = await self.get_snapshot(format_symbol(int(market), code))
|
items = await self.get_snapshot(format_symbol(int(market), code))
|
||||||
return items[0] if items else None
|
return items[0] if items else None
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_goods_list(self, market, start: int = 0,
|
async def get_goods_list(self, market, start: int = 0,
|
||||||
count: int = 100) -> list[SecurityInfo]:
|
count: int = 100) -> list[SecurityInfo]:
|
||||||
"""已废弃 -> :meth:`get_universe`(扩展到 ``SecurityInfo`` 统一模型)。"""
|
"""已废弃 -> :meth:`get_universe`(扩展到 ``SecurityInfo`` 统一模型)。"""
|
||||||
@@ -2573,14 +2148,12 @@ class TdxData:
|
|||||||
return len(await self._ex_all())
|
return len(await self._ex_all())
|
||||||
return len(await self._universe_ex(int(market)))
|
return len(await self._universe_ex(int(market)))
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_goods_quotes(self, stocks, bits=None) -> list[SecurityQuote]:
|
async def get_goods_quotes(self, stocks, bits=None) -> list[SecurityQuote]:
|
||||||
"""已废弃 -> :meth:`get_quotes`。"""
|
"""已废弃 -> :meth:`get_quotes`。"""
|
||||||
_deprecated("get_goods_quotes", "get_quotes")
|
_deprecated("get_goods_quotes", "get_quotes")
|
||||||
return await self.get_quotes([format_symbol(int(m), c) for m, c in stocks],
|
return await self.get_quotes([format_symbol(int(m), c) for m, c in stocks],
|
||||||
fields=bits)
|
fields=bits)
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_goods_quotes_list(self, market, count: int = 80
|
async def get_goods_quotes_list(self, market, count: int = 80
|
||||||
) -> list[SecurityQuote]:
|
) -> list[SecurityQuote]:
|
||||||
"""已废弃 -> :meth:`get_quotes`(或 ``get_universe`` 取清单后自选)。"""
|
"""已废弃 -> :meth:`get_quotes`(或 ``get_universe`` 取清单后自选)。"""
|
||||||
@@ -2596,7 +2169,6 @@ class TdxData:
|
|||||||
bars = await self.get_minutes(format_symbol(int(market), code), sampling=True)
|
bars = await self.get_minutes(format_symbol(int(market), code), sampling=True)
|
||||||
return [b.price for b in bars]
|
return [b.price for b in bars]
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_goods_tick_chart(self, market, code,
|
async def get_goods_tick_chart(self, market, code,
|
||||||
ymd: int = 0) -> list[MinuteBar]:
|
ymd: int = 0) -> list[MinuteBar]:
|
||||||
"""已废弃 -> :meth:`get_minutes`(扩展到统一分时模型)。"""
|
"""已废弃 -> :meth:`get_minutes`(扩展到统一分时模型)。"""
|
||||||
@@ -2604,7 +2176,6 @@ class TdxData:
|
|||||||
return await self.get_minutes(format_symbol(int(market), code),
|
return await self.get_minutes(format_symbol(int(market), code),
|
||||||
date=ymd or None)
|
date=ymd or None)
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_goods_transaction(self, market, code, *, ymd: int = 0,
|
async def get_goods_transaction(self, market, code, *, ymd: int = 0,
|
||||||
start: int = 0, count: int = 1800
|
start: int = 0, count: int = 1800
|
||||||
) -> list[TransactionRecord]:
|
) -> list[TransactionRecord]:
|
||||||
@@ -2613,7 +2184,6 @@ class TdxData:
|
|||||||
return await self.get_ticks(format_symbol(int(market), code),
|
return await self.get_ticks(format_symbol(int(market), code),
|
||||||
date=ymd or None, start=start, count=count)
|
date=ymd or None, start=start, count=count)
|
||||||
|
|
||||||
@framed
|
|
||||||
async def get_goods_transaction_all(self, market, code,
|
async def get_goods_transaction_all(self, market, code,
|
||||||
ymd: int = 0) -> list[TransactionRecord]:
|
ymd: int = 0) -> list[TransactionRecord]:
|
||||||
"""已废弃 -> :meth:`get_ticks`(``count=None`` 即全量翻页)。"""
|
"""已废弃 -> :meth:`get_ticks`(``count=None`` 即全量翻页)。"""
|
||||||
|
|||||||
@@ -162,9 +162,9 @@ def _print_quote(q) -> None:
|
|||||||
print("=" * 58)
|
print("=" * 58)
|
||||||
label = "%s %s" % (mkt, q.code) if mkt else "%s %s" % (q.code, q.server_time or datetime.now().strftime("%H:%M:%S"))
|
label = "%s %s" % (mkt, q.code) if mkt else "%s %s" % (q.code, q.server_time or datetime.now().strftime("%H:%M:%S"))
|
||||||
print(" %s" % label)
|
print(" %s" % label)
|
||||||
print(" 最新 %-10.*f 涨跌 %+8.*f (%+.2f%%)" % (dp, q.price, dp, q.price - q.pre_close, pct))
|
print((" 最新 %-10.*f 涨跌 %+8.*f (%+.2f%%)" % (dp, q.price, dp, q.price - q.pre_close, pct)))
|
||||||
print(" 今开 %-10.*f 最高 %.*f 最低 %.*f" % (dp, q.open, dp, q.high, dp, q.low))
|
print((" 今开 %-10.*f 最高 %.*f 最低 %.*f" % (dp, q.open, dp, q.high, dp, q.low)))
|
||||||
print(" 昨收 %-10.*f 成交量 %s 成交额 %s" % (dp, q.pre_close, _fmt_num(q.vol), _fmt_num(q.amount)))
|
print((" 昨收 %-10.*f 成交量 %s 成交额 %s" % (dp, q.pre_close, _fmt_num(q.vol), _fmt_num(q.amount))))
|
||||||
if q.bid and q.ask:
|
if q.bid and q.ask:
|
||||||
print(" " + "-" * 56)
|
print(" " + "-" * 56)
|
||||||
pf = _price_fmt(dp)
|
pf = _price_fmt(dp)
|
||||||
|
|||||||
@@ -1,431 +0,0 @@
|
|||||||
"""DataFrame 转换层(0.7.0):把模型列表转成 pandas / polars 的 DataFrame。
|
|
||||||
|
|
||||||
为什么单独一层而不是改各方法的返回:
|
|
||||||
|
|
||||||
- **零运行时依赖**是项目的硬约束(``dependencies = []``)。pandas / polars 只在
|
|
||||||
真正用到 ``TdxData(df=...)`` 时才导入,没装就走不到这里。
|
|
||||||
- 转换口径(哪些列留、时间怎么摆、嵌套字段怎么展开)只写一份,40 个返回表格型
|
|
||||||
数据的方法共用,不会各自漂移。
|
|
||||||
|
|
||||||
三条口径约定(0.7.0 定稿):
|
|
||||||
|
|
||||||
1. **时间列替换原始列**:K线/分时的 ``year/month/day/hour/minute`` 不再单列输出,
|
|
||||||
统一给一列 ``datetime``(pandas 为 ``datetime64[ns]``,polars 为 ``Datetime('us')``),
|
|
||||||
另带 ``trading_date``(``YYYY-MM-DD`` 字符串,美股跨日口径见模型注释)。
|
|
||||||
2. **嵌套字段定型**:``SecurityQuote.bid/ask`` 展开成 ``bid1_price``…``ask5_vol`` 共
|
|
||||||
10 列(缺失为 ``None``);``fields`` 这类不定长字典序列化成 JSON 字符串列,
|
|
||||||
空字典给 ``None`` 而不是 ``"{}"``;``TdxBlock.codes`` 保持列表。
|
|
||||||
3. **嵌套调用不二次转换**:``get_kline_batch`` 这类方法内部还会调 ``get_bars``,
|
|
||||||
用 :data:`_FRAME_DEPTH`(``ContextVar``,按 task 隔离)标记层次,内层一律返回
|
|
||||||
原始 ``list``,只有最外层那次才转 DataFrame。
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import functools
|
|
||||||
import json
|
|
||||||
import re
|
|
||||||
from collections.abc import Callable, Sequence
|
|
||||||
from contextvars import ContextVar
|
|
||||||
from datetime import datetime as _dt
|
|
||||||
from enum import IntEnum
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
from . import models as _models
|
|
||||||
from .models import (
|
|
||||||
IndexTradeStats,
|
|
||||||
IndicatorSet,
|
|
||||||
MinuteBar,
|
|
||||||
MinuteTradeStat,
|
|
||||||
SecurityBar,
|
|
||||||
SecurityQuote,
|
|
||||||
TransactionRecord,
|
|
||||||
XdxrRecord,
|
|
||||||
)
|
|
||||||
|
|
||||||
__all__ = ["BACKENDS", "FrameBackendError", "framed", "to_frame"]
|
|
||||||
|
|
||||||
BACKENDS: tuple[str, ...] = ("polars", "pandas")
|
|
||||||
"""支持的后端名,也是 ``TdxData(df=...)`` 的合法取值。"""
|
|
||||||
|
|
||||||
|
|
||||||
class FrameBackendError(ImportError):
|
|
||||||
"""指定的 DataFrame 后端未安装。
|
|
||||||
|
|
||||||
继承 :class:`ImportError` 是刻意的:捕获姿势和缺包一致,
|
|
||||||
且 ``e.name`` 就是缺的那个包名。
|
|
||||||
"""
|
|
||||||
|
|
||||||
def __init__(self, backend: str) -> None:
|
|
||||||
self.backend = backend
|
|
||||||
super().__init__(
|
|
||||||
f"未安装 {backend},无法输出 DataFrame。请执行:pip install pytdxdata[{backend}]",
|
|
||||||
name=backend,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
# 转换层次计数:>0 表示当前正处于某次公开方法的执行过程中(内层),此时不转换。
|
|
||||||
_FRAME_DEPTH: ContextVar[int] = ContextVar("pytdxdata_frame_depth", default=0)
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------- 后端
|
|
||||||
|
|
||||||
|
|
||||||
class _Backend:
|
|
||||||
"""后端适配:只封装「造表」和「拼表」两个动作。"""
|
|
||||||
|
|
||||||
name: str = ""
|
|
||||||
|
|
||||||
def frame(self, cols: dict[str, list]) -> Any: # pragma: no cover - 抽象
|
|
||||||
raise NotImplementedError
|
|
||||||
|
|
||||||
def concat(self, frames: Sequence[Any]) -> Any: # pragma: no cover - 抽象
|
|
||||||
raise NotImplementedError
|
|
||||||
|
|
||||||
def empty(self, cols: Sequence[str]) -> Any:
|
|
||||||
return self.frame({name: [] for name in cols})
|
|
||||||
|
|
||||||
|
|
||||||
class _PolarsBackend(_Backend):
|
|
||||||
name = "polars"
|
|
||||||
|
|
||||||
def __init__(self) -> None:
|
|
||||||
try:
|
|
||||||
import polars as _pl
|
|
||||||
except ImportError as exc: # pragma: no cover - 依赖环境
|
|
||||||
raise FrameBackendError("polars") from exc
|
|
||||||
self._pl = _pl
|
|
||||||
|
|
||||||
def frame(self, cols: dict[str, list]) -> Any:
|
|
||||||
# 全空时必须显式给列名,否则 polars 会造出 0 列的表
|
|
||||||
return self._pl.DataFrame(cols)
|
|
||||||
|
|
||||||
def concat(self, frames: Sequence[Any]) -> Any:
|
|
||||||
frames = [f for f in frames if f is not None and f.height > 0]
|
|
||||||
if not frames:
|
|
||||||
return None
|
|
||||||
if len(frames) == 1:
|
|
||||||
return frames[0]
|
|
||||||
# vertical_relaxed:批量接口各标的分块建表,dtype 可能略有差异(如 Null 列)
|
|
||||||
return self._pl.concat(frames, how="vertical_relaxed")
|
|
||||||
|
|
||||||
|
|
||||||
class _PandasBackend(_Backend):
|
|
||||||
name = "pandas"
|
|
||||||
|
|
||||||
def __init__(self) -> None:
|
|
||||||
try:
|
|
||||||
import pandas as _pd
|
|
||||||
except ImportError as exc: # pragma: no cover - 依赖环境
|
|
||||||
raise FrameBackendError("pandas") from exc
|
|
||||||
self._pd = _pd
|
|
||||||
|
|
||||||
def frame(self, cols: dict[str, list]) -> Any:
|
|
||||||
return self._pd.DataFrame(cols)
|
|
||||||
|
|
||||||
def concat(self, frames: Sequence[Any]) -> Any:
|
|
||||||
frames = [f for f in frames if f is not None and len(f) > 0]
|
|
||||||
if not frames:
|
|
||||||
return None
|
|
||||||
if len(frames) == 1:
|
|
||||||
return frames[0]
|
|
||||||
return self._pd.concat(frames, ignore_index=True)
|
|
||||||
|
|
||||||
|
|
||||||
_BACKEND_CACHE: dict[str, _Backend] = {}
|
|
||||||
|
|
||||||
|
|
||||||
def get_backend(name: str) -> _Backend:
|
|
||||||
"""取后端实例(进程内缓存,重复取不会重复 import)。
|
|
||||||
|
|
||||||
:raises ValueError: 名字不在 :data:`BACKENDS` 里。
|
|
||||||
:raises FrameBackendError: 对应的包没装。
|
|
||||||
"""
|
|
||||||
if name not in BACKENDS:
|
|
||||||
raise ValueError(f"未知的 DataFrame 后端 {name!r},可选:{', '.join(BACKENDS)}")
|
|
||||||
be = _BACKEND_CACHE.get(name)
|
|
||||||
if be is None:
|
|
||||||
be = _PolarsBackend() if name == "polars" else _PandasBackend()
|
|
||||||
_BACKEND_CACHE[name] = be
|
|
||||||
return be
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------- 列计划
|
|
||||||
|
|
||||||
|
|
||||||
def _as_int(value: Any) -> Any:
|
|
||||||
"""``IntEnum`` 转纯 ``int``:polars 对 IntEnum 的类型推断不可靠,转掉最稳。"""
|
|
||||||
if isinstance(value, IntEnum):
|
|
||||||
return int(value)
|
|
||||||
return value
|
|
||||||
|
|
||||||
|
|
||||||
def _json_or_none(value: Any) -> str | None:
|
|
||||||
"""不定长字典序列化成 JSON 列;空字典给 ``None``(别让用户看到一列 ``"{}"``)。"""
|
|
||||||
if not value:
|
|
||||||
return None
|
|
||||||
return json.dumps(value, ensure_ascii=False, separators=(",", ":"))
|
|
||||||
|
|
||||||
|
|
||||||
def _getter(name: str) -> Callable[[Any], Any]:
|
|
||||||
def _get(row: Any) -> Any:
|
|
||||||
return _as_int(getattr(row, name))
|
|
||||||
|
|
||||||
return _get
|
|
||||||
|
|
||||||
|
|
||||||
def _json_getter(name: str) -> Callable[[Any], Any]:
|
|
||||||
def _get(row: Any) -> Any:
|
|
||||||
return _json_or_none(getattr(row, name))
|
|
||||||
|
|
||||||
return _get
|
|
||||||
|
|
||||||
|
|
||||||
def _level_getter(side: str, index: int, pos: int) -> Callable[[Any], Any]:
|
|
||||||
"""``bid``/``ask`` 第 ``index`` 档的价格(pos=0)或数量(pos=1)。"""
|
|
||||||
|
|
||||||
def _get(row: Any) -> Any:
|
|
||||||
levels = getattr(row, side)
|
|
||||||
if len(levels) <= index:
|
|
||||||
return None
|
|
||||||
return levels[index][pos]
|
|
||||||
|
|
||||||
return _get
|
|
||||||
|
|
||||||
|
|
||||||
def _moment(row: SecurityBar | MinuteBar) -> _dt:
|
|
||||||
return row.datetime
|
|
||||||
|
|
||||||
|
|
||||||
def _trading_date(row: SecurityBar | MinuteBar) -> str:
|
|
||||||
return row.trading_date
|
|
||||||
|
|
||||||
|
|
||||||
def _xdxr_day(row: XdxrRecord) -> _dt:
|
|
||||||
# 除权日是「日」不是「时刻」,但 datetime64 / Datetime 才是可排序、可比较的列类型。
|
|
||||||
# 全项目时间口径都是「不带 tzinfo 的北京时间」,这里保持一致。
|
|
||||||
return _dt(row.year, row.month, row.day) # noqa: DTZ001
|
|
||||||
|
|
||||||
|
|
||||||
#: 模型 -> 需要丢弃的原始字段(时间拆解字段,由增强列替代)
|
|
||||||
_DROPS: dict[type, tuple[str, ...]] = {
|
|
||||||
SecurityBar: ("year", "month", "day", "hour", "minute"),
|
|
||||||
MinuteBar: ("hour", "minute", "date", "day_offset"),
|
|
||||||
XdxrRecord: ("year", "month", "day"),
|
|
||||||
SecurityQuote: ("bid", "ask"),
|
|
||||||
}
|
|
||||||
|
|
||||||
#: 模型 -> 在「第一个被丢弃字段的位置」插入的增强列
|
|
||||||
_EXTRAS: dict[type, tuple[tuple[str, Callable[[Any], Any]], ...]] = {
|
|
||||||
SecurityBar: (("datetime", _moment), ("trading_date", _trading_date)),
|
|
||||||
MinuteBar: (("datetime", _moment), ("trading_date", _trading_date)),
|
|
||||||
XdxrRecord: (("date", _xdxr_day),),
|
|
||||||
SecurityQuote: tuple(
|
|
||||||
(f"{side}{level + 1}_{part}", _level_getter(side, level, pos))
|
|
||||||
for side in ("bid", "ask")
|
|
||||||
for level in range(5)
|
|
||||||
for pos, part in ((0, "price"), (1, "vol"))
|
|
||||||
),
|
|
||||||
}
|
|
||||||
|
|
||||||
#: 需要 JSON 化的字典字段(按类区分,避免误伤 ``FinancialRecord.fields: list[float]``)
|
|
||||||
_JSON_FIELDS: dict[type, frozenset[str]] = {
|
|
||||||
SecurityQuote: frozenset({"fields"}),
|
|
||||||
TransactionRecord: frozenset({"fields"}),
|
|
||||||
MinuteBar: frozenset({"fields"}),
|
|
||||||
}
|
|
||||||
|
|
||||||
_PLAN_CACHE: dict[type, tuple[tuple[str, Callable[[Any], Any]], ...]] = {}
|
|
||||||
|
|
||||||
|
|
||||||
def _plan_for(cls: type) -> tuple[tuple[str, Callable[[Any], Any]], ...]:
|
|
||||||
"""构造(并缓存)某模型的输出列计划。
|
|
||||||
|
|
||||||
查找顺序按 MRO 走:``IndexBar`` 没有自己的计划,但它是 ``SecurityBar`` 的子类,
|
|
||||||
直接复用父类计划即可(字段完全相同)。
|
|
||||||
"""
|
|
||||||
cached = _PLAN_CACHE.get(cls)
|
|
||||||
if cached is not None:
|
|
||||||
return cached
|
|
||||||
|
|
||||||
drops: tuple[str, ...] = ()
|
|
||||||
extras: tuple[tuple[str, Callable[[Any], Any]], ...] = ()
|
|
||||||
json_fields: frozenset[str] = frozenset()
|
|
||||||
for klass in cls.__mro__:
|
|
||||||
if not drops and klass in _DROPS:
|
|
||||||
drops = _DROPS[klass]
|
|
||||||
if not extras and klass in _EXTRAS:
|
|
||||||
extras = _EXTRAS[klass]
|
|
||||||
if not json_fields and klass in _JSON_FIELDS:
|
|
||||||
json_fields = _JSON_FIELDS[klass]
|
|
||||||
|
|
||||||
import dataclasses
|
|
||||||
|
|
||||||
plan: list[tuple[str, Callable[[Any], Any]]] = []
|
|
||||||
inserted = False
|
|
||||||
for f in dataclasses.fields(cls): # type: ignore[arg-type]
|
|
||||||
if f.name in drops:
|
|
||||||
if not inserted:
|
|
||||||
plan.extend(extras)
|
|
||||||
inserted = True
|
|
||||||
continue
|
|
||||||
getter = _json_getter(f.name) if f.name in json_fields else _getter(f.name)
|
|
||||||
plan.append((f.name, getter))
|
|
||||||
if not inserted:
|
|
||||||
# 没有丢弃字段的模型(如 SecurityInfo):增强列直接追加到末尾
|
|
||||||
plan.extend(extras)
|
|
||||||
|
|
||||||
_PLAN_CACHE[cls] = tuple(plan)
|
|
||||||
return _PLAN_CACHE[cls]
|
|
||||||
|
|
||||||
|
|
||||||
def _columns(cls: type, rows: Sequence[Any]) -> dict[str, list]:
|
|
||||||
"""列式采集:一次遍历填满所有列,比逐行 ``asdict()`` 快得多。"""
|
|
||||||
return {name: [get(row) for row in rows] for name, get in _plan_for(cls)}
|
|
||||||
|
|
||||||
|
|
||||||
def _columns_of_dicts(rows: Sequence[dict]) -> dict[str, list]:
|
|
||||||
"""``list[dict]``(板块排行这类字段不固定的返回)按首次出现顺序并集列名。"""
|
|
||||||
keys: list[str] = []
|
|
||||||
seen: set[str] = set()
|
|
||||||
for row in rows:
|
|
||||||
for key in row:
|
|
||||||
if key not in seen:
|
|
||||||
seen.add(key)
|
|
||||||
keys.append(key)
|
|
||||||
return {key: [row.get(key) for row in rows] for key in keys}
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------- 容器分发
|
|
||||||
|
|
||||||
|
|
||||||
def _model_by_name(name: str | None) -> type | None:
|
|
||||||
if not name:
|
|
||||||
return None
|
|
||||||
obj = getattr(_models, name, None)
|
|
||||||
return obj if isinstance(obj, type) else None
|
|
||||||
|
|
||||||
|
|
||||||
def _resolve_model(data: Any, model: str | None) -> type | None:
|
|
||||||
"""推断元素类型:非空列表看首元素;空列表只能靠调用方给的类型提示。"""
|
|
||||||
if isinstance(data, list) and data:
|
|
||||||
return type(data[0])
|
|
||||||
return _model_by_name(model)
|
|
||||||
|
|
||||||
|
|
||||||
def _frames_of_rows(be: _Backend, cls: type | None, rows: Sequence[Any]) -> Any:
|
|
||||||
if cls is not None and rows and hasattr(cls, "__dataclass_fields__"):
|
|
||||||
return be.frame(_columns(cls, rows))
|
|
||||||
if rows and isinstance(rows[0], dict):
|
|
||||||
return be.frame(_columns_of_dicts(rows))
|
|
||||||
return be.empty(())
|
|
||||||
|
|
||||||
|
|
||||||
def _from_indicator_sets(be: _Backend, items: Sequence[IndicatorSet]) -> Any:
|
|
||||||
frames = []
|
|
||||||
for item in items:
|
|
||||||
cols = _columns(SecurityBar, item.bars)
|
|
||||||
size = len(item.bars)
|
|
||||||
if item.symbol:
|
|
||||||
cols["symbol"] = [item.symbol] * size
|
|
||||||
for name, series in item.indicators.items():
|
|
||||||
values = list(series[:size])
|
|
||||||
if len(values) < size:
|
|
||||||
values.extend([None] * (size - len(values)))
|
|
||||||
cols[name] = values
|
|
||||||
frames.append(be.frame(cols))
|
|
||||||
merged = be.concat(frames)
|
|
||||||
return merged if merged is not None else be.frame(_columns(SecurityBar, []))
|
|
||||||
|
|
||||||
|
|
||||||
def _from_index_trade_stats(be: _Backend, items: Sequence[IndexTradeStats]) -> Any:
|
|
||||||
frames = []
|
|
||||||
for stat in items:
|
|
||||||
cols = _columns(MinuteTradeStat, stat.minutes)
|
|
||||||
size = len(stat.minutes)
|
|
||||||
# 元信息补成常量列:一张表就能看出是哪个指数、哪一天
|
|
||||||
cols = {"index": [stat.index] * size, "date": [stat.date] * size, **cols}
|
|
||||||
frames.append(be.frame(cols))
|
|
||||||
merged = be.concat(frames)
|
|
||||||
return merged if merged is not None else be.frame(_columns(MinuteTradeStat, []))
|
|
||||||
|
|
||||||
|
|
||||||
def to_frame(data: Any, *, backend: str = "polars", model: str | None = None) -> Any:
|
|
||||||
"""把模型返回值转成 DataFrame。
|
|
||||||
|
|
||||||
:param data: ``list[dataclass]`` / ``list[dict]`` / ``dict[str, list[dataclass]]``
|
|
||||||
/ ``list[IndicatorSet]`` / ``IndexTradeStats`` / ``list[IndexTradeStats]``。
|
|
||||||
:param backend: ``"polars"`` 或 ``"pandas"``。
|
|
||||||
:param model: 元素类型的名字(如 ``"SecurityBar"``);只在 ``data`` 为空、
|
|
||||||
无法从元素推断类型时使用,由 :func:`framed` 从返回注解里自动填。
|
|
||||||
"""
|
|
||||||
be = get_backend(backend)
|
|
||||||
|
|
||||||
if isinstance(data, IndexTradeStats):
|
|
||||||
return _from_index_trade_stats(be, [data])
|
|
||||||
if isinstance(data, IndicatorSet):
|
|
||||||
return _from_indicator_sets(be, [data])
|
|
||||||
|
|
||||||
if isinstance(data, dict):
|
|
||||||
# 批量接口:{标的: [行]} → 拼成一张表(symbol/code 列本来就在行里)
|
|
||||||
frames = []
|
|
||||||
for rows in data.values():
|
|
||||||
if isinstance(rows, list) and rows:
|
|
||||||
frames.append(_frames_of_rows(be, type(rows[0]), rows))
|
|
||||||
merged = be.concat(frames)
|
|
||||||
if merged is not None:
|
|
||||||
return merged
|
|
||||||
cls = _model_by_name(model)
|
|
||||||
return be.empty(tuple(name for name, _ in _plan_for(cls)) if cls else ())
|
|
||||||
|
|
||||||
if isinstance(data, list):
|
|
||||||
if data and isinstance(data[0], IndexTradeStats):
|
|
||||||
return _from_index_trade_stats(be, data)
|
|
||||||
if data and isinstance(data[0], IndicatorSet):
|
|
||||||
return _from_indicator_sets(be, data)
|
|
||||||
cls = _resolve_model(data, model)
|
|
||||||
if data and isinstance(data[0], dict):
|
|
||||||
return _frames_of_rows(be, None, data)
|
|
||||||
if cls is not None and data and hasattr(cls, "__dataclass_fields__"):
|
|
||||||
return _frames_of_rows(be, cls, data)
|
|
||||||
if data:
|
|
||||||
# 元素既不是 dataclass 也不是 dict(如 list[float])——不该走到这里
|
|
||||||
raise TypeError(f"无法转换为 DataFrame 的列表元素:{type(data[0])!r}")
|
|
||||||
return be.frame({name: [] for name, _ in _plan_for(cls)}) if cls else be.empty(())
|
|
||||||
|
|
||||||
raise TypeError(f"无法转换为 DataFrame 的返回值类型:{type(data)!r}")
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------- 接线
|
|
||||||
|
|
||||||
|
|
||||||
_RETURN_MODEL = re.compile(r"list\[(\w+)\]")
|
|
||||||
|
|
||||||
|
|
||||||
def framed(method: Callable[..., Any]) -> Callable[..., Any]:
|
|
||||||
"""给 ``TdxData`` 的表格型方法套一层「按需转 DataFrame」。
|
|
||||||
|
|
||||||
- ``self._df_backend is None``(默认)时零开销,原样返回 ``list[dataclass]``。
|
|
||||||
- 处于某次公开方法执行过程中(:data:`_FRAME_DEPTH` > 0)时不转换,
|
|
||||||
保证批量/别名方法内部拿到的仍是原始 ``list``。
|
|
||||||
"""
|
|
||||||
model: str | None = None
|
|
||||||
annotation = getattr(method, "__annotations__", {}).get("return")
|
|
||||||
if isinstance(annotation, str):
|
|
||||||
matched = _RETURN_MODEL.search(annotation)
|
|
||||||
if matched:
|
|
||||||
model = matched.group(1)
|
|
||||||
|
|
||||||
@functools.wraps(method)
|
|
||||||
async def wrapper(self: Any, *args: Any, **kwargs: Any) -> Any:
|
|
||||||
backend = getattr(self, "_df_backend", None)
|
|
||||||
if backend is None or _FRAME_DEPTH.get() > 0:
|
|
||||||
return await method(self, *args, **kwargs)
|
|
||||||
|
|
||||||
token = _FRAME_DEPTH.set(1)
|
|
||||||
try:
|
|
||||||
result = await method(self, *args, **kwargs)
|
|
||||||
finally:
|
|
||||||
_FRAME_DEPTH.reset(token)
|
|
||||||
return to_frame(result, backend=backend, model=model)
|
|
||||||
|
|
||||||
return wrapper
|
|
||||||
@@ -103,10 +103,6 @@ _US_EX_MARKETS: frozenset[int] = frozenset({int(ExMarket.US_STOCK)})
|
|||||||
_US_DAY_SPLIT_HOUR = 12
|
_US_DAY_SPLIT_HOUR = 12
|
||||||
|
|
||||||
|
|
||||||
# SecurityBar 上「后加的」可选字段:旧磁盘缓存反序列化时要补默认值(见 __setstate__)
|
|
||||||
_BAR_LATE_FIELDS: tuple[str, ...] = ("settle",)
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass(slots=True)
|
@dataclass(slots=True)
|
||||||
class SecurityBar:
|
class SecurityBar:
|
||||||
"""K线记录(价格单位:元)。跨市场统一模型(A股/港股/美股/期货共用)。
|
"""K线记录(价格单位:元)。跨市场统一模型(A股/港股/美股/期货共用)。
|
||||||
@@ -114,10 +110,6 @@ class SecurityBar:
|
|||||||
时间戳口径**统一为北京时间**:A股/港股/期货原始即为北京时间;美股原始是
|
时间戳口径**统一为北京时间**:A股/港股/期货原始即为北京时间;美股原始是
|
||||||
「日期=美东交易日 + 时分=北京时间」的混合值,构造时已还原为真实北京时间
|
「日期=美东交易日 + 时分=北京时间」的混合值,构造时已还原为真实北京时间
|
||||||
(见 ``__post_init__``)。该 bar 归属的交易日见 :attr:`trading_date`。
|
(见 ``__post_init__``)。该 bar 归属的交易日见 :attr:`trading_date`。
|
||||||
|
|
||||||
``amount`` / ``settle`` 的可取性**按通道**:A股标准/MAC 通道两者都有;
|
|
||||||
EX 通道(0x23FF)一个 32B 记录里只有**一个** 4B 槽位可复用,权益类市场用它
|
|
||||||
装成交额、期货期权类用它装结算价,两者不可能同时拿到 —— 详见 :attr:`amount`。
|
|
||||||
"""
|
"""
|
||||||
market: int
|
market: int
|
||||||
code: str
|
code: str
|
||||||
@@ -125,36 +117,14 @@ class SecurityBar:
|
|||||||
high: float
|
high: float
|
||||||
low: float
|
low: float
|
||||||
close: float
|
close: float
|
||||||
# vol 单位按品种:A股/ETF/美股=股;转债=手(1手=10张);⚠️港股主板/基金=手(1手=100股)
|
vol: float # 股票=股 / 转债=手(1 手=10 张,与 vipdoc 一致)
|
||||||
vol: float
|
amount: float # 元。⚠️ amount/vol 只在股票上等于均价;转债还要再 /10(手=10张)
|
||||||
# 元。⚠️ amount/vol 只在 A股上等于均价;转债还要再 /10(手=10张)、港股再 /100(手=100股)。
|
|
||||||
# ⚠️ EX 通道的期货/期权(中金所、上期所、郑商所、大商所及各期权)协议帧内**没有成交额**,
|
|
||||||
# 恒为 0.0 —— 要成交额请改用 EX 报价(``get_quotes``)或其它数据源。
|
|
||||||
amount: float
|
|
||||||
year: int
|
year: int
|
||||||
month: int
|
month: int
|
||||||
day: int
|
day: int
|
||||||
hour: int = 0
|
hour: int = 0
|
||||||
minute: int = 0
|
minute: int = 0
|
||||||
symbol: str = "" # 规范化标的串(sz000001 / hk00700),由 API 层填充
|
symbol: str = "" # 规范化标的串(sz000001 / hk00700),由 API 层填充
|
||||||
# 结算价。仅 EX 通道的**期货/期权**有(日线才有值,分钟线为 None);其余通道恒 None。
|
|
||||||
settle: float | None = None
|
|
||||||
|
|
||||||
def __setstate__(self, state: Any) -> None:
|
|
||||||
"""反序列化兜底:早期版本写下的磁盘缓存里没有后加的可选字段(如 ``settle``)。
|
|
||||||
|
|
||||||
``@dataclass(slots=True)`` 的默认反序列化只把 pickle 里存在的槽位塞回去,
|
|
||||||
缺键的槽位**不会**被赋默认值 —— 直接读 ``bar.settle`` 会抛 ``AttributeError``。
|
|
||||||
缓存键里没有 schema 版本号,所以这里必须容错,否则升级后读旧缓存即崩。
|
|
||||||
"""
|
|
||||||
if isinstance(state, tuple) and len(state) == 2: # slots 形式的 (dict, slots)
|
|
||||||
data = dict(state[1] or {})
|
|
||||||
else:
|
|
||||||
data = dict(state or {})
|
|
||||||
for name in _BAR_LATE_FIELDS:
|
|
||||||
data.setdefault(name, None)
|
|
||||||
for key, value in data.items():
|
|
||||||
object.__setattr__(self, key, value)
|
|
||||||
|
|
||||||
def __post_init__(self) -> None:
|
def __post_init__(self) -> None:
|
||||||
"""美股(EX 通道)时间戳归一化:还原成真实北京时间。
|
"""美股(EX 通道)时间戳归一化:还原成真实北京时间。
|
||||||
|
|||||||
@@ -4,24 +4,9 @@ from __future__ import annotations
|
|||||||
import struct
|
import struct
|
||||||
|
|
||||||
from ..._binary import unpack_from
|
from ..._binary import unpack_from
|
||||||
from ...models import ExMarket
|
|
||||||
from ..datetime_ import get_datetime
|
from ..datetime_ import get_datetime
|
||||||
from .base import BaseCommand
|
from .base import BaseCommand
|
||||||
|
|
||||||
# 0x23FF 的「槽位 5 是成交额」的市场集合(其余市场该槽位是持仓量、槽位 7 是结算价)。
|
|
||||||
# 实测依据(2026-10-28,日线,逐条校验 成交额/(close×vol)):
|
|
||||||
# 31 港股主板 = 100.0(vol 单位手,1手=100股)、49 港股基金 = 100.3、
|
|
||||||
# 48 港股创业板 ≈1.0、44 新三板 ≈0.99、74 美股 ≈1.0(vol 单位股)。
|
|
||||||
# ⚠️ 27 港股指数 / 62 中证指数 该槽位既非成交额也非结算价(量级对不上),故**不在**表内,
|
|
||||||
# 保持 amount=0.0 的原始行为,避免给出可疑数值。
|
|
||||||
_EX_AMOUNT_MARKETS: frozenset[int] = frozenset({
|
|
||||||
int(ExMarket.HK_MAIN_BOARD),
|
|
||||||
int(ExMarket.HK_GEM),
|
|
||||||
int(ExMarket.HK_FUND),
|
|
||||||
int(ExMarket.NEEQ),
|
|
||||||
int(ExMarket.US_STOCK),
|
|
||||||
})
|
|
||||||
|
|
||||||
# pytdx EX setup 握手命令(完整84B: 12B帧头+72B body, 0x2454)—— 与MAC EX login同ID不同body
|
# pytdx EX setup 握手命令(完整84B: 12B帧头+72B body, 0x2454)—— 与MAC EX login同ID不同body
|
||||||
EX_SETUP_CMD = bytes.fromhex("0101486500015200520054241f32c6e5d53dfb411f32c6e5d53dfb411f32c6e5d53dfb411f32c6e5d53dfb411f32c6e5d53dfb411f32c6e5d53dfb411f32c6e5d53dfb411f32c6e5d53dfb41cce16dffd5ba3fb8cbc57a054f7748ea")
|
EX_SETUP_CMD = bytes.fromhex("0101486500015200520054241f32c6e5d53dfb411f32c6e5d53dfb411f32c6e5d53dfb411f32c6e5d53dfb411f32c6e5d53dfb411f32c6e5d53dfb411f32c6e5d53dfb411f32c6e5d53dfb41cce16dffd5ba3fb8cbc57a054f7748ea")
|
||||||
|
|
||||||
@@ -72,24 +57,7 @@ class GetMarketsCmd(BaseCommand[list[dict]]):
|
|||||||
|
|
||||||
|
|
||||||
class GetExInstrumentBarsCmd(BaseCommand[list[dict]]):
|
class GetExInstrumentBarsCmd(BaseCommand[list[dict]]):
|
||||||
"""EX商品K线(0x23FF, 32B/条: 4B时间戳+28B行情)。
|
"""EX商品K线(0x23FF, 32B/条: 4B时间戳+28B行情)。"""
|
||||||
|
|
||||||
后 28B 的字段语义**按市场分两类**(2026-10-28 全市场实测,见 ``_EX_AMOUNT_MARKETS``):
|
|
||||||
|
|
||||||
* 权益类(港股主板/创业板/基金、新三板、美股)::
|
|
||||||
|
|
||||||
open(4) high(4) low(4) close(4) 成交额(4f) 成交量(4I) 保留(4f)
|
|
||||||
|
|
||||||
成交额可用;校验 ``成交额 / (close × 成交量)`` = 100.0(港股主板/基金,
|
|
||||||
``vol`` 单位是「手」=100 股)或 ≈1.0(港股创业板/新三板/美股,``vol`` 单位是股)。
|
|
||||||
|
|
||||||
* 期货期权类(中金所/上期所/郑商所/大商所及各期权)::
|
|
||||||
|
|
||||||
open(4) high(4) low(4) close(4) 持仓量(4I) 成交量(4I) 结算价(4f)
|
|
||||||
|
|
||||||
**协议帧内没有成交额**(``amount`` 恒 0.0,不是解析漏读);末位 float 是结算价,
|
|
||||||
日线有值、分钟线为 0(``settle`` 归一化成 ``None``)。
|
|
||||||
"""
|
|
||||||
|
|
||||||
def __init__(self, market: int, code: str, category: int = 0,
|
def __init__(self, market: int, code: str, category: int = 0,
|
||||||
start: int = 0, count: int = 700) -> None:
|
start: int = 0, count: int = 700) -> None:
|
||||||
@@ -108,7 +76,6 @@ class GetExInstrumentBarsCmd(BaseCommand[list[dict]]):
|
|||||||
if len(body) < 20:
|
if len(body) < 20:
|
||||||
return []
|
return []
|
||||||
(ret_count,) = unpack_from("<H", body, 18, "ex bars count")
|
(ret_count,) = unpack_from("<H", body, 18, "ex bars count")
|
||||||
amount_market = self.market in _EX_AMOUNT_MARKETS
|
|
||||||
items: list[dict] = []
|
items: list[dict] = []
|
||||||
off = 20
|
off = 20
|
||||||
for _ in range(ret_count):
|
for _ in range(ret_count):
|
||||||
@@ -116,23 +83,14 @@ class GetExInstrumentBarsCmd(BaseCommand[list[dict]]):
|
|||||||
break
|
break
|
||||||
# 时间4字节按category分格式:分钟级=2B zipday+2B tminutes;日线及以上=YYYYMMDD
|
# 时间4字节按category分格式:分钟级=2B zipday+2B tminutes;日线及以上=YYYYMMDD
|
||||||
year, month, day, hour, minute, _ = get_datetime(self.category, body, off)
|
year, month, day, hour, minute, _ = get_datetime(self.category, body, off)
|
||||||
raw = body[off + 4: off + 32]
|
open_, high, low, close, position, trade, _ = struct.unpack(
|
||||||
open_, high, low, close = struct.unpack("<ffff", raw[:16])
|
"<ffffIIf", body[off + 4: off + 32])
|
||||||
item = {
|
items.append({
|
||||||
"year": year, "month": month, "day": day,
|
"year": year, "month": month, "day": day,
|
||||||
"hour": hour, "minute": minute,
|
"hour": hour, "minute": minute,
|
||||||
"open": open_, "high": high, "low": low, "close": close,
|
"open": open_, "high": high, "low": low, "close": close,
|
||||||
}
|
"position": position, "trade": trade,
|
||||||
if amount_market:
|
})
|
||||||
# 权益类:槽位 5 = 成交额(float),槽位 7 语义未知(保留不用)
|
|
||||||
amount, trade, _reserved = struct.unpack("<fIf", raw[16:28])
|
|
||||||
item.update(amount=amount, trade=trade, position=0, settle=None)
|
|
||||||
else:
|
|
||||||
# 期货/期权:槽位 5 = 持仓量,槽位 7 = 结算价(分钟线为 0 → None)
|
|
||||||
position, trade, settle = struct.unpack("<IIf", raw[16:28])
|
|
||||||
item.update(amount=0.0, trade=trade, position=position,
|
|
||||||
settle=settle or None)
|
|
||||||
items.append(item)
|
|
||||||
off += 32
|
off += 32
|
||||||
return items
|
return items
|
||||||
|
|
||||||
|
|||||||
@@ -36,12 +36,8 @@ class GetSecurityListCmd(BaseCommand[list[SecurityInfo]]):
|
|||||||
self.start = start
|
self.start = start
|
||||||
|
|
||||||
def build_request(self) -> bytes:
|
def build_request(self) -> bytes:
|
||||||
# 12 字节头 + market(H) + start(H) = **16 字节**。
|
|
||||||
# ⚠️ 曾经多塞一个 H(18 字节):服务端只消费前 16 字节,剩下的 00 00
|
|
||||||
# 留在它的接收缓冲里,会把**下一个请求**的开头吃掉 → 该连接的后续请求
|
|
||||||
# 永远得不到响应(表现为「第一次成功、第二次起全部通信超时」)。
|
|
||||||
header = bytes.fromhex("0c0118640101060006005004".replace(" ", ""))
|
header = bytes.fromhex("0c0118640101060006005004".replace(" ", ""))
|
||||||
return header + struct.pack("<HH", self.market, self.start)
|
return header + struct.pack("<HHH", self.market, self.start, 0)
|
||||||
|
|
||||||
def parse_response(self, body: bytes) -> list[SecurityInfo]:
|
def parse_response(self, body: bytes) -> list[SecurityInfo]:
|
||||||
(num,) = unpack_from("<H", body, 0, "security list header")
|
(num,) = unpack_from("<H", body, 0, "security list header")
|
||||||
|
|||||||
@@ -1,31 +0,0 @@
|
|||||||
"""测试全局隔离。
|
|
||||||
|
|
||||||
默认关掉两个**后台网络行为**:
|
|
||||||
- 权威小数位表同步:否则任何调用 ``TdxData.start()`` 的用例都会顺手发起证券
|
|
||||||
清单请求(26 + 25 个分页),测试变慢且依赖网络。
|
|
||||||
- 当日服务器测速:后台 task 会真的去连主机取K线,会污染「往返次数」这类
|
|
||||||
断言(实测同一个用例的 ``calls`` 在 3/4 之间随机跳),也会拖慢整体。
|
|
||||||
|
|
||||||
需要验证这两个行为本身的用例(``test_decimal_sync.py`` / ``test_probe_cache.py``)
|
|
||||||
自己 ``delenv`` 后再测。
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import os
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
_ENV_KEYS = ("PYPYTDX_DATA_DISABLE_DECIMAL_SYNC", "PYPYTDX_DATA_DISABLE_PROBE")
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.fixture(autouse=True, scope="session")
|
|
||||||
def _disable_background_network():
|
|
||||||
old = {k: os.environ.get(k) for k in _ENV_KEYS}
|
|
||||||
for k in _ENV_KEYS:
|
|
||||||
os.environ[k] = "1"
|
|
||||||
yield
|
|
||||||
for k, v in old.items():
|
|
||||||
if v is None:
|
|
||||||
os.environ.pop(k, None)
|
|
||||||
else:
|
|
||||||
os.environ[k] = v
|
|
||||||
@@ -324,27 +324,6 @@ async def test_universe_and_quotes(td):
|
|||||||
assert len(qex) == 1
|
assert len(qex) == 1
|
||||||
|
|
||||||
|
|
||||||
async def test_decimal_point_sync(td, tmp_path, monkeypatch):
|
|
||||||
"""权威小数位表同步:把证券清单的 decimal_point 灌进价格标度表。"""
|
|
||||||
for name in ("sync_decimal_points", "decimal_point_status"):
|
|
||||||
_use(name)
|
|
||||||
from pytdxdata.protocol.commands.quotes import clear_decimal_map, price_decimal_digits
|
|
||||||
|
|
||||||
# 缓存目录指向临时目录,避免把 mock 数据写进用户真实缓存
|
|
||||||
monkeypatch.setenv("PYPYTDX_DATA_CACHE_DIR", str(tmp_path))
|
|
||||||
before = price_decimal_digits(1, "600000")
|
|
||||||
try:
|
|
||||||
st = await td.sync_decimal_points()
|
|
||||||
assert set(st) >= {"prefixes", "codes", "mixed_prefixes", "stock"}
|
|
||||||
assert st["stock"] >= 0
|
|
||||||
status = await td.decimal_point_status()
|
|
||||||
assert isinstance(status, dict) and "source" in status
|
|
||||||
finally:
|
|
||||||
clear_decimal_map()
|
|
||||||
# 清空后回到静态规则,不受本用例污染
|
|
||||||
assert price_decimal_digits(1, "600000") == before
|
|
||||||
|
|
||||||
|
|
||||||
async def test_snapshot_auction_flow(td):
|
async def test_snapshot_auction_flow(td):
|
||||||
for name in ("get_snapshot", "get_auction", "get_capital_flow"):
|
for name in ("get_snapshot", "get_auction", "get_capital_flow"):
|
||||||
_use(name)
|
_use(name)
|
||||||
@@ -478,24 +457,6 @@ async def test_boards(td):
|
|||||||
assert isinstance(chg, list)
|
assert isinstance(chg, list)
|
||||||
|
|
||||||
|
|
||||||
async def test_index_trade_stats(td):
|
|
||||||
"""指数成分成交汇总:成分 → 逐笔 → 每分钟量/笔数。"""
|
|
||||||
for name in ("get_index_members", "get_index_trade_stats",
|
|
||||||
"get_index_trade_stats_multi"):
|
|
||||||
_use(name)
|
|
||||||
mem = await td.get_index_members("000016")
|
|
||||||
assert isinstance(mem, list) and mem and mem[0].symbol
|
|
||||||
st = await td.get_index_trade_stats("000016")
|
|
||||||
assert st.index == "sh000016"
|
|
||||||
assert st.member_count >= 1
|
|
||||||
assert st.minutes and st.minutes[0].time == "09:30"
|
|
||||||
assert st.total_trade_count >= 1
|
|
||||||
assert st.cum_trade_count()[-1][1] == st.total_trade_count
|
|
||||||
multi = await td.get_index_trade_stats_multi("000016", [20261020, None])
|
|
||||||
assert len(multi) == 2
|
|
||||||
assert multi[0].date == 20261020 and multi[1].date is None
|
|
||||||
|
|
||||||
|
|
||||||
# --------------------------------------------------------------------------- #
|
# --------------------------------------------------------------------------- #
|
||||||
# 六、服务器文件
|
# 六、服务器文件
|
||||||
# --------------------------------------------------------------------------- #
|
# --------------------------------------------------------------------------- #
|
||||||
|
|||||||
+1
-211
@@ -8,13 +8,7 @@ import unittest.mock as mock
|
|||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
from pytdxdata.api import TdxData
|
from pytdxdata.api import TdxData
|
||||||
from pytdxdata.models import (
|
from pytdxdata.models import KlinePeriod, Market, MinuteBar, SecurityBar
|
||||||
KlinePeriod,
|
|
||||||
Market,
|
|
||||||
MinuteBar,
|
|
||||||
SecurityBar,
|
|
||||||
TransactionRecord,
|
|
||||||
)
|
|
||||||
from pytdxdata.protocol.commands.mac_tick import MacTickChartCmd
|
from pytdxdata.protocol.commands.mac_tick import MacTickChartCmd
|
||||||
from pytdxdata.scheduler.paginator import PageSpec, Paginator
|
from pytdxdata.scheduler.paginator import PageSpec, Paginator
|
||||||
|
|
||||||
@@ -249,70 +243,6 @@ async def test_paginator_start_offset_default_zero():
|
|||||||
assert result[0].close == 10.0 # 从 offset 0 开始
|
assert result[0].close == 10.0 # 从 offset 0 开始
|
||||||
|
|
||||||
|
|
||||||
# ---------- Paginator 逆序偏移(服务端 start 从最新往回数) ----------
|
|
||||||
|
|
||||||
class NewestFirstConn:
|
|
||||||
"""模拟 0x122F 真实行为:start=0 返回**最新**的一页,start 越大越旧。
|
|
||||||
|
|
||||||
实测依据(2026-10-21,sh600000):
|
|
||||||
start=0 -> 14:07:48 .. 15:20:16(当天最新 1000 条)
|
|
||||||
start=4000 -> 09:15:03 .. 09:41:18(当天最早)
|
|
||||||
每页内部按时间升序,页与页之间是倒序。
|
|
||||||
"""
|
|
||||||
|
|
||||||
def __init__(self, total: int, page_size: int):
|
|
||||||
self.total = total
|
|
||||||
self.page_size = page_size
|
|
||||||
self.executed: list[tuple[int, int]] = []
|
|
||||||
|
|
||||||
async def execute(self, cmd):
|
|
||||||
start, count = cmd
|
|
||||||
self.executed.append((start, count))
|
|
||||||
# 服务端语义:偏移 start 是从"最新"往回数,所以取倒数区间
|
|
||||||
end_from_oldest = self.total - start
|
|
||||||
begin_from_oldest = max(0, end_from_oldest - count)
|
|
||||||
return [_make_bar(i) for i in range(begin_from_oldest, end_from_oldest)]
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_paginator_newest_first_offset_returns_chronological():
|
|
||||||
"""服务端 start 从最新往回数时,交付列表必须仍是时间升序(BUG 回归)。"""
|
|
||||||
conns = [NewestFirstConn(3000, 1000) for _ in range(3)]
|
|
||||||
pool = FakePool(conns)
|
|
||||||
pag = Paginator(pool, concurrency=3) # type: ignore[arg-type]
|
|
||||||
|
|
||||||
def factory(s, c):
|
|
||||||
return (s, c)
|
|
||||||
|
|
||||||
result = await pag.fetch_all(
|
|
||||||
factory, PageSpec(page_size=1000, total=3000, newest_first=True))
|
|
||||||
assert len(result) == 3000
|
|
||||||
# 时间升序:close 随 index 单调递增
|
|
||||||
closes = [b.close for b in result]
|
|
||||||
assert closes == sorted(closes), "返回列表必须按时间升序拼接"
|
|
||||||
assert result[0].close == pytest.approx(10.0)
|
|
||||||
assert result[-1].close == pytest.approx(10.0 + 2999 * 0.01)
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_paginator_newest_first_respects_start_offset():
|
|
||||||
"""newest_first + start_offset:只取最近 N 条里的后段。"""
|
|
||||||
conns = [NewestFirstConn(3000, 1000) for _ in range(3)]
|
|
||||||
pool = FakePool(conns)
|
|
||||||
pag = Paginator(pool, concurrency=3) # type: ignore[arg-type]
|
|
||||||
|
|
||||||
def factory(s, c):
|
|
||||||
return (s, c)
|
|
||||||
|
|
||||||
result = await pag.fetch_all(
|
|
||||||
factory,
|
|
||||||
PageSpec(page_size=1000, total=1000, newest_first=True, start_offset=0))
|
|
||||||
assert len(result) == 1000
|
|
||||||
# 只取了最新的 1000 条 → 是 2000..2999 这一段
|
|
||||||
assert result[0].close == pytest.approx(10.0 + 2000 * 0.01)
|
|
||||||
assert result[-1].close == pytest.approx(10.0 + 2999 * 0.01)
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- get_kline 日期过滤 ----------
|
# ---------- get_kline 日期过滤 ----------
|
||||||
|
|
||||||
class DateBarApiConn(FakeApiConn):
|
class DateBarApiConn(FakeApiConn):
|
||||||
@@ -399,143 +329,3 @@ async def test_get_kline_since():
|
|||||||
assert bars[-1].year == 2027
|
assert bars[-1].year == 2027
|
||||||
finally:
|
finally:
|
||||||
await td.close()
|
await td.close()
|
||||||
|
|
||||||
# ---------- A股逐笔:b8 过滤 + 单连接串行游标(回归) ----------
|
|
||||||
|
|
||||||
class TickApiConn(FakeApiConn):
|
|
||||||
"""模拟 0x122F:服务端 start 从**最新**往回数,且可注入 bs_flag=8 委托记录。"""
|
|
||||||
|
|
||||||
# 类级:[(time, price, vol, trade_count, bs_flag), ...],时间升序(最早在前)
|
|
||||||
rows: list[tuple] = []
|
|
||||||
calls: list[tuple[int, int]] = []
|
|
||||||
|
|
||||||
async def execute(self, cmd):
|
|
||||||
if "MacTransactionCmd" not in type(cmd).__name__:
|
|
||||||
return await super().execute(cmd)
|
|
||||||
TickApiConn.calls.append((cmd.start, cmd.count))
|
|
||||||
# 从最新往回数:start 偏移落在末尾,往前取 count 条
|
|
||||||
end = len(TickApiConn.rows) - cmd.start
|
|
||||||
begin = max(0, end - cmd.count)
|
|
||||||
chunk = TickApiConn.rows[begin:end]
|
|
||||||
return [TransactionRecord(market=1, code="600000", time=t, price=p,
|
|
||||||
vol=v, trade_count=tc, bs_flag=bs)
|
|
||||||
for (t, p, v, tc, bs) in chunk]
|
|
||||||
|
|
||||||
|
|
||||||
def _build_tick_rows(n_trade: int = 25, n_b8: int = 7) -> list[tuple]:
|
|
||||||
"""构造 n_trade 条正常成交 + n_b8 条竞价委托(穿插在中间)。"""
|
|
||||||
rows: list[tuple] = []
|
|
||||||
for i in range(n_trade):
|
|
||||||
rows.append((f"09:{30 + i // 60:02d}:{i % 60:02d}", 10.0 + i * 0.01,
|
|
||||||
100.0 + i, 3 + i, 1 if i % 2 else 0))
|
|
||||||
if i % 4 == 0 and len([r for r in rows if r[4] == 8]) < n_b8:
|
|
||||||
# 竞价委托:(time, price, vol, tc, bs=8),vol 是累计值、不计入成交
|
|
||||||
rows.append((f"09:15:{i:02d}", 10.0, 9999.0, -1, 8))
|
|
||||||
return rows
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_get_ticks_filters_auction_orders():
|
|
||||||
"""bs_flag=8 是竞价**委托**快照,不是成交 → 必须从 get_ticks 结果中剔除。
|
|
||||||
|
|
||||||
实测(2026-10-21 sh600000):b8 的 vol 是累计撮合量(可被覆盖),
|
|
||||||
计入后 Σvol 比快照 vol 虚高 20%~160%,且不同主机的 b8 集合不同。
|
|
||||||
"""
|
|
||||||
TickApiConn.rows = _build_tick_rows(n_trade=25, n_b8=7)
|
|
||||||
TickApiConn.calls = []
|
|
||||||
assert sum(1 for r in TickApiConn.rows if r[4] == 8) == 7
|
|
||||||
|
|
||||||
with mock.patch("pytdxdata.api.get_standard_hosts", return_value=["s1"]), \
|
|
||||||
mock.patch("pytdxdata.api.get_mac_hosts", return_value=["m1"]), \
|
|
||||||
mock.patch("pytdxdata.pool.pool.TdxConnection",
|
|
||||||
side_effect=lambda h, p=7709, t=10.0: TickApiConn(h)):
|
|
||||||
td = TdxData(cache_dir=None, pool_min=1, pool_max=2)
|
|
||||||
await td.start()
|
|
||||||
try:
|
|
||||||
out = await td.get_ticks("sh600000")
|
|
||||||
finally:
|
|
||||||
await td.close()
|
|
||||||
|
|
||||||
assert out, "应返回成交记录"
|
|
||||||
assert all(r.bs_flag != 8 for r in out), "bs_flag=8 竞价委托必须被剔除"
|
|
||||||
assert len(out) == 25
|
|
||||||
assert sum(r.vol for r in out) == sum(100.0 + i for i in range(25))
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_get_ticks_pages_on_single_connection():
|
|
||||||
"""翻页必须走**同一连接**:并发多主机会让页边界错位(重叠 + 漏条)。"""
|
|
||||||
TickApiConn.rows = _build_tick_rows(n_trade=25, n_b8=0)
|
|
||||||
TickApiConn.calls = []
|
|
||||||
|
|
||||||
with mock.patch("pytdxdata.api.get_standard_hosts", return_value=["s1"]), \
|
|
||||||
mock.patch("pytdxdata.api.get_mac_hosts", return_value=["m1", "m2", "m3"]), \
|
|
||||||
mock.patch("pytdxdata.pool.pool.TdxConnection",
|
|
||||||
side_effect=lambda h, p=7709, t=10.0: TickApiConn(h)):
|
|
||||||
td = TdxData(cache_dir=None, pool_min=1, pool_max=4)
|
|
||||||
await td.start()
|
|
||||||
try:
|
|
||||||
out = await td.get_ticks("sh600000")
|
|
||||||
finally:
|
|
||||||
await td.close()
|
|
||||||
|
|
||||||
# 单页装得下 → 只发一次请求
|
|
||||||
assert len(TickApiConn.calls) == 1
|
|
||||||
assert len(out) == 25
|
|
||||||
# 时间升序
|
|
||||||
times = [r.time for r in out]
|
|
||||||
assert times == sorted(times)
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_get_ticks_multi_page_uses_one_connection_and_no_overlap():
|
|
||||||
"""多页(>_TICK_PAGE):同一连接、游标按 len(rows) 推进、页间不重叠不漏条。
|
|
||||||
|
|
||||||
回归:并发分页时各页来自不同主机,而 0x122F 的 start 是「从最新往回数」的
|
|
||||||
相对锚点 —— 各主机对 bs_flag=8 收录不同,同一 start 落到不同真实成交上,
|
|
||||||
结果页边界既重叠又漏条(实测 sh600000 冗余 116 条、13:03 附近丢 59 条)。
|
|
||||||
"""
|
|
||||||
from pytdxdata.api import _TICK_PAGE
|
|
||||||
|
|
||||||
n = _TICK_PAGE * 2 + 137 # 3 页,末页短页 → 触发 stop_when_short
|
|
||||||
TickApiConn.rows = [
|
|
||||||
(f"{9 + i // 3600:02d}:{i // 60 % 60:02d}:{i % 60:02d}",
|
|
||||||
10.0 + i * 0.001, 1.0 + i, 1 + i, 1)
|
|
||||||
for i in range(n)
|
|
||||||
]
|
|
||||||
TickApiConn.calls = []
|
|
||||||
|
|
||||||
with mock.patch("pytdxdata.api.get_standard_hosts", return_value=["s1"]), \
|
|
||||||
mock.patch("pytdxdata.api.get_mac_hosts", return_value=["m1", "m2", "m3"]), \
|
|
||||||
mock.patch("pytdxdata.pool.pool.TdxConnection",
|
|
||||||
side_effect=lambda h, p=7709, t=10.0: TickApiConn(h)):
|
|
||||||
td = TdxData(cache_dir=None, pool_min=1, pool_max=4)
|
|
||||||
await td.start()
|
|
||||||
try:
|
|
||||||
out = await td.get_ticks("sh600000")
|
|
||||||
finally:
|
|
||||||
await td.close()
|
|
||||||
|
|
||||||
# 恰好 3 页:0 / 1000 / 2000,游标按实际返回条数推进
|
|
||||||
assert TickApiConn.calls == [(0, _TICK_PAGE),
|
|
||||||
(_TICK_PAGE, _TICK_PAGE),
|
|
||||||
(_TICK_PAGE * 2, _TICK_PAGE)]
|
|
||||||
assert len(out) == n, f"必须无重叠、无漏条:期望 {n},实得 {len(out)}"
|
|
||||||
vols = [r.vol for r in out]
|
|
||||||
assert len(set(vols)) == n, "vol 全唯一(无重复记录)"
|
|
||||||
assert vols == sorted(vols), "时间升序"
|
|
||||||
|
|
||||||
# count 参数:游标逐页扣减
|
|
||||||
TickApiConn.calls = []
|
|
||||||
with mock.patch("pytdxdata.api.get_standard_hosts", return_value=["s1"]), \
|
|
||||||
mock.patch("pytdxdata.api.get_mac_hosts", return_value=["m1", "m2", "m3"]), \
|
|
||||||
mock.patch("pytdxdata.pool.pool.TdxConnection",
|
|
||||||
side_effect=lambda h, p=7709, t=10.0: TickApiConn(h)):
|
|
||||||
td = TdxData(cache_dir=None, pool_min=1, pool_max=4)
|
|
||||||
await td.start()
|
|
||||||
try:
|
|
||||||
out2 = await td.get_ticks("sh600000", count=_TICK_PAGE + 50)
|
|
||||||
finally:
|
|
||||||
await td.close()
|
|
||||||
assert len(out2) == _TICK_PAGE + 50
|
|
||||||
assert TickApiConn.calls[1][1] == 50, "末页只补足剩余 50 条"
|
|
||||||
|
|
||||||
@@ -1,234 +0,0 @@
|
|||||||
"""K线批量流水线路径(多连接 × 单连接流水线)。
|
|
||||||
|
|
||||||
背景:单只K线一请求一标的,全市场 5,000 只就是 5,000 次往返。串行时每只固定付
|
|
||||||
一个 RTT(实测 33ms),实测 24 台 × 8 流水能把全市场 1 分钟K线取当天从 43.6s
|
|
||||||
压到 5s 级。这里用替身验证「路径选择」与「结果口径」,不依赖网络。
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import datetime
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
from pytdxdata import TdxData
|
|
||||||
from pytdxdata.api import _auto_pipeline, _bars_for_range, _elapsed_trading_minutes
|
|
||||||
from pytdxdata.models import KlinePeriod, SecurityBar
|
|
||||||
|
|
||||||
# 替身的时间锚点必须跟着**真实 now** 走:API 内部用 ``datetime.now()`` 判断「是不是今天」,
|
|
||||||
# 用固定日期会在 CI(UTC 时区,与本地差 8 小时)上漂到别的时段分支,断言时绿时红。
|
|
||||||
NEWEST = datetime.datetime.now().replace(microsecond=0) # noqa: DTZ001
|
|
||||||
|
|
||||||
|
|
||||||
def _bar(when: datetime.datetime, close: float = 10.0) -> SecurityBar:
|
|
||||||
return SecurityBar(
|
|
||||||
market=1, code="600000", open=10.0, high=10.1, low=9.9, close=close,
|
|
||||||
vol=1000, amount=10000, year=when.year, month=when.month, day=when.day,
|
|
||||||
hour=when.hour, minute=when.minute)
|
|
||||||
|
|
||||||
|
|
||||||
class _MacFakeConn:
|
|
||||||
"""MAC 通道语义替身:**offset 0 = 最新一根**,偏移越大越老。"""
|
|
||||||
|
|
||||||
seen: list[tuple[str, int, int, bool]] = [] # (命令名, start, count, 是否流水线)
|
|
||||||
fail_pipeline = False
|
|
||||||
|
|
||||||
def __init__(self, host="m1", port=7709, timeout=10.0):
|
|
||||||
self.host = host
|
|
||||||
self._alive = True
|
|
||||||
|
|
||||||
@property
|
|
||||||
def alive(self) -> bool:
|
|
||||||
return self._alive
|
|
||||||
|
|
||||||
async def connect(self) -> None:
|
|
||||||
pass
|
|
||||||
|
|
||||||
async def close(self) -> None:
|
|
||||||
self._alive = False
|
|
||||||
|
|
||||||
def _rows(self, cmd, *, record: bool = True):
|
|
||||||
if record:
|
|
||||||
type(self).seen.append(
|
|
||||||
(type(cmd).__name__, getattr(cmd, "start", 0),
|
|
||||||
getattr(cmd, "count", 0), False))
|
|
||||||
if "MacKlineCmd" not in type(cmd).__name__:
|
|
||||||
return []
|
|
||||||
out = []
|
|
||||||
for off in range(cmd.start, cmd.start + cmd.count):
|
|
||||||
if off > 20_000:
|
|
||||||
continue
|
|
||||||
when = NEWEST - datetime.timedelta(minutes=off)
|
|
||||||
out.append(_bar(when, 10.0 + (20_000 - off) * 0.001))
|
|
||||||
return out
|
|
||||||
|
|
||||||
async def execute(self, cmd):
|
|
||||||
return self._rows(cmd)
|
|
||||||
|
|
||||||
async def execute_pipeline(self, cmds, **kwargs):
|
|
||||||
if type(self).fail_pipeline:
|
|
||||||
raise ConnectionError("流水线炸了")
|
|
||||||
for c in cmds:
|
|
||||||
type(self).seen.append(
|
|
||||||
(type(c).__name__, getattr(c, "start", 0), getattr(c, "count", 0), True))
|
|
||||||
return [self._rows(c, record=False) for c in cmds]
|
|
||||||
|
|
||||||
|
|
||||||
def _syms(n: int) -> list[str]:
|
|
||||||
"""n 只沪市股票代码(互不相同,避免按 symbol 归并时互相覆盖)。"""
|
|
||||||
return [f"sh60{i:04d}" for i in range(n)]
|
|
||||||
|
|
||||||
|
|
||||||
async def _run(n: int, period=KlinePeriod.MIN_1, **kw):
|
|
||||||
_MacFakeConn.seen = []
|
|
||||||
_MacFakeConn.fail_pipeline = False
|
|
||||||
with pytest.MonkeyPatch.context() as mp:
|
|
||||||
mp.setattr("pytdxdata.api.get_mac_hosts", lambda: ["m1", "m2"])
|
|
||||||
mp.setattr("pytdxdata.pool.pool.TdxConnection",
|
|
||||||
lambda h, p=7709, t=10.0: _MacFakeConn(h))
|
|
||||||
td = TdxData(cache_dir=None, pool_min=1, pool_max=4)
|
|
||||||
await td.start()
|
|
||||||
try:
|
|
||||||
bars = await td.get_bars(_syms(n), period, **kw)
|
|
||||||
finally:
|
|
||||||
await td.close()
|
|
||||||
return bars
|
|
||||||
|
|
||||||
|
|
||||||
def _data_calls() -> list[tuple[str, int, int, bool]]:
|
|
||||||
"""只看真正的数据请求(排除建连探针:count=100 那条)。"""
|
|
||||||
return [c for c in _MacFakeConn.seen if c[2] != 100]
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_bulk_path_uses_pipeline_for_many_symbols():
|
|
||||||
bars = await _run(20, count=240)
|
|
||||||
|
|
||||||
data = _data_calls()
|
|
||||||
assert data, "应该有数据请求"
|
|
||||||
assert all(via_pipe for *_, via_pipe in data), "20 只必须走流水线"
|
|
||||||
assert len(data) == 20
|
|
||||||
assert len(bars) == 20 * 240
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_small_batch_falls_back_to_single():
|
|
||||||
"""低于阈值走逐只:不经过流水线,结果不变。"""
|
|
||||||
bars = await _run(5, count=240)
|
|
||||||
|
|
||||||
data = _data_calls()
|
|
||||||
assert data and not any(via_pipe for *_, via_pipe in data)
|
|
||||||
assert len(bars) == 5 * 240
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_pipeline_1_disables_bulk():
|
|
||||||
bars = await _run(20, count=240, pipeline=1)
|
|
||||||
|
|
||||||
data = _data_calls()
|
|
||||||
assert data and not any(via_pipe for *_, via_pipe in data)
|
|
||||||
assert len(bars) == 20 * 240
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_result_order_follows_input_order():
|
|
||||||
"""返回按入参标的顺序拼接(组内升序)。"""
|
|
||||||
bars = await _run(20, count=30)
|
|
||||||
|
|
||||||
seen: list[str] = []
|
|
||||||
for b in bars:
|
|
||||||
if not seen or seen[-1] != b.symbol:
|
|
||||||
seen.append(b.symbol)
|
|
||||||
assert seen == _syms(20)
|
|
||||||
# 组内按时间正序
|
|
||||||
head = [b.datetime for b in bars if b.symbol == _syms(20)[0]]
|
|
||||||
assert head == sorted(head)
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_long_count_is_paged_inside_pipeline():
|
|
||||||
"""count 超过单请求上限时按页切分,且页也走流水线(不是逐只串行翻页)。"""
|
|
||||||
bars = await _run(20, count=1600, pipeline=8)
|
|
||||||
|
|
||||||
data = _data_calls()
|
|
||||||
assert all(via_pipe for *_, via_pipe in data)
|
|
||||||
assert len(data) == 20 * 3 # 700 + 700 + 200
|
|
||||||
assert {c[2] for c in data} == {700, 200}
|
|
||||||
assert len(bars) == 20 * 1600
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_pipeline_failure_falls_back_to_single_per_symbol():
|
|
||||||
"""流水线失败 → 该批交回逐只重试,结果不丢。"""
|
|
||||||
_MacFakeConn.seen = []
|
|
||||||
_MacFakeConn.fail_pipeline = True
|
|
||||||
with pytest.MonkeyPatch.context() as mp:
|
|
||||||
mp.setattr("pytdxdata.api.get_mac_hosts", lambda: ["m1", "m2"])
|
|
||||||
mp.setattr("pytdxdata.pool.pool.TdxConnection",
|
|
||||||
lambda h, p=7709, t=10.0: _MacFakeConn(h))
|
|
||||||
td = TdxData(cache_dir=None, pool_min=1, pool_max=4)
|
|
||||||
await td.start()
|
|
||||||
try:
|
|
||||||
bars = await td.get_bars(_syms(20), KlinePeriod.MIN_1, count=240)
|
|
||||||
finally:
|
|
||||||
await td.close()
|
|
||||||
assert len(bars) == 20 * 240
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_dated_range_bulk_trims_to_start_date():
|
|
||||||
"""日期区间走批量时,区间左端仍然精确裁剪。"""
|
|
||||||
start = NEWEST - datetime.timedelta(minutes=30)
|
|
||||||
bars = await _run(20, start_date=start)
|
|
||||||
|
|
||||||
assert bars
|
|
||||||
assert all(b.datetime >= start for b in bars)
|
|
||||||
assert all(b.symbol in set(_syms(20)) for b in bars)
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_dated_future_range_returns_empty_without_requests():
|
|
||||||
future = NEWEST + datetime.timedelta(days=5)
|
|
||||||
bars = await _run(20, start_date=future)
|
|
||||||
|
|
||||||
assert bars == []
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------- 纯函数 ----------------
|
|
||||||
def test_elapsed_trading_minutes():
|
|
||||||
def at(h, m):
|
|
||||||
return datetime.datetime(2026, 10, 28, h, m) # noqa: DTZ001
|
|
||||||
|
|
||||||
assert _elapsed_trading_minutes(at(9, 0)) == 0
|
|
||||||
assert _elapsed_trading_minutes(at(9, 30)) == 0
|
|
||||||
assert _elapsed_trading_minutes(at(10, 0)) == 30
|
|
||||||
assert _elapsed_trading_minutes(at(11, 45)) == 120 # 午休
|
|
||||||
assert _elapsed_trading_minutes(at(13, 30)) == 150
|
|
||||||
assert _elapsed_trading_minutes(at(15, 0)) == 240
|
|
||||||
assert _elapsed_trading_minutes(at(20, 0)) == 240 # 收盘后
|
|
||||||
|
|
||||||
|
|
||||||
def test_bars_for_range_shrinks_intraday():
|
|
||||||
"""盘中只需已走完的根数,不是一整天 240 根。"""
|
|
||||||
morning = datetime.datetime(2026, 10, 28, 10, 0) # noqa: DTZ001
|
|
||||||
midnight = datetime.datetime(2026, 10, 28, 0, 0) # noqa: DTZ001
|
|
||||||
# 10:00 走了 30 分钟 → 30 + 余量,远小于 240
|
|
||||||
assert 30 <= _bars_for_range(KlinePeriod.MIN_1, midnight, morning) <= 40
|
|
||||||
# 收盘后回到一整天的量级
|
|
||||||
close = datetime.datetime(2026, 10, 28, 16, 0) # noqa: DTZ001
|
|
||||||
assert _bars_for_range(KlinePeriod.MIN_1, midnight, close) >= 245
|
|
||||||
|
|
||||||
|
|
||||||
def test_bars_for_range_past_days():
|
|
||||||
now = datetime.datetime(2026, 10, 28, 16, 0) # noqa: DTZ001
|
|
||||||
start = now - datetime.timedelta(days=5)
|
|
||||||
n = _bars_for_range(KlinePeriod.DAY, start, now)
|
|
||||||
assert 5 <= n <= 12 # 5 个自然日 × 1.6 余量
|
|
||||||
|
|
||||||
|
|
||||||
def test_auto_pipeline():
|
|
||||||
# 每根 32B:小 count 用深流水,大 count 用浅流水(在途字节数恒定)
|
|
||||||
assert _auto_pipeline(2, None) == 32
|
|
||||||
assert _auto_pipeline(240, None) == 8
|
|
||||||
assert _auto_pipeline(240, 4) == 4 # 显式指定优先
|
|
||||||
assert _auto_pipeline(2, 1) == 2 # 夹到最小值 2
|
|
||||||
@@ -1,125 +0,0 @@
|
|||||||
"""CLI 价格小数位 / 分时时间列的显示回归。
|
|
||||||
|
|
||||||
覆盖两个曾经出错的地方:
|
|
||||||
|
|
||||||
1. ``tdx minute`` 的时间列按序号推算(``i // 60, i % 60``),240 根分时被打印成
|
|
||||||
``00:00~03:59``,与真实交易时间无关。
|
|
||||||
2. 价格固定写死 ``%8.2f``,可转债(报价步长 0.001 元)会被截成 2 位
|
|
||||||
——``128.892`` 打成 ``128.89``。
|
|
||||||
|
|
||||||
注意 ``price_decimal_digits`` 返回的是**解码位数**(原始整数 / 10**dp),可转债
|
|
||||||
是 4 位,但第 4 位恒为 0,显示层只该给 3 位(见 ``_display_dp``)。
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from pytdxdata.cli import (
|
|
||||||
_display_dp,
|
|
||||||
_dp_of,
|
|
||||||
_price_fmt,
|
|
||||||
cmd_minute,
|
|
||||||
cmd_transactions,
|
|
||||||
parse_symbol,
|
|
||||||
)
|
|
||||||
from pytdxdata.models import MinuteBar, TransactionRecord
|
|
||||||
|
|
||||||
|
|
||||||
class FakeTD:
|
|
||||||
"""只实现 minute / transactions 需要的两个接口。"""
|
|
||||||
|
|
||||||
def __init__(self, minutes=None, ticks=None):
|
|
||||||
self._minutes = minutes or []
|
|
||||||
self._ticks = ticks or []
|
|
||||||
|
|
||||||
async def get_minutes(self, sym, date=None):
|
|
||||||
return self._minutes
|
|
||||||
|
|
||||||
async def get_ticks(self, sym, date=None, count=None):
|
|
||||||
return self._ticks
|
|
||||||
|
|
||||||
|
|
||||||
def _bar(price, hour, minute, vol=100.0, avg=0.0):
|
|
||||||
return MinuteBar(price=price, vol=vol, avg_price=avg or price,
|
|
||||||
hour=hour, minute=minute, fields={})
|
|
||||||
|
|
||||||
|
|
||||||
def _tick(price, vol, trade_count=1, available=True, time="14:59:54"):
|
|
||||||
return TransactionRecord(market=1, code="113708", time=time, price=price, vol=vol,
|
|
||||||
trade_count=trade_count, bs_flag=0, fields={},
|
|
||||||
trade_count_available=available)
|
|
||||||
|
|
||||||
|
|
||||||
# --------------------------------------------------------------------------- #
|
|
||||||
# 小数位判定
|
|
||||||
# --------------------------------------------------------------------------- #
|
|
||||||
|
|
||||||
def test_price_fmt_width_tracks_decimals():
|
|
||||||
assert _price_fmt(2) == "%8.2f" # 股票:保持原宽度,不破坏对齐
|
|
||||||
assert _price_fmt(3) == "%9.3f"
|
|
||||||
assert _price_fmt(4) == "%10.4f"
|
|
||||||
assert _price_fmt(None) == "%8.2f" # 缺省当 2 位
|
|
||||||
|
|
||||||
|
|
||||||
def test_display_dp_by_instrument():
|
|
||||||
assert _display_dp(1, "600000") == 2 # 沪股票
|
|
||||||
assert _display_dp(0, "000001") == 2 # 深股票
|
|
||||||
assert _display_dp(1, "000001") == 2 # 指数
|
|
||||||
assert _display_dp(1, "113708") == 3 # 沪可转债(解码 4 位,显示 3 位)
|
|
||||||
assert _display_dp(1, "110102") == 3
|
|
||||||
assert _display_dp(0, "123235") == 3 # 深可转债
|
|
||||||
assert _display_dp(0, "128136") == 3
|
|
||||||
assert _display_dp(1, "510310") == 3 # ETF
|
|
||||||
assert _display_dp(1, "019547") == 4 # 国债:4 位是真实报价精度
|
|
||||||
|
|
||||||
|
|
||||||
def test_dp_of_uses_cn_only():
|
|
||||||
assert _dp_of(parse_symbol("sh113708")) == 3
|
|
||||||
assert _dp_of(parse_symbol("sh600000")) == 2
|
|
||||||
assert _dp_of(parse_symbol("hk00700")) == 2 # 非 A 股固定 2 位
|
|
||||||
|
|
||||||
|
|
||||||
# --------------------------------------------------------------------------- #
|
|
||||||
# tdx minute
|
|
||||||
# --------------------------------------------------------------------------- #
|
|
||||||
|
|
||||||
async def test_cmd_minute_prints_real_clock(capsys):
|
|
||||||
bars = [_bar(129.271, 9, 30), _bar(129.270, 9, 31), _bar(128.892, 14, 59)]
|
|
||||||
await cmd_minute(FakeTD(minutes=bars), "sh113708", None)
|
|
||||||
out = capsys.readouterr().out
|
|
||||||
assert "09:30" in out and "09:31" in out and "14:59" in out
|
|
||||||
assert "00:00" not in out and "03:59" not in out
|
|
||||||
|
|
||||||
|
|
||||||
async def test_cmd_minute_cb_keeps_three_decimals(capsys):
|
|
||||||
await cmd_minute(FakeTD(minutes=[_bar(128.892, 14, 59)]), "sh113708", None)
|
|
||||||
out = capsys.readouterr().out
|
|
||||||
assert "128.892" in out
|
|
||||||
assert "128.89 " not in out
|
|
||||||
|
|
||||||
|
|
||||||
async def test_cmd_minute_stock_two_decimals(capsys):
|
|
||||||
await cmd_minute(FakeTD(minutes=[_bar(9.62, 15, 0)]), "sh600000", None)
|
|
||||||
out = capsys.readouterr().out
|
|
||||||
assert "9.62" in out and "9.620" not in out
|
|
||||||
|
|
||||||
|
|
||||||
# --------------------------------------------------------------------------- #
|
|
||||||
# tdx transactions
|
|
||||||
# --------------------------------------------------------------------------- #
|
|
||||||
|
|
||||||
async def test_cmd_transactions_cb_price_and_count(capsys):
|
|
||||||
await cmd_transactions(FakeTD(ticks=[_tick(128.892, 3, trade_count=4)]),
|
|
||||||
"sh113708", 50, None)
|
|
||||||
out = capsys.readouterr().out
|
|
||||||
assert "128.892" in out
|
|
||||||
assert "(4笔)" in out
|
|
||||||
|
|
||||||
|
|
||||||
async def test_cmd_transactions_etf_marks_missing_count(capsys):
|
|
||||||
tick = TransactionRecord(market=1, code="510310", time="15:27:35", price=3.989,
|
|
||||||
vol=15, trade_count=0, bs_flag=0, fields={},
|
|
||||||
trade_count_available=False)
|
|
||||||
await cmd_transactions(FakeTD(ticks=[tick]), "sh510310", 50, None)
|
|
||||||
out = capsys.readouterr().out
|
|
||||||
assert "3.989" in out # ETF 也是 3 位
|
|
||||||
assert "(-)" in out # 不打印假笔数
|
|
||||||
assert "笔)" not in out
|
|
||||||
@@ -1,233 +0,0 @@
|
|||||||
"""权威价格小数位表(decimal_sync)单元测试。
|
|
||||||
|
|
||||||
覆盖点:
|
|
||||||
- 权威表(来自 ``SecurityInfo.decimal_point``)**优先于**静态前缀规则
|
|
||||||
- 某前缀出现混合小数位 → 自动退化为逐代码查表,不误伤同前缀其它代码
|
|
||||||
- 清空权威表 → 回到静态规则(与 0.6.5 行为一致)
|
|
||||||
- 磁盘缓存:写入 / 读取 / 跨日失效 / 损坏容错,均不回退到网络
|
|
||||||
- ``PYPYTDX_DATA_DISABLE_DECIMAL_SYNC=1`` → 完全不动作
|
|
||||||
- 证券清单里 ``decimal_point == 0`` 的脏条目被忽略
|
|
||||||
- 后台 task 可取消,且 ``close()`` 后不再改写表
|
|
||||||
(全部注入假 ``TdxData``,不连真实网络)
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import asyncio
|
|
||||||
import json
|
|
||||||
from types import SimpleNamespace
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
from pytdxdata import decimal_sync
|
|
||||||
from pytdxdata.protocol.commands.quotes import (
|
|
||||||
clear_decimal_map,
|
|
||||||
decimal_map_stats,
|
|
||||||
install_decimal_map,
|
|
||||||
price_decimal_digits,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------- #
|
|
||||||
# 夹具
|
|
||||||
# ---------------------------------------------------------------------- #
|
|
||||||
|
|
||||||
@pytest.fixture(autouse=True)
|
|
||||||
def _isolate(tmp_path, monkeypatch):
|
|
||||||
"""每个用例独立缓存目录 + 干净的内存状态。"""
|
|
||||||
monkeypatch.setenv("PYPYTDX_DATA_CACHE_DIR", str(tmp_path))
|
|
||||||
monkeypatch.delenv("PYPYTDX_DATA_DISABLE_DECIMAL_SYNC", raising=False)
|
|
||||||
decimal_sync.reset()
|
|
||||||
yield
|
|
||||||
decimal_sync.reset()
|
|
||||||
|
|
||||||
|
|
||||||
def _info(code: str, dp: int) -> SimpleNamespace:
|
|
||||||
return SimpleNamespace(code=code, decimal_point=dp, market=1)
|
|
||||||
|
|
||||||
|
|
||||||
class FakeTd:
|
|
||||||
"""只有 ``get_universe`` 的假客户端(记录调用次数,便于断言"没走网络")。"""
|
|
||||||
|
|
||||||
def __init__(self, tables: dict[str, list[tuple[str, int]]] | None = None) -> None:
|
|
||||||
self.tables = tables or {}
|
|
||||||
self.calls: list[str] = []
|
|
||||||
|
|
||||||
async def get_universe(self, market: str | None = None):
|
|
||||||
self.calls.append(market or "all")
|
|
||||||
return [_info(c, d) for c, d in self.tables.get(market, [])]
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------- #
|
|
||||||
# 权威表优先级
|
|
||||||
# ---------------------------------------------------------------------- #
|
|
||||||
|
|
||||||
def test_authoritative_map_overrides_static_rule():
|
|
||||||
# 静态规则说 sh019742 是 4 位;权威表说 3 位 → 必须听权威表的
|
|
||||||
assert price_decimal_digits(1, "019742") == 4
|
|
||||||
install_decimal_map([(1, "019742", 3)], source="test")
|
|
||||||
assert price_decimal_digits(1, "019742") == 3
|
|
||||||
|
|
||||||
|
|
||||||
def test_codes_outside_map_still_use_static_rule():
|
|
||||||
install_decimal_map([(1, "019742", 3)], source="test")
|
|
||||||
assert price_decimal_digits(1, "600000") == 2 # 股票
|
|
||||||
assert price_decimal_digits(1, "510300") == 3 # ETF
|
|
||||||
assert price_decimal_digits(0, "131810") == 4 # 深回购(表里没有)
|
|
||||||
|
|
||||||
|
|
||||||
def test_mixed_prefix_falls_back_to_per_code_lookup():
|
|
||||||
"""同一前缀出现两种小数位 → 该前缀不进前缀表,只把例外代码存进逐代码表。"""
|
|
||||||
stats = install_decimal_map(
|
|
||||||
[(1, "110001", 4), (1, "110002", 4), (1, "110003", 3)], source="test")
|
|
||||||
assert stats["mixed_prefixes"] == 1
|
|
||||||
assert stats["codes"] == 3 # 混合前缀下所有代码都进逐代码表
|
|
||||||
assert price_decimal_digits(1, "110001") == 4
|
|
||||||
assert price_decimal_digits(1, "110003") == 3
|
|
||||||
|
|
||||||
|
|
||||||
def test_clear_map_restores_static_rules():
|
|
||||||
install_decimal_map([(1, "019742", 3)], source="test")
|
|
||||||
assert price_decimal_digits(1, "019742") == 3
|
|
||||||
clear_decimal_map()
|
|
||||||
assert decimal_map_stats()["stock"] == 0
|
|
||||||
assert price_decimal_digits(1, "019742") == 4
|
|
||||||
|
|
||||||
|
|
||||||
def test_reinstall_replaces_previous_map():
|
|
||||||
install_decimal_map([(1, "019742", 3)], source="a")
|
|
||||||
install_decimal_map([(1, "019742", 4)], source="b")
|
|
||||||
assert price_decimal_digits(1, "019742") == 4
|
|
||||||
assert decimal_map_stats()["prefixes"] == 1
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------- #
|
|
||||||
# 同步
|
|
||||||
# ---------------------------------------------------------------------- #
|
|
||||||
|
|
||||||
def test_refresh_installs_and_writes_cache(tmp_path):
|
|
||||||
td = FakeTd({"sh": [("019742", 4), ("600000", 2)],
|
|
||||||
"sz": [("000001", 2), ("131810", 4)]})
|
|
||||||
stats = asyncio.run(decimal_sync.refresh(td))
|
|
||||||
|
|
||||||
assert td.calls == ["sh", "sz"]
|
|
||||||
assert stats["stock"] == 4
|
|
||||||
assert price_decimal_digits(1, "600000") == 2
|
|
||||||
assert price_decimal_digits(0, "131810") == 4
|
|
||||||
|
|
||||||
raw = json.loads((tmp_path / "decimal-points.json").read_text(encoding="utf-8"))
|
|
||||||
assert raw["markets"]["1"]["019742"] == 4
|
|
||||||
assert raw["markets"]["0"]["131810"] == 4
|
|
||||||
|
|
||||||
|
|
||||||
def test_refresh_skips_zero_decimal_point():
|
|
||||||
td = FakeTd({"sh": [("019742", 0), ("600000", 2)]})
|
|
||||||
stats = asyncio.run(decimal_sync.refresh(td, ("sh",)))
|
|
||||||
assert stats["stock"] == 1
|
|
||||||
|
|
||||||
|
|
||||||
def test_refresh_ignores_unknown_market():
|
|
||||||
td = FakeTd({"hk": [("00700", 3)]})
|
|
||||||
stats = asyncio.run(decimal_sync.refresh(td, ("hk",)))
|
|
||||||
assert stats["stock"] == 0
|
|
||||||
assert not (decimal_sync.cache_file()).exists()
|
|
||||||
|
|
||||||
|
|
||||||
def test_load_cached_roundtrip(tmp_path):
|
|
||||||
td = FakeTd({"sh": [("019742", 4)]})
|
|
||||||
asyncio.run(decimal_sync.refresh(td))
|
|
||||||
decimal_sync.reset()
|
|
||||||
assert decimal_map_stats()["stock"] == 0
|
|
||||||
|
|
||||||
assert decimal_sync.load_cached() == 1
|
|
||||||
assert price_decimal_digits(1, "019742") == 4
|
|
||||||
|
|
||||||
|
|
||||||
def test_cache_expires_across_days(tmp_path):
|
|
||||||
td = FakeTd({"sh": [("019742", 4)]})
|
|
||||||
asyncio.run(decimal_sync.refresh(td))
|
|
||||||
decimal_sync.reset()
|
|
||||||
|
|
||||||
path = tmp_path / "decimal-points.json"
|
|
||||||
payload = json.loads(path.read_text(encoding="utf-8"))
|
|
||||||
payload["date"] = "2000-01-01"
|
|
||||||
path.write_text(json.dumps(payload), encoding="utf-8")
|
|
||||||
|
|
||||||
assert decimal_sync.load_cached() == 0
|
|
||||||
|
|
||||||
|
|
||||||
def test_corrupt_cache_is_ignored(tmp_path):
|
|
||||||
path = tmp_path / "decimal-points.json"
|
|
||||||
path.write_text("{not json", encoding="utf-8")
|
|
||||||
assert decimal_sync.load_cached() == 0
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------- #
|
|
||||||
# 调度
|
|
||||||
# ---------------------------------------------------------------------- #
|
|
||||||
|
|
||||||
def test_schedule_disabled_by_env(monkeypatch):
|
|
||||||
monkeypatch.setenv("PYPYTDX_DATA_DISABLE_DECIMAL_SYNC", "1")
|
|
||||||
td = FakeTd({"sh": [("019742", 4)]})
|
|
||||||
|
|
||||||
async def run():
|
|
||||||
task = decimal_sync.schedule(td)
|
|
||||||
return task
|
|
||||||
|
|
||||||
assert asyncio.run(run()) is None
|
|
||||||
assert td.calls == []
|
|
||||||
|
|
||||||
|
|
||||||
def test_schedule_loads_cache_without_network(tmp_path):
|
|
||||||
td = FakeTd({"sh": [("019742", 4)]})
|
|
||||||
asyncio.run(decimal_sync.refresh(td))
|
|
||||||
decimal_sync.reset()
|
|
||||||
quiet = FakeTd({"sh": [("019742", 4)]})
|
|
||||||
|
|
||||||
async def run():
|
|
||||||
return decimal_sync.schedule(quiet)
|
|
||||||
|
|
||||||
assert asyncio.run(run()) is None # 缓存命中 → 不起 task
|
|
||||||
assert quiet.calls == [] # 也没走网络
|
|
||||||
assert price_decimal_digits(1, "019742") == 4
|
|
||||||
|
|
||||||
|
|
||||||
def test_schedule_runs_background_sync_without_cache():
|
|
||||||
td = FakeTd({"sh": [("019742", 4)]})
|
|
||||||
|
|
||||||
async def run():
|
|
||||||
task = decimal_sync.schedule(td)
|
|
||||||
assert task is not None
|
|
||||||
await task
|
|
||||||
return decimal_sync.status()
|
|
||||||
|
|
||||||
st = asyncio.run(run())
|
|
||||||
assert st["source"] == "universe"
|
|
||||||
assert st["syncing"] is False
|
|
||||||
assert td.calls == ["sh", "sz"] # 默认两市
|
|
||||||
|
|
||||||
|
|
||||||
def test_cancel_stops_background_task():
|
|
||||||
class SlowTd:
|
|
||||||
def __init__(self) -> None:
|
|
||||||
self.calls = 0
|
|
||||||
|
|
||||||
async def get_universe(self, market=None):
|
|
||||||
self.calls += 1
|
|
||||||
await asyncio.sleep(5)
|
|
||||||
return []
|
|
||||||
|
|
||||||
td = SlowTd()
|
|
||||||
|
|
||||||
async def run():
|
|
||||||
task = decimal_sync.schedule(td)
|
|
||||||
await asyncio.sleep(0.01)
|
|
||||||
await decimal_sync.cancel(task)
|
|
||||||
return task.done() or task.cancelled()
|
|
||||||
|
|
||||||
assert asyncio.run(run()) is True
|
|
||||||
|
|
||||||
|
|
||||||
def test_status_shape():
|
|
||||||
st = decimal_sync.status()
|
|
||||||
assert set(st) >= {"loaded", "source", "syncing", "prefixes", "codes",
|
|
||||||
"mixed_prefixes", "stock"}
|
|
||||||
@@ -1,175 +0,0 @@
|
|||||||
"""EX 0x23FF K线的字段语义:``amount`` / ``settle`` 按市场分别回填。
|
|
||||||
|
|
||||||
背景(2026-10-28 实测):0x23FF 每条 32B = 4B 时间戳 + **28B 行情**,这 28B 只有
|
|
||||||
**一个** 4B 槽位可复用,语义随市场变化 ——
|
|
||||||
|
|
||||||
* 权益类(港股主板/创业板/基金、新三板、美股):该槽位是**成交额**,
|
|
||||||
校验 ``成交额 / (close × vol)`` ≈ 100(港股主板/基金,vol 单位是手)或 ≈ 1(其余,vol 是股);
|
|
||||||
* 期货期权类:该槽位是**结算价**,成交额在帧内根本不存在(恒 0.0)。
|
|
||||||
|
|
||||||
改之前 ``amount`` 被硬编码成 0.0、末位 float 直接丢弃,于是港股/美股拿不到成交额、
|
|
||||||
期货期权拿不到结算价。
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import pickle
|
|
||||||
import struct
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
from pytdxdata.api import TdxData
|
|
||||||
from pytdxdata.models import SecurityBar
|
|
||||||
from pytdxdata.protocol.commands.ex_proto import (
|
|
||||||
_EX_AMOUNT_MARKETS,
|
|
||||||
GetExInstrumentBarsCmd,
|
|
||||||
)
|
|
||||||
from pytdxdata.symbols import parse_symbols
|
|
||||||
|
|
||||||
DAY = 20261028
|
|
||||||
|
|
||||||
|
|
||||||
def _f2u(value: float) -> int:
|
|
||||||
"""float32 -> 其位模式的 uint32(用来拼「同一槽位两种解释」的报文)。"""
|
|
||||||
return struct.unpack("<I", struct.pack("<f", value))[0]
|
|
||||||
|
|
||||||
|
|
||||||
def _ex_body(rows: list[tuple[int, ...]], category: int = 4) -> bytes:
|
|
||||||
"""拼 0x23FF 响应体:20B 头(count@18)+ N × 32B(时间4B + 7×4B)。"""
|
|
||||||
body = bytearray(20 + 32 * len(rows))
|
|
||||||
struct.pack_into("<H", body, 18, len(rows))
|
|
||||||
for i, words in enumerate(rows):
|
|
||||||
off = 20 + i * 32
|
|
||||||
struct.pack_into("<I", body, off, DAY)
|
|
||||||
struct.pack_into("<7I", body, off + 4, *words)
|
|
||||||
return bytes(body)
|
|
||||||
|
|
||||||
|
|
||||||
def _equity_words(close: float, amount: float, vol: int) -> tuple[int, ...]:
|
|
||||||
"""权益类布局:OHLC + 成交额(float) + 成交量(uint) + 保留(float)。"""
|
|
||||||
return (_f2u(close), _f2u(close), _f2u(close), _f2u(close),
|
|
||||||
_f2u(amount), vol, _f2u(0.0))
|
|
||||||
|
|
||||||
|
|
||||||
def _derivative_words(close: float, position: int, vol: int,
|
|
||||||
settle: float) -> tuple[int, ...]:
|
|
||||||
"""期货/期权布局:OHLC + 持仓量(uint) + 成交量(uint) + 结算价(float)。"""
|
|
||||||
return (_f2u(close), _f2u(close), _f2u(close), _f2u(close),
|
|
||||||
position, vol, _f2u(settle))
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- 解析层 ----------
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("market", [31, 48, 49, 44, 74])
|
|
||||||
def test_equity_markets_read_amount_from_fifth_slot(market: int) -> None:
|
|
||||||
cmd = GetExInstrumentBarsCmd(market, "00700", 4, 0, 1)
|
|
||||||
rows = cmd.parse_response(_ex_body([_equity_words(10.2, 2_040_000.0, 2000)]))
|
|
||||||
|
|
||||||
assert len(rows) == 1
|
|
||||||
assert rows[0]["amount"] == pytest.approx(2_040_000.0)
|
|
||||||
assert rows[0]["trade"] == 2000 # 成交量不受影响
|
|
||||||
assert rows[0]["position"] == 0 # 权益类无持仓量概念
|
|
||||||
assert rows[0]["settle"] is None
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("market", [7, 8, 9, 47])
|
|
||||||
def test_derivative_markets_read_settle_from_seventh_slot(market: int) -> None:
|
|
||||||
cmd = GetExInstrumentBarsCmd(market, "IFL0", 4, 0, 1)
|
|
||||||
rows = cmd.parse_response(_ex_body([_derivative_words(4021.0, 49_772, 25_392, 4047.8)]))
|
|
||||||
|
|
||||||
assert rows[0]["amount"] == 0.0 # 帧内没有成交额,不是解析漏读
|
|
||||||
assert rows[0]["position"] == 49_772
|
|
||||||
assert rows[0]["trade"] == 25_392
|
|
||||||
assert rows[0]["settle"] == pytest.approx(4047.8)
|
|
||||||
|
|
||||||
|
|
||||||
def test_intraday_settle_zero_normalized_to_none() -> None:
|
|
||||||
"""结算价是日终量:分钟线该槽位恒 0,归一化成 None 而不是 0.0。"""
|
|
||||||
cmd = GetExInstrumentBarsCmd(47, "IFL0", 7, 0, 1)
|
|
||||||
rows = cmd.parse_response(_ex_body([_derivative_words(4054.6, 49_490, 260, 0.0)], 7))
|
|
||||||
|
|
||||||
assert rows[0]["settle"] is None
|
|
||||||
|
|
||||||
|
|
||||||
def test_index_markets_kept_out_of_amount_table() -> None:
|
|
||||||
"""港股指数(27)/中证指数(62) 该槽位既非成交额也非结算价 —— 不能猜,保持 0.0。"""
|
|
||||||
assert 27 not in _EX_AMOUNT_MARKETS
|
|
||||||
assert 62 not in _EX_AMOUNT_MARKETS
|
|
||||||
cmd = GetExInstrumentBarsCmd(27, "HSI", 4, 0, 1)
|
|
||||||
rows = cmd.parse_response(_ex_body([_equity_words(23041.32, 178_348.0, 17_835)]))
|
|
||||||
assert rows[0]["amount"] == 0.0
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- API 层透传 ----------
|
|
||||||
|
|
||||||
def _fake_api(rows_for: dict[int, list[dict]]) -> TdxData:
|
|
||||||
td = TdxData(cache_dir=None, pool_min=1, pool_max=2)
|
|
||||||
seen: list[tuple[int, str]] = []
|
|
||||||
|
|
||||||
async def fake(make_cmd, spec): # noqa: ANN001 — 与真实签名同形
|
|
||||||
cmd = make_cmd(0, spec.page_size)
|
|
||||||
seen.append((cmd.market, cmd.code))
|
|
||||||
return rows_for[cmd.market]
|
|
||||||
|
|
||||||
td._fetch_paginated_ex = fake # type: ignore[method-assign]
|
|
||||||
td._seen_ex = seen # type: ignore[attr-defined]
|
|
||||||
return td
|
|
||||||
|
|
||||||
|
|
||||||
def _row(market: int, code: str, **kw) -> dict:
|
|
||||||
base = {"year": 2026, "month": 10, "day": 28, "hour": 15, "minute": 0,
|
|
||||||
"open": 10.0, "high": 10.5, "low": 9.5, "close": 10.2,
|
|
||||||
"amount": 0.0, "position": 0, "trade": 2000, "settle": None}
|
|
||||||
base.update(kw)
|
|
||||||
return base
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_bars_ex_backfills_amount_for_hk() -> None:
|
|
||||||
td = _fake_api({31: [_row(31, "00700", amount=2_040_000.0)]})
|
|
||||||
bars = await td.get_bars("hk00700", count=1)
|
|
||||||
|
|
||||||
assert len(bars) == 1
|
|
||||||
assert bars[0].amount == pytest.approx(2_040_000.0)
|
|
||||||
assert bars[0].vol == 2000
|
|
||||||
assert bars[0].settle is None
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_bars_ex_backfills_settle_for_futures() -> None:
|
|
||||||
td = _fake_api({47: [_row(47, "IFL0", position=49_772, trade=25_392, settle=4047.8)]})
|
|
||||||
bars = await td.get_bars("cffex_futures:IFL0", count=1)
|
|
||||||
|
|
||||||
assert bars[0].settle == pytest.approx(4047.8)
|
|
||||||
assert bars[0].amount == 0.0
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- 旧磁盘缓存兼容 ----------
|
|
||||||
|
|
||||||
def test_legacy_pickle_without_settle_still_loads() -> None:
|
|
||||||
"""升级前写下的缓存没有 settle 槽位;默认反序列化不会补默认值,读它必崩。"""
|
|
||||||
bar = SecurityBar(market=1, code="600000", open=1.0, high=2.0, low=0.5,
|
|
||||||
close=1.5, vol=100.0, amount=150.0,
|
|
||||||
year=2026, month=10, day=28, settle=12.3)
|
|
||||||
_, slots = bar.__getstate__()
|
|
||||||
legacy = dict(slots)
|
|
||||||
legacy.pop("settle")
|
|
||||||
|
|
||||||
revived = SecurityBar.__new__(SecurityBar)
|
|
||||||
revived.__setstate__((None, legacy))
|
|
||||||
|
|
||||||
assert revived.settle is None
|
|
||||||
assert revived.close == 1.5
|
|
||||||
|
|
||||||
|
|
||||||
def test_new_pickle_roundtrip_keeps_settle() -> None:
|
|
||||||
bar = SecurityBar(market=47, code="IFL0", open=4021.0, high=4060.0, low=4010.0,
|
|
||||||
close=4054.6, vol=25_618.0, amount=0.0,
|
|
||||||
year=2026, month=10, day=28, settle=4052.6)
|
|
||||||
|
|
||||||
assert pickle.loads(pickle.dumps(bar)).settle == pytest.approx(4052.6)
|
|
||||||
|
|
||||||
|
|
||||||
def test_ex_symbols_route_to_ex_path() -> None:
|
|
||||||
"""守住前提:上面对 31/47 的断言确实走的是 EX 通道。"""
|
|
||||||
assert parse_symbols(["hk00700"])[0].is_ex
|
|
||||||
assert parse_symbols(["cffex_futures:IFL0"])[0].is_ex
|
|
||||||
@@ -1,325 +0,0 @@
|
|||||||
"""DataFrame 转换层测试(0.7.0)。
|
|
||||||
|
|
||||||
覆盖两条独立路径:
|
|
||||||
|
|
||||||
1. **转换口径**:直接调 :func:`to_frame`,喂合成模型对象,断言列名/列序/取值/dtype。
|
|
||||||
不碰网络,双后端各跑一遍。
|
|
||||||
2. **接线**:``TdxData(df=...)`` 的取值校验、装饰器的「内层不二次转换」语义。
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import math
|
|
||||||
import sys
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
from pytdxdata import TdxData
|
|
||||||
from pytdxdata import frames as _frames
|
|
||||||
from pytdxdata.frames import FrameBackendError, framed, get_backend, to_frame
|
|
||||||
from pytdxdata.models import (
|
|
||||||
IndexBar,
|
|
||||||
IndexTradeStats,
|
|
||||||
IndicatorSet,
|
|
||||||
Market,
|
|
||||||
MinuteBar,
|
|
||||||
MinuteTradeStat,
|
|
||||||
SecurityBar,
|
|
||||||
SecurityQuote,
|
|
||||||
TdxBlock,
|
|
||||||
XdxrRecord,
|
|
||||||
)
|
|
||||||
|
|
||||||
BACKENDS = ("polars", "pandas")
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------- 工具
|
|
||||||
|
|
||||||
|
|
||||||
def bar(**kw) -> SecurityBar:
|
|
||||||
base = {
|
|
||||||
"market": int(Market.SH), "code": "600000", "open": 10.0, "high": 11.0,
|
|
||||||
"low": 9.5, "close": 10.5, "vol": 1000.0, "amount": 10500.0,
|
|
||||||
"year": 2026, "month": 10, "day": 27, "hour": 0, "minute": 0,
|
|
||||||
"symbol": "sh600000",
|
|
||||||
}
|
|
||||||
base.update(kw)
|
|
||||||
return SecurityBar(**base)
|
|
||||||
|
|
||||||
|
|
||||||
def rows(df, backend: str) -> int:
|
|
||||||
return len(df) if backend == "pandas" else df.height
|
|
||||||
|
|
||||||
|
|
||||||
def cols(df, backend: str) -> list[str]:
|
|
||||||
return list(df.columns)
|
|
||||||
|
|
||||||
|
|
||||||
def cell(df, backend: str, name: str, index: int = 0):
|
|
||||||
if backend == "pandas":
|
|
||||||
return df[name].iloc[index]
|
|
||||||
# polars 2.x:Series[0] 会返回 Series 而不是标量,统一走 to_list()
|
|
||||||
return df.get_column(name).to_list()[index]
|
|
||||||
|
|
||||||
|
|
||||||
def missing(value) -> bool:
|
|
||||||
"""pandas 的数值列会把 None 变成 NaN,两者都算「缺」。"""
|
|
||||||
return value is None or (isinstance(value, float) and math.isnan(value))
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------- 后端
|
|
||||||
|
|
||||||
|
|
||||||
def test_backend_instances_are_cached():
|
|
||||||
assert get_backend("polars") is get_backend("polars")
|
|
||||||
assert get_backend("pandas") is get_backend("pandas")
|
|
||||||
|
|
||||||
|
|
||||||
def test_unknown_backend_raises_value_error():
|
|
||||||
with pytest.raises(ValueError, match="未知的 DataFrame 后端"):
|
|
||||||
get_backend("numpy")
|
|
||||||
|
|
||||||
|
|
||||||
def test_missing_backend_raises_with_install_hint(monkeypatch):
|
|
||||||
monkeypatch.setitem(sys.modules, "polars", None) # 让 import polars 抛 ImportError
|
|
||||||
monkeypatch.delitem(_frames._BACKEND_CACHE, "polars", raising=False)
|
|
||||||
with pytest.raises(FrameBackendError) as excinfo:
|
|
||||||
get_backend("polars")
|
|
||||||
assert isinstance(excinfo.value, ImportError)
|
|
||||||
assert excinfo.value.name == "polars"
|
|
||||||
assert "pip install pytdxdata[polars]" in str(excinfo.value)
|
|
||||||
monkeypatch.delitem(_frames._BACKEND_CACHE, "polars", raising=False)
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------- 转换口径
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("backend", BACKENDS)
|
|
||||||
def test_bar_replaces_split_time_with_datetime(backend):
|
|
||||||
df = to_frame([bar(), bar(minute=30)], backend=backend)
|
|
||||||
assert cols(df, backend) == [
|
|
||||||
"market", "code", "open", "high", "low", "close", "vol", "amount",
|
|
||||||
"datetime", "trading_date", "symbol", "settle",
|
|
||||||
]
|
|
||||||
assert rows(df, backend) == 2
|
|
||||||
assert cell(df, backend, "market") == 1
|
|
||||||
assert cell(df, backend, "trading_date") == "2026-10-27"
|
|
||||||
stamp = cell(df, backend, "datetime", 1)
|
|
||||||
assert stamp.year == 2026 and stamp.hour == 0 and stamp.minute == 30
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("backend", BACKENDS)
|
|
||||||
def test_int_enum_column_is_plain_int(backend):
|
|
||||||
df = to_frame([bar(market=Market.SZ)], backend=backend)
|
|
||||||
value = cell(df, backend, "market")
|
|
||||||
assert value == 0
|
|
||||||
assert not isinstance(value, Market)
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("backend", BACKENDS)
|
|
||||||
def test_empty_list_falls_back_to_annotation_hint(backend):
|
|
||||||
df = to_frame([], backend=backend, model="SecurityBar")
|
|
||||||
assert rows(df, backend) == 0
|
|
||||||
assert cols(df, backend) == [
|
|
||||||
"market", "code", "open", "high", "low", "close", "vol", "amount",
|
|
||||||
"datetime", "trading_date", "symbol", "settle",
|
|
||||||
]
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("backend", BACKENDS)
|
|
||||||
def test_quote_expands_bid_ask_levels(backend):
|
|
||||||
quote = SecurityQuote(
|
|
||||||
market=1, code="600000", price=10.5, pre_close=10.0, open=10.1, high=10.6,
|
|
||||||
low=9.9, vol=100.0, amount=1000.0, s_vol=40.0, b_vol=60.0,
|
|
||||||
bid=[(10.49, 100.0), (10.48, 200.0)], ask=[(10.51, 300.0)],
|
|
||||||
symbol="sh600000", fields={"0x7": 1.5},
|
|
||||||
)
|
|
||||||
df = to_frame([quote], backend=backend)
|
|
||||||
out = cols(df, backend)
|
|
||||||
assert "bid1_price" in out and "ask5_vol" in out
|
|
||||||
assert "bid" not in out and "ask" not in out
|
|
||||||
assert cell(df, backend, "bid2_vol") == 200.0
|
|
||||||
assert missing(cell(df, backend, "bid3_price")) # 只有两档,缺档给 None
|
|
||||||
assert cell(df, backend, "fields") == '{"0x7":1.5}'
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("backend", BACKENDS)
|
|
||||||
def test_empty_dict_field_becomes_none(backend):
|
|
||||||
df = to_frame([SecurityQuote(
|
|
||||||
market=1, code="600000", price=1.0, pre_close=1.0, open=1.0, high=1.0,
|
|
||||||
low=1.0, vol=1.0, amount=1.0, s_vol=0.0, b_vol=0.0,
|
|
||||||
)], backend=backend)
|
|
||||||
assert missing(cell(df, backend, "fields"))
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("backend", BACKENDS)
|
|
||||||
def test_minute_uses_real_clock(backend):
|
|
||||||
late = MinuteBar(price=1.0, vol=10.0, avg_price=1.0, hour=27, minute=59,
|
|
||||||
symbol="usAAPL", market=74, code="AAPL", date=20261026)
|
|
||||||
df = to_frame([late], backend=backend)
|
|
||||||
assert cols(df, backend) == [
|
|
||||||
"price", "vol", "avg_price", "datetime", "trading_date",
|
|
||||||
"symbol", "market", "code", "fields",
|
|
||||||
]
|
|
||||||
stamp = cell(df, backend, "datetime")
|
|
||||||
assert (stamp.year, stamp.month, stamp.day, stamp.hour, stamp.minute) == (
|
|
||||||
2026, 10, 27, 3, 59,
|
|
||||||
)
|
|
||||||
assert cell(df, backend, "trading_date") == "2026-10-26"
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("backend", BACKENDS)
|
|
||||||
def test_xdxr_date_column(backend):
|
|
||||||
row = XdxrRecord(market=1, code="600000", year=2026, month=6, day=12,
|
|
||||||
category=1, name="分红", fenhong=0.5, symbol="sh600000")
|
|
||||||
df = to_frame([row], backend=backend)
|
|
||||||
assert cols(df, backend)[:3] == ["market", "code", "date"]
|
|
||||||
stamp = cell(df, backend, "date")
|
|
||||||
assert (stamp.year, stamp.month, stamp.day) == (2026, 6, 12)
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("backend", BACKENDS)
|
|
||||||
def test_tdx_block_keeps_codes_list(backend):
|
|
||||||
df = to_frame([TdxBlock(name="银行", category=0, count=2, codes=["600000", "600036"])],
|
|
||||||
backend=backend)
|
|
||||||
assert cols(df, backend) == ["name", "category", "count", "codes"]
|
|
||||||
assert cell(df, backend, "codes") == ["600000", "600036"]
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("backend", BACKENDS)
|
|
||||||
def test_indicator_set_merges_bars_and_indicators(backend):
|
|
||||||
item = IndicatorSet(
|
|
||||||
symbol="sh600000",
|
|
||||||
bars=[bar(), bar(minute=1), bar(minute=2)],
|
|
||||||
indicators={"ma5": [1.0, 2.0, 3.0], "boll": [None, None, 9.0]},
|
|
||||||
)
|
|
||||||
df = to_frame([item], backend=backend)
|
|
||||||
assert rows(df, backend) == 3
|
|
||||||
assert cell(df, backend, "symbol") == "sh600000"
|
|
||||||
assert cell(df, backend, "ma5", 2) == 3.0
|
|
||||||
assert missing(cell(df, backend, "boll"))
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("backend", BACKENDS)
|
|
||||||
def test_indicator_series_shorter_than_bars_is_padded(backend):
|
|
||||||
item = IndicatorSet(symbol="sh600000", bars=[bar(), bar(minute=1)],
|
|
||||||
indicators={"ma5": [7.0]})
|
|
||||||
df = to_frame([item], backend=backend)
|
|
||||||
assert rows(df, backend) == 2
|
|
||||||
assert cell(df, backend, "ma5") == 7.0
|
|
||||||
assert missing(cell(df, backend, "ma5", 1))
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("backend", BACKENDS)
|
|
||||||
def test_index_trade_stats_single_and_multi(backend):
|
|
||||||
stat = IndexTradeStats(
|
|
||||||
index="sh000300", date=20261027, member_count=300, active_members=299,
|
|
||||||
trade_count_members=299,
|
|
||||||
minutes=[MinuteTradeStat(time="09:31", volume=10.0, trade_count=5, amount=1000.0)],
|
|
||||||
)
|
|
||||||
single = to_frame(stat, backend=backend)
|
|
||||||
assert cols(single, backend) == ["index", "date", "time", "volume", "trade_count", "amount"]
|
|
||||||
assert cell(single, backend, "index") == "sh000300"
|
|
||||||
|
|
||||||
multi = to_frame([stat, stat], backend=backend)
|
|
||||||
assert rows(multi, backend) == 2
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("backend", BACKENDS)
|
|
||||||
def test_batch_dict_is_concatenated(backend):
|
|
||||||
data = {
|
|
||||||
"600000": [bar(code="600000"), bar(code="600000", minute=1)],
|
|
||||||
"600036": [bar(code="600036", symbol="sh600036")],
|
|
||||||
}
|
|
||||||
df = to_frame(data, backend=backend)
|
|
||||||
assert rows(df, backend) == 3
|
|
||||||
assert cols(df, backend)[:2] == ["market", "code"]
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("backend", BACKENDS)
|
|
||||||
def test_batch_dict_all_empty_returns_typed_empty(backend):
|
|
||||||
df = to_frame({"600000": [], "600036": []}, backend=backend, model="SecurityBar")
|
|
||||||
assert rows(df, backend) == 0
|
|
||||||
assert "datetime" in cols(df, backend)
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("backend", BACKENDS)
|
|
||||||
def test_list_of_dicts_uses_key_union(backend):
|
|
||||||
df = to_frame([{"name": "银行", "rise": 1.2}, {"name": "券商", "amount": 5.0}],
|
|
||||||
backend=backend)
|
|
||||||
assert cols(df, backend) == ["name", "rise", "amount"]
|
|
||||||
assert missing(cell(df, backend, "amount", 0))
|
|
||||||
|
|
||||||
|
|
||||||
def test_unsupported_payload_raises_type_error():
|
|
||||||
with pytest.raises(TypeError, match="无法转换为 DataFrame 的返回值类型"):
|
|
||||||
to_frame(42, backend="polars")
|
|
||||||
with pytest.raises(TypeError, match="无法转换为 DataFrame 的列表元素"):
|
|
||||||
to_frame([1.0, 2.0], backend="polars")
|
|
||||||
|
|
||||||
|
|
||||||
def test_subclass_reuses_parent_plan():
|
|
||||||
df = to_frame([IndexBar(market=1, code="000001", open=1.0, high=1.0, low=1.0,
|
|
||||||
close=1.0, vol=1.0, amount=1.0, year=2026, month=1, day=5)],
|
|
||||||
backend="polars")
|
|
||||||
assert "datetime" in df.columns
|
|
||||||
assert "year" not in df.columns
|
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------- 接线
|
|
||||||
|
|
||||||
|
|
||||||
def test_tdxdata_default_keeps_legacy_behaviour():
|
|
||||||
td = TdxData()
|
|
||||||
assert td._df_backend is None
|
|
||||||
|
|
||||||
|
|
||||||
def test_tdxdata_rejects_unknown_backend():
|
|
||||||
with pytest.raises(ValueError, match="未知的 DataFrame 后端"):
|
|
||||||
TdxData(df="numpy")
|
|
||||||
|
|
||||||
|
|
||||||
def test_tdxdata_accepts_supported_backend():
|
|
||||||
td = TdxData(df="polars")
|
|
||||||
assert td._df_backend == "polars"
|
|
||||||
|
|
||||||
|
|
||||||
class _Source:
|
|
||||||
"""最小替身:只验装饰器语义,不碰网络。"""
|
|
||||||
|
|
||||||
def __init__(self, backend: str | None) -> None:
|
|
||||||
self._df_backend = backend
|
|
||||||
|
|
||||||
@framed
|
|
||||||
async def single(self) -> list[SecurityBar]:
|
|
||||||
return [bar()]
|
|
||||||
|
|
||||||
@framed
|
|
||||||
async def nested(self) -> dict[str, list[SecurityBar]]:
|
|
||||||
got = await self.single() # 内层必须是原始 list
|
|
||||||
assert isinstance(got, list), f"内层被提前转换成了 {type(got)!r}"
|
|
||||||
return {"600000": got}
|
|
||||||
|
|
||||||
|
|
||||||
async def test_decorator_passthrough_when_no_backend():
|
|
||||||
src = _Source(None)
|
|
||||||
assert isinstance(await src.single(), list)
|
|
||||||
|
|
||||||
|
|
||||||
async def test_decorator_converts_outermost_only():
|
|
||||||
src = _Source("polars")
|
|
||||||
out = await src.nested()
|
|
||||||
assert cols(out, "polars")[0] == "market"
|
|
||||||
assert rows(out, "polars") == 1
|
|
||||||
|
|
||||||
|
|
||||||
def test_model_name_resolution():
|
|
||||||
# 注解是 dict[str, list[SecurityBar]] 时正则也能取到 SecurityBar;
|
|
||||||
# 但 "dict" 这类内建名在 models 里找不到,必须返回 None 而不是内建类型
|
|
||||||
assert _frames._model_by_name("dict") is None
|
|
||||||
assert _frames._model_by_name("SecurityBar") is SecurityBar
|
|
||||||
|
|
||||||
|
|
||||||
def test_datetime_dtype_is_datetime_like():
|
|
||||||
pl = pytest.importorskip("polars")
|
|
||||||
df = to_frame([bar()], backend="polars")
|
|
||||||
assert isinstance(df.schema["datetime"], pl.Datetime)
|
|
||||||
@@ -1,303 +0,0 @@
|
|||||||
"""指数成分成交汇总(get_index_members / get_index_trade_stats)单元测试。
|
|
||||||
|
|
||||||
全部离线:成分与逐笔都用实例属性打补丁,不建立任何真实连接。
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
from types import SimpleNamespace
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
from pytdxdata import TdxData
|
|
||||||
from pytdxdata.models import (
|
|
||||||
IndexTradeStats,
|
|
||||||
MemberQuote,
|
|
||||||
MinuteTradeStat,
|
|
||||||
TransactionRecord,
|
|
||||||
)
|
|
||||||
|
|
||||||
# ---------- _index_pair:裸代码按指数语义解析 ----------
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("text,expect", [
|
|
||||||
("000001", (1, "000001")), # 上证指数(不是平安银行!)
|
|
||||||
("sh000001", (1, "000001")),
|
|
||||||
("000300", (1, "000300")),
|
|
||||||
("sh000300", (1, "000300")),
|
|
||||||
("000905", (1, "000905")),
|
|
||||||
("999999", (1, "999999")),
|
|
||||||
("880305", (1, "880305")),
|
|
||||||
("399001", (0, "399001")), # 深证成指
|
|
||||||
("sz399006", (0, "399006")),
|
|
||||||
("899050", (2, "899050")), # 北证50
|
|
||||||
("bj899050", (2, "899050")),
|
|
||||||
("sh.000300", (1, "000300")), # 点号/冒号都容忍
|
|
||||||
("sz:399001", (0, "399001")),
|
|
||||||
])
|
|
||||||
def test_index_pair_parses(text, expect):
|
|
||||||
assert TdxData._index_pair(text) == expect
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("text", [
|
|
||||||
"", "600000", "sz000001x", "sh12345", "12345", "abc", "zzz000001",
|
|
||||||
])
|
|
||||||
def test_index_pair_rejects(text):
|
|
||||||
with pytest.raises(ValueError):
|
|
||||||
TdxData._index_pair(text)
|
|
||||||
|
|
||||||
|
|
||||||
def test_index_pair_rejects_non_index_cn_code():
|
|
||||||
"""000001 之外的普通股票段(如 600xxx)不是指数,裸写必须报错。"""
|
|
||||||
with pytest.raises(ValueError):
|
|
||||||
TdxData._index_pair("600000")
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- 模型:累计曲线 ----------
|
|
||||||
|
|
||||||
def test_index_trade_stats_cumulative():
|
|
||||||
st = IndexTradeStats(
|
|
||||||
index="sh000300", date=None, member_count=2, active_members=2,
|
|
||||||
trade_count_members=2,
|
|
||||||
minutes=[
|
|
||||||
MinuteTradeStat("09:30", 100.0, 10, 1000.0),
|
|
||||||
MinuteTradeStat("09:31", 200.0, 20, 2000.0),
|
|
||||||
MinuteTradeStat("09:32", 50.0, 5, 500.0),
|
|
||||||
],
|
|
||||||
total_volume=350.0, total_trade_count=35, total_amount=3500.0,
|
|
||||||
)
|
|
||||||
cum = st.cumulative()
|
|
||||||
assert [c.time for c in cum] == ["09:30", "09:31", "09:32"]
|
|
||||||
assert [c.volume for c in cum] == [100.0, 300.0, 350.0]
|
|
||||||
assert [c.trade_count for c in cum] == [10, 30, 35]
|
|
||||||
assert [c.amount for c in cum] == [1000.0, 3000.0, 3500.0]
|
|
||||||
# 累计必须是新对象,不污染原始增量
|
|
||||||
assert st.minutes[1].volume == 200.0
|
|
||||||
assert st.cum_trade_count() == [("09:30", 10), ("09:31", 30), ("09:32", 35)]
|
|
||||||
|
|
||||||
|
|
||||||
def test_index_trade_stats_trade_count_available():
|
|
||||||
ok = IndexTradeStats(index="sh000300", date=None, member_count=1,
|
|
||||||
active_members=1, trade_count_members=1)
|
|
||||||
assert ok.trade_count_available is True
|
|
||||||
bj = IndexTradeStats(index="bj899050", date=None, member_count=1,
|
|
||||||
active_members=1, trade_count_members=0)
|
|
||||||
assert bj.trade_count_available is False
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- get_index_members:分页终止 ----------
|
|
||||||
|
|
||||||
def _member(market: int, code: str, name: str = "") -> MemberQuote:
|
|
||||||
return MemberQuote(market=market, code=code, name=name,
|
|
||||||
fields={"0x0": 9.9, "0x5": 1000.0, "0x7": 99000.0})
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_get_index_members_pages_until_short_batch():
|
|
||||||
"""短页(不足 80 条)即视为最后一页,不再多发一次空请求。"""
|
|
||||||
calls: list[int] = []
|
|
||||||
|
|
||||||
async def fake_exec(cmd):
|
|
||||||
calls.append(cmd.start)
|
|
||||||
if cmd.start == 0:
|
|
||||||
return [_member(1, "600000"), _member(1, "600004")]
|
|
||||||
return []
|
|
||||||
|
|
||||||
td = TdxData(cache_dir=None)
|
|
||||||
td._execute_mac = fake_exec
|
|
||||||
members = await td.get_index_members("000016")
|
|
||||||
assert [m.symbol for m in members] == ["sh600000", "sh600004"]
|
|
||||||
assert calls == [0]
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_get_index_members_continues_on_full_page():
|
|
||||||
"""满页(80 条)后继续翻页,直到空页。"""
|
|
||||||
calls: list[int] = []
|
|
||||||
|
|
||||||
async def fake_exec(cmd):
|
|
||||||
calls.append(cmd.start)
|
|
||||||
if cmd.start == 0:
|
|
||||||
return [_member(1, "600000")] * cmd.page_size
|
|
||||||
return []
|
|
||||||
|
|
||||||
td = TdxData(cache_dir=None)
|
|
||||||
td._execute_mac = fake_exec
|
|
||||||
members = await td.get_index_members("000016")
|
|
||||||
assert len(members) == 80
|
|
||||||
assert calls == [0, 80]
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_get_index_members_respects_count_cap():
|
|
||||||
async def fake_exec(cmd):
|
|
||||||
# 永远返回满页,验证 count 能截断结果
|
|
||||||
return [_member(1, "600000")] * cmd.page_size
|
|
||||||
|
|
||||||
td = TdxData(cache_dir=None)
|
|
||||||
td._execute_mac = fake_exec
|
|
||||||
members = await td.get_index_members("000016", count=5)
|
|
||||||
assert len(members) == 5
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- get_index_trade_stats:按分钟聚合 ----------
|
|
||||||
|
|
||||||
def _tick(sym: str, hhmm: str, price: float, vol: float, tc: int) -> TransactionRecord:
|
|
||||||
market, code = (1, sym[2:]) if sym.startswith("sh") else (0, sym[2:])
|
|
||||||
return TransactionRecord(market=market, code=code, time=f"{hhmm}:00",
|
|
||||||
price=price, vol=vol, trade_count=tc, bs_flag=1,
|
|
||||||
symbol=sym)
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_get_index_trade_stats_aggregates_minutes():
|
|
||||||
async def fake_ticks(symbols, date=None, concurrency=6, **kw):
|
|
||||||
out = []
|
|
||||||
for s in symbols:
|
|
||||||
if s == "sh600000":
|
|
||||||
out += [_tick(s, "09:30", 10.0, 100.0, 10),
|
|
||||||
_tick(s, "09:30", 10.0, 50.0, 5),
|
|
||||||
_tick(s, "09:31", 11.0, 200.0, 20)]
|
|
||||||
else:
|
|
||||||
out += [_tick(s, "09:30", 5.0, 1000.0, 7)]
|
|
||||||
return out
|
|
||||||
|
|
||||||
td = TdxData(cache_dir=None)
|
|
||||||
td.get_ticks = fake_ticks
|
|
||||||
st = await td.get_index_trade_stats(
|
|
||||||
"000016", members=["sh600000", "sz000001"])
|
|
||||||
|
|
||||||
assert st.index == "sh000016"
|
|
||||||
assert st.member_count == 2
|
|
||||||
assert st.active_members == 2
|
|
||||||
assert st.trade_count_members == 2
|
|
||||||
assert st.trade_count_available is True
|
|
||||||
assert st.failed_members == 0
|
|
||||||
|
|
||||||
m = {x.time: x for x in st.minutes}
|
|
||||||
assert sorted(m) == ["09:30", "09:31"]
|
|
||||||
assert m["09:30"].volume == 1150.0 # 100 + 50 + 1000
|
|
||||||
assert m["09:30"].trade_count == 22 # 10 + 5 + 7
|
|
||||||
assert m["09:31"].volume == 200.0
|
|
||||||
# amount = Σ(price × 手 × 100)
|
|
||||||
assert m["09:30"].amount == pytest.approx(
|
|
||||||
10.0 * 100 * 100 + 10.0 * 50 * 100 + 5.0 * 1000 * 100)
|
|
||||||
assert st.total_volume == 1350.0
|
|
||||||
assert st.total_trade_count == 42
|
|
||||||
assert st.total_amount == pytest.approx(
|
|
||||||
10.0 * 100 * 100 + 10.0 * 50 * 100 + 5.0 * 1000 * 100 + 11.0 * 200 * 100)
|
|
||||||
assert st.cum_trade_count() == [("09:30", 22), ("09:31", 42)]
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_get_index_trade_stats_bj_has_no_trade_count():
|
|
||||||
"""北交所成分返回 trade_count=0 → trade_count_available False。"""
|
|
||||||
async def fake_ticks(symbols, date=None, concurrency=6, **kw):
|
|
||||||
return [_tick("sh600000", "09:30", 10.0, 100.0, 0)]
|
|
||||||
|
|
||||||
td = TdxData(cache_dir=None)
|
|
||||||
td.get_ticks = fake_ticks
|
|
||||||
st = await td.get_index_trade_stats("899050", members=["sh600000"])
|
|
||||||
assert st.trade_count_members == 0
|
|
||||||
assert st.trade_count_available is False
|
|
||||||
assert st.total_volume == 100.0
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_get_index_trade_stats_retries_per_symbol_on_batch_failure():
|
|
||||||
"""整批失败 → 退化为逐只重试,只有真正失败的计入 failed_members。"""
|
|
||||||
async def fake_ticks(symbols, date=None, concurrency=6, **kw):
|
|
||||||
single = isinstance(symbols, str)
|
|
||||||
if not single:
|
|
||||||
raise ConnectionError("批量炸了")
|
|
||||||
if symbols == "sh600004":
|
|
||||||
raise ConnectionError("这只也炸了")
|
|
||||||
return [_tick(symbols, "09:30", 10.0, 100.0, 3)]
|
|
||||||
|
|
||||||
td = TdxData(cache_dir=None)
|
|
||||||
td.get_ticks = fake_ticks
|
|
||||||
st = await td.get_index_trade_stats(
|
|
||||||
"000016", members=["sh600000", "sh600004", "sh600006"])
|
|
||||||
assert st.failed_members == 1
|
|
||||||
assert st.active_members == 2
|
|
||||||
assert st.total_trade_count == 6
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_get_index_trade_stats_raises_when_not_skip_errors():
|
|
||||||
async def fake_ticks(symbols, date=None, concurrency=6, **kw):
|
|
||||||
raise ConnectionError("boom")
|
|
||||||
|
|
||||||
td = TdxData(cache_dir=None)
|
|
||||||
td.get_ticks = fake_ticks
|
|
||||||
with pytest.raises(ConnectionError):
|
|
||||||
await td.get_index_trade_stats(
|
|
||||||
"000016", members=["sh600000"], skip_errors=False)
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_get_index_trade_stats_progress_callback():
|
|
||||||
seen: list[tuple[int, int]] = []
|
|
||||||
|
|
||||||
async def fake_ticks(symbols, date=None, concurrency=6, **kw):
|
|
||||||
return [_tick(s, "09:30", 10.0, 1.0, 1) for s in symbols]
|
|
||||||
|
|
||||||
td = TdxData(cache_dir=None)
|
|
||||||
td.get_ticks = fake_ticks
|
|
||||||
st = await td.get_index_trade_stats(
|
|
||||||
"000016", members=[f"sh60000{i}" for i in range(5)],
|
|
||||||
batch_size=2, on_progress=lambda d, t: seen.append((d, t)))
|
|
||||||
assert seen == [(2, 5), (4, 5), (5, 5)]
|
|
||||||
assert st.member_count == 5
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_get_index_trade_stats_fetches_members_when_absent():
|
|
||||||
asked: list[str] = []
|
|
||||||
|
|
||||||
async def fake_members(code):
|
|
||||||
asked.append(code)
|
|
||||||
return [SimpleNamespace(symbol="sh600000")]
|
|
||||||
|
|
||||||
async def fake_ticks(symbols, date=None, concurrency=6, **kw):
|
|
||||||
return [_tick(s, "09:30", 10.0, 1.0, 1) for s in symbols]
|
|
||||||
|
|
||||||
td = TdxData(cache_dir=None)
|
|
||||||
td.get_index_members = fake_members
|
|
||||||
td.get_ticks = fake_ticks
|
|
||||||
await td.get_index_trade_stats("sh000300")
|
|
||||||
assert asked == ["000300"]
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_get_index_trade_stats_rejects_empty_members():
|
|
||||||
td = TdxData(cache_dir=None)
|
|
||||||
|
|
||||||
async def fake_members(code):
|
|
||||||
return []
|
|
||||||
|
|
||||||
td.get_index_members = fake_members
|
|
||||||
with pytest.raises(ValueError, match="没有取到成分股"):
|
|
||||||
await td.get_index_trade_stats("000300")
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_get_index_trade_stats_multi_reuses_members():
|
|
||||||
member_calls: list[str] = []
|
|
||||||
tick_dates: list[int | None] = []
|
|
||||||
|
|
||||||
async def fake_members(code):
|
|
||||||
member_calls.append(code)
|
|
||||||
return [SimpleNamespace(symbol="sh600000")]
|
|
||||||
|
|
||||||
async def fake_ticks(symbols, date=None, concurrency=6, **kw):
|
|
||||||
tick_dates.append(date)
|
|
||||||
return [_tick(s, "09:30", 10.0, 1.0, 2) for s in symbols]
|
|
||||||
|
|
||||||
td = TdxData(cache_dir=None)
|
|
||||||
td.get_index_members = fake_members
|
|
||||||
td.get_ticks = fake_ticks
|
|
||||||
out = await td.get_index_trade_stats_multi(
|
|
||||||
"000300", [20261020, 20261021, None])
|
|
||||||
assert len(out) == 3
|
|
||||||
assert member_calls == ["000300"] # 成分只取一次
|
|
||||||
assert tick_dates == [20261020, 20261021, None]
|
|
||||||
assert all(s.total_trade_count == 2 for s in out)
|
|
||||||
@@ -1,166 +0,0 @@
|
|||||||
"""K线「按日期区间」取数的路径选择:倒推快路径 vs 二分慢路径。
|
|
||||||
|
|
||||||
背景:二分要先探总根数(翻倍 6 次)再跳 log2(N) 次,1 分钟 K 线约 36 次串行
|
|
||||||
往返 = 4 秒级;而「取今天」「增量到最新」这类贴着右端的请求,倒推只要 1~3 次。
|
|
||||||
"""
|
|
||||||
|
|
||||||
import datetime
|
|
||||||
from unittest import mock
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
from pytdxdata import TdxData
|
|
||||||
from pytdxdata.models import KlinePeriod, SecurityBar
|
|
||||||
|
|
||||||
NEWEST = datetime.datetime(2026, 10, 28, 15, 0) # noqa: DTZ001 — 全项目用裸北京时间
|
|
||||||
TOTAL_BARS = 5_000
|
|
||||||
|
|
||||||
|
|
||||||
def _bar(when: datetime.datetime, close: float) -> SecurityBar:
|
|
||||||
return SecurityBar(
|
|
||||||
market=1, code="600000", open=10.0, high=10.1, low=9.9, close=close,
|
|
||||||
vol=1000, amount=10000, year=when.year, month=when.month, day=when.day,
|
|
||||||
hour=when.hour, minute=when.minute)
|
|
||||||
|
|
||||||
|
|
||||||
class _FakeConn:
|
|
||||||
"""标准通道语义替身:**offset 0 = 最新一根**,偏移越大越老。"""
|
|
||||||
|
|
||||||
calls = 0
|
|
||||||
|
|
||||||
def __init__(self, host="s1", port=7709, timeout=10.0):
|
|
||||||
self.host = host
|
|
||||||
self.alive_flag = True
|
|
||||||
|
|
||||||
@property
|
|
||||||
def alive(self):
|
|
||||||
return self.alive_flag
|
|
||||||
|
|
||||||
async def connect(self):
|
|
||||||
pass
|
|
||||||
|
|
||||||
async def close(self):
|
|
||||||
self.alive_flag = False
|
|
||||||
|
|
||||||
async def execute_pipeline(self, cmds, **kwargs):
|
|
||||||
return [await self.execute(c) for c in cmds]
|
|
||||||
|
|
||||||
async def execute(self, cmd):
|
|
||||||
type(self).calls += 1
|
|
||||||
if "GetSecurityBarsCmd" not in type(cmd).__name__:
|
|
||||||
return []
|
|
||||||
rows = []
|
|
||||||
for off in range(cmd.start, min(cmd.start + cmd.count, TOTAL_BARS)):
|
|
||||||
rows.append(_bar(NEWEST - datetime.timedelta(days=off),
|
|
||||||
10.0 + (TOTAL_BARS - off) * 0.01))
|
|
||||||
return rows
|
|
||||||
|
|
||||||
|
|
||||||
class _AscendingConn(_FakeConn):
|
|
||||||
"""方向相反的替身(MAC/EX 语义):offset 0 = **最老**一根。"""
|
|
||||||
|
|
||||||
async def execute(self, cmd):
|
|
||||||
type(self).calls += 1
|
|
||||||
if "GetSecurityBarsCmd" not in type(cmd).__name__:
|
|
||||||
return []
|
|
||||||
rows = []
|
|
||||||
base = datetime.datetime(2020, 1, 1) # noqa: DTZ001
|
|
||||||
for off in range(cmd.start, min(cmd.start + cmd.count, TOTAL_BARS)):
|
|
||||||
rows.append(_bar(base + datetime.timedelta(days=off), 10.0 + off * 0.01))
|
|
||||||
return rows
|
|
||||||
|
|
||||||
|
|
||||||
def _patched(conn_cls):
|
|
||||||
return (
|
|
||||||
mock.patch("pytdxdata.api.get_standard_hosts", return_value=["s1"]),
|
|
||||||
mock.patch("pytdxdata.api.get_mac_hosts", return_value=["m1"]),
|
|
||||||
mock.patch("pytdxdata.pool.pool.TdxConnection",
|
|
||||||
side_effect=lambda h, p=7709, t=10.0: conn_cls(h)),
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
async def _fetch(conn_cls, **kwargs):
|
|
||||||
conn_cls.calls = 0
|
|
||||||
p1, p2, p3 = _patched(conn_cls)
|
|
||||||
with p1, p2, p3:
|
|
||||||
td = TdxData(cache_dir=None, pool_min=1, pool_max=2)
|
|
||||||
await td.start()
|
|
||||||
try:
|
|
||||||
bars = await td.get_bars("sh600000", KlinePeriod.DAY, **kwargs)
|
|
||||||
finally:
|
|
||||||
await td.close()
|
|
||||||
return bars, conn_cls.calls
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_recent_range_uses_backtrack():
|
|
||||||
"""右端贴最新 → 倒推,往返次数是个位数(不是二分那几十次)。"""
|
|
||||||
bars, calls = await _fetch(
|
|
||||||
_FakeConn, start_date=NEWEST - datetime.timedelta(days=5), end_date=NEWEST)
|
|
||||||
|
|
||||||
# 返回按时间升序 → 偏移由老到新(5,4,...,0)
|
|
||||||
assert [b.datetime.date() for b in bars] == [
|
|
||||||
(NEWEST - datetime.timedelta(days=i)).date() for i in range(5, -1, -1)]
|
|
||||||
assert calls <= 4, f"倒推路径应 1~3 次往返,实际 {calls}"
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_end_before_newest_within_one_bar_still_backtracks():
|
|
||||||
"""end 比最新一根略早(K线时间戳是周期结束时刻)仍算贴最新。"""
|
|
||||||
end = NEWEST - datetime.timedelta(minutes=30)
|
|
||||||
bars, calls = await _fetch(
|
|
||||||
_FakeConn, start_date=NEWEST - datetime.timedelta(days=3), end_date=end)
|
|
||||||
|
|
||||||
# offset 0 那根(15:00)晚于 end,被裁掉
|
|
||||||
assert bars[-1].datetime.date() == (NEWEST - datetime.timedelta(days=1)).date()
|
|
||||||
assert calls <= 4
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_middle_range_falls_back_to_bisect():
|
|
||||||
"""历史中段倒推够不着左边 → 回落二分,结果仍要正确。"""
|
|
||||||
bars, calls = await _fetch(
|
|
||||||
_FakeConn,
|
|
||||||
start_date=NEWEST - datetime.timedelta(days=100),
|
|
||||||
end_date=NEWEST - datetime.timedelta(days=90))
|
|
||||||
|
|
||||||
assert [b.datetime.date() for b in bars] == [
|
|
||||||
(NEWEST - datetime.timedelta(days=i)).date() for i in range(100, 89, -1)]
|
|
||||||
assert calls > 10, f"中段应走二分(几十次往返),实际只有 {calls}"
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_future_range_returns_empty_quickly():
|
|
||||||
# 用相对 now 的未来,别用固定日期:固定日期会随运行日漂进 _BACKTRACK_GRACE
|
|
||||||
# (默认 1 天容差覆盖长假),倒推就成立、会多取一轮,断言随日期时绿时红。
|
|
||||||
future = datetime.datetime.now() + datetime.timedelta(days=5) # noqa: DTZ001
|
|
||||||
bars, calls = await _fetch(_FakeConn, start_date=future)
|
|
||||||
|
|
||||||
assert bars == []
|
|
||||||
assert calls <= 3
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_ascending_channel_skips_backtrack():
|
|
||||||
"""offset 0 是最老(MAC/EX 语义)时倒推不成立,必须走二分且结果正确。"""
|
|
||||||
base = datetime.datetime(2020, 1, 1) # noqa: DTZ001
|
|
||||||
bars, calls = await _fetch(
|
|
||||||
_AscendingConn,
|
|
||||||
start_date=base + datetime.timedelta(days=99),
|
|
||||||
end_date=base + datetime.timedelta(days=109))
|
|
||||||
|
|
||||||
assert [b.datetime.date() for b in bars] == [
|
|
||||||
(base + datetime.timedelta(days=i)).date() for i in range(99, 110)]
|
|
||||||
assert calls > 10
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_backtrack_matches_plain_count_result():
|
|
||||||
"""倒推裁剪结果 == 直接取一大段再裁剪(两条路径口径必须一致)。"""
|
|
||||||
start = NEWEST - datetime.timedelta(days=7)
|
|
||||||
by_date, _ = await _fetch(_FakeConn, start_date=start, end_date=NEWEST)
|
|
||||||
whole, _ = await _fetch(_FakeConn, count=60)
|
|
||||||
|
|
||||||
expect = [b for b in whole if b.datetime >= start]
|
|
||||||
assert [(b.datetime, b.close) for b in by_date] == \
|
|
||||||
[(b.datetime, b.close) for b in expect]
|
|
||||||
@@ -1,236 +0,0 @@
|
|||||||
"""单连接流水线(execute_pipeline)单元测试:不连真实网络。
|
|
||||||
|
|
||||||
关键防线:流水线把多条请求塞进同一条 TCP 流,一旦响应与请求错配就是
|
|
||||||
「价格张冠李戴」级别的灾难,所以这里逐项锁死顺序、长度与失败语义。
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import struct
|
|
||||||
import unittest.mock as mock
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
from pytdxdata.pool.connection import TdxConnection
|
|
||||||
from pytdxdata.pool.pool import PooledConnection
|
|
||||||
from pytdxdata.protocol.commands.base import BaseCommand
|
|
||||||
|
|
||||||
|
|
||||||
class EchoCmd(BaseCommand[bytes]):
|
|
||||||
"""请求体即 tag,响应体原样带回,便于断言顺序。"""
|
|
||||||
|
|
||||||
def __init__(self, tag: bytes) -> None:
|
|
||||||
self.tag = tag
|
|
||||||
|
|
||||||
def build_request(self) -> bytes:
|
|
||||||
return b"REQ:" + self.tag
|
|
||||||
|
|
||||||
def parse_response(self, body: bytes) -> bytes:
|
|
||||||
return body
|
|
||||||
|
|
||||||
|
|
||||||
class FakeQuote:
|
|
||||||
"""最小报价替身:stamp() 会写 symbol,需要 market/code 是可 int/str 的真值。"""
|
|
||||||
|
|
||||||
def __init__(self) -> None:
|
|
||||||
self.market = 0
|
|
||||||
self.code = "000001"
|
|
||||||
self.symbol = ""
|
|
||||||
|
|
||||||
|
|
||||||
def frame(body: bytes) -> bytes:
|
|
||||||
"""构造一帧未压缩响应(zipsize == unzipsize)。"""
|
|
||||||
return struct.pack("<IIIHH", 7654321, 0, 0x053E, len(body), len(body)) + body
|
|
||||||
|
|
||||||
|
|
||||||
class FakeReader:
|
|
||||||
def __init__(self, data: bytes) -> None:
|
|
||||||
self.data = data
|
|
||||||
self.pos = 0
|
|
||||||
|
|
||||||
async def read(self, n: int) -> bytes:
|
|
||||||
chunk = self.data[self.pos:self.pos + n]
|
|
||||||
self.pos += len(chunk)
|
|
||||||
return chunk
|
|
||||||
|
|
||||||
|
|
||||||
class FakeWriter:
|
|
||||||
def __init__(self) -> None:
|
|
||||||
self.written = b""
|
|
||||||
self.closed = False
|
|
||||||
self.transport = FakeQuote()
|
|
||||||
|
|
||||||
def write(self, data: bytes) -> None:
|
|
||||||
self.written += data
|
|
||||||
|
|
||||||
async def drain(self) -> None:
|
|
||||||
pass
|
|
||||||
|
|
||||||
def is_closing(self) -> bool:
|
|
||||||
return self.closed
|
|
||||||
|
|
||||||
def close(self) -> None:
|
|
||||||
self.closed = True
|
|
||||||
|
|
||||||
async def wait_closed(self) -> None:
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
def make_conn(payload: bytes, host: str = "1.2.3.4") -> TdxConnection:
|
|
||||||
conn = TdxConnection(host, 7709, 5.0)
|
|
||||||
conn._reader = FakeReader(payload) # type: ignore[assignment]
|
|
||||||
conn._writer = FakeWriter() # type: ignore[assignment]
|
|
||||||
return conn
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- 基本语义 ----------
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_empty_cmds_returns_empty():
|
|
||||||
conn = make_conn(b"")
|
|
||||||
assert await conn.execute_pipeline([]) == []
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_responses_paired_in_order():
|
|
||||||
bodies = [b"AAA", b"BBB", b"CCC"]
|
|
||||||
conn = make_conn(b"".join(frame(b) for b in bodies))
|
|
||||||
cmds = [EchoCmd(t) for t in (b"1", b"2", b"3")]
|
|
||||||
got = await conn.execute_pipeline(cmds)
|
|
||||||
assert got == bodies # 顺序必须与 cmds 严格一致
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_all_requests_sent_at_once():
|
|
||||||
conn = make_conn(b"".join(frame(b"X") for _ in range(3)))
|
|
||||||
cmds = [EchoCmd(t) for t in (b"a", b"b", b"c")]
|
|
||||||
await conn.execute_pipeline(cmds)
|
|
||||||
assert conn._writer.written == b"REQ:aREQ:bREQ:c"
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_closed_connection_raises():
|
|
||||||
conn = make_conn(b"")
|
|
||||||
conn._writer = None
|
|
||||||
with pytest.raises(ConnectionError):
|
|
||||||
await conn.execute_pipeline([EchoCmd(b"1")])
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- 失败语义:响应流错配后必须废连接 ----------
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_truncated_stream_force_closes():
|
|
||||||
# 只给第一帧完整的、第二帧只有半个头 → 读不满,抛错且连接作废
|
|
||||||
conn = make_conn(frame(b"AAA") + b"\x00" * 5)
|
|
||||||
with pytest.raises(ConnectionError):
|
|
||||||
await conn.execute_pipeline([EchoCmd(b"1"), EchoCmd(b"2")])
|
|
||||||
assert conn._writer is None # _force_close 已清空 writer
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_body_length_mismatch_propagates():
|
|
||||||
bad = struct.pack("<IIIHH", 7654321, 0, 0x053E, 99, 99) + b"short"
|
|
||||||
conn = make_conn(bad)
|
|
||||||
# 帧头自称 99 字节但流里没有 → _recv_exact 读空,连接作废
|
|
||||||
with pytest.raises(ConnectionError):
|
|
||||||
await conn.execute_pipeline([EchoCmd(b"1")])
|
|
||||||
assert conn._writer is None
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- 池包装转发 ----------
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_pooled_connection_forwards_pipeline():
|
|
||||||
inner = mock.AsyncMock()
|
|
||||||
inner.execute_pipeline = mock.AsyncMock(return_value=["r1", "r2"])
|
|
||||||
pc = PooledConnection(inner, "host1")
|
|
||||||
got = await pc.execute_pipeline([EchoCmd(b"1"), EchoCmd(b"2")])
|
|
||||||
assert got == ["r1", "r2"]
|
|
||||||
inner.execute_pipeline.assert_awaited_once()
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- api 层:切批是否走流水线 ----------
|
|
||||||
|
|
||||||
def _api_with_fake_pool():
|
|
||||||
from pytdxdata.api import TdxData
|
|
||||||
from pytdxdata.symbols import CN, Symbol
|
|
||||||
|
|
||||||
td = TdxData.__new__(TdxData) # 跳过 __init__(不建网络池)
|
|
||||||
td._std_pool = FakeQuote()
|
|
||||||
td._std_pool.max_size = 24
|
|
||||||
td._std_pool.servers = [f"s{i}" for i in range(11)]
|
|
||||||
td._std_pool.acquire = mock.AsyncMock()
|
|
||||||
td._std_pool.release = mock.AsyncMock()
|
|
||||||
td._std_pool.warm = mock.AsyncMock() # 预建连接是网络动作,测试里空转
|
|
||||||
return td, Symbol, CN
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_quotes_std_uses_pipeline_for_multi_page():
|
|
||||||
td, Symbol, CN = _api_with_fake_pool()
|
|
||||||
syms = [Symbol(0, f"{i:06d}", CN) for i in range(1, 401)] # 5 片
|
|
||||||
chunks = []
|
|
||||||
|
|
||||||
async def fake_pipeline(batch_group):
|
|
||||||
chunks.append(len(batch_group))
|
|
||||||
return [[FakeQuote()] * 80 for _ in batch_group]
|
|
||||||
|
|
||||||
td._execute_std_pipeline = fake_pipeline
|
|
||||||
out = await td._quotes_std(syms, pipeline=True)
|
|
||||||
assert len(out) == 5 * 80
|
|
||||||
assert sum(chunks) == 5 # 每片恰好被处理一次,不多不少
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_quotes_std_single_page_skips_pipeline():
|
|
||||||
td, Symbol, CN = _api_with_fake_pool()
|
|
||||||
called = []
|
|
||||||
td._execute_std_pipeline = mock.AsyncMock(return_value=[])
|
|
||||||
|
|
||||||
async def fake_std(cmd):
|
|
||||||
called.append(1)
|
|
||||||
return []
|
|
||||||
|
|
||||||
td._execute_standard = fake_std
|
|
||||||
syms = [Symbol(0, f"{i:06d}", CN) for i in range(1, 51)] # 1 片
|
|
||||||
await td._quotes_std(syms, pipeline=True)
|
|
||||||
assert called == [1]
|
|
||||||
td._execute_std_pipeline.assert_not_awaited()
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_quotes_std_no_pipeline_flag_uses_plain_path():
|
|
||||||
td, Symbol, CN = _api_with_fake_pool()
|
|
||||||
|
|
||||||
async def fake_std(cmd):
|
|
||||||
return [FakeQuote()] * 80
|
|
||||||
|
|
||||||
td._execute_standard = fake_std
|
|
||||||
td._execute_std_pipeline = mock.AsyncMock(return_value=[])
|
|
||||||
syms = [Symbol(0, f"{i:06d}", CN) for i in range(1, 401)]
|
|
||||||
out = await td._quotes_std(syms, pipeline=False)
|
|
||||||
assert len(out) == 5 * 80
|
|
||||||
td._execute_std_pipeline.assert_not_awaited()
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_pipeline_failure_falls_back_to_per_page():
|
|
||||||
"""节点不支持流水线时,降级逐片,结果条数不变。"""
|
|
||||||
td, Symbol, CN = _api_with_fake_pool()
|
|
||||||
|
|
||||||
pool = td._std_pool
|
|
||||||
pool.acquire = mock.AsyncMock(return_value=mock.Mock(
|
|
||||||
execute_pipeline=mock.AsyncMock(side_effect=ConnectionError("boom"))))
|
|
||||||
pool.release = mock.AsyncMock()
|
|
||||||
|
|
||||||
calls = []
|
|
||||||
|
|
||||||
async def fake_std(cmd):
|
|
||||||
calls.append(1)
|
|
||||||
return [FakeQuote()] * 80
|
|
||||||
|
|
||||||
td._execute_standard = fake_std
|
|
||||||
syms = [Symbol(0, f"{i:06d}", CN) for i in range(1, 241)] # 3 片
|
|
||||||
out = await td._execute_std_pipeline(
|
|
||||||
[syms[i:i + 80] for i in range(0, len(syms), 80)])
|
|
||||||
assert len(out) == 3 and len(calls) == 3
|
|
||||||
assert pool.release.await_args.kwargs.get("broken") is True
|
|
||||||
@@ -12,7 +12,6 @@ from pytdxdata.protocol.commands.ex_proto import GetExInstrumentBarsCmd
|
|||||||
from pytdxdata.protocol.commands.kline import GetSecurityBarsCmd
|
from pytdxdata.protocol.commands.kline import GetSecurityBarsCmd
|
||||||
from pytdxdata.protocol.commands.mac import MacKlineCmd
|
from pytdxdata.protocol.commands.mac import MacKlineCmd
|
||||||
from pytdxdata.protocol.commands.minute import GetHistoryMinuteTimeDataCmd
|
from pytdxdata.protocol.commands.minute import GetHistoryMinuteTimeDataCmd
|
||||||
from pytdxdata.protocol.commands.quotes import price_decimal_digits
|
|
||||||
from pytdxdata.protocol.commands.transaction import MacTransactionCmd
|
from pytdxdata.protocol.commands.transaction import MacTransactionCmd
|
||||||
from pytdxdata.protocol.frame import FrameHeader, decompress_body, parse_header
|
from pytdxdata.protocol.frame import FrameHeader, decompress_body, parse_header
|
||||||
from pytdxdata.protocol.price import decode_ohlc_diffs, get_price, put_price
|
from pytdxdata.protocol.price import decode_ohlc_diffs, get_price, put_price
|
||||||
@@ -182,74 +181,6 @@ def test_mac_transaction_parse():
|
|||||||
assert recs[1].trade_count == 1
|
assert recs[1].trade_count == 1
|
||||||
|
|
||||||
|
|
||||||
def test_mac_transaction_parse_negative_trade_count_auction():
|
|
||||||
"""集合竞价段 trade_count 是有符号 int32,负数不能被读成 42 亿(BUG 回归)。
|
|
||||||
|
|
||||||
实测(2026-10-21,sh600000 09:15:03):18 字节为
|
|
||||||
17820000 0ad71b41 d9000000 96fcffff 0800
|
|
||||||
最后四字节前的 96fcffff = int32 -874,旧解析按 uint32 得 4294966422。
|
|
||||||
"""
|
|
||||||
body = bytearray(39 + 2 * 18)
|
|
||||||
struct.pack_into("<H", body, 29, 2)
|
|
||||||
# 第1条:集合竞价,trade_count = -874
|
|
||||||
struct.pack_into("<IfIiH", body, 39, 33303, 9.74, 217, -874, 8)
|
|
||||||
# 第2条:正常交易,trade_count = 3
|
|
||||||
struct.pack_into("<IfIiH", body, 39 + 18, 34200, 9.75, 100, 3, 1)
|
|
||||||
|
|
||||||
cmd = MacTransactionCmd(1, "600000", 20261021, 0, 100)
|
|
||||||
recs = cmd.parse_response(bytes(body))
|
|
||||||
assert len(recs) == 2
|
|
||||||
assert recs[0].trade_count == -874, "负值必须保留符号,不能变成 4294966422"
|
|
||||||
assert recs[0].bs_flag == 8
|
|
||||||
assert recs[1].trade_count == 3
|
|
||||||
# 正常交易段的量级不受影响
|
|
||||||
assert max(r.trade_count for r in recs) == 3
|
|
||||||
|
|
||||||
|
|
||||||
def test_mac_transaction_etf_trade_count_unavailable():
|
|
||||||
"""ETF/LOF 的 trade_count 是「价格×1000」占位,必须标记为不可用。
|
|
||||||
|
|
||||||
实测(2026-10-22 sh510310 15:27:35):价 3.989、原字段 3994;
|
|
||||||
旧行为把它当笔数打印成「3994笔」,是假的。
|
|
||||||
"""
|
|
||||||
body = bytearray(39 + 2 * 18)
|
|
||||||
struct.pack_into("<H", body, 29, 2)
|
|
||||||
struct.pack_into("<IfIiH", body, 39, 56000, 3.989, 15, 3994, 5)
|
|
||||||
struct.pack_into("<IfIiH", body, 39 + 18, 56003, 3.989, 4, 3994, 5)
|
|
||||||
|
|
||||||
for code, market, expect in (("510310", 1, False), ("159915", 0, False),
|
|
||||||
("600000", 1, True), ("501050", 1, True)):
|
|
||||||
cmd = MacTransactionCmd(market, code, 20261022, 0, 100)
|
|
||||||
recs = cmd.parse_response(bytes(body))
|
|
||||||
assert recs[0].trade_count_available is expect, code
|
|
||||||
if expect:
|
|
||||||
assert recs[0].trade_count == 3994
|
|
||||||
else:
|
|
||||||
assert recs[0].trade_count == 0
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("market,code,expect", [
|
|
||||||
# 股票 / 可转债 / 回购:有笔数
|
|
||||||
(1, "600000", True), (1, "688111", True), (1, "689009", True),
|
|
||||||
(0, "000001", True), (0, "300750", True), (0, "128136", True),
|
|
||||||
(1, "204001", True),
|
|
||||||
# 沪市 LOF 50x:实测有真实笔数,不能一刀切
|
|
||||||
(1, "501050", True), (1, "501018", True), (1, "502000", True), (1, "506000", True),
|
|
||||||
# 沪市 ETF/LOF 51/52/56/58:无笔数
|
|
||||||
(1, "510310", False), (1, "511990", False), (1, "512880", False),
|
|
||||||
(1, "513050", False), (1, "515000", False), (1, "518880", False),
|
|
||||||
(1, "560010", False), (1, "562500", False), (1, "588000", False),
|
|
||||||
# 深市基金:无笔数
|
|
||||||
(0, "159915", False), (0, "159001", False),
|
|
||||||
(0, "161725", False), (0, "160119", False), (0, "165520", False),
|
|
||||||
# 指数:无笔数
|
|
||||||
(1, "000001", False), (0, "399001", False), (1, "880305", False),
|
|
||||||
])
|
|
||||||
def test_has_order_count(market, code, expect):
|
|
||||||
from pytdxdata.symbols import has_order_count
|
|
||||||
assert has_order_count(market, code) is expect
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- 跨时区时间戳归一化(美股 EX 通道) ----------
|
# ---------- 跨时区时间戳归一化(美股 EX 通道) ----------
|
||||||
#
|
#
|
||||||
# 通达信 EX 通道对美股返回的时间戳把两个时区拼在一起:日期字段是**美东交易日**、
|
# 通达信 EX 通道对美股返回的时间戳把两个时区拼在一起:日期字段是**美东交易日**、
|
||||||
@@ -355,146 +286,3 @@ def test_finish_bars_sorts_us_session_chronologically():
|
|||||||
assert [b.datetime for b in out] == [datetime(2026, 10, 12, 21, 31),
|
assert [b.datetime for b in out] == [datetime(2026, 10, 12, 21, 31),
|
||||||
datetime(2026, 10, 13, 4, 0)]
|
datetime(2026, 10, 13, 4, 0)]
|
||||||
assert {b.symbol for b in out} == {"usAAPL"}
|
assert {b.symbol for b in out} == {"usAAPL"}
|
||||||
|
|
||||||
# ---------- 报价小数位:沪市指数必须 2 位(回归:曾整体少 10 倍) ----------
|
|
||||||
|
|
||||||
@pytest.mark.parametrize("market,code,expect", [
|
|
||||||
# 沪市指数:回归 bug —— 曾被判 3 位,导致价格少 10 倍
|
|
||||||
(1, "000001", 2), # 上证指数
|
|
||||||
(1, "000016", 2), # 上证50
|
|
||||||
(1, "000300", 2), # 沪深300
|
|
||||||
(1, "000688", 2), # 科创50
|
|
||||||
(1, "880001", 2), # 板块指数(8 开头,曾被判 3 位)
|
|
||||||
(1, "880305", 2),
|
|
||||||
# 沪市 ETF/基金:仍是 3 位
|
|
||||||
(1, "510300", 3),
|
|
||||||
(1, "588000", 3),
|
|
||||||
(1, "511010", 3),
|
|
||||||
(1, "501018", 3),
|
|
||||||
# 沪市股票:2 位
|
|
||||||
(1, "600000", 2),
|
|
||||||
(1, "601398", 2),
|
|
||||||
(1, "688111", 2), # 科创板
|
|
||||||
# 深市
|
|
||||||
(0, "399001", 2), # 深证成指
|
|
||||||
(0, "399006", 2), # 创业板指
|
|
||||||
(0, "000001", 2), # 平安银行(不是指数!)
|
|
||||||
(0, "300750", 2),
|
|
||||||
(0, "159915", 3), # 深市 ETF
|
|
||||||
# 债券/回购:2026-10-22 修正(沪市曾少 100 倍、深市曾少 10 倍)
|
|
||||||
(1, "019742", 4), # 沪市国债 24特国01(110.228,曾读成 11022.8)
|
|
||||||
(1, "019547", 4), # 沪市国债
|
|
||||||
(1, "010706", 4), # 沪市老国债 07国债06
|
|
||||||
(1, "018014", 4), # 沪市 018 段
|
|
||||||
(1, "020812", 4), # 沪市贴债
|
|
||||||
(1, "110077", 4), # 沪市可转债 洪城转债(199.279,曾读成 19927.9)
|
|
||||||
(1, "113616", 4), # 沪市可转债 韦尔转债
|
|
||||||
(1, "118031", 4), # 沪市可转债
|
|
||||||
(1, "111000", 4), # 沪市 111 段
|
|
||||||
(1, "132024", 4), # 沪市可交换债
|
|
||||||
(1, "184832", 4), # 沪市 184 段
|
|
||||||
(1, "204001", 4), # 沪市国债逆回购 GC001(1.385,曾读成 138.5)
|
|
||||||
(1, "204028", 4),
|
|
||||||
(0, "123107", 4), # 深市可转债 温氏转债(112.189,曾读成 1121.89)
|
|
||||||
(0, "128136", 4), # 深市可转债 立讯转债
|
|
||||||
(0, "127025", 4), # 深市可转债
|
|
||||||
(0, "131810", 4), # 深市国债逆回购 R-001(1.38,曾读成 13.8)
|
|
||||||
(0, "148004", 4), # 深市 148 段
|
|
||||||
(0, "112001", 4), # 深市公司债
|
|
||||||
(0, "101917", 4), # 深市国债
|
|
||||||
(0, "520261", 4), # 深市 520 段
|
|
||||||
# 沪市 B 股:3 位(2026-10-22 修正,曾按 2 位少 10 倍)
|
|
||||||
(1, "900901", 3), # 云赛B股
|
|
||||||
(1, "900948", 3),
|
|
||||||
# 深市 B 股:2 位(与沪市不同!)
|
|
||||||
(0, "200011", 2), # 深物业B
|
|
||||||
(0, "200488", 2),
|
|
||||||
# 边界:999xxx 是「指数」,不能被 9 开头当成 B 股
|
|
||||||
(1, "999999", 2), # 上证指数
|
|
||||||
(1, "999998", 2), # A股指数
|
|
||||||
])
|
|
||||||
def test_price_decimal_digits(market, code, expect):
|
|
||||||
assert price_decimal_digits(market, code) == expect
|
|
||||||
|
|
||||||
|
|
||||||
def test_price_decimal_digits_strips_nul():
|
|
||||||
assert price_decimal_digits(1, "000016\x00\x00") == 2
|
|
||||||
|
|
||||||
|
|
||||||
# ---------- 美股分时跨日:hour 必须是合法时钟值(回归:曾出现 hour=27) ----------
|
|
||||||
|
|
||||||
def _us_minutes(n: int = 390) -> list:
|
|
||||||
"""构造 usAAPL 的 390 条分时:从北京 21:30 起连续 n 分钟。"""
|
|
||||||
from pytdxdata.models import MinuteBar
|
|
||||||
|
|
||||||
start = 21 * 60 + 30
|
|
||||||
return [
|
|
||||||
MinuteBar(price=100.0 + i * 0.01, vol=1.0,
|
|
||||||
hour=(start + i) // 60, minute=(start + i) % 60,
|
|
||||||
market=74, code="AAPL", date=20261022)
|
|
||||||
for i in range(n)
|
|
||||||
]
|
|
||||||
|
|
||||||
|
|
||||||
def test_minute_bar_us_hour_is_valid_clock_value():
|
|
||||||
"""EX 分时的 minutes 是「自开盘起的连续分钟数」,直接 //60 会得到 hour=27。"""
|
|
||||||
bars = _us_minutes()
|
|
||||||
assert all(0 <= b.hour < 24 for b in bars)
|
|
||||||
# 归一化前的原始值确实越界(21:30 + 390min = 28:00)
|
|
||||||
assert max(b.hour + b.day_offset * 24 for b in bars) > 23
|
|
||||||
|
|
||||||
|
|
||||||
def test_minute_bar_us_day_offset_marks_next_day():
|
|
||||||
bars = _us_minutes()
|
|
||||||
assert bars[0].day_offset == 0 # 21:30 当晚
|
|
||||||
assert bars[-1].day_offset == 1 # 03:59 次日
|
|
||||||
assert (bars[-1].hour, bars[-1].minute) == (3, 59)
|
|
||||||
|
|
||||||
|
|
||||||
def test_minute_bar_us_datetime_is_monotonic_across_midnight():
|
|
||||||
bars = _us_minutes()
|
|
||||||
dts = [b.datetime for b in bars]
|
|
||||||
assert dts == sorted(dts)
|
|
||||||
assert dts[0] == datetime(2026, 10, 22, 21, 30)
|
|
||||||
assert dts[-1] == datetime(2026, 10, 23, 3, 59)
|
|
||||||
|
|
||||||
|
|
||||||
def test_minute_bar_us_trading_date_spans_two_calendar_days():
|
|
||||||
"""一个美股交易日的分时横跨两个北京自然日,trading_date 必须都是同一天。"""
|
|
||||||
bars = _us_minutes()
|
|
||||||
tds = {b.trading_date for b in bars}
|
|
||||||
assert tds == {"2026-10-22"}
|
|
||||||
# 而 datetime.date() 会给出两天 —— 这正是不能用它的原因
|
|
||||||
assert len({b.datetime.date() for b in bars}) == 2
|
|
||||||
|
|
||||||
|
|
||||||
def test_minute_bar_cn_unchanged():
|
|
||||||
"""A股/港股分时 hour 本就合法,归一化必须零行为变化。"""
|
|
||||||
from pytdxdata.models import MinuteBar
|
|
||||||
|
|
||||||
for hour, minute in ((9, 30), (11, 29), (13, 0), (14, 59)):
|
|
||||||
b = MinuteBar(price=10.0, vol=1.0, hour=hour, minute=minute,
|
|
||||||
market=1, code="600000", date=20261022)
|
|
||||||
assert (b.hour, b.minute) == (hour, minute)
|
|
||||||
assert b.day_offset == 0
|
|
||||||
assert b.datetime == datetime(2026, 10, 22, hour, minute)
|
|
||||||
assert b.trading_date == "2026-10-22"
|
|
||||||
|
|
||||||
|
|
||||||
def test_minute_bar_cn_datetime_monotonic_full_session():
|
|
||||||
"""A股全天 240 条(09:30~11:29 + 13:00~14:59)时间轴严格递增。"""
|
|
||||||
from pytdxdata.models import MinuteBar
|
|
||||||
|
|
||||||
bars = []
|
|
||||||
for idx in range(240):
|
|
||||||
if idx < 120:
|
|
||||||
hour, minute = 9 + (30 + idx) // 60, (30 + idx) % 60
|
|
||||||
else:
|
|
||||||
hour, minute = 13 + (idx - 120) // 60, (idx - 120) % 60
|
|
||||||
bars.append(MinuteBar(price=10.0, vol=1.0, hour=hour, minute=minute,
|
|
||||||
market=1, code="600000", date=20261022))
|
|
||||||
dts = [b.datetime for b in bars]
|
|
||||||
assert dts == sorted(dts)
|
|
||||||
assert dts[0] == datetime(2026, 10, 22, 9, 30)
|
|
||||||
assert dts[-1] == datetime(2026, 10, 22, 14, 59)
|
|
||||||
assert {b.trading_date for b in bars} == {"2026-10-22"}
|
|
||||||
@@ -1,91 +0,0 @@
|
|||||||
"""证券清单(0x053E)回归测试:请求长度 + 北交所降级。全部离线。
|
|
||||||
|
|
||||||
背景(2026-10-23 实测):`build_request` 曾经多塞一个 ``H``(18 字节),
|
|
||||||
服务端只消费前 16 字节,剩下的 ``00 00`` 留在它的接收缓冲里,把**下一个请求**
|
|
||||||
的开头吃掉 → 同一连接「第一次成功、第二次起全部通信超时」。
|
|
||||||
`get_universe` 的分页器复用连接,于是 66/28 页全靠失败重连,冷启动 12~18s。
|
|
||||||
"""
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import struct
|
|
||||||
import unittest.mock as mock
|
|
||||||
|
|
||||||
import pytest
|
|
||||||
|
|
||||||
from pytdxdata.models import Market
|
|
||||||
from pytdxdata.protocol.commands import GetSecurityListCmd
|
|
||||||
|
|
||||||
_HEADER = bytes.fromhex("0c0118640101060006005004")
|
|
||||||
|
|
||||||
|
|
||||||
def test_request_is_exactly_16_bytes():
|
|
||||||
"""12 字节命令头 + market(H) + start(H),一个字节都不能多。"""
|
|
||||||
req = GetSecurityListCmd(1, 0).build_request()
|
|
||||||
assert len(req) == 16
|
|
||||||
assert req == _HEADER + struct.pack("<HH", 1, 0)
|
|
||||||
|
|
||||||
|
|
||||||
def test_request_encodes_market_and_start_little_endian():
|
|
||||||
req = GetSecurityListCmd(0, 2000).build_request()
|
|
||||||
assert req[:12] == _HEADER
|
|
||||||
market, start = struct.unpack("<HH", req[12:])
|
|
||||||
assert (market, start) == (0, 2000)
|
|
||||||
|
|
||||||
|
|
||||||
def test_request_has_no_trailing_padding():
|
|
||||||
"""尾部不能有 padding:多余的 0 会让服务端解析错位。"""
|
|
||||||
for market in (0, 1, 2):
|
|
||||||
for start in (0, 1000, 27000):
|
|
||||||
req = GetSecurityListCmd(market, start).build_request()
|
|
||||||
assert len(req) == 16, f"market={market} start={start} -> {len(req)} 字节"
|
|
||||||
|
|
||||||
|
|
||||||
def test_parse_response_reads_all_records():
|
|
||||||
"""29 字节/条,头部 2 字节条数。"""
|
|
||||||
code, name = b"600000", "浦发银行".encode("gbk").ljust(8, b"\x00")
|
|
||||||
rec = struct.pack("<6sH8s4sBI4s", code, 100, name, b"\x00" * 4, 2, 0, b"\x00" * 4)
|
|
||||||
assert len(rec) == 29
|
|
||||||
body = struct.pack("<H", 2) + rec + rec
|
|
||||||
rows = GetSecurityListCmd(1, 0).parse_response(body)
|
|
||||||
assert len(rows) == 2
|
|
||||||
assert rows[0].code == "600000"
|
|
||||||
assert rows[0].decimal_point == 2
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_bj_universe_returns_empty_without_network():
|
|
||||||
"""北交所清单协议不支持:必须立刻返回空,且**不碰连接池**。"""
|
|
||||||
from pytdxdata.api import TdxData
|
|
||||||
|
|
||||||
td = TdxData()
|
|
||||||
td._std_pool.acquire = mock.AsyncMock(side_effect=AssertionError("不该联网"))
|
|
||||||
td._std_pool.acquire_many = mock.AsyncMock(
|
|
||||||
side_effect=AssertionError("不该联网"))
|
|
||||||
rows = await td.get_universe("bj")
|
|
||||||
assert rows == []
|
|
||||||
td._std_pool.acquire.assert_not_called()
|
|
||||||
td._std_pool.acquire_many.assert_not_called()
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_bj_market_is_in_unsupported_set():
|
|
||||||
from pytdxdata import api as api_mod
|
|
||||||
|
|
||||||
assert int(Market.BJ) in api_mod._LIST_UNSUPPORTED_MARKETS
|
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
|
||||||
async def test_ex_all_fetched_once_in_get_universe():
|
|
||||||
"""`get_universe()` 跨多个 EX 市场时,EX 全量只能取一次。"""
|
|
||||||
from pytdxdata.api import TdxData
|
|
||||||
|
|
||||||
td = TdxData()
|
|
||||||
calls = {"n": 0}
|
|
||||||
|
|
||||||
async def fake_ex_all():
|
|
||||||
calls["n"] += 1
|
|
||||||
return []
|
|
||||||
|
|
||||||
td._ex_all = fake_ex_all
|
|
||||||
await td.get_universe("all")
|
|
||||||
assert calls["n"] == 1, f"_ex_all 被调用 {calls['n']} 次(应为 1)"
|
|
||||||
Reference in new issue
Block a user