Compare commits
5
Commits
v0.6.5
...
2cf3657f4b
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2cf3657f4b | ||
|
|
dc78d81be8 | ||
|
|
8a78156145 | ||
|
|
2c07400a3a | ||
|
|
76644919de |
No files matched your search
+212
@@ -2,6 +2,218 @@
|
|||||||
|
|
||||||
本项目遵循 [语义化版本](https://semver.org/lang/zh-CN/)。
|
本项目遵循 [语义化版本](https://semver.org/lang/zh-CN/)。
|
||||||
|
|
||||||
|
## [0.6.9] - 2026-10-23
|
||||||
|
|
||||||
|
### 修复
|
||||||
|
|
||||||
|
- **`get_universe` 冷启动 12~18s / 偶发失败:证券清单请求多发了 2 字节,污染连接**。
|
||||||
|
|
||||||
|
`GetSecurityListCmd` 的请求体是「12 字节命令头 + market(H) + start(H)」= **16 字节**,
|
||||||
|
但实现里多塞了一个 `H`(写死 0),实际发出 **18 字节**。服务端只消费前 16 字节,
|
||||||
|
剩下的 `00 00` 停在它的接收缓冲里,把**下一个请求**的开头吃掉 —— 于是同一条连接
|
||||||
|
「第一次成功、第二次起永远无响应」。
|
||||||
|
|
||||||
|
分页器复用连接,等于每拉一页都要先失败重连,冷启动被拖到 12~18s,还经常直接抛
|
||||||
|
`ConnectionError`。
|
||||||
|
|
||||||
|
| 场景 | 修复前 | 修复后 |
|
||||||
|
|---|---|---|
|
||||||
|
| `get_universe("sh")` 冷启动(27,534 条 / 28 页) | 12,267ms | **1,610ms** |
|
||||||
|
| `get_universe("sz")` 冷启动(24,460 条 / 25 页) | 18,260ms | **571ms** |
|
||||||
|
| `get_universe()` 全市场冷启动(146,848 条) | 19,085ms | **4,371ms** |
|
||||||
|
| `get_universe()` 热缓存 | — | 669ms |
|
||||||
|
| 同一连接连发 2 次清单(27 台主机) | **0/27 成功** | **27/27 成功** |
|
||||||
|
|
||||||
|
- **北交所清单不再卡 15s 后报错,改为立即返回空列表**。
|
||||||
|
|
||||||
|
`0x053E` 对 `market=2` 在实测的 **27 台标准主机上全部无响应**(26 台 6s 无回包、
|
||||||
|
1 台立刻拒连),而同市场的 `0x053D` 数量接口正常(返回 389)。原实现会重试 2 次、
|
||||||
|
每次 3s 超时,最后让 `get_universe()`(不带参数)整批失败。
|
||||||
|
|
||||||
|
现在该市场直接返回空列表并 `WARNING` 一次,**北交所标的本身不受影响**:
|
||||||
|
`get_quotes("bj920992")` / `get_bars` / 逐笔全部正常(实测 11.79 与日线 close 一致)。
|
||||||
|
|
||||||
|
### 优化
|
||||||
|
|
||||||
|
- **`get_universe()` 的 EX 全量只取一次**:命中缓存也要反序列化 10 万条(~200ms),
|
||||||
|
原先十几个 EX 市场各来一次,纯属白烧。
|
||||||
|
- **EX 商品列表改并发翻页**(110 页,8 并发):全市场冷启动的一半时间花在这里。
|
||||||
|
|
||||||
|
### 文档
|
||||||
|
|
||||||
|
- `get_universe` docstring 写明北交所限制;`API_REFERENCE` / `llms.txt` / `SKILL.md` 同步。
|
||||||
|
- 新增 `tests/test_universe.py` 7 项:请求长度锁定 16 字节、无尾部 padding、
|
||||||
|
北交所返回空且**不碰连接池**、`_ex_all` 只取一次。
|
||||||
|
|
||||||
|
### 已知限制(非本次引入)
|
||||||
|
|
||||||
|
- EX 全量列表含**非证券条目**(如人员名录),`(market, code)` 在原始数据里就不唯一,
|
||||||
|
且重复项出现在**同一页内部**。属数据源特性,未做去重(去重会丢真实品种)。
|
||||||
|
|
||||||
|
## [0.6.8] - 2026-10-23
|
||||||
|
|
||||||
|
### 修复
|
||||||
|
|
||||||
|
- **CLI 价格按品种小数位显示;`tdx minute` 时间列改回真实时钟**。
|
||||||
|
|
||||||
|
两处纯显示层缺陷,数据本身没问题(分时/逐笔数值见 0.6.7 的核对结论):
|
||||||
|
|
||||||
|
| 现象 | 原因 |
|
||||||
|
|---|---|
|
||||||
|
| `tdx transactions sh113708` 打印 `128.89` | 价格写死 `%8.2f`;可转债报价步长 0.001 元,末位被截掉 |
|
||||||
|
| `tdx minute sh113708` 时间列 `00:00~03:59` | 按序号推算 `i // 60, i % 60`,与 bar 自带时间无关 |
|
||||||
|
| `tdx quotes sh113708` 打印 `128.8920` | `price_decimal_digits` 是**解码位数**(原始整数 / 10⁴),第 4 位恒为 0 |
|
||||||
|
|
||||||
|
| 位置 | 变化 |
|
||||||
|
|---|---|
|
||||||
|
| `_price_fmt(dp)`(新) | 宽度随小数位走(`%8.2f` → `%9.3f`),不破坏表格对齐 |
|
||||||
|
| `_display_dp(market, code)`(新) | 可转债(沪 `110/111/113/118`、深 `123/127/128`)由解码 4 位降到显示 3 位;其余沿用 `price_decimal_digits`(ETF 3 位、国债/回购 4 位、股票/指数 2 位) |
|
||||||
|
| `cmd_kline` / `cmd_transactions` / `cmd_minute` / `_print_quote` | 统一改用上面的格式串 |
|
||||||
|
| `cmd_minute` | 时间取 `MinuteBar.hour/minute`,不再用序号推算 |
|
||||||
|
| `cmd_transactions` / `cmd_minute` | 改调 `get_ticks` / `get_minutes`,消掉 `get_transactions` / `get_minute` 的 DeprecationWarning |
|
||||||
|
|
||||||
|
新增回归测试 8 项(`tests/test_cli_format.py`)。
|
||||||
|
|
||||||
|
文档:`docs/cli.md` 补「价格小数位按品种适配」对照表与 `tdx minute` 时间列说明。
|
||||||
|
|
||||||
|
## [0.6.7] - 2026-10-22
|
||||||
|
|
||||||
|
### 性能
|
||||||
|
|
||||||
|
- **全市场 A股报价 3.5s → 0.33s:单连接流水线 + 工作窃取**(5,227 只 / 66 片实测)。
|
||||||
|
|
||||||
|
老实现 `get_quotes` 是**纯串行 for 循环**逐片发请求:协议硬上限 80 只/请求,
|
||||||
|
全市场必须切 66 片,每片一次往返(≈60ms)→ 3.5s 全花在等网络上。
|
||||||
|
|
||||||
|
关键发现:通达信响应是**自描述长度帧**(16B 帧头 + body)且服务器**按序应答**,
|
||||||
|
因此同一 TCP 连接上可以连续下发多条请求、只付一次往返。实测单连接流水 66 个请求
|
||||||
|
只要 526ms(串行 7,372ms),响应顺序与代码完全对得上(用昨收价交叉比对 30 只,零错配)。
|
||||||
|
|
||||||
|
| 位置 | 变化 |
|
||||||
|
|---|---|
|
||||||
|
| `TdxConnection.execute_pipeline(cmds)`(新) | 一次 write 全部请求 → 按序读回全部响应。中途失败**强制断连**(响应流已错配,不可复用) |
|
||||||
|
| `PooledConnection.execute_pipeline`(新) | 池化包装转发 |
|
||||||
|
| `api._execute_std_pipeline`(新) | 借一条连接流水一整组分片;节点不支持流水线时自动降级为逐片 |
|
||||||
|
| `get_quotes(..., pipeline=True, concurrency=None)` | 新参数;小批量(≤80 只)仍走原路径,零额外开销 |
|
||||||
|
| `_quotes_std` | 改为**工作窃取**:分片放队列,连接做完一批回来再取,快的多干、慢的少干 |
|
||||||
|
| `DynamicPool.warm(n)`(新) | **并发**预建连接。`acquire()` 建连在全局锁内一次一条,22 并发首次会串行握手 22 次 —— 全市场首拉因此多花 5~6s;预热后降到 0.78s |
|
||||||
|
| `CacheManager.get_or_fetch_obj`(新) | 回源直接返回对象,省掉冷路径「pickle→立刻 unpickle」的一次白烧(5,227 条实测 ~40ms) |
|
||||||
|
| `TdxData(pool_max=)` | 默认 12 → 24,让报价并发能到「每台 2 连接」(池仍是懒扩容,不影响其他接口) |
|
||||||
|
|
||||||
|
调优实测(5,227 只 A股,11 台服务器,p50/max):
|
||||||
|
|
||||||
|
| 并发 | 每台连接 | 流水深度 | p50 | max |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| 11 | 1 | 6 | 455ms | 464ms |
|
||||||
|
| 12 | 1~2 | 3 | 457ms | 523ms |
|
||||||
|
| **22** | **2** | **3** | **325ms** | **334ms** |
|
||||||
|
| 33 | 3 | 2 | 328ms | 345ms |
|
||||||
|
|
||||||
|
⚠️ 剩余下限由公网决定:本机 → 服务器 TCP 握手 RTT 59~80ms,66 片最多只能压到
|
||||||
|
「1 次往返」;网络抖动时段实测会到 460~600ms。
|
||||||
|
|
||||||
|
端到端实测(全默认参数 `TdxData()`,11 台服务器,5,227 只):
|
||||||
|
|
||||||
|
| 场景 | 耗时 |
|
||||||
|
|---|---|
|
||||||
|
| 首次调用(含并发建 22 条连接) | 783ms |
|
||||||
|
| 稳态(第 2 次起) | **300~343ms** |
|
||||||
|
| 命中 TTL 缓存 | 67ms |
|
||||||
|
| 老版本对照(串行) | 5,996ms(首次)/ 3,460ms(稳态) |
|
||||||
|
|
||||||
|
### 修复
|
||||||
|
|
||||||
|
- **指数 / ETF / LOF 的 `trade_count` 不再伪装成成交笔数**。
|
||||||
|
|
||||||
|
通达信对**指数**和**多数场内基金**不提供笔数,但逐笔报文里那 4 字节仍塞了值:
|
||||||
|
指数是「当时均价 × 1000」,ETF/LOF 是「价格 × 1000」,货币 ETF(`sh511990`)恒为 `0`。
|
||||||
|
旧版直接把它当笔数,于是 `tdx transactions sh510310` 会打印出「3994笔」这种假数据
|
||||||
|
(实测 2026-10-22,价 3.989,原字段 3994)。
|
||||||
|
|
||||||
|
| 位置 | 变化 |
|
||||||
|
|---|---|
|
||||||
|
| `TransactionRecord.trade_count_available`(新字段) | 该品种协议上是否带笔数;默认 `True` |
|
||||||
|
| `symbols.has_order_count(market, code)`(新函数) | 按品种判定;**沪市 501x/502x/506x 实测有真实笔数**,不能按「5 开头=基金」一刀切 |
|
||||||
|
| `transaction.py` 三个解析器 | 不可用时 `trade_count=0` + `trade_count_available=False`;历史通道(0x013001,无笔数字段)同样标记 |
|
||||||
|
| `tdx transactions` CLI | 不可用时打印 `(-)`,不再打印假笔数 |
|
||||||
|
|
||||||
|
实测样本(2026-10-22,按前缀各取样):沪股票/科创/深股票/深可转债/沪逆回购 → 真实笔数;
|
||||||
|
沪 `51/52/56/58`、深 `159/15x/16x/17x/18x`、所有指数 → 不可用。
|
||||||
|
|
||||||
|
### 文档
|
||||||
|
|
||||||
|
- **可转债的量纲口径补全**(数据本身无缺陷,但对外文档一片空白,很容易被误判成 bug)。
|
||||||
|
|
||||||
|
起因:核查可转债的分时(`0x122D`)与逐笔(`0x122F`)是否有类似 ETF/LOF 的笔数问题。
|
||||||
|
结论是**价格 / 成交量 / 笔数三样全部正确**,顺带确认了 K 线标度也对;唯一的问题是
|
||||||
|
`vol` 的单位没人写清楚 —— 直接拿 `amount / vol` 算均价会**错 10 倍**。
|
||||||
|
|
||||||
|
实测(2026-10-22,16 只 A股可转债当日 + 8 只历史日 `2026-10-21`):
|
||||||
|
|
||||||
|
| 校验项 | 结果 |
|
||||||
|
|---|---|
|
||||||
|
| 分时末价 / 快照现价 | **1.000000** |
|
||||||
|
| Σ分时 vol / 日线 vol | **1.000000** |
|
||||||
|
| Σ逐笔 vol / 日线 vol | **1.000000** |
|
||||||
|
| 逐笔 VWAP vs `amount/(vol×10)` | 吻合(差 ≤0.002,是 amount 精度) |
|
||||||
|
| 快照 `pre_close` vs 上一交易日日线 `close` | **16/16 精确相等** |
|
||||||
|
| `trade_count` | 真实笔数(`sh111000` 3631 条切片 / 20443 笔) |
|
||||||
|
|
||||||
|
关键口径对照(这是文档缺失的部分):
|
||||||
|
|
||||||
|
| 品种 | 分时 / 逐笔 `vol` | 日线 `vol` | `Σ分时vol / 日线vol` | `amount / vol` |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| 股票 / ETF / LOF | 手(= 100 股) | 股 | **0.01** | 均价 ✓ |
|
||||||
|
| 可转债 | 手(= 10 张) | **手** | **1.0** | 均价 **×10** ⚠️ |
|
||||||
|
|
||||||
|
| 位置 | 变化 |
|
||||||
|
|---|---|
|
||||||
|
| `SecurityBar` / `TransactionRecord` / `MinuteBar` | 量纲注释补可转债(`vol`=手;日线 `amount/vol` 需再 `/10`) |
|
||||||
|
| `symbols.has_order_count` | 实测表补沪可转债 `110/111/113/118`、沪 EB `132`、深可转债 `123/127/128`,并说明它们走「排除法」兜底分支 |
|
||||||
|
| `docs/API_REFERENCE.md` | 新增**跨品种量纲对照表** |
|
||||||
|
| `docs/llms.txt` / `SKILL.md` | 同步同一口径 |
|
||||||
|
|
||||||
|
**代码逻辑未改**:可转债在 `has_order_count` 里本来就判为「有笔数」(不在基金/指数的
|
||||||
|
排除前缀里),本次只是把实测依据补进注释与文档。
|
||||||
|
|
||||||
|
## [0.6.6] - 2026-10-22
|
||||||
|
|
||||||
|
### 改进
|
||||||
|
|
||||||
|
- **价格小数位改用协议自带的权威值,前缀规则降为兜底**。
|
||||||
|
|
||||||
|
0.6.5 修好债券/回购标度时,采用的是「按代码前缀推断」+「用全量清单做离线校验」。
|
||||||
|
但同一份证券清单接口(`get_universe`,底层 `GetSecurityListCmd`)本来就**返回**
|
||||||
|
`SecurityInfo.decimal_point`(外加自描述浮点的 `pre_close`)——权威值一直就在手上,
|
||||||
|
只是没接进取数路径。本次把它接进来:
|
||||||
|
|
||||||
|
| 位置 | 说明 |
|
||||||
|
|---|---|
|
||||||
|
| `price_decimal_digits` | 查找顺序改为 **权威表 → 静态前缀规则**,未同步时行为与 0.6.5 完全一致 |
|
||||||
|
| `decimal_sync`(新模块) | 每日后台同步一次 + 磁盘缓存(`%LOCALAPPDATA%\pytdxdata\decimal-points.json`) |
|
||||||
|
| `TdxData.sync_decimal_points()` | 手动强制刷新(排障 / 立刻要权威表) |
|
||||||
|
| `TdxData.decimal_point_status()` | 状态:`source` / `stock` / `mixed_prefixes` / `syncing` |
|
||||||
|
|
||||||
|
**对用户仍无感**:`start()` 命中当日缓存就直接装表(实测 60 ms、零网络),
|
||||||
|
未命中则挂后台 task(实测首次 50 s,含沪 27,520 + 深 24,462 条清单),
|
||||||
|
不阻塞 `start()`(`start()` 仍 1.5 s);`close()` 会取消未完成的同步。
|
||||||
|
|
||||||
|
**表结构刻意做小**:全量 51,982 只标的只产生 **179 个**「三位前缀 → 小数位」映射
|
||||||
|
(沪 96 + 深 83),内存里就是 179 条。若将来某段代码混用两种小数位,该前缀自动
|
||||||
|
退化为**逐代码表**,只存例外代码(`mixed_prefixes` 会 > 0,可据此发现),
|
||||||
|
不会静默算错。清单拉取不完整时表会偏小 —— 缺的部分退回静态规则,不影响正确性。
|
||||||
|
|
||||||
|
为什么保留前缀规则:`get_quotes` 解析是纯同步、无网络的路径,解析那一刻可能还没
|
||||||
|
同步过清单;离线 / 首次安装也必须能出价格。实测两者在全量清单上**零分歧**
|
||||||
|
(51,982 条逐条比对,不一致 0 条),并存不会造成口径冲突。
|
||||||
|
|
||||||
|
开关:`PYPYTDX_DATA_DISABLE_DECIMAL_SYNC=1` 关闭后台同步(CI / 测试用)。
|
||||||
|
|
||||||
|
### 说明
|
||||||
|
|
||||||
|
- 新增回归测试 16 项(权威表优先级 / 混合前缀退化 / 缓存跨日失效 / 损坏容错 /
|
||||||
|
后台任务取消),测试总数 **289 → 306**。
|
||||||
|
|
||||||
## [0.6.5] - 2026-10-22
|
## [0.6.5] - 2026-10-22
|
||||||
|
|
||||||
### 新增
|
### 新增
|
||||||
|
|||||||
@@ -139,13 +139,14 @@ await td.get_bars("cffex:IFL0", KlinePeriod.DAY, count=700) # 期货合约
|
|||||||
|
|
||||||
| 需求 | 方法 |
|
| 需求 | 方法 |
|
||||||
|---|---|
|
|---|---|
|
||||||
| 全市场标的清单(A股+港股+期货…) | `get_universe(market=None)` |
|
| 全市场标的清单(A股+港股+期货…;**北交所返回空**) | `get_universe(market=None)` |
|
||||||
| 日/周/月/分钟 K 线(任意市场) | `get_bars(symbols, period, *, count, adjust, start_date, end_date)` |
|
| 日/周/月/分钟 K 线(任意市场) | `get_bars(symbols, period, *, count, adjust, start_date, end_date)` |
|
||||||
| K 线 + MACD/KDJ/RSI/BOLL | `get_indicators(symbols, indicators, ...)` |
|
| K 线 + MACD/KDJ/RSI/BOLL | `get_indicators(symbols, indicators, ...)` |
|
||||||
| 实时五档报价(A股)/ 基础报价(港股/美股/期货) | `get_quotes(symbols, *, fields=None)` |
|
| 实时五档报价(A股)/ 基础报价(港股/美股/期货) | `get_quotes(symbols, *, fields=None, pipeline=True)` |
|
||||||
| 逐笔成交(任意市场) | `get_ticks(symbols, *, date)` |
|
| 逐笔成交(任意市场) | `get_ticks(symbols, *, date)` |
|
||||||
| 分时图 / 缩略采样 | `get_minutes(symbols, *, date, sampling=True)` |
|
| 分时图 / 缩略采样 | `get_minutes(symbols, *, date, sampling=True)` |
|
||||||
| 全市场股票列表(缓存 1 天) | `get_universe("sz")` / `get_universe("sh")` |
|
| 全市场股票列表(缓存 1 天) | `get_universe("sz")` / `get_universe("sh")` |
|
||||||
|
| ~~北交所全量代码~~ | 拿不到:清单命令对 market=2 无响应(27 台全测)。北交所标的仍可直接查 `bj920992` |
|
||||||
| 除权除息 / 财务快照 | `get_xdxr(symbols)` / `get_finance(symbols)` |
|
| 除权除息 / 财务快照 | `get_xdxr(symbols)` / `get_finance(symbols)` |
|
||||||
| F10 公司资料 | `get_f10(symbol, section)` |
|
| F10 公司资料 | `get_f10(symbol, section)` |
|
||||||
| 板块列表 / 成分股 / 涨跌排行 | `get_boards` / `get_board_members` / `get_board_ranking` |
|
| 板块列表 / 成分股 / 涨跌排行 | `get_boards` / `get_board_members` / `get_board_ranking` |
|
||||||
@@ -169,8 +170,16 @@ await td.get_bars("cffex:IFL0", KlinePeriod.DAY, count=700) # 期货合约
|
|||||||
- **K 线时间戳统一为北京时间**(v0.6.2 起)。美股原始报文里日期是美东交易日、时分是北京时间(两个时区拼在一起),库已自动归一化,`bar.datetime` 可直接排序 / 比较 / 过滤。但**美股一个交易日横跨两个北京自然日**,按交易日聚合(分钟线合成日线、日内统计)必须用 `bar.trading_date`。
|
- **K 线时间戳统一为北京时间**(v0.6.2 起)。美股原始报文里日期是美东交易日、时分是北京时间(两个时区拼在一起),库已自动归一化,`bar.datetime` 可直接排序 / 比较 / 过滤。但**美股一个交易日横跨两个北京自然日**,按交易日聚合(分钟线合成日线、日内统计)必须用 `bar.trading_date`。
|
||||||
- **`count` 很大时会自动多连接并发分页**,无需手动翻页。但标准通道 K 线 `count < 100` 部分服务器会返回空——内部已强制 `min_page=100` 规避。
|
- **`count` 很大时会自动多连接并发分页**,无需手动翻页。但标准通道 K 线 `count < 100` 部分服务器会返回空——内部已强制 `min_page=100` 规避。
|
||||||
- **`get_quotes` 一次 >80 只自动切批**;走 MAC 字段位通道(`fields=`)时上限为 80 只/次。
|
- **`get_quotes` 一次 >80 只自动切批**;走 MAC 字段位通道(`fields=`)时上限为 80 只/次。
|
||||||
|
- **全市场报价用默认参数即可,别自己写循环**。`get_quotes(全部A股)` 内部会切 66 片并走**单连接流水线**(一条连接连发多个分片、只付一次往返)+ 工作窃取调度,5,227 只实测 **p50 325ms**(老版本串行 3.5s)。自己 for 循环逐只/逐片调会把流水线优势全部丢掉。
|
||||||
|
- 想更保守可 `concurrency=11`(每台 1 连接),代价 p50 457ms。
|
||||||
|
- 个别服务器不支持流水线时自动降级逐片,结果不受影响。
|
||||||
|
- **可转债(110/111/113/118/123/127/128)三样都是真的**:价格、`vol`(手=10 张)、`trade_count`(真实笔数,3631 条切片 / 20443 笔量级)全部可用,`Σvol` 与日线 `vol` **1:1 相等**。
|
||||||
|
⚠️ 唯一要小心的是 **日线 `amount / vol` 不等于均价,而是均价的 10 倍**(`vol` 单位是手=10 张)。算可转债均价用 `amount / vol / 10`,或直接用 `Σ(price×vol) / Σvol`——后者跨品种都成立,最稳。
|
||||||
|
对照:股票/ETF 的 `Σ逐笔vol / 日线vol = 0.01`(逐笔手、日线股),而 `amount/vol` 恰好就是均价。
|
||||||
- **指数本身没有成交笔数**。`get_ticks("sh000001")` 的 `trade_count` 是「当时均价×1000」、`vol` 单位是百元(成交额/100)。要笔数只能用 `get_index_trade_stats(index)`(成分股逐笔累加,2230 只约 36-60 s)。北交所成分无笔数字段,该方法会返回 `trade_count_available=False`。
|
- **指数本身没有成交笔数**。`get_ticks("sh000001")` 的 `trade_count` 是「当时均价×1000」、`vol` 单位是百元(成交额/100)。要笔数只能用 `get_index_trade_stats(index)`(成分股逐笔累加,2230 只约 36-60 s)。北交所成分无笔数字段,该方法会返回 `trade_count_available=False`。
|
||||||
|
- **ETF / LOF 也没有成交笔数**。`get_ticks("sh510310")` 的 `trade_count` 是「价格×1000」(货币 ETF 恒为 0)。判定用 `symbols.has_order_count(market, code)`,读记录时看 `TransactionRecord.trade_count_available`——为 `False` 时 `trade_count` 已被置 0,不要求和/求均值。注意**沪市 `501x/502x/506x` 实测有真实笔数**,不能按「5 开头=基金」一刀切。
|
||||||
- **历史逐笔有笔数**(MAC `0x122F` 支持历史日期,实测可回溯最近 5 个交易日;更早返回空)。
|
- **历史逐笔有笔数**(MAC `0x122F` 支持历史日期,实测可回溯最近 5 个交易日;更早返回空)。
|
||||||
|
- **小数位表不用手动管,但可以手动刷**。`start()` 会检查当日缓存并后台同步一次(`SecurityInfo.decimal_point` 权威值);要强制刷新用 `await td.sync_decimal_points()`,看同步状态用 `await td.decimal_point_status()`。`PYPYTDX_DATA_DISABLE_DECIMAL_SYNC=1` 可关闭后台同步。
|
||||||
- 板块文件(`block_*.dat`)/ 行业(`tdxhy.cfg`)/ 财报(`gpcw*.zip`)是**服务器上的静态文件**,解析后是普通 dataclass,**落 JSON 后完全脱网可用**——适合做离线快照。
|
- 板块文件(`block_*.dat`)/ 行业(`tdxhy.cfg`)/ 财报(`gpcw*.zip`)是**服务器上的静态文件**,解析后是普通 dataclass,**落 JSON 后完全脱网可用**——适合做离线快照。
|
||||||
|
|
||||||
## 陷阱清单
|
## 陷阱清单
|
||||||
@@ -189,6 +198,7 @@ await td.get_bars("cffex:IFL0", KlinePeriod.DAY, count=700) # 期货合约
|
|||||||
| 港股/美股五档 bid/ask | 没有(协议不支持) | `fields` 字段位通道有 name 但无五档 |
|
| 港股/美股五档 bid/ask | 没有(协议不支持) | `fields` 字段位通道有 name 但无五档 |
|
||||||
| 美股分钟线用 `datetime.date()` 分组做日聚合 | 一个交易日被拆成两天(美股跨北京零点) | 用 `bar.trading_date`;`bar.datetime` 是归一化后的北京时间 |
|
| 美股分钟线用 `datetime.date()` 分组做日聚合 | 一个交易日被拆成两天(美股跨北京零点) | 用 `bar.trading_date`;`bar.datetime` 是归一化后的北京时间 |
|
||||||
| 拿指数逐笔的 `trade_count` 当笔数 | 拿到的是「均价×1000」,数值荒谬 | 用 `get_index_trade_stats(index)` 由成分累加 |
|
| 拿指数逐笔的 `trade_count` 当笔数 | 拿到的是「均价×1000」,数值荒谬 | 用 `get_index_trade_stats(index)` 由成分累加 |
|
||||||
|
| 拿 ETF/LOF 逐笔的 `trade_count` 当笔数 | 拿到的是「价格×1000」(`sh510310` 显示 3994 笔) | 看 `trade_count_available`,或 `symbols.has_order_count()` |
|
||||||
| 拿深市 `399xxx` 的 `amount` 校验成分合计 | 比值 0.43,误以为数据错 | 深市指数 amount 是**全板块**口径(`sz399001`=深市全部 5081 亿),不是样本股 |
|
| 拿深市 `399xxx` 的 `amount` 校验成分合计 | 比值 0.43,误以为数据错 | 深市指数 amount 是**全板块**口径(`sz399001`=深市全部 5081 亿),不是样本股 |
|
||||||
| 依赖实时性做信号 | 数据是 15 分钟前的 | 换券商实时源 |
|
| 依赖实时性做信号 | 数据是 15 分钟前的 | 换券商实时源 |
|
||||||
| 手动改包内 `servers.json` / 写死 IP | 容器只读、升级被覆盖、快节点捞不回来 | 日常靠内置的**当日测速缓存**自动选路;要强制指定就用 `PYPYTDX_DATA_{STANDARD,MAC,EX}_HOSTS` 环境变量 |
|
| 手动改包内 `servers.json` / 写死 IP | 容器只读、升级被覆盖、快节点捞不回来 | 日常靠内置的**当日测速缓存**自动选路;要强制指定就用 `PYPYTDX_DATA_{STANDARD,MAC,EX}_HOSTS` 环境变量 |
|
||||||
|
|||||||
+21
-4
@@ -47,13 +47,15 @@ from pytdxdata import (
|
|||||||
|
|
||||||
| 方法 | 返回 | 说明 |
|
| 方法 | 返回 | 说明 |
|
||||||
|------|------|------|
|
|------|------|------|
|
||||||
| `get_universe(market=None)` | `list[SecurityInfo]` | 跨市场标的清单(A 股+港股+美股+期货/期权) |
|
| `get_universe(market=None)` | `list[SecurityInfo]` | 跨市场标的清单(A 股+港股+美股+期货/期权);每条带协议自带的 `decimal_point`(价格小数位)与 `pre_close`。**北交所清单协议不支持,返回空列表** |
|
||||||
|
| `sync_decimal_points(markets=("sh","sz"))` | `dict` | 从证券清单同步**权威价格小数位表**(`SecurityInfo.decimal_point`),灌进 `price_decimal_digits` 的运行时表;`start()` 已自动做(当日缓存命中则零网络),此方法用于强制刷新/排障 |
|
||||||
|
| `decimal_point_status()` | `dict` | 小数位表状态:`source`(`cache`/`universe`/空)、`stock`、`mixed_prefixes`、`syncing` |
|
||||||
|
|
||||||
#### 报价
|
#### 报价
|
||||||
|
|
||||||
| 方法 | 返回 | 说明 |
|
| 方法 | 返回 | 说明 |
|
||||||
|------|------|------|
|
|------|------|------|
|
||||||
| `get_quotes(symbols, *, fields=None)` | `list[SecurityQuote]` | 跨市场混合报价(A 股标准五档;给了 fields 走 MAC/EX 字段位有 name) |
|
| `get_quotes(symbols, *, fields=None, concurrency=None, pipeline=True)` | `list[SecurityQuote]` | 跨市场混合报价(A 股标准五档;给了 fields 走 MAC/EX 字段位有 name)。超 80 只自动切批,`pipeline` 开启**单连接流水线**,全市场 5,200 只 ~0.33s |
|
||||||
| `get_snapshot(symbols)` | `list[SymbolSnapshot]` | 个股特征快照(A 股) |
|
| `get_snapshot(symbols)` | `list[SymbolSnapshot]` | 个股特征快照(A 股) |
|
||||||
| `get_auction(symbols)` | `list[AuctionItem]` | 集合竞价(A 股) |
|
| `get_auction(symbols)` | `list[AuctionItem]` | 集合竞价(A 股) |
|
||||||
| `get_capital_flow(symbols)` | `list[CapitalFlow]` | 个股资金流向(A 股) |
|
| `get_capital_flow(symbols)` | `list[CapitalFlow]` | 个股资金流向(A 股) |
|
||||||
@@ -69,7 +71,7 @@ from pytdxdata import (
|
|||||||
|
|
||||||
| 方法 | 返回 | 说明 |
|
| 方法 | 返回 | 说明 |
|
||||||
|------|------|------|
|
|------|------|------|
|
||||||
| `get_ticks(symbols, *, date=None, start=0, count=None, concurrency=6)` | `list[TransactionRecord]` | 跨市场逐笔成交。返回按时间升序;`start` **从最新一笔往回数**;`vol`/`trade_count` 为约 3 秒切片的增量值(非累计)。**已剔除 `bs_flag=8` 竞价委托**(非成交,`vol` 为累计撮合量)。**指数**:`vol` 单位是百元(成交额/100)、`trade_count` 是均价×1000,均非股票语义 |
|
| `get_ticks(symbols, *, date=None, start=0, count=None, concurrency=6)` | `list[TransactionRecord]` | 跨市场逐笔成交。返回按时间升序;`start` **从最新一笔往回数**;`vol`/`trade_count` 为约 3 秒切片的增量值(非累计)。**已剔除 `bs_flag=8` 竞价委托**(非成交,`vol` 为累计撮合量)。**指数**:`vol` 单位是百元(成交额/100)、`trade_count` 是均价×1000,均非股票语义;**可转债**:`vol` 是手(=10 张)、`trade_count` 是**真实笔数**,`Σvol` 与日线 `vol` **1:1** |
|
||||||
| `get_minutes(symbols, *, date=None, sampling=False, concurrency=6)` | `list[MinuteBar]` | 跨市场分时(sampling=True 取缩略采样:A股由 0x122D 派生,扩展市场 0x254D) |
|
| `get_minutes(symbols, *, date=None, sampling=False, concurrency=6)` | `list[MinuteBar]` | 跨市场分时(sampling=True 取缩略采样:A股由 0x122D 派生,扩展市场 0x254D) |
|
||||||
|
|
||||||
#### 公司信息/文件/板块
|
#### 公司信息/文件/板块
|
||||||
@@ -100,7 +102,7 @@ from pytdxdata import (
|
|||||||
|
|
||||||
| 方法 | 返回 | 说明 |
|
| 方法 | 返回 | 说明 |
|
||||||
|------|------|------|
|
|------|------|------|
|
||||||
| `get_quotes(symbols, *, fields=None)` | `list[SecurityQuote]` | 五档报价;>80 只自动切批;TTL 5s |
|
| `get_quotes(symbols, *, fields=None, concurrency=None, pipeline=True)` | `list[SecurityQuote]` | 五档报价;>80 只自动切批(流水线+工作窃取,5,200 只 ~0.33s);TTL 5s |
|
||||||
| `get_stock_quotes(stocks, bits=None)` | `list[MemberQuote]` | MAC 自定义字段报价(≤80只/次) |
|
| `get_stock_quotes(stocks, bits=None)` | `list[MemberQuote]` | MAC 自定义字段报价(≤80只/次) |
|
||||||
| `get_stock_quotes_list(category, *, start=0, count=80, sort_type=0)` | `list[MemberQuote]` | 市场分类报价(0沪A/2深A/6全A/8科创/14创业) |
|
| `get_stock_quotes_list(category, *, start=0, count=80, sort_type=0)` | `list[MemberQuote]` | 市场分类报价(0沪A/2深A/6全A/8科创/14创业) |
|
||||||
| `get_market_stat()` | `MarketStat` | 全市场统计(涨跌家数/总额/市值/涨跌停数) |
|
| `get_market_stat()` | `MarketStat` | 全市场统计(涨跌家数/总额/市值/涨跌停数) |
|
||||||
@@ -221,6 +223,21 @@ from pytdxdata import (
|
|||||||
|
|
||||||
> 注:`__init__` 仅导出 `Market/ExMarket/KlinePeriod/Adjust` 与 6 个核心模型;`BoardInfo/MemberQuote/AuctionItem` 等需 `from pytdxdata.models import ...`。
|
> 注:`__init__` 仅导出 `Market/ExMarket/KlinePeriod/Adjust` 与 6 个核心模型;`BoardInfo/MemberQuote/AuctionItem` 等需 `from pytdxdata.models import ...`。
|
||||||
|
|
||||||
|
**量纲对照(跨品种,2026-10-22 实测)**
|
||||||
|
|
||||||
|
同名字段在不同品种上单位不同。最典型的坑是用 `amount / vol` 算均价——它在可转债上会**错 10 倍**:
|
||||||
|
|
||||||
|
| 品种 | 分时 / 逐笔 `vol` | 日线 `SecurityBar.vol` | `Σ分时vol / 日线vol` | 日线 `amount / vol` |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| 股票 / ETF / LOF | 手(= 100 股) | 股 | **0.01** | 均价 ✓ |
|
||||||
|
| 可转债(110/111/113/118/123/127/128) | 手(= 10 张) | **手** | **1.0** | 均价 **×10** ⚠️ |
|
||||||
|
| 指数 | 百元(成交额/100) | 手 | — | 无意义(`amount` 是全板块口径) |
|
||||||
|
|
||||||
|
算均价:股票 `amount / vol`;可转债 `amount / vol / 10`;跨品种统一用 `Σ(price×vol) / Σvol`。
|
||||||
|
|
||||||
|
> 可转债的价格标度、分时、逐笔已全量校验(16 只当日 + 8 只历史日):分时末价 / 快照现价、`Σvol / 日线vol`、逐笔 VWAP vs `amount/(vol×10)` 比值均为 **1.000000**。
|
||||||
|
|
||||||
|
|
||||||
## 四、辅助模块
|
## 四、辅助模块
|
||||||
|
|
||||||
**`pytdxdata.indicator`**(技术指标,纯函数)
|
**`pytdxdata.indicator`**(技术指标,纯函数)
|
||||||
|
|||||||
+11
@@ -56,6 +56,17 @@ tdx minute sz000001 # 今日分时
|
|||||||
tdx minute sz000001 --date 20260811 # 历史分时
|
tdx minute sz000001 --date 20260811 # 历史分时
|
||||||
```
|
```
|
||||||
|
|
||||||
|
价格列的小数位**按品种自动适配**,不用自己换算:
|
||||||
|
|
||||||
|
| 品种 | 显示小数位 | 示例 |
|
||||||
|
|---|---|---|
|
||||||
|
| 股票 / 指数 | 2 | `12.34` |
|
||||||
|
| ETF / LOF / 沪市 B 股 / 可转债 | 3 | `128.892`(可转债报价步长 0.001 元) |
|
||||||
|
| 债券 / 回购 | 4 | `100.0100` |
|
||||||
|
|
||||||
|
> 可转债的**解码位数**是 4 位(原始整数 / 10⁴),但第 4 位恒为 0,所以显示按 3 位。
|
||||||
|
> `tdx minute` 的时间列是该 bar 的**真实时钟**(`09:31` / `14:57`),不是序号推算。
|
||||||
|
|
||||||
## 板块
|
## 板块
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
|
|||||||
+7
-3
@@ -44,11 +44,15 @@ asyncio.run(main())
|
|||||||
|
|
||||||
标的一律用字符串;以下均为 v0.6 统一接口,跨市场自动选路。
|
标的一律用字符串;以下均为 v0.6 统一接口,跨市场自动选路。
|
||||||
|
|
||||||
- **标的清单**:`get_universe`(A股 + 港股 + 美股 + 期货/期权)
|
- **标的清单**:`get_universe`(A股 + 港股 + 美股 + 期货/期权)。⚠️ 北交所(`market="bj"`)清单**协议不支持**,返回空列表(`0x053E` 对 market=2 无响应,27 台服务器全测);但北交所**标的本身可用** —— 报价/K线/逐笔直接传 `bj920992` 这类代码正常
|
||||||
- **K 线**:`get_bars`(自动并发分页、支持复权与日期区间、多标的并发)
|
- **K 线**:`get_bars`(自动并发分页、支持复权与日期区间、多标的并发)
|
||||||
- **技术指标**:`get_indicators`(K线 + MACD/KDJ/RSI/BOLL/MA/EMA)
|
- **技术指标**:`get_indicators`(K线 + MACD/KDJ/RSI/BOLL/MA/EMA)
|
||||||
- **报价**:`get_quotes`(五档,>80 只自动切批)、`get_snapshot`(个股特征)、`get_market_stat`
|
- **报价**:`get_quotes`(五档,>80 只自动切批;**单连接流水线**默认开启,全市场 5,200 只 A股 ~0.33s)、`get_snapshot`(个股特征)、`get_market_stat`
|
||||||
- **逐笔 / 分时**:`get_ticks`(含 `trade_count`,支持历史日期,返回按时间升序;`vol`/`trade_count` 为约 3 秒切片增量而非累计;已剔除 `bs_flag=8` 竞价委托;**指数**的 `vol` 是百元、`trade_count` 是均价×1000)、`get_minutes`(`sampling=True` 取缩略采样;`hour` 恒为合法时钟值 0~23,美股一个交易日跨两个北京自然日,按交易日聚合用 `MinuteBar.trading_date`、取时刻用 `MinuteBar.datetime`)
|
- 价格标度:协议给的是整数,真实价 = `整数 / 10**小数位`。小数位由 `price_decimal_digits` 决定,
|
||||||
|
优先用**权威表**(证券清单自带的 `SecurityInfo.decimal_point`,`start()` 每日后台同步一次,
|
||||||
|
也可 `await td.sync_decimal_points()` 强制刷新),未同步时退回静态前缀规则。
|
||||||
|
债券/回购是 **4 位**、沪市 B 股 3 位 —— 这里错过两次(整批价格差 100 倍/10 倍)。
|
||||||
|
- **逐笔 / 分时**:`get_ticks`(含 `trade_count`,支持历史日期,返回按时间升序;`vol`/`trade_count` 为约 3 秒切片增量而非累计;已剔除 `bs_flag=8` 竞价委托;**指数**的 `vol` 是百元、`trade_count` 是均价×1000;**可转债**的 `vol` 是手=10 张、`trade_count` 是真实笔数,故日线 `amount/vol` 是均价的 **10 倍**,算均价要再 /10)、`get_minutes`(`sampling=True` 取缩略采样;`hour` 恒为合法时钟值 0~23,美股一个交易日跨两个北京自然日,按交易日聚合用 `MinuteBar.trading_date`、取时刻用 `MinuteBar.datetime`)
|
||||||
- **财务 / 除权**:`get_finance`(最新快照)、`get_xdxr`、`get_price_limits`(本地计算涨跌停)
|
- **财务 / 除权**:`get_finance`(最新快照)、`get_xdxr`、`get_price_limits`(本地计算涨跌停)
|
||||||
- **F10**:`get_f10`(公司资料目录 + 正文)
|
- **F10**:`get_f10`(公司资料目录 + 正文)
|
||||||
- **板块**:`get_boards`、`get_board_members`、`get_board_of`、`get_board_summary`、
|
- **板块**:`get_boards`、`get_board_members`、`get_board_of`、`get_board_summary`、
|
||||||
|
|||||||
+1
-1
@@ -1,6 +1,6 @@
|
|||||||
[project]
|
[project]
|
||||||
name = "pytdxdata"
|
name = "pytdxdata"
|
||||||
version = "0.6.5"
|
version = "0.6.9"
|
||||||
description = "通达信(TongDaXin)行情数据客户端:动态连接池 + 双层TTL缓存 + 3连接并发分页,纯asyncio零运行时依赖,支持A股/港股/美股/期货/期权"
|
description = "通达信(TongDaXin)行情数据客户端:动态连接池 + 双层TTL缓存 + 3连接并发分页,纯asyncio零运行时依赖,支持A股/港股/美股/期货/期权"
|
||||||
readme = "README.md"
|
readme = "README.md"
|
||||||
requires-python = ">=3.11"
|
requires-python = ">=3.11"
|
||||||
|
|||||||
@@ -18,7 +18,7 @@ from .models import (
|
|||||||
XdxrRecord,
|
XdxrRecord,
|
||||||
)
|
)
|
||||||
|
|
||||||
__version__ = "0.6.5"
|
__version__ = "0.6.9"
|
||||||
__all__ = [
|
__all__ = [
|
||||||
"TdxData",
|
"TdxData",
|
||||||
"Market",
|
"Market",
|
||||||
|
|||||||
+228
-30
@@ -25,11 +25,13 @@ import logging
|
|||||||
import re
|
import re
|
||||||
import time
|
import time
|
||||||
import warnings
|
import warnings
|
||||||
|
from collections import deque
|
||||||
from collections.abc import Callable
|
from collections.abc import Callable
|
||||||
from datetime import date as _date, datetime
|
from datetime import date as _date, datetime
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Sequence
|
from typing import Sequence
|
||||||
|
|
||||||
|
from . import decimal_sync
|
||||||
from .cache.disk import DiskCache
|
from .cache.disk import DiskCache
|
||||||
from .cache.key import CacheKey
|
from .cache.key import CacheKey
|
||||||
from .cache.manager import CacheManager
|
from .cache.manager import CacheManager
|
||||||
@@ -146,12 +148,19 @@ _KLINE_MAX_BARS: dict[KlinePeriod, int] = {
|
|||||||
# 自定义字段报价(0x122B/0x122C)基础字段位:昨收/开/高/低/现价/量/量比/额
|
# 自定义字段报价(0x122B/0x122C)基础字段位:昨收/开/高/低/现价/量/量比/额
|
||||||
_QUOTE_BITS = [0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07]
|
_QUOTE_BITS = [0x00, 0x01, 0x02, 0x03, 0x04, 0x05, 0x06, 0x07]
|
||||||
_QUOTE_PAGE = 80 # 0x122B/0x122C 单次上限 80 只
|
_QUOTE_PAGE = 80 # 0x122B/0x122C 单次上限 80 只
|
||||||
|
_QUOTE_PIPE_CHUNK = 3 # 单连接流水线一次取几片(实测 3 片 + 每台 2 连接最稳)
|
||||||
_EX_PAGE = 1000 # EX 商品列表单页
|
_EX_PAGE = 1000 # EX 商品列表单页
|
||||||
_TICK_PAGE = 1000 # MAC 逐笔单页
|
_TICK_PAGE = 1000 # MAC 逐笔单页
|
||||||
_EX_TICK_PAGE = 1800 # 港股逐笔单页
|
_EX_TICK_PAGE = 1800 # 港股逐笔单页
|
||||||
_ALL_PAGES_CAP = 50 # count=None(全量)时的翻页上限
|
_ALL_PAGES_CAP = 50 # count=None(全量)时的翻页上限
|
||||||
# 集合竞价**委托**记录标记(0x122F 第 5 字段)。不是成交,见 _ticks_one 注释。
|
# 集合竞价**委托**记录标记(0x122F 第 5 字段)。不是成交,见 _ticks_one 注释。
|
||||||
_BS_AUCTION_ORDER = 8
|
_BS_AUCTION_ORDER = 8
|
||||||
|
# 证券清单(0x053E)拿不到的市场。实测 27 台标准主机对 market=2(北交所)**全部
|
||||||
|
# 不响应**(26 台 6s 无回包、1 台立刻拒连),而同 market 的 0x053D 数量接口正常
|
||||||
|
# (返回 389)。故清单直接返回空,避免每次调用白等数秒再抛 ConnectionError。
|
||||||
|
# 北交所**标的本身没问题**:报价/K线/逐笔用 ``bj920992`` 这类代码都正常。
|
||||||
|
_LIST_UNSUPPORTED_MARKETS = frozenset({int(Market.BJ)})
|
||||||
|
_list_unsupported_warned: set[int] = set() # 同一市场只 WARNING 一次
|
||||||
|
|
||||||
# 港股系市场:逐笔走 EX 专用 16B 协议(0x23FC/0x2406),其余扩展市场走 MAC 0x122F
|
# 港股系市场:逐笔走 EX 专用 16B 协议(0x23FC/0x2406),其余扩展市场走 MAC 0x122F
|
||||||
_HK_MARKETS = frozenset({27, 31, 48, 49, 71, 98})
|
_HK_MARKETS = frozenset({27, 31, 48, 49, 71, 98})
|
||||||
@@ -236,7 +245,7 @@ class TdxData:
|
|||||||
default_adjust: Adjust = Adjust.NONE,
|
default_adjust: Adjust = Adjust.NONE,
|
||||||
max_inflight: int = 64,
|
max_inflight: int = 64,
|
||||||
pool_min: int = 3,
|
pool_min: int = 3,
|
||||||
pool_max: int = 12,
|
pool_max: int = 24,
|
||||||
mac_pool_cap: int = 4,
|
mac_pool_cap: int = 4,
|
||||||
std_pool_cap: int = 4,
|
std_pool_cap: int = 4,
|
||||||
ex_pool_cap: int = 2,
|
ex_pool_cap: int = 2,
|
||||||
@@ -285,6 +294,7 @@ class TdxData:
|
|||||||
self._paginator_std = Paginator(self._std_pool)
|
self._paginator_std = Paginator(self._std_pool)
|
||||||
self._paginator_mac = Paginator(self._mac_pool)
|
self._paginator_mac = Paginator(self._mac_pool)
|
||||||
self._started = False
|
self._started = False
|
||||||
|
self._decimal_task: asyncio.Task | None = None
|
||||||
|
|
||||||
async def start(self) -> None:
|
async def start(self) -> None:
|
||||||
await self._std_pool.start()
|
await self._std_pool.start()
|
||||||
@@ -292,8 +302,13 @@ class TdxData:
|
|||||||
await self._ex_pool.start()
|
await self._ex_pool.start()
|
||||||
await self._cache.start()
|
await self._cache.start()
|
||||||
self._started = True
|
self._started = True
|
||||||
|
# 权威价格小数位表:当日缓存命中则同步装表(无网络),否则挂后台同步。
|
||||||
|
# 不阻塞 start(),失败静默退回静态前缀规则。
|
||||||
|
self._decimal_task = decimal_sync.schedule(self)
|
||||||
|
|
||||||
async def close(self) -> None:
|
async def close(self) -> None:
|
||||||
|
await decimal_sync.cancel(self._decimal_task)
|
||||||
|
self._decimal_task = None
|
||||||
await self._std_pool.close()
|
await self._std_pool.close()
|
||||||
await self._mac_pool.close()
|
await self._mac_pool.close()
|
||||||
await self._ex_pool.close()
|
await self._ex_pool.close()
|
||||||
@@ -322,16 +337,68 @@ class TdxData:
|
|||||||
返回:
|
返回:
|
||||||
``list[SecurityInfo]``,每条带 ``symbol`` 规范化标的串与 ``kind`` 分类描述。
|
``list[SecurityInfo]``,每条带 ``symbol`` 规范化标的串与 ``kind`` 分类描述。
|
||||||
EX 通道无昨收/单位,对应字段为 0。
|
EX 通道无昨收/单位,对应字段为 0。
|
||||||
|
|
||||||
|
注意:
|
||||||
|
**北交所(``"bj"``)清单返回空列表**:清单命令 0x053E 对 market=2
|
||||||
|
在所有实测服务器上都不响应(27 台全测),协议层根本拿不到。
|
||||||
|
但北交所**标的本身可用** —— 报价/K线/逐笔直接传 ``bj920992`` 这类代码
|
||||||
|
都正常,只是无法枚举全量代码。``market="all"`` 时该市场被跳过,不会报错。
|
||||||
"""
|
"""
|
||||||
out: list[SecurityInfo] = []
|
out: list[SecurityInfo] = []
|
||||||
|
ex_items: list[SecurityInfo] | None = None
|
||||||
for mkt, channel in market_targets(market):
|
for mkt, channel in market_targets(market):
|
||||||
if channel == "ex":
|
if channel == "ex":
|
||||||
out.extend(await self._universe_ex(mkt))
|
# EX 全量只取一次:命中缓存也要反序列化 10 万条(~200ms),
|
||||||
|
# 十几个 EX 市场各来一次纯属白烧。
|
||||||
|
if ex_items is None:
|
||||||
|
ex_items = await self._ex_all()
|
||||||
|
out.extend(it for it in ex_items if it.market == mkt)
|
||||||
else:
|
else:
|
||||||
out.extend(await self._universe_cn(mkt))
|
out.extend(await self._universe_cn(mkt))
|
||||||
return out
|
return out
|
||||||
|
|
||||||
|
async def sync_decimal_points(self, markets: Sequence[str] = ("sh", "sz")
|
||||||
|
) -> dict[str, int]:
|
||||||
|
"""从证券清单同步**权威价格小数位表**,返回统计。
|
||||||
|
|
||||||
|
报价接口返回的是整数,真实价格 = ``整数 / 10**小数位``。小数位不随报价下发,
|
||||||
|
但证券清单接口自带权威值 ``SecurityInfo.decimal_point`` —— 本方法把它灌进
|
||||||
|
:func:`~pytdxdata.protocol.commands.quotes.price_decimal_digits` 使用的运行时表,
|
||||||
|
从此价格标度不再依赖硬编码前缀规则(新增代码段也不会算错)。
|
||||||
|
|
||||||
|
``start()`` 已自动处理:当日缓存命中就直接装表,否则挂后台同步,**不用手动调**。
|
||||||
|
这个方法用于「立刻要权威表」「排障时强制刷新」。
|
||||||
|
|
||||||
|
参数:
|
||||||
|
markets: 拉哪些市场,默认 ``("sh", "sz")``。证券清单走 ``get_universe``,
|
||||||
|
其本身已有缓存层,重复调用只花一次网络。
|
||||||
|
|
||||||
|
返回:
|
||||||
|
``{"prefixes", "codes", "mixed_prefixes", "stock"}`` —— 已覆盖的三位前缀数、
|
||||||
|
例外代码数、出现混合小数位的前缀数、总条目数。
|
||||||
|
``mixed_prefixes`` > 0 说明某段代码混用了不同小数位(已自动退化为逐代码查表)。
|
||||||
|
|
||||||
|
用法::
|
||||||
|
|
||||||
|
async with TdxData() as td:
|
||||||
|
print(await td.sync_decimal_points()) # {'prefixes': 300+, 'codes': 0, ...}
|
||||||
|
q = await td.get_quotes("sh019742") # 110.228(4 位小数)
|
||||||
|
"""
|
||||||
|
return await decimal_sync.refresh(self, tuple(markets))
|
||||||
|
|
||||||
|
async def decimal_point_status(self) -> dict:
|
||||||
|
"""权威小数位表当前状态(``source`` / ``stock`` / ``syncing`` 等,排障用)。"""
|
||||||
|
return decimal_sync.status()
|
||||||
|
|
||||||
async def _universe_cn(self, market: int) -> list[SecurityInfo]:
|
async def _universe_cn(self, market: int) -> list[SecurityInfo]:
|
||||||
|
if market in _LIST_UNSUPPORTED_MARKETS:
|
||||||
|
if market not in _list_unsupported_warned:
|
||||||
|
_list_unsupported_warned.add(market)
|
||||||
|
log.warning(
|
||||||
|
"市场 %s 的证券清单接口协议不支持(0x053E 无响应),返回空列表;"
|
||||||
|
"该市场标的的报价/K线/逐笔不受影响", market)
|
||||||
|
return []
|
||||||
|
|
||||||
key = CacheKey("list", market)
|
key = CacheKey("list", market)
|
||||||
|
|
||||||
async def fetch() -> bytes:
|
async def fetch() -> bytes:
|
||||||
@@ -360,13 +427,18 @@ class TdxData:
|
|||||||
|
|
||||||
async def fetch() -> bytes:
|
async def fetch() -> bytes:
|
||||||
total = int(await self._execute_ex(GetExInstrumentCountCmd()))
|
total = int(await self._execute_ex(GetExInstrumentCountCmd()))
|
||||||
items: list[SecurityInfo] = []
|
starts = list(range(0, total, _EX_PAGE))
|
||||||
pos = 0
|
sem = asyncio.Semaphore(8) # 实测 8 页并发 292ms(串行 110 页 ~4s)
|
||||||
while pos < total:
|
|
||||||
page = await self._execute_ex(
|
async def one(pos: int) -> list[dict]:
|
||||||
|
async with sem:
|
||||||
|
return await self._execute_ex(
|
||||||
GetExInstrumentInfoCmd(0, pos, _EX_PAGE))
|
GetExInstrumentInfoCmd(0, pos, _EX_PAGE))
|
||||||
|
|
||||||
|
items: list[SecurityInfo] = []
|
||||||
|
for page in await asyncio.gather(*(one(s) for s in starts)):
|
||||||
if not page:
|
if not page:
|
||||||
break
|
continue
|
||||||
for g in page:
|
for g in page:
|
||||||
mkt = int(g["market"])
|
mkt = int(g["market"])
|
||||||
code = str(g["code"]).strip()
|
code = str(g["code"]).strip()
|
||||||
@@ -380,7 +452,6 @@ class TdxData:
|
|||||||
symbol=format_symbol(mkt, code),
|
symbol=format_symbol(mkt, code),
|
||||||
kind=str(g.get("desc", "")).strip(),
|
kind=str(g.get("desc", "")).strip(),
|
||||||
))
|
))
|
||||||
pos += len(page)
|
|
||||||
return pack_payload(items)
|
return pack_payload(items)
|
||||||
|
|
||||||
return unpack_payload(await self._cache.get_or_fetch(key, fetch))
|
return unpack_payload(await self._cache.get_or_fetch(key, fetch))
|
||||||
@@ -402,7 +473,9 @@ class TdxData:
|
|||||||
# ================================================================== #
|
# ================================================================== #
|
||||||
|
|
||||||
async def get_quotes(self, symbols: str | Sequence[str], *,
|
async def get_quotes(self, symbols: str | Sequence[str], *,
|
||||||
fields: Sequence[int] | None = None
|
fields: Sequence[int] | None = None,
|
||||||
|
concurrency: int | None = None,
|
||||||
|
pipeline: bool = True
|
||||||
) -> list[SecurityQuote]:
|
) -> list[SecurityQuote]:
|
||||||
"""报价,一次可跨市场、跨通道混合查询。
|
"""报价,一次可跨市场、跨通道混合查询。
|
||||||
|
|
||||||
@@ -412,6 +485,11 @@ class TdxData:
|
|||||||
fields: 可选自定义字段位(如 ``[0x00, 0x04, 0x38]``)。给了就走 MAC/EX 的
|
fields: 可选自定义字段位(如 ``[0x00, 0x04, 0x38]``)。给了就走 MAC/EX 的
|
||||||
0x122B 字段位通道(有 ``name``,无五档);不给时 A股走标准通道
|
0x122B 字段位通道(有 ``name``,无五档);不给时 A股走标准通道
|
||||||
(**有五档 bid/ask、买卖量、涨跌停价,无 name**),扩展市场走 0x122B。
|
(**有五档 bid/ask、买卖量、涨跌停价,无 name**),扩展市场走 0x122B。
|
||||||
|
concurrency: 连接并发数(默认取连接池上限)。协议硬上限 80 只/请求,
|
||||||
|
全市场约 5,200 只 A股需切 66 片;调小可降低对服务器的压力。
|
||||||
|
pipeline: 是否启用**单连接流水线**(默认开启)。开启后一条连接可一次发出
|
||||||
|
多个分片、只付一次往返,全市场 5,200 只 A股由 ~4s 降到 **~0.3s**;
|
||||||
|
个股或小批量不受影响。个别服务器不支持流水线时会自动降级为逐片。
|
||||||
|
|
||||||
返回:
|
返回:
|
||||||
``list[SecurityQuote]``,按入参顺序排列(各记录带 ``symbol``)。
|
``list[SecurityQuote]``,按入参顺序排列(各记录带 ``symbol``)。
|
||||||
@@ -422,50 +500,155 @@ class TdxData:
|
|||||||
bits = [int(b) for b in fields] if fields else None
|
bits = [int(b) for b in fields] if fields else None
|
||||||
tag = "mac:" + _fields_tag(bits) if bits else "std"
|
tag = "mac:" + _fields_tag(bits) if bits else "std"
|
||||||
key = CacheKey.quote_key([s.as_pair() for s in syms], tag=tag)
|
key = CacheKey.quote_key([s.as_pair() for s in syms], tag=tag)
|
||||||
payload = await self._cache.get_or_fetch(
|
quotes = await self._cache.get_or_fetch_obj(
|
||||||
key, lambda: self._fetch_quotes(syms, bits))
|
key,
|
||||||
quotes = unpack_payload(payload)
|
lambda: self._fetch_quotes(syms, bits, concurrency, pipeline),
|
||||||
|
pack=pack_payload, unpack=unpack_payload)
|
||||||
order = {s.as_pair(): i for i, s in enumerate(syms)}
|
order = {s.as_pair(): i for i, s in enumerate(syms)}
|
||||||
quotes.sort(key=lambda q: order.get((int(q.market), q.code), len(order)))
|
quotes.sort(key=lambda q: order.get((int(q.market), q.code), len(order)))
|
||||||
return quotes
|
return quotes
|
||||||
|
|
||||||
async def _fetch_quotes(self, syms: list[Symbol], bits: list[int] | None) -> bytes:
|
async def _fetch_quotes(self, syms: list[Symbol], bits: list[int] | None,
|
||||||
|
concurrency: int | None = None,
|
||||||
|
pipeline: bool = True) -> list[SecurityQuote]:
|
||||||
out: list[SecurityQuote] = []
|
out: list[SecurityQuote] = []
|
||||||
cn = [s for s in syms if not s.is_ex]
|
cn = [s for s in syms if not s.is_ex]
|
||||||
ex = [s for s in syms if s.is_ex]
|
ex = [s for s in syms if s.is_ex]
|
||||||
if bits is None:
|
if bits is None:
|
||||||
if cn:
|
if cn:
|
||||||
out.extend(await self._quotes_std(cn))
|
out.extend(await self._quotes_std(
|
||||||
|
cn, concurrency=concurrency, pipeline=pipeline))
|
||||||
if ex:
|
if ex:
|
||||||
rows = await self._execute_ex(
|
rows = await self._execute_ex(
|
||||||
MacSymbolQuotesCmd([s.as_pair() for s in ex], None))
|
MacSymbolQuotesCmd([s.as_pair() for s in ex], None))
|
||||||
out.extend(_quote_from_member(m) for m in rows)
|
out.extend(_quote_from_member(m) for m in rows)
|
||||||
else:
|
else:
|
||||||
for i in range(0, len(syms), _QUOTE_PAGE):
|
batches = [syms[i:i + _QUOTE_PAGE]
|
||||||
batch = syms[i:i + _QUOTE_PAGE]
|
for i in range(0, len(syms), _QUOTE_PAGE)]
|
||||||
|
sem = asyncio.Semaphore(self._quote_concurrency(concurrency))
|
||||||
|
|
||||||
|
async def one(batch: list[Symbol]) -> list[SecurityQuote]:
|
||||||
|
async with sem:
|
||||||
pairs = [s.as_pair() for s in batch]
|
pairs = [s.as_pair() for s in batch]
|
||||||
cn_pairs = [p for p, s in zip(pairs, batch) if not s.is_ex]
|
cn_pairs = [p for p, s in zip(pairs, batch) if not s.is_ex]
|
||||||
ex_pairs = [p for p, s in zip(pairs, batch) if s.is_ex]
|
ex_pairs = [p for p, s in zip(pairs, batch) if s.is_ex]
|
||||||
|
got: list[SecurityQuote] = []
|
||||||
if cn_pairs:
|
if cn_pairs:
|
||||||
rows = await self._execute_mac(MacSymbolQuotesCmd(cn_pairs, bits))
|
rows = await self._execute_mac(
|
||||||
out.extend(_quote_from_member(m) for m in rows)
|
MacSymbolQuotesCmd(cn_pairs, bits))
|
||||||
|
got.extend(_quote_from_member(m) for m in rows)
|
||||||
if ex_pairs:
|
if ex_pairs:
|
||||||
rows = await self._execute_ex(MacSymbolQuotesCmd(ex_pairs, bits))
|
rows = await self._execute_ex(
|
||||||
out.extend(_quote_from_member(m) for m in rows)
|
MacSymbolQuotesCmd(ex_pairs, bits))
|
||||||
return pack_payload(out)
|
got.extend(_quote_from_member(m) for m in rows)
|
||||||
|
return got
|
||||||
|
|
||||||
async def _quotes_std(self, syms: list[Symbol]) -> list[SecurityQuote]:
|
if batches:
|
||||||
"""A股标准通道五档报价(>80 只自动切批)。"""
|
for got in await asyncio.gather(*(one(b) for b in batches)):
|
||||||
out: list[SecurityQuote] = []
|
out.extend(got)
|
||||||
for i in range(0, len(syms), MAX_QUOTES):
|
return out
|
||||||
batch = syms[i:i + MAX_QUOTES]
|
|
||||||
cmd = GetSecurityQuotesCmd([s.as_pair() for s in batch])
|
def _quote_concurrency(self, requested: int | None) -> int:
|
||||||
rows = await self._execute_standard(cmd)
|
"""连接并发数:显式指定优先,否则按「每台服务器 2 连接」估算。
|
||||||
|
|
||||||
|
实测(5,227 只 A股 / 66 片,11 台服务器):
|
||||||
|
|
||||||
|
========== ===== ====== ====== ======
|
||||||
|
并发 每台连接 流水深度 p50 max
|
||||||
|
========== ===== ====== ====== ======
|
||||||
|
11 1 6 455ms 464ms
|
||||||
|
12 1~2 3 457ms 523ms
|
||||||
|
**22** **2** **3** **325ms** **334ms**
|
||||||
|
33 3 2 328ms 345ms
|
||||||
|
========== ===== ====== ====== ======
|
||||||
|
|
||||||
|
结论:每台 2 连接 × 流水 3 片最稳;再多连接只是把同一台的队列搬个位置,
|
||||||
|
对整体无益反增抖动。池 ``per_server_cap`` 默认 4,2 连接仍留有余量。
|
||||||
|
"""
|
||||||
|
if requested is not None and requested > 0:
|
||||||
|
return max(1, int(requested))
|
||||||
|
n = len(self._std_pool.servers)
|
||||||
|
return max(1, min(self._std_pool.max_size, (n or 1) * 2))
|
||||||
|
|
||||||
|
async def _quotes_std(self, syms: list[Symbol], *,
|
||||||
|
concurrency: int | None = None,
|
||||||
|
pipeline: bool = True) -> list[SecurityQuote]:
|
||||||
|
"""A股标准通道五档报价(协议上限 80 只/请求,超出自动切批)。
|
||||||
|
|
||||||
|
切片多于 1 片时把各片**按轮转分给若干条连接**,每条连接内部用流水线
|
||||||
|
一次发出(1 次往返拿多片),不再逐片往返。
|
||||||
|
"""
|
||||||
|
if not syms:
|
||||||
|
return []
|
||||||
|
|
||||||
|
def stamp(rows: list[SecurityQuote]) -> list[SecurityQuote]:
|
||||||
for q in rows:
|
for q in rows:
|
||||||
q.symbol = format_symbol(q.market, q.code)
|
q.symbol = format_symbol(q.market, q.code)
|
||||||
out.extend(rows)
|
return rows
|
||||||
|
|
||||||
|
batches = [syms[i:i + MAX_QUOTES]
|
||||||
|
for i in range(0, len(syms), MAX_QUOTES)]
|
||||||
|
if len(batches) == 1:
|
||||||
|
cmd = GetSecurityQuotesCmd([s.as_pair() for s in batches[0]])
|
||||||
|
return stamp(await self._execute_standard(cmd))
|
||||||
|
|
||||||
|
conc = min(self._quote_concurrency(concurrency), len(batches))
|
||||||
|
|
||||||
|
# 首次大批量:先把连接并发建好,否则 acquire 会在池锁里串行握手
|
||||||
|
# (22 路并发首次调用实测多花 5~6s)
|
||||||
|
await self._std_pool.warm(conc)
|
||||||
|
|
||||||
|
if not pipeline:
|
||||||
|
sem = asyncio.Semaphore(conc)
|
||||||
|
|
||||||
|
async def one(batch: list[Symbol]) -> list[SecurityQuote]:
|
||||||
|
async with sem:
|
||||||
|
cmd = GetSecurityQuotesCmd([s.as_pair() for s in batch])
|
||||||
|
return stamp(await self._execute_standard(cmd))
|
||||||
|
|
||||||
|
groups = await asyncio.gather(*(one(b) for b in batches))
|
||||||
|
return [q for g in groups for q in g]
|
||||||
|
|
||||||
|
# 工作窃取:连接做完一批就回来再取,快的多干、慢的少干。
|
||||||
|
# 平均分配(每台固定几片)会被最慢的一台拖住 —— 实测 11 台平均分
|
||||||
|
# 时 p50 在 400~580ms 之间抖,改窃取后波动收窄。
|
||||||
|
queue = deque(batches)
|
||||||
|
gate = asyncio.Lock()
|
||||||
|
|
||||||
|
async def worker() -> list[SecurityQuote]:
|
||||||
|
got: list[SecurityQuote] = []
|
||||||
|
while True:
|
||||||
|
async with gate:
|
||||||
|
if not queue:
|
||||||
|
return got
|
||||||
|
take = min(_QUOTE_PIPE_CHUNK, len(queue))
|
||||||
|
chunk = [queue.popleft() for _ in range(take)]
|
||||||
|
for rows in await self._execute_std_pipeline(chunk):
|
||||||
|
got.extend(stamp(rows))
|
||||||
|
|
||||||
|
out: list[SecurityQuote] = []
|
||||||
|
for part in await asyncio.gather(*(worker() for _ in range(conc))):
|
||||||
|
out.extend(part)
|
||||||
return out
|
return out
|
||||||
|
|
||||||
|
async def _execute_std_pipeline(
|
||||||
|
self, batches: list[list[Symbol]]
|
||||||
|
) -> list[list[SecurityQuote]]:
|
||||||
|
"""一条连接流水线执行多片报价;节点不支持流水线时降级为逐片。"""
|
||||||
|
cmds = [GetSecurityQuotesCmd([s.as_pair() for s in b]) for b in batches]
|
||||||
|
pc = await self._std_pool.acquire()
|
||||||
|
try:
|
||||||
|
rows = await pc.execute_pipeline(cmds)
|
||||||
|
except Exception:
|
||||||
|
# 流水线中断后响应流已错配,连接必须废弃;改用逐片(每条独立借还)
|
||||||
|
await self._std_pool.release(pc, broken=True)
|
||||||
|
fallback: list[list[SecurityQuote]] = []
|
||||||
|
for cmd in cmds:
|
||||||
|
fallback.append(await self._execute_standard(cmd))
|
||||||
|
return fallback
|
||||||
|
await self._std_pool.release(pc)
|
||||||
|
return rows
|
||||||
|
|
||||||
async def get_snapshot(self, symbols: str | Sequence[str]) -> list[SymbolSnapshot]:
|
async def get_snapshot(self, symbols: str | Sequence[str]) -> list[SymbolSnapshot]:
|
||||||
"""个股特征快照(当日/换手/活跃度等,仅 A股)。"""
|
"""个股特征快照(当日/换手/活跃度等,仅 A股)。"""
|
||||||
syms = parse_symbols(symbols)
|
syms = parse_symbols(symbols)
|
||||||
@@ -820,6 +1003,10 @@ class TdxData:
|
|||||||
- ``vol`` 单位是**百元**(成交额/100),不是手。
|
- ``vol`` 单位是**百元**(成交额/100),不是手。
|
||||||
- ``trade_count`` 是**当时均价 × 1000**,不是笔数。
|
- ``trade_count`` 是**当时均价 × 1000**,不是笔数。
|
||||||
|
|
||||||
|
**ETF / LOF 同样没有笔数**:``trade_count`` 是「价格 × 1000」(货币 ETF 恒为 0),
|
||||||
|
这类记录的 ``trade_count_available`` 为 ``False`` 且 ``trade_count`` 已置 0。
|
||||||
|
判定规则见 :func:`~pytdxdata.symbols.has_order_count`。
|
||||||
|
|
||||||
详见 :class:`~pytdxdata.models.TransactionRecord` 的类注释。
|
详见 :class:`~pytdxdata.models.TransactionRecord` 的类注释。
|
||||||
"""
|
"""
|
||||||
syms = parse_symbols(symbols)
|
syms = parse_symbols(symbols)
|
||||||
@@ -966,6 +1153,7 @@ class TdxData:
|
|||||||
trade_count=0,
|
trade_count=0,
|
||||||
bs_flag=int(row.get("direction", 0) or 0),
|
bs_flag=int(row.get("direction", 0) or 0),
|
||||||
fields=fields,
|
fields=fields,
|
||||||
|
trade_count_available=False, # 扩展市场逐笔不带笔数字段
|
||||||
)
|
)
|
||||||
|
|
||||||
async def get_minutes(
|
async def get_minutes(
|
||||||
@@ -1352,6 +1540,14 @@ class TdxData:
|
|||||||
|
|
||||||
注意:返回的是**当前**成分,指数调仓后历史日期不适用。
|
注意:返回的是**当前**成分,指数调仓后历史日期不适用。
|
||||||
|
|
||||||
|
⚠️ **综合指数的成员名单不完整**(实测 2026-10-22):``sh000001`` 只返回
|
||||||
|
2230 只(2190 A + 40 B),而当日沪市 A 股实有 2318 只;缺的 128 只
|
||||||
|
(多为近期科创板/主板新股)当日成交 282 亿,正好对上「指数日线 amount −
|
||||||
|
成员日线 amount」的 297 亿(残差 4.9%)。因此
|
||||||
|
:meth:`get_index_trade_stats` 对**综合指数**会系统性低估约 4.6%;
|
||||||
|
样本指数 ``sh000016``(50 只)/ ``sh000300``(300 只)实测比值
|
||||||
|
**1.000000**,不受影响。要精确汇总综指成交,请自行用全市场股票列表。
|
||||||
|
|
||||||
用法::
|
用法::
|
||||||
|
|
||||||
members = await td.get_index_members("000300")
|
members = await td.get_index_members("000300")
|
||||||
@@ -1468,10 +1664,12 @@ class TdxData:
|
|||||||
for t in ticks:
|
for t in ticks:
|
||||||
hhmm = t.time[:5]
|
hhmm = t.time[:5]
|
||||||
vol_by_min[hhmm] = vol_by_min.get(hhmm, 0.0) + t.vol
|
vol_by_min[hhmm] = vol_by_min.get(hhmm, 0.0) + t.vol
|
||||||
|
# 成分里若混入 ETF/LOF(协议无笔数),不能把占位值累加进笔数
|
||||||
|
if getattr(t, "trade_count_available", True):
|
||||||
tc_by_min[hhmm] = tc_by_min.get(hhmm, 0) + t.trade_count
|
tc_by_min[hhmm] = tc_by_min.get(hhmm, 0) + t.trade_count
|
||||||
amt_by_min[hhmm] = amt_by_min.get(hhmm, 0.0) + t.price * t.vol * 100.0
|
amt_by_min[hhmm] = amt_by_min.get(hhmm, 0.0) + t.price * t.vol * 100.0
|
||||||
active.add(t.symbol)
|
active.add(t.symbol)
|
||||||
if t.trade_count:
|
if getattr(t, "trade_count_available", True) and t.trade_count:
|
||||||
with_tc.add(t.symbol)
|
with_tc.add(t.symbol)
|
||||||
done += len(batch)
|
done += len(batch)
|
||||||
if on_progress is not None:
|
if on_progress is not None:
|
||||||
|
|||||||
Vendored
+32
@@ -90,6 +90,38 @@ class CacheManager:
|
|||||||
# 确保清理锁(即使 fetcher 异常也清理,避免无限增长)
|
# 确保清理锁(即使 fetcher 异常也清理,避免无限增长)
|
||||||
self._locks.pop(ks, None)
|
self._locks.pop(ks, None)
|
||||||
|
|
||||||
|
async def get_or_fetch_obj(
|
||||||
|
self,
|
||||||
|
key: CacheKey,
|
||||||
|
fetcher: Callable[[], Awaitable[Any]],
|
||||||
|
*,
|
||||||
|
pack: Callable[[Any], bytes],
|
||||||
|
unpack: Callable[[bytes], Any],
|
||||||
|
) -> Any:
|
||||||
|
"""同 :meth:`get_or_fetch`,但**回源时直接返回新取到的对象**。
|
||||||
|
|
||||||
|
用于「一次取回上千条记录」的场景(如全市场报价):常规路径对冷数据
|
||||||
|
会先 ``pack`` 再立刻 ``unpack`` 一次,纯属往返浪费 —— 5,200 条报价实测
|
||||||
|
白烧约 90ms。这里命中缓存才解序列化,回源则把对象原样返回,只在写缓存时打包。
|
||||||
|
"""
|
||||||
|
ks = key.to_str()
|
||||||
|
cached = await self.get(key)
|
||||||
|
if cached is not None:
|
||||||
|
return unpack(cached)
|
||||||
|
|
||||||
|
lock = self._locks.setdefault(ks, asyncio.Lock())
|
||||||
|
try:
|
||||||
|
async with lock:
|
||||||
|
cached = await self.get(key)
|
||||||
|
if cached is not None:
|
||||||
|
return unpack(cached)
|
||||||
|
async with self._semaphore:
|
||||||
|
obj = await fetcher()
|
||||||
|
await self.set(key, pack(obj))
|
||||||
|
return obj
|
||||||
|
finally:
|
||||||
|
self._locks.pop(ks, None)
|
||||||
|
|
||||||
async def invalidate(self, category: str, *, market: int | None = None,
|
async def invalidate(self, category: str, *, market: int | None = None,
|
||||||
code: str | None = None) -> int:
|
code: str | None = None) -> int:
|
||||||
"""按前缀批量失效,返回失效条数。"""
|
"""按前缀批量失效,返回失效条数。"""
|
||||||
|
|||||||
+63
-13
@@ -38,6 +38,7 @@ from .api import TdxData
|
|||||||
from dataclasses import asdict
|
from dataclasses import asdict
|
||||||
|
|
||||||
from .models import Adjust, ExMarket, KlinePeriod, Market
|
from .models import Adjust, ExMarket, KlinePeriod, Market
|
||||||
|
from .protocol.commands.quotes import price_decimal_digits
|
||||||
|
|
||||||
PERIODS = {
|
PERIODS = {
|
||||||
"min1": KlinePeriod.MIN_1, "min5": KlinePeriod.MIN_5, "min15": KlinePeriod.MIN_15,
|
"min1": KlinePeriod.MIN_1, "min5": KlinePeriod.MIN_5, "min15": KlinePeriod.MIN_15,
|
||||||
@@ -107,6 +108,40 @@ def _pct(price: float, pre: float) -> str:
|
|||||||
return "%+.2f%%" % ((price / pre - 1) * 100)
|
return "%+.2f%%" % ((price / pre - 1) * 100)
|
||||||
|
|
||||||
|
|
||||||
|
def _price_fmt(dp: int) -> str:
|
||||||
|
"""按品种小数位生成右对齐格式串。
|
||||||
|
|
||||||
|
股票 2 位、**可转债/ETF 3 位**、债券/回购 4 位。固定写 ``%8.2f`` 会把
|
||||||
|
可转债的 128.891 打成 128.89(少一位有效报价)。
|
||||||
|
"""
|
||||||
|
dp = max(0, min(int(dp or 2), 4))
|
||||||
|
return "%%%d.%df" % (6 + dp, dp)
|
||||||
|
|
||||||
|
|
||||||
|
# 可转债后缀(解码 4 位,但交易所报价步长 0.001 元,第 4 位恒为 0)
|
||||||
|
_CB_PREFIX = {1: ("110", "111", "113", "118"), 0: ("123", "127", "128")}
|
||||||
|
|
||||||
|
|
||||||
|
def _display_dp(market: int, code: str) -> int:
|
||||||
|
"""报价**显示**小数位。
|
||||||
|
|
||||||
|
注意 ``price_decimal_digits`` 给的是**解码位数**(原始整数 / 10**dp),
|
||||||
|
可转债是 4 位;但真实报价是 0.001 元的步长,第 4 位恒为 0,
|
||||||
|
按 4 位打印只会多一个 ``0``(``128.8920``)。
|
||||||
|
"""
|
||||||
|
dp = price_decimal_digits(market, code)
|
||||||
|
if str(code).startswith(_CB_PREFIX.get(int(market), ())):
|
||||||
|
return 3
|
||||||
|
return dp
|
||||||
|
|
||||||
|
|
||||||
|
def _dp_of(sym) -> int:
|
||||||
|
"""Symbol -> 价格小数位(非 A 股/指数一律 2 位)。"""
|
||||||
|
if getattr(sym, "kind", "") == "cn":
|
||||||
|
return _display_dp(sym.market, sym.code)
|
||||||
|
return 2
|
||||||
|
|
||||||
|
|
||||||
# ------------------------------------------------------------------ #
|
# ------------------------------------------------------------------ #
|
||||||
# 各子命令
|
# 各子命令
|
||||||
# ------------------------------------------------------------------ #
|
# ------------------------------------------------------------------ #
|
||||||
@@ -120,24 +155,31 @@ async def cmd_quotes(td: TdxData, symbols: list[str]) -> None:
|
|||||||
def _print_quote(q) -> None:
|
def _print_quote(q) -> None:
|
||||||
pct = (q.price / q.pre_close - 1) * 100 if q.pre_close else 0
|
pct = (q.price / q.pre_close - 1) * 100 if q.pre_close else 0
|
||||||
mkt = {31: "港股", 74: "美股"}.get(q.market, "")
|
mkt = {31: "港股", 74: "美股"}.get(q.market, "")
|
||||||
|
if q.market in (0, 1):
|
||||||
|
dp = _display_dp(q.market, q.code)
|
||||||
|
else:
|
||||||
|
dp = getattr(q, "decimal_point", 2) or 2
|
||||||
print("=" * 58)
|
print("=" * 58)
|
||||||
label = "%s %s" % (mkt, q.code) if mkt else "%s %s" % (q.code, q.server_time or datetime.now().strftime("%H:%M:%S"))
|
label = "%s %s" % (mkt, q.code) if mkt else "%s %s" % (q.code, q.server_time or datetime.now().strftime("%H:%M:%S"))
|
||||||
print(" %s" % label)
|
print(" %s" % label)
|
||||||
print(" 最新 %-10.2f 涨跌 %+8.2f (%+.2f%%)" % (q.price, q.price - q.pre_close, pct))
|
print(" 最新 %-10.*f 涨跌 %+8.*f (%+.2f%%)" % (dp, q.price, dp, q.price - q.pre_close, pct))
|
||||||
print(" 今开 %-10.2f 最高 %.2f 最低 %.2f" % (q.open, q.high, q.low))
|
print(" 今开 %-10.*f 最高 %.*f 最低 %.*f" % (dp, q.open, dp, q.high, dp, q.low))
|
||||||
print(" 昨收 %-10.2f 成交量 %s 成交额 %s" % (q.pre_close, _fmt_num(q.vol), _fmt_num(q.amount)))
|
print(" 昨收 %-10.*f 成交量 %s 成交额 %s" % (dp, q.pre_close, _fmt_num(q.vol), _fmt_num(q.amount)))
|
||||||
if q.bid and q.ask:
|
if q.bid and q.ask:
|
||||||
print(" " + "-" * 56)
|
print(" " + "-" * 56)
|
||||||
|
pf = _price_fmt(dp)
|
||||||
for i in range(max(len(q.bid), len(q.ask))):
|
for i in range(max(len(q.bid), len(q.ask))):
|
||||||
b = q.bid[i] if i < len(q.bid) else (0, 0)
|
b = q.bid[i] if i < len(q.bid) else (0, 0)
|
||||||
a = q.ask[i] if i < len(q.ask) else (0, 0)
|
a = q.ask[i] if i < len(q.ask) else (0, 0)
|
||||||
print(" 买%-1d %8.2f %10s | 卖%-1d %8.2f %10s" % (
|
print((" 买%-1d " + pf + " %10s | 卖%-1d " + pf + " %10s") % (
|
||||||
i + 1, b[0], _fmt_num(b[1]), i + 1, a[0], _fmt_num(a[1])))
|
i + 1, b[0], _fmt_num(b[1]), i + 1, a[0], _fmt_num(a[1])))
|
||||||
print()
|
print()
|
||||||
|
|
||||||
|
|
||||||
async def cmd_kline(td: TdxData, sym: str, period: KlinePeriod, count: int,
|
async def cmd_kline(td: TdxData, sym: str, period: KlinePeriod, count: int,
|
||||||
start: int, adjust: Adjust) -> None:
|
start: int, adjust: Adjust) -> None:
|
||||||
|
s = parse_symbol(sym)
|
||||||
|
pf = _price_fmt(_dp_of(s))
|
||||||
bars = await td.get_bars(sym, period, start=start, count=count, adjust=adjust)
|
bars = await td.get_bars(sym, period, start=start, count=count, adjust=adjust)
|
||||||
print("%s K线 (%s, %d根):" % (sym, period.name, len(bars)))
|
print("%s K线 (%s, %d根):" % (sym, period.name, len(bars)))
|
||||||
print(" %-12s %8s %8s %8s %8s %12s" % ("日期", "开盘", "最高", "最低", "收盘", "成交量"))
|
print(" %-12s %8s %8s %8s %8s %12s" % ("日期", "开盘", "最高", "最低", "收盘", "成交量"))
|
||||||
@@ -145,30 +187,38 @@ async def cmd_kline(td: TdxData, sym: str, period: KlinePeriod, count: int,
|
|||||||
dt = "%04d-%02d-%02d" % (b.year, b.month, b.day)
|
dt = "%04d-%02d-%02d" % (b.year, b.month, b.day)
|
||||||
if b.hour:
|
if b.hour:
|
||||||
dt += " %02d:%02d" % (b.hour, b.minute)
|
dt += " %02d:%02d" % (b.hour, b.minute)
|
||||||
print(" %-12s %8.2f %8.2f %8.2f %8.2f %12s" % (
|
print((" %-12s " + " ".join([pf] * 4) + " %12s") % (
|
||||||
dt, b.open, b.high, b.low, b.close, _fmt_num(b.vol)))
|
dt, b.open, b.high, b.low, b.close, _fmt_num(b.vol)))
|
||||||
|
|
||||||
|
|
||||||
async def cmd_transactions(td: TdxData, sym: str, count: int, date: int | None) -> None:
|
async def cmd_transactions(td: TdxData, sym: str, count: int, date: int | None) -> None:
|
||||||
s = parse_symbol(sym)
|
s = parse_symbol(sym)
|
||||||
tx = await td.get_transactions(Market(s.market), s.code, date=date, count=count)
|
pf = _price_fmt(_dp_of(s))
|
||||||
|
tx = await td.get_ticks(sym, date=date, count=count)
|
||||||
print("%s 逐笔成交 (%d条):" % (sym, len(tx)))
|
print("%s 逐笔成交 (%d条):" % (sym, len(tx)))
|
||||||
print(" 时间 价格 成交量 方向")
|
print(" 时间 价格 成交量 方向")
|
||||||
for t in tx:
|
for t in tx:
|
||||||
bs = "买" if getattr(t, "bs_flag", 0) == 1 else "卖" if getattr(t, "bs_flag", 0) == 2 else "-"
|
bs = "买" if getattr(t, "bs_flag", 0) == 1 else "卖" if getattr(t, "bs_flag", 0) == 2 else "-"
|
||||||
tc = getattr(t, "trade_count", 0)
|
if not getattr(t, "trade_count_available", True):
|
||||||
print(" %-10s %8.2f %10s %s%s" % (t.time, t.price, _fmt_num(t.vol), bs,
|
tc = " (-)" # 指数/ETF/LOF:协议不提供笔数
|
||||||
" (%d笔)" % tc if tc else ""))
|
elif getattr(t, "trade_count", 0):
|
||||||
|
tc = " (%d笔)" % t.trade_count
|
||||||
|
else:
|
||||||
|
tc = ""
|
||||||
|
print((" %-10s " + pf + " %10s %s%s") % (
|
||||||
|
t.time, t.price, _fmt_num(t.vol), bs, tc))
|
||||||
|
|
||||||
|
|
||||||
async def cmd_minute(td: TdxData, sym: str, date: int | None) -> None:
|
async def cmd_minute(td: TdxData, sym: str, date: int | None) -> None:
|
||||||
s = parse_symbol(sym)
|
s = parse_symbol(sym)
|
||||||
bars = await td.get_minute(Market(s.market), s.code, date=date)
|
pf = _price_fmt(_dp_of(s))
|
||||||
|
bars = await td.get_minutes(sym, date=date)
|
||||||
print("%s 分时 (%d条):" % (sym, len(bars)))
|
print("%s 分时 (%d条):" % (sym, len(bars)))
|
||||||
print(" 时间 价格 均价 成交量")
|
print(" 时间 价格 均价 成交量")
|
||||||
for i, b in enumerate(bars):
|
for b in bars:
|
||||||
print(" %02d:%02d %8.2f %8.2f %10s" % (
|
# 用 bar 自带时间:早期按序号推算会打成 00:00~03:59
|
||||||
i // 60, i % 60, b.price, b.avg_price, _fmt_num(b.vol)))
|
print((" %02d:%02d " + pf + " " + pf + " %10s") % (
|
||||||
|
b.hour, b.minute, b.price, b.avg_price, _fmt_num(b.vol)))
|
||||||
|
|
||||||
|
|
||||||
async def cmd_board(td: TdxData, args) -> None:
|
async def cmd_board(td: TdxData, args) -> None:
|
||||||
|
|||||||
@@ -33,6 +33,7 @@ _SERVERS_FILE = Path(__file__).resolve().parent / "servers.json" # 包内清单
|
|||||||
# 当日测速缓存(结构变更时递增,旧版本缓存自动失效)
|
# 当日测速缓存(结构变更时递增,旧版本缓存自动失效)
|
||||||
PROBE_CACHE_VERSION = 1
|
PROBE_CACHE_VERSION = 1
|
||||||
_PROBE_CACHE_NAME = "hosts-today.json"
|
_PROBE_CACHE_NAME = "hosts-today.json"
|
||||||
|
_DECIMAL_CACHE_NAME = "decimal-points.json"
|
||||||
|
|
||||||
|
|
||||||
# ---------- 包内清单(只读基线) ----------
|
# ---------- 包内清单(只读基线) ----------
|
||||||
@@ -97,6 +98,18 @@ def probe_cache_file() -> Path:
|
|||||||
return probe_cache_dir() / _PROBE_CACHE_NAME
|
return probe_cache_dir() / _PROBE_CACHE_NAME
|
||||||
|
|
||||||
|
|
||||||
|
def decimal_cache_file() -> Path:
|
||||||
|
"""权威价格小数位表缓存路径(每次调用重新解析,便于测试改环境变量后生效)。"""
|
||||||
|
return probe_cache_dir() / _DECIMAL_CACHE_NAME
|
||||||
|
|
||||||
|
|
||||||
|
def decimal_sync_disabled() -> bool:
|
||||||
|
"""是否已通过 ``PYPYTDX_DATA_DISABLE_DECIMAL_SYNC`` 关闭小数位后台同步。"""
|
||||||
|
return os.environ.get("PYPYTDX_DATA_DISABLE_DECIMAL_SYNC", "").strip().lower() in {
|
||||||
|
"1", "true", "yes", "on",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
def probe_disabled() -> bool:
|
def probe_disabled() -> bool:
|
||||||
"""是否已通过 ``PYPYTDX_DATA_DISABLE_PROBE`` 关闭当日测速缓存。"""
|
"""是否已通过 ``PYPYTDX_DATA_DISABLE_PROBE`` 关闭当日测速缓存。"""
|
||||||
return os.environ.get("PYPYTDX_DATA_DISABLE_PROBE", "").strip().lower() in {
|
return os.environ.get("PYPYTDX_DATA_DISABLE_PROBE", "").strip().lower() in {
|
||||||
|
|||||||
@@ -0,0 +1,223 @@
|
|||||||
|
"""权威价格小数位表的同步与缓存(对用户无感)。
|
||||||
|
|
||||||
|
为什么需要它
|
||||||
|
------------
|
||||||
|
``get_quotes`` 解析报价时,协议给的是**整数**,真实价格 = ``整数 / 10**小数位``。
|
||||||
|
小数位在协议里**不是**随报价一起下发的,只能靠推断(见
|
||||||
|
:func:`pytdxdata.protocol.commands.quotes.price_decimal_digits`)。
|
||||||
|
|
||||||
|
但证券清单接口(``get_universe``,底层 ``GetSecurityListCmd``)**自带**这个字段:
|
||||||
|
``SecurityInfo.decimal_point``,同时还带 ``pre_close``。这就是权威来源。本模块把它
|
||||||
|
灌进 ``price_decimal_digits`` 使用的运行时表,使价格标度不再依赖硬编码前缀规则。
|
||||||
|
|
||||||
|
行为
|
||||||
|
----
|
||||||
|
- **当日首次启动**检查缓存文件(``decimal-points.json``):命中就直接装表(纯内存,
|
||||||
|
无网络、约 10 ms);未命中则挂一个**后台 task** 拉证券清单并写缓存,
|
||||||
|
当天不再重复。
|
||||||
|
- 后台 task 不阻塞 ``start()``,不打印、不抛异常(取消除外),失败只落 debug 日志。
|
||||||
|
- 未同步成功时 ``price_decimal_digits`` 静默退回静态前缀规则,行为与 0.6.5 一致。
|
||||||
|
- 证券清单本身已被 ``cache/`` 层缓存,因此同步的实际网络开销只在**首次**
|
||||||
|
(约 26 + 25 个分页请求)。
|
||||||
|
|
||||||
|
开关
|
||||||
|
----
|
||||||
|
- ``PYPYTDX_DATA_DISABLE_DECIMAL_SYNC=1`` 关闭后台同步(CI / 测试用,保证确定性)。
|
||||||
|
- ``PYPYTDX_DATA_CACHE_DIR`` 覆盖缓存目录(测试指向临时目录)。
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import json
|
||||||
|
import logging
|
||||||
|
import os
|
||||||
|
from datetime import date
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import TYPE_CHECKING, Any
|
||||||
|
|
||||||
|
from .config import decimal_cache_file, decimal_sync_disabled
|
||||||
|
from .protocol.commands.quotes import (
|
||||||
|
clear_decimal_map,
|
||||||
|
decimal_map_source,
|
||||||
|
decimal_map_stats,
|
||||||
|
install_decimal_map,
|
||||||
|
)
|
||||||
|
|
||||||
|
if TYPE_CHECKING: # pragma: no cover
|
||||||
|
from .api import TdxData
|
||||||
|
|
||||||
|
log = logging.getLogger(__name__)
|
||||||
|
|
||||||
|
#: 同步哪些市场(只对 A 股有意义:扩展市场报价本身不含标度歧义)
|
||||||
|
DEFAULT_MARKETS: tuple[str, ...] = ("sh", "sz")
|
||||||
|
|
||||||
|
_CN_NAME = {"sh": 1, "sz": 0}
|
||||||
|
|
||||||
|
_state: dict[str, Any] = {"loaded": "", "task": None}
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------- #
|
||||||
|
# 磁盘缓存
|
||||||
|
# ---------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
def cache_file() -> Path:
|
||||||
|
"""缓存文件路径(每次重新解析,便于测试改环境变量)。"""
|
||||||
|
return decimal_cache_file()
|
||||||
|
|
||||||
|
|
||||||
|
def _read_cache(path: Path | None = None) -> dict[str, dict[str, int]] | None:
|
||||||
|
"""读缓存 → ``{"1": {code: dp}}``;缺失 / 损坏 → ``None``。"""
|
||||||
|
target = Path(path) if path is not None else cache_file()
|
||||||
|
try:
|
||||||
|
raw = json.loads(target.read_text(encoding="utf-8"))
|
||||||
|
except (OSError, ValueError):
|
||||||
|
return None
|
||||||
|
if not isinstance(raw, dict):
|
||||||
|
return None
|
||||||
|
if raw.get("date") != date.today().isoformat():
|
||||||
|
return None
|
||||||
|
markets = raw.get("markets")
|
||||||
|
if not isinstance(markets, dict) or not markets:
|
||||||
|
return None
|
||||||
|
out: dict[str, dict[str, int]] = {}
|
||||||
|
for k, v in markets.items():
|
||||||
|
if not isinstance(v, dict):
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
out[str(int(k))] = {str(c): int(d) for c, d in v.items() if int(d) > 0}
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
continue
|
||||||
|
return out or None
|
||||||
|
|
||||||
|
|
||||||
|
def load_cached(path: Path | None = None) -> int:
|
||||||
|
"""把当日缓存装进权威表;成功返回覆盖条目数,否则 0。"""
|
||||||
|
if decimal_sync_disabled():
|
||||||
|
return 0
|
||||||
|
markets = _read_cache(path)
|
||||||
|
if not markets:
|
||||||
|
return 0
|
||||||
|
items = [(int(m), c, d) for m, codes in markets.items() for c, d in codes.items()]
|
||||||
|
stats = install_decimal_map(items, source="cache")
|
||||||
|
_state["loaded"] = date.today().isoformat()
|
||||||
|
log.debug("小数位表已从缓存加载: %s", stats)
|
||||||
|
return int(stats["stock"])
|
||||||
|
|
||||||
|
|
||||||
|
def _write_cache(markets: dict[str, dict[str, int]], path: Path | None = None) -> None:
|
||||||
|
"""原子写缓存(tmp + ``os.replace``),失败只落 debug 日志。"""
|
||||||
|
target = Path(path) if path is not None else cache_file()
|
||||||
|
payload = {"date": date.today().isoformat(), "markets": markets}
|
||||||
|
tmp = target.with_suffix(".tmp")
|
||||||
|
try:
|
||||||
|
target.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
tmp.write_text(
|
||||||
|
json.dumps(payload, ensure_ascii=False, separators=(",", ":")),
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
os.replace(tmp, target)
|
||||||
|
except OSError as e: # pragma: no cover - 磁盘不可写等
|
||||||
|
log.debug("小数位表缓存写入失败: %s", e)
|
||||||
|
try:
|
||||||
|
tmp.unlink(missing_ok=True)
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------- #
|
||||||
|
# 同步
|
||||||
|
# ---------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
async def refresh(td: "TdxData", markets: tuple[str, ...] = DEFAULT_MARKETS,
|
||||||
|
*, path: Path | None = None) -> dict[str, int]:
|
||||||
|
"""拉证券清单 → 装表 + 写缓存。返回统计。
|
||||||
|
|
||||||
|
证券清单走 ``get_universe``(含 ``cache/`` 层缓存),只有首次真正发请求。
|
||||||
|
"""
|
||||||
|
tables: dict[str, dict[str, int]] = {}
|
||||||
|
items: list[tuple[int, str, int]] = []
|
||||||
|
for name in markets:
|
||||||
|
m = _CN_NAME.get(name)
|
||||||
|
if m is None:
|
||||||
|
continue
|
||||||
|
rows = await td.get_universe(name)
|
||||||
|
table: dict[str, int] = {}
|
||||||
|
for it in rows:
|
||||||
|
dp = int(getattr(it, "decimal_point", 0) or 0)
|
||||||
|
code = str(getattr(it, "code", "") or "").strip().rstrip("\x00")
|
||||||
|
if dp <= 0 or not code:
|
||||||
|
continue
|
||||||
|
table[code] = dp
|
||||||
|
items.append((m, code, dp))
|
||||||
|
if table:
|
||||||
|
tables[str(m)] = table
|
||||||
|
|
||||||
|
stats = install_decimal_map(items, source="universe")
|
||||||
|
_state["loaded"] = date.today().isoformat()
|
||||||
|
if tables:
|
||||||
|
_write_cache(tables, path)
|
||||||
|
log.debug("小数位表同步完成: %s", stats)
|
||||||
|
return stats
|
||||||
|
|
||||||
|
|
||||||
|
async def _run(td: "TdxData") -> dict[str, int]:
|
||||||
|
try:
|
||||||
|
return await refresh(td)
|
||||||
|
except asyncio.CancelledError:
|
||||||
|
raise
|
||||||
|
except Exception as e: # pragma: no cover - 网络/解析异常都不该影响用户
|
||||||
|
log.debug("小数位表后台同步失败: %s", e)
|
||||||
|
return {}
|
||||||
|
|
||||||
|
|
||||||
|
def schedule(td: "TdxData") -> asyncio.Task | None:
|
||||||
|
"""[内部] 需要时把同步挂到后台并返回 Task(调用方持有 / 取消)。
|
||||||
|
|
||||||
|
- 已关闭(``PYPYTDX_DATA_DISABLE_DECIMAL_SYNC=1``)→ ``None``
|
||||||
|
- 当日缓存命中 → 立即装表(同步、无网络),返回 ``None``
|
||||||
|
- 否则创建后台 task;进程内单例,重复调用复用同一个 task
|
||||||
|
"""
|
||||||
|
if decimal_sync_disabled():
|
||||||
|
return None
|
||||||
|
if decimal_map_stats()["stock"] and _state["loaded"] == date.today().isoformat():
|
||||||
|
return None
|
||||||
|
if load_cached():
|
||||||
|
return None
|
||||||
|
task = _state.get("task")
|
||||||
|
if task is not None and not task.done():
|
||||||
|
return task
|
||||||
|
task = asyncio.get_running_loop().create_task(_run(td), name="pytdxdata-decimal-sync")
|
||||||
|
_state["task"] = task
|
||||||
|
return task
|
||||||
|
|
||||||
|
|
||||||
|
async def cancel(task: asyncio.Task | None) -> None:
|
||||||
|
"""取消并等待后台同步 task(供 ``TdxData.close()`` 调用)。"""
|
||||||
|
if task is None or task.done():
|
||||||
|
return
|
||||||
|
task.cancel()
|
||||||
|
try:
|
||||||
|
await task
|
||||||
|
except (asyncio.CancelledError, Exception): # noqa: BLE001 - 关闭路径不抛
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def status() -> dict[str, Any]:
|
||||||
|
"""当前状态快照(排障 / 测试用)。"""
|
||||||
|
task = _state.get("task")
|
||||||
|
return {
|
||||||
|
"loaded": _state["loaded"],
|
||||||
|
"source": decimal_map_source(),
|
||||||
|
"syncing": bool(task is not None and not task.done()),
|
||||||
|
**decimal_map_stats(),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def reset() -> None:
|
||||||
|
"""清空内存状态(测试用;不动磁盘缓存)。"""
|
||||||
|
task = _state.get("task")
|
||||||
|
if task is not None and not task.done():
|
||||||
|
task.cancel()
|
||||||
|
_state["task"] = None
|
||||||
|
_state["loaded"] = ""
|
||||||
|
clear_decimal_map()
|
||||||
@@ -117,8 +117,8 @@ class SecurityBar:
|
|||||||
high: float
|
high: float
|
||||||
low: float
|
low: float
|
||||||
close: float
|
close: float
|
||||||
vol: float # 股票=股 / 转债=手(与vipdoc一致)
|
vol: float # 股票=股 / 转债=手(1 手=10 张,与 vipdoc 一致)
|
||||||
amount: float # 元
|
amount: float # 元。⚠️ amount/vol 只在股票上等于均价;转债还要再 /10(手=10张)
|
||||||
year: int
|
year: int
|
||||||
month: int
|
month: int
|
||||||
day: int
|
day: int
|
||||||
@@ -247,17 +247,42 @@ class TransactionRecord:
|
|||||||
- ``trade_count`` **不是笔数**,而是**当时均价 × 1000**(实测 tc/1000 与分时
|
- ``trade_count`` **不是笔数**,而是**当时均价 × 1000**(实测 tc/1000 与分时
|
||||||
``avg_price`` 的差 < 1.1)。指数没有「成交笔数」概念。
|
``avg_price`` 的差 < 1.1)。指数没有「成交笔数」概念。
|
||||||
- ``bs_flag`` 语义不成立(实测返回 2,而模型只定义 0=卖 / 1=买)。
|
- ``bs_flag`` 语义不成立(实测返回 2,而模型只定义 0=卖 / 1=买)。
|
||||||
|
|
||||||
|
**场内基金同样没有笔数**(ETF / 多数 LOF,见
|
||||||
|
:func:`~pytdxdata.symbols.has_order_count`):
|
||||||
|
|
||||||
|
- ``trade_count`` 是 ``价格 × 1000``(实测 ``sh510310`` 收盘 tc=3994,
|
||||||
|
当时价 3.989 → 3989,偏差 σ≈2;货币 ETF ``sh511990`` 恒为 0)。
|
||||||
|
- ``vol`` 仍是**手**,``bs_flag`` 正常。
|
||||||
|
- 这类标的的 :attr:`trade_count_available` 为 ``False``,``trade_count`` 已被
|
||||||
|
置 ``0``,不要拿去求和/求均值。
|
||||||
|
|
||||||
|
``trade_count_available=False`` 表示该品种**协议上就不带笔数**,
|
||||||
|
与「本切片恰好 0 笔」不是一回事。
|
||||||
|
|
||||||
|
**可转债(110/111/113/118/123/127/128/132)三样都是真的**(实测 2026-10-22,
|
||||||
|
16 只 A股可转债当日 + 8 只历史日全部自洽):
|
||||||
|
|
||||||
|
- ``vol`` 单位是**手**(1 手 = 10 张),与股票逐笔一致;``trade_count`` 是**真实成交笔数**
|
||||||
|
(``sh111000`` 3631 条切片 / 20443 笔,可转债在
|
||||||
|
:func:`~pytdxdata.symbols.has_order_count` 里判为**可用**)。
|
||||||
|
- ``Σvol`` 与**日线** ``SecurityBar.vol`` **1:1 相等**(比值 1.000000)——
|
||||||
|
这点和股票相反:股票逐笔是「手」、日线是「股」,Σ比值是 **0.01**。
|
||||||
|
- ⚠️ 但**日线** ``amount / vol`` **不等于**均价,而是均价的 **10 倍**(``vol`` 的「手」= 10 张)。
|
||||||
|
要算可转债均价须用 ``amount / vol / 10``;股票则是 ``amount / vol`` 直接就对。
|
||||||
|
实测:``sz127025`` 逐笔 VWAP 105.9063 vs ``amount/vol`` 1059.0957,比值 10.0003。
|
||||||
"""
|
"""
|
||||||
|
|
||||||
market: int
|
market: int
|
||||||
code: str
|
code: str
|
||||||
time: str # HH:MM:SS(扩展市场为 HH:MM)
|
time: str # HH:MM:SS(扩展市场为 HH:MM)
|
||||||
price: float
|
price: float
|
||||||
vol: float # 股票=手;指数=百元(成交额/100),见类注释
|
vol: float # 股票/转债=手;指数=百元(成交额/100),见类注释
|
||||||
trade_count: int # 股票=笔数增量;指数=均价×1000,见类注释
|
trade_count: int # 股票=笔数增量;指数/基金=占位值,见类注释
|
||||||
bs_flag: int # 1=主动买 0=主动卖 2=中性 5=盘后固定价(指数语义不成立)
|
bs_flag: int # 1=主动买 0=主动卖 2=中性 5=盘后固定价(指数语义不成立)
|
||||||
symbol: str = ""
|
symbol: str = ""
|
||||||
fields: dict[str, float] = field(default_factory=dict) # 扩展市场额外字段(增仓等)
|
fields: dict[str, float] = field(default_factory=dict) # 扩展市场额外字段(增仓等)
|
||||||
|
trade_count_available: bool = True # 指数/ETF/LOF 为 False(协议无笔数字段)
|
||||||
|
|
||||||
|
|
||||||
@dataclass(slots=True)
|
@dataclass(slots=True)
|
||||||
@@ -268,7 +293,7 @@ class MinuteBar:
|
|||||||
:attr:`datetime` / :attr:`trading_date`,不要自己拿 ``hour`` 拼日期。
|
:attr:`datetime` / :attr:`trading_date`,不要自己拿 ``hour`` 拼日期。
|
||||||
"""
|
"""
|
||||||
price: float
|
price: float
|
||||||
vol: float
|
vol: float # 手(股票/转债同);Σ分时vol 与当日日线 vol 同口径
|
||||||
avg_price: float = 0.0 # 均价(协议第2字段)
|
avg_price: float = 0.0 # 均价(协议第2字段)
|
||||||
hour: int = 0
|
hour: int = 0
|
||||||
minute: int = 0
|
minute: int = 0
|
||||||
|
|||||||
@@ -3,6 +3,7 @@ from __future__ import annotations
|
|||||||
|
|
||||||
import asyncio
|
import asyncio
|
||||||
import time
|
import time
|
||||||
|
from collections.abc import Sequence
|
||||||
from typing import Generic, TypeVar
|
from typing import Generic, TypeVar
|
||||||
|
|
||||||
from ..protocol.commands.base import BaseCommand
|
from ..protocol.commands.base import BaseCommand
|
||||||
@@ -22,6 +23,7 @@ class TdxConnection(Generic[T]):
|
|||||||
self._reader: asyncio.StreamReader | None = None
|
self._reader: asyncio.StreamReader | None = None
|
||||||
self._writer: asyncio.StreamWriter | None = None
|
self._writer: asyncio.StreamWriter | None = None
|
||||||
self._last_result: T | None = None
|
self._last_result: T | None = None
|
||||||
|
self._pipeline_result: list[T] = []
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def alive(self) -> bool:
|
def alive(self) -> bool:
|
||||||
@@ -107,6 +109,58 @@ class TdxConnection(Generic[T]):
|
|||||||
body = decompress_body(header, raw_body)
|
body = decompress_body(header, raw_body)
|
||||||
self._last_result: T = cmd.parse_response(body)
|
self._last_result: T = cmd.parse_response(body)
|
||||||
|
|
||||||
|
async def execute_pipeline(
|
||||||
|
self, cmds: Sequence[BaseCommand[T]], *, timeout: float | None = None
|
||||||
|
) -> list[T]:
|
||||||
|
"""单连接**流水线**:一次性发出全部请求,再按序读取全部响应。
|
||||||
|
|
||||||
|
通达信协议是自描述长度帧(16 字节帧头 + body),且服务器**按序**应答,
|
||||||
|
因此可以在同一 TCP 连接上连续下发多条请求、省掉 ``(N-1)`` 次往返。
|
||||||
|
实测 66 个「80 只报价」请求:逐条 7,372ms → 流水线 526ms(14 倍)。
|
||||||
|
|
||||||
|
与 :meth:`execute` 的关键差异:``execute`` 逐条严格配对,失败可小范围重试;
|
||||||
|
流水线一旦中途失败,**响应流已错配**,剩余响应无法再与请求对齐,
|
||||||
|
故此处强制断开连接(上层应换连接重试)。
|
||||||
|
|
||||||
|
参数:
|
||||||
|
cmds: 同一连接的请求序列(通常为同一类命令的不同分片)。
|
||||||
|
timeout: 整体超时(默认 ``self.timeout * 2``,流水线耗时随条数增长)。
|
||||||
|
|
||||||
|
返回:
|
||||||
|
与 ``cmds`` **等长且同序**的结果列表。
|
||||||
|
"""
|
||||||
|
if not cmds:
|
||||||
|
return []
|
||||||
|
if not self.alive:
|
||||||
|
raise ConnectionError(f"连接未建立或已关闭: {self.host}")
|
||||||
|
assert self._reader is not None and self._writer is not None
|
||||||
|
budget = timeout if timeout is not None else self.timeout * 2
|
||||||
|
try:
|
||||||
|
try:
|
||||||
|
await asyncio.wait_for(self._io_pipeline(cmds), budget)
|
||||||
|
except asyncio.TimeoutError:
|
||||||
|
self._force_close()
|
||||||
|
raise ConnectionError(
|
||||||
|
f"流水线超时 {self.host} ({budget:.1f}s, {len(cmds)} 条)") from None
|
||||||
|
except (ConnectionError, OSError) as e:
|
||||||
|
self._force_close()
|
||||||
|
raise ConnectionError(f"流水线错误 {self.host}: {e}") from e
|
||||||
|
except (ConnectionError, OSError):
|
||||||
|
raise
|
||||||
|
return self._pipeline_result
|
||||||
|
|
||||||
|
async def _io_pipeline(self, cmds: Sequence[BaseCommand[T]]) -> None:
|
||||||
|
assert self._reader is not None and self._writer is not None
|
||||||
|
self._writer.write(b"".join(c.build_request() for c in cmds))
|
||||||
|
await self._writer.drain()
|
||||||
|
results: list[T] = []
|
||||||
|
for cmd in cmds:
|
||||||
|
header_buf = await self._recv_exact(HEADER_SIZE)
|
||||||
|
header = parse_header(header_buf)
|
||||||
|
raw_body = await self._recv_exact(header.zipsize)
|
||||||
|
results.append(cmd.parse_response(decompress_body(header, raw_body)))
|
||||||
|
self._pipeline_result: list[T] = results
|
||||||
|
|
||||||
async def _send_setup(self) -> None:
|
async def _send_setup(self) -> None:
|
||||||
"""按序发送3条握手命令并丢弃响应。"""
|
"""按序发送3条握手命令并丢弃响应。"""
|
||||||
assert self._reader is not None and self._writer is not None
|
assert self._reader is not None and self._writer is not None
|
||||||
|
|||||||
@@ -59,6 +59,11 @@ class PooledConnection:
|
|||||||
self.last_used = time.monotonic()
|
self.last_used = time.monotonic()
|
||||||
return await self.conn.execute(cmd)
|
return await self.conn.execute(cmd)
|
||||||
|
|
||||||
|
async def execute_pipeline(self, cmds, *, timeout: float | None = None):
|
||||||
|
"""在同一条连接上流水线执行多条命令(见 TdxConnection.execute_pipeline)。"""
|
||||||
|
self.last_used = time.monotonic()
|
||||||
|
return await self.conn.execute_pipeline(cmds, timeout=timeout)
|
||||||
|
|
||||||
async def close(self) -> None:
|
async def close(self) -> None:
|
||||||
self.alive = False
|
self.alive = False
|
||||||
try:
|
try:
|
||||||
@@ -273,6 +278,46 @@ class DynamicPool:
|
|||||||
]
|
]
|
||||||
return random.choice(alive) if alive else None
|
return random.choice(alive) if alive else None
|
||||||
|
|
||||||
|
async def warm(self, n: int) -> None:
|
||||||
|
"""并发预建连接至 ``n`` 条(最多到 ``max_size``)。
|
||||||
|
|
||||||
|
:meth:`acquire` 建连时持有池锁,**一次只能建一条**——22 路并发首次
|
||||||
|
调用会排队 22×握手时间(实测全市场首拉多花 5~6s)。大批量请求前先调
|
||||||
|
本方法把连接并发建好,首拉只多付一次握手。
|
||||||
|
|
||||||
|
失败静默忽略:缺的连接后续由 ``acquire`` 按需补齐。
|
||||||
|
"""
|
||||||
|
if self._closed:
|
||||||
|
return
|
||||||
|
need = min(n, self.max_size) - self._total
|
||||||
|
if need <= 0:
|
||||||
|
return
|
||||||
|
|
||||||
|
# 用局部计数挑 host,避免并发建连时反复选中同一台
|
||||||
|
counts = dict(self._server_count)
|
||||||
|
ranked = self.health.rank(self.servers)
|
||||||
|
hosts: list[str] = []
|
||||||
|
for _ in range(need):
|
||||||
|
host = next((h for h in ranked
|
||||||
|
if not self.health.in_cooldown(h)
|
||||||
|
and counts.get(h, 0) < self.per_server_cap), None)
|
||||||
|
if host is None:
|
||||||
|
break
|
||||||
|
hosts.append(host)
|
||||||
|
counts[host] = counts.get(host, 0) + 1
|
||||||
|
if not hosts:
|
||||||
|
return
|
||||||
|
|
||||||
|
results = await asyncio.gather(
|
||||||
|
*(self._new_conn(h) for h in hosts), return_exceptions=True)
|
||||||
|
async with self._lock:
|
||||||
|
for pc in results:
|
||||||
|
if isinstance(pc, PooledConnection):
|
||||||
|
self._total += 1
|
||||||
|
self.stats.total = self._total
|
||||||
|
if not self._closed and pc.alive:
|
||||||
|
self._idle.append(pc)
|
||||||
|
|
||||||
async def release(self, pc: PooledConnection, *, broken: bool = False) -> None:
|
async def release(self, pc: PooledConnection, *, broken: bool = False) -> None:
|
||||||
if self.stats.borrowed > 0:
|
if self.stats.borrowed > 0:
|
||||||
self.stats.borrowed -= 1
|
self.stats.borrowed -= 1
|
||||||
|
|||||||
@@ -12,30 +12,93 @@ from .base import BaseCommand
|
|||||||
MAX_QUOTES = 80
|
MAX_QUOTES = 80
|
||||||
|
|
||||||
|
|
||||||
def price_decimal_digits(market: int, code: str) -> int:
|
# ====================================================================== #
|
||||||
"""推断报价的小数位:**协议原始整数除以 ``10**该值``** 才是真实价格。
|
# 权威小数位表(由证券清单 SecurityInfo.decimal_point 灌入)
|
||||||
|
# ====================================================================== #
|
||||||
|
# 查找顺序:**权威表 → 静态前缀规则**。权威表来自 ``get_universe`` 返回的
|
||||||
|
# ``SecurityInfo.decimal_point``(协议自带,见 protocol/commands/security_list.py),
|
||||||
|
# 由 ``pytdxdata.decimal_sync`` 在后台每日同步一次;未同步时静默退回静态规则,
|
||||||
|
# 行为与 0.6.5 完全一致。
|
||||||
|
_DECIMAL_PREFIX: dict[tuple[int, str], int] = {}
|
||||||
|
_DECIMAL_CODE: dict[tuple[int, str], int] = {}
|
||||||
|
_DECIMAL_MIXED: set[tuple[int, str]] = set()
|
||||||
|
_DECIMAL_SOURCE: str = ""
|
||||||
|
|
||||||
四档:股票/指数/板块 ``2``、基金/ETF/B股 ``3``、债券/回购 ``4``。
|
|
||||||
|
|
||||||
该函数只是**离线推断**(``get_quotes`` 解析时拿不到证券清单)。真正的权威来源
|
def install_decimal_map(
|
||||||
是证券列表里协议自带的 ``SecurityInfo.decimal_point`` 字段——2026-10-22 拉取
|
items: "list[tuple[int, str, int]] | tuple[tuple[int, str, int], ...]",
|
||||||
全量 51,982 只(沪 27,520 + 深 24,462)逐条比对,本函数与之一一吻合、**零例外**。
|
*,
|
||||||
能对上是因为「三位前缀 → 小数位」在全量清单里没有任何一个前缀出现混合小数位。
|
source: str = "",
|
||||||
|
) -> dict[str, int]:
|
||||||
|
"""用证券清单灌入权威小数位表。
|
||||||
|
|
||||||
历史修正(两次都是同一类错误:把某段前缀想当然地归错档):
|
``items`` 为 ``(market, code, decimal_point)`` 序列(直接来自
|
||||||
|
``SecurityInfo``)。按**三位代码前缀**归并成一个极小的表(全量 5 万只标的
|
||||||
|
只产生数百个前缀);若某个前缀出现**不同**小数位,则该前缀退化为逐代码查表,
|
||||||
|
只把这些例外代码存进 ``_DECIMAL_CODE``——这样即使将来交易所新增代码段,
|
||||||
|
也不会因为前缀规则过时而算错价格。
|
||||||
|
|
||||||
- **2026-10-21**:沪市 ``000``/``8`` 被判成 3 位 → 上证指数/上证50/科创50/
|
返回统计:``{"prefixes", "codes", "mixed_prefixes", "stock"}``。
|
||||||
板块指数**价格少 10 倍**(3713 读成 371)。它们实际是 2 位。
|
|
||||||
- **2026-10-22**:债券与回购全线偏差 —— 沪市 ``010/018/019/020/1xx/2xx``
|
|
||||||
与深市 ``10x/11x/12x/13x/14x/19x/50x/52x/56x`` 实际是 **4 位**,原先
|
|
||||||
沪市落到 2 位(**少 100 倍**)、深市落到 3 位(**少 10 倍**)。实测例:
|
|
||||||
``sh019742`` 110.228 读成 11022.8、``sh204001`` 1.385 读成 138.5、
|
|
||||||
``sz131810`` 1.38 读成 13.8。同时沪市 B 股 ``900xxx`` 是 3 位(原先 2 位)。
|
|
||||||
|
|
||||||
实证方法:拿 ``pre_close``(快照口径)除以上一交易日**日线收盘**(K线口径,
|
|
||||||
走 ``/1000`` 独立路径),比值即标度误差 —— 1.0 正确、10 少 10 倍、100 少 100 倍。
|
|
||||||
"""
|
"""
|
||||||
code = (code or "").strip().rstrip("\x00")
|
global _DECIMAL_SOURCE
|
||||||
|
votes: dict[tuple[int, str], dict[int, int]] = {}
|
||||||
|
per_code: dict[tuple[int, str], int] = {}
|
||||||
|
for market, code, dp in items:
|
||||||
|
c = str(code or "").strip().rstrip("\x00")
|
||||||
|
if len(c) < 3:
|
||||||
|
continue
|
||||||
|
m = int(market)
|
||||||
|
if m not in (0, 1):
|
||||||
|
continue
|
||||||
|
d = int(dp)
|
||||||
|
if d <= 0:
|
||||||
|
continue
|
||||||
|
per_code[(m, c)] = d
|
||||||
|
key = (m, c[:3])
|
||||||
|
bucket = votes.setdefault(key, {})
|
||||||
|
bucket[d] = bucket.get(d, 0) + 1
|
||||||
|
|
||||||
|
_DECIMAL_PREFIX.clear()
|
||||||
|
_DECIMAL_CODE.clear()
|
||||||
|
_DECIMAL_MIXED.clear()
|
||||||
|
for key, bucket in votes.items():
|
||||||
|
if len(bucket) == 1:
|
||||||
|
_DECIMAL_PREFIX[key] = next(iter(bucket))
|
||||||
|
else:
|
||||||
|
_DECIMAL_MIXED.add(key)
|
||||||
|
for (m, c), d in per_code.items():
|
||||||
|
if (m, c[:3]) in _DECIMAL_MIXED:
|
||||||
|
_DECIMAL_CODE[(m, c)] = d
|
||||||
|
_DECIMAL_SOURCE = source or "universe"
|
||||||
|
return decimal_map_stats()
|
||||||
|
|
||||||
|
|
||||||
|
def clear_decimal_map() -> None:
|
||||||
|
"""清空权威表(回到纯静态前缀规则;测试与排障用)。"""
|
||||||
|
global _DECIMAL_SOURCE
|
||||||
|
_DECIMAL_PREFIX.clear()
|
||||||
|
_DECIMAL_CODE.clear()
|
||||||
|
_DECIMAL_MIXED.clear()
|
||||||
|
_DECIMAL_SOURCE = ""
|
||||||
|
|
||||||
|
|
||||||
|
def decimal_map_stats() -> dict[str, int]:
|
||||||
|
"""权威表状态:已覆盖前缀数 / 例外代码数 / 混合前缀数 / 总条目数。"""
|
||||||
|
return {
|
||||||
|
"prefixes": len(_DECIMAL_PREFIX),
|
||||||
|
"codes": len(_DECIMAL_CODE),
|
||||||
|
"mixed_prefixes": len(_DECIMAL_MIXED),
|
||||||
|
"stock": len(_DECIMAL_PREFIX) + len(_DECIMAL_CODE),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def decimal_map_source() -> str:
|
||||||
|
"""权威表来源标记(``universe`` / ``cache`` / ``""``=未加载)。"""
|
||||||
|
return _DECIMAL_SOURCE
|
||||||
|
|
||||||
|
|
||||||
|
def _decimal_digits_static(market: int, code: str) -> int:
|
||||||
|
"""静态前缀规则(**兜底**,权威表缺失时使用)。"""
|
||||||
if market == 1: # 沪市
|
if market == 1: # 沪市
|
||||||
if code.startswith("5"):
|
if code.startswith("5"):
|
||||||
return 3 # 基金/ETF/LOF(50/51/52/56/58 系)
|
return 3 # 基金/ETF/LOF(50/51/52/56/58 系)
|
||||||
@@ -56,6 +119,48 @@ def price_decimal_digits(market: int, code: str) -> int:
|
|||||||
return 2
|
return 2
|
||||||
|
|
||||||
|
|
||||||
|
def price_decimal_digits(market: int, code: str) -> int:
|
||||||
|
"""报价小数位:**协议原始整数除以 ``10**该值``** 才是真实价格。
|
||||||
|
|
||||||
|
查找顺序:
|
||||||
|
|
||||||
|
1. **权威表**(:func:`install_decimal_map` 灌入)—— 来自证券清单里协议自带的
|
||||||
|
``SecurityInfo.decimal_point``,由 ``pytdxdata.decimal_sync`` 每日后台同步一次。
|
||||||
|
2. **静态前缀规则**(:func:`_decimal_digits_static`)—— 权威表未就绪时兜底。
|
||||||
|
四档:股票/指数/板块 ``2``、基金/ETF/B股 ``3``、债券/回购 ``4``。
|
||||||
|
|
||||||
|
为什么还要静态规则:``get_quotes`` 解析是**纯同步、无网络**的路径,解析那一刻
|
||||||
|
可能还没同步过证券清单;而现网首次安装/离线环境也必须能算出价格。
|
||||||
|
|
||||||
|
权威表与静态规则的一致性:2026-10-22 拉取全量 51,982 只(沪 27,520 + 深 24,462)
|
||||||
|
逐条比对,静态规则与 ``decimal_point`` **一一吻合、零例外**——因为全量清单里
|
||||||
|
没有任何一个三位前缀出现混合小数位。因此两者并存不会产生口径冲突。
|
||||||
|
|
||||||
|
历史修正(两次都是同一类错误:把某段前缀想当然地归错档):
|
||||||
|
|
||||||
|
- **2026-10-21**:沪市 ``000``/``8`` 被判成 3 位 → 上证指数/上证50/科创50/
|
||||||
|
板块指数**价格少 10 倍**(3713 读成 371)。它们实际是 2 位。
|
||||||
|
- **2026-10-22**:债券与回购全线偏差 —— 沪市 ``010/018/019/020/1xx/2xx``
|
||||||
|
与深市 ``10x/11x/12x/13x/14x/19x/50x/52x/56x`` 实际是 **4 位**,原先
|
||||||
|
沪市落到 2 位(**少 100 倍**)、深市落到 3 位(**少 10 倍**)。实测例:
|
||||||
|
``sh019742`` 110.228 读成 11022.8、``sh204001`` 1.385 读成 138.5、
|
||||||
|
``sz131810`` 1.38 读成 13.8。同时沪市 B 股 ``900xxx`` 是 3 位(原先 2 位)。
|
||||||
|
|
||||||
|
实证方法:拿 ``pre_close``(快照口径)除以上一交易日**日线收盘**(K线口径,
|
||||||
|
走 ``/1000`` 独立路径),比值即标度误差 —— 1.0 正确、10 少 10 倍、100 少 100 倍。
|
||||||
|
"""
|
||||||
|
code = (code or "").strip().rstrip("\x00")
|
||||||
|
m = int(market)
|
||||||
|
if m in (0, 1):
|
||||||
|
hit = _DECIMAL_PREFIX.get((m, code[:3]))
|
||||||
|
if hit is not None:
|
||||||
|
return hit
|
||||||
|
hit = _DECIMAL_CODE.get((m, code))
|
||||||
|
if hit is not None:
|
||||||
|
return hit
|
||||||
|
return _decimal_digits_static(m, code)
|
||||||
|
|
||||||
|
|
||||||
def format_server_time(raw: int) -> str:
|
def format_server_time(raw: int) -> str:
|
||||||
"""服务器时间编码:小时 + 百万分之一小时。14999212 -> 14:59:57.163"""
|
"""服务器时间编码:小时 + 百万分之一小时。14999212 -> 14:59:57.163"""
|
||||||
hours, fractional_hour = divmod(raw, 1_000_000)
|
hours, fractional_hour = divmod(raw, 1_000_000)
|
||||||
|
|||||||
@@ -36,8 +36,12 @@ class GetSecurityListCmd(BaseCommand[list[SecurityInfo]]):
|
|||||||
self.start = start
|
self.start = start
|
||||||
|
|
||||||
def build_request(self) -> bytes:
|
def build_request(self) -> bytes:
|
||||||
|
# 12 字节头 + market(H) + start(H) = **16 字节**。
|
||||||
|
# ⚠️ 曾经多塞一个 H(18 字节):服务端只消费前 16 字节,剩下的 00 00
|
||||||
|
# 留在它的接收缓冲里,会把**下一个请求**的开头吃掉 → 该连接的后续请求
|
||||||
|
# 永远得不到响应(表现为「第一次成功、第二次起全部通信超时」)。
|
||||||
header = bytes.fromhex("0c0118640101060006005004".replace(" ", ""))
|
header = bytes.fromhex("0c0118640101060006005004".replace(" ", ""))
|
||||||
return header + struct.pack("<HHH", self.market, self.start, 0)
|
return header + struct.pack("<HH", self.market, self.start)
|
||||||
|
|
||||||
def parse_response(self, body: bytes) -> list[SecurityInfo]:
|
def parse_response(self, body: bytes) -> list[SecurityInfo]:
|
||||||
(num,) = unpack_from("<H", body, 0, "security list header")
|
(num,) = unpack_from("<H", body, 0, "security list header")
|
||||||
|
|||||||
@@ -5,6 +5,7 @@ import struct
|
|||||||
|
|
||||||
from ..._binary import unpack_from
|
from ..._binary import unpack_from
|
||||||
from ...models import TransactionRecord
|
from ...models import TransactionRecord
|
||||||
|
from ...symbols import has_order_count
|
||||||
from ..datetime_ import get_time
|
from ..datetime_ import get_time
|
||||||
from ..mac_frame import build_mac_request
|
from ..mac_frame import build_mac_request
|
||||||
from ..price import get_price
|
from ..price import get_price
|
||||||
@@ -32,6 +33,8 @@ class GetTransactionDataCmd(BaseCommand[list[TransactionRecord]]):
|
|||||||
pos = 2
|
pos = 2
|
||||||
last_price = 0
|
last_price = 0
|
||||||
records: list[TransactionRecord] = []
|
records: list[TransactionRecord] = []
|
||||||
|
code = self.code.decode("utf-8")
|
||||||
|
has_tc = has_order_count(self.market, code)
|
||||||
for _ in range(num):
|
for _ in range(num):
|
||||||
hour, minute, pos = get_time(body, pos)
|
hour, minute, pos = get_time(body, pos)
|
||||||
price_diff, pos = get_price(body, pos)
|
price_diff, pos = get_price(body, pos)
|
||||||
@@ -41,10 +44,12 @@ class GetTransactionDataCmd(BaseCommand[list[TransactionRecord]]):
|
|||||||
_unknown, pos = get_price(body, pos)
|
_unknown, pos = get_price(body, pos)
|
||||||
last_price += price_diff
|
last_price += price_diff
|
||||||
records.append(TransactionRecord(
|
records.append(TransactionRecord(
|
||||||
market=self.market, code=self.code.decode("utf-8"),
|
market=self.market, code=code,
|
||||||
time=f"{hour:02d}:{minute:02d}:00",
|
time=f"{hour:02d}:{minute:02d}:00",
|
||||||
price=last_price / 100.0, vol=float(vol),
|
price=last_price / 100.0, vol=float(vol),
|
||||||
trade_count=int(num_orders), bs_flag=bs_flag,
|
# 指数/ETF/LOF 该字段是「价格×1000」占位,不是笔数
|
||||||
|
trade_count=int(num_orders) if has_tc else 0,
|
||||||
|
bs_flag=bs_flag, trade_count_available=has_tc,
|
||||||
))
|
))
|
||||||
return records
|
return records
|
||||||
|
|
||||||
@@ -68,6 +73,7 @@ class GetHistoryTransactionDataCmd(BaseCommand[list[TransactionRecord]]):
|
|||||||
pos = 6 # 2(num) + 4(skip)
|
pos = 6 # 2(num) + 4(skip)
|
||||||
last_price = 0
|
last_price = 0
|
||||||
records: list[TransactionRecord] = []
|
records: list[TransactionRecord] = []
|
||||||
|
code = self.code.decode("utf-8")
|
||||||
for _ in range(num):
|
for _ in range(num):
|
||||||
hour, minute, pos = get_time(body, pos)
|
hour, minute, pos = get_time(body, pos)
|
||||||
price_diff, pos = get_price(body, pos)
|
price_diff, pos = get_price(body, pos)
|
||||||
@@ -76,10 +82,11 @@ class GetHistoryTransactionDataCmd(BaseCommand[list[TransactionRecord]]):
|
|||||||
_unknown, pos = get_price(body, pos)
|
_unknown, pos = get_price(body, pos)
|
||||||
last_price += price_diff
|
last_price += price_diff
|
||||||
records.append(TransactionRecord(
|
records.append(TransactionRecord(
|
||||||
market=self.market, code=self.code.decode("utf-8"),
|
market=self.market, code=code,
|
||||||
time=f"{hour:02d}:{minute:02d}:00",
|
time=f"{hour:02d}:{minute:02d}:00",
|
||||||
price=last_price / 100.0, vol=float(vol),
|
price=last_price / 100.0, vol=float(vol),
|
||||||
trade_count=0, bs_flag=bs_flag, # 历史无笔数
|
trade_count=0, bs_flag=bs_flag,
|
||||||
|
trade_count_available=False, # 历史通道整条链路都没有笔数字段
|
||||||
))
|
))
|
||||||
return records
|
return records
|
||||||
|
|
||||||
@@ -108,6 +115,7 @@ class MacTransactionCmd(BaseCommand[list[TransactionRecord]]):
|
|||||||
def parse_response(self, body: bytes) -> list[TransactionRecord]:
|
def parse_response(self, body: bytes) -> list[TransactionRecord]:
|
||||||
(count,) = unpack_from("<H", body, 29, "mac transaction count")
|
(count,) = unpack_from("<H", body, 29, "mac transaction count")
|
||||||
records: list[TransactionRecord] = []
|
records: list[TransactionRecord] = []
|
||||||
|
has_tc = has_order_count(self.market, self.code)
|
||||||
for i in range(count):
|
for i in range(count):
|
||||||
offset = 39 + i * 18
|
offset = 39 + i * 18
|
||||||
try:
|
try:
|
||||||
@@ -124,6 +132,8 @@ class MacTransactionCmd(BaseCommand[list[TransactionRecord]]):
|
|||||||
market=self.market, code=self.code,
|
market=self.market, code=self.code,
|
||||||
time=f"{h:02d}:{m:02d}:{s:02d}",
|
time=f"{h:02d}:{m:02d}:{s:02d}",
|
||||||
price=price, vol=float(volume),
|
price=price, vol=float(volume),
|
||||||
trade_count=int(trade_count), bs_flag=bs_flag,
|
# 指数/ETF/LOF 该字段是「价格×1000」占位,不是笔数
|
||||||
|
trade_count=int(trade_count) if has_tc else 0,
|
||||||
|
bs_flag=bs_flag, trade_count_available=has_tc,
|
||||||
))
|
))
|
||||||
return records
|
return records
|
||||||
@@ -238,3 +238,53 @@ def is_cn_index(market: int, code: str) -> bool:
|
|||||||
or (code.startswith("000") and int(market) == int(Market.SH))
|
or (code.startswith("000") and int(market) == int(Market.SH))
|
||||||
or (code.startswith("399") and int(market) == int(Market.SZ))
|
or (code.startswith("399") and int(market) == int(Market.SZ))
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def has_order_count(market: int, code: str) -> bool:
|
||||||
|
"""该标的的逐笔 ``trade_count`` 是否是**真实成交笔数**。
|
||||||
|
|
||||||
|
通达信对**指数**和**多数场内基金**不提供笔数,但那 4 字节里仍塞了值:
|
||||||
|
|
||||||
|
- 指数:``当时均价 × 1000``(见 :class:`~pytdxdata.models.TransactionRecord`)
|
||||||
|
- ETF / LOF:``价格 × 1000``;货币 ETF(如 ``sh511990``)恒为 ``0``
|
||||||
|
|
||||||
|
直接当笔数用会得到 3994 这种荒谬值(实测 ``sh510310``)。
|
||||||
|
|
||||||
|
实测覆盖(2026-10-22,按代码前缀各取样 1~7 只):
|
||||||
|
|
||||||
|
================== ========================== ==========
|
||||||
|
品种 前缀 笔数可用
|
||||||
|
================== ========================== ==========
|
||||||
|
沪股票/科创 600/601/603/605/688/689 是
|
||||||
|
沪 LOF 501/502/506 是
|
||||||
|
沪 ETF/LOF 51/52/56/58 **否**
|
||||||
|
沪指数 000/88 否
|
||||||
|
沪可转债 110/111/113/118 是
|
||||||
|
沪 EB(可交换债) 132 是
|
||||||
|
沪逆回购 204 是
|
||||||
|
深股票 000/001/002/003/300/301 是
|
||||||
|
深 ETF 159 **否**
|
||||||
|
深 LOF/基金 15/16/17/18 **否**
|
||||||
|
深指数 399 否
|
||||||
|
深可转债 123/127/128 是
|
||||||
|
================== ========================== ==========
|
||||||
|
|
||||||
|
``501x/502x/506x``(沪市 LOF)实测**有**真实笔数,所以不能用「沪市 5 开头
|
||||||
|
一律当基金」的粗规则;必须按上面这组前缀精确判定。
|
||||||
|
|
||||||
|
可转债(沪 110/111/113/118/132、深 123/127/128)**走的是兜底分支**——
|
||||||
|
它们既不在沪市 ``51/52/56/58`` 里,也不在深市 ``159/15/16/17/18`` 里,
|
||||||
|
所以两个分支都落到「是」。这不是巧合而是设计:兜底按「**排除法**」认定
|
||||||
|
基金/指数,其余一律当普通品种。实测验证(2026-10-22,16 只当日 + 8 只历史日):
|
||||||
|
笔数合计与日线量级吻合(``sh111000`` 3631 条切片 / 20443 笔,
|
||||||
|
``sz128136`` 1939 条 / 14593 笔),且 ``Σvol`` 与日线 ``vol`` 比值 **1.000000**。
|
||||||
|
"""
|
||||||
|
code = str(code)
|
||||||
|
if is_cn_index(market, code):
|
||||||
|
return False
|
||||||
|
m = int(market)
|
||||||
|
if m == int(Market.SH):
|
||||||
|
return code[:2] not in ("51", "52", "56", "58")
|
||||||
|
if m == int(Market.SZ):
|
||||||
|
return not (code[:3] == "159" or code[:2] in ("15", "16", "17", "18"))
|
||||||
|
return True
|
||||||
@@ -0,0 +1,22 @@
|
|||||||
|
"""测试全局隔离。
|
||||||
|
|
||||||
|
默认关掉**权威小数位表的后台同步** —— 否则任何调用 ``TdxData.start()`` 的用例都会
|
||||||
|
顺手发起证券清单请求(26 + 25 个分页),测试会变慢且依赖网络。
|
||||||
|
需要验证同步行为本身的用例(``test_decimal_sync.py``)自己 ``delenv`` 后再测。
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import os
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True, scope="session")
|
||||||
|
def _disable_decimal_sync():
|
||||||
|
old = os.environ.get("PYPYTDX_DATA_DISABLE_DECIMAL_SYNC")
|
||||||
|
os.environ["PYPYTDX_DATA_DISABLE_DECIMAL_SYNC"] = "1"
|
||||||
|
yield
|
||||||
|
if old is None:
|
||||||
|
os.environ.pop("PYPYTDX_DATA_DISABLE_DECIMAL_SYNC", None)
|
||||||
|
else:
|
||||||
|
os.environ["PYPYTDX_DATA_DISABLE_DECIMAL_SYNC"] = old
|
||||||
@@ -324,6 +324,27 @@ async def test_universe_and_quotes(td):
|
|||||||
assert len(qex) == 1
|
assert len(qex) == 1
|
||||||
|
|
||||||
|
|
||||||
|
async def test_decimal_point_sync(td, tmp_path, monkeypatch):
|
||||||
|
"""权威小数位表同步:把证券清单的 decimal_point 灌进价格标度表。"""
|
||||||
|
for name in ("sync_decimal_points", "decimal_point_status"):
|
||||||
|
_use(name)
|
||||||
|
from pytdxdata.protocol.commands.quotes import clear_decimal_map, price_decimal_digits
|
||||||
|
|
||||||
|
# 缓存目录指向临时目录,避免把 mock 数据写进用户真实缓存
|
||||||
|
monkeypatch.setenv("PYPYTDX_DATA_CACHE_DIR", str(tmp_path))
|
||||||
|
before = price_decimal_digits(1, "600000")
|
||||||
|
try:
|
||||||
|
st = await td.sync_decimal_points()
|
||||||
|
assert set(st) >= {"prefixes", "codes", "mixed_prefixes", "stock"}
|
||||||
|
assert st["stock"] >= 0
|
||||||
|
status = await td.decimal_point_status()
|
||||||
|
assert isinstance(status, dict) and "source" in status
|
||||||
|
finally:
|
||||||
|
clear_decimal_map()
|
||||||
|
# 清空后回到静态规则,不受本用例污染
|
||||||
|
assert price_decimal_digits(1, "600000") == before
|
||||||
|
|
||||||
|
|
||||||
async def test_snapshot_auction_flow(td):
|
async def test_snapshot_auction_flow(td):
|
||||||
for name in ("get_snapshot", "get_auction", "get_capital_flow"):
|
for name in ("get_snapshot", "get_auction", "get_capital_flow"):
|
||||||
_use(name)
|
_use(name)
|
||||||
|
|||||||
@@ -0,0 +1,125 @@
|
|||||||
|
"""CLI 价格小数位 / 分时时间列的显示回归。
|
||||||
|
|
||||||
|
覆盖两个曾经出错的地方:
|
||||||
|
|
||||||
|
1. ``tdx minute`` 的时间列按序号推算(``i // 60, i % 60``),240 根分时被打印成
|
||||||
|
``00:00~03:59``,与真实交易时间无关。
|
||||||
|
2. 价格固定写死 ``%8.2f``,可转债(报价步长 0.001 元)会被截成 2 位
|
||||||
|
——``128.892`` 打成 ``128.89``。
|
||||||
|
|
||||||
|
注意 ``price_decimal_digits`` 返回的是**解码位数**(原始整数 / 10**dp),可转债
|
||||||
|
是 4 位,但第 4 位恒为 0,显示层只该给 3 位(见 ``_display_dp``)。
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pytdxdata.cli import (
|
||||||
|
_display_dp,
|
||||||
|
_dp_of,
|
||||||
|
_price_fmt,
|
||||||
|
cmd_minute,
|
||||||
|
cmd_transactions,
|
||||||
|
parse_symbol,
|
||||||
|
)
|
||||||
|
from pytdxdata.models import MinuteBar, TransactionRecord
|
||||||
|
|
||||||
|
|
||||||
|
class FakeTD:
|
||||||
|
"""只实现 minute / transactions 需要的两个接口。"""
|
||||||
|
|
||||||
|
def __init__(self, minutes=None, ticks=None):
|
||||||
|
self._minutes = minutes or []
|
||||||
|
self._ticks = ticks or []
|
||||||
|
|
||||||
|
async def get_minutes(self, sym, date=None):
|
||||||
|
return self._minutes
|
||||||
|
|
||||||
|
async def get_ticks(self, sym, date=None, count=None):
|
||||||
|
return self._ticks
|
||||||
|
|
||||||
|
|
||||||
|
def _bar(price, hour, minute, vol=100.0, avg=0.0):
|
||||||
|
return MinuteBar(price=price, vol=vol, avg_price=avg or price,
|
||||||
|
hour=hour, minute=minute, fields={})
|
||||||
|
|
||||||
|
|
||||||
|
def _tick(price, vol, trade_count=1, available=True, time="14:59:54"):
|
||||||
|
return TransactionRecord(market=1, code="113708", time=time, price=price, vol=vol,
|
||||||
|
trade_count=trade_count, bs_flag=0, fields={},
|
||||||
|
trade_count_available=available)
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# 小数位判定
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
def test_price_fmt_width_tracks_decimals():
|
||||||
|
assert _price_fmt(2) == "%8.2f" # 股票:保持原宽度,不破坏对齐
|
||||||
|
assert _price_fmt(3) == "%9.3f"
|
||||||
|
assert _price_fmt(4) == "%10.4f"
|
||||||
|
assert _price_fmt(None) == "%8.2f" # 缺省当 2 位
|
||||||
|
|
||||||
|
|
||||||
|
def test_display_dp_by_instrument():
|
||||||
|
assert _display_dp(1, "600000") == 2 # 沪股票
|
||||||
|
assert _display_dp(0, "000001") == 2 # 深股票
|
||||||
|
assert _display_dp(1, "000001") == 2 # 指数
|
||||||
|
assert _display_dp(1, "113708") == 3 # 沪可转债(解码 4 位,显示 3 位)
|
||||||
|
assert _display_dp(1, "110102") == 3
|
||||||
|
assert _display_dp(0, "123235") == 3 # 深可转债
|
||||||
|
assert _display_dp(0, "128136") == 3
|
||||||
|
assert _display_dp(1, "510310") == 3 # ETF
|
||||||
|
assert _display_dp(1, "019547") == 4 # 国债:4 位是真实报价精度
|
||||||
|
|
||||||
|
|
||||||
|
def test_dp_of_uses_cn_only():
|
||||||
|
assert _dp_of(parse_symbol("sh113708")) == 3
|
||||||
|
assert _dp_of(parse_symbol("sh600000")) == 2
|
||||||
|
assert _dp_of(parse_symbol("hk00700")) == 2 # 非 A 股固定 2 位
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# tdx minute
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
async def test_cmd_minute_prints_real_clock(capsys):
|
||||||
|
bars = [_bar(129.271, 9, 30), _bar(129.270, 9, 31), _bar(128.892, 14, 59)]
|
||||||
|
await cmd_minute(FakeTD(minutes=bars), "sh113708", None)
|
||||||
|
out = capsys.readouterr().out
|
||||||
|
assert "09:30" in out and "09:31" in out and "14:59" in out
|
||||||
|
assert "00:00" not in out and "03:59" not in out
|
||||||
|
|
||||||
|
|
||||||
|
async def test_cmd_minute_cb_keeps_three_decimals(capsys):
|
||||||
|
await cmd_minute(FakeTD(minutes=[_bar(128.892, 14, 59)]), "sh113708", None)
|
||||||
|
out = capsys.readouterr().out
|
||||||
|
assert "128.892" in out
|
||||||
|
assert "128.89 " not in out
|
||||||
|
|
||||||
|
|
||||||
|
async def test_cmd_minute_stock_two_decimals(capsys):
|
||||||
|
await cmd_minute(FakeTD(minutes=[_bar(9.62, 15, 0)]), "sh600000", None)
|
||||||
|
out = capsys.readouterr().out
|
||||||
|
assert "9.62" in out and "9.620" not in out
|
||||||
|
|
||||||
|
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
# tdx transactions
|
||||||
|
# --------------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
async def test_cmd_transactions_cb_price_and_count(capsys):
|
||||||
|
await cmd_transactions(FakeTD(ticks=[_tick(128.892, 3, trade_count=4)]),
|
||||||
|
"sh113708", 50, None)
|
||||||
|
out = capsys.readouterr().out
|
||||||
|
assert "128.892" in out
|
||||||
|
assert "(4笔)" in out
|
||||||
|
|
||||||
|
|
||||||
|
async def test_cmd_transactions_etf_marks_missing_count(capsys):
|
||||||
|
tick = TransactionRecord(market=1, code="510310", time="15:27:35", price=3.989,
|
||||||
|
vol=15, trade_count=0, bs_flag=0, fields={},
|
||||||
|
trade_count_available=False)
|
||||||
|
await cmd_transactions(FakeTD(ticks=[tick]), "sh510310", 50, None)
|
||||||
|
out = capsys.readouterr().out
|
||||||
|
assert "3.989" in out # ETF 也是 3 位
|
||||||
|
assert "(-)" in out # 不打印假笔数
|
||||||
|
assert "笔)" not in out
|
||||||
@@ -0,0 +1,233 @@
|
|||||||
|
"""权威价格小数位表(decimal_sync)单元测试。
|
||||||
|
|
||||||
|
覆盖点:
|
||||||
|
- 权威表(来自 ``SecurityInfo.decimal_point``)**优先于**静态前缀规则
|
||||||
|
- 某前缀出现混合小数位 → 自动退化为逐代码查表,不误伤同前缀其它代码
|
||||||
|
- 清空权威表 → 回到静态规则(与 0.6.5 行为一致)
|
||||||
|
- 磁盘缓存:写入 / 读取 / 跨日失效 / 损坏容错,均不回退到网络
|
||||||
|
- ``PYPYTDX_DATA_DISABLE_DECIMAL_SYNC=1`` → 完全不动作
|
||||||
|
- 证券清单里 ``decimal_point == 0`` 的脏条目被忽略
|
||||||
|
- 后台 task 可取消,且 ``close()`` 后不再改写表
|
||||||
|
(全部注入假 ``TdxData``,不连真实网络)
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import asyncio
|
||||||
|
import json
|
||||||
|
from types import SimpleNamespace
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from pytdxdata import decimal_sync
|
||||||
|
from pytdxdata.protocol.commands.quotes import (
|
||||||
|
clear_decimal_map,
|
||||||
|
decimal_map_stats,
|
||||||
|
install_decimal_map,
|
||||||
|
price_decimal_digits,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------- #
|
||||||
|
# 夹具
|
||||||
|
# ---------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
@pytest.fixture(autouse=True)
|
||||||
|
def _isolate(tmp_path, monkeypatch):
|
||||||
|
"""每个用例独立缓存目录 + 干净的内存状态。"""
|
||||||
|
monkeypatch.setenv("PYPYTDX_DATA_CACHE_DIR", str(tmp_path))
|
||||||
|
monkeypatch.delenv("PYPYTDX_DATA_DISABLE_DECIMAL_SYNC", raising=False)
|
||||||
|
decimal_sync.reset()
|
||||||
|
yield
|
||||||
|
decimal_sync.reset()
|
||||||
|
|
||||||
|
|
||||||
|
def _info(code: str, dp: int) -> SimpleNamespace:
|
||||||
|
return SimpleNamespace(code=code, decimal_point=dp, market=1)
|
||||||
|
|
||||||
|
|
||||||
|
class FakeTd:
|
||||||
|
"""只有 ``get_universe`` 的假客户端(记录调用次数,便于断言"没走网络")。"""
|
||||||
|
|
||||||
|
def __init__(self, tables: dict[str, list[tuple[str, int]]] | None = None) -> None:
|
||||||
|
self.tables = tables or {}
|
||||||
|
self.calls: list[str] = []
|
||||||
|
|
||||||
|
async def get_universe(self, market: str | None = None):
|
||||||
|
self.calls.append(market or "all")
|
||||||
|
return [_info(c, d) for c, d in self.tables.get(market, [])]
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------- #
|
||||||
|
# 权威表优先级
|
||||||
|
# ---------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
def test_authoritative_map_overrides_static_rule():
|
||||||
|
# 静态规则说 sh019742 是 4 位;权威表说 3 位 → 必须听权威表的
|
||||||
|
assert price_decimal_digits(1, "019742") == 4
|
||||||
|
install_decimal_map([(1, "019742", 3)], source="test")
|
||||||
|
assert price_decimal_digits(1, "019742") == 3
|
||||||
|
|
||||||
|
|
||||||
|
def test_codes_outside_map_still_use_static_rule():
|
||||||
|
install_decimal_map([(1, "019742", 3)], source="test")
|
||||||
|
assert price_decimal_digits(1, "600000") == 2 # 股票
|
||||||
|
assert price_decimal_digits(1, "510300") == 3 # ETF
|
||||||
|
assert price_decimal_digits(0, "131810") == 4 # 深回购(表里没有)
|
||||||
|
|
||||||
|
|
||||||
|
def test_mixed_prefix_falls_back_to_per_code_lookup():
|
||||||
|
"""同一前缀出现两种小数位 → 该前缀不进前缀表,只把例外代码存进逐代码表。"""
|
||||||
|
stats = install_decimal_map(
|
||||||
|
[(1, "110001", 4), (1, "110002", 4), (1, "110003", 3)], source="test")
|
||||||
|
assert stats["mixed_prefixes"] == 1
|
||||||
|
assert stats["codes"] == 3 # 混合前缀下所有代码都进逐代码表
|
||||||
|
assert price_decimal_digits(1, "110001") == 4
|
||||||
|
assert price_decimal_digits(1, "110003") == 3
|
||||||
|
|
||||||
|
|
||||||
|
def test_clear_map_restores_static_rules():
|
||||||
|
install_decimal_map([(1, "019742", 3)], source="test")
|
||||||
|
assert price_decimal_digits(1, "019742") == 3
|
||||||
|
clear_decimal_map()
|
||||||
|
assert decimal_map_stats()["stock"] == 0
|
||||||
|
assert price_decimal_digits(1, "019742") == 4
|
||||||
|
|
||||||
|
|
||||||
|
def test_reinstall_replaces_previous_map():
|
||||||
|
install_decimal_map([(1, "019742", 3)], source="a")
|
||||||
|
install_decimal_map([(1, "019742", 4)], source="b")
|
||||||
|
assert price_decimal_digits(1, "019742") == 4
|
||||||
|
assert decimal_map_stats()["prefixes"] == 1
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------- #
|
||||||
|
# 同步
|
||||||
|
# ---------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
def test_refresh_installs_and_writes_cache(tmp_path):
|
||||||
|
td = FakeTd({"sh": [("019742", 4), ("600000", 2)],
|
||||||
|
"sz": [("000001", 2), ("131810", 4)]})
|
||||||
|
stats = asyncio.run(decimal_sync.refresh(td))
|
||||||
|
|
||||||
|
assert td.calls == ["sh", "sz"]
|
||||||
|
assert stats["stock"] == 4
|
||||||
|
assert price_decimal_digits(1, "600000") == 2
|
||||||
|
assert price_decimal_digits(0, "131810") == 4
|
||||||
|
|
||||||
|
raw = json.loads((tmp_path / "decimal-points.json").read_text(encoding="utf-8"))
|
||||||
|
assert raw["markets"]["1"]["019742"] == 4
|
||||||
|
assert raw["markets"]["0"]["131810"] == 4
|
||||||
|
|
||||||
|
|
||||||
|
def test_refresh_skips_zero_decimal_point():
|
||||||
|
td = FakeTd({"sh": [("019742", 0), ("600000", 2)]})
|
||||||
|
stats = asyncio.run(decimal_sync.refresh(td, ("sh",)))
|
||||||
|
assert stats["stock"] == 1
|
||||||
|
|
||||||
|
|
||||||
|
def test_refresh_ignores_unknown_market():
|
||||||
|
td = FakeTd({"hk": [("00700", 3)]})
|
||||||
|
stats = asyncio.run(decimal_sync.refresh(td, ("hk",)))
|
||||||
|
assert stats["stock"] == 0
|
||||||
|
assert not (decimal_sync.cache_file()).exists()
|
||||||
|
|
||||||
|
|
||||||
|
def test_load_cached_roundtrip(tmp_path):
|
||||||
|
td = FakeTd({"sh": [("019742", 4)]})
|
||||||
|
asyncio.run(decimal_sync.refresh(td))
|
||||||
|
decimal_sync.reset()
|
||||||
|
assert decimal_map_stats()["stock"] == 0
|
||||||
|
|
||||||
|
assert decimal_sync.load_cached() == 1
|
||||||
|
assert price_decimal_digits(1, "019742") == 4
|
||||||
|
|
||||||
|
|
||||||
|
def test_cache_expires_across_days(tmp_path):
|
||||||
|
td = FakeTd({"sh": [("019742", 4)]})
|
||||||
|
asyncio.run(decimal_sync.refresh(td))
|
||||||
|
decimal_sync.reset()
|
||||||
|
|
||||||
|
path = tmp_path / "decimal-points.json"
|
||||||
|
payload = json.loads(path.read_text(encoding="utf-8"))
|
||||||
|
payload["date"] = "2000-01-01"
|
||||||
|
path.write_text(json.dumps(payload), encoding="utf-8")
|
||||||
|
|
||||||
|
assert decimal_sync.load_cached() == 0
|
||||||
|
|
||||||
|
|
||||||
|
def test_corrupt_cache_is_ignored(tmp_path):
|
||||||
|
path = tmp_path / "decimal-points.json"
|
||||||
|
path.write_text("{not json", encoding="utf-8")
|
||||||
|
assert decimal_sync.load_cached() == 0
|
||||||
|
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------- #
|
||||||
|
# 调度
|
||||||
|
# ---------------------------------------------------------------------- #
|
||||||
|
|
||||||
|
def test_schedule_disabled_by_env(monkeypatch):
|
||||||
|
monkeypatch.setenv("PYPYTDX_DATA_DISABLE_DECIMAL_SYNC", "1")
|
||||||
|
td = FakeTd({"sh": [("019742", 4)]})
|
||||||
|
|
||||||
|
async def run():
|
||||||
|
task = decimal_sync.schedule(td)
|
||||||
|
return task
|
||||||
|
|
||||||
|
assert asyncio.run(run()) is None
|
||||||
|
assert td.calls == []
|
||||||
|
|
||||||
|
|
||||||
|
def test_schedule_loads_cache_without_network(tmp_path):
|
||||||
|
td = FakeTd({"sh": [("019742", 4)]})
|
||||||
|
asyncio.run(decimal_sync.refresh(td))
|
||||||
|
decimal_sync.reset()
|
||||||
|
quiet = FakeTd({"sh": [("019742", 4)]})
|
||||||
|
|
||||||
|
async def run():
|
||||||
|
return decimal_sync.schedule(quiet)
|
||||||
|
|
||||||
|
assert asyncio.run(run()) is None # 缓存命中 → 不起 task
|
||||||
|
assert quiet.calls == [] # 也没走网络
|
||||||
|
assert price_decimal_digits(1, "019742") == 4
|
||||||
|
|
||||||
|
|
||||||
|
def test_schedule_runs_background_sync_without_cache():
|
||||||
|
td = FakeTd({"sh": [("019742", 4)]})
|
||||||
|
|
||||||
|
async def run():
|
||||||
|
task = decimal_sync.schedule(td)
|
||||||
|
assert task is not None
|
||||||
|
await task
|
||||||
|
return decimal_sync.status()
|
||||||
|
|
||||||
|
st = asyncio.run(run())
|
||||||
|
assert st["source"] == "universe"
|
||||||
|
assert st["syncing"] is False
|
||||||
|
assert td.calls == ["sh", "sz"] # 默认两市
|
||||||
|
|
||||||
|
|
||||||
|
def test_cancel_stops_background_task():
|
||||||
|
class SlowTd:
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self.calls = 0
|
||||||
|
|
||||||
|
async def get_universe(self, market=None):
|
||||||
|
self.calls += 1
|
||||||
|
await asyncio.sleep(5)
|
||||||
|
return []
|
||||||
|
|
||||||
|
td = SlowTd()
|
||||||
|
|
||||||
|
async def run():
|
||||||
|
task = decimal_sync.schedule(td)
|
||||||
|
await asyncio.sleep(0.01)
|
||||||
|
await decimal_sync.cancel(task)
|
||||||
|
return task.done() or task.cancelled()
|
||||||
|
|
||||||
|
assert asyncio.run(run()) is True
|
||||||
|
|
||||||
|
|
||||||
|
def test_status_shape():
|
||||||
|
st = decimal_sync.status()
|
||||||
|
assert set(st) >= {"loaded", "source", "syncing", "prefixes", "codes",
|
||||||
|
"mixed_prefixes", "stock"}
|
||||||
@@ -0,0 +1,236 @@
|
|||||||
|
"""单连接流水线(execute_pipeline)单元测试:不连真实网络。
|
||||||
|
|
||||||
|
关键防线:流水线把多条请求塞进同一条 TCP 流,一旦响应与请求错配就是
|
||||||
|
「价格张冠李戴」级别的灾难,所以这里逐项锁死顺序、长度与失败语义。
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import struct
|
||||||
|
import unittest.mock as mock
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from pytdxdata.pool.connection import TdxConnection
|
||||||
|
from pytdxdata.pool.pool import PooledConnection
|
||||||
|
from pytdxdata.protocol.commands.base import BaseCommand
|
||||||
|
|
||||||
|
|
||||||
|
class EchoCmd(BaseCommand[bytes]):
|
||||||
|
"""请求体即 tag,响应体原样带回,便于断言顺序。"""
|
||||||
|
|
||||||
|
def __init__(self, tag: bytes) -> None:
|
||||||
|
self.tag = tag
|
||||||
|
|
||||||
|
def build_request(self) -> bytes:
|
||||||
|
return b"REQ:" + self.tag
|
||||||
|
|
||||||
|
def parse_response(self, body: bytes) -> bytes:
|
||||||
|
return body
|
||||||
|
|
||||||
|
|
||||||
|
class FakeQuote:
|
||||||
|
"""最小报价替身:stamp() 会写 symbol,需要 market/code 是可 int/str 的真值。"""
|
||||||
|
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self.market = 0
|
||||||
|
self.code = "000001"
|
||||||
|
self.symbol = ""
|
||||||
|
|
||||||
|
|
||||||
|
def frame(body: bytes) -> bytes:
|
||||||
|
"""构造一帧未压缩响应(zipsize == unzipsize)。"""
|
||||||
|
return struct.pack("<IIIHH", 7654321, 0, 0x053E, len(body), len(body)) + body
|
||||||
|
|
||||||
|
|
||||||
|
class FakeReader:
|
||||||
|
def __init__(self, data: bytes) -> None:
|
||||||
|
self.data = data
|
||||||
|
self.pos = 0
|
||||||
|
|
||||||
|
async def read(self, n: int) -> bytes:
|
||||||
|
chunk = self.data[self.pos:self.pos + n]
|
||||||
|
self.pos += len(chunk)
|
||||||
|
return chunk
|
||||||
|
|
||||||
|
|
||||||
|
class FakeWriter:
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self.written = b""
|
||||||
|
self.closed = False
|
||||||
|
self.transport = FakeQuote()
|
||||||
|
|
||||||
|
def write(self, data: bytes) -> None:
|
||||||
|
self.written += data
|
||||||
|
|
||||||
|
async def drain(self) -> None:
|
||||||
|
pass
|
||||||
|
|
||||||
|
def is_closing(self) -> bool:
|
||||||
|
return self.closed
|
||||||
|
|
||||||
|
def close(self) -> None:
|
||||||
|
self.closed = True
|
||||||
|
|
||||||
|
async def wait_closed(self) -> None:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def make_conn(payload: bytes, host: str = "1.2.3.4") -> TdxConnection:
|
||||||
|
conn = TdxConnection(host, 7709, 5.0)
|
||||||
|
conn._reader = FakeReader(payload) # type: ignore[assignment]
|
||||||
|
conn._writer = FakeWriter() # type: ignore[assignment]
|
||||||
|
return conn
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- 基本语义 ----------
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_empty_cmds_returns_empty():
|
||||||
|
conn = make_conn(b"")
|
||||||
|
assert await conn.execute_pipeline([]) == []
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_responses_paired_in_order():
|
||||||
|
bodies = [b"AAA", b"BBB", b"CCC"]
|
||||||
|
conn = make_conn(b"".join(frame(b) for b in bodies))
|
||||||
|
cmds = [EchoCmd(t) for t in (b"1", b"2", b"3")]
|
||||||
|
got = await conn.execute_pipeline(cmds)
|
||||||
|
assert got == bodies # 顺序必须与 cmds 严格一致
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_all_requests_sent_at_once():
|
||||||
|
conn = make_conn(b"".join(frame(b"X") for _ in range(3)))
|
||||||
|
cmds = [EchoCmd(t) for t in (b"a", b"b", b"c")]
|
||||||
|
await conn.execute_pipeline(cmds)
|
||||||
|
assert conn._writer.written == b"REQ:aREQ:bREQ:c"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_closed_connection_raises():
|
||||||
|
conn = make_conn(b"")
|
||||||
|
conn._writer = None
|
||||||
|
with pytest.raises(ConnectionError):
|
||||||
|
await conn.execute_pipeline([EchoCmd(b"1")])
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- 失败语义:响应流错配后必须废连接 ----------
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_truncated_stream_force_closes():
|
||||||
|
# 只给第一帧完整的、第二帧只有半个头 → 读不满,抛错且连接作废
|
||||||
|
conn = make_conn(frame(b"AAA") + b"\x00" * 5)
|
||||||
|
with pytest.raises(ConnectionError):
|
||||||
|
await conn.execute_pipeline([EchoCmd(b"1"), EchoCmd(b"2")])
|
||||||
|
assert conn._writer is None # _force_close 已清空 writer
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_body_length_mismatch_propagates():
|
||||||
|
bad = struct.pack("<IIIHH", 7654321, 0, 0x053E, 99, 99) + b"short"
|
||||||
|
conn = make_conn(bad)
|
||||||
|
# 帧头自称 99 字节但流里没有 → _recv_exact 读空,连接作废
|
||||||
|
with pytest.raises(ConnectionError):
|
||||||
|
await conn.execute_pipeline([EchoCmd(b"1")])
|
||||||
|
assert conn._writer is None
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- 池包装转发 ----------
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_pooled_connection_forwards_pipeline():
|
||||||
|
inner = mock.AsyncMock()
|
||||||
|
inner.execute_pipeline = mock.AsyncMock(return_value=["r1", "r2"])
|
||||||
|
pc = PooledConnection(inner, "host1")
|
||||||
|
got = await pc.execute_pipeline([EchoCmd(b"1"), EchoCmd(b"2")])
|
||||||
|
assert got == ["r1", "r2"]
|
||||||
|
inner.execute_pipeline.assert_awaited_once()
|
||||||
|
|
||||||
|
|
||||||
|
# ---------- api 层:切批是否走流水线 ----------
|
||||||
|
|
||||||
|
def _api_with_fake_pool():
|
||||||
|
from pytdxdata.api import TdxData
|
||||||
|
from pytdxdata.symbols import CN, Symbol
|
||||||
|
|
||||||
|
td = TdxData.__new__(TdxData) # 跳过 __init__(不建网络池)
|
||||||
|
td._std_pool = FakeQuote()
|
||||||
|
td._std_pool.max_size = 24
|
||||||
|
td._std_pool.servers = [f"s{i}" for i in range(11)]
|
||||||
|
td._std_pool.acquire = mock.AsyncMock()
|
||||||
|
td._std_pool.release = mock.AsyncMock()
|
||||||
|
td._std_pool.warm = mock.AsyncMock() # 预建连接是网络动作,测试里空转
|
||||||
|
return td, Symbol, CN
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_quotes_std_uses_pipeline_for_multi_page():
|
||||||
|
td, Symbol, CN = _api_with_fake_pool()
|
||||||
|
syms = [Symbol(0, f"{i:06d}", CN) for i in range(1, 401)] # 5 片
|
||||||
|
chunks = []
|
||||||
|
|
||||||
|
async def fake_pipeline(batch_group):
|
||||||
|
chunks.append(len(batch_group))
|
||||||
|
return [[FakeQuote()] * 80 for _ in batch_group]
|
||||||
|
|
||||||
|
td._execute_std_pipeline = fake_pipeline
|
||||||
|
out = await td._quotes_std(syms, pipeline=True)
|
||||||
|
assert len(out) == 5 * 80
|
||||||
|
assert sum(chunks) == 5 # 每片恰好被处理一次,不多不少
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_quotes_std_single_page_skips_pipeline():
|
||||||
|
td, Symbol, CN = _api_with_fake_pool()
|
||||||
|
called = []
|
||||||
|
td._execute_std_pipeline = mock.AsyncMock(return_value=[])
|
||||||
|
|
||||||
|
async def fake_std(cmd):
|
||||||
|
called.append(1)
|
||||||
|
return []
|
||||||
|
|
||||||
|
td._execute_standard = fake_std
|
||||||
|
syms = [Symbol(0, f"{i:06d}", CN) for i in range(1, 51)] # 1 片
|
||||||
|
await td._quotes_std(syms, pipeline=True)
|
||||||
|
assert called == [1]
|
||||||
|
td._execute_std_pipeline.assert_not_awaited()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_quotes_std_no_pipeline_flag_uses_plain_path():
|
||||||
|
td, Symbol, CN = _api_with_fake_pool()
|
||||||
|
|
||||||
|
async def fake_std(cmd):
|
||||||
|
return [FakeQuote()] * 80
|
||||||
|
|
||||||
|
td._execute_standard = fake_std
|
||||||
|
td._execute_std_pipeline = mock.AsyncMock(return_value=[])
|
||||||
|
syms = [Symbol(0, f"{i:06d}", CN) for i in range(1, 401)]
|
||||||
|
out = await td._quotes_std(syms, pipeline=False)
|
||||||
|
assert len(out) == 5 * 80
|
||||||
|
td._execute_std_pipeline.assert_not_awaited()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_pipeline_failure_falls_back_to_per_page():
|
||||||
|
"""节点不支持流水线时,降级逐片,结果条数不变。"""
|
||||||
|
td, Symbol, CN = _api_with_fake_pool()
|
||||||
|
|
||||||
|
pool = td._std_pool
|
||||||
|
pool.acquire = mock.AsyncMock(return_value=mock.Mock(
|
||||||
|
execute_pipeline=mock.AsyncMock(side_effect=ConnectionError("boom"))))
|
||||||
|
pool.release = mock.AsyncMock()
|
||||||
|
|
||||||
|
calls = []
|
||||||
|
|
||||||
|
async def fake_std(cmd):
|
||||||
|
calls.append(1)
|
||||||
|
return [FakeQuote()] * 80
|
||||||
|
|
||||||
|
td._execute_standard = fake_std
|
||||||
|
syms = [Symbol(0, f"{i:06d}", CN) for i in range(1, 241)] # 3 片
|
||||||
|
out = await td._execute_std_pipeline(
|
||||||
|
[syms[i:i + 80] for i in range(0, len(syms), 80)])
|
||||||
|
assert len(out) == 3 and len(calls) == 3
|
||||||
|
assert pool.release.await_args.kwargs.get("broken") is True
|
||||||
@@ -206,6 +206,50 @@ def test_mac_transaction_parse_negative_trade_count_auction():
|
|||||||
assert max(r.trade_count for r in recs) == 3
|
assert max(r.trade_count for r in recs) == 3
|
||||||
|
|
||||||
|
|
||||||
|
def test_mac_transaction_etf_trade_count_unavailable():
|
||||||
|
"""ETF/LOF 的 trade_count 是「价格×1000」占位,必须标记为不可用。
|
||||||
|
|
||||||
|
实测(2026-10-22 sh510310 15:27:35):价 3.989、原字段 3994;
|
||||||
|
旧行为把它当笔数打印成「3994笔」,是假的。
|
||||||
|
"""
|
||||||
|
body = bytearray(39 + 2 * 18)
|
||||||
|
struct.pack_into("<H", body, 29, 2)
|
||||||
|
struct.pack_into("<IfIiH", body, 39, 56000, 3.989, 15, 3994, 5)
|
||||||
|
struct.pack_into("<IfIiH", body, 39 + 18, 56003, 3.989, 4, 3994, 5)
|
||||||
|
|
||||||
|
for code, market, expect in (("510310", 1, False), ("159915", 0, False),
|
||||||
|
("600000", 1, True), ("501050", 1, True)):
|
||||||
|
cmd = MacTransactionCmd(market, code, 20261022, 0, 100)
|
||||||
|
recs = cmd.parse_response(bytes(body))
|
||||||
|
assert recs[0].trade_count_available is expect, code
|
||||||
|
if expect:
|
||||||
|
assert recs[0].trade_count == 3994
|
||||||
|
else:
|
||||||
|
assert recs[0].trade_count == 0
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.parametrize("market,code,expect", [
|
||||||
|
# 股票 / 可转债 / 回购:有笔数
|
||||||
|
(1, "600000", True), (1, "688111", True), (1, "689009", True),
|
||||||
|
(0, "000001", True), (0, "300750", True), (0, "128136", True),
|
||||||
|
(1, "204001", True),
|
||||||
|
# 沪市 LOF 50x:实测有真实笔数,不能一刀切
|
||||||
|
(1, "501050", True), (1, "501018", True), (1, "502000", True), (1, "506000", True),
|
||||||
|
# 沪市 ETF/LOF 51/52/56/58:无笔数
|
||||||
|
(1, "510310", False), (1, "511990", False), (1, "512880", False),
|
||||||
|
(1, "513050", False), (1, "515000", False), (1, "518880", False),
|
||||||
|
(1, "560010", False), (1, "562500", False), (1, "588000", False),
|
||||||
|
# 深市基金:无笔数
|
||||||
|
(0, "159915", False), (0, "159001", False),
|
||||||
|
(0, "161725", False), (0, "160119", False), (0, "165520", False),
|
||||||
|
# 指数:无笔数
|
||||||
|
(1, "000001", False), (0, "399001", False), (1, "880305", False),
|
||||||
|
])
|
||||||
|
def test_has_order_count(market, code, expect):
|
||||||
|
from pytdxdata.symbols import has_order_count
|
||||||
|
assert has_order_count(market, code) is expect
|
||||||
|
|
||||||
|
|
||||||
# ---------- 跨时区时间戳归一化(美股 EX 通道) ----------
|
# ---------- 跨时区时间戳归一化(美股 EX 通道) ----------
|
||||||
#
|
#
|
||||||
# 通达信 EX 通道对美股返回的时间戳把两个时区拼在一起:日期字段是**美东交易日**、
|
# 通达信 EX 通道对美股返回的时间戳把两个时区拼在一起:日期字段是**美东交易日**、
|
||||||
|
|||||||
@@ -0,0 +1,91 @@
|
|||||||
|
"""证券清单(0x053E)回归测试:请求长度 + 北交所降级。全部离线。
|
||||||
|
|
||||||
|
背景(2026-10-23 实测):`build_request` 曾经多塞一个 ``H``(18 字节),
|
||||||
|
服务端只消费前 16 字节,剩下的 ``00 00`` 留在它的接收缓冲里,把**下一个请求**
|
||||||
|
的开头吃掉 → 同一连接「第一次成功、第二次起全部通信超时」。
|
||||||
|
`get_universe` 的分页器复用连接,于是 66/28 页全靠失败重连,冷启动 12~18s。
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import struct
|
||||||
|
import unittest.mock as mock
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from pytdxdata.models import Market
|
||||||
|
from pytdxdata.protocol.commands import GetSecurityListCmd
|
||||||
|
|
||||||
|
_HEADER = bytes.fromhex("0c0118640101060006005004")
|
||||||
|
|
||||||
|
|
||||||
|
def test_request_is_exactly_16_bytes():
|
||||||
|
"""12 字节命令头 + market(H) + start(H),一个字节都不能多。"""
|
||||||
|
req = GetSecurityListCmd(1, 0).build_request()
|
||||||
|
assert len(req) == 16
|
||||||
|
assert req == _HEADER + struct.pack("<HH", 1, 0)
|
||||||
|
|
||||||
|
|
||||||
|
def test_request_encodes_market_and_start_little_endian():
|
||||||
|
req = GetSecurityListCmd(0, 2000).build_request()
|
||||||
|
assert req[:12] == _HEADER
|
||||||
|
market, start = struct.unpack("<HH", req[12:])
|
||||||
|
assert (market, start) == (0, 2000)
|
||||||
|
|
||||||
|
|
||||||
|
def test_request_has_no_trailing_padding():
|
||||||
|
"""尾部不能有 padding:多余的 0 会让服务端解析错位。"""
|
||||||
|
for market in (0, 1, 2):
|
||||||
|
for start in (0, 1000, 27000):
|
||||||
|
req = GetSecurityListCmd(market, start).build_request()
|
||||||
|
assert len(req) == 16, f"market={market} start={start} -> {len(req)} 字节"
|
||||||
|
|
||||||
|
|
||||||
|
def test_parse_response_reads_all_records():
|
||||||
|
"""29 字节/条,头部 2 字节条数。"""
|
||||||
|
code, name = b"600000", "浦发银行".encode("gbk").ljust(8, b"\x00")
|
||||||
|
rec = struct.pack("<6sH8s4sBI4s", code, 100, name, b"\x00" * 4, 2, 0, b"\x00" * 4)
|
||||||
|
assert len(rec) == 29
|
||||||
|
body = struct.pack("<H", 2) + rec + rec
|
||||||
|
rows = GetSecurityListCmd(1, 0).parse_response(body)
|
||||||
|
assert len(rows) == 2
|
||||||
|
assert rows[0].code == "600000"
|
||||||
|
assert rows[0].decimal_point == 2
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_bj_universe_returns_empty_without_network():
|
||||||
|
"""北交所清单协议不支持:必须立刻返回空,且**不碰连接池**。"""
|
||||||
|
from pytdxdata.api import TdxData
|
||||||
|
|
||||||
|
td = TdxData()
|
||||||
|
td._std_pool.acquire = mock.AsyncMock(side_effect=AssertionError("不该联网"))
|
||||||
|
td._std_pool.acquire_many = mock.AsyncMock(
|
||||||
|
side_effect=AssertionError("不该联网"))
|
||||||
|
rows = await td.get_universe("bj")
|
||||||
|
assert rows == []
|
||||||
|
td._std_pool.acquire.assert_not_called()
|
||||||
|
td._std_pool.acquire_many.assert_not_called()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_bj_market_is_in_unsupported_set():
|
||||||
|
from pytdxdata import api as api_mod
|
||||||
|
|
||||||
|
assert int(Market.BJ) in api_mod._LIST_UNSUPPORTED_MARKETS
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_ex_all_fetched_once_in_get_universe():
|
||||||
|
"""`get_universe()` 跨多个 EX 市场时,EX 全量只能取一次。"""
|
||||||
|
from pytdxdata.api import TdxData
|
||||||
|
|
||||||
|
td = TdxData()
|
||||||
|
calls = {"n": 0}
|
||||||
|
|
||||||
|
async def fake_ex_all():
|
||||||
|
calls["n"] += 1
|
||||||
|
return []
|
||||||
|
|
||||||
|
td._ex_all = fake_ex_all
|
||||||
|
await td.get_universe("all")
|
||||||
|
assert calls["n"] == 1, f"_ex_all 被调用 {calls['n']} 次(应为 1)"
|
||||||
Reference in new issue
Block a user