feat: DataFrame 直出(df="polars"/"pandas")+ 0.7.0 (#12)
* feat: DataFrame 直出(df="polars"/"pandas")+ 版本 0.7.0 TdxData 新增构造参数 df: - None(默认)→ 行为与 0.6.x 完全一致,返回 list[dataclass] - "polars"/"pandas" → 42 个表格型接口直接返回 DataFrame 转换层 src/pytdxdata/frames.py: - 列计划按模型缓存,列式采集(不做逐行 asdict) - 时间列替换:year/month/day/hour/minute → datetime + trading_date;Xdxr 给 date - 嵌套定型:bid/ask 展开 10 列、fields 字典 JSON 化、codes 保持列表 - 容器展开:get_indicators 合并指标、get_index_trade_stats 展开 minutes、 get_kline_batch/get_minute_batch 纵向拼接 - ContextVar 层次标记:批量/别名方法内层不二次转换,并发互不干扰 - 后端惰性加载:缺包抛 FrameBackendError(带安装命令),构造时即失败 依赖:新增 extras pandas/polars/dataframe,核心包仍零依赖;dev 组加 pandas+polars 测试:tests/test_frames.py 40 项(双后端×双格式);全量 401 项通过 文档:新增 docs/dataframe.md,同步 API_REFERENCE/SKILL/llms.txt/README/mkdocs * perf(kline): 按日期取K线贴最新时走倒推,1分钟 4.2s → 82ms 问题:start_date/end_date 一律走「探总根数 + 二分定位」。1 分钟 K 线历史 二十万根量级 = 6 次翻倍探测 + 2 次方向探测 + 两次各 ~18 次跳转,共 36+ 次 串行往返。「取今天」这个最常见的调用要 4.2 秒。 改法(_bars_backtrack 快路径): - 右端离最新不超过 _BACKTRACK_GRACE(10 天) 时,直接 start=0, count=N 拿最新 N 根再裁剪;N 按时间跨度估(_BARS_PER_DAY),一天内一次命中, 不够才翻倍,放大到 _BACKTRACK_MAX_BARS 仍不够则回落二分 - 历史中段仍走二分,结果口径完全一致(已逐条比对基准) 两个关键细节: - 偏移方向是通道级属性而非标的属性,首次探测后缓存在实例上 (_kline_dir_cache)——全市场 5,225 只批量只付一次探测成本 - K线 hour/minute 是周期**结束**时刻,最新一根可能比 now() 略大(14:29 那根还在走),「右端贴最新」必须留周期容差,否则 end_date=now() 被误判 成历史中段 实测(1 分钟,冷启动): 取今天 4,208ms → 82ms 2天前~now 2,211ms → 75ms 最近3天 2,000ms+ → 239ms 历史中段 1,512ms → 285ms 全市场5225只取今天 32.1s(1,140,699 根,精确当天口径) 注:route_for 让 MIN_1 一律走 MAC(0x122E),所以按通道做方向缓存而不是按 「标准/扩展」写死分支。 测试:新增 tests/test_kline_date.py 6 项(倒推次数上界、结束时刻容差、 中段回落二分、未来区间、反向通道、与 count 路径口径一致);全量 407 项通过 * perf(kline): 全市场1分钟K线改多连接流水线抓取(43.6s → 5~7s,盘中 2s) 瓶颈其实是**往返次数**不是带宽:单只K线一请求一标的,串行 33ms/只 ≈ 1 个 RTT。 同一台服务器 16 连接 × 深度 8 实测能跑 5.2 MB/s(此前「单 IP 限流 1.5MB/s」 是浅流水下的假象)。 - get_bars 新增 pipeline 参数(None=按单请求字节自适应 8~32,1=关闭走逐只) - 多标的(≥16 只)走 _bulk_bars:多连接 work-stealing + 单连接 execute_pipeline - 只用实测最快的 12 台:池里 24 台全用反而被慢节点拖长尾 - 盘中按交易时段只拉已走过的根数(10:00 → 35 根,数据量差 7 倍) - 标准通道(日线不复权)同样走批量:全市场 49~72s → 13s - 修 pipeline 失败时队列剩余标的静默丢失(实测 20 只只回 16 只) - 修 test_future_range_returns_empty_quickly 时间炸弹(固定日期会漂进 _BACKTRACK_GRACE) - conftest 关闭后台测速 task,避免污染「往返次数」类断言 实测(5,225 只):count=2 → 2.7s;count=35 → 2.2s;count=240 → 6.9s。 逐根一致性 30/30 相同。新增 tests/test_bulk_kline.py 12 项,合计 419 项全绿。 * test: 批量K线测试的时间锚点改用真实 now(CI 是 UTC,固定日期会漂到别的时段分支) * fix(ex): EX 通道 K 线回填成交额与结算价(0x23FF 槽位按市场分两类) 报告:EX 通道 get_bars 返回的 SecurityBar.amount 恒为 0.0。 定位属实,但实际情况比报告更细 —— 32B 记录里那 28B 行情只有**一个** 4B 槽位 可复用,语义按市场分两类(全市场实测,用 成交额/(close×vol) 逐条校验): | 市场 | 槽位5 | 槽位6 | 槽位7 | |---|---|---|---| | 港股主板31/港股基金49 | 成交额(f) | 量(手=100股) | 未知 | | 港股创业板48/新三板44/美股74 | 成交额(f) | 量(股) | 0 | | 期货/期权 7/8/9/47… | 持仓量(I) | 量 | 结算价(f) | 改前 amount 硬编码 0.0、第 7 个 float 用 `_` 丢弃 —— 于是港股/美股/新三板 明明帧内有成交额却全为 0,期货期权的结算价也拿不到。 - 新增 SecurityBar.settle(结算价,float|None):仅 EX 期货/期权日线有值, 分钟线该槽位恒 0 已归一化为 None,其它通道恒 None。 - 权益类市场回填真实 amount;港股主板/基金的 vol 单位是手(=100股), amount/vol 是均价的 100 倍,已写进模型注释与量纲表。 - 期货/期权的 amount 仍是 0.0(帧内确实没有成交额,协议限制), 已在 SecurityBar / API_REFERENCE / llms.txt / SKILL.md 写清, 避免使用者再按旧文档判断成调用方 bug。 - 港股指数27/中证指数62 该槽位量级对不上(既非额也非结算价),保守保持 0.0。 - SecurityBar.__setstate__ 兜底旧磁盘缓存:缓存键无 schema 版本,升级后读旧 pickle 缺 settle 槽位,默认反序列化不补默认值 → AttributeError,已容错; EX K 线缓存键额外加 @v2,让 amount 恒 0 的旧缓存自然失效。 - K 线 DataFrame 因此多一列 settle(0.7.0 未发布,列清单同步更新)。 测试:tests/test_ex_bars_fields.py 16 项(解析层双布局、日内 settle 归零、 API 透传、旧 pickle 兼容);全套 435 项通过。 * docs: 0.7.0 全站口径同步(DataFrame 出口 + EX 成交额限制 + 构造参数表) - 首页:特性表补「DataFrame 直出」与 K 线批量性能,导航新增 DataFrame 卡片, 统一方法数按实测修正为 35(+33 别名 = 68 个公开方法) - 安装页:补 extras(polars/pandas/dataframe)与 df 用法;构造参数表补 df / std_pool_cap / ex_pool_cap,并把 pool_max 默认值从 12 修正为 24(实际值) - 场景示例:`转成 pandas` 一节改为先推 df= 开关,手动转换降级为局部场景 - 数据模型页:不再说「刻意不提供 DataFrame」,改为 dataclass 与 DataFrame 是同一份数据、两种容器 - AI 接入页:返回值约定补 `df=` 开关的说法 - README:K 线列注释补 settle;已知限制补 EX 通道成交额/结算价的协议口径 - CHANGELOG:新增「升级」小节(升级命令、旧缓存无需清理) --------- Co-authored-by: openbot-coder <openbot-coder@users.noreply.github.com>
This commit is contained in:
1 parent
2cf3657f4b
commit
04e0c57569
24 files changed
+3022
-496
No files matched your search
+19
-10
@@ -1,8 +1,13 @@
|
||||
"""测试全局隔离。
|
||||
|
||||
默认关掉**权威小数位表的后台同步** —— 否则任何调用 ``TdxData.start()`` 的用例都会
|
||||
顺手发起证券清单请求(26 + 25 个分页),测试会变慢且依赖网络。
|
||||
需要验证同步行为本身的用例(``test_decimal_sync.py``)自己 ``delenv`` 后再测。
|
||||
默认关掉两个**后台网络行为**:
|
||||
- 权威小数位表同步:否则任何调用 ``TdxData.start()`` 的用例都会顺手发起证券
|
||||
清单请求(26 + 25 个分页),测试变慢且依赖网络。
|
||||
- 当日服务器测速:后台 task 会真的去连主机取K线,会污染「往返次数」这类
|
||||
断言(实测同一个用例的 ``calls`` 在 3/4 之间随机跳),也会拖慢整体。
|
||||
|
||||
需要验证这两个行为本身的用例(``test_decimal_sync.py`` / ``test_probe_cache.py``)
|
||||
自己 ``delenv`` 后再测。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
@@ -10,13 +15,17 @@ import os
|
||||
|
||||
import pytest
|
||||
|
||||
_ENV_KEYS = ("PYPYTDX_DATA_DISABLE_DECIMAL_SYNC", "PYPYTDX_DATA_DISABLE_PROBE")
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True, scope="session")
|
||||
def _disable_decimal_sync():
|
||||
old = os.environ.get("PYPYTDX_DATA_DISABLE_DECIMAL_SYNC")
|
||||
os.environ["PYPYTDX_DATA_DISABLE_DECIMAL_SYNC"] = "1"
|
||||
def _disable_background_network():
|
||||
old = {k: os.environ.get(k) for k in _ENV_KEYS}
|
||||
for k in _ENV_KEYS:
|
||||
os.environ[k] = "1"
|
||||
yield
|
||||
if old is None:
|
||||
os.environ.pop("PYPYTDX_DATA_DISABLE_DECIMAL_SYNC", None)
|
||||
else:
|
||||
os.environ["PYPYTDX_DATA_DISABLE_DECIMAL_SYNC"] = old
|
||||
for k, v in old.items():
|
||||
if v is None:
|
||||
os.environ.pop(k, None)
|
||||
else:
|
||||
os.environ[k] = v
|
||||
@@ -0,0 +1,234 @@
|
||||
"""K线批量流水线路径(多连接 × 单连接流水线)。
|
||||
|
||||
背景:单只K线一请求一标的,全市场 5,000 只就是 5,000 次往返。串行时每只固定付
|
||||
一个 RTT(实测 33ms),实测 24 台 × 8 流水能把全市场 1 分钟K线取当天从 43.6s
|
||||
压到 5s 级。这里用替身验证「路径选择」与「结果口径」,不依赖网络。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import datetime
|
||||
|
||||
import pytest
|
||||
|
||||
from pytdxdata import TdxData
|
||||
from pytdxdata.api import _auto_pipeline, _bars_for_range, _elapsed_trading_minutes
|
||||
from pytdxdata.models import KlinePeriod, SecurityBar
|
||||
|
||||
# 替身的时间锚点必须跟着**真实 now** 走:API 内部用 ``datetime.now()`` 判断「是不是今天」,
|
||||
# 用固定日期会在 CI(UTC 时区,与本地差 8 小时)上漂到别的时段分支,断言时绿时红。
|
||||
NEWEST = datetime.datetime.now().replace(microsecond=0) # noqa: DTZ001
|
||||
|
||||
|
||||
def _bar(when: datetime.datetime, close: float = 10.0) -> SecurityBar:
|
||||
return SecurityBar(
|
||||
market=1, code="600000", open=10.0, high=10.1, low=9.9, close=close,
|
||||
vol=1000, amount=10000, year=when.year, month=when.month, day=when.day,
|
||||
hour=when.hour, minute=when.minute)
|
||||
|
||||
|
||||
class _MacFakeConn:
|
||||
"""MAC 通道语义替身:**offset 0 = 最新一根**,偏移越大越老。"""
|
||||
|
||||
seen: list[tuple[str, int, int, bool]] = [] # (命令名, start, count, 是否流水线)
|
||||
fail_pipeline = False
|
||||
|
||||
def __init__(self, host="m1", port=7709, timeout=10.0):
|
||||
self.host = host
|
||||
self._alive = True
|
||||
|
||||
@property
|
||||
def alive(self) -> bool:
|
||||
return self._alive
|
||||
|
||||
async def connect(self) -> None:
|
||||
pass
|
||||
|
||||
async def close(self) -> None:
|
||||
self._alive = False
|
||||
|
||||
def _rows(self, cmd, *, record: bool = True):
|
||||
if record:
|
||||
type(self).seen.append(
|
||||
(type(cmd).__name__, getattr(cmd, "start", 0),
|
||||
getattr(cmd, "count", 0), False))
|
||||
if "MacKlineCmd" not in type(cmd).__name__:
|
||||
return []
|
||||
out = []
|
||||
for off in range(cmd.start, cmd.start + cmd.count):
|
||||
if off > 20_000:
|
||||
continue
|
||||
when = NEWEST - datetime.timedelta(minutes=off)
|
||||
out.append(_bar(when, 10.0 + (20_000 - off) * 0.001))
|
||||
return out
|
||||
|
||||
async def execute(self, cmd):
|
||||
return self._rows(cmd)
|
||||
|
||||
async def execute_pipeline(self, cmds, **kwargs):
|
||||
if type(self).fail_pipeline:
|
||||
raise ConnectionError("流水线炸了")
|
||||
for c in cmds:
|
||||
type(self).seen.append(
|
||||
(type(c).__name__, getattr(c, "start", 0), getattr(c, "count", 0), True))
|
||||
return [self._rows(c, record=False) for c in cmds]
|
||||
|
||||
|
||||
def _syms(n: int) -> list[str]:
|
||||
"""n 只沪市股票代码(互不相同,避免按 symbol 归并时互相覆盖)。"""
|
||||
return [f"sh60{i:04d}" for i in range(n)]
|
||||
|
||||
|
||||
async def _run(n: int, period=KlinePeriod.MIN_1, **kw):
|
||||
_MacFakeConn.seen = []
|
||||
_MacFakeConn.fail_pipeline = False
|
||||
with pytest.MonkeyPatch.context() as mp:
|
||||
mp.setattr("pytdxdata.api.get_mac_hosts", lambda: ["m1", "m2"])
|
||||
mp.setattr("pytdxdata.pool.pool.TdxConnection",
|
||||
lambda h, p=7709, t=10.0: _MacFakeConn(h))
|
||||
td = TdxData(cache_dir=None, pool_min=1, pool_max=4)
|
||||
await td.start()
|
||||
try:
|
||||
bars = await td.get_bars(_syms(n), period, **kw)
|
||||
finally:
|
||||
await td.close()
|
||||
return bars
|
||||
|
||||
|
||||
def _data_calls() -> list[tuple[str, int, int, bool]]:
|
||||
"""只看真正的数据请求(排除建连探针:count=100 那条)。"""
|
||||
return [c for c in _MacFakeConn.seen if c[2] != 100]
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_bulk_path_uses_pipeline_for_many_symbols():
|
||||
bars = await _run(20, count=240)
|
||||
|
||||
data = _data_calls()
|
||||
assert data, "应该有数据请求"
|
||||
assert all(via_pipe for *_, via_pipe in data), "20 只必须走流水线"
|
||||
assert len(data) == 20
|
||||
assert len(bars) == 20 * 240
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_small_batch_falls_back_to_single():
|
||||
"""低于阈值走逐只:不经过流水线,结果不变。"""
|
||||
bars = await _run(5, count=240)
|
||||
|
||||
data = _data_calls()
|
||||
assert data and not any(via_pipe for *_, via_pipe in data)
|
||||
assert len(bars) == 5 * 240
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_pipeline_1_disables_bulk():
|
||||
bars = await _run(20, count=240, pipeline=1)
|
||||
|
||||
data = _data_calls()
|
||||
assert data and not any(via_pipe for *_, via_pipe in data)
|
||||
assert len(bars) == 20 * 240
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_result_order_follows_input_order():
|
||||
"""返回按入参标的顺序拼接(组内升序)。"""
|
||||
bars = await _run(20, count=30)
|
||||
|
||||
seen: list[str] = []
|
||||
for b in bars:
|
||||
if not seen or seen[-1] != b.symbol:
|
||||
seen.append(b.symbol)
|
||||
assert seen == _syms(20)
|
||||
# 组内按时间正序
|
||||
head = [b.datetime for b in bars if b.symbol == _syms(20)[0]]
|
||||
assert head == sorted(head)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_long_count_is_paged_inside_pipeline():
|
||||
"""count 超过单请求上限时按页切分,且页也走流水线(不是逐只串行翻页)。"""
|
||||
bars = await _run(20, count=1600, pipeline=8)
|
||||
|
||||
data = _data_calls()
|
||||
assert all(via_pipe for *_, via_pipe in data)
|
||||
assert len(data) == 20 * 3 # 700 + 700 + 200
|
||||
assert {c[2] for c in data} == {700, 200}
|
||||
assert len(bars) == 20 * 1600
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_pipeline_failure_falls_back_to_single_per_symbol():
|
||||
"""流水线失败 → 该批交回逐只重试,结果不丢。"""
|
||||
_MacFakeConn.seen = []
|
||||
_MacFakeConn.fail_pipeline = True
|
||||
with pytest.MonkeyPatch.context() as mp:
|
||||
mp.setattr("pytdxdata.api.get_mac_hosts", lambda: ["m1", "m2"])
|
||||
mp.setattr("pytdxdata.pool.pool.TdxConnection",
|
||||
lambda h, p=7709, t=10.0: _MacFakeConn(h))
|
||||
td = TdxData(cache_dir=None, pool_min=1, pool_max=4)
|
||||
await td.start()
|
||||
try:
|
||||
bars = await td.get_bars(_syms(20), KlinePeriod.MIN_1, count=240)
|
||||
finally:
|
||||
await td.close()
|
||||
assert len(bars) == 20 * 240
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_dated_range_bulk_trims_to_start_date():
|
||||
"""日期区间走批量时,区间左端仍然精确裁剪。"""
|
||||
start = NEWEST - datetime.timedelta(minutes=30)
|
||||
bars = await _run(20, start_date=start)
|
||||
|
||||
assert bars
|
||||
assert all(b.datetime >= start for b in bars)
|
||||
assert all(b.symbol in set(_syms(20)) for b in bars)
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_dated_future_range_returns_empty_without_requests():
|
||||
future = NEWEST + datetime.timedelta(days=5)
|
||||
bars = await _run(20, start_date=future)
|
||||
|
||||
assert bars == []
|
||||
|
||||
|
||||
# ---------------- 纯函数 ----------------
|
||||
def test_elapsed_trading_minutes():
|
||||
def at(h, m):
|
||||
return datetime.datetime(2026, 10, 28, h, m) # noqa: DTZ001
|
||||
|
||||
assert _elapsed_trading_minutes(at(9, 0)) == 0
|
||||
assert _elapsed_trading_minutes(at(9, 30)) == 0
|
||||
assert _elapsed_trading_minutes(at(10, 0)) == 30
|
||||
assert _elapsed_trading_minutes(at(11, 45)) == 120 # 午休
|
||||
assert _elapsed_trading_minutes(at(13, 30)) == 150
|
||||
assert _elapsed_trading_minutes(at(15, 0)) == 240
|
||||
assert _elapsed_trading_minutes(at(20, 0)) == 240 # 收盘后
|
||||
|
||||
|
||||
def test_bars_for_range_shrinks_intraday():
|
||||
"""盘中只需已走完的根数,不是一整天 240 根。"""
|
||||
morning = datetime.datetime(2026, 10, 28, 10, 0) # noqa: DTZ001
|
||||
midnight = datetime.datetime(2026, 10, 28, 0, 0) # noqa: DTZ001
|
||||
# 10:00 走了 30 分钟 → 30 + 余量,远小于 240
|
||||
assert 30 <= _bars_for_range(KlinePeriod.MIN_1, midnight, morning) <= 40
|
||||
# 收盘后回到一整天的量级
|
||||
close = datetime.datetime(2026, 10, 28, 16, 0) # noqa: DTZ001
|
||||
assert _bars_for_range(KlinePeriod.MIN_1, midnight, close) >= 245
|
||||
|
||||
|
||||
def test_bars_for_range_past_days():
|
||||
now = datetime.datetime(2026, 10, 28, 16, 0) # noqa: DTZ001
|
||||
start = now - datetime.timedelta(days=5)
|
||||
n = _bars_for_range(KlinePeriod.DAY, start, now)
|
||||
assert 5 <= n <= 12 # 5 个自然日 × 1.6 余量
|
||||
|
||||
|
||||
def test_auto_pipeline():
|
||||
# 每根 32B:小 count 用深流水,大 count 用浅流水(在途字节数恒定)
|
||||
assert _auto_pipeline(2, None) == 32
|
||||
assert _auto_pipeline(240, None) == 8
|
||||
assert _auto_pipeline(240, 4) == 4 # 显式指定优先
|
||||
assert _auto_pipeline(2, 1) == 2 # 夹到最小值 2
|
||||
@@ -0,0 +1,175 @@
|
||||
"""EX 0x23FF K线的字段语义:``amount`` / ``settle`` 按市场分别回填。
|
||||
|
||||
背景(2026-10-28 实测):0x23FF 每条 32B = 4B 时间戳 + **28B 行情**,这 28B 只有
|
||||
**一个** 4B 槽位可复用,语义随市场变化 ——
|
||||
|
||||
* 权益类(港股主板/创业板/基金、新三板、美股):该槽位是**成交额**,
|
||||
校验 ``成交额 / (close × vol)`` ≈ 100(港股主板/基金,vol 单位是手)或 ≈ 1(其余,vol 是股);
|
||||
* 期货期权类:该槽位是**结算价**,成交额在帧内根本不存在(恒 0.0)。
|
||||
|
||||
改之前 ``amount`` 被硬编码成 0.0、末位 float 直接丢弃,于是港股/美股拿不到成交额、
|
||||
期货期权拿不到结算价。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import pickle
|
||||
import struct
|
||||
|
||||
import pytest
|
||||
|
||||
from pytdxdata.api import TdxData
|
||||
from pytdxdata.models import SecurityBar
|
||||
from pytdxdata.protocol.commands.ex_proto import (
|
||||
_EX_AMOUNT_MARKETS,
|
||||
GetExInstrumentBarsCmd,
|
||||
)
|
||||
from pytdxdata.symbols import parse_symbols
|
||||
|
||||
DAY = 20261028
|
||||
|
||||
|
||||
def _f2u(value: float) -> int:
|
||||
"""float32 -> 其位模式的 uint32(用来拼「同一槽位两种解释」的报文)。"""
|
||||
return struct.unpack("<I", struct.pack("<f", value))[0]
|
||||
|
||||
|
||||
def _ex_body(rows: list[tuple[int, ...]], category: int = 4) -> bytes:
|
||||
"""拼 0x23FF 响应体:20B 头(count@18)+ N × 32B(时间4B + 7×4B)。"""
|
||||
body = bytearray(20 + 32 * len(rows))
|
||||
struct.pack_into("<H", body, 18, len(rows))
|
||||
for i, words in enumerate(rows):
|
||||
off = 20 + i * 32
|
||||
struct.pack_into("<I", body, off, DAY)
|
||||
struct.pack_into("<7I", body, off + 4, *words)
|
||||
return bytes(body)
|
||||
|
||||
|
||||
def _equity_words(close: float, amount: float, vol: int) -> tuple[int, ...]:
|
||||
"""权益类布局:OHLC + 成交额(float) + 成交量(uint) + 保留(float)。"""
|
||||
return (_f2u(close), _f2u(close), _f2u(close), _f2u(close),
|
||||
_f2u(amount), vol, _f2u(0.0))
|
||||
|
||||
|
||||
def _derivative_words(close: float, position: int, vol: int,
|
||||
settle: float) -> tuple[int, ...]:
|
||||
"""期货/期权布局:OHLC + 持仓量(uint) + 成交量(uint) + 结算价(float)。"""
|
||||
return (_f2u(close), _f2u(close), _f2u(close), _f2u(close),
|
||||
position, vol, _f2u(settle))
|
||||
|
||||
|
||||
# ---------- 解析层 ----------
|
||||
|
||||
@pytest.mark.parametrize("market", [31, 48, 49, 44, 74])
|
||||
def test_equity_markets_read_amount_from_fifth_slot(market: int) -> None:
|
||||
cmd = GetExInstrumentBarsCmd(market, "00700", 4, 0, 1)
|
||||
rows = cmd.parse_response(_ex_body([_equity_words(10.2, 2_040_000.0, 2000)]))
|
||||
|
||||
assert len(rows) == 1
|
||||
assert rows[0]["amount"] == pytest.approx(2_040_000.0)
|
||||
assert rows[0]["trade"] == 2000 # 成交量不受影响
|
||||
assert rows[0]["position"] == 0 # 权益类无持仓量概念
|
||||
assert rows[0]["settle"] is None
|
||||
|
||||
|
||||
@pytest.mark.parametrize("market", [7, 8, 9, 47])
|
||||
def test_derivative_markets_read_settle_from_seventh_slot(market: int) -> None:
|
||||
cmd = GetExInstrumentBarsCmd(market, "IFL0", 4, 0, 1)
|
||||
rows = cmd.parse_response(_ex_body([_derivative_words(4021.0, 49_772, 25_392, 4047.8)]))
|
||||
|
||||
assert rows[0]["amount"] == 0.0 # 帧内没有成交额,不是解析漏读
|
||||
assert rows[0]["position"] == 49_772
|
||||
assert rows[0]["trade"] == 25_392
|
||||
assert rows[0]["settle"] == pytest.approx(4047.8)
|
||||
|
||||
|
||||
def test_intraday_settle_zero_normalized_to_none() -> None:
|
||||
"""结算价是日终量:分钟线该槽位恒 0,归一化成 None 而不是 0.0。"""
|
||||
cmd = GetExInstrumentBarsCmd(47, "IFL0", 7, 0, 1)
|
||||
rows = cmd.parse_response(_ex_body([_derivative_words(4054.6, 49_490, 260, 0.0)], 7))
|
||||
|
||||
assert rows[0]["settle"] is None
|
||||
|
||||
|
||||
def test_index_markets_kept_out_of_amount_table() -> None:
|
||||
"""港股指数(27)/中证指数(62) 该槽位既非成交额也非结算价 —— 不能猜,保持 0.0。"""
|
||||
assert 27 not in _EX_AMOUNT_MARKETS
|
||||
assert 62 not in _EX_AMOUNT_MARKETS
|
||||
cmd = GetExInstrumentBarsCmd(27, "HSI", 4, 0, 1)
|
||||
rows = cmd.parse_response(_ex_body([_equity_words(23041.32, 178_348.0, 17_835)]))
|
||||
assert rows[0]["amount"] == 0.0
|
||||
|
||||
|
||||
# ---------- API 层透传 ----------
|
||||
|
||||
def _fake_api(rows_for: dict[int, list[dict]]) -> TdxData:
|
||||
td = TdxData(cache_dir=None, pool_min=1, pool_max=2)
|
||||
seen: list[tuple[int, str]] = []
|
||||
|
||||
async def fake(make_cmd, spec): # noqa: ANN001 — 与真实签名同形
|
||||
cmd = make_cmd(0, spec.page_size)
|
||||
seen.append((cmd.market, cmd.code))
|
||||
return rows_for[cmd.market]
|
||||
|
||||
td._fetch_paginated_ex = fake # type: ignore[method-assign]
|
||||
td._seen_ex = seen # type: ignore[attr-defined]
|
||||
return td
|
||||
|
||||
|
||||
def _row(market: int, code: str, **kw) -> dict:
|
||||
base = {"year": 2026, "month": 10, "day": 28, "hour": 15, "minute": 0,
|
||||
"open": 10.0, "high": 10.5, "low": 9.5, "close": 10.2,
|
||||
"amount": 0.0, "position": 0, "trade": 2000, "settle": None}
|
||||
base.update(kw)
|
||||
return base
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_bars_ex_backfills_amount_for_hk() -> None:
|
||||
td = _fake_api({31: [_row(31, "00700", amount=2_040_000.0)]})
|
||||
bars = await td.get_bars("hk00700", count=1)
|
||||
|
||||
assert len(bars) == 1
|
||||
assert bars[0].amount == pytest.approx(2_040_000.0)
|
||||
assert bars[0].vol == 2000
|
||||
assert bars[0].settle is None
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_bars_ex_backfills_settle_for_futures() -> None:
|
||||
td = _fake_api({47: [_row(47, "IFL0", position=49_772, trade=25_392, settle=4047.8)]})
|
||||
bars = await td.get_bars("cffex_futures:IFL0", count=1)
|
||||
|
||||
assert bars[0].settle == pytest.approx(4047.8)
|
||||
assert bars[0].amount == 0.0
|
||||
|
||||
|
||||
# ---------- 旧磁盘缓存兼容 ----------
|
||||
|
||||
def test_legacy_pickle_without_settle_still_loads() -> None:
|
||||
"""升级前写下的缓存没有 settle 槽位;默认反序列化不会补默认值,读它必崩。"""
|
||||
bar = SecurityBar(market=1, code="600000", open=1.0, high=2.0, low=0.5,
|
||||
close=1.5, vol=100.0, amount=150.0,
|
||||
year=2026, month=10, day=28, settle=12.3)
|
||||
_, slots = bar.__getstate__()
|
||||
legacy = dict(slots)
|
||||
legacy.pop("settle")
|
||||
|
||||
revived = SecurityBar.__new__(SecurityBar)
|
||||
revived.__setstate__((None, legacy))
|
||||
|
||||
assert revived.settle is None
|
||||
assert revived.close == 1.5
|
||||
|
||||
|
||||
def test_new_pickle_roundtrip_keeps_settle() -> None:
|
||||
bar = SecurityBar(market=47, code="IFL0", open=4021.0, high=4060.0, low=4010.0,
|
||||
close=4054.6, vol=25_618.0, amount=0.0,
|
||||
year=2026, month=10, day=28, settle=4052.6)
|
||||
|
||||
assert pickle.loads(pickle.dumps(bar)).settle == pytest.approx(4052.6)
|
||||
|
||||
|
||||
def test_ex_symbols_route_to_ex_path() -> None:
|
||||
"""守住前提:上面对 31/47 的断言确实走的是 EX 通道。"""
|
||||
assert parse_symbols(["hk00700"])[0].is_ex
|
||||
assert parse_symbols(["cffex_futures:IFL0"])[0].is_ex
|
||||
@@ -0,0 +1,325 @@
|
||||
"""DataFrame 转换层测试(0.7.0)。
|
||||
|
||||
覆盖两条独立路径:
|
||||
|
||||
1. **转换口径**:直接调 :func:`to_frame`,喂合成模型对象,断言列名/列序/取值/dtype。
|
||||
不碰网络,双后端各跑一遍。
|
||||
2. **接线**:``TdxData(df=...)`` 的取值校验、装饰器的「内层不二次转换」语义。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
import sys
|
||||
|
||||
import pytest
|
||||
|
||||
from pytdxdata import TdxData
|
||||
from pytdxdata import frames as _frames
|
||||
from pytdxdata.frames import FrameBackendError, framed, get_backend, to_frame
|
||||
from pytdxdata.models import (
|
||||
IndexBar,
|
||||
IndexTradeStats,
|
||||
IndicatorSet,
|
||||
Market,
|
||||
MinuteBar,
|
||||
MinuteTradeStat,
|
||||
SecurityBar,
|
||||
SecurityQuote,
|
||||
TdxBlock,
|
||||
XdxrRecord,
|
||||
)
|
||||
|
||||
BACKENDS = ("polars", "pandas")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 工具
|
||||
|
||||
|
||||
def bar(**kw) -> SecurityBar:
|
||||
base = {
|
||||
"market": int(Market.SH), "code": "600000", "open": 10.0, "high": 11.0,
|
||||
"low": 9.5, "close": 10.5, "vol": 1000.0, "amount": 10500.0,
|
||||
"year": 2026, "month": 10, "day": 27, "hour": 0, "minute": 0,
|
||||
"symbol": "sh600000",
|
||||
}
|
||||
base.update(kw)
|
||||
return SecurityBar(**base)
|
||||
|
||||
|
||||
def rows(df, backend: str) -> int:
|
||||
return len(df) if backend == "pandas" else df.height
|
||||
|
||||
|
||||
def cols(df, backend: str) -> list[str]:
|
||||
return list(df.columns)
|
||||
|
||||
|
||||
def cell(df, backend: str, name: str, index: int = 0):
|
||||
if backend == "pandas":
|
||||
return df[name].iloc[index]
|
||||
# polars 2.x:Series[0] 会返回 Series 而不是标量,统一走 to_list()
|
||||
return df.get_column(name).to_list()[index]
|
||||
|
||||
|
||||
def missing(value) -> bool:
|
||||
"""pandas 的数值列会把 None 变成 NaN,两者都算「缺」。"""
|
||||
return value is None or (isinstance(value, float) and math.isnan(value))
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 后端
|
||||
|
||||
|
||||
def test_backend_instances_are_cached():
|
||||
assert get_backend("polars") is get_backend("polars")
|
||||
assert get_backend("pandas") is get_backend("pandas")
|
||||
|
||||
|
||||
def test_unknown_backend_raises_value_error():
|
||||
with pytest.raises(ValueError, match="未知的 DataFrame 后端"):
|
||||
get_backend("numpy")
|
||||
|
||||
|
||||
def test_missing_backend_raises_with_install_hint(monkeypatch):
|
||||
monkeypatch.setitem(sys.modules, "polars", None) # 让 import polars 抛 ImportError
|
||||
monkeypatch.delitem(_frames._BACKEND_CACHE, "polars", raising=False)
|
||||
with pytest.raises(FrameBackendError) as excinfo:
|
||||
get_backend("polars")
|
||||
assert isinstance(excinfo.value, ImportError)
|
||||
assert excinfo.value.name == "polars"
|
||||
assert "pip install pytdxdata[polars]" in str(excinfo.value)
|
||||
monkeypatch.delitem(_frames._BACKEND_CACHE, "polars", raising=False)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 转换口径
|
||||
|
||||
|
||||
@pytest.mark.parametrize("backend", BACKENDS)
|
||||
def test_bar_replaces_split_time_with_datetime(backend):
|
||||
df = to_frame([bar(), bar(minute=30)], backend=backend)
|
||||
assert cols(df, backend) == [
|
||||
"market", "code", "open", "high", "low", "close", "vol", "amount",
|
||||
"datetime", "trading_date", "symbol", "settle",
|
||||
]
|
||||
assert rows(df, backend) == 2
|
||||
assert cell(df, backend, "market") == 1
|
||||
assert cell(df, backend, "trading_date") == "2026-10-27"
|
||||
stamp = cell(df, backend, "datetime", 1)
|
||||
assert stamp.year == 2026 and stamp.hour == 0 and stamp.minute == 30
|
||||
|
||||
|
||||
@pytest.mark.parametrize("backend", BACKENDS)
|
||||
def test_int_enum_column_is_plain_int(backend):
|
||||
df = to_frame([bar(market=Market.SZ)], backend=backend)
|
||||
value = cell(df, backend, "market")
|
||||
assert value == 0
|
||||
assert not isinstance(value, Market)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("backend", BACKENDS)
|
||||
def test_empty_list_falls_back_to_annotation_hint(backend):
|
||||
df = to_frame([], backend=backend, model="SecurityBar")
|
||||
assert rows(df, backend) == 0
|
||||
assert cols(df, backend) == [
|
||||
"market", "code", "open", "high", "low", "close", "vol", "amount",
|
||||
"datetime", "trading_date", "symbol", "settle",
|
||||
]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("backend", BACKENDS)
|
||||
def test_quote_expands_bid_ask_levels(backend):
|
||||
quote = SecurityQuote(
|
||||
market=1, code="600000", price=10.5, pre_close=10.0, open=10.1, high=10.6,
|
||||
low=9.9, vol=100.0, amount=1000.0, s_vol=40.0, b_vol=60.0,
|
||||
bid=[(10.49, 100.0), (10.48, 200.0)], ask=[(10.51, 300.0)],
|
||||
symbol="sh600000", fields={"0x7": 1.5},
|
||||
)
|
||||
df = to_frame([quote], backend=backend)
|
||||
out = cols(df, backend)
|
||||
assert "bid1_price" in out and "ask5_vol" in out
|
||||
assert "bid" not in out and "ask" not in out
|
||||
assert cell(df, backend, "bid2_vol") == 200.0
|
||||
assert missing(cell(df, backend, "bid3_price")) # 只有两档,缺档给 None
|
||||
assert cell(df, backend, "fields") == '{"0x7":1.5}'
|
||||
|
||||
|
||||
@pytest.mark.parametrize("backend", BACKENDS)
|
||||
def test_empty_dict_field_becomes_none(backend):
|
||||
df = to_frame([SecurityQuote(
|
||||
market=1, code="600000", price=1.0, pre_close=1.0, open=1.0, high=1.0,
|
||||
low=1.0, vol=1.0, amount=1.0, s_vol=0.0, b_vol=0.0,
|
||||
)], backend=backend)
|
||||
assert missing(cell(df, backend, "fields"))
|
||||
|
||||
|
||||
@pytest.mark.parametrize("backend", BACKENDS)
|
||||
def test_minute_uses_real_clock(backend):
|
||||
late = MinuteBar(price=1.0, vol=10.0, avg_price=1.0, hour=27, minute=59,
|
||||
symbol="usAAPL", market=74, code="AAPL", date=20261026)
|
||||
df = to_frame([late], backend=backend)
|
||||
assert cols(df, backend) == [
|
||||
"price", "vol", "avg_price", "datetime", "trading_date",
|
||||
"symbol", "market", "code", "fields",
|
||||
]
|
||||
stamp = cell(df, backend, "datetime")
|
||||
assert (stamp.year, stamp.month, stamp.day, stamp.hour, stamp.minute) == (
|
||||
2026, 10, 27, 3, 59,
|
||||
)
|
||||
assert cell(df, backend, "trading_date") == "2026-10-26"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("backend", BACKENDS)
|
||||
def test_xdxr_date_column(backend):
|
||||
row = XdxrRecord(market=1, code="600000", year=2026, month=6, day=12,
|
||||
category=1, name="分红", fenhong=0.5, symbol="sh600000")
|
||||
df = to_frame([row], backend=backend)
|
||||
assert cols(df, backend)[:3] == ["market", "code", "date"]
|
||||
stamp = cell(df, backend, "date")
|
||||
assert (stamp.year, stamp.month, stamp.day) == (2026, 6, 12)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("backend", BACKENDS)
|
||||
def test_tdx_block_keeps_codes_list(backend):
|
||||
df = to_frame([TdxBlock(name="银行", category=0, count=2, codes=["600000", "600036"])],
|
||||
backend=backend)
|
||||
assert cols(df, backend) == ["name", "category", "count", "codes"]
|
||||
assert cell(df, backend, "codes") == ["600000", "600036"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("backend", BACKENDS)
|
||||
def test_indicator_set_merges_bars_and_indicators(backend):
|
||||
item = IndicatorSet(
|
||||
symbol="sh600000",
|
||||
bars=[bar(), bar(minute=1), bar(minute=2)],
|
||||
indicators={"ma5": [1.0, 2.0, 3.0], "boll": [None, None, 9.0]},
|
||||
)
|
||||
df = to_frame([item], backend=backend)
|
||||
assert rows(df, backend) == 3
|
||||
assert cell(df, backend, "symbol") == "sh600000"
|
||||
assert cell(df, backend, "ma5", 2) == 3.0
|
||||
assert missing(cell(df, backend, "boll"))
|
||||
|
||||
|
||||
@pytest.mark.parametrize("backend", BACKENDS)
|
||||
def test_indicator_series_shorter_than_bars_is_padded(backend):
|
||||
item = IndicatorSet(symbol="sh600000", bars=[bar(), bar(minute=1)],
|
||||
indicators={"ma5": [7.0]})
|
||||
df = to_frame([item], backend=backend)
|
||||
assert rows(df, backend) == 2
|
||||
assert cell(df, backend, "ma5") == 7.0
|
||||
assert missing(cell(df, backend, "ma5", 1))
|
||||
|
||||
|
||||
@pytest.mark.parametrize("backend", BACKENDS)
|
||||
def test_index_trade_stats_single_and_multi(backend):
|
||||
stat = IndexTradeStats(
|
||||
index="sh000300", date=20261027, member_count=300, active_members=299,
|
||||
trade_count_members=299,
|
||||
minutes=[MinuteTradeStat(time="09:31", volume=10.0, trade_count=5, amount=1000.0)],
|
||||
)
|
||||
single = to_frame(stat, backend=backend)
|
||||
assert cols(single, backend) == ["index", "date", "time", "volume", "trade_count", "amount"]
|
||||
assert cell(single, backend, "index") == "sh000300"
|
||||
|
||||
multi = to_frame([stat, stat], backend=backend)
|
||||
assert rows(multi, backend) == 2
|
||||
|
||||
|
||||
@pytest.mark.parametrize("backend", BACKENDS)
|
||||
def test_batch_dict_is_concatenated(backend):
|
||||
data = {
|
||||
"600000": [bar(code="600000"), bar(code="600000", minute=1)],
|
||||
"600036": [bar(code="600036", symbol="sh600036")],
|
||||
}
|
||||
df = to_frame(data, backend=backend)
|
||||
assert rows(df, backend) == 3
|
||||
assert cols(df, backend)[:2] == ["market", "code"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("backend", BACKENDS)
|
||||
def test_batch_dict_all_empty_returns_typed_empty(backend):
|
||||
df = to_frame({"600000": [], "600036": []}, backend=backend, model="SecurityBar")
|
||||
assert rows(df, backend) == 0
|
||||
assert "datetime" in cols(df, backend)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("backend", BACKENDS)
|
||||
def test_list_of_dicts_uses_key_union(backend):
|
||||
df = to_frame([{"name": "银行", "rise": 1.2}, {"name": "券商", "amount": 5.0}],
|
||||
backend=backend)
|
||||
assert cols(df, backend) == ["name", "rise", "amount"]
|
||||
assert missing(cell(df, backend, "amount", 0))
|
||||
|
||||
|
||||
def test_unsupported_payload_raises_type_error():
|
||||
with pytest.raises(TypeError, match="无法转换为 DataFrame 的返回值类型"):
|
||||
to_frame(42, backend="polars")
|
||||
with pytest.raises(TypeError, match="无法转换为 DataFrame 的列表元素"):
|
||||
to_frame([1.0, 2.0], backend="polars")
|
||||
|
||||
|
||||
def test_subclass_reuses_parent_plan():
|
||||
df = to_frame([IndexBar(market=1, code="000001", open=1.0, high=1.0, low=1.0,
|
||||
close=1.0, vol=1.0, amount=1.0, year=2026, month=1, day=5)],
|
||||
backend="polars")
|
||||
assert "datetime" in df.columns
|
||||
assert "year" not in df.columns
|
||||
|
||||
|
||||
# ---------------------------------------------------------------- 接线
|
||||
|
||||
|
||||
def test_tdxdata_default_keeps_legacy_behaviour():
|
||||
td = TdxData()
|
||||
assert td._df_backend is None
|
||||
|
||||
|
||||
def test_tdxdata_rejects_unknown_backend():
|
||||
with pytest.raises(ValueError, match="未知的 DataFrame 后端"):
|
||||
TdxData(df="numpy")
|
||||
|
||||
|
||||
def test_tdxdata_accepts_supported_backend():
|
||||
td = TdxData(df="polars")
|
||||
assert td._df_backend == "polars"
|
||||
|
||||
|
||||
class _Source:
|
||||
"""最小替身:只验装饰器语义,不碰网络。"""
|
||||
|
||||
def __init__(self, backend: str | None) -> None:
|
||||
self._df_backend = backend
|
||||
|
||||
@framed
|
||||
async def single(self) -> list[SecurityBar]:
|
||||
return [bar()]
|
||||
|
||||
@framed
|
||||
async def nested(self) -> dict[str, list[SecurityBar]]:
|
||||
got = await self.single() # 内层必须是原始 list
|
||||
assert isinstance(got, list), f"内层被提前转换成了 {type(got)!r}"
|
||||
return {"600000": got}
|
||||
|
||||
|
||||
async def test_decorator_passthrough_when_no_backend():
|
||||
src = _Source(None)
|
||||
assert isinstance(await src.single(), list)
|
||||
|
||||
|
||||
async def test_decorator_converts_outermost_only():
|
||||
src = _Source("polars")
|
||||
out = await src.nested()
|
||||
assert cols(out, "polars")[0] == "market"
|
||||
assert rows(out, "polars") == 1
|
||||
|
||||
|
||||
def test_model_name_resolution():
|
||||
# 注解是 dict[str, list[SecurityBar]] 时正则也能取到 SecurityBar;
|
||||
# 但 "dict" 这类内建名在 models 里找不到,必须返回 None 而不是内建类型
|
||||
assert _frames._model_by_name("dict") is None
|
||||
assert _frames._model_by_name("SecurityBar") is SecurityBar
|
||||
|
||||
|
||||
def test_datetime_dtype_is_datetime_like():
|
||||
pl = pytest.importorskip("polars")
|
||||
df = to_frame([bar()], backend="polars")
|
||||
assert isinstance(df.schema["datetime"], pl.Datetime)
|
||||
@@ -0,0 +1,166 @@
|
||||
"""K线「按日期区间」取数的路径选择:倒推快路径 vs 二分慢路径。
|
||||
|
||||
背景:二分要先探总根数(翻倍 6 次)再跳 log2(N) 次,1 分钟 K 线约 36 次串行
|
||||
往返 = 4 秒级;而「取今天」「增量到最新」这类贴着右端的请求,倒推只要 1~3 次。
|
||||
"""
|
||||
|
||||
import datetime
|
||||
from unittest import mock
|
||||
|
||||
import pytest
|
||||
|
||||
from pytdxdata import TdxData
|
||||
from pytdxdata.models import KlinePeriod, SecurityBar
|
||||
|
||||
NEWEST = datetime.datetime(2026, 10, 28, 15, 0) # noqa: DTZ001 — 全项目用裸北京时间
|
||||
TOTAL_BARS = 5_000
|
||||
|
||||
|
||||
def _bar(when: datetime.datetime, close: float) -> SecurityBar:
|
||||
return SecurityBar(
|
||||
market=1, code="600000", open=10.0, high=10.1, low=9.9, close=close,
|
||||
vol=1000, amount=10000, year=when.year, month=when.month, day=when.day,
|
||||
hour=when.hour, minute=when.minute)
|
||||
|
||||
|
||||
class _FakeConn:
|
||||
"""标准通道语义替身:**offset 0 = 最新一根**,偏移越大越老。"""
|
||||
|
||||
calls = 0
|
||||
|
||||
def __init__(self, host="s1", port=7709, timeout=10.0):
|
||||
self.host = host
|
||||
self.alive_flag = True
|
||||
|
||||
@property
|
||||
def alive(self):
|
||||
return self.alive_flag
|
||||
|
||||
async def connect(self):
|
||||
pass
|
||||
|
||||
async def close(self):
|
||||
self.alive_flag = False
|
||||
|
||||
async def execute_pipeline(self, cmds, **kwargs):
|
||||
return [await self.execute(c) for c in cmds]
|
||||
|
||||
async def execute(self, cmd):
|
||||
type(self).calls += 1
|
||||
if "GetSecurityBarsCmd" not in type(cmd).__name__:
|
||||
return []
|
||||
rows = []
|
||||
for off in range(cmd.start, min(cmd.start + cmd.count, TOTAL_BARS)):
|
||||
rows.append(_bar(NEWEST - datetime.timedelta(days=off),
|
||||
10.0 + (TOTAL_BARS - off) * 0.01))
|
||||
return rows
|
||||
|
||||
|
||||
class _AscendingConn(_FakeConn):
|
||||
"""方向相反的替身(MAC/EX 语义):offset 0 = **最老**一根。"""
|
||||
|
||||
async def execute(self, cmd):
|
||||
type(self).calls += 1
|
||||
if "GetSecurityBarsCmd" not in type(cmd).__name__:
|
||||
return []
|
||||
rows = []
|
||||
base = datetime.datetime(2020, 1, 1) # noqa: DTZ001
|
||||
for off in range(cmd.start, min(cmd.start + cmd.count, TOTAL_BARS)):
|
||||
rows.append(_bar(base + datetime.timedelta(days=off), 10.0 + off * 0.01))
|
||||
return rows
|
||||
|
||||
|
||||
def _patched(conn_cls):
|
||||
return (
|
||||
mock.patch("pytdxdata.api.get_standard_hosts", return_value=["s1"]),
|
||||
mock.patch("pytdxdata.api.get_mac_hosts", return_value=["m1"]),
|
||||
mock.patch("pytdxdata.pool.pool.TdxConnection",
|
||||
side_effect=lambda h, p=7709, t=10.0: conn_cls(h)),
|
||||
)
|
||||
|
||||
|
||||
async def _fetch(conn_cls, **kwargs):
|
||||
conn_cls.calls = 0
|
||||
p1, p2, p3 = _patched(conn_cls)
|
||||
with p1, p2, p3:
|
||||
td = TdxData(cache_dir=None, pool_min=1, pool_max=2)
|
||||
await td.start()
|
||||
try:
|
||||
bars = await td.get_bars("sh600000", KlinePeriod.DAY, **kwargs)
|
||||
finally:
|
||||
await td.close()
|
||||
return bars, conn_cls.calls
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_recent_range_uses_backtrack():
|
||||
"""右端贴最新 → 倒推,往返次数是个位数(不是二分那几十次)。"""
|
||||
bars, calls = await _fetch(
|
||||
_FakeConn, start_date=NEWEST - datetime.timedelta(days=5), end_date=NEWEST)
|
||||
|
||||
# 返回按时间升序 → 偏移由老到新(5,4,...,0)
|
||||
assert [b.datetime.date() for b in bars] == [
|
||||
(NEWEST - datetime.timedelta(days=i)).date() for i in range(5, -1, -1)]
|
||||
assert calls <= 4, f"倒推路径应 1~3 次往返,实际 {calls}"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_end_before_newest_within_one_bar_still_backtracks():
|
||||
"""end 比最新一根略早(K线时间戳是周期结束时刻)仍算贴最新。"""
|
||||
end = NEWEST - datetime.timedelta(minutes=30)
|
||||
bars, calls = await _fetch(
|
||||
_FakeConn, start_date=NEWEST - datetime.timedelta(days=3), end_date=end)
|
||||
|
||||
# offset 0 那根(15:00)晚于 end,被裁掉
|
||||
assert bars[-1].datetime.date() == (NEWEST - datetime.timedelta(days=1)).date()
|
||||
assert calls <= 4
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_middle_range_falls_back_to_bisect():
|
||||
"""历史中段倒推够不着左边 → 回落二分,结果仍要正确。"""
|
||||
bars, calls = await _fetch(
|
||||
_FakeConn,
|
||||
start_date=NEWEST - datetime.timedelta(days=100),
|
||||
end_date=NEWEST - datetime.timedelta(days=90))
|
||||
|
||||
assert [b.datetime.date() for b in bars] == [
|
||||
(NEWEST - datetime.timedelta(days=i)).date() for i in range(100, 89, -1)]
|
||||
assert calls > 10, f"中段应走二分(几十次往返),实际只有 {calls}"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_future_range_returns_empty_quickly():
|
||||
# 用相对 now 的未来,别用固定日期:固定日期会随运行日漂进 _BACKTRACK_GRACE
|
||||
# (默认 1 天容差覆盖长假),倒推就成立、会多取一轮,断言随日期时绿时红。
|
||||
future = datetime.datetime.now() + datetime.timedelta(days=5) # noqa: DTZ001
|
||||
bars, calls = await _fetch(_FakeConn, start_date=future)
|
||||
|
||||
assert bars == []
|
||||
assert calls <= 3
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_ascending_channel_skips_backtrack():
|
||||
"""offset 0 是最老(MAC/EX 语义)时倒推不成立,必须走二分且结果正确。"""
|
||||
base = datetime.datetime(2020, 1, 1) # noqa: DTZ001
|
||||
bars, calls = await _fetch(
|
||||
_AscendingConn,
|
||||
start_date=base + datetime.timedelta(days=99),
|
||||
end_date=base + datetime.timedelta(days=109))
|
||||
|
||||
assert [b.datetime.date() for b in bars] == [
|
||||
(base + datetime.timedelta(days=i)).date() for i in range(99, 110)]
|
||||
assert calls > 10
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_backtrack_matches_plain_count_result():
|
||||
"""倒推裁剪结果 == 直接取一大段再裁剪(两条路径口径必须一致)。"""
|
||||
start = NEWEST - datetime.timedelta(days=7)
|
||||
by_date, _ = await _fetch(_FakeConn, start_date=start, end_date=NEWEST)
|
||||
whole, _ = await _fetch(_FakeConn, count=60)
|
||||
|
||||
expect = [b for b in whole if b.datetime >= start]
|
||||
assert [(b.datetime, b.close) for b in by_date] == \
|
||||
[(b.datetime, b.close) for b in expect]
|
||||
Reference in new issue
Block a user