refactor: cleanup dead code, fix model confusion, streamline CLI

- Remove 3 dead modules: academic/, wechat/, social/ (merged into engines/)
- Remove 4 stale news scripts (tech_news.py, wallstreetcn.py, cls.py, rss.py)
- scripts/core/base.py: canonical SearchResult + BaseEngine (was in engines/base.py)
- scripts/engines/base.py: re-exports from core.base for compatibility
- scripts/engines/news_engine.py: Bing News + RSS unified (was in scripts/news/rss.py)
- scripts/engines/academic_engine.py: ArXiv + CrossRef unified (was in scripts/academic/arxiv.py)
- scripts/cli.py: complete rewrite — 10 commands (search/web/news/academic/wechat/social/rss/baidu/health/sources)
- scripts/unified_search.py: clean async Python API (was a stub)
- ws.py: lightweight launcher
- README.md + SKILL.md: updated to reflect actual 15+ engines
- All EngineResult → SearchResult
- All engines return list[SearchResult]
This commit is contained in:
openbot-coder committed 2026-05-30 16:25:41 +08:00
1 parent e20478fcbd
commit c077a7ae40
31 files changed
+1188 -3148

No files matched your search

+10 -13
View File
@@ -1,31 +1,31 @@
"""
Generic URL-based search engine.
Builds search URLs from templates defined in engines.json
Builds search URLs from templates (config/engines.json)
and returns them as URL results.
"""
from __future__ import annotations
import logging
from typing import Any, Optional
from urllib.parse import quote_plus
from typing import Any
from scripts.core.base import SearchEngine, SearchResult
logger = logging.getLogger(__name__)
from scripts.core.base import BaseEngine, SearchResult
class UrlSearchEngine(SearchEngine):
class UrlSearchEngine(BaseEngine):
"""
Generic search engine that builds URLs from templates.
Generic engine that builds search URLs from templates.
Reads URL templates from config (engines.json) and returns
the constructed search URL as a SearchResult.
"""
def __init__(self, name: str, engine_def: dict[str, Any]):
super().__init__(name, engine_def.get("config", {}))
name: str = "url"
def __init__(self, name: str, engine_def: dict[str, Any], timeout: float = 10.0):
super().__init__(timeout=timeout)
self.name = name
self.url_template = engine_def.get("url", "")
self.region = engine_def.get("region", "global")
@@ -55,6 +55,3 @@ class UrlSearchEngine(SearchEngine):
extra={"url": url, "region": self.region},
)
]
async def close(self):
pass