Files
openbot-coder c077a7ae40 refactor: cleanup dead code, fix model confusion, streamline CLI
- Remove 3 dead modules: academic/, wechat/, social/ (merged into engines/)
- Remove 4 stale news scripts (tech_news.py, wallstreetcn.py, cls.py, rss.py)
- scripts/core/base.py: canonical SearchResult + BaseEngine (was in engines/base.py)
- scripts/engines/base.py: re-exports from core.base for compatibility
- scripts/engines/news_engine.py: Bing News + RSS unified (was in scripts/news/rss.py)
- scripts/engines/academic_engine.py: ArXiv + CrossRef unified (was in scripts/academic/arxiv.py)
- scripts/cli.py: complete rewrite — 10 commands (search/web/news/academic/wechat/social/rss/baidu/health/sources)
- scripts/unified_search.py: clean async Python API (was a stub)
- ws.py: lightweight launcher
- README.md + SKILL.md: updated to reflect actual 15+ engines
- All EngineResult → SearchResult
- All engines return list[SearchResult]
2026-05-30 16:25:41 +08:00

3.7 KiB

name, description, version, license, compatibility, metadata
name description version license compatibility metadata
web-search Multi-source web search — 15+ engines (web, news, academic, social, RSS) running concurrently in <5s. No API key needed for most engines. Use when user asks to search the web, find news, look up papers, check social media, or search WeChat. 3.0.0 MIT Python 3.10+, httpx, beautifulsoup4, lxml
author category tags
openbot-coder search
search
web
news
academic
social
rss
cli
async

Web Search Skills

Triggers

Use this skill when the user says:

Intent Example
Web search "搜索一下 xxx", "search the web for xxx"
News "查一下新闻", "搜索最新的科技新闻"
Academic "搜论文 xxx", "查学术资料"
WeChat "搜微信公众号 xxx"
Social "看看 Twitter 上关于"
Health check "检查一下搜索引擎状态"
List sources "有哪些搜索引擎可以用"

Commands

# Full search (all 15 engines in parallel, ~10s)
web-search search "AI news" -n 10

# Web only (6 engines, ~5s)
web-search web "Python async" -n 5

# News (Bing News + 15 RSS, ~8s)
web-search news "量子计算" -n 5

# Academic (ArXiv + CrossRef, ~6s)
web-search academic "machine learning"

# WeChat (Sogou, ~5s)
web-search wechat 蒙面财经

# Social (Nitter, ~5s)
web-search social "AI regulation" -n 10

# RSS feeds only
web-search rss "AI newsletter" -n 5

# Baidu Qianfan API (needs BAIDU_API_KEY)
web-search baidu "中美元首会晤"

# Engine health check
web-search health

# List all sources
web-search sources

Decision Tree

用户说了什么?
│
├─ "微信公众号" / "wechat"
│   → web-search wechat <query>
│
├─ "学术" / "论文" / "paper" / "ArXiv"
│   → web-search academic <query>
│
├─ "Twitter" / "X" / "社交"
│   → web-search social <query>
│
├─ "新闻" / "news"
│   → web-search news <query>
│
├─ "健康检查" / "health"
│   → web-search health
│
├─ "有哪些源" / "sources"
│   → web-search sources
│
└─ 通用搜索
    → web-search search <query> -n 10

Python API

from scripts.unified_search import UnifiedSearch

async def main():
    us = UnifiedSearch()
    resp = await us.search(
        "quantum computing",
        sources=["bing", "arxiv"],  # or None for all
        max_results=5,
    )
    for r in resp.results:
        print(f"[{r.rank}] {r.title}")
        print(f"    {r.url}")
        print(f"    {r.snippet[:100]}")
    print(f"Engines used: {resp.metadata.engines_used}")
    print(f"Duration: {resp.metadata.duration_ms}ms")
    await us.close()

SearchResult Fields

Field Type Description
rank int Position after deduplication (1-based)
title str Article/title text
url str Direct URL
snippet str Summary (~300 chars)
source str Engine name (e.g. "bing", "arxiv")
category str "web" | "news" | "academic" | "github"
published str ISO date (optional)
extra dict Engine-specific metadata

Error Handling

Symptom Cause Action
Empty Bing News Anti-scraping / region Not critical, skip
Baidu Qianfan empty Missing BAIDU_API_KEY Set env var
Sogou WeChat empty Sogou anti-scraping Show URL fallback
Nitter all fail Instance blocked Show direct twitter.com link

Notes

  • Most engines require no API key
  • Baidu Qianfan needs BAIDU_API_KEY env var
  • All engines run concurrently via asyncio.gather
  • Max 5 concurrent engines (semaphore)
  • Per-engine timeout: 15s
  • Results are deduplicated by URL before ranking