--- name: web-search description: "Multi-source web search — 15+ engines (web, news, academic, social, RSS) running concurrently in <5s. No API key needed for most engines. Use when user asks to search the web, find news, look up papers, check social media, or search WeChat." version: 3.0.0 license: MIT compatibility: Python 3.10+, httpx, beautifulsoup4, lxml metadata: author: openbot-coder category: search tags: [search, web, news, academic, social, rss, cli, async] --- # Web Search Skills ## Triggers Use this skill when the user says: | Intent | Example | |--------|---------| | Web search | "搜索一下 xxx", "search the web for xxx" | | News | "查一下新闻", "搜索最新的科技新闻" | | Academic | "搜论文 xxx", "查学术资料" | | WeChat | "搜微信公众号 xxx" | | Social | "看看 Twitter 上关于" | | Health check | "检查一下搜索引擎状态" | | List sources | "有哪些搜索引擎可以用" | ## Commands ```bash # Full search (all 15 engines in parallel, ~10s) web-search search "AI news" -n 10 # Web only (6 engines, ~5s) web-search web "Python async" -n 5 # News (Bing News + 15 RSS, ~8s) web-search news "量子计算" -n 5 # Academic (ArXiv + CrossRef, ~6s) web-search academic "machine learning" # WeChat (Sogou, ~5s) web-search wechat 蒙面财经 # Social (Nitter, ~5s) web-search social "AI regulation" -n 10 # RSS feeds only web-search rss "AI newsletter" -n 5 # Baidu Qianfan API (needs BAIDU_API_KEY) web-search baidu "中美元首会晤" # Engine health check web-search health # List all sources web-search sources ``` ## Decision Tree ``` 用户说了什么? │ ├─ "微信公众号" / "wechat" │ → web-search wechat │ ├─ "学术" / "论文" / "paper" / "ArXiv" │ → web-search academic │ ├─ "Twitter" / "X" / "社交" │ → web-search social │ ├─ "新闻" / "news" │ → web-search news │ ├─ "健康检查" / "health" │ → web-search health │ ├─ "有哪些源" / "sources" │ → web-search sources │ └─ 通用搜索 → web-search search -n 10 ``` ## Python API ```python from scripts.unified_search import UnifiedSearch async def main(): us = UnifiedSearch() resp = await us.search( "quantum computing", sources=["bing", "arxiv"], # or None for all max_results=5, ) for r in resp.results: print(f"[{r.rank}] {r.title}") print(f" {r.url}") print(f" {r.snippet[:100]}") print(f"Engines used: {resp.metadata.engines_used}") print(f"Duration: {resp.metadata.duration_ms}ms") await us.close() ``` ## SearchResult Fields | Field | Type | Description | |-------|------|-------------| | `rank` | int | Position after deduplication (1-based) | | `title` | str | Article/title text | | `url` | str | Direct URL | | `snippet` | str | Summary (~300 chars) | | `source` | str | Engine name (e.g. "bing", "arxiv") | | `category` | str | "web" \| "news" \| "academic" \| "github" | | `published` | str | ISO date (optional) | | `extra` | dict | Engine-specific metadata | ## Error Handling | Symptom | Cause | Action | |---------|-------|--------| | Empty Bing News | Anti-scraping / region | Not critical, skip | | Baidu Qianfan empty | Missing `BAIDU_API_KEY` | Set env var | | Sogou WeChat empty | Sogou anti-scraping | Show URL fallback | | Nitter all fail | Instance blocked | Show direct twitter.com link | ## Notes - Most engines require **no API key** - Baidu Qianfan needs `BAIDU_API_KEY` env var - All engines run concurrently via `asyncio.gather` - Max 5 concurrent engines (semaphore) - Per-engine timeout: 15s - Results are deduplicated by URL before ranking