Files
web-search-skills/SKILL.md
T
openbot-coder c077a7ae40 refactor: cleanup dead code, fix model confusion, streamline CLI
- Remove 3 dead modules: academic/, wechat/, social/ (merged into engines/)
- Remove 4 stale news scripts (tech_news.py, wallstreetcn.py, cls.py, rss.py)
- scripts/core/base.py: canonical SearchResult + BaseEngine (was in engines/base.py)
- scripts/engines/base.py: re-exports from core.base for compatibility
- scripts/engines/news_engine.py: Bing News + RSS unified (was in scripts/news/rss.py)
- scripts/engines/academic_engine.py: ArXiv + CrossRef unified (was in scripts/academic/arxiv.py)
- scripts/cli.py: complete rewrite — 10 commands (search/web/news/academic/wechat/social/rss/baidu/health/sources)
- scripts/unified_search.py: clean async Python API (was a stub)
- ws.py: lightweight launcher
- README.md + SKILL.md: updated to reflect actual 15+ engines
- All EngineResult → SearchResult
- All engines return list[SearchResult]
2026-05-30 16:25:41 +08:00

141 lines
3.7 KiB
Markdown

---
name: web-search
description: "Multi-source web search — 15+ engines (web, news, academic, social, RSS) running concurrently in <5s. No API key needed for most engines. Use when user asks to search the web, find news, look up papers, check social media, or search WeChat."
version: 3.0.0
license: MIT
compatibility: Python 3.10+, httpx, beautifulsoup4, lxml
metadata:
author: openbot-coder
category: search
tags: [search, web, news, academic, social, rss, cli, async]
---
# Web Search Skills
## Triggers
Use this skill when the user says:
| Intent | Example |
|--------|---------|
| Web search | "搜索一下 xxx", "search the web for xxx" |
| News | "查一下新闻", "搜索最新的科技新闻" |
| Academic | "搜论文 xxx", "查学术资料" |
| WeChat | "搜微信公众号 xxx" |
| Social | "看看 Twitter 上关于" |
| Health check | "检查一下搜索引擎状态" |
| List sources | "有哪些搜索引擎可以用" |
## Commands
```bash
# Full search (all 15 engines in parallel, ~10s)
web-search search "AI news" -n 10
# Web only (6 engines, ~5s)
web-search web "Python async" -n 5
# News (Bing News + 15 RSS, ~8s)
web-search news "量子计算" -n 5
# Academic (ArXiv + CrossRef, ~6s)
web-search academic "machine learning"
# WeChat (Sogou, ~5s)
web-search wechat 蒙面财经
# Social (Nitter, ~5s)
web-search social "AI regulation" -n 10
# RSS feeds only
web-search rss "AI newsletter" -n 5
# Baidu Qianfan API (needs BAIDU_API_KEY)
web-search baidu "中美元首会晤"
# Engine health check
web-search health
# List all sources
web-search sources
```
## Decision Tree
```
用户说了什么?
│
├─ "微信公众号" / "wechat"
│ → web-search wechat <query>
│
├─ "学术" / "论文" / "paper" / "ArXiv"
│ → web-search academic <query>
│
├─ "Twitter" / "X" / "社交"
│ → web-search social <query>
│
├─ "新闻" / "news"
│ → web-search news <query>
│
├─ "健康检查" / "health"
│ → web-search health
│
├─ "有哪些源" / "sources"
│ → web-search sources
│
└─ 通用搜索
→ web-search search <query> -n 10
```
## Python API
```python
from scripts.unified_search import UnifiedSearch
async def main():
us = UnifiedSearch()
resp = await us.search(
"quantum computing",
sources=["bing", "arxiv"], # or None for all
max_results=5,
)
for r in resp.results:
print(f"[{r.rank}] {r.title}")
print(f" {r.url}")
print(f" {r.snippet[:100]}")
print(f"Engines used: {resp.metadata.engines_used}")
print(f"Duration: {resp.metadata.duration_ms}ms")
await us.close()
```
## SearchResult Fields
| Field | Type | Description |
|-------|------|-------------|
| `rank` | int | Position after deduplication (1-based) |
| `title` | str | Article/title text |
| `url` | str | Direct URL |
| `snippet` | str | Summary (~300 chars) |
| `source` | str | Engine name (e.g. "bing", "arxiv") |
| `category` | str | "web" \| "news" \| "academic" \| "github" |
| `published` | str | ISO date (optional) |
| `extra` | dict | Engine-specific metadata |
## Error Handling
| Symptom | Cause | Action |
|---------|-------|--------|
| Empty Bing News | Anti-scraping / region | Not critical, skip |
| Baidu Qianfan empty | Missing `BAIDU_API_KEY` | Set env var |
| Sogou WeChat empty | Sogou anti-scraping | Show URL fallback |
| Nitter all fail | Instance blocked | Show direct twitter.com link |
## Notes
- Most engines require **no API key**
- Baidu Qianfan needs `BAIDU_API_KEY` env var
- All engines run concurrently via `asyncio.gather`
- Max 5 concurrent engines (semaphore)
- Per-engine timeout: 15s
- Results are deduplicated by URL before ranking