# web-search-skills > Multi-source web search CLI & Python library — 15+ engines, <5s, no API key required. ## Features - **15+ engines** across 5 categories: Web, News, Academic, Social, RSS - **Concurrent execution** via `asyncio.gather` with semaphore (max 5 parallel) - **Free by default** — most engines work without any API key - **Python API** for programmatic use (`UnifiedSearch`) - **Single model** — `SearchResult` dataclass across all engines ## Quick Start ```bash # Install pip install -e . # CLI web-search search "AI news" -n 10 web-search news "quantum computing" -n 5 web-search academic "machine learning" web-search wechat 蒙面财经 web-search health ``` ## Commands | Command | Description | Speed | |---------|------------|-------| | `search` | All 15 engines in parallel | ~10s | | `web` | Web engines only (6) | ~5s | | `news` | Bing News + 15 RSS feeds | ~8s | | `academic` | ArXiv + CrossRef in parallel | ~6s | | `wechat` | WeChat via Sogou | ~5s | | `social` | Twitter/X via Nitter | ~5s | | `rss` | RSS feed aggregation | ~8s | | `baidu` | Baidu Qianfan API (needs key) | varies | | `health` | Engine connectivity check | ~10s | | `sources` | List all sources | instant | ## Options - `-n N` — max results per engine (default: 10) - `-s SRC...` — run specific sources only - `-c CAT` — run by category (web/news/academic/social/rss) - `-r cn|global` — region (default: cn) - `-j` — JSON output ## Python API ```python from scripts.unified_search import UnifiedSearch async def main(): us = UnifiedSearch() resp = await us.search("quantum computing", sources=["bing", "arxiv"], max_results=5) for r in resp.results: print(f"[{r.rank}] {r.title}") print(f" {r.url}") await us.close() ``` ## Architecture ``` scripts/ cli.py # CLI entry point unified_search.py # Python API models.py # Data models config.py # Settings core/ base.py # SearchResult + BaseEngine (canonical) aggregator.py # Deduplication + ranking execution_log.py # Timing + error tracking engines/ bing_scraper.py # Bing web search sogou_scraper.py # Sogou web search baidu_scraper.py # Baidu web search search360_scraper.py # 360 search parser_engines.py # DuckDuckGo + Brave (HTML parser) news_engine.py # Bing News + RSS aggregator academic_engine.py # ArXiv + CrossRef github_engine.py # GitHub API baidu_qianfan.py # Baidu Qianfan API (needs key) parallel_search.py # Parallel orchestrator ``` ## Engines ### Web (无需 API key) | Engine | URL | Notes | |--------|-----|-------| | Bing CN | cn.bing.com | Primary | | Bing INT | www.bing.com | Global fallback | | Sogou | sogou.com | Primary CN | | Baidu | baidu.com | Primary CN | | 360 | so.com | CN fallback | | DuckDuckGo | html.duckduckgo.com | Global | | Brave | search.brave.com | Global | ### News | Engine | Source | |--------|--------| | Bing News | bing.com/news | | RSS (15 feeds) | 36kr, IT之家, 钛媒体, HackerNews, TechCrunch... | ### Academic (Free APIs) | Engine | API | |--------|-----| | ArXiv | export.arxiv.org/api/query | | CrossRef | api.crossref.org | ### Social | Engine | Notes | |--------|-------| | Nitter | twitter.com 免登录 (多个实例自动切换) | ### API (需配置密钥) | Engine | Env var | |--------|---------| | Baidu Qianfan | `BAIDU_API_KEY` | ## SearchResult Model ```python @dataclass class SearchResult: title: str # Article/title text url: str # Direct URL snippet: str # Summary text (~300 chars) source: str # Engine name (e.g. "bing", "arxiv") category: str # "web" | "news" | "academic" | "github" rank: int # Position after deduplication published: str # ISO date (optional) extra: dict # Engine-specific metadata ```