refactor: restructure project with CLI tool, multi-engine support, and skill-builder compliant SKILL.md

- Replace setup.py with pyproject.toml (uv-compatible)
- Add CLI entry point (web-search) with 9 subcommands
- Add multi-engine architecture (14 web + 7 special sources)
- Add WeChat/CLS/WallStreetCN/ArXiv/Twitter search engines
- Remove old src/ and examples/ directories
- Remove redundant requirements.txt and setup.py
- Update README.md and SKILL.md with full documentation
- SKILL.md now compliant with skill-builder standards (frontmatter, triggers, structure)
This commit is contained in:
vex1023 committed 2026-05-14 16:31:54 +08:00
1 parent 80ac14491e
commit 87935b59ce
29 files changed
+2364 -354

No files matched your search

+68 -77
View File
@@ -1,95 +1,86 @@
# Web Search Skills
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
Multi-source web search CLI tool — search across 20+ engines from your terminal.
A collection of web search tools and skills for AI agents, designed to enhance search capabilities with customizable configurations, multiple search engines support, and intelligent result processing.
## Features
- **Multi-Engine Support** — Built-in support for Google, Bing, DuckDuckGo, and custom search backends
- **Intelligent Result Processing** — Extract, summarize, and rank search results automatically
- **Configurable Filters** — Filter by date, domain, language, region, and content type
- **Async & Concurrent** — Non-blocking requests with concurrent search execution
- **Extensible Architecture** — Plugin-based design for easy integration of new search sources
- **Agent-Ready** — Designed as a CodeBuddy Agent Skill for seamless AI integration
## Getting Started
### Prerequisites
- Python 3.8+
- pip / poetry
### Installation
## Quick Start
```bash
git clone https://github.com/openbot-coder/web-search-skills.git
# Install with uv (recommended)
cd web-search-skills
pip install -r requirements.txt
uv tool install -e .
# Now use globally from anywhere
web-search wechat 蒙面财经
web-search news "AI" -n 5
web-search search "量子计算" -o results.json
```
### Quick Start
```python
import asyncio
from src.web_search import search_web
results = asyncio.run(search_web("latest AI developments"))
for result in results:
print(f"- {result.title}: {result.url}")
Or run directly without install:
```bash
python ws.py wechat 蒙面财经
```
## Usage
```bash
web-search <command> <query> [options]
```
| Command | Description |
|------------|------------------------------------------|
| `search` | Search all sources (default) |
| `web` | Web search engines (Google, Baidu, DDG) |
| `news` | News (财联社, 华尔街见闻, RSS) |
| `wechat` | WeChat public account articles |
| `academic` | Academic papers (ArXiv) |
| `social` | Social media (Twitter/X) |
| `rss` | RSS news feeds |
| `sources` | List all available search sources |
| `urls` | Generate search URLs without executing |
Options:
| Flag | Description |
|------|-------------|
| `-n N` | Max results per source (default: 10) |
| `-j` | JSON output |
| `-o FILE` | Save to JSON file |
| `-s NAME` | Specific source name(s) |
| `-r cn/global` | Region filter (web search) |
| `-v` | Verbose/debug logs |
## Project Structure
```
web-search-skills/
├── README.md # Project documentation
├── LICENSE # MIT License
├── skill.md # CodeBuddy Agent Skill definition
├── requirements.txt # Python dependencies
├── src/ # Source code
│ └── web_search.py # Core search implementation
└── examples/ # Usage examples
└── basic_search.py
├── scripts/
│ ├── cli.py # CLI entry point
│ ├── unified_search.py # Unified search orchestrator
│ ├── core/
│ │ ├── base.py # SearchResult & SearchEngine base class
│ │ └── config_loader.py # Load engine config from JSON
│ ├── engines/
│ │ ├── url_engine.py # URL-template search engines
│ │ └── parser_engines.py # HTML-parsed search (DuckDuckGo)
│ ├── news/
│ │ ├── cls.py # 财联社
│ │ ├── wallstreetcn.py # 华尔街见闻
│ │ └── rss.py # RSS feeds
│ ├── academic/
│ │ └── arxiv.py # ArXiv papers
│ ├── wechat/
│ │ └── sogou_weixin.py # WeChat (via Sogou)
│ └── social/
│ └── twitter.py # Twitter/X
├── config/
│ └── engines.json # Search engine definitions
├── ws.py # Direct run entry point
├── pyproject.toml # Package config
├── SKILL.md # CodeBuddy skill definition
├── LICENSE
└── README.md
```
## Usage
### As a CodeBuddy Agent Skill
This project is designed as a CodeBuddy Agent Skill. To install:
1. Copy `skill.md` to your CodeBuddy skills directory
2. Configure your search engine API keys
3. The skill is now available to your AI agent
### API Reference
```python
# Core class
searcher = WebSearch(config=None)
results = await searcher.search(query, engine="duckduckgo", max_results=10)
content = await searcher.extract_content(url)
# Convenience function
results = await search_web(query, engine="duckduckgo", max_results=10)
```
## Contributing
We welcome contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
1. Fork the repository
2. Create your feature branch (`git checkout -b feature/amazing-feature`)
3. Commit your changes
4. Push to the branch
5. Open a Pull Request
## License
This project is licensed under the MIT License — see the [LICENSE](LICENSE) file for details.
## Acknowledgments
- Built for the CodeBuddy ecosystem
- Inspired by the need for better web search capabilities in AI agents
MIT