Writing & Documentation
Real file from a public repo · synced Sep 3, 2026
crawl
Crawl any website and save pages as local markdown files. Use when you need to download documentation, knowledge bases, or web content for offline access or analysis. No code required - just provide a URL.
Works with
Claude CodeCursorCodex CLIGitHub CopilotGemini CLI
Install in three steps
- 1
Download the ZIP
One file per agent, plus INSTALL.md with the exact paths below.
- 2
Claude Code: copy SKILL.md into your skills folder
.claude/skills/crawl/SKILL.md # this project ~/.claude/skills/crawl/SKILL.md # every project
- 3
Other agents: use the matching file
skill.mdc goes in Cursor’s .cursor/rules/. AGENTS.md works as GitHub Copilot instructions, a legacy .cursorrules, or a Codex / Gemini CLI instructions file.

