parsew-scrape
Scrape a single web page and convert it to markdown using the Parsew API. Use when the user wants to scrape a URL, fetch page content, convert a webpage to markdown, grab text from a website, read a web page, or mentions Parsew scrape. Also use when the user wants to get the HTML or links from a page. For multiple URLs, see parsew-batch-scrape. For structured data extraction with a schema, see parsew-extract.
Works with
Agent Skills format with YAML frontmatter. Claude Code reads it as-is.
---
name: "parsew-scrape"
description: "Scrape a single web page and convert it to markdown using the Parsew API. Use when the user wants to scrape a URL, fetch page content, convert a webpage to markdown, grab text from a website, read a web page, or mentions Parsew scrape. Also use when the user wants to get the HTML or links from a page. For multiple URLs, see parsew-batch-scrape. For structured data extraction with a schema, see parsew-extract."
license: "MIT"
---
## Authentication
Requires a `PARSEW_API_KEY` environment variable with a secret key (`sr_` prefix).
If the key is not set, tell the user to get one from https://dash.parsew.com and set it:
```bash
export PARSEW_API_KEY=sr_your_secret_key
```
## Endpoint
`POST https://api.parsew.com/v1/scrape` — 1 credit per request.
### Parameters
| Parameter | Type | Required | Description |
|-----------|------|----------|-------------|
| `url` | string | Yes | The URL to scrape |
| `waitForSelector` | string | No | CSS selector to wait for before returning (useful for JS-rendered content) |
| `waitFor` | integer | No | Max wait time in milliseconds (1–60,000) |
| `actions` | array | No | Up to 20 browser actions to perform before scraping (see Actions below) |
### Response
```json
{
"markdown": "# Page Title\n\nPage content as markdown...",
"html": "<html>...</html>",
"links": ["https://example.com/page1", "https://example.com/page2"],
"warning": "Optional warning message"
}
```
## Usage
### With the SDK (`@parsew/sdk`)
```typescript
import { Parsew } from '@parsew/sdk/server'
const parsew = new Parsew({ apiKey: process.env.PARSEW_API_KEY })
const result = await parsew.scrape('https://example.com')
console.log(result.markdown)
```
With options:
```typescript
const result = await parsew.scrape('https://example.com', {
waitForSelector: '#content',
waitFor: 5000,
actions: [
{ type: 'wait', milliseconds: 2000 },
{ type: 'click', selector: '.load-more' },
{ type: 'scroll', direction: 'down' },
],
})
```
### With curl
```bash
curl -X POST https://api.parsew.com/v1/scrape \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $PARSEW_API_KEY" \
-d '{"url": "https://example.com"}'
```
## Actions
Actions are browser interactions performed before scraping. Max 20 per request.
| Action | Fields | Description |
|--------|--------|-------------|
| `scroll` | `direction`: `"up"` or `"down"` | Scroll the page |
| `wait` | `milliseconds`: 1–60,000 | Wait for a duration |
| `click` | `selector`: CSS selector | Click an element |
Use actions for pages that need interaction to load content — clicking "load more" buttons, dismissing modals, scrolling to trigger lazy loading.
## Error Handling
| Status | Meaning | Action |
|--------|---------|--------|
| 401 | Invalid or missing API key | Check `PARSEW_API_KEY` is set correctly |
| 429 | Rate limit or insufficient credits | Back off and retry, or check credit balance |
| 502 | Upstream scrape error | The target site may be blocking or down |
| 504 | Scrape timed out | Increase `waitFor` or simplify the target page |
## When to Use Something Else
- **Multiple URLs?** Use `parsew-batch-scrape` to scrape 1–100 URLs asynchronously.
- **Need structured/typed data?** Use `parsew-extract` to extract data matching a schema (costs 5 credits). But if the data is straightforward, it's cheaper to scrape (1 credit) and parse the markdown yourself.
- **Just need links?** Use `parsew-map` for link discovery with filtering.More General & Other skills
find-skills
vercel-labs/skills
Helps users discover and install agent skills when they ask questions like "how do I do X", "find a skill for X", "is there a skill that can...", or express interest in extending capabilities. This skill should be used when the user is looking for functionality that might exist as an installable skill.
grill-me
mattpocock/skills
A relentless interview to sharpen a plan or design.
grill-with-docs
mattpocock/skills
A relentless interview to sharpen a plan or design, which also creates docs (ADR's and glossary) as we go.
