POST /v1/crawler:batch
Crawl new or recrawl existing URLs. This endpoint is only available for Advanced and Enterprise plans. Please, contact support to enable the feature.
Payload
{
"indexPublicKey": "SITEKEY",
"urls": [
"http://foo.com/bar1.html",
"http://foo.com/bar2.html",
"http://foo.com/bar3.html"
]
}
| Field | Description | Type | Notes |
|---|---|---|---|
| indexPublicKey | The public site key | string | A sitekey is an identifier of your AddSearch account that can be found in the dashboard |
| urls | URLs of documents | string | A list of full URLs of your documents including protocol (http/https), domain, and path. |
Response
{
"documents_queued_for_crawling": [
{
"url": "http://foo.com/bar1.html",
"docId": "aac7142057fcc5b55e6dd5b70969066d"
},
{
"url": "http://foo.com/bar2.html",
"docId": "59ead4c38325d6ee98a99c21dbd162f2"
}
]
}
| Field | Description | Type | Notes |
|---|---|---|---|
| documents_queued_for_crawling | List of documents scheduled for crawling | array | Each object in the array consists of a URL and id of the document scheduled for crawling |
URLs are queued for recrawl and processed one by one with a small delay to prevent high load on your website’s server.
Returns HTTP 202 ACCEPTED with payload indicating the status of the request.
Please note that the Crawl API endpoint requires authentication. See authentication for more information.
