POST /v1/crawler:batch

Crawl new or recrawl existing URLs.

Payload

{
  "indexPublicKey": "SITEKEY",
  "urls": [
     "http://foo.com/bar1.html",
     "http://foo.com/bar2.html",
     "http://foo.com/bar3.html"
  ]
}

Field	Description	Type	Notes
indexPublicKey	The public site key	string	A sitekey is an identifier of your AddSearch account that can be found in the dashboard
urls	URLs of documents	string	A list of full URLs of your documents including protocol (http/https), domain, and path.

Response

{
  "documents_queued_for_crawling": [
    {
      "url": "http://foo.com/bar1.html",
      "docId": "aac7142057fcc5b55e6dd5b70969066d"
    },
    {
      "url": "http://foo.com/bar2.html",
      "docId": "59ead4c38325d6ee98a99c21dbd162f2"
    }
  ]
}

Field	Description	Type	Notes
documents_queued_for_crawling	List of documents scheduled for crawling	array	Each object in the array consists of a URL and id of the document scheduled for crawling

URLs are queued for recrawl and processed one by one with a small delay to prevent high load on your website’s server.

Returns HTTP 202 ACCEPTED with payload indicating the status of the request.

Please note that the Crawl API endpoint requires authentication. See authentication for more information.