AddSearch
  • Product
    • Overview
    • Result Management
    • Design & Search UI
    • Analytics
    • Personalization
    • Crawler
    • WordPress Plugin
    • Book a demo
  • Pricing
  • Customers
    • Overview
    • Enterprise
    • Ecommerce
    • Media
    • B2B
    • Docs and Helpdesk
    • Education
  • For Agencies
  • Support
    • Documentation
    • API Reference
    • Product Updates
    • FAQ
    • Blog
    • Webinars
    • Contact Support
Login Signup
  • Product
    • Overview
    • Result Management
    • Design & Search UI
    • Analytics
    • Personalization
    • Crawler
    • WordPress Plugin
    • Book a demo
  • Pricing
  • Customers
    • Overview
    • Enterprise
    • Ecommerce
    • Media
    • B2B
    • Docs and Helpdesk
    • Education
  • For Agencies
  • Support
    • Documentation
    • API Reference
    • Product Updates
    • FAQ
    • Blog
    • Webinars
    • Contact Support
AddSearch
Login Signup
  • Indexing and Crawling
    • Choosing the Search Index
    • Adding Multiple Domains to Search
    • Determining the Language of Webpages
    • Indexing PDF, DOCX and PPTX Files
    • Whitelisting AddSearch Bot
    • Crawling adjustment
    • Indexing and Updating Single Pages
    • Real-time Indexing with RSS Feeds or API
    • Excluding Content
    • Restricting Search to a Certain Area
    • Crawling content within pages
    • Defining Featured Image for Thumbnails
    • Defining Page’s Publishing Time
    • Defining Custom Fields
    • Using Custom Fields Manager
  • Building Search UI
    • Different Search Result Views
    • Custom Implementation
    • Go to API Reference
    • Search UI Library
    • Go to Search UI Library (GitHub)
    • Introduction
    • Getting Started
    • Create Autocomplete Suggestions
    • Ready-Made Views
    • Search Widget Settings
    • Separate Results Page Settings
    • Look and Feel of Ready-Made Views
    • Customizing Look & Feel
    • Defining Category Information with Metatags
    • Defining Category Information with Aliases
    • Hiding Thumbnails and Screen Captures from Search Results
    • Showing Meta Description in Search Results
    • Additional Settings
    • Implementing Search Suggestions Autocomplete
    • Search Filters
  • Result Management and Ranking
    • Search Algorithm
    • Ranking Adjustment
    • Site Areas – Boost Ranking or Exclude a Site Section
    • Pinned Results – Curate Results for Specific Keywords
    • Promotions – Add Promotional Banners to Your Search
    • Relevancy
    • What are Custom Fields?
    • Configuring Search Suggestions Autocomplete
    • Synonyms – Add Keywords with Similar Meaning
    • Stemming and Language Settings
    • Hidden Keywords – Add New Keywords to Pages
  • Installation
    • Supported CMS Platforms
    • Basic Installation Guidelines
    • CMS-Specific Guidelines
    • WordPress
    • WooCommerce
    • Jimdo
    • Shopify
    • HubSpot
  • Analytics
    • Statistics Dashboard
    • Integrating with Google Analytics
    • Excluding AddSearch Bot from Google Analytics Traffic
    • Getting Analytics Data from Search API Usage
  • Personalization
    • Overview
  • Account Management
    • Users
    • User Roles and Management
    • Multi-Factor Authentication
    • Billing
    • Subscribing to AddSearch
    • Adding Another AddSearch Account
    • Updating Billing Information
    • Billing and Invoicing
    • Canceling Subscription

Indexing and Crawling

AddSearch crawls your website and indexes the content for the search. Read more about our crawlers and follow the instructions on adding content for the search.

  • Adding Multiple Domains to Search
  • AJAX Crawling – Indexing of Dynamic Content
  • Authenticated Crawling
  • Canonical Links
  • Choosing the Search Index
  • Crawling Frequency
  • Defining Custom Fields
  • Defining Featured Image for Thumbnails
  • Defining Page’s Publishing Time
  • Determining the Language of Webpages
  • Excluding Content
  • Excluding Single Page
  • Exporting Indexed URLs and IDs
  • Failure during Demo Creation
  • Index Meta Keywords
  • Indexing and Filtering PDFs, Microsoft Word and Powerpoint Documents
  • Indexing and Updating Single Pages
  • Indexing Status
  • Real-time Indexing with RSS Feeds or API
  • Requesting Full Recrawl of Search Index
  • Restricting Search to a Certain Area of Your Website
  • Setting Up Ignored URL Parameters
  • Use Meta Tags to Optimize Your Search Results
  • Using Custom Fields Manager
  • Using Text Extraction Rules
  • Whitelisting AddSearch Bot
  • Whitelisting AddSearch to Work with Content Security Policy

Was this helpful?

Let us know if you liked the post. That’s the only way we can improve.
Need more help?

We’re always happy to help with code or other questions you might have. Search our documentation, contact support, or connect with our sales team.

AddSearch
  • Product
  • Product Updates
  • Pricing
  • Partners
  • Documentation
  • API Reference
  • Customers
  • Contact Support
  • Blog
  • Webinars
  • About AddSearch
  • Careers
  • Contact Info
  • Privacy Policy
  • Terms & Conditions
AddSearch Oy
PL 8
FI-02611 Espoo
Finland
info@addsearch.com
Corporate ID #: 2545782-4
VAT ID #: FI25457824