Back to Dashboard
Managing

    Crawl

    Crawl your website and add content to your knowledge base

    Selling products?
    If your website has a product catalogue, connect it as a data feed instead of crawling it. Your AI then always has current products, prices and stock, and your crawl stays free for your pages and guides.
    Set up data feed
    No websites found. Please add a website first.
    Crawl Mode
    Crawl a Specific Subdomain or URL By default the crawl starts at your website's homepage. Enable this to crawl a subdomain instead (e.g. your help center) — it must belong to the selected website.

    You can also paste a sitemap.xml URL — every page listed in it will be crawled.

    Password Protected Pages Works with standard login forms. May not work with JavaScript-based logins, CAPTCHA, 2FA, or OAuth.
    Advanced: Custom field names

    Only change these if the login form uses field names other than username and password, for example email.

    Maximum Pages