Overview
Scan Website (WPiko AI Chatbot Pro) adds the content of your website to your chatbot’s knowledge base. It lists your pages, blog posts and the articles of documentation or knowledge-base plugins, so you can pick what the chatbot should know and scan it all in one run. There’s no limit on how much you can scan.
Each content type is handled the way that suits it:
- Pages are rewritten into questions and answers by AI. Q&A pairs help the chatbot find precise answers in marketing-style pages. This uses your OpenAI credits.
- Posts, docs and other articles are already written to be read, so they’re added as clean Markdown with a short header, with no AI request. They’re free and instant to scan, and steps, code samples, tables and links keep their exact wording.
Scan Website vs. Quick Learn
| Quick learn (free) | Scan Website (Pro) | |
|---|---|---|
| Content | Published pages | Pages, posts, docs and other articles |
| How it works | Copies the page text | Pages: AI writes Q&A Posts and docs: clean Markdown |
| Speed and cost | Ready in seconds, no AI cost | Pages use AI credits; posts and docs are free and instant |
| How many | Up to 10 pages | Any number |
You can use both. When you scan a page, it’s automatically removed from Quick learn, so nothing is stored twice. If you remove a scanned page from the knowledge base, Quick learn shows it as available again.
Requirements
- WPiko AI Chatbot and WPiko AI Chatbot Pro installed, with an active Pro license
- A working OpenAI connection (see API Key Configuration)
Your Content List
Go to WPiko AI Chatbot → AI & Training → AI Settings → Model & Knowledge and click Scan Website in the Train AI assistant section. The window lists your content with its scan state:
| State | Meaning |
|---|---|
| Not scanned | Not in the knowledge base yet |
| Up to date | Scanned, and not edited since |
| Changed since scan | Edited after it was scanned. Rescan it so the chatbot learns the new content. |
| Rescan needed | Scanned in a different format (for example a post scanned as Q&A). Rescan it to use the appropriate format for its content type. |
To find items quickly:
- Content type: switch between pages, posts and the content types of your documentation or knowledge-base plugins. Each plugin’s types are grouped under its own name and icon.
- Filter by title: type part of a title.
- Status filters: All, Not scanned, Changed or Scanned.
Every public content type with an editor is listed, with no plugin-specific setup. WooCommerce products, media files and page-builder templates are left out, and so are WooCommerce’s Cart, Checkout, My account and Shop pages, which have no text to learn from. Products stay in sync through the WooCommerce integration instead. Up to 500 items of each type are listed.
Published, private, draft and pending items are listed. Private and password-protected items have an amber tag: once scanned, the chatbot can share their content with any visitor, so only scan them if that’s fine.
Scanning Your Content
- Tick the items you want to scan, or tick Select all pages shown to select every item in the current list.
- Click Scan.
- Each item shows its progress while it’s processed. Click Stop to stop after the current item.
- At the end, a summary shows how many items were added. Items that failed stay selected, so you can click Scan again to retry them.
Selecting 5 or more pages asks for confirmation first, because each page is sent to OpenAI to write Q&A, which uses your API credits. Posts and docs never need confirmation. If you close the window while a scan is running, you’re asked whether to stop it.
Pages: AI-Written Q&A
Pages are rewritten into questions and answers with GPT-6.1 Terra using Medium reasoning and Medium verbosity, independently of the model and settings selected for chatbot replies. Large pages are split into parts automatically, with the same model and settings used for each part. If any part fails, the whole result is discarded instead of saving incomplete Q&A, so just try again. Links in the Q&A are kept exactly as written, including percent-encoded addresses such as Greek or other non-Latin page URLs.
Posts and Docs: Clean Markdown
Posts, docs and other articles are converted to Markdown with no AI request. Each file starts with a short header so the chatbot knows what it’s reading:
- Title and link
- Content type
- Publish and last-updated dates
- Categories, tags and other taxonomies
Blog posts also note their publish date, so answers can mention that details may have changed since then.
Keeping Posts and Docs Up to Date
Turn on Keep scanned posts and docs up to date (off by default) to update the knowledge base automatically:
- When you edit a post or doc that’s already in the knowledge base, it’s updated about a minute later, with no AI cost.
- Moving it to the trash removes it from the knowledge base.
- Updates run in the background in small batches, so saving stays fast and bulk edits work on shared hosting. Saves that don’t change the text aren’t uploaded again.
Pages aren’t included, because their Q&A is written by AI and uses credits. Rescan changed pages yourself: the Changed filter shows which ones.
Managing Scanned Content
Each scanned item has its own actions in the list:
- Rescan: scan it again after you change it.
- View page: open it on your site.
- Remove from knowledge base: the chatbot stops using it. You can scan it again at any time.
Other scanned files lists files whose content was deleted or is no longer listed, and files scanned before content was tracked. Delete the ones you no longer need.
Rescanning a renamed page replaces its previous file, and two pages with the same title each keep their own file.
Downloading a Copy
Turn on Download a copy of each scanned file to have your browser also save each file you scan (Q&A for pages, Markdown for posts and docs), so you can review what the chatbot learned. The setting saves as soon as you change it.
License
If your license shows as expired (for example when a free trial ends), you can still view and manage existing scanned content, but you can’t scan new items until you activate a lifetime license. Licenses bought on wpiko.com never expire.
For Developers
| Filter | Use it to |
|---|---|
wpiko_chatbot_pro_scan_post_types | Change which content types are listed |
wpiko_chatbot_pro_scan_mode | Return qa or markdown to choose how an item is scanned |
wpiko_chatbot_pro_markdown_header | Change the header lines of Markdown files |
wpiko_chatbot_pro_scan_excluded_ids | Hide more items, for example other plugins’ account pages |
wpiko_chatbot_pro_scan_pages_limit | Change how many items of each type are listed (default 500) |
wpiko_chatbot_pro_auto_sync_delay, wpiko_chatbot_pro_auto_sync_batch | Tune automatic updates (default 60 seconds, 5 items per batch) |
wpiko_chatbot_qa_chunk_threshold, _chunk_size, _chunk_overlap | Tune how large pages are split for Q&A |
wpiko_chatbot_ai_feature_config | Change the model, reasoning and verbosity for Scan Website Q&A using the qa_generation feature |
See Developer Hooks and Filters for examples.
Troubleshooting
- An item failed: it stays selected, so click Scan to retry. Check Advanced → Debug Log for the OpenAI error, for example no credit or a rate limit.
- “Timed out. The page may be too long”: try again. Large pages are split automatically, but a very slow OpenAI response can still time out.
- Something isn’t listed: check the Content type selector and the status filter. WooCommerce products and store pages, media files and page-builder templates are never listed.
- Upload failed: check the OpenAI connection with Test connection in AI Settings → API Connection, then retry.
- Chatbot still gives old answers: rescan the item (use the Changed filter to find edited ones), or turn on automatic updates for posts and docs.
