Website Content Scanning

The Website Content Scanning feature allows you to automatically generate Q&A pairs from your website’s content to enhance your AI Assistant’s knowledge base. This premium feature helps you quickly train your chatbot with your website’s content.

Feature Overview

• Smart Page Search: Search and select specific pages from your website
• Automated Q&A Generation: Convert page content into relevant question-answer pairs
• Knowledge Base Integration: Automatically upload generated content to your AI Assistant
• Content Management: View and manage scanned pages in your knowledge base

Prerequisites

• Active Premium License
• Valid Assistant ID configuration
• Live, accessible web pages (not under maintenance or coming soon pages)

Using Website Content Scanning

1. Accessing the Scanner

• Navigate to AI Configuration in your WordPress admin panel
• Click the “Scan Website” button in the “Train Assistant” section
• The Website Content Scanner modal will open

2. Scanning Pages

• Search for Pages:

– Enter at least 3 letters in the search box
– The system will display matching pages from your website
– Click on a page from the search results to select it

• Generate Q&A Content:

– Click the “Generate Q&A” button after selecting a page
– The system will process the page content
– Generation may take several minutes for large pages

3. Processing Status

The system provides real-time status updates during the scanning process:

• “Generating Q&A content…” – Initial processing
• “Q&A content generated successfully” – Content creation complete
• “Uploading to Assistant API…” – Knowledge base integration
• “File uploaded successfully” – Process completion

Q&A Download Option

Enabling Downloads

• Locate the “Enable Q&A Download” option in the scanner interface
• Check the box to enable downloads
• Click “Save Setting” to apply the change

When enabled:

Managing Scanned Content

Website Pages List

Best Practices

• Page Selection:

– Choose content-rich pages for scanning
– Avoid scanning duplicate content
– Ensure pages are fully loaded and accessible

• Content Processing:

– For large pages, consider breaking content into smaller sections
– Wait for each processing task to complete before starting another
– Regularly review generated Q&A pairs for relevance

• Knowledge Base Management:

– Remove outdated scanned content
– Update important pages periodically
– Monitor the AI Assistant’s performance with the scanned content

Error Handling

Common issues and solutions:

| Error | Solution |
|——-|———-|
| “Page too large” | Break the content into smaller sections |
| “Server timeout” | Try processing during off-peak hours |
| “Error searching pages” | Verify the page is accessible and try again |
| “Upload failed” | Check your internet connection and retry |

Technical Notes

• Maximum processing time: 3 minutes per page
• Content is processed server-side
• Files are stored in the Assistant API’s knowledge base
• Supports both public and private pages on your website
• Real-time AJAX-based processing and status updates

License Requirements

• Premium feature requiring active license
• Expired licenses can view but not scan new pages
• License renewal restores full scanning capabilities

Benefits

• Time Efficiency:

– Automate Q&A pair creation
– Quickly build comprehensive knowledge base
– Reduce manual content entry

• Content Accuracy:

– Direct extraction from your website
– Maintains context and relevance
– Consistent information across platforms

• Easy Maintenance:

– Update content as your website changes
– Remove outdated information
– Track scanned pages efficiently