Skip to main content
Create powerful AI Agents that can answer questions and provide information based on website content. This process automatically scrapes, processes, and trains your website content to build a smart assistant.

​
What you can do with this

  • Build customer support agents that answer questions using your help documentation
  • Create internal knowledge assistants to help employees find information quickly
  • Develop product information bots that provide detailed specs and features
  • Design training assistants for interactive learning experiences
  • Ensure compliance with accurate information delivery for regulatory requirements

​
What you get

Knowledge-powered intelligence
  • Instant expertise: Your Agent immediately knows your content and can answer related questions
  • Contextual responses: Provides accurate answers based on your specific information
  • Content updates: Automatically stays current as you update your Knowledge Base
  • Source citations: Can reference specific documents or sections when answering
  • Domain expertise: Specialized knowledge in your specific field or industry

​
Step-by-step creation process

1

Click Create from Knowledge card

Navigate to the Create Agent page and click on the Create from Knowledge card to begin the process.
Create Agent page showing the Create from Knowledge card option
This opens a modal dialog where you can configure your knowledge-powered Agent.
2

Select Website data source and customize Agent

In the modal, select Website as the data source for training your Agent’s Knowledge Base and customize the Agent’s appearance:
Data source selection and agent customization options
Customization options:
  • Data Source: Choose Website to train from web content
  • Agent Icon: Select or change the icon representing your Agent
You can choose from various data sources and customize your Agent’s icon. Additional data source options will be introduced in future updates.
3

Configure Knowledge Base settings

Enter the Knowledge Base Name and Website Base URL, then configure advanced scraping settings if needed.
  • Basic Configuration
  • Advanced Settings
Basic Knowledge Base configuration showing name and URL fields
Optimal Settings: For most websites, the default settings (200 URLs, 20 levels deep) work well. Use the browser option for JavaScript-heavy sites, and choose legacy scraper if you encounter compatibility issues with the modern Crawl4AI engine.
4

Fetch and process content

Click the Fetch Links button to begin the web scraping process.
  • Pre-fetch
  • While fetching
Pre-fetch state showing configuration before starting the scraping process
Scraping process: - Content discovery: System discovers and crawls web pages, extracting content from each page - Data processing: Cleans and organizes extracted content for optimal training preparation
The fetching process may take several minutes depending on the size of your website and the number of pages to scrape.
5

Train the Knowledge Base

After content is successfully fetched, click Train to process and train your Knowledge Base.
  • Pre-training
  • While training
  • Post-training
Pre-training state showing content ready for Knowledge Base training
Page selection options: - Select the pages you want to train - Deselect the pages you don’t need - View metadata about the number of words fetched for each page
Training process: - Vector storage: Content is converted to vector embeddings and stored in vector databases for fast retrieval - Index optimization: Creates efficient search and retrieval system for AI responses
6

Agent creation complete

Your knowledge-powered Agent is now ready!
Completed Agent creation showing the new Agent with Knowledge Base gambit connected
What happens automatically: - Agent creation: Agent is created with your specified name and Knowledge Base gambit connected - Ready to use: Agent can immediately answer questions based on your website content
Content Quality: The effectiveness of your Agent depends on the quality and organization of your source content. Ensure your website is well-structured and comprehensive.
Content Updates: If your source website changes significantly, you may need to retrain your Knowledge Base to ensure the Agent has access to the latest information.
⌘I