Skip to main content

Web Scraping

learn about the web scraping feature of the knowledge library

Bryce DeCora avatar
Written by Bryce DeCora
Updated over 3 weeks ago

Overview

Scraping a website is the easiest way to load a bunch of information into an agent for it to reference. However, it's also the easiest way to feed too much information into your agent. Websites typically contain a lot of text, and despite what you may have heard, more doesn't always mean better when it comes to knowledge bases for agents. When possible, use document uploads or text inputs instead of web scraping.

Did this answer your question?