Overview
Scraping a website is the easiest way to load a bunch of information into an agent for it to reference. However, it's also the easiest way to feed too much information into your agent. Websites typically contain a lot of text, and despite what you may have heard, more doesn't always mean better when it comes to knowledge bases for agents. When possible, use document uploads or text inputs instead of web scraping.