What are “site crawlers, robots, spiders” and what do they mean to me?

These are all terms for software packages used by Search Engines to scan through the millions of web pages on the internet, indexing the pages and categorizing them as they find them. As they travel, they collect a variety of data and send back to central indexing warehouses where it is stored and check for relevancy. Due to the sheer volume, this process takes time…hence why it takes time for your website to show up on searches. It’s not enough to have a good website. We need to give the crawlers time to find your good website/content and index those pages so that they can found in searches in the future.