Spider:

Also called web Spider, web crawler, web robot or bot. A program or automated script which combs the Internet, visits websites , gathers all relevant data about it – especially an index of all keywords appearing on the site - and sends it to a search engine index data base.

Before a new site is not entered in a search engine’s index it cannot appear in the search result pages. Therefore webmasters want their site visited by Spiders as soon as possible. They can either submit their site manually (for Google you can do this here) or – better as a lot faster if done correctly – put their link out on those other sites which are often visited by spiders, e.g. blog  sites, social bookmarking sites or other popular high traffic websites . Spiders follow these links and arrive at the new site sometimes within 24 hours.

[Spider comes from Old English spibra = spider]

Bookmark this page

BlinkListDeliciousDiggFacebookFurlRedditSpurlStumbleuponYahoo My WebGoogle Bookmarks