6-17-03 - Googlebot 216.239.46.* disappears
Without any fanfare or spectacular occurrence, the well known
Google deep crawl web spider disappeared after a final tour
of
the net in April. However, the Googlebot is neither missing in
action nor caught in some spider trap in the deep recesses of
cyberspace.
As a part of major changes being employed by engineers deep within
the Googleplex, the main crawl spider is being replaced by a more
vigorous search robot, formerly and affectionately called the
Fresh Bot by many astute webmasters.
Traditionally, the "main" crawler's job was to pick
up new pages and new websites, placing them in a temporary database
where they would be added to the main index during Google's monthly
update. The Fresh Bot, however, was designed to quickly pick up
on changes to existing web pages ( the greater the website popularity
rating, the greater the chance of visits by the Fresh Bot ). These
updated pages would often show up with a date tag in search results.
The new and improved Fresh Bot, now fully deployed, will serve
both to crawl the internet, picking up new pages and websites,
and serve in its traditional role to quickly catch changes to
existing pages.
