Importance:
High
In a
footnote to
this article on
Google Watch, GoogleGuy is quoted as saying that Google winds down its crawl after 2 billion plus pages. I will leave the analysis and speculation up to you.
GoogleGuy Says: [
Link to quote]
Hey, I should have been more clear in one of my posts. When I said 'it could be that we just ran out of space or time,' I was referring to space in our repository. That is, we wind down the crawl after fetching 2B+ URLs, and the URL in question might not have been in that set of documents. Regarding address space, we're not in danger of running out of docIDs for our documents. The story about the engineer claiming that he almost fell out of his chair laughing at the suggestion that we're running out of docID/address space is true. You can choose not to believe me, but that's your call. :)
