6 {{ upvoteCount | shortNum }}
6 {{ upvoteCount | shortNum }}

Typical Time from Re-Crawl to Re-Index

by consultant1027
5 replies
  • SEO
    • |
For existing sites, already in the Google index, on average how long are people seeing Google take before recrawling and discovering new pages or page changes, and when the changes are reflected in the index (you can see the Title or Meta Description change or see the page preview image change, or, see new pages appear)

Are people seeing new pages for existing sites take longer to show up than changes to existing pages?

How dependent is it on the size and age of the site?
#recrawl #reindex #time #typical
  • Profile picture of the author ~kev~
    Originally Posted by consultant1027 View Post

    How dependent is it on the size and age of the site?
    A lot.

    Google will crawl older sites with lots of backlinks faster then new sites. There was a show on the discovery channel a few months ago where an engineer from google was talking about how search engines work. Sites like yahoo are being crawled thousands of times a day to find new content.

    On my blog, new articles are indexed within seconds of being posted.

    New sites might only be crawled a few times a week.
    Signature
    Country Lifestyle Network
    {{ DiscussionBoard.errors[4356573].message }}
  • Profile picture of the author consultant1027
    I've noticed that too.. blogs and especially forum posts showing up on Google almost instantly. I'd love to get more specific insight on what factors they use specifically to determine how quickly the page gets indexed and their relative importance - but wouldn't we all?
    {{ DiscussionBoard.errors[4356694].message }}
  • Profile picture of the author JamesGw
    It has a lot to do with backlinks, as that's one way spiders can reach your site. I think Google traffic also has a lot to do with it, which kind of makes it difficult to increase your indexing rate. The more pages you have indexed, the more Google crawls them, but that doesn't help you when you're waiting to have the pages indexed. That said, having a crapload of pages definitely helps.
    Signature
    Brazilian Jiu-Jitsu Techniques | Precision SEO
    {{ DiscussionBoard.errors[4356731].message }}
  • Profile picture of the author consultant1027
    It's interesting, I've got a site that is 7 years old with 2700+ backlinks. I'm seeing new pages appear in the index in 3-5 days. I've seeing updates to Title and Meta Description appear within 12 hours. I'm seeing the page preview being updated in a week or two, longer for sub-pages, shorter for home page.

    It seems like the re-indexing takes even longer but it may just be there wasn't enough changes to the page to make it change position.

    Why would a page not be re-indexed immediately after it is crawled?
    {{ DiscussionBoard.errors[4386395].message }}
  • Profile picture of the author lexilexi
    Blog pages are often crawled immediately because the blog software (i.e. Wordpress) has a built in software that pings Google to let it know that the blog has been updated. You can also see how long it takes on average by looking at the "cached" link in the serps, it has a date on it and looking through any top 10 serp there are often pages which have not been crawled for 2-6 weeks.
    Signature

    "If there is no door, it becomes necessary to break out through the wall."

    {{ DiscussionBoard.errors[4388647].message }}

Trending Topics