crawling and indexing

2 replies
  • SEO
  • |
I guess my question is why is it taking soooooooo long for my site to get crawled and idnexed?? it says index pending in webmaster......i know it doesnt happen magically overnight so is there a way to check which areas of my site have been indexed and not; like seeing a live status??

could it be possible that the more information you have in robots.txt to alllow and disallow the longer the search bots will take to fully index every piece of the site?

at what point should i assume that their is an error of indexing and crawling are taking longer time than expected like i doubt it would take a full weeks for a small 20 article site to be fully indexed and crawled
#crawling #indexing
  • Profile picture of the author MikeFriedman
    Originally Posted by TAHERSYED34 View Post

    I guess my question is why is it taking soooooooo long for my site to get crawled and idnexed?? it says index pending in webmaster......i know it doesnt happen magically overnight so is there a way to check which areas of my site have been indexed and not; like seeing a live status??

    could it be possible that the more information you have in robots.txt to alllow and disallow the longer the search bots will take to fully index every piece of the site?

    at what point should i assume that their is an error of indexing and crawling are taking longer time than expected like i doubt it would take a full weeks for a small 20 article site to be fully indexed and crawled
    It could easily take 3-4 weeks sometimes.

    Just use the 'site:' operator in the Google search bar and you will see what is indexed.
    {{ DiscussionBoard.errors[10058788].message }}
  • Profile picture of the author CPABeyondNetwork
    Ping your site to pingomatic, it should index your site after a day or two. If not then just wait for it to index naturally.


    Best regards,

    CPA Beyond
    Signature

    AM at CPA Beyond - #1 Private Network - Daily Payments (No Fees)

    {{ DiscussionBoard.errors[10058942].message }}

Trending Topics