9 {{ upvoteCount | shortNum }}
9 {{ upvoteCount | shortNum }}

Crawling Issue

by amzhr5
8 replies
  • SEO
    • |
I recently got a SEO project from my client and the site is 1 year old. I first created the webmaster and verified then submit sitemap but Google is not crawling it. I checked robots.txt file it is opened but still not crawling. What can be the issue?
#crawling #issue
  • Profile picture of the author amzhr5
    Originally Posted by dmcmark View Post

    Did you tried Fetch as Google Bot option in webmaster? If no then give it a try.
    Hope this helps.
    Yes I did that too..
    {{ DiscussionBoard.errors[8854628].message }}
  • Profile picture of the author yukon
    Banned
    Originally Posted by amzhr5 View Post

    I recently got a SEO project from my client and the site is 1 year old. I first created the webmaster and verified then submit sitemap but Google is not crawling it. I checked robots.txt file it is opened but still not crawling. What can be the issue?
    This might sound crazy, but I've seen it happen plenty of times on this forum.

    Check your HTML source code & do a browser search for noindex.

    If it's a WP site, sometimes people use SEO plugins that noindex their pages while they're trying rank pages.
    {{ DiscussionBoard.errors[8854722].message }}
    • Profile picture of the author amzhr5
      Originally Posted by yukon View Post

      This might sound crazy, but I've seen it happen plenty of times on this forum.

      Check your HTML source code & do a browser search for noindex.

      If it's a WP site, sometimes people use SEO plugins that noindex their pages while they're trying rank pages.
      I checked there is no noindex found. but there are some plugins added on WP admin.
      {{ DiscussionBoard.errors[8854953].message }}
  • Profile picture of the author rozerr
    It takes some time to crawl a website when you setup webmaster and submit your sitemap. So wait for sometime.
    {{ DiscussionBoard.errors[8855087].message }}
    • Profile picture of the author Enuke
      Yes some issues comes when your website does not crawled some points are here

      1. In robots.txt sometime use Disallow : /

      Remove this

      2. Source page of your home website have code noindex

      check this issue , if not crawled then in webmaster use Fetch as Google option.
      By the way above my friends saying right thing, I repeat those things only . please send your URL if not possible for more help
      {{ DiscussionBoard.errors[8855171].message }}
  • Profile picture of the author Deepak09
    Originally Posted by amzhr5 View Post

    I recently got a SEO project from my client and the site is 1 year old. I first created the webmaster and verified then submit sitemap but Google is not crawling it. I checked robots.txt file it is opened but still not crawling. What can be the issue?
    do some social bookmarking or blog commenting and wait for few days, your site will be crawled.
    {{ DiscussionBoard.errors[8855187].message }}
  • Profile picture of the author Michael Nguyen
    in most cases, a good strong link will get it index.
    Signature
    FREE Cold Calling Course: >>> Looking for Serious & Aspiring Cold Callers to Train for FREE <<<
    {{ DiscussionBoard.errors[8855283].message }}
    • Profile picture of the author paulgl
      Come on people. Any seo person would know how linking gets a site crawled.
      Why keep repeating nonsense?

      Apparently, either one of two things are happening.

      Site was never crawled, never cached. If that's the case, there is a problem.
      Especially for a 1 year old site. Tons of reasons. But then, a person taking clients
      would know the list. From website banned, to just no reason to even be indexed.

      The other scenario, is that the site has been crawled and cached. Just not
      often enough. Again, any seo person taking clients would know how to
      start getting regular crawls. Very simple, and very basic stuff.

      But here's a clue:
      Originally Posted by amzhr5 View Post

      I checked robots.txt file it is opened but still not crawling. What can be the issue?
      I have no idea what that means.

      If you have set up webmaster tools, then messages should start appearing if they
      truly cannot crawl your site.

      Some of you people are too polite. Why are some people taking paying clients?
      That's the real issue.

      Paul
      Signature

      If you were disappointed in your results today, lower your standards tomorrow.

      {{ DiscussionBoard.errors[8855457].message }}

Trending Topics