Crawling Issue

by 8 replies
9
I recently got a SEO project from my client and the site is 1 year old. I first created the webmaster and verified then submit sitemap but Google is not crawling it. I checked robots.txt file it is opened but still not crawling. What can be the issue?
#search engine optimization #crawling #issue
  • Yes I did that too..
  • Banned
    This might sound crazy, but I've seen it happen plenty of times on this forum.

    Check your HTML source code & do a browser search for noindex.

    If it's a WP site, sometimes people use SEO plugins that noindex their pages while they're trying rank pages.
    • [1] reply
    • I checked there is no noindex found. but there are some plugins added on WP admin.
  • It takes some time to crawl a website when you setup webmaster and submit your sitemap. So wait for sometime.
    • [1] reply
    • Yes some issues comes when your website does not crawled some points are here

      1. In robots.txt sometime use Disallow : /

      Remove this

      2. Source page of your home website have code noindex

      check this issue , if not crawled then in webmaster use Fetch as Google option.
      By the way above my friends saying right thing, I repeat those things only . please send your URL if not possible for more help
  • do some social bookmarking or blog commenting and wait for few days, your site will be crawled.
  • in most cases, a good strong link will get it index.
    • [1] reply
    • Come on people. Any seo person would know how linking gets a site crawled.
      Why keep repeating nonsense?

      Apparently, either one of two things are happening.

      Site was never crawled, never cached. If that's the case, there is a problem.
      Especially for a 1 year old site. Tons of reasons. But then, a person taking clients
      would know the list. From website banned, to just no reason to even be indexed.

      The other scenario, is that the site has been crawled and cached. Just not
      often enough. Again, any seo person taking clients would know how to
      start getting regular crawls. Very simple, and very basic stuff.

      But here's a clue:
      I have no idea what that means.

      If you have set up webmaster tools, then messages should start appearing if they
      truly cannot crawl your site.

      Some of you people are too polite. Why are some people taking paying clients?
      That's the real issue.

      Paul
Join the discussion

Next Topics on Trending Feed

  • 9

    Crawling Issue

    I recently got a SEO project from my client and the site is 1 year old. I first created the webmaster and verified then submit sitemap but Google is not crawling it. I checked robots.txt file it is opened but still not crawling. What can be the issue?