7 {{ upvoteCount | shortNum }}
7 {{ upvoteCount | shortNum }}

Google crawler bot help ?

by classicseo
6 replies
Hello
I need some help from you to assist me. I installed wordpress theme to my domain but unfortunate, Google bot has been blocked to crawl on site in privacy option so my site gone to out of cache in Google and all major search engines, now i made changed in wordpress theme and unblock the Google bot crawlers. so this changes would be automated updated in webmaster tool because i find there also site crawler access is blocked in robot file. i have made changes this few hours ago today or i have to made manually correction in robot text file to access Google my site..

Thanks for your valuable opinions
#bot #crawler #google
  • Profile picture of the author alv22
    Install any of the popular seo plugins for wordpress. Then configure all the settings, create a sitemap and send it to google. Should be good.
    {{ DiscussionBoard.errors[3446965].message }}
  • Profile picture of the author Bruce Hearder
    Using your web browser, type in:

    www.yourdomain.com/robots.txt (obviously change yourdomain to your url)

    and see what your robots.txt file is saying. Make sure all is good before telling BigG to come back again.

    Bruce
    {{ DiscussionBoard.errors[3451266].message }}
  • Profile picture of the author classicseo
    All is ok there...
    User-agent: *
    Disallow:
    in robots.txt file but still Google webmaster tool is showing 16 urls restricted
    they would be automatically updated in webmaster tool ?
    {{ DiscussionBoard.errors[3456421].message }}
  • Profile picture of the author SteveJohnson
    Some themes block /archive/ and /tag/ URLs. You might check to see if that's the case.
    Signature

    The 2nd Amendment, 1789 - The Original Homeland Security.

    Gun control means never having to say, "I missed you."

    {{ DiscussionBoard.errors[3456549].message }}
  • Profile picture of the author classicseo
    where i should check that blocked archive/ and /tag/ URLs
    {{ DiscussionBoard.errors[3456991].message }}
    • Profile picture of the author donkey007
      user agent should be allowed in robot.txt file . Generate it again from your webmaster tools and upload again it to your web server.

      aprt from this you can create your sitemap from this site Create your Google Sitemap Online - XML Sitemaps Generator and submit sitemap url to your sitemap section in webmaster tool . just like yourdomain.com/sitemap.xml
      {{ DiscussionBoard.errors[3457208].message }}

Trending Topics