1 {{ upvoteCount | shortNum }}
1 {{ upvoteCount | shortNum }}

Robots.txt and Wordpress

by cybernetuk
1 replies
  • SEO
    • |
Normally when building a Wordpress site I set it to private before its finished. On my new site I did this and then took the privacy off on completion. I noticed Google still had not indexed it after a couple of days and when I submit the sitemap to Google Webmaster, it says its blocked by the Robots.txt.

I know Wordpress creates a virtual Robots so I created a manual one to see what happens, but still it says it is been blocked. Should I just wait and see - normally I have never had a problem. The Robots.txt look like this:

User-agent: *
Disallow: /cgi-bin
Disallow: /wp-admin
Disallow: /wp-includes

Sitemap: http://www.xyxyx.com/sitemap.xml.gz

thanks
#robotstxt #wordpress
  • Profile picture of the author PerformanceMan
    Google may not have grabbed the new one yet. I think I read somewhere they only check once every 24 hours. Be patient and check again tomorrow.

    Your new one looks right.
    Signature
    Free Special Report on Mindset - Level Up with Positive Thinking
    {{ DiscussionBoard.errors[7055137].message }}

Trending Topics