3 {{ upvoteCount | shortNum }}
3 {{ upvoteCount | shortNum }}

Problems with robots.txt file on sniper site

by Clojo
2 replies
Hi there!

I am new on this forum and have over the last week or so just discovered it and can't believe how much great information there is on here. It has helped me know end, so I thought I would become a member and join in.

Today I am having a problem. I set up a new sniper site - thanks to George Brown - great guy.

Anyway I have submitted it to webmaster tools and with google xml sitemaps creating the sitemap, but Google is telling me that the robots file is restricting access. This is what I have for the robots file:

User-agent: *
Disallow:

Sitemap: sitemap url here

(not sure if I am allowed to put urls?)

When I initially submitted it to webmaster tools I didn't realised that I had the WP Privacy setting set to don't crawl, but have since changed this to visible to all.

Could the fact that it was on that mean that it is just a hangover from that and rectify itself, or do I need to do something? It is driving me crazy, I have never had this problem before with any of my other sites. Any help would be very much appreciated.

Thank you
#file #problems #robotstxt #site #sniper
  • Profile picture of the author BenQ
    If you start a site with the privacy setting on, my experience is it takes a little longer for it to get indexed once you change that setting. Just give it a couple days.
    Signature
    {{ DiscussionBoard.errors[3472804].message }}
    • Profile picture of the author RyanAndrews
      I had a site up that was meta noindex, and when I decided to build the site it took several weeks for google to up the crawl rate.
      I'm going to avoid doing that in future because of the hangover.
      {{ DiscussionBoard.errors[3480445].message }}

Trending Topics