A description for this result is not available because of this site's robots.txt - learn more

4 replies
  • SEO
  • |
I have a wordpress website stupidscam.co.nf however i have recieved a message in webmaster and on google results page it says "A description for this result is not available because of this site's robots.txt – learn more"

it is a new website and only has worpress and a few plugins?

help please

thank you do i just delete the robot.txt?

but i never made a robot.txt? and its not in my ftp

it seems ai had the option in wordpress to block search engines....never clicked that oh well, dont really know why they have that option anyway hopefully it should work now

nope i have even made a robot.txt file to overwrite the virus wordpress puts on your website and still when fetch as google bot nothing still denied from my own website

does google need to find and crawl my website again on it own ?
#description #learn #result #robotstxt #site
  • Profile picture of the author Igal Zeifman
    Hi,

    "but i never made a robot.txt?"

    Every site has robot.txt file, as so does yours. See here:
    http://stupidscam.co.nf/robots.txt
    You don't need to create it, as most CMS (WP included) will create it for you.
    Also, it is very easy to spot, because it will always be in your root.

    "do i just delete the robot.txt?"
    Please don't delete your robot.txt - you`ll need it.
    To modify, simply go in (i.e. via FTP) an edit it by removing the "Disallow:" rule.

    Here you can learn more about Robots.txt...
    Also, in case your are thinking about managing bot traffic, for bot info, IPs and user-agents you should go to Botopedia.org.
    {{ DiscussionBoard.errors[6823953].message }}
  • Profile picture of the author roycewong
    I have the same problem to, I've deleted the robots.txt for several days, but it still show me the message in googles SERP, how could I do, my site has no index in google now
    {{ DiscussionBoard.errors[7169484].message }}
  • Profile picture of the author jovykhan
    Another reason for this issue, especially if you are using CMS, is that you blocked Google using the Privacy settings found in WP for example when you just started building your website because don't want yet to index your website.

    Just let it pass and make sure you're not actually blocking it via robots.txt.
    Signature
    LocalFinder.net Australia's Leading Online Business Directory
    Australian Local Citation Service
    {{ DiscussionBoard.errors[7169536].message }}
    • Profile picture of the author roycewong
      Originally Posted by jovykhan View Post

      Another reason for this issue, especially if you are using CMS, is that you blocked Google using the Privacy settings found in WP for example when you just started building your website because don't want yet to index your website.

      Just let it pass and make sure you're not actually blocking it via robots.txt.
      This message just show for my index page, the inner page was normal in google,
      about 2 weeks
      {{ DiscussionBoard.errors[7169727].message }}

Trending Topics