6 {{ upvoteCount | shortNum }}
6 {{ upvoteCount | shortNum }}

Robots.txt file?

by katied772
5 replies
  • SEO
    • |
When looking through webmaster tools and attempting to improve adsense and comply well with Google, it mentions making sure that the robots.txt file includes access. Where do I find this in my WP site?

Thanks
#file #robotstxt
  • Profile picture of the author lovboa
    Banned
    www,yoursite,com/robots.txt

    The last line should say sitemap: (with your sitemap here)
    {{ DiscussionBoard.errors[6548293].message }}
  • Profile picture of the author sudhanshu013
    Originally Posted by katied772 View Post

    When looking through webmaster tools and attempting to improve adsense and comply well with Google, it mentions making sure that the robots.txt file includes access. Where do I find this in my WP site?

    Thanks
    You have to check with your root directory, you can have it in www directory, the simple URL format of the same is url/robots.txt




    __________________________________________________ _________________


    The Green Room
    {{ DiscussionBoard.errors[6548535].message }}
  • Profile picture of the author getacho
    Banned
    [DELETED]
    {{ DiscussionBoard.errors[6548593].message }}
    • Profile picture of the author vgvetter
      Our site was not indexed by Webmaster Tools for over a week. Searching around, the word was that WP generates a virtual robots.txt file if your setup process disallows robots during construction which prevents indexing.. Our fix was

      Site Robots.txt File
      Generate a site robots.txt file as shown in bold below with notepad or the Google robots.txt tool and upload via FTP into the site root directory.

      User-agent: *
      Allow: /

      User-agent: Googlebot
       Allow: / sitemap: http://www.mysite.com/sitemap.xml
      |
       In the event the site uses Google AdSense, add the following:
      User-agent: Mediapartners-Google
      Disallow:

      Upload the above robots.txt file via FTP into the site root directory. From the browser if you now type in www.mysite.com/robots.txt, the file shown in bold above should display.

      The result was that the entire site was indexed in 24 hours.
      We omitted the "media partners Disallow" section and are running AdSense without problem.
      Signature
      Websites with WordPress
      Free eBook: Download; Backup; Navigation; Forms; Graphics; RSS; Traffic; SEO; Customization; News Letters; eCommerce
      {{ DiscussionBoard.errors[6548689].message }}
  • Profile picture of the author Clickthroo
    All good advice.

    Have seen many occasions in the past where the developer's had forgotten to remove disallow commands and so the site wasn't indexed for months..
    Signature
    Dave Bird (CEO): Clickthroo.com - Landing Page Marketing Solution for advertisers, agencies, and affiliates.

    Landing page builder, tracking platform, split-testing, traffic segmentation, auto-optimization, 100+ templates, 1000+ stock images, optional affiliate marketing module.
    {{ DiscussionBoard.errors[6548763].message }}
  • Profile picture of the author robertwilston
    Banned
    [DELETED]
    {{ DiscussionBoard.errors[6552884].message }}
    • Profile picture of the author katied772
      My site is indexed, so does that mean that my file is allowing Google as needed for adsense and ranking purposes?
      Signature


      {{ DiscussionBoard.errors[6552919].message }}

Trending Topics