6 {{ upvoteCount | shortNum }}
6 {{ upvoteCount | shortNum }}

Spiders not visiting Wordpress Site

by beachowl
5 replies
  • WEB DESIGN
    • |
I have a site that gets around 300 visitors a day. (according to statpress) Recently i noticed in the stats that zero spiders were visiting. and apparantly this has been the case since january. Ive checked various settings and it is selected to be "visible" to the spiders. Anyone know the specifics i should check or what else may cause this? no new plugins were added prior to.

Thanks.
#site #spiders #visiting #wordpress
  • Profile picture of the author turntablist
    are you using google webmaster? Paste a copy of your robots.txt so we can review it
    {{ DiscussionBoard.errors[3842859].message }}
  • Profile picture of the author Mkj
    You can block spiders by commands in the robots.txt file. Check that your site is using one and take a look at the contents of it.

    Block or remove pages using a robots.txt file - Webmaster Tools Help
    {{ DiscussionBoard.errors[3842864].message }}
    • Profile picture of the author beachowl
      ugh. i didnt even look at the robots. Cant seem to find one for this site. i have multiple domains hosted under 1 so not sure if each site directory has a root or just in the main directory. i just created one with notepad and im gonna drop it into the main folder. i found a suggested robot file online.

      User-agent: *
      Disallow: /wp-
      Disallow: /feed/
      Disallow: /trackback/


      is the above sufficient ? and about the "root" or "multiple root" directories ?
      thanks a lot for the responses.
      {{ DiscussionBoard.errors[3843390].message }}
      • Profile picture of the author jamesg3
        If you are attempting to make multiple sites that will generate and income, do not use sub domains. Go buy domains for those sites, they are usually 6 to 12 bucks a year.
        Signature
        Serious Wordpress sites, html/css sites and flash sites Cheap. I provide killer SEO, easy to use e-commerce solutions and payment gateways. Check me out at Serious Website Design.
        Need Hosting with serious speed and power? 3.99 a month. Take a look here. Serious Website Hosting.
        {{ DiscussionBoard.errors[3846839].message }}
        • Profile picture of the author beachowl
          that's sound advice and i am doing so. I have multiple domains hosted under one account, add on domains as hostgator puts it. And each separate add on is under a different file in the file structure. This makes me not sure if things have to be done to the main root, or each of the add on domain folder roots.

          thanks for all your replies
          {{ DiscussionBoard.errors[3847631].message }}

Trending Topics