Wordpress SEO and crawling problem

by 13 replies
15
Hi,

Around 10 days ago I moved from the reliable Headspace 2 to Yoast's Wordpress SEO on the basis of rave reviews. Before moving, most of my pages were ranked well enough to give me a living from my site.

After the 'upgrade', traffic dropped off alarmingly and when I check Google using th 'site:' function, and click on 'cache' option I see that the site has not been crawled/indexed since Jan 29/30! Plus, my pages have dropped maybe 5 or 6 pages or more.

I assumed it was a sitemap failure and disabled the built-in XML and enabled the trusty Workpress plug in. Still, I am not being crawled.

Can anyone please help me. I am losing money hand over first.

Charlotte
#search engine optimization #crawling #problem #seo #wordpress
  • PS. I have just installed Google Webmaster Tools, and see I that while I AM being crawled, I am also getting a LOT of crawl errors 404.

    Perplexed!
    • [3] replies
    • Hi
      It may be that the new plugins may have altered the http response codes, not sure. If you are using Firefox, there is a plugin called livehttpheaders
      Then install this HEaderMonitor plugin. It will show you the header code on your browser status bar.

      Now check the list at Google for what the different codes stand for and make sure you dont get an unsavory header response. Try this and see what happens.

      Also did you change all the metatags when you changed the plugins?

      Best
      Raviv
    • can you see the robots.txt file in your webiste. <your website>/robots.txt

      You can see this in the webmaster tools as well. all the bots should be allowed.
      I have seen few days before that, if you dont specify the robots.txt, the default robots txt is not allowing the bots.
      Note: you havent placed the robots.txt. but <your website>/robots.txt serves you a page, then you have to prepare a robots.txt and place in the wwwroot.
    • Hi Charlotte
      If you do not have a robots.txt file, then it means by default the bots are allowed to spider your site. If you want to specifically block certain areas of your site, then you have to use the robots.txt file (create it first and upload it to your web server). Nothing to worry about if it is not there
      Best
      Raviv
    • [DELETED]
  • Try this and see what happens.
    • [1] reply
    • Another one! Try WHAT and see what happens?
      • [1] reply
  • I have done the site+robots.text, and now get this:xxxxxx being my site name
    Does this mean my sitemap is being disallowed?

    User-agent: *
    Disallow:Sitemap: xxxxxxxxxxxxxxx/sitemap.xml.gz
    • [1] reply
    • No, I don't believe so, the commands have just run onto one line. They should read as I quoted them earlier, with the 'Disallow:' on a separate line, but I doubt that makes a difference to the robot.

      However, just in case, you might want to separate out those declarations like I did above.

      Again, though I find it hard to believe this would be the cause. If Google likes a site they aren't going to knock you down the SERPS just because your sitemap is unavailable. Go hassle that plugin writer!
      • [ 1 ] Thanks
  • I haven't seen this brought up yet and hopefully it's not a dumb question, but did you change your URL structure when you installed the WordPress SEO plugin from Yoast?

    Have you also ported over your previous SEO Titles/Descriptions, etc.? (there are plugins to help do this)
    • [1] reply
    • Yes I transported all my settings using the Yoast import option within WP SEO.

      I have emailed the man himself and he is diagnosing it as i type.

      Fingers crossed

      C
      • [1] reply
Join the discussion

Next Topics on Trending Feed