20 {{ upvoteCount | shortNum }}
20 {{ upvoteCount | shortNum }}

How can I check if my site has been crawled?

by zannix
13 replies
  • SEO
    • |
Hi.

I'm interested if there are any online free ways/services to check if my site was crawled by google, and when it last occured?

I need this because the way I'm doing it now is changing the site description by one word and then checking the google's listings to see if it's updated the description. Then I know it's crawled.


But I'm interested, there surely must a way around this? Something quicker.
#check #crawled #site
  • Profile picture of the author renukoot
    Hi,

    I have some site address at my site. Hope that may help you.

    Search Engine Optimization (SEO) - Better SEM & SEO Optimization Tips Tutorials

    Thanks & Regards
    Abhishek Srivastava
    (www.needtruehelp.com - 1st Shopper Approved website selling Traffic)
    Signature
    www.caressl.com - Upto 75% Discount on SSL Certificates & Website Scanner. If you don't find what you looking for, raise a support query and we will get you that SSL Certificate.
    {{ DiscussionBoard.errors[3686094].message }}
  • Profile picture of the author Steve Faber
    look in your site logs at your hosting account. You'll be able to see if any of the search engine bots have crawled your site, and when they did so. If you're using CPanel for your hosting interface, it's really easy. Just go to the logs section and use awstats. On the left, after entering awstats, look under the "who" section under robots/spiders visitors. It's al right there.
    Signature
    For Killer Marketing Tips that Will Grow Your Business Follow Me on Twitter Now
    After all, you're probably following a few hundred people already that aren't doing squat for you.....
    {{ DiscussionBoard.errors[3686190].message }}
  • Profile picture of the author SmartHealthShop
    you can use Google analytics or Engineseeker to see all the IPs browsing your site. check the IP and you can see its location (look for CA IPs and also WA state IPs)
    {{ DiscussionBoard.errors[3686199].message }}
    • Profile picture of the author hyderkhan
      Signup for Google Webmaster Tools. You will be able to login and visibly see the date and time of when was the last time Google crawled your website, as well as a whole host of other useful statistics that Google maintains (eg- backlinks, keywords used to search your site, your site's ranking for certain keywords, etc.)
      Signature
      Looking To Help A Fellow Warrior Out.

      Need a Wordpress website built? I know Wordpress, MySQL, PHP, and even GIMP, and can help you build your website.

      I'm looking to JV with anyone with a drive to succeed. Send me a PM.
      {{ DiscussionBoard.errors[3686250].message }}
  • Profile picture of the author Christian Little
    Signup for Google Webmaster Tools (https://www.google.com/webmasters/tools/). It takes a few minutes to setup for your site, but it provides you with a ton of useful data including how/when Google crawls your site. It even provides you with a list of problems that Google has found when crawling your site, which is very handy.

    EDIT: hyderkhan beat me to it by a few seconds lol.
    Signature
    cPanel Shared and Reseller Web Hosting - 30 Day Free Trial Exclusive for Warriors!
    {{ DiscussionBoard.errors[3686259].message }}
    • Profile picture of the author zannix
      Thank you guys! Great help!

      BTW - do you know any method which kind of "let's google know that it should crawl my website", or do I just wait for it to happen?

      I mean I'm indexed already, but I'm building backlinks, and I want to monitor progress in listing rankings.
      {{ DiscussionBoard.errors[3686412].message }}
      • Profile picture of the author hyderkhan
        Originally Posted by zannix View Post

        Thank you guys! Great help!

        BTW - do you know any method which kind of "let's google know that it should crawl my website", or do I just wait for it to happen?

        I mean I'm indexed already, but I'm building backlinks, and I want to monitor progress in listing rankings.
        One way is to make sure that you have an updated sitemap.xml hosted on your website.

        And then log onto Google Webmaster Tools and select the option to have Google download an updated sitemap.xml from your website. (Not sure of the exact phrasing, but the option is there.)

        This is one way of signalling to Google "hey I just updated my website... please crawl me!" But Google will still take its own good old time to do so. But it least you'll be queued up.
        Signature
        Looking To Help A Fellow Warrior Out.

        Need a Wordpress website built? I know Wordpress, MySQL, PHP, and even GIMP, and can help you build your website.

        I'm looking to JV with anyone with a drive to succeed. Send me a PM.
        {{ DiscussionBoard.errors[3686429].message }}
      • Profile picture of the author dburk
        Originally Posted by zannix View Post

        Thank you guys! Great help!

        BTW - do you know any method which kind of "let's google know that it should crawl my website", or do I just wait for it to happen?

        I mean I'm indexed already, but I'm building backlinks, and I want to monitor progress in listing rankings.
        Hi zannix,

        There is a special protocol designed specifically for this purpose, it's called a XML-RPC ping.
        Signature

        * Get Results - Outsource Your PPC Management |
        *         Don Burk Advertising & Marketing - www.donburk.com

        {{ DiscussionBoard.errors[3686473].message }}
  • Profile picture of the author Andy Yusuf
    use google webmaster tools mate...
    Signature

    no sign today...

    {{ DiscussionBoard.errors[3686446].message }}
  • Profile picture of the author Christian Little
    Google Webmaster Tools will tell you how often Googlebot is crawling your site. You can also ask them to crawl it more frequently in the interface. They give you a pretty little chart that shows you these details, here's for one of my sites:



    That shows me that Googlebot is crawling 14 pages/day on my site usually, but it had a big spike in February.
    Signature
    cPanel Shared and Reseller Web Hosting - 30 Day Free Trial Exclusive for Warriors!
    {{ DiscussionBoard.errors[3686646].message }}
  • Profile picture of the author RogerBell
    you might try wordpress-crawl-rate-tracker you'll have to google it I can't post links
    {{ DiscussionBoard.errors[3687116].message }}
    • Profile picture of the author zannix
      Thank you guys.

      I suppose pinging a site at pingomatic won't make google go crawl?
      {{ DiscussionBoard.errors[3688032].message }}
      • Profile picture of the author dburk
        Originally Posted by zannix View Post

        Thank you guys.

        I suppose pinging a site at pingomatic won't make google go crawl?
        Technically you are not pinging a "site", you are pinging a URL, and pinging with Ping-O-Matic will almost always send the Googlebots a calling. You can also use Twitter and it will often work even faster than pinging.
        Signature

        * Get Results - Outsource Your PPC Management |
        *         Don Burk Advertising & Marketing - www.donburk.com

        {{ DiscussionBoard.errors[3688213].message }}

Trending Topics