Get crawled every few days

by burtie
14 replies
  • SEO
  • |
Hi guys,

I have a page on my site that I want to get crawled every few days or as much as possible. The reason for this is that it is a page that shows items/pages that have been recently added to my website, so I want to do it to get these new pages indexed as I shell be adding a few each and every day.

The page in question is;

Most Recent

Any help or tips on how to do this would be fantastic.
#crawled #days
  • Profile picture of the author alby
    Add frequent new content, google decides how often it crawls your site, i beleive it's possible to put a comment in the header tag, to give google's spiders a request to visit more often, also if you add a sitemap, you can stipulate that your content changes daily.
    {{ DiscussionBoard.errors[437205].message }}
  • Profile picture of the author Stephen Crooks
    Open a webmaster tools account with google, submit a google sitemap and you get options available about spider activity. Also make sure you have a robots file on your server to instruct search engine bots where they can and can't go.

    Originally Posted by burtie View Post

    Hi guys,

    I have a page on my site that I want to get crawled every few days or as much as possible. The reason for this is that it is a page that shows items/pages that have been recently added to my website, so I want to do it to get these new pages indexed as I shell be adding a few each and every day.

    The page in question is;

    Most Recent

    Any help or tips on how to do this would be fantastic.
    {{ DiscussionBoard.errors[437211].message }}
  • Profile picture of the author burtie
    Cheers guys. Working on your suggestions now.
    {{ DiscussionBoard.errors[437247].message }}
    • Profile picture of the author Bruce Hearder
      Doing a couple of manual pings to the page won't do any harm either..

      Go to places like :
      Ping-o-Matic!
      www.pingoat.com and
      Blog and Ping Tool - Use Pingler.com to Drive Traffic your Blogs and Websites

      will soon get the bots racing over to your site..

      Hope this helps

      Bruce
      {{ DiscussionBoard.errors[437279].message }}
      • Profile picture of the author burtie
        Originally Posted by Bruce Hearder View Post

        Doing a couple of manual pings to the page won't do any harm either..

        Go to places like :
        Ping-o-Matic!
        www.pingoat.com and
        Blog and Ping Tool - Use Pingler.com to Drive Traffic your Blogs and Websites

        will soon get the bots racing over to your site..

        Hope this helps

        Bruce
        All pinged (except for the middle one that is down for maintenance).
        {{ DiscussionBoard.errors[437287].message }}
        • Profile picture of the author Bruce Hearder
          Yeah PinGoat.com has been down for a few days now..
          Not quite sure what is happening there..

          I have found that once you ping a site with Pingler, GoogleBot can be on the site with 60seconds (thats right, seconds)

          Pingomatic sometimes works, bit most of the time it never gets the BOTS to turn up..

          Hope this helps..

          Bruce
          {{ DiscussionBoard.errors[437293].message }}
          • Profile picture of the author Deedah
            I've been using pingler but noticed that the captcha text is always 'wrong' the first time round, not that it actually is. Is their a reason for that? I always check about three times to make sure I've put in the right text but it never works, I always have to fill in another one before it accepts it.
            {{ DiscussionBoard.errors[437306].message }}
            • Profile picture of the author Bruce Hearder
              Originally Posted by Deedah View Post

              I've been using pingler but noticed that the captcha text is always 'wrong' the first time round, not that it actually is.
              I've noticed the same thing..

              Maybe its their attempt to reduce the amount of spam pings (whats the new name for that.. SPLINGS)

              Anyway once you get it right, its seems to work fine after that..

              Take care

              Bruce
              {{ DiscussionBoard.errors[437361].message }}
          • Profile picture of the author burtie
            Originally Posted by Bruce Hearder View Post

            Yeah PinGoat.com has been down for a few days now..
            Not quite sure what is happening there..

            I have found that once you ping a site with Pingler, GoogleBot can be on the site with 60seconds (thats right, seconds)

            Pingomatic sometimes works, bit most of the time it never gets the BOTS to turn up..

            Hope this helps..

            Bruce
            Well, looking forward to its visits.
            {{ DiscussionBoard.errors[437320].message }}
  • Profile picture of the author luckycharmz336
    I can get a page crawled in hours if I create a Press Release about the page on PRLOG and link to it from my press release.
    {{ DiscussionBoard.errors[438159].message }}
    • Profile picture of the author burtie
      Originally Posted by luckycharmz336 View Post

      I can get a page crawled in hours if I create a Press Release about the page on PRLOG and link to it from my press release.
      Interesting,

      I will bear that in mind.
      {{ DiscussionBoard.errors[438164].message }}
  • Profile picture of the author berryhill
    Banned
    [DELETED]
    {{ DiscussionBoard.errors[438883].message }}
    • Profile picture of the author burtie
      Originally Posted by berryhill View Post

      I have learnt a lot here,thanks guys.
      Another way to get your site indexed is to write quality article and post it in a high traffic forum and then put a link that will link back to your site in the article you wrote.
      Yeah I am going to start article marketing for the site soon.

      Does anybody know the quantity of quality links that I should be aiming for? 1000+? 10000+?

      Any ideas?
      {{ DiscussionBoard.errors[440314].message }}
  • Profile picture of the author makepurple
    build your inlinks for fast indexing.
    {{ DiscussionBoard.errors[440081].message }}
  • Profile picture of the author maestro2010
    webmaster account with google helps you to choose the crawlers activity on your website.........
    {{ DiscussionBoard.errors[446958].message }}

Trending Topics