16 {{ upvoteCount | shortNum }}
16 {{ upvoteCount | shortNum }}

Stop Google From Crawling Down Site?

by CJWrite
14 replies
  • WEB DESIGN
    • |
So, I'm in the middle of switching one of my websites from Weebly to HostGator with WordPress. At the moment, it seems to be taking forever to propagate (?) and all I can reach when I go to the address is a "Page not found" from my old host (Weebly). A friend of mine can see the new unset up Host Gator site though.

Anyway, my question is this:

Will the "Page not found" errors effect my PR or standing with Google at all? Should I try to stop Google from crawling my site for a few days? Is that even possible? Or should I just not worry about it?
#crawling #google #site #stop
  • Profile picture of the author justmerob
    One my clients is using Weebly as well and I have been trying to get him over to ANYWHERE other than here.

    I have thought about this same thing for when we do the move. Now depending on the situation I think is going to depend on what you need to do.

    1. If they are using the hxxp://www.SITENAME.WEEBLY.com address and your going to your own domain, setting up 301 redirects in Weebly settings would be your best bet when the new site is up. Just direct each page from the .weebly.com site to your new domains pages.

    2. If they are using their own domain currently with Weebly (bought from Weebly or elsewhere), I'm thinking that once the new site is set up doing the 301's in the NEW HOSTING cpanel to direct any prior ranking pages to the NEW correct pages.

    Now will this effect PR tho, that I am unsure of but I would assume doing the 301's is saying "Hey my new page/site is here" would be fine until the new pages index.

    Side note: I guess in theory you could set up the new site pages with the same URL's from Weebly (if on your own domain of course) and it should work too. Again, not positive about this one but it's a thought.

    Hope this helps you some and I will keep an eye on this for when I have to move this client as well.

    Rob
    Signature
    "There are no magic wands, no hidden tricks, and no secret handshakes that can bring you immediate success, but with time, energy, and determination you can get there."
    -- Darren Rowse, Founder - Problogger
    {{ DiscussionBoard.errors[7090613].message }}
    • Profile picture of the author CJWrite
      Oh, der... I never gave redirects a consideration. The way I have it set up now is a "splash" page, since there are 2 different sections of the website. The main header goes to the Weebly main site (I did convert it to a free weebly site - mysite.weebly.com - while I switched everything over), and the "For this, click here" and "For that, click here" links go to the corresponding pages.

      Only problem now is the MySite.com is still going to the "Page not found" on Weebly. It's driving me nuts, since I know SHE can see the new site but I can't. I don't know if there's something off with my computer, or if its just taking longer for my "cache stuff" (?) to update. I changed to the HG nameservers yesterday.

      It's such a pain to explain. Be so much easier to just show you.

      And, yes. Switching from Weebly to another host is turning out to be a much more difficult situation than originally anticipated... I would almost advise your client to just stick it out with Weebly for the time being, since I basically have to rebuild from scratch. Which is a nasty pain on a 2-section website!!!

      **On a side note - I picked the WRONG week to quit smoking!! I have no cigarettes, a fist full of Dum-Dums, and a website that doesn't work!!

      AAARRRRRGHH!!! HULK SMAAAASH!!!
      {{ DiscussionBoard.errors[7090799].message }}
      • Profile picture of the author justmerob
        Originally Posted by CJWrite View Post

        Oh, der... I never gave redirects a consideration. The way I have it set up now is a "splash" page, since there are 2 different sections of the website. The main header goes to the Weebly main site (I did convert it to a free weebly site - mysite.weebly.com - while I switched everything over), and the "For this, click here" and "For that, click here" links go to the corresponding pages.

        Only problem now is the MySite.com is still going to the "Page not found" on Weebly. It's driving me nuts, since I know SHE can see the new site but I can't. I don't know if there's something off with my computer, or if its just taking longer for my "cache stuff" (?) to update. I changed to the HG nameservers yesterday.

        It's such a pain to explain. Be so much easier to just show you.

        And, yes. Switching from Weebly to another host is turning out to be a much more difficult situation than originally anticipated... I would almost advise your client to just stick it out with Weebly for the time being, since I basically have to rebuild from scratch. Which is a nasty pain on a 2-section website!!!

        **On a side note - I picked the WRONG week to quit smoking!! I have no cigarettes, a fist full of Dum-Dums, and a website that doesn't work!!

        AAARRRRRGHH!!! HULK SMAAAASH!!!
        24 hours, it's probably just not populated with the new host just yet or some weird (it is here but not there) type thing going on. You cleared your cache and that didn't work so just give it time. If the domain was bought from Weebly I could see it taking a few days with their speed at doing anything. If you want me to look feel free to shoot me a PM with your e.mail so I can get back to you, or get me on Skype.

        The building from scratch is also what I told my client and were upto 80+ pages on his site right now!!! So it WILL be a nice job when it happens.

        Hope this gets resolved for you soon and stay strong on the quitting smoking thing too! I'm in that boat with you but mine has sunk quite a few times over the past month :/ But I do have to say the HULKSMASH made me LOL

        Take care!
        Signature
        "There are no magic wands, no hidden tricks, and no secret handshakes that can bring you immediate success, but with time, energy, and determination you can get there."
        -- Darren Rowse, Founder - Problogger
        {{ DiscussionBoard.errors[7092390].message }}
        • Profile picture of the author CJWrite
          Originally Posted by justmerob View Post

          24 hours, it's probably just not populated with the new host just yet or some weird (it is here but not there) type thing going on. You cleared your cache and that didn't work so just give it time. If the domain was bought from Weebly I could see it taking a few days with their speed at doing anything. If you want me to look feel free to shoot me a PM with your e.mail so I can get back to you, or get me on Skype.

          The building from scratch is also what I told my client and were upto 80+ pages on his site right now!!! So it WILL be a nice job when it happens.

          Hope this gets resolved for you soon and stay strong on the quitting smoking thing too! I'm in that boat with you but mine has sunk quite a few times over the past month :/ But I do have to say the HULKSMASH made me LOL

          Take care!
          I don't wanna give it more time!! GAAAAARRRRRRRGH!! It'll be going on 48 hours down as of 2 PM today.

          However, I can see the site through a proxy server for some reason. The only thing I've come up with is that my ISP sucks and hasn't refreshed something. But I have no idea, since I'm not a techy person at ALL.

          And oops on the smoking. But only 2 all day yesterday. I loaded up on Snickers, Milky Ways, Caramellos, and those chocolate Hostess cupcakes with the cream filling, so I'll be fat soon, but at least I won't be smoking! Much...
          {{ DiscussionBoard.errors[7096791].message }}
  • Profile picture of the author HorseStall
    You can modify the robot.txt to temporarily stop spidering and then update it.
    Signature
    Horse Gifts :: Dog Gifts :: Wild Animal Gifts :: Mythical Creatures
    {{ DiscussionBoard.errors[7091153].message }}
    • Profile picture of the author rhinocl
      If someone else is seeing it one of you has a cache that needs to be cleared out. I suppose propogation could happen differently for people in different places but I think a caching issue is more likely.
      Signature
      cheap search engine optimization
      {{ DiscussionBoard.errors[7091891].message }}
      • Profile picture of the author CJWrite
        Originally Posted by rhinocl View Post

        If someone else is seeing it one of you has a cache that needs to be cleared out. I suppose propogation could happen differently for people in different places but I think a caching issue is more likely.
        That's exactly what I thought. I cleared the cache in Opera and IE, and both are showing the Weebly error page. I also flushed my DNS cache in Windows - still no luck.

        So, I did a fresh download of FireFox - and it shows the error page as well. It's been almost 24 hours. The other computer in my home also shows an error page, sooooo.......... i have no idea what's going wrong.

        I also did a DNS check tool thingie, and it says the NS are HostGator...

        I'm stumped. And getting frustrated!!!
        {{ DiscussionBoard.errors[7091930].message }}
    • Profile picture of the author Bryan Zazz
      Originally Posted by HorseStall View Post

      You can modify the robot.txt to temporarily stop spidering and then update it.
      I don't believe weebly allows users to play with the robots.txt file.
      Besides, robots.txt is just a convention, some crawlers could simply ignore it and continue crawlling the site regardless ...
      Signature
      wsoProto-WSO
      wsoProto-app
      {{ DiscussionBoard.errors[7099697].message }}
  • Profile picture of the author rhinocl
    There is one other thing that could cause this which I won't say here (it's something all web site developers have done at one time or other) but if you pm the url, I'll get back to you by e-mail).Oh you can't pm. I'll send one with an email address.
    Signature
    cheap search engine optimization
    {{ DiscussionBoard.errors[7092867].message }}
    • Profile picture of the author Bryan Zazz
      Originally Posted by rhinocl View Post

      There is one other thing that could cause this which I won't say here (it's something all web site developers have done at one time or other) but if you pm the url, I'll get back to you by e-mail).Oh you can't pm. I'll send one with an email address.
      hey there friend, why wouldn't you share your secret with the rest of us here?
      no need to be shy, we won't bite.
      (or are you just trying to get a date here - haha)
      I believe her weebly site is cjwebwriter.com
      Signature
      wsoProto-WSO
      wsoProto-app
      {{ DiscussionBoard.errors[7099754].message }}
      • Profile picture of the author CJWrite
        Originally Posted by Bryan Zazz View Post

        you probably did not have much of a PR to begin with, so I would not worry about that.
        Sometimes you could see an old version of a page beacuse the browser shows you a "cached" copy. This is definitely true if your friend can see the "new" site, while you see the old version: Clear your browser cache!!
        Well, it's the first PR I've ever had and it may not be much to y'all, but I really would rather not lose it.

        I DID clear my caches. On all my browsers.

        I cleared the cache in Opera and IE, and both are showing the Weebly error page. I also flushed my DNS cache in Windows - still no luck.
        Originally Posted by Bryan Zazz View Post

        I believe her weebly site is cjwebwriter.com
        No - that's not the site I'm working on right now.
        {{ DiscussionBoard.errors[7104793].message }}
      • Profile picture of the author rhinocl
        Originally Posted by Bryan Zazz View Post

        hey there friend, why wouldn't you share your secret with the rest of us here?
        no need to be shy, we won't bite.
        (or are you just trying to get a date here - haha)
        I believe her weebly site is cjwebwriter.com
        Because the other logical causes had to do with possible dumb errors on the OP's part which turned out not to be the case, and I'm afraid I'm too old for her.
        Signature
        cheap search engine optimization
        {{ DiscussionBoard.errors[7105239].message }}
  • Profile picture of the author Bryan Zazz
    Originally Posted by CJWrite View Post

    So, I'm in the middle of switching one of my websites from Weebly to HostGator with WordPress. At the moment, it seems to be taking forever to propagate (?) and all I can reach when I go to the address is a "Page not found" from my old host (Weebly). A friend of mine can see the new unset up Host Gator site though.

    Anyway, my question is this:

    Will the "Page not found" errors effect my PR or standing with Google at all? Should I try to stop Google from crawling my site for a few days? Is that even possible? Or should I just not worry about it?
    you probably did not have much of a PR to begin with, so I would not worry about that.
    Sometimes you could see an old version of a page beacuse the browser shows you a "cached" copy. This is definitely true if your friend can see the "new" site, while you see the old version: Clear your browser cache!!
    Signature
    wsoProto-WSO
    wsoProto-app
    {{ DiscussionBoard.errors[7099712].message }}
  • Profile picture of the author VirtuaSoft
    Remember when you buy a hostgator account to use the coupon "404page" 0,01 cent. good luck, also you need to wait for like 48 hours about the dns updates.
    {{ DiscussionBoard.errors[7105472].message }}

Trending Topics