GoDaddy Warning!

Agreed, but if your robots.txt is set to not crawl, it does not even get to your sitemap. Therein lies the problem. Sitemap and robots are two different things.

Thanks
2 replies

Signature

0 - 2456 Unique Visitors A Day With Reverse Blogging

{{ DiscussionBoard.errors[422459].message }}

Nathan Hangen

15 years ago

Originally Posted by SaSeoPete

Agreed, but if your robots.txt is set to not crawl, it does not even get to your sitemap. Therein lies the problem. Sitemap and robots are two different things.

Right, but the plugin can generate a new robots.txt for you.

Thanks
1 reply

{{ DiscussionBoard.errors[422959].message }}

Sonni 15 years ago

If you're not convinced not to use Godaddy go to the Nodaddy forum and read some of their horror stories. Yes, there's a Nodaddy forum no kidding.
Sonni
- Thanks
- 1 reply
Signature
www.makeblogs.org
www.easytipsforweightloss.geteasytips.com
{{ DiscussionBoard.errors[422988].message }}
- jmidas 15 years ago
  
  Thanks Pete, going to check mine now.
  
  Thanks
  
  {{ DiscussionBoard.errors[423010].message }}

RDGatchel 15 years ago

Yeah ...

I have also had some "fun" with GoDaddy hosting and dropped them like a hot potato!
- Thanks
{{ DiscussionBoard.errors[427190].message }}

IMChick

15 years ago

So did GoDaddy actually change something in your hosting account for this to happen all of a sudden? Did you trace how it got there?

I just got multiple notices from them that my hosting was migrating to a new server, and I wonder if anything changed or popped back to default along the way.

And, I know, GoDaddy is not the place to host. I'm looking for a new home.

Thanks
1 reply

{{ DiscussionBoard.errors[422796].message }}

Sonni

15 years ago

I am techie dumb and don't know how to locate the robot thing you speak of where exactly is it? Somewhere in the coding I suppose but where? Thanks
Sonni

Thanks
1 reply

Signature

www.makeblogs.org
www.easytipsforweightloss.geteasytips.com

{{ DiscussionBoard.errors[422809].message }}

Eric Lorence

15 years ago

Yet another reason to steer clear of Godaddy hosting.

You will find many low quality hosts will do this in order to save their bandwidth costs, and many newbies are none the wiser, so they get away with it.

Try Hostgator next time.

Thanks
2 replies

{{ DiscussionBoard.errors[422839].message }}

Derek Pankaew

15 years ago

Originally Posted by Eric Lorence

Yet another reason to steer clear of Godaddy hosting.

You will find many low quality hosts will do this in order to save their bandwidth costs, and many newbies are none the wiser, so they get away with it.

Try Hostgator next time.

Are you freakin' serious? They're de-indexing their customer's sites to save bandwidth?

Thanks
1 reply

{{ DiscussionBoard.errors[422867].message }}

Eric Lorence 15 years ago

Originally Posted by Derek Pankaew

Are you freakin' serious? They're de-indexing their customer's sites to save bandwidth?

Been going on for a while.
- Thanks
{{ DiscussionBoard.errors[422886].message }}

Karen Blundell 15 years ago

GoDaddy is a great domain registrar...I've been using them for years. Their hosting, though, is not so good, especially for WordPress-driven sites. I've had clients who had nothing but problems with their WordPress sites hosted on GoDaddy. I helped them move their blogs to another host. My suggestion is to find another web host.
- Thanks
Signature
---------------
{{ DiscussionBoard.errors[422874].message }}

JustDean

15 years ago

Sorry I'm a Dumb A$$ when it come to programing, where do I find it and change it,is it some thing I should even worry about.

Thanks Eric I will try Host Gator next time

Thanks

Signature

ARK PIF

Acts of Random Kindness
Pay It Forward

{{ DiscussionBoard.errors[422859].message }}

Mark Brian

15 years ago

Wow, why on earth would they not want their client's site to not be indexed?

Thanks

Signature

{{ DiscussionBoard.errors[422871].message }}

Red Tomatoe

15 years ago

Thanks for info guys

Red Tomatoe

)

Thanks

{{ DiscussionBoard.errors[422877].message }}

cypherslock

15 years ago

Another good host with FAST and excellent support is HASTYHOST - cheap web hosting | shared, reseller, and vps

Thanks

{{ DiscussionBoard.errors[423051].message }}

SaSeoPete

15 years ago

A robots.txt is very easy to make. All you need to do is open notepad, the copy and paste this:

User-Agent: *
Allow: /

and save is as robots (the .txt will come automatically) and upload it into your root folder.

All it does is tell search engines where they are allowed to go and where they aren't.

Thanks
1 reply

Signature

0 - 2456 Unique Visitors A Day With Reverse Blogging

{{ DiscussionBoard.errors[423065].message }}

riveradjr

15 years ago

Originally Posted by SaSeoPete

A robots.txt is very easy to make. All you need to do is open notepad, the copy and paste this:

User-Agent: *
Allow: /

and save is as robots (the .txt will come automatically) and upload it into your root folder.

All it does is tell search engines where they are allowed to go and where they aren't.

thanks for the info. will be looking to implement it now.

Thanks

{{ DiscussionBoard.errors[426717].message }}

blackhatbastard

Banned 15 years ago

Wordpress 2.6 & 2.7, by default have an option in the administrator panel that says
"I would like to block search engines but allow normal visitors". It is under Settings/Privacy Settings. It is enabled by default, which I'm guessing could possibly change the Robots.txt to make it seem like GoDaddy doesn't want your blog to be indexed by search engines. (Which is ridculous)

You should check that before you make assumptions about your web host

[ 1 ] Thanks
1 reply

{{ DiscussionBoard.errors[423100].message }}

kevinw1

15 years ago

Originally Posted by blackhat*******

Wordpress 2.6 & 2.7, by default have an option in the administrator panel that says
"I would like to block search engines but allow normal visitors". It is under Settings/Privacy Settings. It is enabled by default,

None of the (several dozen) WP blogs I have installed have ever had this enabled. Are you sure this is correct? Could it be that certain hosts set up their fantastico WP installs like this?

Kevin

Thanks
1 reply

{{ DiscussionBoard.errors[426964].message }}

jasonl70

15 years ago

Originally Posted by kevinw1

None of the (several dozen) WP blogs I have installed have ever had this enabled. Are you sure this is correct? Could it be that certain hosts set up their fantastico WP installs like this?

Kevin

I've installed around 5 wp blogs over the past 2 months (at go-daddy), and every one of them had this as their default setting

I installed the same version on my own server, and they were 'clean'..

Thanks

Signature

-Jason

{{ DiscussionBoard.errors[427222].message }}

peterwhite

15 years ago

This is really useful information - esp for a newbie like me. I was wondering though... I am having problems right now with one of my webistes. Google is not indexing it... its a simple forwading site... i dont use a host for this one... do you think because I bought the domain from GoDaddy? I need to try and get indexed on Google quicker.. any ideas??? Thanks!

Thanks
1 reply

{{ DiscussionBoard.errors[423103].message }}

SaSeoPete

15 years ago

Originally Posted by blackhat*******

Wordpress 2.6 & 2.7, by default have an option in the administrator panel that says
"I would like to block search engines but allow normal visitors". It is under Settings/Privacy Settings. It is enabled by default, which I'm guessing could possibly change the Robots.txt to make it seem like GoDaddy doesn't want your blog to be indexed by search engines. (Which is ridculous)

You should check that before you make assumptions about your web host

Mine was actually set to " I would like my blog to be visible to everyone, including search engines (like Google, Sphere, Technorati) and archivers"

Originally Posted by peterwhite

This is really useful information - esp for a newbie like me. I was wondering though... I am having problems right now with one of my webistes. Google is not indexing it... its a simple forwading site... i dont use a host for this one... do you think because I bought the domain from GoDaddy? I need to try and get indexed on Google quicker.. any ideas??? Thanks!

No, if you don't host on GoDaddy you shouldn't have a problem at all.

Thanks
1 reply

Signature

0 - 2456 Unique Visitors A Day With Reverse Blogging

{{ DiscussionBoard.errors[423150].message }}

peterwhite 15 years ago

Thanks SaSeoPete - back to the drawing board... not sure why it is not being indexed! I have a feeling it is because the site simply just forwards to another and maybe google does not like it!
- Thanks
{{ DiscussionBoard.errors[423259].message }}

Troy McDonald

15 years ago

I had heard this a long time ago, that the spiders couldn't get in to a Godaddy hosted account. Looks like this is what they are doing.

Originally Posted by SaSeoPete

...
Anyway my DEFAULT robots.txt file was telling the SE's not to crawl my site!

...

Thanks

Signature

Troy McDonald
I will build your IM list for FREE!

{{ DiscussionBoard.errors[423109].message }}

bb7400

15 years ago

Thanks Pete! Great Heads up!

Thanks

{{ DiscussionBoard.errors[423262].message }}

clarissa25

Banned 15 years ago

Thanks so much for the heads up

Thanks

{{ DiscussionBoard.errors[424160].message }}

Didi

15 years ago

Thanks Pete for sharing. I've seen a lot of complains from people using GoDaddy and I think it's best to stay away from them when it comes to hosting issues.

Thanks

{{ DiscussionBoard.errors[424191].message }}

Kim Standerline

15 years ago

Godaddy servers are tied up tighter than a ducks arse, (Try running a script on them)

Pretty crappy for domain registering as well, I buy all mine from namecheap now, tho I must confess I still have a lot left at godaddy

Kim

Thanks

Signature

The Easiest Dimesale Script Ever!!

EasyMemberPro is back with Mama
How To Write And Profit From Hot Romance (Kindle Product)

{{ DiscussionBoard.errors[424200].message }}

cypherslock

15 years ago

most hosts have either windows or linux. Are you running asp scripts or something that you absolutely need windows for? beyond that difference you can use a linux based host (cpanel, ftp etc) just as well.

Thanks

{{ DiscussionBoard.errors[424418].message }}

GarrieWilson

15 years ago

David,

I have above average intelligence and I would *never* think about a host blocking spiders. After all, it's *my* bandwidth not theirs.

Your little analogy about charging batteries dont work either because GoDaddies welcome email doesn't tell you to "charge the batteries" so yes, it is a reflection on them.

GoDaddy SUCKS and this is just one more reason why.

Garrie

[ 1 ] Thanks

Signature

Screw You, NameCheap!
$1 Off NameSilo Domain Coupons:
SAVEABUCKDOMAINS & DOLLARDOMAINSAVINGS

{{ DiscussionBoard.errors[424892].message }}

SaSeoPete

15 years ago

David chill man,

I am not saying GoDaddy does that by default to ALL blogs/websites etc. As all I was saying is that they did it to one of my blogs. I couldn't quite work out why it wasn't being indexed and I found this out.

Just trying to help here bro, take it easy.

Thanks

Signature

0 - 2456 Unique Visitors A Day With Reverse Blogging

{{ DiscussionBoard.errors[424915].message }}

Frenzic

15 years ago

Godaddy makes me confused, well I should have been warned with the name godaddy. If you asked some random people what they though godaddy.com was 90% would answer its a pornsite. Now who wants to be hosted at a porn site?

I use hostgator it litterally eats up all comeption gogo crocodile.

Thanks

{{ DiscussionBoard.errors[427389].message }}

samples

15 years ago

I can tell you that I use hostgator and it is very good hosting, up time 99%

Thanks

{{ DiscussionBoard.errors[437226].message }}

MuayThaiGuy

15 years ago

Thanks for pointing this out. I have wordpress blogs on godaddy hosting.

But I'm confused. Where do I access the robots txt file and make these changes? Do I make the changes from within my wordpress theme editor?

Thanks
1 reply

{{ DiscussionBoard.errors[437251].message }}

Bruce Hearder

15 years ago

The file is in the root directory of your site,

You can always see what your robots.txt files looks like by doing the following using your web browser :

www.your-web-site.com/robots.txt

Of course, change www.your-web-site.com to your actual domain..

To change it (with out using any WP plugins), is usually done using your favorite FTP such as FTPZilla etc, and going to your / directory

Hope this helps

Bruce

Thanks
1 reply

{{ DiscussionBoard.errors[437273].message }}

MuayThaiGuy

15 years ago

Originally Posted by Bruce Hearder

The file is in the root directory of your site,

You can always see what your robots.txt files looks like by doing the following using your web browser :

www.your-web-site.com/robots.txt

Of course, change www.your-web-site.com to your actual domain..

To change it (with out using any WP plugins), is usually done using your favorite FTP such as FTPZilla etc, and going to your / directory

Hope this helps

Bruce

Thanks for the reply ..

When I typed in my domain with the robots.txt extension .. the following was displayed ..

User-agent: *
Disallow:

Do I need to make any changes?

Thanks
1 reply

{{ DiscussionBoard.errors[437283].message }}

Bruce Hearder 15 years ago

No change necessary..

It means that every is just fine..

You can find out more about robots.txt files at

The Web Robots Pages

A good site, but lately they have gort a bit carried away with the amount of Adsense plastered all over it..

Otherwise, definitely worth looking at..

Hope this helps..

Bruce
- Thanks
{{ DiscussionBoard.errors[437288].message }}

madison_avenue

15 years ago

Thetoolwiz said this:

"Do you suspect that most people hosting at GD have blogs they're trying to get indexed? Or a sh*tpile of videos and audios that they share with their friends?

If you've got 40GB of video and audio on your site, SOME spiders WILL READ IT. EVERY SINGLE FREAKING KILOBYTE OF IT. EVERY TIME THEY VISIT.

"Do you look at your server logs? For sites where I have no robots.txt file -- equivalent to "Allow: /" -- twice as much bandwidth is used by spiders than by visitors. I had one site that was nothing but YouTube videos -- and spiders were somehow draining 400MB/mo just on that one folder! I have no clue what they were doing, since there was less than 200kb of content in the folder they were searching."

I think this is a valid point. I have videos on my site: Is there away to get the seo benefit of a spider visit while preventing the spiders from crawling every last byte of video. i.e something like, set it up so they can crawl the titles of the videos but not the content?. Is there something like a alt text system as there is for images?

After all how much benefit do we get from from a video being spidered, can keywords be seen, I don't think so.

Thanks

{{ DiscussionBoard.errors[437271].message }}

Don Schenk

15 years ago

Now I am really confused :-(

MuayThaiGuy says,

QUOTE "Thanks for the reply ..

When I typed in my domain with the robots.txt extension .. the following was displayed ..

User-agent: *
Disallow:

Do I need to make any changes? "

Bruce Hearder says,

QUOTE "No change necessary..

It means that every is just fine..

You can find out more about robots.txt files at..."

but...

SaSeoPete says,

QUOTE
User-agent: *
Disallow: /
That forward slash after the disallow tell Google and co. they can't crawl my site.

So I have changed it now to

User-Agent: *
Allow: /

and

"A robots.txt is very easy to make. All you need to do is open notepad, the copy and paste this:

User-Agent: *
Allow: /

and save is as robots (the .txt will come automatically) and upload it into your root folder."

This seems contradictory.

I'm lost

Thanks for the help.

:-Don

Thanks
1 reply

Signature

Studies prove happier people have massively more success. It has worked for me. Use this one trick too... Click Here

Total Competition Stomper - Simply the most powerful tactic your IM or Offline business will ever use... See it here!

{{ DiscussionBoard.errors[437357].message }}

Bruce Hearder 15 years ago

Hi Don..

The following snippet

User-Agent: *
Allow: /

Means allow everything from the root directory (/) onwards

User-Agent: *
Disallow: /

Means block everything from the root directory onwards..

A small diffrence in letters, but a huge difference in meaning to the BOTS

Does this make things clearer or am I misunderstanding your question

Take care

Bruce
- Thanks
{{ DiscussionBoard.errors[437364].message }}

Don Schenk

15 years ago

I think that helps. So if...

User-Agent: *
Disallow: /

then no robots would follow past the root directory, and the root directory would get listed when a seach engine bot finds is. It that correct.

What happens if there is not a robots.txt on the site?

Thanks

:-Don

Thanks
1 reply

Signature

Studies prove happier people have massively more success. It has worked for me. Use this one trick too... Click Here

Total Competition Stomper - Simply the most powerful tactic your IM or Offline business will ever use... See it here!

{{ DiscussionBoard.errors[437371].message }}

Bruce Hearder

15 years ago

Originally Posted by Don Schenk

I think that helps. So if...

User-Agent: *
Disallow: /

then no robots would follow past the root directory, and the root directory would get listed when a seach engine bot finds is. It that correct.

What happens if there is not a robots.txt on the site?

Thanks

:-Don

Not only would they not go past the root directory, they won't read the root directory either..

If the file is not there, I believe that the Search Engines (SE) will assume that everything can be indexed..

What I have found, is that the SE will call your robots.txt file many times over the first few days and if they get 404 errors they will eventually assume that the file does not exist, and start spidering your site..

But if the file is there from the onset, then your site starts getting spidered really fast..

So its well worth while having a robots.txt file there from the beginning..

Plus, now days you can put a sitemap into your robots.txt file and all the major search engines will follow the sitemap..

Very worthwhile doing..

Hope that helps

Bruce

Thanks
1 reply

{{ DiscussionBoard.errors[437378].message }}

Srikanth D

15 years ago

Originally Posted by Bruce Hearder

Not only would they not go past the root directory, they won't read the root directory either..

If the file is not there, I believe that the Search Engines (SE) will assume that everything can be indexed..

What I have found, is that the SE will call your robots.txt file many times over the first few days and if they get 404 errors they will eventually assume that the file does not exist, and start spidering your site..

But if the file is there from the onset, then your site starts getting spidered really fast..

So its well worth while having a robots.txt file there from the beginning..

Plus, now days you can put a sitemap into your robots.txt file and all the major search engines will follow the sitemap..

Very worthwhile doing..

Hope that helps

Bruce

Nice you mentioned these Bruce. Having a robots.txt file with allow option will definetly make your site indexed faster than not having a file.

Thanks

Signature

>> Getting back into IM game <<

{{ DiscussionBoard.errors[437392].message }}

JustEarnIt

15 years ago

Thanks for the info,i will need to change my settings now..

Thanks

{{ DiscussionBoard.errors[449634].message }}

blairezon blenn

15 years ago

i guess they have reason why they do it but y? can anyone find exact solutions?

cheers!

Thanks

{{ DiscussionBoard.errors[449676].message }}

tghva

15 years ago

They're ok for domains but that's about it

Thanks

Signature

RidersCorral.com The New Equestrian Hangout
Over 20,000 grocery items FREE shipping on most items ClicknShipGroceries.com
Pet products for all your favorite pals ShabbyPaws.com
Handiest Buckets You'll Ever Own! TubTrugBuckets.com

{{ DiscussionBoard.errors[449781].message }}

GeorgR.

15 years ago

just delete robots.txt altogether.

By the way...doesn't Wordpress create/overwrite the robots.txt if you change the privacy settings?

Thanks

Signature

*** Affiliate Site Quick --> The Fastest & Easiest Way to Make Affiliate Sites!<--
-> VISIT www.1UP-SEO.com *** <- Internet Marketing, SEO Tips, Reviews & More!! ***
*** HIGH QUALITY CONTENT CREATION +++ Manual Article Spinning (Thread Here) ***
Content Creation, Blogging, Articles, Converting Sales Copy, Reviews, Ebooks, Rewrites

{{ DiscussionBoard.errors[449793].message }}

KathyBaka

15 years ago

never had a problem with godaddy, but have not used them for hosting
(name cheap is also good for domains) Thanks for the great information.

Kathy Baka
Review Crushers

Thanks

{{ DiscussionBoard.errors[449885].message }}

GoDaddy Warning!

Trending Topics

Best AI for spell checking & grammar (2024)?

Google Ads, landing pages, and Clickbank links

Carry out better target audience research

A Very Strange Internet Problem

YouTube gets shopping display tools and affiliate partnership updates