Googlebot is blocked from my site... How do I fix it?

by rimam1

Posted: 15 years ago 11 replies

PROGRAMMING

I have a site Website Marketing Consultants | Pay For Performance Internet Marketing Consultants and get this message in Google webmaster tools:

"Googlebot is blocked from http://websitemarketingconsultants.org/"

It also says:

"User-agent: *
Disallow: /

Sitemap: http://websitemarketingconsultants.org/sitemap.xml.gz"

How do I fix this?

Thanks,
Raza

#blocked #fix #googlebot #site

TristanPerry 15 years ago

Change the word "Disallow:" to the word "Allow:" in your robots.txt file. So edit this file:

http://websitemarketingconsultants.org/robots.txt

And make it so that it says:

User-agent: *
Allow: /

Sitemap: http://websitemarketingconsultants.org/sitemap.xml.gz
- Thanks
- 1 reply
Signature
Plagiarism Guard - Protect Against Content Theft
{{ DiscussionBoard.errors[1116766].message }}
- rimam1 15 years ago
  
  Thanks.
  
  The problem is I don't know where to find my robots.txt file. Also, I did a re-direct to my site recently. I wanted it to go from Website Marketing Consultants | Pay For Performance Internet Marketing Consultants to websitemarketingconsultants.org
  
  Any suggestions on how where to find robots.txt and if the redirect has anything to do with it?
  
  Thanks again,
  Raza
  
  Thanks
  
  Signature
  5x Your Focus: Exactly How a Busy Father of 3, With a Full-Time Job, and a Grueling 2 Hour Commute Makes an Extra $500 to $2000 per Month... on Autopilot
  
  -->Click Here for Free Access<--
  
  {{ DiscussionBoard.errors[1116929].message }}
TristanPerry 15 years ago

Hi Raza,
The re-direct shouldn't affect it. The robots.txt file is just in your public_html/base directory with all your other files. How did you get your website up in the first place? Did you upload them via cPanel or via FTP? You can also always ask your host on instructions on how to find/edit it.

The re-direct shouldn't affect it; a re-direct won't change the robots.txt file. Robots.txt is a text file and will only really be manually changed.

Thanks,
Tristan
- Thanks
- 1 reply
Signature
Plagiarism Guard - Protect Against Content Theft
{{ DiscussionBoard.errors[1117114].message }}
- rapidprog 15 years ago
  
  Your Wordpress settings are probably causing this.
  
  Go to Settings -> Privacy.
  
  Under "Blog Visibility" make sure this is checked:
  "I would like my blog to be visible to everyone, including search engines (like Google, Sphere, Technorati) and archivers"
  
  Otherwise, Wordpress will generate code to block your site from the spiders.
  
  - Ken
  
  [ 1 ] Thanks
  
  1 reply
  
  {{ DiscussionBoard.errors[1123530].message }}
  
  rimam1 15 years ago
  
  Thanks Ken,
  
  I think that was the problem. Still not cached according to SEOBook toolbar, but I'm building links to my site again so it should get indexed and cached soon.
  
  Thanks
  Raza
  
  Thanks
  
  Signature
  5x Your Focus: Exactly How a Busy Father of 3, With a Full-Time Job, and a Grueling 2 Hour Commute Makes an Extra $500 to $2000 per Month... on Autopilot
  
  -->Click Here for Free Access<--
  
  {{ DiscussionBoard.errors[1128429].message }}
absolutelee 15 years ago

Hi Rapidprog!

I've been struggling with this same issue on a WP blog for a couple of hours. You're answer solved my problem. I would never have thought to look at the privacy settings. I thought it was a problem with some of my plugins.

Thanks!

Lee
- Thanks
Signature

Snag My YouTube Ads Case Study...FREE!
{{ DiscussionBoard.errors[1178792].message }}
marynorth21 15 years ago

I know it's overtime but may I ask if how can I put nofollow tag on my site page?
- Thanks
- 1 reply
{{ DiscussionBoard.errors[1181696].message }}
- TristanPerry 15 years ago
  
  Originally Posted by marynorth21
  
  I know it's overtime but may I ask if how can I put nofollow tag on my site page?
  
  Just edit any <a> tags to include the attribute:
  
  rel="nofollow"
  
  So a link which is:
  
  <a href="index.html">Homepage</a>
  
  Would becme:
  
  <a href="index.html" rel="nofollow">Homepage</a>
  
  etc.
  
  Thanks
  
  Signature
  Plagiarism Guard - Protect Against Content Theft
  
  {{ DiscussionBoard.errors[1181851].message }}
warrich 15 years ago

try robots.txt to allow it ,and check in webmaster tool where is the problem and why it is not crawling ur site
- Thanks
Signature

High Quality Blog Post Services | Strong PA/DA | Strong Trust Flow | 100% Guaranteed Rank
Affiliate Marketing Forum
{{ DiscussionBoard.errors[1207533].message }}
mad 15 years ago

If you don't have robots.txt, make sure you have

<meta name="robots" content="index,follow">

in meta tags in the index page of your site. Also you can create a site map in Google webmaster tool
- Thanks
{{ DiscussionBoard.errors[1267823].message }}
amarcs427 10 years ago

Just you visit my site it will help you for creating your best wordpress blog or site click here or IT Solution Tricks – WordPress
- Thanks
{{ DiscussionBoard.errors[8832562].message }}

Googlebot is blocked from my site... How do I fix it?

Trending Topics

Newbie

How did you turn it all around?

best high quality crypto traffic.

What's the Best IM-Related Skill to Learn Today That Can Help in the Future?

A healthy fascination for Hollywood...