Robot.txt question ?

by webgain

Posted: 12 years ago 10 replies

I have a site xyz.com and I am using the below code in the robots.txt file
User-agent: *
Disallow: /bl

I have a blog on my website with the url- xyz.com/blog

Now suggest, will the above mentioned robots.txt code affect the indexing of blog. If yes then why?
please give you experienced answer

Thanks

#question #robottxt

UMS 12 years ago

It prevents search engines which honor robots.txt from indexing anything under /bl
- Thanks
Signature

SECockpit - Get the world's most POWERFUL Keyword Research Tool for ONLY $1

Grab a LIFETIME Discount off Elegant Themes

Grab a LIFETIME 40% Discount off The Best Spinner
{{ DiscussionBoard.errors[6883631].message }}
MatthewWoodward 12 years ago

/blog will still be indexed
- Thanks
- 1 reply
Signature

â–ºâ–ºâ–º FREE! How To Build (Steal) An Intelligent SEO Strategy In Minutes...
{{ DiscussionBoard.errors[6884284].message }}
- webgain 12 years ago
  
  Originally Posted by MatthewWoodward
  
  /blog will still be indexed
  
  why it will be index...
  
  someone told me me that if i m using /bl parameter then my url xyz.com/blog will also won't index...
  
  if it won't affect then why not?
  
  Thanks
  
  1 reply
  
  {{ DiscussionBoard.errors[6885543].message }}
  
  MatthewWoodward 12 years ago
  
  Originally Posted by webgain
  
  why it will be index...
  
  someone told me me that if i m using /bl parameter then my url xyz.com/blog will also won't index...
  
  if it won't affect then why not?
  
  Yes if you included a wildcard parameter but in the example you gave you are not.
  
  Thanks
  
  Signature
  
  â–ºâ–ºâ–º FREE! How To Build (Steal) An Intelligent SEO Strategy In Minutes...
  
  {{ DiscussionBoard.errors[6909401].message }}
She 12 years ago

no problem..your blog will be still index by crawler
- Thanks
{{ DiscussionBoard.errors[6884553].message }}
rightknight 12 years ago

If you sign up for webmaster tools, then you can clearly see which pages will be indexed and which will be blocked
- Thanks
{{ DiscussionBoard.errors[6886515].message }}
jyotsna 12 years ago

hey frnd..
your blog will still b indexd as it only hide the content that you do not want google ot crawl..
otherwise it will be indexed..!!
- Thanks
Signature
www.theultimatemania.com
{{ DiscussionBoard.errors[6887500].message }}
webgain 12 years ago

thank you all guys... for you help and support
- Thanks
{{ DiscussionBoard.errors[6918220].message }}
webdevpro 12 years ago

In your case /blog will still be indexed and if you want to block that also then use the wildcards like:

/bl* - this will block /blog also
- Thanks
- 1 reply
Signature

80+ Free Webmaster Tools | Duplicate Content Checker | SEO Forums
{{ DiscussionBoard.errors[6921588].message }}
- webgain 12 years ago
  
  ya got it @webdevpro thanks for the information and help.
  
  Thanks
  
  {{ DiscussionBoard.errors[6924773].message }}

Robot.txt question ?

Trending Topics

Best AI for spell checking & grammar (2024)?

Google Ads, landing pages, and Clickbank links

Carry out better target audience research

A Very Strange Internet Problem

YouTube gets shopping display tools and affiliate partnership updates