Problem with Robots.txt

by AmitShaw

Posted: 14 years ago 6 replies

INTERNET MARKETING

I have installed Yoast Plugin and Disable Author,Tag, Category (Nofollow and NoIndex)
And i have disallowed tag author and category from robots.txt.
Still my tags, category,author's are indexing on Google.
Here is my Robots.txt Link : www.smartphonegyan.com/robots.txt
Here you can check my site's links are indexed.
site:smartphonegyan.com

Waiting for your reply. Let me know how to solve it. Thanks.

#problem #robotstxt

Bill_Z 14 years ago

They might have been indexed before you blocked them with your robots file.
- Thanks
- 1 reply
Signature

Affordable Whiteboard Animation Videos!

Get More Buyers & Increase Conversions
Grab Visitors Attention With Hottest Trend of 2013!
{{ DiscussionBoard.errors[7578524].message }}
- AmitShaw 14 years ago
  
  Nope. . . First i followed everything then i published my article after 7 days......
  
  Thanks
  
  Signature
  
  Blogger at iTechCode. Visit Here : iTechCode Resources
  
  {{ DiscussionBoard.errors[7578526].message }}
Bill_Z 14 years ago

I'm not sure what you mean in your last reply. When I do the site: command I only see 2 pages indexed which are your homepage and your author page. If you created the author page before blocking the /author directory in robots then it will get indexed.
- Thanks
- 1 reply
Signature

Affordable Whiteboard Animation Videos!

Get More Buyers & Increase Conversions
Grab Visitors Attention With Hottest Trend of 2013!
{{ DiscussionBoard.errors[7578531].message }}
- AmitShaw 14 years ago
  
  Thanks for your reply Bill : Please check this screehsot : site:smartphonegyan.com - Google Search | Awesome Screenshot
  Here almost my tags are indexed too. . . . First i created robots.txt then i published my article including tag and category and its got indexed. That is why i am worried.
  
  Thanks
  
  Signature
  
  Blogger at iTechCode. Visit Here : iTechCode Resources
  
  {{ DiscussionBoard.errors[7578542].message }}
Bill_Z 14 years ago

Here is what I see: 2013-01-09_1341 - Billy_Z's library

I know robot.txt isn't foolproof and stuff still does get indexed. In past what I've done is remove them from the index manually via webmaster fools if robots.txt hasn't blocked them properly.
- [ 1 ] Thanks
- 1 reply
Signature

Affordable Whiteboard Animation Videos!

Get More Buyers & Increase Conversions
Grab Visitors Attention With Hottest Trend of 2013!
{{ DiscussionBoard.errors[7578676].message }}
- AmitShaw 14 years ago
  
  Thanks for checking Bill. Let me know what should i do now? Should i manually remove those and wait for next article to recheck its indexing or not?
  Thanks.
  
  Thanks
  
  Signature
  
  Blogger at iTechCode. Visit Here : iTechCode Resources
  
  {{ DiscussionBoard.errors[7578699].message }}

Problem with Robots.txt

Trending Topics

Lightening strikes :(

If you had to start over today -- what would you do ?

" It is what it is" ... overused way too much imo

Are you into the Stock Market??

What would you do without the Internet ??