Go Back   WarriorForum - Internet Marketing Forums > The Warrior Forum > Adsense / PPC / SEO Discussion Forum
Register Blogs FAQ Social Groups CalendarHelp Desk

Reply
 
LinkBack Thread Tools
Old 10-12-2010, 05:35 AM   #1
Warrior Member
 
Join Date: Oct 2010
Posts: 9
Thanks: 0
Thanked 0 Times in 0 Posts
Default Quick Reply need about Robots.txt file

Which is the better way to restrict Search Engines to index my Directory: "uploads" and the content in it?

User-agent: *
1: Disallow: /httpdocs/php/uploads/

2: User-agent: *
Disallow: /php/uploads/

3: User-agent: *
Disallow: /uploads/

if I use /php/uploads/ will search engines not index my "PHP" Directory?

asghar is offline   Reply With Quote
Old 10-13-2010, 04:07 AM   #2
Warrior Member
 
Join Date: Oct 2010
Posts: 9
Thanks: 0
Thanked 0 Times in 0 Posts
Default Re: Quick Reply need about Robots.txt file

Also,
can I hide robots.txt file from visitors like if someone open the link "/httpdocs/robots.txt" but I dont want to show that which directory is most important but Search Engines can still read this robots txt file, is there any way?

asghar is offline   Reply With Quote
Old 10-13-2010, 04:22 AM   #3
Warrior Member
 
Join Date: Oct 2010
Posts: 4
Thanks: 0
Thanked 1 Time in 1 Post
Default Re: Quick Reply need about Robots.txt file

I am looking for same stuff now a days if any have more idea about this then please mention here I am eagerly waiting for this .

emmamartin123 is offline   Reply With Quote
Old 10-13-2010, 04:54 AM   #4
Active Warrior
 
Join Date: Oct 2010
Posts: 29
Thanks: 0
Thanked 2 Times in 2 Posts
Default Re: Quick Reply need about Robots.txt file

They will index. You have to restrict for php folder too..

Maheshpandit is offline   Reply With Quote
Old 10-13-2010, 05:44 AM   #5
Senior Warrior Member
 
Join Date: May 2009
Posts: 1,259
Thanks: 21
Thanked 54 Times in 52 Posts
Default Re: Quick Reply need about Robots.txt file

Agree w/ the comment above - they should index, so you will have to restrict bots from indexing them.

I believe the first one should work:

User-agent: *
Disallow: /httpdocs/php/uploads/

bigcat1967 is offline   Reply With Quote
Old 10-13-2010, 06:17 AM   #6
Active Warrior
 
Join Date: Oct 2010
Location: cochin
Posts: 34
Thanks: 1
Thanked 2 Times in 2 Posts
Social Networking View Member's FaceBook Profile  View Member's Twitter Profile 
Default Re: Quick Reply need about Robots.txt file

Google will index robots.txt and sitemap files first so there is no way to hide these files from Google and the format for not crawling that 'upload' folder is

Useragent : Google
Disallow : /uploads

leapharris is offline   Reply With Quote
Old 10-13-2010, 08:42 AM   #7
Plundering the Web
War Room Member
 
paulgl's Avatar
 
Join Date: Feb 2007
Location: , , .
Posts: 4,849
Thanks: 804
Thanked 1,199 Times in 886 Posts
Default Re: Quick Reply need about Robots.txt file

Put your uploads in a password protected folder.

Google is not the only one indexing stuff...

Paul

How to Make Money off Facebook: Login to your account. Deactivate your account. Get your butt to work.
paulgl is offline   Reply With Quote
Old 10-13-2010, 08:54 AM   #8
Warrior Member
 
Join Date: Oct 2010
Posts: 9
Thanks: 0
Thanked 0 Times in 0 Posts
Default Re: Quick Reply need about Robots.txt file

Quote:
Originally Posted by paulgl View Post
Put your uploads in a password protected folder.

Google is not the only one indexing stuff...

Paul
that looks like a decent answer, I will have to try this one and I will let you know the results

Thanks buddy

asghar is offline   Reply With Quote
Reply

  WarriorForum - Internet Marketing Forums > The Warrior Forum > Adsense / PPC / SEO Discussion Forum

Tags
file, quick, reply, robotstxt

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off



All times are GMT -6. The time now is 10:59 PM.