7 {{ upvoteCount | shortNum }}
7 {{ upvoteCount | shortNum }}

About Robots.txt

by easypr
6 replies
  • SEO
    • |
Recently i get one clients, who want to promote their website, when i see the issues of website, and found client disallow index.php in robots. Is it is makes problems in indexing and ranking?
#robotstxt
  • Profile picture of the author danielcollins
    Originally Posted by easypr View Post

    Recently i get one clients, who want to promote their website, when i see the issues of website, and found client disallow index.php in robots. Is it is makes problems in indexing and ranking?
    Yes obviously.. It will create problem in indexing..
    {{ DiscussionBoard.errors[8827397].message }}
  • Profile picture of the author ibarena
    This is incorrect. It will exclude index.php
    Signature
    Stay Fit 24 hours a day
    {{ DiscussionBoard.errors[8827408].message }}
  • Profile picture of the author nettiapina
    It might not be a problem. If you can see their front page in Google search results (without "blocked by robots.txt" text) then it seems to be working just fine. There's usually more than one URL to access the front page (/, index.xxx), and the client may have tried to remove one of those alternatives.
    Signature
    Links in signature will not help your SEO. Not on this site, and not on any other forum.
    Who told me this? An ex Google web spam engineer.

    What's your excuse?
    {{ DiscussionBoard.errors[8829203].message }}
    • Profile picture of the author easypr
      Originally Posted by nettiapina View Post

      It might not be a problem. If you can see their front page in Google search results (without "blocked by robots.txt" text) then it seems to be working just fine. There's usually more than one URL to access the front page (/, index.xxx), and the client may have tried to remove one of those alternatives.
      I also see that, their is not any problem in indexing of pages. When i check the indexing is google, home page is indexed and cache by google. So, we disallow index.php page of website, with the help of robots.txt. its doesn't effect in SEO.
      Signature
      Press Release Writing Services
      {{ DiscussionBoard.errors[8831741].message }}
  • Profile picture of the author affilorama-portal
    Agreed on what nettiapina said. The index.php version of the site may have been de-indexed simply to avoid duplicate content-page issue. De-indexing removes the duplicate version so only ONE version of the homepage link is visible to Google.
    {{ DiscussionBoard.errors[8829555].message }}
  • Profile picture of the author jamesemma4545
    Banned
    [DELETED]
    {{ DiscussionBoard.errors[8831344].message }}
    • Profile picture of the author nettiapina
      Originally Posted by jamesemma4545 View Post

      There are two important considerations when using /robots.txt:
      1] robots can ignore your /robots.txt. Especially malware robots that scan the web for security vulnerabilities, and email address harvesters used by spammers will pay no attention.
      2] the /robots.txt file is a publicly available file. Anyone can see what sections of your server you don't want robots to use.
      Robots.txt is not a security feature, and you should not try to use it as one. It's a wish list for certain bots to not index some parts of your site.

      If you're using a well-known CMS your admin address is usually pretty easy to find, but unless there's a bug or misconfiguration it should not disclose anything important.
      Signature
      Links in signature will not help your SEO. Not on this site, and not on any other forum.
      Who told me this? An ex Google web spam engineer.

      What's your excuse?
      {{ DiscussionBoard.errors[8831640].message }}

Trending Topics