Need tool which does this

3 replies
  • SEO
  • |
Crawl a site, obeying NoIndex/NoFollow and show me exactly like Google would index the page, eg. pages which are NoIndex/NF in different colors.

G.
#tool
  • Profile picture of the author kernelpaniker
    I dont know about "exactly like googlebot" but Xenu Link Sleuth is good for crawling.
    {{ DiscussionBoard.errors[3302796].message }}
  • Profile picture of the author GeorgR.
    yes i have xenu, but not sure whether it does that.
    Signature
    *** Affiliate Site Quick --> The Fastest & Easiest Way to Make Affiliate Sites!<--
    -> VISIT www.1UP-SEO.com *** <- Internet Marketing, SEO Tips, Reviews & More!! ***
    *** HIGH QUALITY CONTENT CREATION +++ Manual Article Spinning (Thread Here) ***
    Content Creation, Blogging, Articles, Converting Sales Copy, Reviews, Ebooks, Rewrites
    {{ DiscussionBoard.errors[3302801].message }}
    • Profile picture of the author paulgl
      Crawl a site, obeying NoIndex/NoFollow and show me exactly like Google would index the page, eg. pages which are NoIndex/NF in different colors.
      Not sure what you mean by obeying nofollow. That has nothing
      to do with crawling or indexing. Your pages would not be nofollow.
      The links to the pages could be nofollow, but that does not keep
      them from being indexed. Only noindex will. If a page is on the web,
      and does not have the noindex, it could be indexed.

      Paul
      Signature

      If you were disappointed in your results today, lower your standards tomorrow.

      {{ DiscussionBoard.errors[3302933].message }}

Trending Topics