What does the symbol * mean in robots.txt?
- SEO |
For example, what is the difference between the three below:
Disallow: /category/
Disallow: /category/*/*
Disallow: /category/*/*/*
Does it mean not to crawl any further pages or files in sub-directories? Just confused, your help would be greatly appreciated!
-
jsmichael6 -
Thanks
{{ DiscussionBoard.errors[4580922].message }} -