• 2 {{ upvoteCount | shortNum }}

    Shopify Sites Can Now Edit Their Robots.txt File

    WarriorForum.com in eCommerce

    A new article on Search Engine Journal says that Shopify has started letting stores edit their robots.txt file, which allows greater control over a site's SEO. According to CEO Tobi ... [read more]

  • 4 {{ upvoteCount | shortNum }}

    Does blocking a site from robots.txt also blocks its subdomains access by default?

    CloudGeek in SEO

    Suppose there is a site abc.com . If I stop its indexing from Robots.txt then does it also block indexing for pq.abc.com and yz.abc.com?

  • 8 {{ upvoteCount | shortNum }}

    Robots.txt In SEO ?

    adamandy in SEO

    Do Subdirectories with gTLD Need separate Robots.txt? wwwdotexampledotcom is the global version The global version is included with all robots.txt. I wanted to know do we need to add a ... [read more]

  • 18 {{ upvoteCount | shortNum }}

    Blocking bots with htaccss and robots.txt files

    Shaun J in SEO

    Hi Does anyone know why you should use both the htaccess file and robots.txt to block certain bots from accessing your site instead of just using the robots.txt? I thought ... [read more]

  • 10 {{ upvoteCount | shortNum }}

    Need only tens of pages to be indexed out of hundreds: Robots.txt is Okay to proceed with?

    michealtech in SEO

    Hi all, We 2 sub domains with hundreds of pages where we need only 50 pages to get indexed which are important. Unfortunately the CMS of these sub domains is ... [read more]

  • 4 {{ upvoteCount | shortNum }}

    Robots.txt & internal filter links blocked

    DigitalKnowledge in SEO

    Our internal links appear to be blocked by robots.txt when they are permutated beyond category or brand pages. We did this so that Google wouldn't index all the filters we ... [read more]

  • 8 {{ upvoteCount | shortNum }}

    robots.txt vs meta

    Voora in SEO

    Robots.txt Vs Meta Robots.Which one is better for Effective SEO?

  • 6 {{ upvoteCount | shortNum }}

    Which is the best option Meta Robot tags or robots.txt?

    stephenrobes in SEO

    Hi friends help me to know Which is the best option Meta Robot tags or robots.txt?

  • 0 {{ upvoteCount | shortNum }}

    Disallow Widget Php Directory in robots.txt

    Hims in Programming

    Hi everybody. I have a weather forecast site that offers weather widgets to other sites. The widget is an iframe that pulls data from, let's say, mysite.com/widget/yourforecast.php Search engines follow ... [read more]

  • 2 {{ upvoteCount | shortNum }}

    What is disallow in robots.txt file?

    avemfly619619 in SEO

    Hlo Friends, What is disallow in robots.txt file?

  • 3 {{ upvoteCount | shortNum }}

    Help in Robots.txt file !!!

    benthomas642 in SEO

    Hi there, I need your help. I m confused after looking this in 1 website: Disallow: *? What does it mean? Can anyone please explain here.

  • 3 {{ upvoteCount | shortNum }}

    Search Console can't fetch Robots.txt !

    TheOptimizer in SEO

    Hello everyone, I'm having an issue I've not had before with google search console. https://www.allianzemeklilik.com.tr/robots.txt When I check the robots.txt file it says: "robots.txt fetch failed You have a robots.txt ... [read more]

  • 7 {{ upvoteCount | shortNum }}

    How important is robots.txt for SEO

    diff122 in

    Hi Warriors, Just need some suggestions regarding the importance of robots.txt for good SEO. How important is that? The reason why I am asking is I have been ranking on ... [read more]