AdSense and robots.txt

mesmerist · 07-20-2011, 10:35 PM

I was just exploring my adsense account and I clicked on the "Site Diagnostics" link and I saw a report that lists 4 of my pages under "Blocked URL". The reason for this is Robots.txt File. It then gives instructions on how to edit the robot.txt file. Is this necessary? There are ads displaying on those 4 pages its just that google says in the report: "We encourage you to resolve these issues in order to allow us to crawl more of your pages and provide relevant ads."

Also, since there is one robots.txt, why does it only affect the 4 pages and not the other pages? Someone with technical knowledge out there please help.

07-20-2011, 10:35 PM	#1
mesmerist HyperActive Warrior Join Date: Jan 2011 Posts: 362 Thanks: 25 Thanked 29 Times in 29 Posts	AdSense and robots.txt I was just exploring my adsense account and I clicked on the "Site Diagnostics" link and I saw a report that lists 4 of my pages under "Blocked URL". The reason for this is Robots.txt File. It then gives instructions on how to edit the robot.txt file. Is this necessary? There are ads displaying on those 4 pages its just that google says in the report: "We encourage you to resolve these issues in order to allow us to crawl more of your pages and provide relevant ads." Also, since there is one robots.txt, why does it only affect the 4 pages and not the other pages? Someone with technical knowledge out there please help.