Need help resolving (restricted by) robot.txt errors

fivepennies · 01-10-2012, 02:42 PM

I'm trying to resolve all my crawl errors for one of my domains but there is one crawl error that I just can't figure out. The error is for:

www[dot]domain[dot]com/wp-includes/js/jquery/jquery.js?ver=1.7.1

Here are the steps I took to resolve the robot.txt error myself:

Step One:

I tried to set up 301 redirect in the .htaccess file, but the 301 redirect did not esolve the error in the robot.txt file.

Step Two:

The next step I took was to review the robot.txt file itself and all that is in that file is:

User-agent: *
Disallow: /wp-admin/
Disallow: /wp-includes/

Step Three:

I'm guessing the problem has something to do with my includes file, but I'm not very familiar with robot.txt files so I thought I might consult the Internet marketing community for possible resolution.

Any help you can offer would be greatly appreciated. I tried to search the forum for a thread on this topic, but didn't find the answers I was looking for.

01-10-2012, 02:42 PM	#1
fivepennies Warrior Member Join Date: Nov 2011 Location: Los Angeles Posts: 1 Thanks: 0 Thanked 0 Times in 0 Posts Social Networking Contact Info	Need help resolving (restricted by) robot.txt errors I'm trying to resolve all my crawl errors for one of my domains but there is one crawl error that I just can't figure out. The error is for: www[dot]domain[dot]com/wp-includes/js/jquery/jquery.js?ver=1.7.1 Here are the steps I took to resolve the robot.txt error myself: Step One: I tried to set up 301 redirect in the .htaccess file, but the 301 redirect did not esolve the error in the robot.txt file. Step Two: The next step I took was to review the robot.txt file itself and all that is in that file is: User-agent: * Disallow: /wp-admin/ Disallow: /wp-includes/ Step Three: I'm guessing the problem has something to do with my includes file, but I'm not very familiar with robot.txt files so I thought I might consult the Internet marketing community for possible resolution. Any help you can offer would be greatly appreciated. I tried to search the forum for a thread on this topic, but didn't find the answers I was looking for.