html Redirect Link

Pat Jackson · 10-07-2009, 12:51 AM

Hey guys,

I've been using a basic html re-direct to cloak my affiliate links. It looks like this;

Code:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type"
content="text/html; charset=iso-8859-1" />
<meta http-equiv="Refresh"
content="1; URL=AFF LINK" target="_blank">
<title>TITLE</title>
</head>
<body>
<br />
</body>
</html>

I've just been reading that I should be making these links nofollow because Google is beginning to penalize them if you don't.

Am I right that I just have to add

Code:

<meta name="robots" content="nofollow" />

to do this?

If yes, is this the place where I'm supposed to put it?

Code:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type"
content="text/html; charset=iso-8859-1" />
<meta http-equiv="Refresh"
content="1; URL=AFF LINK" target="_blank">
<title>TITLE</title>
<meta name="robots" content="nofollow" />
</head>
<body>
<br />
</body>
</html>

Is that in the right spot or do I need to put it somewhere else?

Thanks in advance!

TheLimeDesign · 10-07-2009, 04:08 AM

Yep, that's the place you put them (anywhere within the <head> tag)

Alternatively, you can also centralize it in your robot.txt files. Helps a lot if you have a huge number of pages.

Pat Jackson · 10-07-2009, 04:49 AM

Quote:

Originally Posted by TheLimeDesign

Yep, that's the place you put them (anywhere within the <head> tag)

Alternatively, you can also centralize it in your robot.txt files. Helps a lot if you have a huge number of pages.

Yeah, there is quite a few pages so that would be better!

What do I need to put in my robots.txt?

TheLimeDesign · 10-07-2009, 05:59 AM

okay, this is your basic robot.txt file content:

Code:

User-agent: *
Disallow: /examplefolder1/

User-Agent: Googlebot
Disallow: /examplefolder2/

Following the top code, everything within "examplefolder1" folder will NOT be indexed by ALL search engines, while the "examplefolder2" folder will NOT be indexed ONLY by Google.

This way, you can sort out which FOLDERS you want to index and which you want to leave out. This covers the WHOLE FOLDER. To customize this further, you can use the following:

To block the entire site, use a forward slash.

Code:

Disallow: /

To block a directory and everything in it, follow the directory name with a forward slash.

Code:

Disallow: /junk-directory/

To block a page, list the page.

Code:

Disallow: /private_file.html

To remove a specific image from Google Images, add the following:

Code:

User-agent: Googlebot-Image
Disallow: /images/dogs.jpg

To remove all images on your site from Google Images:

Code:

User-agent: Googlebot-Image
Disallow: /

To block files of a specific file type (for example, .gif), use the following:

Code:

User-agent: Googlebot
Disallow: /*.gif$

To prevent pages on your site from being crawled, while still displaying AdSense ads on those pages, disallow all bots other than Mediapartners-Google. This keeps the pages from appearing in search results, but allows the Mediapartners-Google robot to analyze the pages to determine the ads to show. The Mediapartners-Google robot doesn't share pages with the other Google user-agents. For example:

Code:

User-agent: *
Disallow: /

User-agent: Mediapartners-Google
Allow: /

For multiple items,

Code:

User-agent: *
Disallow: /file1.html
Disallow: /page4.html

User-Agent: Googlebot
Disallow: /examplefolder2/
Disallow: /junk-page.html

Hope that helps

Pat Jackson · 10-07-2009, 06:51 AM

Quote:

Originally Posted by TheLimeDesign

okay, this is your basic robot.txt file content:

Code:

User-agent: *
Disallow: /examplefolder1/

User-Agent: Googlebot
Disallow: /examplefolder2/

Following the top code, everything within "examplefolder1" folder will NOT be indexed by ALL search engines, while the "examplefolder2" folder will NOT be indexed ONLY by Google.

This way, you can sort out which FOLDERS you want to index and which you want to leave out. This covers the WHOLE FOLDER. To customize this further, you can use the following:

Thanks for that. noindex and nofollow aren't the same thing, though, are they?

All of the links I'm talking about are in a folder called "Recommends" so would the following set up;

Code:

User-agent: *
Disallow: /Recommends/

Stop Google from following the links or just stop them from indexing them? (Forgive me if it's the same thing)

TheLimeDesign · 10-07-2009, 07:42 AM

Hey Pat,

I'm really sorry, I overlooked the post. If index is what you are looking for, then follow the post above I gave. You are right, nofollow and noindex are not the same. A page can be indexed and nofollowed at the same time and vice versa.

The main purpose of these are actually related to the indexing, ability of crawling and the PR of the page. Basically, nofollow could not be integrated into robots.txt. That said, you will have to insert the nofollow meta tag in the head section of all pages of yours (I hope you are using PHP).

To make it easier to understand,
noindex (meta) - disables indexing of the page, but your page will still be crawled and the PR will be affected
nofollow (meta) - does index your page still, but the links are NOT crawled and it does NOT contribute to your PR at all
robots.txt - This is the file the bot searches first before finding the pages. robots file will only send the URL of your pages to google and whether or not it should be processed further. It does not have any affect on the crawling of your pages but it does affect PR rate of your page.

Once again, sorry for the confusion. I hope I made it clearer this time.

Cheers~

10-07-2009, 12:51 AM	#1
Pat Jackson Senior Warrior Member War Room Member Join Date: Jan 2009 Location: Melbourne, Australia Posts: 1,380 Thanks: 648 Thanked 191 Times in 129 Posts	html Redirect Link Hey guys, I've been using a basic html re-direct to cloak my affiliate links. It looks like this; Code: <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head> <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" /> <meta http-equiv="Refresh" content="1; URL=AFF LINK" target="_blank"> <title>TITLE</title> </head> <body> <br /> </body> </html> I've just been reading that I should be making these links nofollow because Google is beginning to penalize them if you don't. Am I right that I just have to add Code: <meta name="robots" content="nofollow" /> to do this? If yes, is this the place where I'm supposed to put it? Code: <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <head> <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" /> <meta http-equiv="Refresh" content="1; URL=AFF LINK" target="_blank"> <title>TITLE</title> <meta name="robots" content="nofollow" /> </head> <body> <br /> </body> </html> Is that in the right spot or do I need to put it somewhere else? Thanks in advance!

10-07-2009, 04:08 AM	#2
TheLimeDesign Ultimate War Machine War Room Member Join Date: Oct 2009 Posts: 63 Thanks: 2 Thanked 12 Times in 12 Posts Contact Info	Re: html Redirect Link Yep, that's the place you put them (anywhere within the <head> tag) Alternatively, you can also centralize it in your robot.txt files. Helps a lot if you have a huge number of pages.
	Quality Graphics with INSANE Prices coming soon! Be the first to know! \| TheLimeDesign Official Site "Building Relationships. Not websites" Last edited by TheLimeDesign; 10-07-2009 at 04:08 AM. Reason: extra tip

10-07-2009, 05:59 AM	#4
TheLimeDesign Ultimate War Machine War Room Member Join Date: Oct 2009 Posts: 63 Thanks: 2 Thanked 12 Times in 12 Posts Contact Info	Re: html Redirect Link okay, this is your basic robot.txt file content: Code: User-agent: * Disallow: /examplefolder1/ User-Agent: Googlebot Disallow: /examplefolder2/ Following the top code, everything within "examplefolder1" folder will NOT be indexed by ALL search engines, while the "examplefolder2" folder will NOT be indexed ONLY by Google. This way, you can sort out which FOLDERS you want to index and which you want to leave out. This covers the WHOLE FOLDER. To customize this further, you can use the following: To block the entire site, use a forward slash. Code: Disallow: / To block a directory and everything in it, follow the directory name with a forward slash. Code: Disallow: /junk-directory/ To block a page, list the page. Code: Disallow: /private_file.html To remove a specific image from Google Images, add the following: Code: User-agent: Googlebot-Image Disallow: /images/dogs.jpg To remove all images on your site from Google Images: Code: User-agent: Googlebot-Image Disallow: / To block files of a specific file type (for example, .gif), use the following: Code: User-agent: Googlebot Disallow: /.gif$ To prevent pages on your site from being crawled, while still displaying AdSense ads on those pages, disallow all bots other than Mediapartners-Google. This keeps the pages from appearing in search results, but allows the Mediapartners-Google robot to analyze the pages to determine the ads to show. The Mediapartners-Google robot doesn't share pages with the other Google user-agents. For example: Code: User-agent: Disallow: / User-agent: Mediapartners-Google Allow: / For multiple items, Code: User-agent: * Disallow: /file1.html Disallow: /page4.html User-Agent: Googlebot Disallow: /examplefolder2/ Disallow: /junk-page.html Hope that helps
	Quality Graphics with INSANE Prices coming soon! Be the first to know! \| TheLimeDesign Official Site "Building Relationships. Not websites"

10-07-2009, 07:42 AM	#6
TheLimeDesign Ultimate War Machine War Room Member Join Date: Oct 2009 Posts: 63 Thanks: 2 Thanked 12 Times in 12 Posts Contact Info	Re: html Redirect Link Hey Pat, I'm really sorry, I overlooked the post. If index is what you are looking for, then follow the post above I gave. You are right, nofollow and noindex are not the same. A page can be indexed and nofollowed at the same time and vice versa. The main purpose of these are actually related to the indexing, ability of crawling and the PR of the page. Basically, nofollow could not be integrated into robots.txt. That said, you will have to insert the nofollow meta tag in the head section of all pages of yours (I hope you are using PHP). To make it easier to understand, noindex (meta) - disables indexing of the page, but your page will still be crawled and the PR will be affected nofollow (meta) - does index your page still, but the links are NOT crawled and it does NOT contribute to your PR at all robots.txt - This is the file the bot searches first before finding the pages. robots file will only send the URL of your pages to google and whether or not it should be processed further. It does not have any affect on the crawling of your pages but it does affect PR rate of your page. Once again, sorry for the confusion. I hope I made it clearer this time. Cheers~
	Quality Graphics with INSANE Prices coming soon! Be the first to know! \| TheLimeDesign Official Site "Building Relationships. Not websites"