Splinter

I have asked my web designer to track the URL and Keyword for each click coming from my Adwords campaigns usin the HTTP_REFERRER variable but he seems to think that when my online form is submitted, it will only track the last URL.

Can someone who has PHP experience please give me a short briefing on what I should be telling my programmer to enable him to capture the first URL when a user enters my site from a campaign so that we can assign the URL + Keyword to each form submission.

The reasoning for this is that it will enable me to track what keywords from my my website leads convert to actual deals!

Thanks

Aaron Sustar

Your programmer is right, the formal definition of the $_SERVER['HTTP_REFERER'] variable is the following: "The address of the page (if any) which referred the user agent to the current page. This is set by the user agent. Not all user agents will set this, and some provide the ability to modify HTTP_REFERER as a feature. In short, it cannot really be trusted."

So, yes, the "HTTP_REFERER" only tracks the last URL.

However, there is a simple way around it. When users come to your website, check if they already have the cookie "first_http_referer" and if they don't have it, set the value of the cookie "first_http_referer" to $_SERVER['HTTP_REFERER'].

This way you will always have the information you want saved in user's cookie - and the information won't get overwriten as your user continues to browse around your website.

I hope I've helped you with my reply.

Aaron

Try our Revolutionary Article Rewriter (Before All Spots Are Taken!)
and Build 1000's of Highly Relevant Backlinks! (Your Traffic will Explode from this!)

One of RAVING Spin Rewriter Reviews: "Aaron, this is 20 years ahead of the competition!"

rtrotter

Splinter,

If you want to track the AdWords info when a from is submitted you will probably need to do it using javascript.

window.document.referrer - will get the URL and
window.location.search - will get you the query string

The you can place these using javascript as a hidden input and when form is submitted the info will be passed.

Also, this does not rely on cookies.

Your programmer should know the details on how to do this.

Hope that helps,
Rodney

Ping All Your Feed On Auto-Pilot
www.kping.com

Splinter

Aaron my programmer did confirm it will only work providing the user has cookies enabled... Rodney, the code you placed above, is that possible to do in PHP? This is the code we have running now:
================================================
<?php
session_start();
if(!$_SESSION['referer'])
{

$URL = $_SERVER['HTTP_REFERER'];
//first see if the url contains google
if(($pos = strpos($URL, 'google'))||($pos = strpos($URL, 'msn'))||($pos = strpos($URL, 'bing')))
{
// get everything after the q=
$query=strstr($URL, "q=");

$array=explode('&', $query); // get rid of everything after the first &

foreach ($array as $param)
{
if ($param{0} == 'q') // its q, so it's either google or msn/bing
{
$word_string = substr($param, 2); // strip q=
}
}

$keywords = str_replace('"', '', urldecode($word_string)); //clean keywords, get rid of the + and quotes

$_SESSION['referer'] = $URL; // set the url in the session

if($keywords){ $_SESSION['keywords'] = $keywords; } // set the keywords in the session
}
elseif($pos = strpos($URL, 'yahoo'))
{
// get everything after the p=
$query=strstr($URL, "p=");

$array=explode('&', $query); // get rid of everything after the first &

foreach ($array as $param)
{
if ($param{0} == 'p') // its a p so its yahoo
{
$word_string = substr($param, 2); // strip the p=
}
}

$keywords = str_replace('"', '', urldecode($word_string)); //clean keywords, get rid of the + and quotes

$_SESSION['referer'] = $URL; // set the url in the session

if($keywords){ $_SESSION['keywords'] = $keywords; } // set the keywords in the session
}
else
{
//not one of the 3 search engines so just hold the URL
$_SESSION['referer'] = $URL;
}
}
?>

rtrotter

Splinter,

That code looks fine. However, you are going to miss visitors who are not accepting cookies. Also you have to be sure and set those session cookies before anything is written to browser.

You said when form submitted, so your script for processing form reads the cookies. The method I mentioned lets you pass the data with the form and will work even if cookies are not enabled. If your form uses post method then values will be in $_POST global array with key having whatever name you used in the hidden input elements.

Assume you used URL and KW then $_POST['URL'] would replace $URL in your script and $_POST['KW'] would replace $query.

Rodney

Ping All Your Feed On Auto-Pilot
www.kping.com

Splinter

Thanks Rodney, I will swing this past my chap and ask him to make the necessary amendments.

Splinter

Hi Rodney, can you please comment on my programmers reply below. If it is only 5% of users that do not have cookies enabled, it is probably not worth it for the amount of work that is entailed.

====================

What they are suggesting is based on form submission which is fine in so far as the referrer will be the last page that was visited before the form was submitted. This obviously wont work as it is as the visitor may well have visited other pages on your site first which means the referring url will be one of yours. The problem here lies with passing the initial referring url from page to page without using cookies. Having run this past our other programmer we have come up with a way of doing it but it is very complex and would involve changing every link in your site. I will try and explain the process below

• On every page we place a check to determine if $_POST[‘referer’] is set.
• If it isn’t we populate it with $_SERVER[‘HTTP_REFERER’]
• On every page we also have a form with 2 hidden fields, $referer and $location where $referer is set as $_POST[‘referer’] and location (form action) is blank
• Change all the links on your site so instead of going to the page straight away the link calls a javascript function
• The js function will send the destination location e.g. contact-us.php to the hidden form and set it as its action
• The js function then submits the form.

Basically this forces every link on your site to submit the hidden form and post the original referrer each time so the correct value is always carried over to your valuation form.

I have spoken with Jay and he strongly recommends against this as Google does not like links to javascript functions and it may well affect SEO.

To be honest though all browsers have cookies enabled by default and the percentage of people that know how to turn them off let alone turn them off is very small. It seems like a hell of a lot of work to cater for about 5% of internet users. Google Answers: Cookies

To implement this would take between 3-4 hours to code, test and implement.

CMartin

The way you programmer is doing, by using session cookies, is the preferred way. I would use a similar approach and would set a session cookie once a user visits your page and do subsequent checks to see if the session cookie was already created on the other pages he/she might visit. The session cookie would only be set if the referrer host contains the domain (or part of the domain) for the search engines that you are targeting and the respective keyword.

Regarding cookies, almost all browsers accepts *SESSION* cookies by default and security software often don't block it... but persistent cookies can be blocked by security software by default and/or by users.

Even if session cookies were blocked, there are other ways to keep track of the info - for example by appending the session id to the url or hide it in a form, but for this case I wouldn't do it because it can be very complex to implement it, depending on the way your web pages are created.

Tip: tell you programmer to check the "parse_url" and "parse_str" at php.net to parse the referrer url and the query string, as it might help to simplify the code.

Carlos

Ken Durham

Perhaps I am misunderstanding but doesn't Google Analytics have this capability?

Webmaster Services
List Your Wealth Building Systems and Services for Free

Insanity is doing the same thing over and over and expecting a different result ~ Einstein
Insanity is doing the same thing over and over and never getting the same results ~ Ken

Splinter

Not entirely for what I want it for as I want to track what keywords are being typed by each lead which is submitted to me via a form. Then of these leads, a very small % convert into deals so again I can track which keywords are mainly converting to deals.

Analytics will therefore not associate the keyword being used to a specific lead.

Hope that answers your Q

Ken Durham

I see,
This would not be much different than tracking in a shopping cart.
The original referer can be parsed as mentioned above, the IP can be captured, an ID assigned, and all of the info logged in your database when a person first visits the site. The ID can be set to follow them around via a query string if cookies are not used. Any action that this person makes can then be recorded.
If that person submits the form their record is updated. If they convert then the record is further updated.
If URLs on your site are dynamically created, assigning the user id would be simple enough. If not then you would have your work cut out for you if your site is very large.

Or you could always ask the person what the keywords used were as part of your conversion process.

Webmaster Services
List Your Wealth Building Systems and Services for Free

Insanity is doing the same thing over and over and expecting a different result ~ Einstein
Insanity is doing the same thing over and over and never getting the same results ~ Ken

Splinter

HomeComputerGames, thanks for showing me another way it can be done. The site is not that large so we have decided to just track it be capturing the URL and Q string on the users first visit via a cookie. If they do not have cookies enabled, we will loose out however I think the amount of times this will happen will be so far and few between.

Should it become a problem, the idea you have just mentioned along with a couple others above would have to be implemented.

Ken Durham

The more I think about it can you not just match ip's and dates in a report pulled from your server logs?
You should be able to log just about anything needed via your server.
So writing custom reports pulling from these logs may work best by matching IP and date. If %page entry% exists run the report.

Then parse the First referrer entry for this IP + date for your keywords.

Webmaster Services
List Your Wealth Building Systems and Services for Free

Insanity is doing the same thing over and over and expecting a different result ~ Einstein
Insanity is doing the same thing over and over and never getting the same results ~ Ken

Splinter

That is a bit too techie for me. I will run it past my friend who looks after all the IT stuff for me. Thanks

Foresights

I've also been looking for the same solution for a while, and found the following:
Best Contact Form - Features
Seems these guys have cracked it, but it's a hosted service that costs a little too much.

So the knowledge is out there, so hope someone will share someday :-)

- Vince

DatingAngel - free trial
There is No App for That!

RedMatrix

This is an interesting thread!

I wish the [ php ] tags were working.

~Dave

rtrotter

It is not a good idea to track by IP address because it is not necessarily unique to a user. That is, an IP address that is seen at your site can be sured by multiple users. For example, I am in a hotel on local address 192.168.0.99 but every room in the hotel is going out to the web on the same IP. It works better just to assign a unique id generated using md5 and place in cookie.

Another interesting way to track is using flash shared objects. They actually work, per Adobe's report, on 99% of browsers and are much harder to delete than cookies and cannot be turned off like cookies.

Rodney

Ping All Your Feed On Auto-Pilot
www.kping.com

Splinter

Just to let you guys know, we went live with our new site this weekend and our first few leads have come in all with the URL and keywords. So it is working but we will monitor to see if and how many come in with a blank field.

Foresights

Hi,
Would love to hear an update on this.
Thanks,

DatingAngel - free trial
There is No App for That!

Splinter

So far it has been working fine for me... very interesting to see exactly what your users are typing into the query string to find your website

Dan Grossman

You shouldn't be usin' referrers for this anyway. You won't always get one, and what they searched isn't the keyword you bid on, so you will have trouble tieing the referrers back to what keywords to eliminate or spend more on with your AdWords account.

Instead tell Google to pass in the keyword that was matched to your landing page as part of the URL.

Use {keyword} in the destination URL of the ad.

Ex: http://www.your-domain.com/some-page.php?kw={keyword}

Then you can read the keyword that the user's search matched from $_GET['kw'].

Or use a proper tracking system like Google Analytics, or the one I'm releasing this month, to track this kinda thing without having your programmer reinvent the wheel.

I'm Dan Grossman. I run Awio Web Services LLC.

>> Know which 20% of your advertising generates 80% of your sales <<

Splinter

Dan surely what I am doing is matching the keyword to the Q string. Each time a form submission comes through, as part of the email we are sent the HTTP Referrer which includes the keyword. There has been maybe a couple out of a few hundred leads which has not had the keyword so it seems to be fairly accurate.

Now when a lead converts into a purchase (deal), we can start matching what are the real converting keywords for the deals and not just the leads.

Dan Grossman

I'm just saying that the "q string" is not the same as what keyword you bid on that matched that search. Sometimes it won't contain a single word you bid on if Google thinks the concept is close enough to something you bid on. But with only capturing the "q string", you won't be sure what keyword you bid on is performing well (or poorly). You can capture both the searched keyword and the bidded keyword by also tracking the {keyword}.

I'm Dan Grossman. I run Awio Web Services LLC.

>> Know which 20% of your advertising generates 80% of your sales <<

Foresights

Hi Splinter,
Would you mind updating us on the code you are actually using now please.
Maybe we can all collaborate to make it more powerful and try and achieve something similar to the example 3rd party service I pointed to earlier in this thread:
A Website Form the Retrieves Keywords & Search Engine Referrer Information

Dan,
Are you referring to your w3roi.com service?
If so, it doesn't seem to be what we are trying to achieve here; i.e. have the referrer, PPC keywords and organic search terms info sent/saved with each submit of our own web forms.

Thanks,

- Vince

DatingAngel - free trial
There is No App for That!

Splinter

Dan, at the moment I bring back the entire URL in my web form submission as well as the Q string in a separate line. When I look at the Q string and the keywords in the URL that the user typed, they seem the same.

Now are you saying that the KW query actually tells you what keyword made my advert appear on Adwords or organic listings alongside what the user actually typed in (Q String) ?


Vince - I am using the code as show at the beginning of the thread...

Dan Grossman

That's exactly what a service like w3roi does,



but I'm not specifically referring to any site.

You might bid on the keyword "online dating", and the searcher typed in the phrase "online singles". Google might decide those are close enough to match if you have broad match enabled and show your ad.

So what you see in the "q" piece of the referrer will be "online singles" even though that doesn't appear anywhere in your AdWords account; the {keyword} would've given you "online dating" so you can properly match the referral to your bid.

I'm Dan Grossman. I run Awio Web Services LLC.

>> Know which 20% of your advertising generates 80% of your sales <<

Foresights

Hi Dan,
So are you saying the info from the w3roi system can also be posted with the lead via our website contact forms, and therefore sent via email with all lead details?

In other words, same as what bestcontactform.com does?

Thanks,

- Vince

DatingAngel - free trial
There is No App for That!

Splinter

Guys, my Google manager has left for the day and I wanted to raise a quick question. I noticed one of my staff members keeps clicking on the HTTP_Referrer link that is passed back in each lead submission we get. Now since this link sometimes contains that "gclid" id reference Google generates, would it continue to add to be analytics conversion tracking stats each time they click on this link?

jgsketch

I recently tried to implement the same solution you have in place to track keyword conversions. I got the cookie script up and is running. However I cannot seem to pass the values to the hidden fields in my form.

What method did you use to pass the values to your form?

Splinter

From what I can see in my programmers code is that he is storing the values in a session variable which then gets emailed to me when the form submits as well as stored in the db.

Is this what you are doing?

Dan Grossman

If you are storing the values in the cookies, then all you have to do is print them in the form.

<input type="hidden" name="keyword" value="<?php echo $_COOKIE['keyword']; ?>" />

Replacing "keyword" with your actual field and cookie names

I'm Dan Grossman. I run Awio Web Services LLC.

>> Know which 20% of your advertising generates 80% of your sales <<