Go Back   WarriorForum - Internet Marketing Forums > Warrior Support Forums > Programming Talk
Register Blogs FAQ Social Groups CalendarHelp Desk

Reply
 
LinkBack Thread Tools
Old 07-18-2009, 02:38 PM   #1
SocialAdr.com
War Room Member
 
khtm's Avatar
 
Join Date: Apr 2009
Location: San Diego
Posts: 318
Thanks: 56
Thanked 29 Times in 27 Posts
Social Networking View Member's Myspace Profile  View Member's FaceBook Profile  View Member's Twitter Profile  View Member's YouTube Profile
Default Remote form submission via PHP Curl

Hey nerd-buddies!

I have a problem -

I need to write a bunch of interfaces to social networking websites that don't have APIs available. I've been using PHP Curl to connect and in some cases have been successful at logging in but then can't seem to submit a form on another page.

The code needs to be ran automatically via a cron job (so...behind the scenes).

Does anyone have some mad skills at simulating remote form submissions via PHP? Or should I be using javascript or another language?

Thanks!

khtm is offline   Reply With Quote
Old 07-18-2009, 03:57 PM   #2
Active Warrior
 
Join Date: Jul 2009
Location: Pennsylvania, US
Posts: 30
Thanks: 2
Thanked 12 Times in 10 Posts
Lightbulb Re: Remote form submission via PHP Curl

Kane,

It's going to depend on the site you're using. Something like Facebook runs on Javascript 99% of the time, so straight form submissions aren't going to work for you there.

If the site is 100% non-RPC based, then you might just need to capture the login cookie(s) that gets set and make sure you reply with that header when posting new form submissions.

Code:
$user_agent = 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0';
$headers = array (
    'HTTP_ACCEPT' => 'application/x-shockwave-flash,text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5', 
    'HTTP_ACCEPT_LANGUAGE' => 'en-us,en;q=0.5',
    'HTTP_ACCEPT_CHARSET' => 'ISO-8859-1,utf-8;q=0.7,*;q=0.7',
    'HTTP_KEEP_ALIVE' => '300',
    'HTTP_CONNECTION' => 'keep-alive'
    );
    
    

function read_header($ch, $string)
{
   global $location; #keep track of location/redirects
   global $cookiearr; #store cookies here
   global $ch;
       # ^overrides the function param $ch  this is okay because 
       # we need to update the global $ch with new cookies
  
   $length = strlen($string);
   if(!strncmp($string, "Location:", 9)) #keep track of last redirect
   { $location = trim(substr($string, 9, -1)); }
   
   if(!strncmp($string, "Set-Cookie:", 11)) #get the cookie
   { 
     $cookiestr = trim(substr($string, 11, -1));
     $cookie = explode(';', $cookiestr);
     $cookie = explode('=', $cookie[0]);
     $cookiename = trim(array_shift($cookie));
     $cookiearr[$cookiename] = trim(implode('=', $cookie));
   }
   $cookie = "";
   if(trim($string) == "") #execute only at end of header
   {  
     foreach ($cookiearr as $key=>$value) { $cookie .= "$key=$value; "; }
     curl_setopt($ch, CURLOPT_COOKIE, $cookie);
   }

   return $length;
}



$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADERFUNCTION, 'read_header');  # take their cookie and love it!
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 5);
curl_setopt($ch, CURLOPT_AUTOREFERER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_USERAGENT, $user_agent);
curl_setopt($ch, CURLOPT_REFERER, $url_base);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_URL, $url_login);

$string = curl_exec($ch);
Set your unset variables accordingly and give it a run.

Hope this helps!

Hit the Thanks in the lower right if you found this post useful. Thanks! ;-)
Kiva.org - Help aspiring entrepreneurs by giving low sum loans.
maxleadford is offline   Reply With Quote
Old 07-18-2009, 04:32 PM   #3
SocialAdr.com
War Room Member
 
khtm's Avatar
 
Join Date: Apr 2009
Location: San Diego
Posts: 318
Thanks: 56
Thanked 29 Times in 27 Posts
Social Networking View Member's Myspace Profile  View Member's FaceBook Profile  View Member's Twitter Profile  View Member's YouTube Profile
Default Re: Remote form submission via PHP Curl

Hey Max, thanks for the help!

I think I have a lot to learn about cookies and HTTP headers...this stuff is still confusing for me.

The first site I'm trying this on is :: BuddyMarks ::

I wanted to first initiate a login and then insert a new bookmark. I simply viewed the form details and tried to use Curl POSTFIELDS to copy them all.

Here's my original code:

PHP Code:
 'myusername';
 = 
'mypassword';

// Post variables
 
'form_username='.urlencode().'&form_password='.urlencode();
 .= 
'&action=login';

// Curl setup
 
curl_init();
curl_setopt(, CURLOPT_URL'http://www.buddymarks.com/login.php');
curl_setopt(, CURLOPT_POST1);
curl_setopt(, CURLOPT_POSTFIELDS, );
curl_setopt(, CURLOPT_HEADER0);
curl_setopt(, CURLOPT_FOLLOWLOCATION1);
curl_setopt(, CURLOPT_SSL_VERIFYPEERfalse);
curl_setopt(, CURLOPT_COOKIEJAR"my_cookies.txt");
curl_setopt(, CURLOPT_COOKIEFILE"my_cookies.txt");
curl_setopt(, CURLOPT_RETURNTRANSFER1);
curl_setopt(, CURLOPT_USERAGENT"Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.9) Gecko/2008052906 Firefox/3.0 (.NET CLR 3.5.30729) Creative ZENcast v2.00.14");

// Login
 
curl_exec();

// Post variables
 
'&back_url=http://buddymarks.com/view_group.php?id=938139';
 .= 
'&form_buddymark=true';
 .= 
'&form_public_link=true';
 .= 
'&form_group_type=existing';
 .= 
'&form_group_id=938139';
 .= 
'&form_group_title=';
 .= 
'&form_group_parent_id=0';
 .= 
'&form_cat_id=2';
 .= 
'&bookmark_title=hello';
 .= 
'&bookmark_url=http://im-fun.com';
 .= 
'&form_description=my blog';
 .= 
'&form_tags=tags';
 .= 
'&form_private=2';

// Curl setup
curl_setopt(, CURLOPT_URL'http://www.buddymarks.com/add_bookmark.php?action=insert_bookmark');
curl_setopt(, CURLOPT_POST1);
curl_setopt(, CURLOPT_POSTFIELDS, );
curl_setopt(, CURLOPT_HEADER0);
curl_setopt(, CURLOPT_FOLLOWLOCATION1);
curl_setopt(, CURLOPT_SSL_VERIFYPEERfalse);
curl_setopt(, CURLOPT_COOKIEJAR"my_cookies.txt");
curl_setopt(, CURLOPT_COOKIEFILE"my_cookies.txt");
curl_setopt(, CURLOPT_RETURNTRANSFER1);
curl_setopt(, CURLOPT_USERAGENT"Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US; rv:1.9) Gecko/2008052906 Firefox/3.0 (.NET CLR 3.5.30729) Creative ZENcast v2.00.14");

// Insert bookmark
 
curl_exec();

// Display the follow location response
echo ; 
But unfortunately I was only able to login and couldn't add the bookmark.

I'm not really sure how to proceed with your example. Does that code just grab the header info and save it as a cookie? Then I'd still need to initiate a LOGIN using the same header info?

I tried putting the "meat & potatoes" of my code after your code and it seemed to get me further. Instead of the ADD_BOOKMARK.php page redirecting me back to itself after the curl_exec(), it actually says "Public Bookmark not saved!" this time. But I'm not sure how to debug that particular message.

khtm is offline   Reply With Quote
Old 07-18-2009, 04:35 PM   #4
SocialAdr.com
War Room Member
 
khtm's Avatar
 
Join Date: Apr 2009
Location: San Diego
Posts: 318
Thanks: 56
Thanked 29 Times in 27 Posts
Social Networking View Member's Myspace Profile  View Member's FaceBook Profile  View Member's Twitter Profile  View Member's YouTube Profile
Default Re: Remote form submission via PHP Curl

Wait a sec!

It actually did work! Even though it says "Public Bookmark not saved" I checked and it was actually there.

Sweeeeet

You're a genius, Max.

khtm is offline   Reply With Quote
Old 07-18-2009, 05:55 PM   #5
Active Warrior
 
Join Date: Jul 2009
Location: Pennsylvania, US
Posts: 30
Thanks: 2
Thanked 12 Times in 10 Posts
Default Re: Remote form submission via PHP Curl

khtm,

Glad to hear it helped.

I wrote this as part of a data scraper three years ago to grab some data from a public records website for some real estate stuff. I'm actually a bit foggy on the specifics, but I'm pretty sure that as long as you stay in the same cURL session (in my example $ch), your cookies should stay in place.

Good luck on your endeavor!

Hit the Thanks in the lower right if you found this post useful. Thanks! ;-)
Kiva.org - Help aspiring entrepreneurs by giving low sum loans.
maxleadford is offline   Reply With Quote
Old 07-18-2009, 07:28 PM   #6
Active Warrior
 
Join Date: Jul 2009
Location: Pennsylvania, US
Posts: 30
Thanks: 2
Thanked 12 Times in 10 Posts
Default Re: Remote form submission via PHP Curl

khtm,

It just occurred to me that I should have recommended Firebug for Firefox. I've you've not used this plugin, it'll greatly assist you in picking apart your form calls on your target websites.

You want to use the "net" panel when you send a form submission. It'll list out every URL hit in the submission process and show all the headers and responses from every URL. Insanely useful when there's more than a single URL involved in a submission process. [Those often go overlooked if they load quickly during your submission process and you're not running Firebug or some other logger on port 80 traffic.]

Hit the Thanks in the lower right if you found this post useful. Thanks! ;-)
Kiva.org - Help aspiring entrepreneurs by giving low sum loans.
maxleadford is offline   Reply With Quote
Old 07-18-2009, 09:12 PM   #7
SocialAdr.com
War Room Member
 
khtm's Avatar
 
Join Date: Apr 2009
Location: San Diego
Posts: 318
Thanks: 56
Thanked 29 Times in 27 Posts
Social Networking View Member's Myspace Profile  View Member's FaceBook Profile  View Member's Twitter Profile  View Member's YouTube Profile
Default Re: Remote form submission via PHP Curl

Thanks Max! I wasn't aware of that plugin - I'm gonna check it out now.

khtm is offline   Reply With Quote
Reply

  WarriorForum - Internet Marketing Forums > Warrior Support Forums > Programming Talk

Tags
cron, curl, form, form submission, logging, php, remote, simulating, submitting, website

Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off



All times are GMT -6. The time now is 09:11 AM.