![]() | | ||||||||
| | #1 |
| SocialAdr.com War Room Member Join Date: Apr 2009 Location: San Diego
Posts: 318
Thanks: 56
Thanked 29 Times in 27 Posts
|
Hey nerd-buddies! I have a problem - I need to write a bunch of interfaces to social networking websites that don't have APIs available. I've been using PHP Curl to connect and in some cases have been successful at logging in but then can't seem to submit a form on another page. The code needs to be ran automatically via a cron job (so...behind the scenes). Does anyone have some mad skills at simulating remote form submissions via PHP? Or should I be using javascript or another language? Thanks! |
| | |
| | |
| | #2 |
| Active Warrior Join Date: Jul 2009 Location: Pennsylvania, US
Posts: 30
Thanks: 2
Thanked 12 Times in 10 Posts
|
Kane, It's going to depend on the site you're using. Something like Facebook runs on Javascript 99% of the time, so straight form submissions aren't going to work for you there. If the site is 100% non-RPC based, then you might just need to capture the login cookie(s) that gets set and make sure you reply with that header when posting new form submissions. Code: $user_agent = 'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1) Gecko/20061010 Firefox/2.0';
$headers = array (
'HTTP_ACCEPT' => 'application/x-shockwave-flash,text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5',
'HTTP_ACCEPT_LANGUAGE' => 'en-us,en;q=0.5',
'HTTP_ACCEPT_CHARSET' => 'ISO-8859-1,utf-8;q=0.7,*;q=0.7',
'HTTP_KEEP_ALIVE' => '300',
'HTTP_CONNECTION' => 'keep-alive'
);
function read_header($ch, $string)
{
global $location; #keep track of location/redirects
global $cookiearr; #store cookies here
global $ch;
# ^overrides the function param $ch this is okay because
# we need to update the global $ch with new cookies
$length = strlen($string);
if(!strncmp($string, "Location:", 9)) #keep track of last redirect
{ $location = trim(substr($string, 9, -1)); }
if(!strncmp($string, "Set-Cookie:", 11)) #get the cookie
{
$cookiestr = trim(substr($string, 11, -1));
$cookie = explode(';', $cookiestr);
$cookie = explode('=', $cookie[0]);
$cookiename = trim(array_shift($cookie));
$cookiearr[$cookiename] = trim(implode('=', $cookie));
}
$cookie = "";
if(trim($string) == "") #execute only at end of header
{
foreach ($cookiearr as $key=>$value) { $cookie .= "$key=$value; "; }
curl_setopt($ch, CURLOPT_COOKIE, $cookie);
}
return $length;
}
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADERFUNCTION, 'read_header'); # take their cookie and love it!
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 5);
curl_setopt($ch, CURLOPT_AUTOREFERER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_USERAGENT, $user_agent);
curl_setopt($ch, CURLOPT_REFERER, $url_base);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_URL, $url_login);
$string = curl_exec($ch); Hope this helps! |
|
Hit the Thanks in the lower right if you found this post useful. Thanks! ;-) Kiva.org - Help aspiring entrepreneurs by giving low sum loans. | |
| | |
| | #3 |
| SocialAdr.com War Room Member Join Date: Apr 2009 Location: San Diego
Posts: 318
Thanks: 56
Thanked 29 Times in 27 Posts
|
Hey Max, thanks for the help! ![]() I think I have a lot to learn about cookies and HTTP headers...this stuff is still confusing for me. ![]() The first site I'm trying this on is :: BuddyMarks :: I wanted to first initiate a login and then insert a new bookmark. I simply viewed the form details and tried to use Curl POSTFIELDS to copy them all. Here's my original code: PHP Code: I'm not really sure how to proceed with your example. Does that code just grab the header info and save it as a cookie? Then I'd still need to initiate a LOGIN using the same header info? I tried putting the "meat & potatoes" of my code after your code and it seemed to get me further. Instead of the ADD_BOOKMARK.php page redirecting me back to itself after the curl_exec(), it actually says "Public Bookmark not saved!" this time. But I'm not sure how to debug that particular message. |
| | |
| | |
| | #4 |
| SocialAdr.com War Room Member Join Date: Apr 2009 Location: San Diego
Posts: 318
Thanks: 56
Thanked 29 Times in 27 Posts
|
Wait a sec! It actually did work! Even though it says "Public Bookmark not saved" I checked and it was actually there. Sweeeeet You're a genius, Max. |
| | |
| | |
| | #5 |
| Active Warrior Join Date: Jul 2009 Location: Pennsylvania, US
Posts: 30
Thanks: 2
Thanked 12 Times in 10 Posts
|
khtm, Glad to hear it helped. I wrote this as part of a data scraper three years ago to grab some data from a public records website for some real estate stuff. I'm actually a bit foggy on the specifics, but I'm pretty sure that as long as you stay in the same cURL session (in my example $ch), your cookies should stay in place. Good luck on your endeavor! |
|
Hit the Thanks in the lower right if you found this post useful. Thanks! ;-) Kiva.org - Help aspiring entrepreneurs by giving low sum loans. | |
| | |
| | #6 |
| Active Warrior Join Date: Jul 2009 Location: Pennsylvania, US
Posts: 30
Thanks: 2
Thanked 12 Times in 10 Posts
|
khtm, It just occurred to me that I should have recommended Firebug for Firefox. I've you've not used this plugin, it'll greatly assist you in picking apart your form calls on your target websites. You want to use the "net" panel when you send a form submission. It'll list out every URL hit in the submission process and show all the headers and responses from every URL. Insanely useful when there's more than a single URL involved in a submission process. [Those often go overlooked if they load quickly during your submission process and you're not running Firebug or some other logger on port 80 traffic.] |
|
Hit the Thanks in the lower right if you found this post useful. Thanks! ;-) Kiva.org - Help aspiring entrepreneurs by giving low sum loans. | |
| | |
| | #7 |
| SocialAdr.com War Room Member Join Date: Apr 2009 Location: San Diego
Posts: 318
Thanks: 56
Thanked 29 Times in 27 Posts
|
Thanks Max! I wasn't aware of that plugin - I'm gonna check it out now.
|
| | |
| | |
![]() |
|
| Tags |
| cron, curl, form, form submission, logging, php, remote, simulating, submitting, website |
| Thread Tools | |
| |
![]() |