PHP cURL - POST is being sent to an empty page? - php

I'm trying to automate the usage of this site, where users can use temporary E-Mail addresses which can also be specified by them manually. It uses a very simple anti-spam protection by having this <input name="csrf" type="hidden"> input inside the form with a randomized set of characters, which then needs to be included in the POST request. So, if the CSRF is bj152nvua2ob, and I want my new address to be "john#l0real.net", my POST needs to be:
csrf=bj152nvua2ob&mail=john&domain=#l0real.net
Problem is, I can't do this with cURL and PHP. Here's the top of my code:
$ch = curl_init ();
curl_setopt ($ch, CURLOPT_URL, "temp-mail.org/en/option/change/");
// Without CURLOPT_FOLLOWLOCATION, the page is not going to load.
curl_setopt ($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt ($ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt ($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt ($ch, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)");
$html = curl_exec ($ch);
Then, after I've got the response, I start generating a new mail address and start getting the CSRF, therefore I've written two simple functions called "generate_mail" and "get_csrf". I've tested both of them, and they seem to work without any issues.
$csrf = get_csrf ($html);
$mail = generate_mail ();
$post = "csrf=$csrf&mail=$mail&domain=#l0real.net";
curl_setopt ($ch, CURLOPT_POST, true);
curl_setopt ($ch, CURLOPT_POSTFIELDS, $post);
file_put_contents ("final_response.html", curl_exec ($ch));
curl_close ($ch);
After creating the file final_response.html on my PC and then viewing it, I've noticed that the results aren't what I've expected. Some tries later, I've decided to debug my connection using Fiddler, and then I've noticed something interesting. This is what I get when I use this site with my browser:
and this with cURL:
.
Notice how there is 1 element (excluding the tunnels) against... 4! First element is an empty response, but the second one contains the HTML page. Last two are the same: third is an empty response (and that's where my POST was sent), and forth contains the HTML page with different CSRF and incorrect mail address (the one that was generated by the site itself, not by me). From what I can tell, each time I use curl_exec, it first loads an empty page, but then loads the correct one. All of my requests are being sent to the empty one, thus being ignored later on. Is this a security measure, or I didn't configure cURL correctly? I've tried to provide as much information as I could, showcasing each, and every step of mine, hoping that this problem can be fixed.

Solved by adding these lines of code:
curl_setopt ($ch, CURLOPT_POSTREDIR, 3);
curl_setopt ($ch, CURLOPT_COOKIEJAR, $path_to_cookies);
curl_setopt ($ch, CURLOPT_COOKIEFILE, $path_to_cookies);
Now, it works the way intended!

Related

Logging in to a webpage through CURL, differently

There's a webpage that I need to log in to. I used CURL with post to login, but it's not enough. When you log in from the website the post also includes a string that is always changing. Is threre a way to get over that?
I use this:
$ch = curl_init();
curl_setopt ($ch, CURLOPT_URL, $url);
curl_setopt ($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; he-IL; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6");
curl_setopt ($ch, CURLOPT_REFERER, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($ch, CURLOPT_POST, TRUE);
$post = "username=$username&password=$password";
curl_setopt($ch, CURLOPT_POSTFIELDS, $post);
$result = curl_exec($ch);
$info = curl_getinfo($ch);
curl_close($ch);
It's like I need the code to actually go to the webpage and fill the form regularly.
I looked everywhere but all I could find was using post data.
Thanks!
To pass that you need to visit the page with the form, grab the field and then use it in POST request when you submit the form.
I suggest you visit the form page not only for that, but also for the following reasons (some of which can be used to figure people using automatic requests):
You recieve cookies
You don't fake referrer, you actually visited the page
You might want to check form fields to see if there's any new ones added since you wrote the script. That could be the case if form setup changes and you might want to adapt to that, if you don't then your script might stop working one day

php cURL and Instagram Photos

So someone Tweets a link to a photo on Instagram : http://instagr.am/p/QSVkR8LS3H/
This redirects from http://t.co/bOJ4EX2j to http://instagr.am/p/QSVkR8LS3H/ to the actual Instagram page http://instagram.com/p/QSVkR8LS3H/ where the photo resides.
Cool. All that is good and well. Now I want a cURL to follow the tweeted link and download that final page, that contains the < img > of the photo. Script looks basically like this :
$target = 'http://t.co/bOJ4EX2j';
$ch = curl_init();
curl_setopt ($ch, CURLOPT_HTTPGET, TRUE);
curl_setopt ($ch, CURLOPT_POST, FALSE);
curl_setopt ($ch, CURLOPT_COOKIEJAR, COOKIE_FILE); // Defined Constant
curl_setopt ($ch, CURLOPT_COOKIEFILE, COOKIE_FILE);
curl_setopt ($ch, CURLOPT_TIMEOUT, CURL_TIMEOUT); // Defined Constant
curl_setopt ($ch, CURLOPT_USERAGENT, WEBBOT_NAME); // Defined Constant
curl_setopt ($ch, CURLOPT_URL, $target); // Target site
curl_setopt ($ch, CURLOPT_REFERER, ''); // Referer value
curl_setopt ($ch, CURLOPT_VERBOSE, FALSE); // Minimize logs
curl_setopt ($ch, CURLOPT_SSL_VERIFYPEER, FALSE); // No certificate
curl_setopt ($ch, CURLOPT_FOLLOWLOCATION, TRUE); // Follow redirects
curl_setopt ($ch, CURLOPT_MAXREDIRS, 4); // Limit redirections to four
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, TRUE); // Return in string
# Create return array
$return_array['FILE'] = curl_exec($ch);
$return_array['STATUS'] = curl_getinfo($ch);
$return_array['ERROR'] = curl_error($ch);
# Close PHP/CURL handle
curl_close($ch);
return $return_array;
Now this script has many more components but that's the just of the cURL part. Now, it does manage to spit back that it landed on the correct final page http://instagram.com/p/QSVkR8LS3H/) where the image is - but this is what the $return_array['FILE'] spits out :
500 Server Error
An internal server error occurred.
Even when if you navigate to the page in your browser, with cookie's off, and not signed into Instagram (if you were) the page loads completely!
What the hell am I missing that's not allowing this to cURL script to download the Instagram page?! It works on just about every other page I try it! Just not Instagram.com?!
Please someone help me crack this nut of a issue - I'd greatly appreciate any help or insight anyone might have.
If you're running this code on a server and not your local machine, it's possible that there's some misconfigured proxy between the server and Instagram.com. Check the code from your local machine, making sure it also works in a browser on the same machine.
If you get this working and discover, as Justin Wood has said, that you have an HTML page and not the image you wanted, I can help you with some PHP to get the image URL (for which you'd then have to run another cURL request).
As said by many of us, that code seems to work perfectly fine. All i can suggest is that you make sure error reporting is on and then check your PHP error logs in your base directory.
You might also want to check that the cURL module is enabled in PHP.ini.
Not sure what else could be wrong.

PHP/Curl/Wordpress Posting data without refreshing page, curl not working

I am writing code for a donation page and want to submit CC information without refreshing the page, and display the results using an overlay. I have the overlay working correctly using jQuery, and using a php if statement, I can post back to the page I am on and get the variables correctly the the curl block for the CC transaction. However I never get any results. If I allow the page to POST/refresh it works fine, but it seems Wordpress will not allow me to run the curl from a page that is not directly run in the enviroment.
Any suggestions?
jQuery.post('sameurl', jQuery("#donateform").serialize(), function(data) {
jQuery('#overlay_msg').html(data);
and
$ch = curl_init ();
curl_setopt ($ch, CURLOPT_URL,$hoststring);
curl_setopt ($ch, CURLOPT_POST, 1);
curl_setopt ($ch, CURLOPT_POSTFIELDS, $xml); # the string we built above
curl_setopt ($ch, CURLOPT_SSLCERT, $cert);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt ($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt ($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt ($ch, CURLOPT_VERBOSE, 1);
$result = curl_exec ($ch);
I hard coded the $xml string as well trying to get it to work. I'm new to php/curl/ajax but know a bit about coding in general.
Thanks.
From what I can see your code is not calling any wordpress functions, so there is no dependency there. It sounds like the cURL call is not completing successfully, you will need to debug this. Make sure php error reporting is on and set to E_All in your php.ini, also set the curl option FAILONERROR to true and be sure to catch any errors using the curl error functions.
Failing this you may need to analyse the HTTP headers to see whats being received from the server. You may simply be getting a redirect try setting CURLOPT_FOLLOWLOCATION to true.
Well, it turns out it was a directory issue. Wordpress sets you to the website root directory, while regular html has your directory relative to the file location. So I could not get my cert.

Bypassing Captcha with curl in PHP

I am trying to automate the login progress on a captcha protected page. I am using Death By Captcha to translate the image into text and it seems to be working well. I am using curl to load the login page, retrieve the captcha image url, send it to DBC, get the text back and submit a POST request to the login page with the captcha text.
The problem that I'm having is that the captcha image changes when I submit the post request. Since I do not get the same behavior when reloading/or wrongly submitting the form through a browser (I get the same image over and over again), I am assuming that the problem has to do with the cookies or something else that I'm missing that relates to the session.
This is the code that I use to retrieve the data and submit the form:
$ch = curl_init();
// Not sure that I need it, just make sure that the session doesn't change...
curl_setopt($ch, CURLOPT_COOKIESESSION, false);
curl_setopt($ch, CURLOPT_URL, $loginUrl);
// It seems that PHPSESSID cookie parameter might be the parameter that keep the image the same, but it didn't work. I even read it dynamically from the cookie file but it still didn't work
//curl_setopt($ch, CURLOPT_COOKIE, "PHPSESSID=2bp3nhkp3bgftfrr1rjekg03o2");
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookieName);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookieName);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_REFERER, $loginUrl);
$result = curl_exec($ch);
// Resolve the captcha and append it to the post parameters
$captchaText = $this->resolveCaptcha($result);
$postData .= '&LoginForm%5BverifyCode%5D='.$captchaText;
// Resubmit the form with the updated form data
curl_setopt($ch, CURLOPT_REFERER, $loginUrl);
curl_setopt($ch, CURLOPT_URL, $loginUrl);
curl_setopt ($ch, CURLOPT_POST, 1); //FIXED
curl_setopt ($ch, CURLOPT_POSTFIELDS, $postData);
$result = curl_exec($ch);
When I print the end result, I can see that the captcha text was submitted successfully but that the image itself has changed...
I am also attaching a screenshot of the request params as captured with Tamper in a standard Firefox session (so someone might spot if I'm missing something).
The PHP/curl submit code is fully working for non-captcha based sites so the POST parameters submission seems to be working.
It could be that I'm missing something very basic here, any help will be much appreciated.
I also took a look at these posts but couldn't find the answer that I'm looking for.
How CURL Login with Captcha and Session
How to retrieve captcha and save session with PHP cURL?
https://stackoverflow.com/questions/8633282/curl-to-download-a-captcha-and-submit-it
you're using
curl_setopt ($ch, CURLOPT_POST, 0);
in second curl_exec. shoudn't it be
curl_setopt ($ch, CURLOPT_POST, 1);
?

Download file attached to header with Curl and Php

I'm connecting to a website daily to collect some statistics, the website runs .net to make things extra difficult. What i would like to do is to mechanize this process.
I go to http://www.thesite.com:8080/statistics/Login.aspx?ReturnUrl=%2Fstatistics%2Fdataexport.ashx%3FReport%3D99, (the return url is /statistics/dataexport.ashx?Report=99 decoded).
The Login.aspx displays a form, in which I enter my user/pass and when the form is submitted the dataexport.ashx starts to download the file directly. The filename delivered is always statistics.csv.
I have experimented with this a few days now. Are there any resources or does anyone have some kind of hint of what I should try next?
Here is some of my code.
<?php
// INIT CURL
$ch = curl_init();
// SET URL FOR THE POST FORM LOGIN
curl_setopt($ch, CURLOPT_URL, $url);
// ENABLE HTTP POST
curl_setopt ($ch, CURLOPT_POST, 1);
// SET POST PARAMETERS : FORM VALUES FOR EACH FIELD
$viewstate = urlencode('/wEPDwUKM123123daE2MGQYAQUeX19Db250cm9sc1JlcXVpcmVQb3N0QmFja0tleV9fFgEFGG1fTG9naW4kTG9naW5JbWFnZUJ1dHASdasdRvbij2MVoasdasdYibEXm/eSdad4hS');
$eventval = urlencode('/wEWBAKMasd123LKJJKfdAvD8gd8KAoCt878OED00uk0pShTQHkXmZszVXtBJtVc=');
curl_setopt ($ch, CURLOPT_POSTFIELDS, "__VIEWSTATE=$viewstate"."__EVENTVALIDATION=$eventval&UserName=myuser&Password=mypassword");
// IMITATE CLASSIC BROWSER'S BEHAVIOUR : HANDLE COOKIES
curl_setopt ($ch, CURLOPT_COOKIEJAR, 'cookie.txt');
# Setting CURLOPT_RETURNTRANSFER variable to 1 will force cURL
# not to print out the results of its query.
# Instead, it will return the results as a string return value
# from curl_exec() instead of the usual true/false.
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
// FOLLOW REDIRECTS AND READ THE HEADER
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_HEADER, true);
// EXECUTE REQUEST (FORM LOGIN)
$store = curl_exec ($ch);
// print the result
print_r($store);
// CLOSE CURL
curl_close ($ch);
?>
Thanks
Trikks
You also need to use CURLOPT_COOKIEFILE to send the cookies along with the next request. Another thing if i remember correctly is that ASPX would set unique value each time for variables like __VIEWSTATE. See if these 2 pointers help.

Categories