When I add an parameter to the CURL init function:
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
It does print the site content I'm sending a request to, the problem is that I want to manually parse the returned content ($response = curl_exec($ch);) but the problem is that, the site is displaying the page content and I want to keep having the site content on my $response variable so I could parse it, but at the same, stop it from displaying it.
Is that feasible?
The curl code:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $action);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6");
curl_setopt($ch, CURLOPT_TIMEOUT, 60);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_COOKIEJAR, '');
curl_setopt($ch, CURLOPT_COOKIEFILE, '');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $fields);
$response = curl_exec($ch);
curl_close($ch);
You should set the CURLOPT_RETURNTRANSFER flag to true. From PHP.NET manual
CURLOPT_RETURNTRANSFER TRUE
To return the transfer as a string of the return value of curl_exec() instead of outputting it out directly.
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
Related
I am trying to do API calls to Airbnb, but you must be logged in to make these calls, and with no public API, I am left to try and come up with a solution.
I am first trying to login via CURL, I am trying to use this piece of code but I'm not having much luck.
$url = "https://www.airbnb.co.uk/login";
$postinfo = "email=".$username."&password=".$password;
$cookie_file_path = $path."/cookie.txt";
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_NOBODY, false);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file_path);
//for certain features, set the cookie the site has - this is optional
curl_setopt($ch, CURLOPT_COOKIE, "cookiename=0");
curl_setopt($ch, CURLOPT_USERAGENT,
"Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_REFERER, $_SERVER['REQUEST_URI']);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postinfo);
curl_exec($ch);
curl_setopt($ch, CURLOPT_URL, $url);
$html_data = curl_exec($ch);
curl_close($ch);
echo $html_data;
The response I get is:
{
success: false,
redirect: ""
}
Where am I going wrong ( I have declared the vars, username, and password but have obviously left that out).
You can't login to Airbnb by curl because Airbnb uses google captcha which can't be resolved. This is why it returns success: false.
If you want to do something with their application you should try to go here and apply as partner in order to use their API.
I've been trying to log into a site (www.namemc.com/login) with a curl script for about 5 hours now and I think I'm going crazy trying to do so.
I was wondering if you would be able to check what's wrong with my code. I think the site may be blocking it somehow, but I'm not sure.
Code:
<?php
$username = 'example#gmail.com';
$password = 'example';
$accountUrl = 'https://namemc.com/login';
$postdata = "email=".$username."&password=".$password;
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_NOBODY, 0);
curl_setopt($ch, CURLOPT_URL, $accountUrl);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_USERAGENT,
"Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.7.12) Gecko/20050915 Firefox/1.0.7");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postdata);
$result = curl_exec($ch);
echo $result;
?>
curl is a tool to transfer data from or to a server. The command is designed to work without user interaction.
You should be using either a cookie or a Header to maintain a session in your code like:
cookie="cookie.txt";
curl_setopt ($ch, CURLOPT_COOKIEJAR, $cookie);
Also have you tried arguements in double quotes instead of single .
and ensure send all form data which is needed in $postdata.
<?php
$url='https://source.amazon.com/forcelogin?returnToURL=https://www.amazon.com/';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_USERAGENT,
"Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7");
curl_setopt($ch, CURLOPT_COOKIEFILE,'cookies.txt');
curl_setopt($ch, CURLOPT_COOKIEJAR, "cookies.txt");
curl_setopt($ch, CURLOPT_COOKIEFILE, "cookies.txt");
$data = curl_exec($ch);
curl_close($ch);
?>
I don't know why it doesn't load the website.
If I put regular sites like google.com, amazon.com or anything else is working.Anyone can help me out?
You need to add CURLOPT_SSL_VERIFYPEER and CURLOPT_SSL_VERIFYHOST options to access HTTPS URL.
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
I would like to get the html of the page. After some google-ing, i found following code, which isn't working.
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERPWD, "$username:$passwd");
curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_BASIC);
$output = curl_exec($ch);
$info = curl_getinfo($ch);
curl_close($ch);
echo $output;
edit: something like this then:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, '');
curl_setopt($ch, CURLOPT_POSTFIELDS,'username='.urlencode($username).'&passwd='.urlencode($passwd));
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_COOKIEJAR, "/tmp/my_cookies.txt");
curl_setopt($ch, CURLOPT_COOKIEFILE, "/tmp/my_cookies.txt");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt( $ch, CURLOPT_SSL_VERIFYPEER, false );
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3");
curl_setopt($ch, CURLOPT_REFERER, "");
$output = curl_exec($ch);
echo $output
This just shows the target page, it doesn't log in, nor does it return the html code.
edit 2: now tried with $data = array('username' => $username, 'passwd' => $passwd); and curl_setopt($ch, CURLOPT_POSTFIELDS,$data);
This shows the same as above + an error: Request Entity Too Large
That page does not use HTTP Basic Authentication.
You need to make an HTTP request to match what submitting the form would sent, then you need to process the response (which will probably involve storing cookies).
I am trying to automate the login process for the site: winpossible.com. The site is running on .NET and expects the VIEWSTATE variable to be appropriately set and that is what is most probably tripping up the login function?
<?php
$username=urlencode('<something>');
$password="<something>";
$cookie="<cookie_file>";
$viewstate="...";
$postdata="__EVENTARGUMENT=&__EVENTTARGET=&__VIEWSTATE=$viewstate&_ctl0%3AContentPlaceHolder1%3APassword=$password&_ctl0%3AContentPlaceHolder1%3AUserName=$username&_ctl0%3AContentPlaceHolder1%3AbtnLogin.x=0&_ctl0%3AContentPlaceHolder1%3AbtnLogin.y=0";
$ch = curl_init();
$headers = 'Connection: Keep-Alive';
// First go to the home-page
curl_setopt($ch, CURLOPT_URL,"http://www.winpossible.com/");
curl_setopt($ch, CURLOPT_COOKIESESSION, true);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie);
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6");
curl_setopt($ch, CURLOPT_HEADER, TRUE);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
$result0 = curl_exec ($ch);
echo "Hi there - results of hitting the home-page\n";
echo $result0;
// Now try the login
curl_setopt($ch, CURLOPT_URL,"http://www.winpossible.com/LoginCheck.aspx");
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, FALSE);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.6) Gecko/20070725 Firefox/2.0.0.6");
curl_setopt($ch, CURLOPT_TIMEOUT, 60);
curl_setopt($ch, CURLOPT_HEADER, TRUE);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_COOKIESESSION, true);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie);
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_ANY);
curl_setopt($ch, CURLOPT_REFERER, 'http://www.winpossible.com/Login.aspx');
curl_setopt($ch, CURLOPT_POSTFIELDS, $postdata);
curl_setopt($ch, CURLOPT_POST, 1);
$result = curl_exec ($ch);
echo $result;
// Now access my-account
curl_setopt($ch, CURLOPT_URL, "http://www.winpossible.com/");
$result2 = curl_exec ($ch);
echo $result2;
curl_close($ch);
// unlink($cookie);
exit;
?>
i've done similar stuff in php and everything seems fine, so try these:
use curl_close after every curl_exec
check if post_datat is correctly
encoded remove ssl options(I've never used them, so I'm not sure)
are you sure there isn't any token or dynamic stuff in login?
looking at source I've seen this
<input type="hidden" name="__VIEWSTATE" id="__VIEWSTATE" value="LGwSEpu70f..." /></form>
if that's the problem you'll have to isolate it with a regex and send it as post parameter