I'm trying by PHP and cURL to log in a website, but this website use a CSRF token. My PHP code is catching the token (checked with a echo) but I couldn't log in after that (CSRF token error).
Do you have any idea of how I could keep the same cURL session in order to have the same CSRF token (otherwise the server will generate a new one).
I have added comment to my code in order to make it more understandable
Thank you in advance for all your help
Here my code :
<?php
$url = "https://www.boursedirect.fr/fr/login";
// obtain the CSRF Token
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:100.0) Gecko/20100101 Firefox/100.0");
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_COOKIESESSION, true);
curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookie'); // to handle cookies during login
curl_setopt($ch, CURLOPT_COOKIEFILE, 'tmp'); // idem
$headers = [
'Keep-Alive: timeout=5, max=100',
'Connection: keep-alive'
];
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
$res = curl_exec($ch);
$dom = new DomDocument();
# $dom->loadHTML($res);
$elem = $dom->getElementById("bd_auth_login_type__token");
$token = $elem->getAttribute("value");
/* extract the token value hidden in <input type="hidden" id="bd_auth_login_type__token" name="bd_auth_login_type[_token]" class="form-control" value="yDVyvTXUhIJjnAj9mTfBO3OKgRpI0zLCUZY2BM_O1E8" /> __ FULLY WORKING */
// second part, trying to log in (same URL)
$username = "X";
$password = "X";
curl_setopt($ch, CURLOPT_POST, true); // switching to POST
curl_setopt($ch, CURLOPT_POSTFIELDS, 'bd_auth_login_type[login]='.$username.'&bd_auth_login_type[password]='.$password.'&bd_auto_login_type[submit]=&bd_auth_login_type[token]='.$token);
$res = curl_exec($ch);
echo $res; // result is a CSRF token error, so unable to log in
curl_close($ch);
?>
Working script thanks to #miken32 help !
<?php
$url = "/";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:100.0) Gecko/20100101 Firefox/100.0");
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
$headers = [
'Keep-Alive: timeout=5, max=100',
'Connection: keep-alive'
];
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
$res = curl_exec($ch);
$dom = new DomDocument();
# $dom->loadHTML($res);
$elem = $dom->getElementById("bd_auth_login_type__token");
$token = $elem->getAttribute("value");
$data = array(
"bd_auth_login_type[login]" => "X",
"bd_auth_login_type[password]" => "X",
"bd_auto_login_type[submit]" => "",
"bd_auth_login_type[_token]" => $token
);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($data));
$res = curl_exec($ch);
curl_close($ch);
?>
Related
I am Trying to Make A PHP Curl Request to a site that apparently seems to be protected agains Cross-Site Forgery
and have done all my best to edit the headers, parameters and cookies to resemble that of a browser but I still keep getting ERROR 400 (Bad Request) Please I would like to know how to make this work with php curl the code below works on site with and without the crsf protection but doesn't work in the one i am currently trying to access
$url="https://candidate.scholastica.ng/candidate/login";
$cookie_file_path = "cookie.txt";
// I first load the page to save the cookie and the request token
// Below are the required curl settings that works in other sites
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_NOBODY, false);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file_path);
//set the cookie the site has for certain features, this is optional
// curl_setopt($ch, CURLOPT_COOKIE, "cookiename=0");
curl_setopt($ch, CURLOPT_USERAGENT,"Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:66.0) Gecko/20100101 Firefox/66.0");
curl_setopt($ch, CURLOPT_ENCODING,'gzip, deflate');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
// curl_setopt($ch, CURLOPT_REFERER, "http://www.google.com/bot.html");
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
$headers = array("User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.0.8)
Gecko/20061025 Firefox/1.5.0.8","origin:http://www.google.com/bot.html","accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9", "accept-language:en-US,en;q=0.5","accept-encoding:gzip, deflate","connection:keep-alive","upgrade-insecure-requests:1","keep-alive:300","accept-charset:ISO-8859-1,utf-8;q=0.7,*;q=0.7");
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "GET");
curl_setopt($ch, CURLOPT_POST, 1);
// curl_setopt($ch, CURLOPT_POSTFIELDS, "");
$details = curl_exec($ch);// I load the Page containing the form
// I remove all newline and return statements from the form to make my regex simple for me
$pattern = '/\r\n*/m';
$replace = '';
$details = preg_replace( $pattern, $replace, $details);
// Regular Expression to grab request token from form page
$pattern = '/<input.+?name="__RequestVerificationToken".+?value="(.*?)".+?>/m';
preg_match($pattern, $details, $matches);
$token = $matches[1];
// I set up urlencoded post form values and get content length
$field1 = urlencode("__RequestVerificationToken") ."=". urlencode($token);
$field2 = urlencode("Username") ."=". urlencode("example#gmail.com");
$field3 = urlencode("Pwd") ."=". urlencode("examplepassword");
$fields = "{$field1}&{$field2}&{$field3}";
$content_length = strlen($fields);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_NOBODY, false);
curl_setopt($ch, CURLINFO_HEADER_OUT, true);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, "POST");
curl_setopt($ch, CURLOPT_POST, 1);
// $headers = array("Host:candidate.scholastica.ng",
// "User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:71.0) Gecko/20100101 Firefox/71.0",
//I Modify header to add content length
$headers = array("Host:candidate.scholastica.ng","User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:71.0) Gecko/20100101 Firefox/71.0","Accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8","Accept-Language:en-US,en;q=0.5","Accept-Encoding:gzip, deflate, br","Connection:keep-alive","Upgrade-Insecure-Requests:1","Origin:https://candidate.scholastica.ng","Referer:https://candidate.scholastica.ng/candidate/login","content-type:application/x-www-form-urlencoded","TE:Trailers","content-length:{$content_length}");
// I assign the headers and set the post fields
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $fields);
I make request with existing curl thereby sending back the set cookie
$details = curl_exec($ch);
echo $details;
curl_close($ch);
This still returns a Bad Request
Your Response will be Greatly appreciated
i am having a problem with PHP file_get_contents.i am trying to fetch inforamtion following url but is getting captcha page.
$link = 'http://www.wayfair.com/a/product_review_page/get_update_reviews_json?_format=json&product_sku=KUS1523&page_number=5&sort_order=relevance&filter_rating=&filter_tag=&item_per_page=5';
$Page_information = file_get_contents($link);
print_r($Page_information);
Also i am trying to get page information using php curl but same captcha page is display.
$cookie='cookie.txt';
if(!file_exists($cookie)){
$fh = fopen($cookie, "w");
fwrite($fh, "");
fclose($fh);
}
$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_URL, "http://www.wayfair.com/a/product_review_page/get_update_reviews_json?_format=json&product_sku=KUS1523&page_number=5&sort_order=relevance&filter_rating=&filter_tag=&item_per_page=5");
curl_setopt($ch, CURLOPT_BINARYTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_COOKIE,1);
curl_setopt($ch, CURLOPT_COOKIEJAR,$cookie);
curl_setopt($ch, CURLOPT_COOKIEFILE,$cookie);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2);
$result11 = curl_exec($ch);
print_r($result11);
If you analyze the headers from a browser where cookies and javascript are disabled you should see the bare minimum sent - some, perhaps all might be required and are set with the context argument.
/* set the options for the stream context */
$args=array(
'http'=>array(
'method' => "GET",
'header' => array(
'User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:44.0) Gecko/20100101 Firefox/44.0',
'Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8',
'Host: www.wayfair.com',
'Accept-Encoding: gzip, deflate'
)
)
);
/* create the context */
$context=stream_context_create( $args );
$link = 'http://www.wayfair.com/a/product_review_page/get_update_reviews_json?_format=json&product_sku=KUS1523&page_number=5&sort_order=relevance&filter_rating=&filter_tag=&item_per_page=5';
/* Get the response from remote url */
$res = file_get_contents( $link, FILE_TEXT, $context );
/* process the response */
print_r( $res );
$url = "http://www.wayfair.com/a/product_review_page/get_update_reviews_json?_format=json&product_sku=KUS1523&page_number=5&sort_order=relevance&filter_rating=&filter_tag=&item_per_page=5";
$cookie = getcwd().DIRECTORY_SEPARATOR.'cookie.txt';
$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_BINARYTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_COOKIE,1);
curl_setopt($ch, CURLOPT_COOKIEJAR,$cookie);
curl_setopt($ch, CURLOPT_COOKIEFILE,$cookie);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2);
//added
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.95 Safari/537.36");
$result11 = curl_exec($ch);
print_r($result11);
try this
I'm trying to login to Walmart using CURL, the code works perfectly. However whenever i activate the CURLOPT_POST I'm getting an error Access Denied.
This is the code that I tried:
<?php
$user_agent = "Mozilla/5.0 (X11; Linux i686; rv:24.0) Gecko/20140319 Firefox/24.0 Iceweasel/24.4.0";
$curl_crack = curl_init();
CURL_SETOPT($curl_crack,CURLOPT_URL,"https://www.walmart.com/account/login");
CURL_SETOPT($curl_crack,CURLOPT_USERAGENT,$user_agent);
//CURL_SETOPT($curl_crack,CURLOPT_PROXY,"183.78.169.60:37899");
//CURL_SETOPT($curl_crack,CURLOPT_PROXYTYPE,CURLPROXY_SOCKS5);
CURL_SETOPT($curl_crack,CURLOPT_POST,True);
CURL_SETOPT($curl_crack,CURLOPT_POSTFIELDS,"login-username=test&login-password=test");
CURL_SETOPT($curl_crack,CURLOPT_RETURNTRANSFER,True);
CURL_SETOPT($curl_crack,CURLOPT_FOLLOWLOCATION,True);
CURL_SETOPT($curl_crack,CURLOPT_COOKIEFILE,"cookie.txt"); //Put the full path of the cookie file if you want it to write on it
CURL_SETOPT($curl_crack,CURLOPT_COOKIEJAR,"cookie.txt"); //Put the full path of the cookie file if you want it to write on it
CURL_SETOPT($curl_crack,CURLOPT_TIMEOUT,30);
echo $exec = curl_exec($curl_crack);
?>
I don't know what I'm doing wrong, please help me.
Here is a working example.
You need to first request the home page to establish base cookies, and then post the credentials to the login page. If a 403 error is returned, the credentials are incorrect.
<?php
$base_url = 'https://www.walmart.com/';
$login_url = 'https://www.walmart.com/account/api/signin';
$user_agent = "Mozilla/5.0 (X11; Linux i686; rv:24.0) Gecko/20140319 Firefox/24.0 Iceweasel/24.4.0";
$username = 'user#example.com';
$password = 'password';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $base_url);
curl_setopt($ch, CURLOPT_USERAGENT,$user_agent);
curl_setopt($ch, CURLOPT_RETURNTRANSFER,true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION,True);
curl_setopt($ch, CURLOPT_AUTOREFERER, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_ENCODING, 'gzip, deflate');
curl_setopt($ch, CURLOPT_COOKIEFILE, '');
curl_setopt($ch, CURLOPT_TIMEOUT,30);
//curl_setopt($ch, CURLOPT_VERBOSE, 1);
//curl_setopt($ch, CURLOPT_STDERR, fopen('php://output', 'r'));
//curl_setopt($ch,CURLOPT_PROXY,"183.78.169.60:37899");
//curl_setopt($ch,CURLOPT_PROXYTYPE,CURLPROXY_SOCKS5);
$resp = curl_exec($ch);
$headers = array(
'X-Requested-With: XMLHttpRequest',
);
$post = array(
'username' => $username,
'password' => $password,
'login-captcha-value' => '',
'sensor-data' => '',
'clearPCID' => '1',
);
curl_setopt($ch, CURLOPT_URL, $login_url);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($post));
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
$exec = curl_exec($ch);
$info = curl_getinfo($ch);
if ($info['http_code'] != 200) {
echo "Login failed! HTTP code {$info['http_code']}<br>\n";
var_dump($exec);
exit;
}
echo "Login successful!<br>\n";
// you are now logged in, use $ch to request pages as the logged in user
$url = 'https://www.walmart.com/account';
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_POST, 0);
$account = curl_exec($ch);
I am trying to login to ets.org/toefl account using php curl. But I am unable to login to the website. I usually get an error saying server is busy, but it works when I login using a browser. I have attached my code. Can anyone see what is wrong?
<?php
include('simple_html_dom.php');
$login_url = 'https://toefl-registration.ets.org/TOEFLWeb/logon.do';
$username='****';
$password='***';
$ck = 'cookie.txt';
$agent = 'Mozilla/5.0 (Windows NT 6.1; rv:22.0) Gecko/20100101 Firefox/22.0';
// extra headers
$headers[] = "Connection: keep-alive";
//$headers[]= "Accept-Encoding: gzip, deflate";
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_COOKIEJAR, $ck);
curl_setopt ($ch, CURLOPT_COOKIEFILE, $ck);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
//curl_setopt($ch, CURLOPT_URL, 'https://toefl-registration.ets.org/TOEFLWebextISERLogonPrompt.do');
$output = curl_exec($ch);
//echo $output;
$html = new simple_html_dom();
$html = str_get_html($output);
$e = $html->find(".loginform");
$a = $e[0]->find('input');
$str = $a[0]->outertext;
preg_match("/value=\"(.*)\"/",$str,$match);
$h_attr = $match[1];
$fields['org.apache.struts.taglib.html.TOKEN'] = $h_attr;
$fields['currentLocale']= 'en_US';
$fields['username'] = $username;
$fields['password'] = $password;
$fields['x'] = 11;
$fields['y'] = 4;
//print_r($fields);
//echo "\r\n";
$POSTFIELDS = http_build_query($fields);
//echo $POSTFIELDS;
$headers[] = "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
$headers[] = "Accept-Language: en-US,en;q=0.5";
$headers[]="Referer: https://toefl-registration.ets.org/TOEFLWeb/extISERLogonPrompt.do";
curl_setopt($ch, CURLOPT_URL, $login_url);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $POSTFIELDS);
$result = curl_exec($ch);
print $result;
(Update from comments)
Post by browser:
org.apache.struts.taglib.html.TOKEN=c1b88957e9914492fe8cc20b33ef1cdd¤tLocale=en_US&username=name&password=pass&x=23&y=3
By me.
org.apache.struts.taglib.html.TOKEN=345a9f935b2db8a69f55c5b4d3372190¤tLocale=en_US&username=name&password=pass&x=11&y=4
Post generated by php curl verbose:
POST /TOEFLWeb/logon.do HTTP/1.1 User-Agent: Mozilla/5.0 (Windows NT
6.1; rv:22.0) Gecko/20100101 Firefox/22.0 Host: toefl-registration.ets.org Cookie: au=MTM3Mjc4ODQwMg%3d%3d; server=3;
JSESSIONID=23C39022E2641B8F5AC944295837315E Connection: keep-alive
Accept: / Accept:
text/html,application/xhtml+xml,application/xml;q=0.9,/;q=0.8
Accept-Language: en-US,en;q=0.5 Referer:
toefl-registration.ets.org/TOEFLWeb/extISERLogonPrompt.do
Content-Length: 134 Content-Type: application/x-www-form-urlencoded
Try comparing the HTTP headers sent by your CURL script to those headers sent by your browser (use chrome dev tools). Maybe the remote server is refusing you due to some missing header info.
Ensure cookie files have full permissions. From php.net:
When specifing CURLOPT_COOKIEFILE or CURLOPT_COOKIEJAR options, don't
forget to "chmod 777" that directory where cookie-file must be
created.
I got it working somehow... I added certificate verification to the code. Further i found that some delay needs to be present between the two functions get cookie and login. The working code is below
<?php
include('simple_html_dom.php');
$login_url = 'https://toefl-registration.ets.org/TOEFLWeb/logon.do';
$cookie_page = 'https://toefl-registration.ets.org/TOEFLWeb/extISERLogonPrompt.do';
$username='******';
$password='******';
//$ck = 'E:\Projects\Web Development\toefl_script\cookie.txt';
$ck = 'D:\Nikhil\Projects\Wamp\toeflscript\cookie.txt';
//$agent = 'Mozilla/5.0 (Windows NT 6.1; rv:22.0) Gecko/20100101 Firefox/22.0';
$agent = 'Mozilla/5.0 (Windows NT 6.1; rv:21.0) Gecko/20100101 Firefox/21.0';
$headers[] = "Connection: keep-alive";
$headers[] = "Accept: */*";
/* Begin Program Execution */
init_curl();
get_cookie();
sleep(30);
login();
function get_cookie()
{
global $ch, $ck, $h_attr, $headers, $cookie_page;
global $ck;
curl_setopt($ch, CURLOPT_URL, $cookie_page);
//curl_setopt($ch, CURLOPT_VERBOSE, true);
$output = curl_exec($ch);
//echo $output;
/*
$html = new simple_html_dom();
$html = str_get_html($output);
$e = $html->find(".loginform");
$a = $e[0]->find('input');
$str = $a[0]->outertext;
preg_match("/value=\"(.*)\"/",$str,$match);
$h_attr = $match[1];
*/
}
function init_curl()
{
global $ch, $ck, $h_attr, $headers, $agent;
global $ck;
ini_set('max_execution_time', 300);
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2);
curl_setopt($ch, CURLOPT_CAINFO, getcwd() . '/cacert.pem');
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_COOKIEJAR, $ck);
curl_setopt ($ch, CURLOPT_COOKIEFILE, $ck);
}
function login()
{
global $ch, $login_url, $password, $username, $ck, $h_attr, $headers;
//$fields['org.apache.struts.taglib.html.TOKEN'] = 'abc';//$h_attr;
$fields['currentLocale']= 'en_US';
$fields['username'] = $username;
$fields['password'] = $password;
$fields['x'] = 11;
$fields['y'] = 4;
$POSTFIELDS = http_build_query($fields);
//print_r($fields);
//echo $POSTFIELDS;
$headers[] = "Accept-Language: en-US,en;q=0.5";
$headers[]="Referer: https://toefl-registration.ets.org/TOEFLWeb/extISERLogonPrompt.do";
curl_setopt($ch, CURLOPT_URL, $login_url);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $POSTFIELDS);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
$result = curl_exec($ch);
print $result;
}
I'm trying to use CURL with PHP to login to Vimeo.com, Vimeo login.
To provide the data for CURL to use (cookie and field data), I'm using a browser extension to read the field data off of the webpage and get the cookies. I'm then passing that data through to my server and am trying to login using curl.
I'm quite positive that the browser extension part works correctly (gets the correct data) because I can verify what it's passing with what it should be passing, and it matches correctly.
Additionally, I've used this on other sites as well, and it has no problem logging in, but on vimeo the exec returns false.
Any thoughts?
function curlpage(){
$ch = curl_init();
$url = $this->input->post('url');
$data = $this->input->post('data');
$cookie = $this->input->post('cookie');
$method = $this->input->post('method');
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_POSTFIELDS,$data);
if(strtolower($method)=="put"){
curl_setopt($ch, CURLOPT_PUT, 1);
}
else{
curl_setopt($ch, CURLOPT_PUT, 0);
}
if(strtolower($method)=="get"){
curl_setopt($ch, CURLOPT_HTTPGET, 1);
}
else{
curl_setopt($ch, CURLOPT_HTTPGET, 0);
}
if(strtolower($method)=="post"){
curl_setopt($ch, CURLOPT_POST, 1);
}
else{
curl_setopt($ch, CURLOPT_POST, 0);
}
curl_setopt($ch, CURLOPT_FAILONERROR, true);
curl_setopt($ch, CURLOPT_UNRESTRICTED_AUTH, true);
curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_ANY);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_CAINFO, getcwd().'/certificates/BuiltinObjectToken-EquifaxSecureCA.crt');
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_COOKIE, $cookie);
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:17.0) Gecko/20100101 Firefox/17.0");
curl_setopt($ch, CURLOPT_REFERER, $url);
$error = curl_error($ch);
$url=curl_getinfo($ch, CURLINFO_EFFECTIVE_URL);
if(!preg_match('/^http(s)?:\/\//', $url)){
$url = 'http://' . $url;
}
$host = parse_url($url, PHP_URL_HOST);
$page = curl_exec($ch);
curl_close($ch);
return array('page'=>$page, 'url'=>$host, 'error'=>$error);
}
Here's a sample of the data I'm sending to the above function on my server (with a bogus email an password and altered cookies):
data=action%253Dlogin%2526service%253Dvimeo%2526email%253Dhou%2540fah.com%2526password%253Dudwt%2526token%253D6b2fc081bcdf02b1f58a390d6a3f8b83
cookie=__utma%3D18392654.1284111214.1456668252.1456678435.1456181183.3%3B__utmb%3D18302654.2.10.1454681883%3B__utmc%3D18232154%3B__utmz%3D17202654.1456675435.2.2.utmcsr%3Dgoogle%7Cutmccn%3D(organic)%7Cutmcmd%3Dorganic%7Cutmctr%3D(not%2520provided)%3B
method=POST
url=http%3A%2F%2Fvimeo.com%2Flog_in
$ret = customSendDataByCurl("https://vimeo.com/log_in");
preg_match("/xsrft: \'(.*)\',/i",$ret,$token);
$token = $token[1];
echo "$token <hr>";
$cookie = '(copy from your browser using tamper data)... xsrft='.$token;
$headers = array(
"Referer: https://vimeo.com/log_in",
"User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:26.0) Gecko/20100101 Firefox/26.0",
"Accept: application/json",
"Accept-Language: en-US,en;q=0.5",
"X-Requested-With: XMLHttpRequest",
"X-Request: JSON",
"Content-Type: application/x-www-form-urlencoded; charset=utf-8",
);
$ret = customSendDataByCurl("https://vimeo.com/log_in?action=warm", "POSTDATA=email=(email url encoded)&token=".$token, $headers, $cookie);
$fields = array(
"action" => "login",
"service" => "vimeo",
"email" => "(email)",
"password" => "(pass)",
"token" => $token,
);
$headers = array(
"Referer: https://vimeo.com/log_in",
"User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64; rv:26.0) Gecko/20100101 Firefox/26.0",
"Content-Type: application/x-www-form-urlencoded; charset=utf-8",
);
$ret = customSendDataByCurl("https://vimeo.com/log_in", http_build_query($fields), $headers, $cookie);
$ret = customSendDataByCurl("https://vimeo.com/stats/video/84142281/totals/export:csv", http_build_query($fields), $headers, $cookie);
var_export($ret);
function customSendDataByCurl($agateway, $apostfields=null, $headers=array(), $cookie="") {
if(is_array($apostfields)) $apostfields = http_build_query($apostfields);
$cookiesFile = 'cookies.txt';
$ch = curl_init($agateway);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt ($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt ($ch, CURLOPT_SSL_VERIFYHOST, FALSE);
curl_setopt( $ch, CURLOPT_COOKIESESSION, true );
if ($headers) curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt( $ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 6.3; WOW64; rv:26.0) Gecko/20100101 Firefox/26.0' );
curl_setopt($ch,CURLOPT_COOKIEJAR,$cookiesFile);
curl_setopt($ch,CURLOPT_COOKIEFILE,$cookiesFile);
curl_setopt($ch, CURLOPT_COOKIE, $cookie);
if(!empty($apostfields)) curl_setopt($ch, CURLOPT_POSTFIELDS, $apostfields);
$response = curl_exec($ch);
if(!$response) $response="CURL #".curl_errno($ch).": ".curl_error($ch);
return $response;
}
Vimeo, I've actually found to be a weird case for websites. They don't set all of their cookies up front, but rather set certain cookies needed to login when the form is submitted. So one of my problems was that I was not submitting all of the correct cookie information.
My second problem was that I was not correctly encoding my data as it was being sent.
All said and done, I now have got it to work!