I'm trying to post a request to http://iate.europa.eu/ using the following script:
<?php
$output_array = array();
$post = "method=search&saveStats=true&screenSize=1920x1080&query=bond&valid=Szukaj+&sourceLanguage=en&targetLanguages=pl&domain=&domain=12&typeOfSearch=s&request=";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, 'http://iate.europa.eu/SearchByQueryEdit.do');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: application/x-www-form-urlencoded'));
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
echo $server_output = curl_exec ($ch);
?>
The data I used:
When I run the script, I get "Access Denied. Your request has been denied for security reason."
Does the script trigger some security mechansims or does it simply make bad requests?
Thx in advance,
Lukasz
I have added an agent. Try this:
$post = "method=search&saveStats=true&screenSize=1920x1080&query=bond&valid=Szukaj+&sourceLanguage=en&targetLanguages=pl&domain=&domain=12&typeOfSearch=s&request=";
$ch = curl_init();
/* Additional Code Start */
$agent = "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:30.0) Gecko/20100101 Firefox/30.0";
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
/* Additional Code End*/
curl_setopt($ch, CURLOPT_URL, 'http://iate.europa.eu/SearchByQueryEdit.do');
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post);
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Type: application/x-www-form-urlencoded'));
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
echo $server_output = curl_exec ($ch);
Related
Sorry for my English. Tell me how to pass non-standard headers in cURL
$headers = array("Accept: application/json, text/plain, /*",
"Captcha: NDRMR9",
"Referer: https://avtokod.mos.ru/AutoHistory/");
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "https://avtokod.mos.ru/AutoHistory/GetByLicensePlate");
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query(array('licensePlate' => $licensePlate, 'sts' => $sts)));
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLINFO_HEADER_OUT, true);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:61.0) Gecko/20100101 Firefox/61.0');
curl_setopt($ch, CURLOPT_COOKIEFILE, $_SERVER["DOCUMENT_ROOT"]."/cookie.txt");
curl_setopt($ch, CURLOPT_COOKIEJAR, $_SERVER["DOCUMENT_ROOT"]."/cookie.txt");
$response = curl_exec($ch);
$info = curl_getinfo($ch);
curl_close($ch);
echo "<pre>";
echo $response;
print_r($info['request_header']);
But I get an error
{"code":1,"message":"Необходимо ввести капчу"}
I. e. I didn't send a captcha.
I can see that in the debugger the Captcha parameter is passed in the header
open image
What am I doing wrong?
I am trying to log into an .asp site with a https login page from PHP. I can't get logged into the site . There dosen't seem to be a cookie generated, viewstate, etc nor does it work leaving these parameters out. The form fields seem to fill correctly (i can physically see the login name) but im not sure about the password field which is a password type, but i don't think there is a issue there, its correctly spelled etc.
I have tried all the related posts including http://www.mishainthecloud.com/2009/12/screen-scraping-aspnet-application-in.html?showComment=1368565341638#c9104469935977149435
My code is below and returning "not Found" (error code 7 i think..) on the final call. No curl errors are present on the 1st two calls.
Can anyone help with this?
$ckfile = tempnam ("/tmp", "CURLCOOKIE");
// URL to login page
$url = "https://secure2.clubwise.com/glenview/memberlogin.asp";
// Get Login page and its cookies/ viewstate , etc
$ch = curl_init();
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($ch, CURLOPT_URL, $url);
//curl_setopt($ch, CURLOPT_COOKIEJAR, $ckfile); // no cookie stored
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
$output = curl_exec($ch);
//print(esc_html($output));
$viewstate = regexExtract($output,$regexViewstate,$regs,1);
$eventval = regexExtract($output, $regexEventVal,$regs,1);
// rebuild post info -- view state and eventvalidate empty! cant find on page
$fields_string =
'__VIEWSTATE='.rawurlencode($viewstate).
'&__EVENTVALIDATION='.rawurlencode($eventval).
'&login='.urlencode('xxx#xxx.com').
'&password='.urlencode('xxxx').
'&submit='.urlencode('Sign in').
'&redirect=';
echo $fields_string;
// Post login form -- password field ok?
$ch = curl_init();
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); // Accepts all CAs
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_POST, 5);
curl_setopt($ch, CURLOPT_POSTFIELDS, $fields_string);
curl_setopt($ch, CURLOPT_COOKIEFILE, $ckfile);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
//curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:9.0.1) Gecko/20100101 Firefox/9.0.1");
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); // Tells cURL to follow redirects
$outputb = curl_exec($ch);
print curl_error;
//var_dump($outputb);
$url = "https://secure2.clubwise.com/glenview/bookclass.asp";
$ch = curl_init();
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_COOKIEFILE, $ckfile);
//curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:9.0.1) Gecko/20100101 Firefox/9.0.1");
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$outputc = curl_exec($ch);
print curl_error;
var_dump($outputc);
header information:
//$header= array(
//'HTTP/1.1 200 OK',
//'Date: Mon, 09 Jun 2014 00:55:17',
//'GMT Server: Microsoft-IIS/6.0',
//'X-Powered-By: ASP.NET',
//'Content-Length: 21035',
//'Content-Type: text/html',
//'Set-Cookie: ASPSESSIONIDCCSBSTDC=IHJBDOOBIDMJDDOFLAOOBENL; path=/',
//'Cache-control: private'
//);
SOLUTION: it was curl_init() on each operation that was causing this to break. Viewstate, headers are not needed.
$ourFileName = get_stylesheet_directory()."/cookieFile.txt";
$ckfile = $ourFileName;
// URL to login page
$url = "https://secure2.clubwise.com/glenview/memberlogin.asp";
$ch = curl_init();
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); // Accepts all CAs
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_COOKIEJAR, $ckfile); // Stores cookies in the temp file
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
$output = curl_exec($ch);
//Cookie obtained, login
$fields_string =
'&redirect='.
'&login='.urlencode('vvv#xxx.com').
'&password='.urlencode('xxx').
'&submit='.urlencode('"Sign in"' )
;
//cookie in the header + header not required
/*
$cookielines =file($ckfile);
foreach($cookielines as $row) {
if($row[0] != '#') {
$cookie=$row;
}
}
$header= array( // not needed at moment
'HTTP/1.1 200 OK',
'Date: Mon, 09 Jun 2014 00:55:17',
'GMT Server: Microsoft-IIS/6.0',
'X-Powered-By: ASP.NET',
'Content-Length: 21035',
'Content-Type: text/html',
'Set-Cookie: $cookie; path=/',
'Cache-control: private'
);
*/
$url = "https://secure2.clubwise.com/glenview/memberlogin.asp";
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); // Accepts all CAs
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_POST, 4);
curl_setopt($ch, CURLOPT_POSTFIELDS, $fields_string);
curl_setopt($ch, CURLOPT_COOKIEFILE, $ckfile); //Uses cookies from the temp file
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:9.0.1) Gecko/20100101 Firefox/9.0.1");
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); // Tells cURL to follow redirects
curl_setopt($ch, CURLOPT_HTTPHEADER, $header);
curl_setopt($ch, CURLOPT_ENCODING,"");
curl_setopt($ch, CURLOPT_REFERER,"https://secure2.clubwise.com/glenview/memberlogin.asp");
$outputb = curl_exec($ch);
$err = curl_errno ( $ch ); echo "<br>error=".$err;
$errmsg = curl_error ( $ch ); echo "<br>errmsg=".$errmsg;
$header = curl_getinfo ( $ch ); echo "<br>header="; var_dump($header);
$httpCode = curl_getinfo ( $ch, CURLINFO_HTTP_CODE ); echo "<br>httpcode=".$httpCode;
print curl_error;
//Now you should be able to access any pages within the password-restricted area by just including the cookies for each call:
$url = "https://secure2.clubwise.com/glenview/bookclass.asp?Mode=Area&RecId=67";
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); // Accepts all CAs
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_POST, 0 );
curl_setopt($ch, CURLOPT_COOKIEFILE, $ckfile); //Uses cookies from the temp file
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:9.0.1) Gecko/20100101 Firefox/9.0.1");
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$outputc = curl_exec($ch);
print curl_error;
var_dump($outputc);
Please check again that cookie stored on first page. Set CURLOPT_COOKIEFILE and CURLOPT_COOKIEJAR for Session cookies and not reinit or close curl for save a session.
Set CURLOPT_REFERER, some sites check it for login page.
I am trying to login into oscommerce admin panel via cURL and PHP, I put already the right login and password in the post_array but I keep getting "Error: Invalid administrator login attempt." Am I sending the POST data and saving the cookies correctly?
Here is my code:
<?php
$agent = 'Mozilla/5.0 (Windows; U; Windows NT 5.1; tr; rv:1.9.2.12) Gecko/20101026 Firefox/3.6.12 ( .NET CLR 3.5.30729; .NET4.0E)';
$username = 'PromptUsername';
$password = 'PromptPassword';
$URL = 'http://www.example.com/admin/login.php'; //'http://www.example.com/admin/categories.php?cPath=21';
$path1 = 'cookies/';
$post_array = array(
'username'=>'usernAme',
'password'=>'passw0rd'
);
$authURL = $URL.'?action=process';
// Login here
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $authURL);
curl_setopt($ch,CURLOPT_USERAGENT,$agent);
curl_setopt($ch, CURLOPT_COOKIEJAR, realpath($path1.'cookies.txt') );
curl_setopt($ch, CURLOPT_COOKIEFILE, realpath($path1.'cookies.txt'));
curl_setopt($ch, CURLOPT_COOKIESESSION, true);
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_MAXREDIRS, 10);
curl_setopt($ch, CURLOPT_REFERER, $URL);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($post_array));
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
'method' => 'POST',
"Authorization: Basic ".base64_encode("$username:$password")."\r\n",
));
$debug = curl_getinfo($ch, CURLINFO_HTTP_CODE);
var_dump($debug);
$store = curl_exec ($ch);
var_dump($store);
?>
You are adding \r\n with the authorization header which upsetting the server. Just remove them.
"Authorization: Basic ".base64_encode("$username:$password"),
I would like to get the html of the page. After some google-ing, i found following code, which isn't working.
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_USERPWD, "$username:$passwd");
curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_BASIC);
$output = curl_exec($ch);
$info = curl_getinfo($ch);
curl_close($ch);
echo $output;
edit: something like this then:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, '');
curl_setopt($ch, CURLOPT_POSTFIELDS,'username='.urlencode($username).'&passwd='.urlencode($passwd));
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_COOKIEJAR, "/tmp/my_cookies.txt");
curl_setopt($ch, CURLOPT_COOKIEFILE, "/tmp/my_cookies.txt");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt( $ch, CURLOPT_SSL_VERIFYPEER, false );
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.3) Gecko/20070309 Firefox/2.0.0.3");
curl_setopt($ch, CURLOPT_REFERER, "");
$output = curl_exec($ch);
echo $output
This just shows the target page, it doesn't log in, nor does it return the html code.
edit 2: now tried with $data = array('username' => $username, 'passwd' => $passwd); and curl_setopt($ch, CURLOPT_POSTFIELDS,$data);
This shows the same as above + an error: Request Entity Too Large
That page does not use HTTP Basic Authentication.
You need to make an HTTP request to match what submitting the form would sent, then you need to process the response (which will probably involve storing cookies).
I am trying to log into a website using PHP CURL. It all works fine on websites which doesn't require cookies and session but it doesn't seem to work with the websites which rquire you so here is my code I found this code over here
any help on this would be apritiated thanks
Code
<?php
// 1-Get First Login Page http://signin.ebay.com/aw-cgi/eBayISAPI.dll?SignIn
$ebay_user_id = "username"; // Please set your Ebay ID
$ebay_user_password = "password"; // Please set your Ebay Password
$cookie_file_path = "cookie.txt"; // Please set your Cookie File path
$LOGINURL = "__";
$agent = "Mozilla/5.0 (Windows; U; Windows NT 5.0; en-US; rv:1.4) Gecko/20030624 Netscape/7.1 (ax)";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,$LOGINURL);
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie_file_path);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file_path);
$result = curl_exec ($ch);
// curl_close ($ch);
// 2- Post Login Data to Page http://signin.ebay.com/aw-cgi/eBayISAPI.dll
$LOGINURL = "url";
$POSTFIELDS = 'postfiends';
$reffer = "url";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,$LOGINURL);
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS,$POSTFIELDS);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_REFERER, $reffer);
curl_setopt($ch, CURLOPT_COOKIEFILE, $cookie_file_path);
curl_setopt($ch, CURLOPT_COOKIEJAR, $cookie_file_path);
$result = curl_exec ($ch);
// curl_close ($ch);
print $result;
?>
Maybe you need more options:
// define some HTTP headers
$headers[] = "Accept: */*";
$headers[] = "Connection: Keep-Alive";
$headers[] = "Content-type: application/x-www-form-urlencoded;charset=UTF-8";
// to GET add
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
// to POST add
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_HEADER, 1);
// check for errors before close
$result = curl_exec($ch);
if ($result === false)
{
echo curl_error($ch);
}
curl_close($ch);
Be sure that your $cookie_file_path file is writable (if Linux). Play with CURLOPT_SSL_VERIFYHOST and CURLOPT_SSL_VERIFYPEER.
http://signin.ebay.com/aw-cgi/eBayISAPI.dll not only expects username and password as post data, but other parameters as well, maybe you didn't take account of that. Use Net tab on firebug to see the parameters passed, and try to duplicate it.