Well, I'm trying to get some results from a form in a Webpage, I've already logged in, having the PHPSESSID a the responses are OK in the previous steps. If I execute from command line like this:
curl "http://domain/"%"27/ajaxform" -H "Cookie: PHPSESSID=l2pgk9rqihvo5n5gf3uq79c5e6" -H "Origin: http://domain" -H "Accept-Encoding: gzip,deflate" -H "Accept-Language: es,en-US;q=0.8,en;q=0.6,es-419;q=0.4" -H "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.124 Safari/537.36" -H "Content-Type: application/x-www-form-urlencoded; charset=UTF-8" -H "Accept: */*" -H "Referer: http://domain/inicio" -H "X-Requested-With: XMLHttpRequest" -H "Connection: keep-alive" --data "formName=option1&other=attributes" --compressed > resp.txt
I can get the results in the resp.txt file.
But, running it in PHP all I get is http_code 500, I'm using the following code in PHP
function check_cdd($optionname, $sessionid){
$url="http://domain/".urlencode("'")."/ajaxform";
$cookie="cookie.txt";
$postvars = 'formName='.$optionname.'&other=attributes';
$ch = curl_init();
curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt ($ch, CURLOPT_URL, $url);
curl_setopt ($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt ($ch, CURLOPT_TIMEOUT, 60);
curl_setopt ($ch, CURLOPT_FOLLOWLOCATION, 0);
curl_setopt ($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt ($ch, CURLOPT_ENCODING , "");
curl_setopt ($ch, CURLOPT_COOKIEJAR, $cookie);
curl_setopt ($ch, CURLOPT_POSTFIELDS, $postvars);
curl_setopt ($ch, CURLOPT_POST, 1);
curl_setopt ($ch, CURLINFO_HEADER_OUT, false);
curl_setopt ($ch, CURLOPT_HEADER, false);
curl_setopt($ch,CURLOPT_HTTPHEADER,array(
'Cookie: PHPSESSID='.$sessionid,
'Origin: http://domain',
'Accept-Encoding:gzip,deflate',
'Accept-Language: es,en-US;q=0.8,en;q=0.6,es-419;q=0.4',
'User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.120 Safari/537.36',
'Content-Type: application/x-www-form-urlencoded; charset=UTF-8',
'Accept: */*',
'Referer: http://domain/inicio',
'X-Requested-With: XMLHttpRequest',
'Connection: keep-alive'));
$result = curl_exec ($ch);
$header_size = curl_getinfo($ch, CURLINFO_HEADER_SIZE);
$header = substr($result, 0, $header_size);
print("<pre>");
print_r(curl_error($ch));
print_r(curl_getinfo($ch));
echo $result;
print("</pre>");
curl_close($ch);}.
I've tryed differents params and options in PHP side, but always get 500 code in response, don't know what else to try. Thanks in advance for your answers
500 means internal server error...so help me to debug your code, by removing $postvars from your curl_setopt() function, it does not exist from your command line, and i suspect that it can be the origin of server error
Related
I tried to fetch some data using php curl_setopt in my code. But it went through 2 minutes of loading, then got 504 Gateway Timeout error. Here's the code:
function sendRequest($url, $data, $token = '') {
$header[] = "Connection: keep-alive";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HTTPHEADER, $header);
curl_setopt($ch, CURLOPT_HTTP_VERSION, 'CURL_HTTP_VERSION_1_1');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLOPT_CUSTOMREQUEST, 'POST');
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.120 Safari/537.36");
$result = curl_exec($ch);
curl_close($ch);
return $result;
}
Strangely, I succeeded fetching the data instantly using command line cURL with the same options:
curl --url "http://x.x.x.x" --header "Connection: keep-alive" --http1.1 --verbose --request "POST" -d "data1=value1" -d "data2=value2" --user-agent "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/37.0.2062.120 Safari/537.36"
The server that hosts the code and the destination is in the same network and firewall is not activated in both servers. What could be the cause of the failure?
No clue what $data is. If it's an associative array, try
curl_setopt($ch, CURLOPT_POSTFIELDS, http_build_query($data));
I want to create App Pinterest with PHP Curl but this is not working. The application I follow the POST process: Postman Interceptor
Request:
curl 'https://developers.pinterest.com/apps/'
-H 'User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:74.0) Gecko/20100101 Firefox/74.0'
-H 'Accept: */*'
-H 'Accept-Language: tr-TR,tr;q=0.8,en-US;q=0.5,en;q=0.3' --compressed
-H 'Content-Type: application/json'
-H 'X-CSRFToken: IjExYmYxYzZlZDgwM2RlMGQxNzEyZjdlZTRkZTk3NmQ0OTliZWQwYTki.XlzcOw.zET53I_suQKC2TbplvhpouiuaAM'
-H 'X-Requested-With: XMLHttpRequest'
-H 'Origin: https://developers.pinterest.com'
-H 'DNT: 1'
-H 'Connection: keep-alive'
-H 'Referer: https://developers.pinterest.com/apps/'
-H 'Cookie: _pinterest_sess=TWc9PSZESDJWQ0dCNmN0TERrZThaSEFLMnhrekhLVVRzaVMwKzBxY3YxaTJmMDdZMS9kR1V4SHB0UVorajRyY21TU1A4TCtXUElEdlpDMGVNd3BCcGhFYXg0Q3JjTm1uK2VxQ2dMTlo1UHJoQ2ZXc2t6YytLV01VYTNqNU5pL0tqQlZtWU9EMjQ0NGh4UG95VC93b1BPM21kRzNWSHlQT0tpaEdIZ2xNWmd1WUlvdUFDaHBrZDVLRHdYdWVUbnN6cEtRV1QrU1pUdzNTbElaZ0Y3VVc5VW81c1ArQzRZam41dEFCTVV6dEFCMngySnZ6dDZWeUNkeXA3cW5ZcGZUZkFGL1o4TjBnMFlCYUUzaEtEL0VwOTZIdkVHWXpQNjJ5NHRZUm5YNk5YU1hWWWgwV0FCWVJHbzFEb2Z3U24zR0FrbzZTcVJPaXZFYzduTCtCc3RPejYrVFFFd2pDOXM0czRpby96V3lzQlpFVzBLdEk1REVDVk0rajgxeGZRNldXUE9sc24xak50S3pPV3BzdWFXdkhEZ3Q5OUhiYlVjNHBPeUNsYkoySVpRMFpnd2QrcjNtNW0waUhZWkk4YjlXakFiTEtTaGJUOXpiRE9CWDJxcEUzLzZWYVNhZ1NmRDJFWTFDenhJZkZVdmoyeGI1WFZ6TldpZWFWeVhDNFozNkFpcEFsM0k3YlZUMkZiTEROZHpoNnFEdjdpdmJxMW14dVFSWnlETkRTUGFrWDNOd1MycUdETGZWYWNMT0JkbEVxS2N4TmhQekZiaE5zSkltWkVtQTZ6STV0Nm5yMzg2R2toREdiTHRPZ0ZlRXVPc1V2R3hIYUMrNjdNWGFOOGcxL1VHOStNWmZBRHFXd3JOZFJhRFVheXY2emVuRUszZVhuZFRkemdMS3hiL05aajd5WFYyMk5YQWR6MmcvYU9Pa1BtUnU1MTVZQjVmbDZyQjkyMDJXSGh1dXBjamFINGhwU2tYZXpRN1Z1UE1zbU83bytadkFnRURQSy9XcEF3d3VodDh5MXcwTU1LblFGbDUwdWk0a1VleC8vQTRlV3lzejNyeWpRc2sralV6dFZ3c0lMYjZjWFNRMTZreVh2R0Q4OHlSWG91JkE3eERJQXpWS1YyUG1GcldkZHhFeXpCdDlyST0=; _auth=1; session=eyJjc3JmX3Rva2VuIjoiMTFiZjFjNmVkODAzZGUwZDE3MTJmN2VlNGRlOTc2ZDQ5OWJlZDBhOSJ9.XlzdMQ.qu4RJUJgbv0UygBZFyPjw6-aGkw; csrftoken=lwBLncs70zm1Y2Xhx9D7M6w6pVZQw6ro; _dev_sess=TWc9PSZXMDN0NjdEK3FQSWw3WFFUaUNHSEdZVFdLYVgxME9WUkZDTlU1UGczUXRUeWpNZEVrUmZIbjFQSFlXK2V1NTBaS3JrNFZCaW1FVnY3WFRhODNVT3pwQ3ljUldQUXFmU1JFcVNuc2orNi9jSGpnRFAwVjlhRDdJUnBRekNGb3FZK21idk9IdFdlVXpJVzJkZnptaGZpRUtSQ2xpTnZPU2ZZMVdNNFY5MEM5cXV5ZUlzelYrM0tEL1JodU9yM3BBaUw2KzV2WEdJa3RJeEJQdTJSRjYrcUlIakFTS3BXRS9aNldUOXM1dmVLK2VYT1I5TENYQ0hSNGpzd21HcE9icU95M3poSVB4RDVyT1BQeUdnUnhjaGdVTFlvNHR4cUl6Zk9uZllmL1g5NEVZRDgzSzNxWjJPZFJjZXdQaVhuWnBkQzFhVUQzMm42NFplb2k5VUFMWTM3bnc9PSZEd0Y2VUlRQVlvOS9NbmMzcFBmaFNzdlQ4cGs9; country-code=TR'
-H 'TE: Trailers' --data '{"name":"test1","description":"test description"}'
Full Codes:
<?php
//preg_match('/csrftoken(.*?)[\b;\s]/i', $headers, $csrf_token);
error_reporting(E_ALL);
ini_set('display_errors', 1);
$appname = "deneme33";
$appdescription = "lorem ipsum deneme";
$username = "talithae.matson_53#hotmail.com";
$password = "SahsZGHx";
// this is the http post data for logging in - username & password are substituted in later
$login_post = array(
'source_url' => '/login/',
'data' => '{"options":{"username_or_email":"%s","password":"%s"},"context":{}}',
'module_path' => 'App()>LoginPage()>Login()>Button(class_name=primary, text=Log In, type=submit, size=large',
);
$create_app = array(
'data' => '{"name":"%s","description":"%s"}',
);
$pinterest_url = 'https://www.pinterest.com/'; //'https://www.pinterest.com/'; // pinterest home url
$developer_url = 'https://developers.pinterest.com/apps/'; // Developer home url
$login_url = $pinterest_url . 'login/'; // pinterest login page url
$login_post_url = $pinterest_url . 'resource/UserSessionResource/create/'; // pinterest login post url
// http headers to send with requests
$httpheaders = array(
'Connection: keep-alive',
'Pragma: no-cache',
'Cache-Control: no-cache',
'Accept-Language: en-US,en;q=0.5',
);
// http headers to send when logging in
$login_header = array(
'X-NEW-APP: 1',
'X-APP-VERSION: 08930e3', // THIS WILL UPDATE FREQUENTLY, CHANGE IT!!!
'X-Requested-With: XMLHttpRequest',
'Accept: application/json, text/javascript, */*; q=0.01');
$developer_header = array(
'Accept: */*',
'Sec-Fetch-Dest: empty',
'X-Requested-With: XMLHttpRequest',
'Content-Type: application/json');
// ----------------------------------------------------------------------------
// request home page to establish cookies and a session, set curl options
$ch = curl_init($pinterest_url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_AUTOREFERER, 1);
curl_setopt($ch, CURLOPT_ENCODING, 'gzip,deflate');
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:72.0) Gecko/20100101 Firefox/72.0');
curl_setopt($ch, CURLOPT_COOKIEFILE, 'cookie.txt');
curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookie.txt');
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_STDERR, fopen('debug.txt', 'w+'));
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_HTTPHEADER, $httpheaders);
$data = curl_exec($ch);
sleep(5);
//echo "Giriş Datası".$data;
// ----------------------------------------------------------------------------
// parse the csrf token out of the cookies to set later when logging in
list($headers, $body) = explode("\r\n\r\n", $data, 2);
$headers = file_get_contents('cookie.txt', true);
//print_r($headers);
preg_match("/csrftoken+(.*?)\n/i",$headers,$matches);
$csrf_token=trim($matches[1]);
preg_match("/session+(.*?)\n/i",$headers,$matches);
$session=trim($matches[1]);
echo "Token:". $csrf_token.PHP_EOL;
echo "Session:". $session.PHP_EOL;
//die();
// next request the login page
curl_setopt($ch, CURLOPT_URL, $login_url);
$data = curl_exec($ch);
//Developer Cookie Get
$ch = curl_init($developer_url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_AUTOREFERER, 1);
curl_setopt($ch, CURLOPT_ENCODING, 'gzip,deflate');
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:72.0) Gecko/20100101 Firefox/72.0');
curl_setopt($ch, CURLOPT_COOKIEFILE, 'cookie.txt');
curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookie.txt');
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_STDERR, fopen('debug.txt', 'w+'));
curl_setopt($ch, CURLOPT_HEADER, 1);
curl_setopt($ch, CURLOPT_HTTPHEADER, $httpheaders);
$data = curl_exec($ch);
//Developer Cookie Get
//Make APP
$create_app['data'] = sprintf($create_app['data'], $appname, $appdescription);
$post = http_build_query($create_app);
print_r($post);
die();
curl_setopt($ch, CURLOPT_URL, $developer_url);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post);
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
"Accept: */*",
"Sec-Fetch-Dest: empty",
"X-CSRFToken: ".$csrf_token,
"X-Requested-With: XMLHttpRequest",
"User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:72.0) Gecko/20100101 Firefox/72.0",
"Content-Type: application/json",
"Cache-Control: no-cache",
"Host: developers.pinterest.com",
"Accept-Encoding: gzip, deflate, br",
"Cookie: session=".$session,
"Connection: keep-alive"
));
curl_setopt($ch, CURLOPT_COOKIEFILE, 'cookie.txt'); // Cookie aware
curl_setopt($ch, CURLOPT_COOKIEJAR, 'cookie.txt'); // Cookie aware
curl_setopt($ch, CURLOPT_HEADER, 1);
$data = curl_exec($ch);
//Make APP
//echo "Baş Header".print_r($developer_header);
echo $data;
die();
$login_header[] = 'X-CSRFToken: ' . $csrf_token;
$login_post['data'] = sprintf($login_post['data'], $username, $password);
$post = http_build_query($login_post);
curl_setopt($ch, CURLOPT_URL, $login_post_url);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post);
curl_setopt($ch, CURLOPT_HTTPHEADER, array_merge($httpheaders, $login_header));
curl_setopt($ch, CURLOPT_REFERER, $login_url);
curl_setopt($ch, CURLOPT_HEADER, 0);
$data = curl_exec($ch);
echo $data;
// ----------------------------------------------------------------------------
// perform login post
//print_r($csrf_token);
// check response and output status
if (curl_getinfo($ch, CURLINFO_HTTP_CODE) != 200) {
echo "Error logging in.<br />";
var_dump(curl_getinfo($ch));
} else {
$response = json_decode($data, true);
if ($response === null) {
echo "Failed to decode JSON response.<br /><br />";
var_dump($response);
} else if ($response['resource_response']['status'] === "success") {
print_r($response['resource_response']);
//print_r($response['resource_response']['data']);
//print_r($response['resource_response']['username']);
echo "Login successful, " . $response['resource_response']['user']['username'] . "<br /><br />";
echo "You have {$response['resource_response']['data']['follower_count']} followers, are following {$response['resource_response']['data']['following_count']} users. You have liked {$response['resource_response']['data']['like_count']} pins.";
}
}
I think there is a problem with the data I sent after //Make APP.
Cookies: https://i.imgur.com/OoyVh4C.jpg
Body: https://i.imgur.com/A02mfGe.jpg
Headers: https://i.imgur.com/Adc8Dmd.jpg
Result: https://i.imgur.com/sXBPQxM.jpg
I might be missing something during the post operation.
What am I missing in xhr post sending?
The URL you are trying to call is not part of their normal REST API it looks like. I believe you are supposed to visit that manually and create your app first. Then work with their API after that.
Source
Did you find documentation from them that allows you to do that, or are you trying to hack a method together to accomplish it?
include_once('simple_html_dom.php');
$usuario = "username";
$password = "password";
$url = 'https://www.instagram.com/';
$url_login = 'https://www.instagram.com/accounts/login/ajax/';
$user_agent = array("Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 ",
"(KHTML, like Gecko) Chrome/48.0.2564.103 Safari/537.36");
$ch = curl_init();
$headers = [
'Accept-Encoding: gzip, deflate',
'Accept-Language: en-US;q=0.6,en;q=0.4',
'Connection: keep-alive',
'Content-Length: 0',
'Host: www.instagram.com',
'Origin: https://www.instagram.com',
'Referer: https://www.instagram.com/',
'User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.103 Safari/537.36',
'X-Instagram-AJAX: 1',
'X-Requested-With: XMLHttpRequest'
];
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($ch, CURLOPT_USERAGENT, $_SERVER['HTTP_USER_AGENT']);
curl_setopt($ch, CURLOPT_COOKIEFILE, "/tmp/cookie/pruebalogininsta2.txt");
curl_setopt($ch, CURLOPT_REFERER, $sTarget);
curl_setopt($ch, CURLOPT_HEADER, TRUE);
$html = curl_exec($ch);
preg_match_all('/^Set-Cookie:\s*([^;]*)/mi', $html, $matches);
$cookies = array();
foreach($matches[1] as $item) {
parse_str($item, $cookie);
$cookies = array_merge($cookies, $cookie);
}
$headers = [
'Accept-Encoding: gzip, deflate',
//'Accept-Language: ru-RU,ru;q=0.8,en-US;q=0.6,en;q=0.4',
'Accept-Language: en-US;q=0.6,en;q=0.4',
'Connection: keep-alive',
'Content-Length: 0',
'Host: www.instagram.com',
'Origin: https://www.instagram.com',
'Referer: https://www.instagram.com/',
'User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.103 Safari/537.36',
'X-Instagram-AJAX: 1',
'X-Requested-With: XMLHttpRequest'
];
$cadena_agregar_vector = 'X-CSRFToken:'. $cookies["csrftoken"];
$headers[] = $cadena_agregar_vector ;
$sPost = "username=".$usuario . "&password=". $password ;
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_POSTFIELDS, $sPost);
curl_setopt($ch, CURLOPT_URL, $url_login);
$html2 = curl_exec($ch);
curl_setopt($ch, CURLOPT_URL, "http://www.instagram.com/");
$html4 = curl_exec($ch);
echo $html4;
this is what I get
the problem is the way you hardcode Accept-Encoding: gzip, deflate, this makes curl send the encoding header indeed, but it does not turn on the decoding feature of curl, thus you get the raw data, without curl decoding it for you.
remove 'Accept-Encoding: gzip, deflate', and add curl_setopt($ch, CURLOPT_ENCODING, 'gzip, deflate'); , and curl will decode it for you (provided that curl is compiled with gzip & deflate support) - or better yet, just do curl_setopt($ch, CURLOPT_ENCODING, ''); , and curl will automatically list all supported encodings, so you dont run into the encoding problem where curl isn't compiled with gzip support.
on an unrelated note, you probably want to use CURLOPT_USERAGENT, not set the user-agent header manually. else, the UA-string will just be sent with this 1 request, and be reset on the next request, while CURLOPT_USERAGENT is kept until curl_close($ch)
edit: on my first revision of this post, i wrote CURLOPT_POSTFIELDS instead of CURLOPT_ENCODING, sorry, fixed that
edit 2: on another unrelated note, you're encoding the username/password wrong. instead of $sPost = "username=".$usuario . "&password=". $password ;, do
$sPost=http_build_query(array('username'=>$usuario,'password'=>$password));, else accounts with & or = or NULLs in the password or username wont work properly
The answer posted by #hanshenrik should really be accepted. But if you just want an easy solution that works and is not incorrect, remove the 'Accept-Encoding: gzip, deflate' from your headers array.
I am trying to make a website scraper, but the website is acting diferrently, than normal request via browser.
How can i make perfect cURL reguest, that the website will not filter it and block it?
Any help would be appriciated.
$curl_handle = curl_init ("***");
$header = array();
$header[] = "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:37.0) Gecko/20100101 Firefox/37.0";
$header[] = "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8";
$header[] = "Accept-Language: cs,en-US;q=0.7,en;q=0.3";
$header[] = "Accept-Encoding: utf-8";
$header[] = "Connection: keep-alive";
$header[] = "Host: ****";
curl_setopt($curl_handle, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:37.0) Gecko/20100101 Firefox/37.0');
curl_setopt($curl_handle, CURLOPT_HTTPHEADER, $header);
curl_setopt ($curl_handle, CURLOPT_COOKIEFILE, dirname(__FILE__) . '/cookie.txt');
curl_setopt ($curl_handle, CURLOPT_COOKIEJAR, dirname(__FILE__) . '/cookie.txt');
curl_setopt ($curl_handle, CURLOPT_RETURNTRANSFER, true);
curl_setopt ($curl_handle, CURLOPT_AUTOREFERER, true);
$output = curl_exec ($curl_handle);
This is, what i got so far, but it is still getting blocked.
The following CURL options might help:
curl_setopt($ch, CURLOPT_REFERER, $_SERVER['REQUEST_URI']);
curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
I have 0 experince with curl, and i want to run a request that will return a json body inside php.
using firefox firbug i can copy curl order and it looks like this
curl 'http://*****.com/index.php/home/search/keyword/c815f2fec5?value=keyworkd' -H
'Host: *****.com' -H
'User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:29.0) Gecko/20100101 Firefox/29.0' -H
'Accept: */*' -H
'Accept-Language: en-US,en;q=0.5' -H
'Accept-Encoding: gzip, deflate' -H
'X-Requested-With: XMLHttpRequest' -H
'Referer: http://****.com/' -H
'Cookie: _ga=GA1.2.373422434.1399222050; PHPSESSID=l8sf036kjaijt6cjvcqnu992l4'
can i simulate such request with php ?
Of course it is, without going much into unnecessary things and based on your limited info:
<?php
$c = curl_init('http://*****.com/index.php/home/search/keyword/c815f2fec5?value=keyworkd');
curl_setopt($c, CURLOPT_USERAGENT, "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:29.0) Gecko/20100101 Firefox/29.0");
curl_setopt($c, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($c, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($c, CURLOPT_COOKIE,
'_ga=GA1.2.373422434.1399222050; PHPSESSID=l8sf036kjaijt6cjvcqnu992l4');
curl_setopt($c, CURLOPT_REFERER, 'http://****.com/');
$z = curl_getinfo($c);
$s = curl_exec($c);
curl_close($c);
?>
Edit: Damn though it is POST not COOKIE, repaired.
Edit2: With cookie file.
<?php
$cookie_file = "cookie.txt"; //remember to check if it exists
$c = curl_init('http://*****.com/index.php/home/search/keyword/c815f2fec5?value=keyworkd');
curl_setopt($c, CURLOPT_USERAGENT, "User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:29.0) Gecko/20100101 Firefox/29.0");
curl_setopt($c, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($c, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($c, CURLOPT_COOKIEJAR, $cookie_file);
curl_setopt($c, CURLOPT_COOKIEFILE, $cookie_file);
curl_setopt($c, CURLOPT_REFERER, 'http://****.com/');
$z = curl_getinfo($c);
$s = curl_exec($c);
curl_close($c);
?>