Trying to do some local testing on a virtual server, the problem is cURL is returning a http_code => 0
I think it's to do with my virtual host naming.
Virtual Host Name: dev.project
the cURL request is adding http://
if I ping: dev.project from the command line, I get a hit.
If I try it with http://dev.project I get unknown host.
Is there a curl_setopt option just to use the hostname? I'm no sure if I can use the IP as there are several projects on the server, or would Apache handle this?
Here is what I have tried:
$request_url = 'dev.project';
$request_args = 'parm=1234';
$user_agent = 'Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.16 (KHTML, like Gecko) Chrome/10.0.648.151 Safari/534.16';
$ch = curl_init();
// set curl options (GET)
curl_setopt($ch, CURLOPT_URL, $request_url.'?'.$request_args);
curl_setopt($ch, CURLOPT_HEADER, FALSE);
//curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
//curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
//curl_setopt($ch, CURLOPT_FAILONERROR, TRUE);
//curl_setopt($ch, CURLOPT_USERAGENT, $user_agent);
//curl_setopt($ch, CURLOPT_COOKIEJAR, "cookie.txt");
//curl_setopt($ch, CURLOPT_COOKIEFILE, "cookie.txt");
//curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE); // CHANGE THIS TO TRUE
// Set curl options (POST)
//curl_setopt($ch, CURLOPT_URL, $request_url);
//curl_setopt($ch, CURLOPT_POST, TRUE);
//curl_setopt($ch, CURLOPT_POSTFIELDS, $request_args);
//curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
//curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
//curl_setopt($ch, CURLOPT_USERAGENT, $user_agent);
//curl_setopt($ch, CURLOPT_COOKIEJAR, "cookie.txt");
//curl_setopt($ch, CURLOPT_COOKIEFILE, "cookie.txt");
//curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
$response = curl_exec($ch);
echo "<pre>".print_r($response,true)."</pre><br />\n"; // nothing is returned
print_r(curl_getinfo($ch));
curl_close($ch);
Response from curl_getinfo() : (NOTE: the http:// is pre-pended in the url)
Array
(
[url] => http://dev.project?parm=1234
[http_code] => 0
[header_size] => 0
[request_size] => 0
[filetime] => -1
[ssl_verify_result] => 0
[redirect_count] => 0
[total_time] => 0
[namelookup_time] => 0
[connect_time] => 0
[pretransfer_time] => 0
[size_upload] => 0
[size_download] => 0
[speed_download] => 0
[speed_upload] => 0
[download_content_length] => 0
[upload_content_length] => 0
[starttransfer_time] => 0
[redirect_time] => 0
)
If anyone has this issue, here is the fix:
// This is your Virtual Hosts name
$request_host = 'dev.project';
// This is the IP
$request_url = '192.168.0.1';
$headers = array("Host: ".$request_host);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_URL, $request_url.'?'.$request_args);
curl_setopt($ch, CURLOPT_HEADER, FALSE);
Related
To restart my router I can access to web page at 192.168.1.1/tools.lp and click on the restart button.
This is the code of the page
<form name="gwRestart" method="post" action="restartingAG.lp">
<input type="hidden" name="action" id="restartAction" value="saveRestart" />
<table border='0' cellspacing='0' cellpadding='0' width='100%' style="text-align:center">
<tr>
<td>
<a href="javascript:document.gwRestart.submit();">
<div class="midarea7-1 mainButton">RESTART</div></a>
</td><td> </td>
</tr>
</table>
</form>
I would restart my router using a php page and I tried to do it with cURL in this way
$post_data['action'] = 'saveRestart';
//traverse array and prepare data for posting (key1=value1)
foreach ( $post_data as $key => $value) {
$post_items[] = $key . '=' . $value;
}
//create the final string to be posted using implode()
$post_string = implode ('&', $post_items);
//create cURL connection
$ch = curl_init('http://192.168.1.1/restartingAG.lp');
//set options
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 30);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_AUTOREFERER, true);
curl_setopt($ch, CURLOPT_COOKIESESSION, true);
curl_setopt($ch, CURLOPT_COOKIEJAR, "my_cookies.txt");
curl_setopt($ch, CURLOPT_COOKIEFILE, "my_cookies.txt");
//set data to be posted
curl_setopt($ch, CURLOPT_POSTFIELDS, $post_string);
//perform our request
$result = curl_exec($ch);
//show information regarding the request
print_r(curl_getinfo($ch));
//echo curl_errno($ch) . '-' . curl_error($ch);
//close the connection
curl_close($ch);
Unfortunately, I noticed that my request is redirected to index_auth.lp (login function is disabled!) and not to that required.
Array ( [url] => http://192.168.1.1/index_auth.lp [content_type] => text/html [http_code] => 200 [header_size] => 858 [request_size] => 630 [filetime] => -1 [ssl_verify_result] => 0 [redirect_count] => 2 [total_time] => 1.216 [namelookup_time] => 0 [connect_time] => 0 [pretransfer_time] => 0 [size_upload] => 0 [size_download] => 7233 [speed_download] => 5948 [speed_upload] => 0 [download_content_length] => 7233 [upload_content_length] => 0 [starttransfer_time] => 0.53 [redirect_time] => 0.686 [redirect_url] => [primary_ip] => 192.168.1.1 [certinfo] => Array ( ) [primary_port] => 80 [local_ip] => 192.168.1.5 [local_port] => 50687 )
Perhaps because the request is from a different ip? Can I somehow make a request as if it were made by a user on the web page? Thanks
$post_data = array();
$post_data['action'] = 'saveRestart';
foreach ( $post_data as $key => $value) {
$post_items[] = $key . '=' . $value;
}
$post_string = implode ('&', $post_items);
$ch = curl_init('http://192.168.1.1/restartingAG.lp');
$headers = array();
$headers[] = "text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
$headers[] = "Cache-Control: max-age=0";
$headers[] = "Connection: keep-alive";
$headers[] = "Keep-Alive: 300";
$headers[] = "Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7";
$headers[] = "Accept-Language: en-us,en;q=0.5";
$headers[] = "Pragma: ";
//set options
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 30);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_AUTOREFERER, true);
curl_setopt($ch, CURLOPT_COOKIESESSION, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_REFERER, 'http://192.168.1.1/tools.lp');
curl_setopt($ch, CURLOPT_COOKIEJAR, "my_cookies.txt");
curl_setopt($ch, CURLOPT_COOKIEFILE, "my_cookies.txt");
curl_setopt($ch, CURLOPT_POSTFIELDS, $post_string);
$result = curl_exec($ch);
curl_close($ch);
echo "<pre>";
echo $result;
From what i can, you are not to actually send your request as a POST request. Currently you are sending the message as a GET request, but with some post-data attached.
Your code is missing this line
curl_setopt($ch, CURLOPT_POST, 1);
I'm trying to scrap a website but it always said that Empty Reply from server
can any one look at the code and tell me what am I doing wrong?
Here is the code
function spider($url){
$header = array(
"Host" => "www.example.net",
//"Accept-Encoding:gzip,deflate,sdch",
"Accept-Language:en-US,en;q=0.8",
"Cache-Control:max-age=0",
"Connection:keep-alive","Content-Length:725","Content-Type:application/x-www-form-urlencoded",
'Accept' => 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8'
,"X-Requested-With:XMLHttpRequest"
);
$cookie = "cookie.txt";
$ch = curl_init();
curl_setopt($ch, CURLOPT_HTTPHEADER, $header);
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, 0); // return headers 0 no 1 yes
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); // return page 1:yes
curl_setopt($ch, CURLOPT_TIMEOUT, 200); // http request time-out 20 seconds
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true); // Follow redirects, need this if the URL changes
curl_setopt($ch, CURLOPT_MAXREDIRS, 2); //if http server gives redirection response
curl_setopt($ch, CURLOPT_USERAGENT,
"Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7");
curl_setopt($ch, CURLOPT_COOKIEJAR, realpath( $cookie)); // cookies storage / here the changes have been made
curl_setopt($ch, CURLOPT_COOKIEFILE, realpath( $cookie));
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false); // false for https
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, FALSE);
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS,"view=ViewDistrict¶m=7&uniqueid=1397991494188&PHPSESSID=f134vrnv7glosgojvf4n1mp7o2&page=http%3A%2F%2Fwww.example.com%2Fxhr.php");
curl_setopt($ch, CURLOPT_REFERER, $url);
curl_setopt($ch, CURLOPT_REFERER, "http://www.example.com/");
$data = curl_exec($ch); // execute the http request
$info = curl_getinfo($ch);
curl_close($ch); // close the connection
return $data;
}
Here is function call
echo spider("http://www.example.net/");
Edit
Array ( [url] => http://www.example.net/ [content_type] => text/html [http_code] => 301 [header_size] => 196 [request_size] => 840 [filetime] => -1 [ssl_verify_result] => 0 [redirect_count] => 1 [total_time] => 61.359 [namelookup_time] => 0 [connect_time] => 0.281 [pretransfer_time] => 0.281 [size_upload] => 0 [size_download] => 0 [speed_download] => 0 [speed_upload] => 0 [download_content_length] => 178 [upload_content_length] => 0 [starttransfer_time] => 60.593 [redirect_time] => 0.766 [certinfo] => Array ( ) [redirect_url] => ) Empty reply from server
this is the header now
also I'd updated my post data
it's now
curl_setopt($ch, CURLOPT_POSTFIELDS,"view=ViewDistrict¶m=7&uniqueid=".time(). rand(101,500)."&PHPSESSID=f134vrnv7glosgojvf4n1mp7o2&page=http%3A%2F%2Fexample.com%2Fxhr.php");
and also had removed "X-Requested-With:XMLHttpRequest" from headers
Have you tried removing this from the headers ?
X-Requested-With:XMLHttpRequest
My guess is that your problem is in this line:
curl_setopt(
$ch,
CURLOPT_POSTFIELDS,
"view=ViewDistrict¶m=7&uniqueid=1397991494188&PHPSESSID=f134vrnv7glosgojvf4n1mp7o2&page=http%3A%2F%2Fwww.example.com%2Fxhr.php"
);
Notice that you're passing a value for PHPSESSID. I'm guessing you copied & pasted a URL from a visit to the site, right? That session ID was probably valid when you visited the site, but the odds of it being valid now are pretty slim. And if the server doesn't like the session ID, chances are it's not going to give you any data.
here is my URL: https://webservices-dev.compuscan.co.za:9443/PersonStatusService/user2/password2/8310240031083/XML
i am try to get its content with PHP curl but no success hope some can help.
here is my code:
// you can add anoother curl options too
// see here - http://php.net/manual/en/function.curl-setopt.php
function get_data($url) {
$ch = curl_init();
$timeout = 5;
curl_setopt($ch, CURLOPT_URL, $url);
//curl_setopt($ch, CURLOPT_POST, TRUE); // Use POST method
//curl_setopt($ch, CURLOPT_POSTFIELDS, "var1=1&var2=2&var3=3"); // Define POST values
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0)");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST,false);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER,false);
curl_setopt($ch, CURLOPT_MAXREDIRS, 10);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
$data = curl_exec($ch);
$output = curl_getinfo($ch);
print_r($output);
curl_close($ch);
return $data;
}
$url = "https://webservices-dev.compuscan.co.za:9443/PersonStatusService/user2/password2/8310240031083/XML" ;
$variablee = get_data($url);
echo $variable;
Well i am getting results as expected.
It should be
$variablee = get_data($url);
echo $variablee; //<--- Must be $variablee (as you have defined it above)
OUTPUT:
Array ( [url] =>
https://webservices-dev.compuscan.co.za:9443/PersonStatusService/user2/password2/8310240031083/XML
[content_type] => text/plain; charset=utf-8 [http_code] => 200
[header_size] => 81 [request_size] => 192 [filetime] => -1
[ssl_verify_result] => 19 [redirect_count] => 0 [total_time] => 1.544
[namelookup_time] => 0 [connect_time] => 0.343 [pretransfer_time] =>
1.186 [size_upload] => 0 [size_download] => 656 [speed_download] => 424 [speed_upload] => 0 [download_content_length] => 656
[upload_content_length] => 0 [starttransfer_time] => 1.544
[redirect_time] => 0 [certinfo] => Array ( ) [primary_ip] =>
196.34.30.23 [primary_port] => 9443 [local_ip] => 192.168.72.37 [local_port] => 61090 [redirect_url] => )
8310240031083CLAIRECSAPLARN1983/10/24FY2012/03/242013/09/308 SKALIE
STWEST ACRES EXT 1312012013/08/130768333118
function get_data($url) {
$ch = curl_init($url);
$timeout = 5;
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 8.0;
Windows NT 6.0)");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, false);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
curl_setopt($ch, CURLOPT_MAXREDIRS, 10);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
$data = curl_exec($ch);
$httpCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
if ($httpCode == 404) {
curl_close($ch);
return '404';
} else {
curl_close($ch);
return $data;
}
}
This function worked perfectly for me, maybe somebody finds it useful as I did. I'm not pretending to answer your question (well, I'm 9 months late) but solve people's similar questions about a similar issue.
i'm trying to call a url in a page on a remote site. Decided on using curl. On the remote site the url vars are showing as:
$_REQUEST Array
(
[var1] => val1
[amp;var2] => val2
[amp;var3] => val3
)
the url being called is:
http://site.com/controller/action.php?var1=val1&var2=val2&var3=val3
Notice that I'm not using & in the url, but the request global has it, or nearly - it has amp; instead of & and not & like I used!!! Not that it should have any of them.
The curl object has logged in to a form, set cookies, posted a form on another page, and now this is the last link I have to follow.
here is the php i'm using:
curl_setopt($this->ch, CURLOPT_URL, "http://site.com/controller/action.php?var1=val1&var2=val2&var3=val3");
curl_setopt($this->ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($this->ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($this->ch, CURLOPT_REFERER, "http://site.com/");
curl_setopt($this->ch, CURLOPT_VERBOSE, 1);
curl_setopt($this->ch, CURLOPT_COOKIEJAR, $this->cookie);
curl_setopt($this->ch, CURLOPT_COOKIEFILE, $this->cookie);
curl_setopt($this->ch, CURLOPT_POST, 0);
curl_setopt($this->ch, CURLOPT_HTTPGET, 1);
$output = curl_exec($this->ch);
$info = curl_getinfo($this->ch);
I've ran another curl request after this and I'm still logged in so issue is not cookies. Because the last request (the login form) used $_POST have have set the CURLOPT_POST to 0 and the CURLOPT_HTTPGET to 1.
Here is the output from $info:
info Array
(
[url] => http://site.com/controller/action.php?var1=val1&var2=val2&var3=val3
[content_type] => text/html; charset=utf-8
[http_code] => 403
[header_size] => 403
[request_size] => 541
[filetime] => -1
[ssl_verify_result] => 0
[redirect_count] => 0
[total_time] => 0.781186
[namelookup_time] => 3.7E-5
[connect_time] => 3.7E-5
[pretransfer_time] => 4.2E-5
[size_upload] => 1093
[size_download] => 3264
[speed_download] => 4178
[speed_upload] => 1399
[download_content_length] => 3264
[upload_content_length] => 0
[starttransfer_time] => 0.781078
[redirect_time] => 0
[certinfo] => Array
(
)
)
if i copy and past $info['url'] into the browser it works. Completely lost, hours lost on the clock, any help would be appreciated ;)
Try modifying your code as follows:
$data = array('val1'=>"val1", 'val2'=>"val2", 'val3'=>"val3");
curl_setopt($this->ch, CURLOPT_URL, "http://site.com/controller/action.php");
curl_setopt($this->ch, CURLOPT_POSTFIELDS, http_build_query($data));
curl_setopt($this->ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($this->ch, CURLOPT_FOLLOWLOCATION, 1);
curl_setopt($this->ch, CURLOPT_REFERER, "http://site.com/");
curl_setopt($this->ch, CURLOPT_VERBOSE, 1);
curl_setopt($this->ch, CURLOPT_COOKIEJAR, $this->cookie);
curl_setopt($this->ch, CURLOPT_COOKIEFILE, $this->cookie);
curl_setopt($this->ch, CURLOPT_POST, 0);
curl_setopt($this->ch, CURLOPT_HTTPGET, 1);
$output = curl_exec($this->ch);
$info = curl_getinfo($this->ch);
EDIT
Removed custom encode function as per Brad's comment - you don't need it, as PHP has a build-in function that does the same thing.
I created an API for customers using cURL. I just moved to a new server this domain and now the api doesnt not work. Everything seems to be working fine modules wise but I can't get it to work:
This is the response I get.
Array (
[url] => https://www.1800pay.com/api/process.php
[content_type] => text/html; charset=iso-8859-1
[http_code] => 404
[header_size] => 179
[request_size] => 506
[filetime] => -1
[ssl_verify_result] => 0
[redirect_count] => 0
[total_time] => 0.038607
[namelookup_time] => 0.002688
[connect_time] => 0.002737
[pretransfer_time] => 0.038372
[size_upload] => 0
[size_download] => 294
[speed_download] => 7615
[speed_upload] => 0
[download_content_length] => 294
[upload_content_length] => 0
[starttransfer_time] => 0.038597
[redirect_time] => 0 )
Curl error:
Not Found
The requested URL /api/process.php was not found on this server.
Apache/2.2.3 (CentOS) Server at www.1800pay.com Port 443
CODE USED:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "https://www.1800pay.com/api/process.php");
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 15);
curl_setopt($ch, CURLOPT_TIMEOUT, 15);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, 0);
//curl_setopt($ch, CURLOPT_VERBOSE, true);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST,0);
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, 0);
curl_setopt($ch, CURLOPT_POSTFIELDS, $output_transaction);
curl_setopt($ch, CURLOPT_POST, 1);
if (!($data = curl_exec($ch))) {print_r(curl_error($ch));echo "error";
return ERROR;
}
print_r(curl_getinfo($ch));
echo 'Curl error: ' . curl_error($ch);
curl_close($ch);
print_r($data);
Thanks for the help :)
And YES THE FILE EXISTS ON THE SERVER..... :|
The answer is within the response.. a not found error is exactly that. Just make sure the file api/process.php exists on the new domain.
I'm not sure what your problem is but when I try to curl the URL using this:
function curl_get($url)
{
$curl_handle = curl_init();
curl_setopt($curl_handle, CURLOPT_URL, $url);
curl_setopt($curl_handle, CURLOPT_CONNECTTIMEOUT, 2);
curl_setopt($curl_handle, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($curl_handle, CURLOPT_COOKIEJAR, "cookie.txt");
curl_setopt($curl_handle, CURLOPT_COOKIEFILE, "cookie.txt");
//curl_setopt($curl_handle, CURLOPT_USERPWD, $co.":".$apikey);
curl_setopt($curl_handle, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($curl_handle, CURLOPT_DNS_USE_GLOBAL_CACHE, FALSE);
$buffer = curl_exec($curl_handle);
$error = curl_error($curl_handle);
curl_close($curl_handle);
//if (empty($buffer))
//echo "No response from server: ".$error;
//else
return $buffer;
}
I get this:
Error [Code:1043] Payment ()Key - incorrect data