i am fetching somesite page..
but it display nothing
and url address change.
example i have typed
http://localhost/sushant/EXAMPLE_ROUGH/curl.php
in curl page my coding is=
$fp = fopen("cookie.txt", "w");
fclose($fp);
$agent= 'Mozilla/5.0 (Windows; U; Windows NT 5.1; pl; rv:1.9) Gecko/2008052906 Firefox/3.0';
$ch = curl_init();
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
// 2. set the options, including the url
curl_setopt($ch, CURLOPT_URL, "http://www.fnacspectacles.com/place-spectacle/manifestation/Grand-spectacle-LE-ROI-LION-ROI4.htm");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt ($ch, CURLOPT_COOKIEJAR, 'cookie.txt');
curl_setopt($ch, CURLOPT_COOKIEFILE, "cookie.txt");
// 3. execute and fetch the resulting HTML output
if(curl_exec($ch) === false)
{
echo 'Curl error: ' . curl_error($ch);
}
else
echo $output = curl_exec($ch);
// 4. free up the curl handle
curl_close($ch);
but it canege url like this..
http://localhost/aide.do?sht=_aide_cookies_
object not found.
how can solve these problem help me
It looks like you're both trying to save cookies to cookies.txt, and read them from there. What you would normally do is that the first url you visit, you have curl save the cookies to a file. Then, for subseqent requests, you supply that file.
I'm not sure of the php aspects, but from the curl aspects it looks like you're trying to read a cookie file that doesn't exist yet.
edit: oh, and if you're only doing one request, you shouldn't even need cookies.
Seems like there is javascript in the output, which is causing the redirect.
So for testing purpose, instead of using:
echo $output = curl_exec($ch);
Use:
$output = curl_exec($ch);
echo strip_tags($output);
Update:
The code below will put the contents into contents.htm .. everything u need for pasring should be in there and in the output variable.
if(curl_exec($ch) === false)
{
echo 'Curl error: ' . curl_error($ch);
}
else{
$output = curl_exec($ch);
$fp2 = fopen("content.htm" , "w");
fwrite($fp2 , $output);
fclose($fp2);
}
Related
I'm currently developing a nagios plugin with PHP and cURL.
My problem is that my script is working well when i use it with PHP like this :
#php /usr/local/nagios/plugins/script.php
I mean it returns me a 200 HTTP CODE.
But with nagios it returns me a 0 HTTP CODE. It's strange because the php is working with NAGIOS (i can read variables...). So the problem is that Nagios can't use cURL.
Can someone give me a clue ? Thanks.
Here you can see my code.
<?php
$widgeturl = "http://google.com";
$agent = "Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.0.12) Gecko/2009070611 Firefox/3.0.12";
if (!function_exists("curl_init")) die("pushMeTo needs CURL module, please install CURL on your php.");
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $widgeturl);
$page = curl_exec($ch); //or die("Curl exe failed");
$code=curl_getinfo($ch, CURLINFO_HTTP_CODE);
if ($code==200) {
fwrite(STDOUT, $page.'Working well : '.$code);
exit(0);
}
else {
fwrite(STDOUT, $page.'not working : '.$code);
exit(1);
}
curl_close($ch);
Solution :
It was because the proxy was basically set on my OS (centOS), but Nagios was not using it instead of PHP. So i just had to put : curl_setopt($ch, CURLOPT_PROXY, 'myproxy:8080'); curl_setopt($ch, CURLOPT_PROXYUSERPWD, "user:pass"); Hope it could help someone
curl_setopt($ch, CURLOPT_PROXY, 'myproxy:8080');
curl_setopt($ch, CURLOPT_PROXYUSERPWD, "user:pass")
Can you try making the CURL request like this (i.e. header only request):
<?php
// config
$url = 'http://www.google.com/';
// make request & parse response
$curl = curl_init();
curl_setopt($curl, CURLOPT_URL, $url);
curl_setopt($curl, CURLOPT_FILETIME, true);
curl_setopt($curl, CURLOPT_NOBODY, true);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, true);
$response_header = curl_exec($curl);
$response_info = curl_getinfo($curl);
curl_close($curl);
// debug
echo "<b>response_header</b>\r\n";
var_dump($response_header);
echo "<b>response_info</b>\r\n";
var_dump($response_info);
The above will output the following:
I have been fighting with this for hours now I am trying to retrive a rss feed from maxhire:
rsslink, parse the content and display it using jfeed. now i am aware of the ajax not allowing for cross domain and i have been using the proxy.php that jfeed comes packaged with, but to no avail it just tells me there are to many redirects in the url so i have increased them like so:
<?php
header('Content-type: text/html');
$context = array(
'http'=>array('max_redirects' => 99)
);
$context = stream_context_create($context);
// hand over the context to fopen()
$handle = fopen($_REQUEST['url'], "r", false, $context);
if ($handle) {
while (!feof($handle)) {
$buffer = fgets($handle, 4096);
echo $buffer;
}
fclose($handle);
}
?>
but still no luck it just returns a message telling me that the object has been moved. So i have moved on to using curl like so:
$ch = curl_init('http://www.maxhire.net/cp/?EC5A6C361E43515B7A591C6539&L=EN');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HEADER, false);
$result = curl_exec($ch);
var_dump($result);
to retrive the xml page locally but it just returns the same error the object has moved:
<body>string(237) "<title>Object moved</title>
<h2>Object moved to here.</h2>
"
</body>
then redirects me to a url locally with : &AspxAutoDetectCookieSupport=1 added to the end.
Can someone please explain what i'm doing wrong?
Right I managed to get curl working by faking the useragent and the cookies and i am using a custom metafield in wordpress to assign the url like so:
<?php
$mykey_values = get_post_custom_values('maxhireurl');
foreach ( $mykey_values as $key => $value ) {
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $value);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 5.1) AppleWebKit/535.6 (KHTML, like Gecko) Chrome/16.0.897.0 Safari/535.6');
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_COOKIEFILE, "cookie.txt");
curl_setopt($ch, CURLOPT_COOKIEJAR, "cookie.txt");
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 30);
curl_setopt($ch, CURLOPT_REFERER, "http://www.maxhire.net");
$html = curl_exec($ch);
curl_close($ch);
echo $html;
}
?>
I made a nice simple userscript:
When I browse the web, I can "bookmark" any image in 1 click
My userscript
Grab the img src
Grab the url of the webpage
Copy the .jpg .png .gif to my server
Everything works perfectly, BUT in some cases, the script cannot copy the file...
Actually the file is created but do not contains the img data, it only contains the content of an error webpage:
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>403 Forbidden</title>
</head><body>
<h1>Forbidden</h1>
<p>You don't have permission to access /data/x/xxx_xxx_x.jpg on this server.</p>
<p>Additionally, a 403 Forbidden
error was encountered while trying to use an ErrorDocument to handle the request.</p>
<hr>
<address>Apache Server at xxxxxxxx.net Port 80</address>
</body></html>
The "copy" code (php):
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $urlimg);
curl_setopt($ch, CURLOPT_HEADER, false);
curl_setopt($ch, CURLOPT_BINARYTRANSFER, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
set_time_limit(300); # 5 minutes for PHP
curl_setopt($ch, CURLOPT_TIMEOUT, 300); # and also for CURL
$path = $dirpix.'/'.$aa.'/'.$mm;
if ( ! is_dir($path)) {
mkdir($path);
}
$outfile = fopen($path.'/'.$id.'.'.$ext, 'wb');
curl_setopt($ch, CURLOPT_FILE, $outfile);
curl_exec($ch);
fclose($outfile);
curl_close($ch);
Maybe the website blocks that kind of "copy" script?
Thanks!
2 things I can think of here are,
Set a user agent to your curl request. Because from what you say, you are able to view the image but curl is getting 403 error, it could very well be userAgent filtering on server side.
Add referer to your curl request. You can send the referer information from your userscript to the php script. You'd have to post or get window.location.href's value.
Try it below code it working fine in my server. it is tested code:-
<?php
$img[]='http://i.indiafm.com/stills/celebrities/sada/thumb1.jpg';
$img[]='http://i.indiafm.com/stills/celebrities/sada/thumb5.jpg';
$path="images/";
foreach($img as $i){
save_image($i, $path);
if(getimagesize($path.basename($i))){
echo '<h3 style="color: green;">Image ' . basename($i) . ' Downloaded OK</h3>';
}else{
echo '<h3 style="color: red;">Image ' . basename($i) . ' Download Failed</h3>';
}
}
//Alternative Image Saving Using cURL seeing as allow_url_fopen is disabled - bummer
function save_image($img,$fullpath='basename'){
if($fullpath!='basename'){
$fullpath = $fullpath.basename($img);
}
$ch = curl_init ($img);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_BINARYTRANSFER,1);
$rawdata=curl_exec($ch);
curl_close ($ch);
if(file_exists($fullpath)){
unlink($fullpath);
}
$fp = fopen($fullpath,'x');
fwrite($fp, $rawdata);
fclose($fp);
}
for correct work add
$agent= 'Accept:text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.64 Safari/537.11';
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
I had a hard time to access my DLink camera using this method.
But finally I found the issue: authentication.
Don't forget authentication.
This is the solution that worked for me, thanks to all contributors.
<?php
function download_image1($image_url, $image_file){
$fp = fopen ($image_file, 'w+'); // open file handle
$ch = curl_init($image_url);
$agent= 'Accept:image/jpeg,text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8 User-Agent:Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/537.11 (KHTML, like Gecko) Chrome/23.0.1271.64 Safari/537.11';
curl_setopt($ch, CURLOPT_USERAGENT, $agent);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_BINARYTRANSFER,true);
curl_setopt($ch, CURLOPT_TIMEOUT, 400);
curl_setopt($ch, CURLOPT_AUTOREFERER, false);
curl_setopt($ch, CURLOPT_REFERER, "http://google.com");
curl_setopt($ch, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE); // Follows redirect responses.
curl_setopt($ch, CURLOPT_USERPWD, "user:password");
$raw=curl_exec($ch);
if ($raw === false) {
trigger_error(curl_error($ch));
}
curl_close ($ch);
$localName = $image_file; // The file name of the source can be used locally
if(file_exists($localName)){
unlink($localName);
}
$fp = fopen($localName,'x');
fwrite($fp, $raw);
fclose($fp);
}
download_image1("http://url_here/image.jpg","/path/filename.jpg"); // to access DLink cameras
// be sure you have rights to the path
?>
The code above probably has some redundance, since I am openning fopen twice. To be honest, I won't correct, sinve it is working!
I have a web service which you post data to and it returns you a txt.gz file. I'm trying to use cURL to post up the information, however I'm not sure how to be ready and handle the file which comes back at me to download.
I'm getting a successful response; but the file is 0 KB, and obviously isn't downloading. Any help would be greatly appreciated!
Here is currently what I'm doing:
$url = 'http://www.mywonderfulurl.com';
$fields = array('id'=>'123');
$fields_string = 'id=123';
$useragent="Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1";
$ch = curl_init();
// set user agent
curl_setopt($ch, CURLOPT_USERAGENT, $useragent);
//set the url, number of POST vars, POST data
curl_setopt($ch,CURLOPT_URL,$url);
curl_setopt($ch,CURLOPT_POST,count($fields));
curl_setopt($ch,CURLOPT_POSTFIELDS,$fields_string);
curl_setopt($ch, CURLOPT_TIMEOUT, 250);
curl_setopt($ch, CURLOPT_FILE, 'download.txt.gz');
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
if (file_put_contents('download.txt.gz', curl_exec($ch)) === false) {
// Handle unable to write to file error.
echo('you failed');
exit;
}
echo curl_getinfo( $ch, CURLINFO_HTTP_CODE );
//close connection
curl_close($ch);
You are never writing to the file.
You need to write the result to file.
fwrite($fp, $result);
You are probably better off using file_put_contents, then you don't need to open/close the file.
http://php.net/manual/en/function.file-put-contents.php
if ($file_put_contents('download.txt.gz', curl_exec($ch)) === false) {
// Handle unable to write to file error.
}
But maybe using the $result in between to check the request was actually ok before saving it to file.
You should check the return from curl_exec()
$result = curl_exec($ch);
if ($result === false) {
echo 'Curl error: ' . curl_error($ch);
}
Writing false to a file would result in an empty file, so this could be whats happening.
A similar question has been posted at but i could not find the solution there
Curl error Could not resolve host: saved_report.xml; No data record of requested type"
<?php
$url="http://en.wikipedia.org/wiki/Pakistan";
$ch = curl_init(urlencode($url));
echo $ch;
// used to spoof that coming from a real browser so we don't get blocked by some sites
$useragent="Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.1) Gecko/20061204 Firefox/2.0.0.1";
curl_setopt($ch, CURLOPT_USERAGENT, $useragent);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 4);
curl_setopt($ch, CURLOPT_TIMEOUT, 8);
curl_setopt($ch, CURLOPT_LOW_SPEED_TIME, 10);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
$content = curl_exec($ch);
$info = curl_getinfo($ch);
if ($content === false || $info['http_code'] != 200) {
$content = "No cURL data returned for $url [". $info['http_code']. "]";
if (curl_error($ch))
$content .= "\n". curl_error($ch);
}
else {
// 'OK' status; format $output data if necessary here:
echo "...";
}
echo $content;
curl_close($ch);
?>
when i paste the same address in browser i am able to access the webpage. but when i run this script i get the error message. Can anyone please help me.
Thanks
Remove the urlencode call.
remove the urlencode($url) it should be:
$ch = curl_init($url);
Well.
If you remove urlencode() with instantiating your $ch-var, you go just fine. urlencode() is definitely wrong here.
Good:
$ch = curl_init($url);
Bad:
$ch = curl_init(urlencode($url));
$ch = curl_init($url);
instead of
$ch = curl_init(urlencode($url));