I am working on a website, that will have a large number of files. So, I made a separate server for my files such as images and txt files. The problem is that php's file_get_contents function does not work for this server.
I have tried echo file_get_contents("http://url"); and I get nothing, but when I do echo file_get_contents("http://google.com"); I get google's homepage. This the same case for a curl connection.
$ch = curl_init();
$url = "http://running-files.rf.gd/hello.html";
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE);
curl_setopt($ch, CURLOPT_MAXREDIRS, 5);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
$body = curl_exec($ch);
$info = curl_getinfo($ch);
$error = curl_errno($ch);
curl_close($ch);
echo $body;
My guess is that there is something need in the .htaccess file. Anyone have some suggestions?
If you're opening a URI with special characters,
(such as spaces) you need to encode the URI with urlencode().
Ex:
file_get_contents("http://domain-name.com?id=".urlencode("something with special characters"));
"something with special characters" can be a variable at most cases
Related
I'm trying to get some data from a website that is not mine, using this code.
<?
$text = file_get_contents("https://ninjacourses.com/explore/4/");
echo $text;
?>
However, nothing is being echo'd, and the string length is 0.
I've done this method before, and it has worked no problem, but with this website, it is not working at all.
Thanks!
I managed to get the contents using curl like this:
$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_URL, "https://ninjacourses.com/explore/4/");
$result = curl_exec($ch);
curl_close($ch);
cURL is a way you can hit a URL from your code to get a html response from it. cURL means client URL which allows you to connect with other URLs and use their responses in your code
$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_URL, "https://ninjacourses.com/explore/4/");
$result = curl_exec($ch);
curl_close($ch);
i think this is useful for you curl-with-php and another
I want to grab a set of images using the URL in PHP. I've tried using file_get_contents and curl. Below is the code that I have tried using.
$image = file_get_contents('http://user:pwd#server/directory/images/image1.jpg');
file_put_contents('D:/images/image1.jpg', $image);
and
$url = 'http://server/directory/images/image1.jpg';
$localFilePath = 'D:/images/image1.jpg';
$ch = curl_init ($url);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_BINARYTRANSFER,1);
curl_setopt($ch, CURLOPT_USERPWD, "user:pwd");
$raw = curl_exec($ch);
curl_close ($ch);
if(file_exists($localFilePath)){
unlink($localFilePath);
}
$fp = fopen($localFilePath,'wb');
fwrite($fp, $raw);
fclose($fp);
In both cases, I am getting the following error:
401 - Unauthorized : Access is denied due to invalid credentials.
The password has a special character. I can't change it to a plain password, as the password policies don't allow it.
I Don't see the place where you tell CURL to use AUTHentication:
curl_setopt($ch, CURLOPT_HTTPAUTH, CURLAUTH_ANY);
I think it also might be because you're not using cookies, enable them:
curl_setopt($ch, CURLOPT_COOKIEJAR, "cookies.txt");
The equivalent of this line is
curl_setopt($ch, CURLOPT_USERPWD, "user:pwd");
This one, and you can try this by replacing your one
curl_setopt($ch, CURLOPT_HTTPHEADER,
array("Authorization: Basic ".base64_encode("user:pwd")));
// also you can try by changing + / characters into _-
Now come to the point. Base64 has several implementation based on RFC. For example : RFC3538. Different library or different language implemented different RFC for base64 encoding/decoding. For example at some implementation it uses + and / character and some use the _ and - character.
So lets say your curl is sending the base64 string for the authorization is xyz+12= but your server is expecting the string to be as xyz_12=. So it will obviously fail to decode.
Got a php script using cURL grabbing the contents of a url that has colons in the source name:
$url = 'http://www.awebsite.com/anxml:file:thatoddly:hascolons:allovertheplace:';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 5);
$data = curl_exec($ch);
if(curl_errno($ch)) {
echo 'Curl error: ' . curl_error($ch);
}
curl_close($ch);
I am getting the error.
Could not resolve host: http; nodename nor servname provided, or not known <url here>
I've double checked that the url is working fine otherwise, but I suspect cURL is choking on the colons in the filename. The source isn't mine, so I can't remove the colons.
Is there another way around this?
Provider fixed their files, so I don't have to deal with colons any longer. Turns out I was using cURL improperly after all and likely the urlencode() with the code below would have worked.
This DIDN'T WORK:
$url = urlencode($url);
$url = str_replace("http%3A","http:",$url);
$ch = curl_init();
$timeout = 5;
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, $timeout);
$data = curl_exec($ch);
This DID WORK:
$url = urlencode($url);
$url = str_replace("http%3A","http:",$url);
$c = curl_init($url);
curl_setopt($c, CURLOPT_RETURNTRANSFER, true);
$data = iconv("UTF-8","ISO-8859-1",curl_exec($c));
Hope that helps someone out.
Looking at the man page, cURL has a --data-urlencode flag.
If it's just one URL not being done via CLI but PHP, you could use PHP's urlencode().
I get files by their urls by this code
file_get_contents($_POST['url'];
Then I do something with them.
But I don't want to operate with big files, how do I limit size of received file?
It should throw an error if file is bigger than 500kb.
See my answer to this question. You need to have the cURL extension, with which you can make a HEAD HTTP request to the remote server. The response will let you know how big the file is, and you can then decide accordingly.
You are interested specifically in this line:
$size = curl_getinfo($ch, CURLINFO_CONTENT_LENGTH_DOWNLOAD);
Agree with #Jon
$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, true);
curl_setopt($ch, CURLOPT_NOBODY, true);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_URL, $url); //specify the url
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
$head = curl_exec($ch);
$size = curl_getinfo($ch,CURLINFO_CONTENT_LENGTH_DOWNLOAD);
if(<limit the $size>){
file_get_contents($url);
}
I have a script where I have to download some files and to make sure that everything worked fine I'm comparing MD5 checksums.
I have found that the checksums are not correct when downloading with CURL. The script below demonstrates this. It downloads the Google logo and compares checksums.
$url = 'http://www.google.com/intl/en_ALL/images/logo.gif';
echo md5_file($url)."\n";
$path = 'f1';
file_put_contents($path, file_get_contents($url));
echo md5_file($path)."\n";
$path = 'f2';
$out = fopen($path, 'wb');
$ch = curl_init();
curl_setopt($ch, CURLOPT_FILE, $out);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_URL, $url);
curl_exec($ch);
curl_close($ch);
echo md5_file($path)."\n";
the output is:
e80d1c59a673f560785784fb1ac10959
e80d1c59a673f560785784fb1ac10959
d83892759d58a1281e3f3bc7503159b5
The first two are correct (they match the MD5 checksum when I download the logo using firefox) and the result produced by curl is not OK.
any ideas how to fix that?
thanks for your help
UPDATE:
interestingly the code below works just fine and produces the correct output. The problem really only seems to exist when saving to a file. Unfortunately I have to save directly to a file since the files I'm downloading can get rather large.
$path = 'f3';
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL,$url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
file_put_contents($path, curl_exec ($ch));
echo md5_file($path)."\n";
curl_close ($ch);
You're missing an fclose($out), which could account for md5_file seeing an incomplete file.
Try to add
curl_setopt($ch, CURLOPT_BINARYTRANSFER, 1);
To your curl options