This question already has answers here:
How to scrape websites when cURL and allow_url_fopen is disabled
(4 answers)
Closed 9 years ago.
I've got several functions in my php app that relay on calls to file_get_contents(), file_put_contents, and getimagesize().
The problem is that when allow_url_fopen an allow_url_include are disabled in php.ini I'm getting errors on these critical functions.
Warning: getimagesize() [function.getimagesize]: URL file-access is
disabled in the server configuration in
/home/content/.../html/_sites/mysite/wp-content/themes/mytheme/functions.php
on line 2534
What are the options for working around these issues?
EDIT: These are all local files on the same webserver as the calling php script. Is there a preferred method for reading/writing file contents?
You can use curl to get remote pages. You can store the results of the curl request to a variable and echo() in place of using the url wrapper to fetch content.
In theory, you could also eval() the returned data. But running remotely fetched PHP code is a ENORMOUS security risk, any PHP code included in this way can literally do anything you can. DON'T DO IT! The same goes for allow_url_include too
If you have access to your webserver, you may have to find your php.ini file, for example:
/etc/php5/apache/php.ini
And use these options:
; http://php.net/allow-url-fopen
allow_url_fopen = On
; http://php.net/allow-url-include
allow_url_include = Off
If your using someking of hosting account, these options may be interactively available at a control panel.
The second solution might be to use cURL. I suppose you were trying to call getimagesize with an URL. Documentation here: http://php.net/manual/en/book.curl.php
$ch = curl_init('http://example.com/image.php');
$fp = fopen('/my/folder/flower.gif', 'wb');
curl_setopt($ch, CURLOPT_FILE, $fp);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_exec($ch);
curl_close($ch);
fclose($fp);
Related
I am using file_get_contents in PHP to get information from a client's collections on contentDM. CDM has an API so you can get that info by making php queries, like, say:
http://servername:port/webutilities/index.php?q=function/arguments
It has worked pretty well thus far, across computers and operating systems. However, this time things work a little differently.
http://servername/utils/collection/mycollectionname/id/myid/filename/myname
For this query I fill in mycollection, myid, and myname with relevant values. myid and mycollection have to exist in the system, obviously. However, myname can be anything you want. When you run the query, it doesn't return a web page or anything to your browser. It just automatically downloads a file with myname as the name of the file, and puts it in your local /Downloads folder.
I DON'T WISH TO DOWNLOAD THIS FILE. I just want to read the contents of the file it returns directly into PHP as a string. The file I am trying to get just contains xml data.
file_get_contents works to get the data in that file, if I use it with PHP7 and Apache on my laptop running Ubuntu. But, on my desktop which runs Windows 10, and XAMPP (Apache and PHP5), I get this error (I've replaced sensitive data with ###):
Warning:
file_get_contents(###/utils/collection/###/id/1110/filename/1111.cpd):
failed to open stream: No such file or directory in
D:\Titus\Documents\GitHub\NativeAmericanSCArchive\NASCA-site\api\update.php
on line 18
My coworkers have been unable to help me so I am curious if anyone here can confirm or deny whether this is an operating system issue, or a PHP version issue, and whether there's a solid alternative method that is likely to work in PHP5 and on both Windows and Ubuntu.
file_get_contents() is a simple screwdriver. It's very good for getting data by simply GET requests where the header, HTTP request method, timeout, cookiejar, redirects, and other important things do not matter.
fopen() with a stream context or cURL with setopt are powerdrills with every bit and option you can think of.
In addition to this, due to some recent website hacks, we had to secure our sites more. In doing so, we discovered that file_get_contents failed to work, where curl still would work.
Not 100%, but I believe that this php.ini setting may have been blocking the file_get_contents request.
; Disable allow_url_fopen for security reasons
allow_url_fopen = 0
Either way, our code now works with curl.
reference :
http://25labs.com/alternative-for-file_get_contents-using-curl/
http://phpsec.org/projects/phpsecinfo/tests/allow_url_fopen.html
So, You can solve this problem by using PHP cURL extension. Here is an example that does the same thing you were trying:
function curl($url)
{
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
$url = 'your_api_url';
$data = curl($url);
And finally you can check your data by print_r($data). Hope it you it will works and you will understand.
Reference : http://php.net/manual/en/book.curl.php
I want to be able to read the contents of a file from its full domain and filepath, so something like:
http://www.example.com/files/file.txt
What I can't do in this instance is:
../files/file.txt
I have tried curl, fopen, file_get_contents and would rather use curl but cannot get it to work for any of them.
Is there an obvious reason why this isn't working that I am missing?
Here are the code snippets for each attempt, parhaps someone knows what's wrong with one of them?
Incidentally, if I could do ../files/file.txt it works for each option.
$file = "http://www.example.com/files/file.txt";
fopen:
$f=fopen($file,'r');
$data='';
while(!feof($f))
$data.=fread($f,$size);
fclose($f);
curl:
function curl($url){
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER,1);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
$data = curl($file);
file_get_contents:
$data = file_get_contents($file);
Thanks in advance for all help.
file_get_contents works fine for URLs with PHP 4 >= 4.3.0 and PHP 5:
"A URL can be used as a filename with this function if the fopen wrappers have been enabled"
Many shared hosters have this option disabled, or are using an older version of PHP and are blocking loading external files using PHP's safe mode.
Start with enabling error_reporting and have a look at your hosters web site to see if he's blocking external files.
Perhaps url fopen are disabled on your hosting and urls from another domains are just disabled?
php.ini var allow_url_fopen must be on
According to the description of the Google Custom Search API you can invoke it using the GET verb of the REST interface, like with the example:
GET https://www.googleapis.com/customsearch/v1?key=INSERT-YOUR-KEY&cx=017576662512468239146:omuauf_lfve&q=lectures
I setup my API key and custom search engine, and when pasted my test query directly on my browser it worked fine, and I got the JSON file displayed to me.
Then I tried to invoke the API from my PHP code by using:
$json = file_get_contents("$url") or die("failed");
Where $url was the same one that worked on the browser, but my PHP code was dying when trying to open it.
After that I tried with curl, and it worked. The code was this:
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
$body = curl_exec($ch);
Questions:
How come file_get_contents() didn't work and curl did?
Could I use fsocket for this as well?
Question 1:
At first you should check ini setting allow_url_fopen, AFAIK this is the only reason why file_get_contents() shouldn't work. Also deprecated safe_mode may cause this.
Oh, based on your comment, you have to add http:// to URL when using with file system functions, it's a wrapper that tells php that you need to use http request, without it function thinks you require to open ./google.com (the same as google.txt).
Question 2:
Yes, you can build almost any cURL request with sockets.
My personal opinion is that you should stick with cURL because:
timeout settings
handles all possible HTTP states
easy and detailed configuration (there is no need for detailed knowledge of HTTP headers)
file_get_contents probably will rewrite your request after getting the IP, obtaining the same thing as:
file_get_contents("xxx.yyy.www.zzz/app1",...)
Many servers will deny you access if you go through IP addressing in the request.
With cURL this problem doesn't exists. It resolves the hostname leaving the request as you set it, so the server is not rude in response.
This could be the "cause", too..
1) Why are you using the quotes when calling file_get_contents?
2) As it was mentioned in the comment, file_get_contents requires allow_url_fopen to be enabled on your php.ini.
3) You could use fsockopen, but you would have to handle HTTP requests/responses manually, which would be to reinvent the wheel when you have cURL. The same goes for socket_create.
4) Regarding the title of this question: cURL can be more customizable and useful to work with complex HTTP transactions than file_get_contents. Though, it should be mentioned, that working with stream contexts allows you to make a lot of settings for your file_get_contents calls. However, I think cURL is still more complete since it gives you, for instance, the possibility of working with multiple parallel handlers.
I use following PHP function:
file_get_contents('http://example.com');
Whenever I do this on a certain server, the result is empty. When I do it anywhere else, the result is whatever the page's content may be. When I however, on the server where the result is empty, use the function locally - without accessing an external URL (file_get_contents('../simple/internal/path.html');), it does work.
Now, I am pretty sure it has something to do with a certain php.ini configuration. What I am however not sure about is, which one. Please help.
Complementing Aillyn's answer, you could use a function like the one below to mimic the behavior of file_get_contents:
function get_content($URL){
$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_URL, $URL);
$data = curl_exec($ch);
curl_close($ch);
return $data;
}
echo get_content('http://example.com');
The setting you are looking for is allow_url_fopen.
You have two ways of getting around it without changing php.ini, one of them is to use fsockopen(), and the other is to use cURL.
I recommend using cURL over file_get_contents() anyways, since it was built for this.
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://www.your_external_website.com");
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
$result = curl_exec($ch);
curl_close($ch);
is best for http url,
But how to open https url help me
The is related to the ini configuration setting allow_url_fopen.
You should be aware that enable that option may make some bugs in your code exploitable.
For instance, this failure to validate input may turn into a full-fledged remote code execution vulnerability:
copy($_GET["file"], ".");
The answers provided above solve the problem but don't explain the strange behaviour the OP described. This explanation should help anyone testing communication between sites in a development environment where these sites all reside on the same host (and the same virtualhost; I'm working with apache 2.4 and php7.0).
There's a subtlety with file_get_contents() I came across that is absolutely relevant here but unaddressed (probably because it's either barely documented or not documented from what I can tell or is documented in an obscure php security model whitepaper I can't find).
With allow_url_fopen set to Off in all relevant contexts (e.g. /etc/php/7.0/apache2/php.ini, /etc/php/7.0/fpm/php.ini, etc...) and allow_url_fopen set to On in the command line context (i.e. /etc/php/7.0/cli/php.ini), calls to file_get_contents() for a local resource will be allowed and no warning will be logged such as:
file_get_contents('php://input');
or
// Path outside document root that webserver user agent has permission to read. e.g. for an apache2 webserver this user agent might be www-data so a file at /etc/php/7.0/filetoaccess would be successfully read if www-data had permission to read this file
file_get_contents('<file path to file on local machine user agent can access>');
or
// Relative path in same document root
file_get_contents('data/filename.dat')
To conclude, the restriction allow_url_fopen = Off is analogous to an iptables rule in the OUTPUT chain, where the restriction is only applied when an attempt to "exit the system" or "change contexts" is made.
N.B. allow_url_fopen set to On in the command line context (i.e. /etc/php/7.0/cli/php.ini) is what I had on my system but I suspect it would have no bearing on the explanation I provided even if it were set to Off unless of course you're testing by running your scripts from the command line itself. I did not test the behaviour with allow_url_fopen set to Off in the command line context.
This will also give external links an absolute path without having to use php.ini
<?php
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, "http://www.your_external_website.com");
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
$result = curl_exec($ch);
curl_close($ch);
$result = preg_replace("#(<\s*a\s+[^>]*href\s*=\s*[\"'])(?!http)([^\"'>]+)([\"'>]+)#",'$1http://www.your_external_website.com/$2$3', $result);
echo $result
?>
Enable allow_url_fopen From cPanel Or WHM in PHP INI Section
Add:
allow_url_fopen=1
in your php.ini file. If you are using shared hosting, create one first.
We are using shared hosting and the follow features are disabled.
file_uploads = Off
allow_url_fopen = Off
allow_url_include = Off
We are unable to change hosting and need to figure out some workarounds. The hosting co is also not able/willing to enable these features.
For example:
We are calling 1 server from another in order to get content. So we do an include but since URL file include is disabled we are not sure what options we have to get the content on that second server and store it there using some kind of cache.
We control the content server fully (dedicated) so we can do whatever necessary just not sure if there is some easy solution to the problem.
Since you're looking to retrieve remote content the easiest way will be to write the functionality to fetch the content yourself with something like curl (php.net/curl)
Have you tried something like this:
http://www.humanumbrella.com/2007/12/08/how-to-download-a-remote-file-in-php-and-then-save-it/
It depends on how locked down the server is. The given examples (using curl functions or fsockopen) should not be hampered by the restrictions you mentioned.
You can solve your problem like this
a) Create mechanism in dedicated server to fetch any file (plus some kind of key based authentication and restrictions on paths where files can be fetched from)
eg: A url that says get_file?path=/path/to/file&key=security_key
b) Write a function to fetch this as if from a local file
function fetch_file($path) {
$ch = curl_init("http://www.example.com/get_file?path=$path&key=security_key");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_BINARYTRANSFER, true);
$output = curl_exec($ch);
curl_close($ch);
return $output;
}
Then you can eval the returned string and that would be like including the file
eval fetch_file($path);
Another solution to write to the server if php file upload is prevented is to ftp the file on to your server and include the file.