i use file_get_contents function to grab data from sites and store the data in database. it will be very inconvenient for me, if one day the script will start not working.
I know, that it can start not working, if they change the structure of site, but now i'm afraid, that maybe there are mechanisms to disable the working of this function, maybe from server?
i tried to find documentation about it, but can't get, so maybe you will help me?
Thanks
I know, that it can start not working,
if they change the structure of site,
but now i'm afraid, that maybe there
are mechanisms to disable the working
of this function, maybe from server?
Yes, it can be disabled from php.ini with allow_url_fopen option. You have other options such as CURL extension too.
Note also that you will need to have openssl extension turned on from php.ini if you are going to use the file_get_contents function to read from a secure protocol.
So in case file_get_contents is/gets disabled, you can go for CURL extension.
It is possible to disable certain functions using disable_function. Furthermore the support of URLs with filesystem functions like file_get_contents can be disabled with allow_url_fopen. So chances are that file_get_contents might not work as expected one day.
There are at least two PHP configuration directives that can break your script :
If allow_url_fopen is disabled, then, file_get_contents() will not be able to fetch files that are not on the local disk
i.e. it will not be able to load remote pages via HTTP.
Note : I've seen that option disabled quite a few times
And, of course, with disable_functions, any PHP function can be disabled.
Chances are pretty low that file_get_contents() itself will ever get disabled...
But remote-file loading... Well, it might be wise to add an alternative loading mecanism to your script, that would use curl in case allow_url_fopen is disabled.
Related
In PHP, the allow_url_fopen flag controls whether or not remote URLs can be used by various file system functions, in order to access remote files.
It is recommended security best practice nowadays to disable this option, as it is a potential attack vector. However, any code which depends on this functionality in order to work would be broken if the setting is disabled. For example, I know of at least one reCaptcha plugin which uses file_get_contents() to access the Google API and which therefore depends on this flag.
In order to check the code in our applications to determine whether it is safe to disable this flag (with a view to rewriting, where necessary) I need a canonical list of the PHP functions that it affects. However, I have been unable to find such a list - there doesn't seem to be one on the PHP website and a Google search didn't turn anything up.
Can anyone provide a list of all PHP functions whose behaviour is affected by allow_url_fopen?
The accepted answer should reference an authoritative source or provide details about methodology used to compile the list, to demonstrate its correctness and completeness.
The list of functions is massive, as the allow_url_fopen ini directive is implemented in PHP's streams system, meaning anything that uses PHP's network streams are affected.
This includes functions from pretty much every extension of PHP that does not use an external library for gaining access to a remote file. As some extensions like cURL uses its own transport layer outside that of PHP.
Some extensions, notoriously ext/soap does bypass this directive in some ways (for what reason I don't exactly know as I'm not familiar with the internals of this extension).
Any function from the standard library (implemented in: main/, Zend/, ext/standard, ext/spl), meaning every Filesystem, Stream, Includes and URL Wrappers respect this directive. From on top of my head I also know that ext/exif does this.
I cannot remember on top of my head if XML based extensions (such as ext/libxml, ext/simplexml, ext/xmlreader, ext/xmlwriter, ext/dom) does this, but I'm certain that there was a point in the past where they did not respect it as the path was directly supplied to LibXML2 underneath.
This is crying out for a list of functions/methods that can take either a file path or a URL when allow_url_fopen is enabled. Making it community wiki, as the reason I found this question was that I was looking for such a list and am unsure that I am considering every corner case.
Opens a file
copy
file
file_get_contents
file_put_contents
fopen
simplexml_load_file
Stats a file
file_exists
filemtime
filesize
filetype
is_dir
is_file
Note: not all of these will work for every kind of URL. For example, "https://" URLs do not allow writing, so copy and file_put_contents will fail on such destinations. Meanwhile, ftp:// URLs do allow writing. Similar issues with file_exists.
I am deliberately not including functions like fwrite and fclose. Because those in particular take the results of fopen. So to my mind, it is fopen that is impacted, not fwrite nor fclose. Because fwrite can't open a file, only fopen can (of those three). So it is fopen that needs checked, not subsequent uses of fwrite or fclose. Those will work or fail if fopen does.
This is why I find answers like "Any function from the standard library" less than helpful. Most of those functions will work with streams opened under allow_url_fopen, but they will not themselves open such a stream. There may be many functions and methods that take resources that were originally opened via a URL, but I don't care about them unless they participate in the opening.
Another way of stating this is that I'm trying to list all functions that accept a URL (e.g. https://stackoverflow.com/ ) as a file path when allow_url_fopen is enabled (but not when it is disabled). Functions like fwrite and fclose do not do this (they take resources, not file paths). So I don't care about them even if their behavior is impacted by allow_url_fopen. I realize that the original question does not make this clear, but I believe that this was what was intended.
Related: list of supported protocols and wrappers.
The fsockopen and curl functions can open URLs even with allow_url_fopen turned off.
I'm trying to write a website in PHP that allows the user to enter PHP code, and then be able to run it on my server. However, I want to be able to disable certain features (file access, database access, etc.). Basically, I want the code to run without any risk to my server, and if the code does attempt to do something dangerous, I just want the code to stop running (I don't mind if it just stops, produces an error, or carries on while ignoring the dangerous code).
Is this possible, and if so, how could I achieve this?
Thanks :)
It is possible using libraries that do some simple checking or limiting.
Take a look at a PECL (PHP Extensions) extension called RunKit_Sandbox http://php.net/manual/en/runkit.sandbox.php or PHPSandbox.
The key to look for on Google is PHP Sandbox, it will find you similar libraries.
vi php.ini
and then find disable_functions,
disable the functions as you want! like this :
disable_functions = exec,passthru,popen,proc_open,shell_exec,system,phpinfo,assert,chroot,getcwd,scandir,delete,rmdir,rename,chgrp,chmod,chown,copy,mkdir,file,file_get_contents,fputs,fwrite,dir
I actually developed a package specifically for these kinds of use cases. It can be fully configured and even used to override dangerous functions and globals.
https://github.com/fieryprophet/php-sandbox
for security reasons I have disabled the function glob in the php.ini and it works as expected, but I also noticed that phpinfo reveals the following information:
Registered PHP Streams: php, file, glob, data, http, ftp, zip, compress.zlib, phar
So if I take following source:
$it = new DirectoryIterator("glob://C:\wamp\www\*");
foreach($it as $f) {
printf("%s: %.1FK\n", $f->getFilename(), $f->getSize()/1024);
}
It would still return the contents of the specified directory.
How can I globally unregister PHP Streams such as glob?
The short answer is: don't even bother trying.
PHP is a complete enough language that if someone is going to write dirty of vulnerable code, they can do it through any block you put in place. The only thing disabling functions like that does is make application developers' lives hell.
It's been proven that things like safe_mode and open_basedir don't actually secure anything. The reason is twofold:
Black lists (which is what safe_mode is) don't work. This has been proven over and over and over.
You can't secure on top of an insecure base. It's already too late. PHP itself already has enough access that even if you disable all the fun parts, people can still get around it.
Instead, protect from the bottom up. Install a chroot jail, and run PHP inside that. Use proper permissions. Vet the code that you run on your server. Monitor the server for intrusions. Nothing fancy. Just good old fashioned sys-admin work...
To answer your original question
The only way that you can unregister a stream wrapper is to do it yourself via stream_wrapper_unregister(). You could use an auto-prepend-file to do it for you (to run that code before every script).
But realize that it's trivial to implement glob in PHP. So there really isn't much point in disabling it...
I am trying to talk to a buggy webserver that I don't control over SSL from PHP+curl.
I wrote a little C-program directly against openssl lib, and through that identified that, if I enable the openssl options SSL_OP_NO_COMPRESSION and SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS explicitly in call to SSL_CTX_set_options(), I can get it to work. Like this:
SSL_CTX_set_options(ctx, SSL_OP_NO_COMPRESSION | SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS)
but, ...that is in my isolated C program.
Can I, somehow, make PHP+curl set these options, when it establishes the SSL connection? CURL seems to operate on a much higher level.
Here are my own findings, so far:
I know about curl_setopt, but I see no options like those in its list.
I have found something called stream options, but I am not clear on how or if they are used with CURL, and again, I see no match for the options I need.
The PHP manual on openssl seems to only be about functions to handle keys and certificates.
Then, there is HttpRequest setSslOptions, but again, that seems to closely match options to CURL context.
UPDATE
After the response from "user2076645" on the option on disabling compression, I git cloned the source of PHP and took a look around myself.
Specifically, I found this piece of code:
#if OPENSSL_VERSION_NUMBER >= 0x0090605fL
ssl_ctx_options &= ~SSL_OP_DONT_INSERT_EMPTY_FRAGMENTS;
#endif
which explicitly disables the option I need. I looked up the commit message, too, and it was a fix to a possible attack on SSL.
So, I guess it can't be done from PHP, currently :-(
Not sure if it helps but there is an option to disable compression
http://git.php.net/?p=php-src.git;a=commitdiff;h=4a01ddfb5569da1b87dd4cac95c3f709fb607396;hp=bb4d11b405ae1f37a8b0e4db630e80c5678f0746
I have a script that users can input a image URL (from another website) and then crop it using JS and have it saved on my server.
My question is... when getting the image from another server is it safer to use CURL or allow_url_fopen (via file_get_contents())? Or is there a preferred/safer method available?
Security is a big concern for me as I know this is a very dangerous procedure - The script will only need to work for image files if that makes a difference.
Thanks
curl's error handling is much better than file_get_contents(). If you care about that, curl's probably the way to go. If a simple "oops, that didn't work" is enough for you, though, file_get_contents() is a perfectly acceptable shortcut.
First of all, if you want to get into a deep security discussion. Downloading files is in fact a security concern if you don't know what you are doing.
You can overwrite vital files or even overwrite system files in some cases. Uploading scripts,etc on the server with intention of executing them via web server is also an issue.
So it's not sunshine and rainbows like people pointing out here.
Back to your question, allow_url_fopen is a configuration directive. I assume you meant file_get_contents(). Either will do fine. As others pointed out Curl is a bit more verbose and it's also faster.
If you do end up using file_get_contents(), make sure you never include an unfiltered variable as a parameter.
Downloading a file is not a security concern at all, no matter whether it's your server or your own computer, or the application/code you are using to download it :) It's whether you are executing the file :D
All you have to do is just make sure you are not going to EXECUTE / INCLUDE anything in the file. Since you are only going to crop the image, I think you are good to go :)
I suggest cURL tho, allow_url_fopen may raise security problems in other places in your code.
cURL has more options ans possibilities.
Both are equally safe (or unsafe if misused).
It is wiser to use cURL because you will uprise your experience with a more powerful function, which may serve you in future projects.
Also, if this very project needs new functionalities later on, you will not have to rewrite everything with cURL if file_get_contents is not enough.
The answer of this thread shows a nice cURL function: Replace file_get_content() with cURL?
curl would generally be a safer way. It'd take an explicit design/coding decision on your part to allow the results from curl to directly affect your program, whereas allowing urls in the f*() functions would let
include('http://example.com/really_nasty_remote_takeover.php');
occur without error.