I am having some trouble with PHP's file_get_contents function.
My setup is basically as follows:
-> From a php page (let's call it a.php), a POST request is sent to another php page (b.php) through file_get_contents.
-> b.php does some stuff with the POST input and then sends another POST request through file_get_contents to itself (b.php)
->This is repeated a couple of times (for example 4 times), so basically it looks like this:
a.php -> POST request through file_get_contents -> b.php -> POST request through file_get_contents -> b.php -> POST request through file_get_contents -> b.php -> POST request through file_get_contents -> b.php
At the last post request to b.php, the script echoes something to the "poster", he adds something to it etc. etc. all the way back to a.php.
For clarity's sake: in production all those php files will be on different servers, and each server has an added value in the process.
For testing however, all pages are on the same server (and I add "?server=x" to the URL so that the same file uses a different database at every "call").
This works like a charm :) ... Unless there are more than 5 file_get_contents are "active" simultaneously ...
This works fine:
a.php->b.php->b.php->b.php->b.php
This doesn't:
a.php->b.php->b.php->b.php->b.php->b.php
As a matter of fact it crashes my server (not responding to ANY http requests anymore), and only restarting Apache "deblocks" it.
The same happens when I load the working "circuit" a.php->b.php->b.php->b.php->b.php different times from the browser.
Error in the Apache error log:
failed to open stream: HTTP request failed!
I thought it might be related to the POST size being too large, but sending a HUGE POST request through the a.php->b.php->b.php->b.php->b.php circuit works just fine ....
So it looks like somehow only 5 simultaneous file_get_contents are allowed ...
Anyone's got some ideas ?
EDIT: As mentioned below, it looks like the real problem is a deadlock, which will not happen in PRD since there will be no "loop" on the same server ... I solved this issue by using CURL instead with a timeout. When a deadlock risks to occur, the curl requests simply time out without freezing the server.
I'm still interested however to get an answer to this question: How can I check/reconfigure the amount of simultaneous requests in Apache2? It's not in the conf file afaik.
Thanks !!!
I would suggest using CURL instead of file_get_contents in this case - although I've never had an issue with this except(!) when using sessions since they lock the process up till the session closes.
Related
I am running Windows Server and it hosts my PHP files.
I am using "file_get_contents()" to call another PHP script and return the results. (I have also tried cURL with the same result)
This works fine. However if I execute my script, then re execute it almost straight away, I get an error:
"Warning: file_get_contents(http://...x.php): failed to open stream: HTTP request failed!"
So this works fine if I leave a minute or two between calling this PHP file via the browser. But after a successful attempt, if I retry too quickly, then it fails. I have even changed the URL in the line "$html = file_get_contents($url, false, $context);" to an empty file that simply prints out a line, and the HTTP stream still doesn't open.
What could be preventing me to open a new HTTP stream?
I suspect my server is blocking further outgoing streams but cannot find out where this would be configured in IIS.
Any help on this problem would be much appreciated.
**EDIT: ** In the script, I am calling a Java file that takes around 1.5 mins, and it is after this that I then call the PHP script that fails.
Also, when it fails, the page hangs for quite some time. During this time, if I open another connection to the initial PHP page then the previous page (still hanging) then completes. It seems like a connection timeout somewhere.
I have set the timeout appropriately in IIS Manager and in PHP
The URL in question : http://www.roblox.com/asset/?id=149996624
When accessed in a browser, it will correctly download a file (which is an XML document). I wanted to get the file in php, and simply display its contents on a page.
$contents = file_get_contents("http://www.roblox.com/asset/?id=149996624");
The above is what I've tried using (as far as I know, the page does not expect any headers). I get a 500 HTTP error. However, in Python, the following code works and I receive the file.
r = requests.get("http://www.roblox.com/asset/?id=147781188")
I'm confused as to what the distinction is between how these two requests are sent. I am almost 100% it is not a header problem. I've also tried the cURL library in PHP to no avail. Nothing I've tried in PHP seems to succeed with the URL (with any valid id parameter); but Python is able to bring success nonchalantly.
Any insight as to why this issue may be happening would be great.
EDIT : I have already tried copying Python's headers into my PHP request.
EDIT2 : It also appears that there are two requests happening upon navigating to the link.
Is this on a linux/mac host by chance? If so you could use ngrep to see the differences on the request themselves on the wire. Something like the following should work
ngrep -t '^(GET) ' 'src host 127.0.0.1 and tcp and dst port 80'
EDIT - The problem is that your server is responding with a 302 and the PHP library is not following it automatically. Cheers!
I tried using curl to post to a local file and it fails. Can it be done? my two management systems are on the same server and it seems unnecessary to have it traverse the entire internet system just to go to a file on the same hard drive.
Using localhost didn't do the trick.
I also tried to $_SERVER[DOCUMENT_ROOT].'/dir/to/file.php' using post data. It's for an API that is encrypted, so I'm not sure exactly how it works. It's for a billing system I have and I just realized that it sends data back (API).
It's simply post data and an XML response. I could write an html form tag and input fields and get the same result, but there isn't really anything else to know.
The main question is: Does curl have the ability to post to a local file or not?
it is post data. it's for an API that is encrypted so i'm not sure exactly how it works
Without further details nobody can answer then what you should do.
But if it's indeed a POST receival script on the local server, then you can send a POST request to it using the URL:
$url = "https://$_SERVER[SERVER_NAME]/path/to/api.php";
And then receive its output from the cURL call.
$data = curl($url)->post(1)->postdata(array("billing"=>1234345))
->returntransfer(1)->exec();
// (you would use the cumbersome curl_setopt() calls instead)
So you get a XML or JSON or whatever response.
If they're on the same drive, then use file operations instead:
file_put_contents('/path/to/the/file', $contents);
Using CURL should only be done if you absolutely NEED the http layer to get involved for some reason, or if you're dealing with a remote server. Using HTTP would also mean you need to have the 'target' script be able to handle a file upload plus whatever other data you need to send, and then that script would end up having to do file operations ANYWAYS, so in effect you've gone on a round-the-world flight just so you can move from your living room to the kitchen.
file://locafilespec.ext worked for me. I had 2 files in the same folder on a linux box, in a folder that is not served by my webserver, and I used the file:// wrapper to post to file://test.php and it worked great. it's not pretty, but it'll work for dev until I move it to it's final resting place.
Does curl have the ability to post to a local file or not?
To curl local file, you need to setup HTTP server as file:// won't work, so:
npm install http-server -g
Then run the HTTP server in the folder where is the file:
$ http-server
See: Using node.js as a simple web server.
Then test the curl request from the command-line to localhost like:
curl http://127.0.0.1:8081/file.html
Then you can do the same in PHP.
I am using a cURL based php application to make requests to another webserver that does asynchronous requests. So what I am doing is creating files with the name as .req with the info I will need on the return and as the identification in the request. The requests are done using HTTP-XML-POST. The file is written using: -
file_get_contents(reqs/<databaseid>.req, FILE_APPEND);
What happens is that while the requests are being generated in bulk (about 1500 per second), the responses start coming back from the webserver. The response is caught by a another script which received the from the response and opens the request file based on it using: -
$aResponse = file(reqs/<databaseid>.req);
Now what happens is that in about 15% of requests, the file() request fails and generates a log entry in apache log like this: -
file(reqs/<databaseid>.req): failed to open stream: No such file or directory in <scriptname> on line <xyz>
It has been verified using a cleaner script that runs later that the file did exist.
Any ideas?!!!
There are some functions to handle simultaneous file access such as flock() but it's normally easier to simply use a database. Any decent DBMS has already worked it out for you.
I'm attempting to use curl inside php to grab a page from my own web server. The page is pretty simple, just has some plain text output. However, it returns 'null'. I can successfully retrieve other pages on other domains and on my own server with it. I can see it in the browser just fine, and I can grab it with command line wget just fine, it's just that when I try to grab that one particular page with curl, it simply comes up null. We can't use file_get_contents because our host has it disabled.
Why in the world would this be different behavior be happening?
Found the issue. I was putting my url someplace that was not in curl_init(), and that place was truncating the query string. Once I moved it back to curl_init, it worked.
Try setting curl's user agent. Sometimes hosts will block "bots" by blocking things like wget or curl - but usually they do this just by examining the user agent.
You should check the output of curl_error() and also take a look at your logfiles for the http server.