I'm having some trouble using gae php as a simple proxy using "file_get_contents"
When i load a file for the first time I get the latest version available.
But if I change the content of the file, I dont get the latest version immediately.
$result = file_get_contents('http://example.com/'.$url);
The temporary solution I found was to add a random variable at the end of the query string, which allowed me to get a fresh version of the file every time :
$result = file_get_contents('http://example.com/'.$url.'?r=' . rand(0, 9999));
But this trick doesn't work for api calls with parameters for example.
I tried disabling APC cache in the php.ini of gae (using apc.enabled = "0") and i used clearstatcache(); in my script, but neither work.
Any ideas ?
Thanks.
As described in the appengine documentation the http stream wrapper uses urlfetch. As seen in another question urlfetch provides a public/shared cache and as such does not allow individual apps to clear it. For your own services you can set the HTTP cache headers to reduce or void the cache as necessary.
Additionally, you can also add HTTP request headers indicating the maximum age of data that is allowed to be returned. The python example given in mailing list thread is:
result = urlfetch.fetch(url, headers = {'Cache-Control' : 'max-age=300'})
Per php.net file_get_contents http header example and HTTP header documentation a modified example would be:
<?php
$opts = [
'http' => [
'method' => 'GET',
'header' => "Cache-Control: max-age=60\r\n",
],
];
$context = stream_context_create($opts);
$file = file_get_contents('http://www.example.com/', false, $context);
?>
Related
I am using PHP to get JSON from a remote server via file_get_contents command. Here is the piece of code I used:
$opts = array(
'https'=>array(
'method'=>'GET',
'header'=>'Accept-language: en\r\n' .
'Authorization: MAC ["3","ios2.5.0","123","123abc","123=","abc="]\r\n' .
'User-Agent: abc/1.1.1 iOS/10.0.2 iPhone/iPhone7,1\r\n'
)
);
$context = stream_context_create($opts);
$file = file_get_contents('https://www.google.com/v11/file?search=ios&with=users%2Cfiles%2Cquestions', false, $context);
echo $file;
I did a quick debugging:
Using Postman I was able to get the json file with the same header.
I tried a different json from a different url, it works.
I tried a local file, it works.
You have to understand what file_get_contents is. This command is a request to get the file on the server, in this case it is requesting to get https://www.google.com/v11/file/index.html on the server as in one single step. Since your url seems to use header to verify your origin, it might be an ajax request, meaning the server components didn't set up to allow an output from file_get_contents requests, instead they probably accept cURL requests.
So you can use:
curl_exec()
I'm trying to use a web service REST API for which I need to add a parameter for authorization (with the appropriate key, of course) to get a XML result. I'm developing in PHP. How can I add a parameter to the request header in such a situation?
Edit: The way I'm doing the request right now is $xml = simplexml_load_file($query_string);
Are you using curl? (recommended)
I assume that you are using curl to do these requests towards the REST API, if you aren't; use it.
When using curl you can add a custom header by calling curl_setopt with the appropriate parameters, such as in below.
curl_setopt (
$curl_handle, CURLOPT_HTTPHEADER,
array ('Authentication-Key: foobar')
); // make curl send a HTTP header named 'Authentication-key'
// with the value 'foobar'
Documentation:
PHP: cURL - Manual
PHP: curl_setopt - Manual
Are you using file_get_contents or similar?
This method is not recommended, though it is functional.
Note: allow_url_fopen needs to be enabled for file_get_contents to be able to access resources over HTTP.
If you'd like to add a custom header to such request you'll need to create yourself a valid stream context, as in the below snippet:
$context_options = array(
'http' =>array (
'method' => 'GET',
'header' => 'Authentication-Key'
)
);
$context = stream_context_create ($context_options);
$response = file_get_contents (
'http://www.stackoverflow.com', false, $context_options
);
Documentation:
PHP: file_get_contents - Manual
PHP: stream_context_create - Manual
PHP: Runtime Configuration, allow_url_fopen
I'm using neither of the above solutions, what should I do?
[Post OP EDIT]
My recommendation is to fetch the data using curl and then pass it off to the parser in question when all the data is received. Separate data fetching from the processing of the returned data.
[/Post OP EDIT]
When you use $xml = simplexml_load_file($query_string);, the PHP interpreter invokes it's wrapper over fopen to open the contents of a file located at $query_string. If $query_string is a remote file, the PHP interpreter opens a stream to that remote URL and retrieves the contents of the file there (if the HTTP response code 200 OK). It uses the default stream context to do that.
There is a way to alter the headers sent by altering that stream context, however, in most cases, this is a bad idea. You're relying on PHP to always open all files, local or remote, using a function that was meant to take a local file name only. Not only is it a security problem but it also could be the source of a bug that is very hard to track down.
Instead, consider splitting the loading of the remote content using cURL (checking the returned HTTP status code and other sanity checks) and then parsing that content into a SimpleXMLElement object to use. When you use cURL, you can set any headers you want to send with the request by invoking something similar to curl_setopt($ch, CURLOPT_HTTPHEADER, array('HeaderName' => 'value');
Hope this helps.
I have been made aware of the Accept-Range header.
I have a URL that I am calling that always returns a 2mb file. I don't need this much and only need the last section 20-50k.
I am not sure how to go about using it? Would I need to use cURL? I am currently using file_get_contents().
Would someone be able to provide me with an example / tutorial?
Thanks.
EDIT: If this isn't possible then what is post on about? Here ...
EDIT: Ulrika! I'm not insane.
This is possible using the Range header, provided the server supports it. See the HTTP 1.1 spec. You would want to send a header in the following format in your request:
Range: bytes=-50000
This would give you the last 50,000 bytes. Adjust to whatever you need.
You can specify this header in file_get_contents using a context. For example:
// Create a stream
$opts = array(
'http'=>array(
'method' => "GET",
'header' => "Range: bytes=-50000\r\n"
)
);
$context = stream_context_create($opts);
// Open the file using the HTTP headers set above
$file = file_get_contents('http://www.example.com/', false, $context);
If you were to file_get_contents() and dump that to a passthrough 'cache' file on disk, then you could use the unix/linux tail -c to only grab back the last 20kb or so. This doesn't mitigate the actual transfer, but gets that 20kb into the application.
This is indeed possible - see this question for an example of the HTTP headers sent and received
you can't do that. You're going to have to load the entire file (which is sent in its entirety, sequentially, by the source server), and just discard most of it.
What you're asking is like "I'm tuning to this radio station on my car stereo and I only want to hear the last 5 minutes of the show, without having to wait for the rest to complete or change channels".
I am using PHP with the Amazon Payments web service. I'm having problems with some of my requests. Amazon is returning an error as it should, however the way it goes about it is giving me problems.
Amazon returns XML data with a message about the error, but it also throws an HTTP 400 (or even 404 sometimes). This makes file_get_contents() throw an error right away and I have no way to get the content. I've tried using cURL also, but never got it to give me back a response.
I really need a way to get the XML returned regardless of HTTP status code. It has an important "message" element that gives me clues as to why my billing requests are failing.
Does anyone have a cURL example or otherwise that will allow me to do this? All my requests currently use file_get_contents() but I am not opposed to changing them. Everyone else seems to think cURL is the "right" way.
You have to define custom stream context (3rd argument of function file_get_contents) with ignore_errors option on.
As a follow-up to DoubleThink's post, here is a working example:
$url = 'http://whatever.com';
//Set stream options
$opts = array(
'http' => array('ignore_errors' => true)
);
//Create the stream context
$context = stream_context_create($opts);
//Open the file using the defined context
$file = file_get_contents($url, false, $context);
I need a function that I can use in my script to contact another script to send it some GET data. But I need to be able to set a timeout so that it only loads for a few seconds, then continues with the rest of the script. I know I could easily use cURL to do this, but I'd like to know if there are any alternatives?
You can specify a timeout for the standard file access functions (like file_get_contents()) using stream_context_create():
<?php
$opts = array(
'http'=>array(
'method'=>"GET",
'timeout' => 5
)
);
$context = stream_context_create($opts);
$fp = fopen('http://www.example.com', 'r', false, $context);
fpassthru($fp);
fclose($fp);
?>
See the list of context options for an explanation on the timeout option.
This requires, of course, that you can access external URLs using fopen() and consorts.
The nice thing about curl, is it lets you uses threads even though php doesn't support them. So you can make the call to curl_multi, give it a callback, and let the rest of the script run. This way your regular processing isn't blocked. This reduces the need for a short timeout.