file_get_contents fails, but POST request is made successfully - php

spoiler* convoluted code ahead
Context
main.php has a command-line like input, the file's form action attribute is submit.php. submit.php decides what other file to call, based on the input from main.php. In submit.php I adjust variables $url and $data, based on the input, then call file_get_contents with these parameters.
With post requests that do not have session headers I do not have any problem, but once I needed these and tried sending them, I hit a roadblock:
the function file_get_contents itself fails.
the request made by the function is completed successfully
Example and Symptoms
I hit submit on main.php to call submit.php with parameters to make a post request for list/access/index.php:
Server hangs until timeout
On timeout I get the following output.
Warning: file_get_contents(http://localhost/example/list/access/index.php): failed to open stream: HTTP request failed! in /Users/josetaveras/Sites/example/php_toolbox/toolbox.php on line 83
Warning: Cannot modify header information - headers already sent by (output started at /Users/josetaveras/Sites/example/php_toolbox/toolbox.php:83) in /Users/josetaveras/Sites/example/list/submit.php on line 373
When I go back to my main.php I know the post request is completed successfully because the request made in list/access/index.php simply sets a session variable. This variable is visible when going back to main.php.
Relevant code
submit.php
$result = post_request($url,$data,$headers);
header(
'Location: '
.$root_folder_path
.'list/?h='
.$hash
);}
// after post_request we get back to main.php with simple header redirect
toolbox.php
/*
* recursive merging 2 arrays
*/
function rec_array_merge($ar1,$ar2){
foreach($ar2 as $k=>$v){
if(is_array($v)){
if(isset($ar1[$k])){
$ar1[$k] = rec_array_merge($ar1[$k],$v);
}else{
$ar1[$k]=$v;
}
}else{
$ar1[$k]=$v;
}
}
return $ar1;
}
function serialize_array($array,$map_symbol,$separator_symbol){
$returnstr = "";
foreach($array as $k=>$v){
$returnstr.=$k.$map_symbol.$v.$separator_symbol;
}
return $returnstr;
}
function post_request($url,$data,$headers=NULL){
$post_content = http_build_query($data);
$default_headers = Array(
"Content-type"=>"application/x-www-form-urlencoded"
,"Content-Length"=>strlen($post_content)
);
if(isset($headers)){
try{
$headers = rec_array_merge($default_headers,$headers);
}
catch(Exception $e){
print_r($e);
}
}else{
$headers = $default_headers;
}
$serialized = serialize_array($headers,": ","\r\n");
$options = array(
'http' => array(
'header' => $serialized
,'method' => 'POST'
,'content' => $post_content
,'timeout' => 1 // for debugging purposes
)
);
$context = stream_context_create($options);
$result = file_get_contents($url, false, $context); // line 83 on toolbox.php
return $result;
}
References
I used How do I send a POST request with PHP? to construct my post_request.
Final thoughts
It is redundant to say that I am not so deft with PHP, and I know and understand that I have bad habits. This leads to a little bit of disorder in the code and many poor choices. My impression is that overall this pattern feels fragile. I'm not trying to reinvent the wheel, but I am pointedly avoiding libraries and packages that facilitate any part of this. I understand that a routing package might help me with all of this.
I appreciate any feedback you may have on this question. All criticism is welcome and I will try to answer as soon as I can but I have limited connectivity and access to my computing devices. Having said all of that, thank you for reading up to this point.

After a few months I finally found the reason why this was happening.
The worst aspect of this is that I am familiar with error_reporting(E_ALL) and ini('error_notice',1) but I completely forgot about it. I did learn about var_dump($http_response_header) so there is that.
Honestly it was in the documentation and I should have found it sooner. Oh well.
I quote myself
With post requests that do not have session headers I do not have any problem, but once I needed these and tried sending them, I hit a roadblock
I was not familiar exactly with how session mechanics work in general, so i mostly ignored the documentation until I found it a few minutes ago. I refer to session_write_close as what I should have read before diving into sessions (or more specifically, attempting to open multiple sessions in a chain of requests (that I thought of as a single request)):
session data is locked to prevent concurrent writes only one script may operate on a session at any time
In my submit.php I open the session to access some variables. I basically forgot to close it before making a new request which itself attempted to modify session data. The fix was to use session_write_close() right before post_request().
Thank you to all who commented.

Related

How do you echo a SQL SELECT statement from a PHP file called by AJAX?

There's a lot of code in each file, too much to post, so I'm giving you a general idea of what's happening in each file.
index.php
[html dropdown menu code etc.]
scripts.js
[AJAX detects user selection from dropdown, grabs fetch.php which pulls database to generate html code for secondary dropdown selections to put in index.php]
fetch.php
[Generates secondary dropdown code based on user selection and query of the database]
I need to see what exactly is being queried to debug, so I'd like to echo the sql select statement:
$query = "SELECT * FROM databasename WHERE.."
That is in fetch.php when user makes a selection from index.php - How do I do this?
When I deal with AJAX, that I return as JSON, one trick I use is to take advantage of output buffering. You can't just echo or output anything you want because it will mess up the JSON data so for an example,
ob_start(); //turn on buffering at beginning of script.
.... other code ...
print_r($somevar);
.... other code ...
$debug = ob_get_clean(); //put output in a var
$data['debug'] = $debug;
header('Content-Type: application/json');
echo json_encode($data); //echo JSON data.
What this does, is wrap any output from you script into your JSON data so that it's format is not messed up.
Then on the javascript side you can use console.log
$.post(url, input, function(data){
if(data.debug) console.log(data.debug);
});
If you are not used to debugging with console.log(), you can usually hit F12 and open the debugger in most browsers. Then in there the output will be sent to the "console". IE9 had a bit of an issue with console.log() if I recall, but I don't want to go to far off track.
NOTE: Just make sure to not leave this stuff in the code when you move it to production, its very simple to just comment this line out,
//$data['debug'] = $debug;
And then your debug information wont be exposed in production. There are other ways to automatically do this, but it depends on if you do development local then publish to the server. For example you can switch it on the $_SERVER['SERVER_ADDR']; which will be ::1 or 127.0.0.1 when it's local. This has a few drawbacks, mainly the server address is not available from the Command Line Interface (CLI). So typically I will tie it into a global constant that says what "mode" the site is in (included in the common entry point, typically index.php).
if(!defined('ENV_DEVELOPMENT')) define('ENV_DEVELOPMENT','DEVELOPMENT');
if(!defined('ENV_PRODUCTION')) define('ENV_PRODUCTION','PRODUCTION');
if(!defined('ENVIRONMENT')) define('ENVIRONMENT',ENV_DEVELOPMENT);
//site is in Development mode, uncomment for production
//if(!defined('ENVIRONMENT')) define('ENVIRONMENT',ENV_DEVELOPMENT);
Then it is a simple matter to check it:
if(ENVIRONMENT == ENV_PRODUCTION ) $data['debug'] = $debug;
If you know how to use error reporting you can even tie into that using
if(ini_get('display_errors') == 1) $data['debug'] = $debug;
Which will only show the debug when display errors is on.
Hope that helps.
UPDATE
Because I mentioned it in the comments, here is an example of it wrapped in a class (this is a simplified version, so I didn't test this)
class LibAjax{
public static function respond($callback, $options=0, $depth=32){
$result = ['userdata' => [
'debug' => false,
'error' => false
]];
ob_start();
try{
if(!is_callable($callback)){
//I have better exception in mine, this is just more portable
throw new Exception('Callback is not callable');
}
$callback($result);
}catch(\Exception $e){
//example 'Exception[code:401]'
$result['userdata']['error'] = get_class($e).'[code:'.$e->getCode().']';
//if(ENVIRONMENT == ENV_DEVELOPMENT){
//prevents leaking data in production
$result['userdata']['error'] .= ' '.$e->getMessage();
$result['userdata']['error'] .= PHP_EOL.$e->getTraceAsString();
//}
}
$debug = '';
for($i=0; $i < ob_get_level(); $i++){
//clear any nested output buffers
$debug .= ob_get_clean();
}
//if(ENVIRONMENT == ENV_DEVELPMENT){
//prevents leaking data in production
$result['userdata']['debug'] = $debug;
//}
header('Content-Type: application/json');
echo self::jsonEncode($result, $options, $depth);
}
public static function jsonEncode($result, $options=0, $depth=32){
$json = json_encode($result, $options, $depth);
if(JSON_ERROR_NONE !== json_last_error()){
//debug is not passed in this case, because you cannot be sure that, that was not what caused the error. Such as non-valid UTF-8 in the debug string, depth limit, etc...
$json = json_encode(['userdata' => [
'debug' => false,
'error' => json_last_error_msg()
]],$options);
}
return $json;
}
}
Then when you make a AJAX response you just wrap it like this (note $result is pass by reference, this way we don't have to do return, and in the case of an exception we update $result in "real time" instead of on completion)
LibAjax::respond( function(&$result){
$result['data'] = 'foo';
});
If you need to pass additional data into the closure don't forget you can use the use statement, like this.
$otherdata = 'bar';
LibAjax::respond( function(&$result) use($otherdata){
$result['data'][] = 'foo';
$result['data'][] = $otherdata;
});
Sandbox
This handles catching any output and puts it in debug, if the environment is correct (commented out). Please pleas make sure to implement some kind of protection so that the output is not sent to clients on production, I cant stress that enough. It also catches any exceptions puts it in error. And it also handles the header and encoding.
One big benefit to this is consistent structure to your JSON, you will know (on the client side) that if if(data.userdata.error) then you have an exception on the back end. It gives you one place to tweak your headers, JSON encoding etc...
One note in PHP7 you'll have to or should add the Throwable interface (instead of Exception). If you want to catch Error and Exception classes Or do two catch blocks.
Let's just say I do a lot of AJAX and got sick of re-writing this all the time, my actual class is more extensive then this, but that's the gist of it.
Cheers.
UPDATE1
One thing I had to do for things to display was to parse the data variable before I console.log() it
This is typically because you are not passing the correct header back to the browser. If you send (just before calling json_encode)
header('Content-Type: application/json');
This just lets the browser know what type of data it is getting back. One thing most people forget is that on the web all responses are done in text. Even images or file download and web pages. It's all just text, what makes that text into something special is the Content-Type that the browser thinks it is.
One thing to note about header is you cannot output anything before sending the headers. However this plays well with the code I posted because that code will capture all the output and send it after the header is sent.
I updated the original code to have the header, I had it in the more complex class one I posted later. But if you add that in it should get rid of the need to manually parse the JSON.
One last thing I should mention I do is check if I got JSON back or text, you could still get text in the event that some error occurs before the output buffering is started.
There are 2 ways to do this.
If Data is a string that needs to be parsed
$.post(url, {}, function(data){
if( typeof data == 'string'){
try{
data = $.parseJSON(data);
}catch(err){
data = {userdata : {error : data}};
}
}
if(data.userdata){
if( data.userdata.error){
//...etc.
}
}
//....
}
Or if you have the header and its always JSON, then its a bit simpler
$.post(url, {}, function(data){
if( typeof data == 'string'){
data = {userdata : {error : data}};
}
if(data.userdata){
if( data.userdata.error){
//...etc.
}
}
//....
}
Hope that helps!
UPDATE2
Because this topic comes up a lot, I put a modified version of the above code on my GitHub you can find it here.
https://github.com/ArtisticPhoenix/MISC/blob/master/AjaxWrapper/AjaxWrapper.php
Echo the contents and do a die() or exit; afterwards... then in the Network tab of your browser, start it recording, run the Ajax request (it'll fail) but check the resource/name and then view the Response, and it'll show you what was echo'd in the script
Taken from: Request Monitoring in Chrome
Chrome currently has a solution built in.
Use CTRL+SHIFT+I (or navigate to Current Page Control > Developer > Developer Tools.
In the newer versions of Chrome, click the Wrench icon > Tools > Developer Tools.) to enable the Developer Tools.
From within the developer tools click on the Network button. If it isn't already, enable it for the session or always.
Click the "XHR" sub-button.
Initiate an AJAX call.
You will see items begin to show up in the left column under "Resources".
Click the resource and there are 2 tabs showing the headers and return content.
Other browsers also have a Network tab, but you will need to use what I commented to get the string value of the query.
ArtisticPhoenix solution above is delightful.

SOAP client throws "Error fetching http headers" after first request

I need to make the acquaintance of SOAP, and wrote a simple client connecting to some random web service. (Turns out even finding a working service is a bit of a hassle.)
The code I have so far seems to work - but here's the thing: it only works once every ten seconds.
When I first load the page it shows the result I expect - a var_dump of an object - but when I reload the page right after that, all I see is Error Fetching http headers. Now matter how many times I refresh, it takes around ten seconds until I get the right result again, and then the process repeats - refresh too quickly, get an error.
I can't see what's going on at the HTTP level, and even if I could, I'm not sure I'd be able to draw the right conclusions.
Answers to similar questions posted here include setting the keep_alive option to false, or extending the default_socket_timeout, but neither solution worked for me.
So, long story short: is this an issue on the service's end or a problem I can remedy, and if it's the latter, how?
Here's the code I got so far:
<?php
error_reporting(-1);
ini_set("display_errors", true);
ini_set("max_execution_time", 600);
ini_set('default_socket_timeout', 600);
$wsdl = "http://api.chartlyrics.com/apiv1.asmx?WSDL";
try
{
$client = new SoapClient($wsdl, array(
"keep_alive" => false,
"trace" => true
));
$response = $client->SearchLyricDirect(array(
"artist" => "beatles",
"song" => "norwegian wood"
));
var_dump($response);
}
catch (Exception $e)
{
echo $e->getMessage();
}
?>
Any help would be appreciated. (And as a bonus, if you could enlighten me as to why saving the WSDL locally speeds the process up by 30 seconds, that'd be great as well. I assume it's the DNS lookup that takes so much time?)
As it turns out, the connection to the server as a whole is rather shaky.
I (and a few others I've asked to) had similar issues just trying to open the WSDL file in a browser - it works the first time, but refreshing somehow aborts the connection for a good ten seconds.
Though I really can't say what its problem is, this does strongly suggest that the fault lies with the server, not my client.

Advanced PHP: Configure an onBefore and/or onAfter callback for a cURL handle?

I'm working with the cURL implementation in PHP and leveraging curl_multi_init() and curl_multi_exec() to execute batches of requests in parallel. I've been doing this for a while, and understand this piece of it.
However, the request bodies contain a signature that is calculated with a timestamp. From the moment this signature is generated, I have a limited window of time to make the request before the server will reject the request once it's made. Most of the time this is fine. However, in some cases, I need to do large uploads (5+ GB).
If I batch requests into a pool of 100, 200, 1000, 20000, or anything in-between, and I'm uploading large amounts of data to the server, the initial requests that execute will complete successfully. Later requests, however, won't have started until after the timestamp in the signature expires, so the server rejects those requests out-of-hand.
The current flow I'm using goes something like this:
Do any processing ahead of time.
Add the not-yet-executed cURL handles to the batch.
Let cURL handle executing all of the requests.
Look at the data that came back and parse it all.
I'm interested in finding a way to execute a callback function that can generate a signature on-demand and update the request body at the moment that PHP/cURL goes to execute that particular request. I know that you can bind a callback function to a cURL handle that will execute repeatedly while the request is happening, and you have access to the cURL handle all along the way.
So my question is this: Is there any way to configure an onBefore and/or onAfter callback for a cURL handle? Something that can execute immediately before the cURL executes the request, and then something that can execute immediately after the response comes back so that the response data can be parsed.
I'd like to do something a bit more event oriented, like so:
Add a not-yet-executed cURL handle to the batch, assigning a callback function to execute when cURL (not myself) executes the request (both before and after).
Take the results of the batch request and do whatever I want with the data.
No, this isn't possible with the built in functions of cURL. However, it would be trivial to implement a wrapper around the native functions to do what you want.
For instance, vaguely implementing the Observer pattern:
<?php
class CurlWrapper {
private $ch;
private $listeners;
public function __construct($url) {
$this->ch = curl_init($url);
$this->setopt(CURLOPT_RETURNTRANSFER, true);
}
public function setopt($opt, $value) {
$this->notify('setopt', array('option' => $opt, 'value' => $value));
curl_setopt($this->ch, $opt, $value);
}
public function setopt_array($opts) {
$this->notify('setopt_array', array('options' => $opts));
curl_setopt_array($this->ch, $opts);
}
public function exec() {
$this->notify('beforeExec', array());
$ret = curl_exec($this->ch);
$this->notify('afterExec', array('result' => $ret));
return $ret;
}
public function attachListener($event, $fn) {
if (is_callable($fn)) {
$this->listeners[$event][] = $fn;
}
}
private function notify($event, $data) {
if (isset($this->listeners[$event])) {
foreach ($this->listeners[$event] as $listener) {
$listener($this, $data);
}
}
}
}
$c = new CurlWrapper('http://stackoverflow.com');
$c->setopt(CURLOPT_HTTPGET, true);
$c->attachListener('beforeExec', function($handle, $data) {
echo "before exec\n";
});
$result = $c->exec();
echo strlen($result), "\n";
You can add event listeners (which must be callables) to the object with addListener, and they will automatically be called at the relevant moment.
Obviously you would need to do some more work to this to make it fit your requirements, but it isn't a bad start, I think.
Anything to do with cURL is not advanced PHP. It's "advanced mucking about".
If you have these huge volumes of data going through cURL I would recommend not using cURL at all (actually, I would always recommend not using cURL)
I'd look into a socket implementation. Good ones aren't easy to find, but not that hard to write yourself.
Ok, so you say that the requests are parallelized, I'm not sure exactly what that means, but that's not too important.
As an aside, I'll explain what I mean by Asynchronous. If you open a raw TCP socket, you can call the socket_set_blocking function on the connection, this means that read / write operations don't block. You can take several of these connections and write a small amount of data to each of them in a loop, this way you are sending your requests "at once".
The reason I asked whether you have to wait until the whole message is consumed before the endpoint validates the signature is that even if Curl is sending the requests "all at once", there's always a possibility that the time it takes to upload will mean that the validation fails. Presumably it's slower to upload 2000 requests at once than to upload 5, so you'd expect more failures for the former case? Similarly, if your requests are processing synchronously (i.e. one at a time) then you'll see the same error for the same reason, although in this case it's the later requests that are expected to fail. Maybe you need to think about the data upload rate required to upload a message of a particular size within a particular time frame, then try and calculate an optimum multi-payload size. Perhaps the best approach is the simplest: upload one at a time and calculate the signature just before each upload?
A better approach might be to put the signature in a message header, this way the signature can be read earlier in the upload process.

PHP looking for a neat way to prevent unauthorised users from viewing pages

I'm looking for a way to prevent unauthorised users from viewing pages without, lets say, wrapping everything in an if authed { show page } else { show error}
My website is currently setup like:
index.php
require_once __WEBROOT__ . '/templates/default/header.tmpl';
require_once content('p');
require_once __WEBROOT__ . '/templates/default/footer.tmpl';
content()
function content($GETvar)
{
$content = '';
$root = __WEBROOT__;
$location = 'content';
$files = scanDirRecursive($root . '/content/');
if (isset ($_GET[$GETvar]))
{
$path = str_replace('\\', '/', $_GET[$GETvar]->toHTML());
if (in_array("$root/$location/$path", $files))
{
$content = "$root/$location/$path";
}
else
{
$content = $root . '/templates/default/errors/404.php';
}
}
else
{
$content = __WEBROOT__ . '/content/home.php';
}
return $content;
}
This works nicely. When I was playing around with auth options, I chucked in a 'return' at the top of 'content' page. Which ended up preventing the content page from loading but keeping the template in tact (unlike a die()).
So I was wondering, is this safe? Or is there an error occurring that I'm not seeing...
Use the front controller pattern. Instead of having all your pages as individual PHP files, have a single "point of entry".
Basically, have your index.php file work like index.php?p=foo where foo defines what page to show. This way, all your requests will go through index.php, and you can include all your access checking in a single place. Remember to be careful to not allow including arbitrary files though - a common beginner mistake with this approach.
However, as pointed out, you may wish to research how frameworks like Cake or Zend perform this job.
Require a login page which sets a session variable with, say, the userid. Then on every page, call a function to check for authorization. It could probably be put in the header if it considers both the page and the user.
If no user is logged in, or they aren't allowed for the page, redirect to the login pageā€”it would be nice to add a message saying they can't use the page they requested without logging in.
Logging out should clear the session variables. Also, if there is to be a session timeout, record the timestamp in a session variable at times which reset the timeout.
Why to reinvent the wheel? Every php framework have it's acl module, where you can set security policy with minimal amount of coding. Take a look at cakephp or in google acl framework...
don't do a if logged in do this {} else {complain,} just redirect them to the login page if they aren't identified then die();
I've found it convenient to simply throw an Exception for such things. There are several strategies, but one might involve a scenario like:
function show_content()
{
if( ! $user_is_allowed_to_see_this_content ) {
throw new Exception('This user may not see this content', 403);
}
// Continue on with the content code
}
By default, this will simply error out, but you can use the set_exception_handler() function to define what specifically happens when the exception is thrown. This lets you define the "what to do when stuff goes wrong" logic in a separate place from your content-handling code, which I find makes things tidier.
For example:
function custom_exception_handler( Exception $exception )
{
// Log the Exception
error_log( $exception->getMessage(), 0 );
// Return the generic "we screwed up" http status code
header( "HTTP/1.0 500 Internal Server Error" );
// return some error content
die("We're sorry. Something broke. Please try again.");
}
// Now tell php to use this function to handle un-caught exceptions
set_exception_handler('custom_exception_handler');
It's worth noting that this is a good general-purpose way to handle all logical failure events, and not just authentication failures. File-not-found exceptions, validation exceptions, database query exceptions, demand-throttling exceptions; these can all be handled in the same way and in the same place.

Updating an array before MySQL insert?

I'm receiving an XML via php://input and after using simpleXML to break the elements down into variables and then what I want to do is append an array, or create an array of the variables every 30 seconds or so.
The reason is this script will be getting regular inputs, and rather than doing loads of mySQL updates or inserts, I assume it might be better for efficiency.
So, a couple of questions if anyone has a moment.
1) is there a way to check for a new input on php://input.
2) is there a better way to do this repeat check than sleep function?
3) how do I append/add to an array with these updating variables?
I haven't gone too far yet, so the code isn't useful but if you can forgive me simpleness:-
function input() {
$xml = new SimpleXMLElement($input);
$session_id = $xml->session_id;
$ip = $xml->ip;
$browser = $xml->browser;
store($session_id, $ip, $browser);
}
function store() {
$session_id = array();
$ip = array();
$browser = array();
}
If I understand you correctly, it seems that you are trying to use PHP for a long-running stateful program. I trust you know the following:
PHP programs generally don't run for longer than a few miliseconds, at most a few seconds for the typical web application. Every time a resource is requested from the PHP handler, the parsing begins anew and there is no program state that remains from the previous execution. Being a stateless environment, it is up to you to maintain the state. For this reason, PHP is not made to handle input that changes with time, or to maintain a state.
This being said, the simplest way to append to an array is the follwing:
$myarray[] = "newvalue";
or
$myarray['newkey'] = "newvalue";
To process the stream:
while (!feof($handle)){ $data = fgets($handle, 4096); }

Categories