Much of my work lately has involved expanding and bug fixing ajax actions. But the action lists are fairly unmanageable in size, and since I was not the original author and the comments are sparse, I spend a great deal of time tracing the code paths and trying to figure out which jquery event triggered the action and if it sent the proper data with the request.
Right now the ajax request scripts are basically just about a hundred if-else blocks split up into different files based loosely on their function.
Is there a relevant design pattern or a php idiom to help me better organize the php part of the ajax requests?
I was thinking of maybe making some sort of dispatch interface. (Don't know if it is a good or workable idea.) where I can register actions and somehow indicate what data they require. The dispatcher would then call the function from the appropriate place. Then I could route all ajax requests through a single script and organize my functions however I want. And I can have an overview of what data is required to call a certain action without reading its implementation line by line. I would potentially have access to my server-side class heirarchy from the client side.
Does this sound workable? Is this safe? Are there other ways that might work better? The inspiration for this was basically smalltalk style message passing. My major worry is that I am going to introduce a cross-side request forgery vulnerability or that there is already one present in the code and due to it being difficult to read, I missed it.
I use a RPC-style mechanism to achieve what I think you want.
Disclaimer: I've successfully implemented this scheme in JS+PHP and JS+Python, so it is workable. But it might not be secure. You have to take all appropriate verification steps to make sure it is secure (especially w.r.t. to code/SQL injection and XSS attacks)
The idea is to have a single PHP script that processes the RPC requests, receiving the method name and its argument through both GET and POST, and outputs JSON back to the Javascript side.
For instance, on the client side:
API.rpc('getItemById', 1532, function(item) { console.log(item); });
would write
Object(id=1532,name="foo",whatever="bar")
on the console.
The communication protocol I use is the following:
the client sends an HTTP request to the RPC handler script, using
either GET or POST. The restrictions are that the 'method' must
always be provided in the GET, and that all arguments must be
URL-encoded. Otherwise, all arguments are given as key=value pairs and can be part of the request (GET) or the payload (POST)
the server always responds with an HTTP 200 (otherwise it means that a very nasty thing happened). It responds only with JSON data. The returned object has at least 2 members.
the 'success' member is always there, and indicates if the call succeeded - i.e. that no exception was thrown
if successful, the 'ret' members contains the return value of the function
if an exception was thrown, the 'message' member contains the exception message (I prefer sending the whole backtrace here, but that's certainly not good for sensitive environments)
(1) On the javascript side (assuming jQuery, coding as I think, so this may be buggy):
API = function() {
this.rpc = function(method, args, callback) {
return $.ajax({
url: 'rpcscript.php?method='+encodeURIComponent(args.method),
data: args,
type: 'post', //only the method name is sent as a GET arg
dataType: 'json'
error: function() {
alert('HTTP error !'); // This is e.g. an HTTP 500, or 404
},
success: function(data) {
if (data.success) {
callback(data.ret);
} else {
alert('Server-side error:\n'+data.message);
}
},
});
}
}
You can then add shortcut functions such as syncRPC() to perform synchronous calls, etc.
(2) On the PHP side (slightly modified running code):
class MyAPI
{
function getItemById($id)
{
// Assuming the $db is a database connection returning e.g. an associative array with the result of the SQL query. Note the (int) typecast to secure the query - all defensive measures should be used as usual.
return $db->query("SELECT * FROM item WHERE id = ".(int)$id.";");
}
}
class RemoteProcedureCall
{
function __construct()
{
$this->api = new MyAPI();
}
function serve()
{
header("Content-Type: application/json; charset=utf-8");
try
{
if (!isset($_GET['method']))
throw new Exception("Invalid parameters");
$methodDesc = array($this->api, $_GET['method']);
if (!method_exists($methodDesc[0], $methodDesc[1]) || !is_callable($methodDesc))
throw new Exception("Invalid parameters");
$method = new ReflectionMethod($methodDesc[0], $methodDesc[1]);
$params = array();
foreach ($method->getParameters() as $param)
{
// The arguments of the method must be passed as $_POST, or $_GET
if (isset($_POST[$param->getName()]))
// OK, arg is in $_POST
$paramSrc = $_POST[$param->getName()];
elseif (!in_array($param->getName(),array('action','method'))
&& isset($_GET[$param->getName()])
&& !isset($paramSrc[$param->getName()]))
// 'action' and 'method' are reserved $_GET arguments. Arguments for the RPC method
// can be any other args in the query string, unless they are already in $_POST.
$paramSrc = $_GET[$param->getName()];
if (!isset($paramSrc))
{
// If the argument has a default value (as specified per the PHP declaration
// of the method), we allow the caller to use it - that is, not sending the
// corresponding parameter.
if ($param->isDefaultValueAvailable())
$p = $param->getDefaultValue();
else
throw new Exception("Invalid parameters");
}
else
{
$p = $paramSrc;
}
$params[$param->getName()] = $p;
unset($paramSrc);
}
$ret = $method->invokeArgs($db, $params);
echo json_encode(array('success' => true, 'ret' => $ret));
}
catch (Exception $e)
{
echo json_encode(array('success' => false, 'message' => $e->getMessage()."\n".$e->getBacktrace()));
}
}
};
$rpc = RemoteProcedureCall();
$rpc->serve();
There are many application-specific assumptions here, including the kind of exceptions that may be thrown, the reserved keywords, etc ...
Anyway I hope this provides a good starting point for your problem.
You can have a look here: http://www.phpapi.org/
From description:
"This is the skeleton upon which you can develop a web-system from a simple Web Calculator to the most sofisticated CRM/ERP/CMS/ETC. What PHP-API provides is: a general structure of the code, a very simple extendable API code structure,JavaScript connectivity with the API ( with an easy way of adding new modules/method handlers ), ...."
Related
From what I understand, basically, PHP server-side apps (PHP-FPM) load the entire app from scratch on every request and then close it down at the end of a request. Meaning that variables, containers, config and everything else gets read and built from zero in each separate request and there is no crossover. I can use this knowledge to structure the app better. For example, I would know that class statics hold their data only for the duration of the request and each new request will have its own value.
A Node.js server like Express.js works very differently, however. It is a single Node.js process that is running continually and listens for any new requests and passes them along to the correct handlers. This requires a different approach to development, as there is data that is kept in memory between requests. For example, class statics in such a case sound like they would hold data for the entire duration of the server uptime, not just for the duration of a single request.
So I have some questions about this:
Does it make sense to pre-load some data during Express.js startup (like reading private keys from file) so that it is already in memory when needed by a request and it would get re-used each time without being re-read from file? In a PHP server framework this wouldn't matter that much as everything gets built from 0 with each request.
How do I properly handle exceptions in a Node.js server process? If a PHP server script throws a fatal exception only that specific request dies, all other requests and any new ones run fine. If a fatal error happens in a Node.js server, it sounds like it would kill the entire process and thus all requests with it.
If you have any resources about how this topic, it'd be great if you could share them also.
1-
Does it make sense to pre-load some data during Express.js startup (like reading private keys from file) so that it is already in memory when needed by a request and it would get re-used each time without being re-read from file? In a PHP server framework this wouldn't matter that much as everything gets built from 0 with each request.
Yes, totally. You would bootstrap connections to databases, data read for files and similar tasks at application startup, so they are always available in every request.
There are some things to consider in this scenario:
During application startup, you can safely call synchronous methods, like fs.readFileSync etc, because there are not concurrent request on the single thread at this point.
CommonJS modules does cache their first value exported. So if you choose to use a dedicate module to handle secrets read from a file, database connections etc., you can:
secrets.js
const fs = require('fs');
const gmailSecretApiKey = fs.readFileSync('path_to_file');
const mailgunSecretApiKey = fs.readFileSync('path_to_file');
...
module.exports = {
gmailSecretApiKey,
mailgunSecretApiKey,
...
}
Then require this as your application startup. After this, any modules that does:
const gmailKey = require('.../secrets').gmailSecretApiKey won't read from file again. The results are cached in the module.
This is important because allow you to use require and import for consuming configuration in your controllers and modules, without bothering passing extra parameters to your http controllers or adding them to req objects.
Depending upon infrastructure, you may not be able to allow your application to not handling requests during startup (i.e. you have only one machine up and don't want to give service unavailble to your clients). In such cases, you can expose all the configuration and shared resources in promises, and bootstrap your web controllers as fast as possible, waiting for the promises inside. Let's say we need kafka up and running when handling a request on '/user':
kafka.js
function kafka() {
// return some promise of an object that can publish and read from kafka in a given port etc. etc.
}
module.exports = kafka();
So now in:
userController.js
const kafka = require('.../kafka');
router.get('/user', (req,res) => {
kafka.then(k => {
k.publish(req.user, 'userTopic'); // or whatever. This is just an example.
});
})
In this way, in the event an user makes a request during bootstrap, the request will still be handled (but will take some time). Requests made when the promise is already resolved won't notice anything.
There's no such thing as multiple threads in node. Anything you declare in a commonJS module or you write to process will be available in every request.
2-
How do I properly handle exceptions in a Node.js server process? If a PHP server script throws a fatal exception only that specific request dies, all other requests and any new ones run fine. If a fatal error happens in a Node.js server, it sounds like it would kill the entire process and thus all requests with it.
This really depends in the kind of exception that you find. It is specifically related to the request being processed, or is something critical for the whole application?
In the former case, you want to catch the exception and don't allow the whole thread to die. Now, 'catch the exception' in javascript is tricky, because you cannot catch asynchronous exceptions/errors, and you would likely use process.on('unhandledRejection') to handle that, like:
// main.js
try {
bootstrapMongoDb();
bootstrapKafka();
bootstrapSecrets();
... wahtever
bootstrapExpress();
} catch(e){
// read what `e` brings and decide.
// however, is worth to mention that errors raised during handling
// http request won't ever get handled here, because they are
// asynchronous. try/catch in javascript don't catch asynchronous errors.
}
process.on('unhandledRejection', e => {
// now here we are treating unhandled promise rejections, and errors that raise
// in express controllers are likely end up here. of course, I'm talking about
// promise rejections. I am not sure if this can catch Errors thrown in callbacks.
// You should never `throw new Error` inside an asynchronous callback.
});
Handling errors in node application is a whole topic on its own, too broad to be considered here. However some tips shouldn't do harm:
Never throw errors in callbacks. throw is synchronous. Callbacks and asynchrony should rely on an error parameter or a promise rejection.
You better get used to promises. Promises really improve error management in asynchronous code.
Javascript errors can be decorated with extra fields, so you can fill in trace id's and other id's that may be useful when reading logs of your system, given you will log your unhandled errors.
Now, in the latter case... sometimes there are failures that are totally disastrous for your app. Maybe you totally need a connection to a kafka or a mongo server, and if it is broken, then you may want to kill your application so clients receive a 503 when trying to connect.
Then, in some scenarios, you may want to kill your app, then let another service to reboot it when database is available again. This depends a lot on infrastructure and you may as well not kill your app never.
If you don't have a infrastructure that handles the health and reboot of your web service for you, it is probably safer to never let your application die. Said so, it's a good thing to at least use tools like nodemon or PM2 to ensure your app will relaunch after going down.
Bonus: why you should not throw errors in callbacks
Thrown errors propagates through the callstack. You have, let's say, function A who calls B, who in turn then calls C. Then C throw an Error. All of them only have synchronous code.
In such scenario, error propagates to B and, if it don't catch it, it propagates to A, and so on.
Now let's say that, instead, C doesn't throw an error by itself, but do call fs.readFile(path, callback). In the callback function, an error is thrown.
Here, when the callback is invoked, and the error thrown, A is already done and left the stack long ago, hundreds of milliseconds ago, maybe even more.
This means that any catch block in A won't catch the error, because is not even there already:
function bootstrapTimeout() {
try {
setTimeout(() => {
throw new Error('foo');
console.log('paco');
}, 200);
} catch (e) {
console.log('error trapped!');
}
}
function bootstrapInterval() {
setInterval(() => {
console.log('interval')
}, 50);
}
console.log('start');
bootstrapTimeout();
bootstrapInterval();
If you run that snippet, you would see how the error reach the top level and kill the process, even if the throw new Error('foo'); line was placed within a try/catch block.
error, result interface
Instead of using Errors to handle exceptions in asynchronous code, node.js has the standard behavior of expose an (error, result) interface for every callback you pass to an asynchronous method. If, for instance, fs.readFile happens to go wrong because the filename did not exist, it does not throw an error, it invokes the callback with the corresponding Error as the error parameter.
Like:
fs.readFile('notexists.png', (error, callback) => {
if(error){
// foo
}
else {
http.post('http://something.com', result, (error, callback) => {
if(error){
// oops, something went wrong with an http request
} else {
// keep working
// etc.
// maybe more callbacks, always with the dreadful 'if (error)'...
}
})
}
});
You always control errors in async operations in the callback, you should never throw.
Now this is a pain in the ass. Promises allow for much better error control because you can control async errors in one single catch block:
fsReadFilePromise('something.png')
.then(res => someHttpRequestPromise(res))
.then(httpResponse => someOtherAsyncMethod(httpResponse))
.then(_ => maybeSomeLoggingOrWhatever() )
.catch(e => {
// here you can control any error thrown in the previous chain.
});
And there's also async/await that allow you to mix async and sync code and treat promise rejections in catch blocks:
await function main() {
try {
a(); // some sync code
await b(); // some promise
} catch(e) {
console.log(e); // either an error throw in a() or a promise rejection reason in b();
}
}
However keep in mind that await is no magic and you really need to understand promises and asynchrony well in order to use it properly.
At the end, you always end up with one error control flow for synchronous errors via try/catch, and another for asynchronous errors, via callback parameters or promise rejections.
Callbacks can use try/catch when consuming synchronous api's, but should never throw. Any function can use catch to handle synchronous errors, but cannot rely on catch blocks to handle asynchronous errors. Kinda messy.
Does it make sense to pre-load some data during Express.js startup (like reading private keys from file) so that it is already in memory when needed by a request and it would get re-used each time without being re-read from file?
Yes it make sense if you structure your code to let these data be available in the request handler. In the following example, based on what i know, the staticResponse is readed only one time.
const express = require('express');
const staticResponse = fs.readFileSync('./data');
const app = express();
app.get('/', function (req, res) {
res.json(staticResponse);
});
app.listen(3000, function () {
console.log('Example app listening on port 3000!');
});
How do I properly handle exceptions in a Node.js server process? If a fatal error happens in a Node.js server, it sounds like it would kill the entire process and thus all requests with it.
Exactly, an unhandled exception make the entire nodejs process crash. There are multiple ways to manage error, there isn't 'the one for all' solution. Depends on how you write you're code.
all requests with it => keep in mind that nodejs is single thread.
app.post('/', function (req, res, next) {
try {
const data = JSON.parse(req.body.stringedData);
// use data
res.sendStatus(200);
} catch (err) {
return next(err);
}
});
Background: I've been drafted to maintain and update a flash/flex/as3 front-end to website using php on the backend. I'm getting better at it, but I'm still new to the environment.
I need some processed information received back in the return handler of a POST. I thought it would be clever to do the next POST within the return handler. In other words, I've got something like this:
private function finishThis():void
{
var s:HTTPService = new HTTPService();
rq.action = "do_something";
s.url = "inc/php/test.php";
s.method = "POST";
s.request = rq;
s.requestTimeout = 120000;
s.addEventListener(ResultEvent.RESULT, doSomethingRH);
s.send();
s.disconnect(); // That's a test, it didn't help
}
and then in doSomethingRH(), I've got
private function doSomethingRH(event:ResultEvent):void
{
doSomethingElse();
}
private function doSomethingElse():void
{
var s:HTTPService = new HTTPService();
rq.action = "do_something_else";
s.url = "inc/php/test.php";
s.method = "POST";
s.request = rq;
s.requestTimeout = 120000;
s.addEventListener(ResultEvent.RESULT, doSomethingElseRH);
s.send();
s.disconnect(); // That's new, it wasn't there before
}
All of this works as expected using http://localhost (WAMP). But online, though I have backend indications that function do_something_else runs (so far), but function doSomethingElseRH() never gets called. Instead, I get the error I've seen from several posts :
[RPC Fault faultString="HTTP request error" faultCode="Server.Error.Request" faultDetail="Error: [IOErrorEvent type="ioError" bubbles=false cancelable=false eventPhase=2 text="Error #2032: Stream Error. URL: http://test.net/inc/php/test.php"]. URL: http://test.net/inc/php/test.php"]
The problem pointed out most often is too many simultaneous connections. I don't need to do multiple connections, but thought perhaps I had them since I was starting the new POST in the previous POST return handler. Instead of calling doSomethingElse() directly, I added an event and did a dispatchEvent() to call it, but at least according to the call stack shown in FlashDevelop, the eventual call to doSomethingElse() was under the doSomethingRH() function (still learning this stuff). And all of that is conjecture based on a post I saw about getting the HTTPService instance from within event delivered to the return handler.
My ultimate question is, how can I achieve sequential POSTs like this without the error?
And if I'm getting the error because of the way I'm chaining them together, is there a clean-up function I can do so that when the return handler is called, the connection can be fully closed?
Update - it's fine
A pdf engine was being called within both functions, one to create a base file, the other to append some pages. It was working fine, but was updated recently. Blocking those activities allowed the http flow to work as expected, no disconnect() needed.
The best I can say for this is it's an example of chaining calls together that I haven't seen online, I suppose that's something.
It's possible that you s.disconnect(); too soon (hence PHP file is not found). Why disconnect immediately after .send before that even gets a chance to do anything? File access on hard drive is faster than between web server. If still failing then try adding a delay (ie: disconnect, then start a timer that counts to 3 seconds) before attempting to do doSomethingElse();...
Untested but try :
public var s:HTTPService; //make one and re-use
private function finishThis():void
{
s = new HTTPService();
rq.action = "do_something";
s.url = "inc/php/test.php";
s.method = "POST";
s.request = rq;
s.requestTimeout = 120000;
s.addEventListener(ResultEvent.RESULT, doSomethingRH);
s.send();
}
...then in doSomethingRH() :
private function doSomethingRH(event:ResultEvent):void
{
s.disconnect(); //is now finished so disconnect before re-use
doSomethingElse();
}
private function doSomethingElse():void
{
s = new HTTPService();
rq.action = "do_something_else";
s.url = "inc/php/test.php";
s.method = "POST";
s.request = rq;
s.requestTimeout = 120000;
s.addEventListener(ResultEvent.RESULT, doSomethingElseRH);
s.send();
}
h t t p should be http
relative urls are relative to where the swf is loaded from... i would have a baseUrl variable that you want to be the relative url
I am very very new to PHP and Kohana. Already created a sample/demo "hello World" PHP Kohana application which is running sucessfully in WAMP server.
I want my application to be work as a complete server side component.
Since i'll have only server side logic in this application, it should use a ORM to communicate with my MySQL database.
I'll have a separate client side application which will have UI parts.
So I want my PHP-Kohana should recognize the RestFul webservices call from my client and give the JSON response accordingly.
Is it possible to create a Kohana application which supports RestFul webservices?
If yes, give me a guidance to create the webservices in Kohana application.
Is there any such sample code for demo?
There is no specific demo or sample code that I know of, so hopefully these tips will help to get you started with it...
It is possible, and relatively easy, to accept AJAX requests and produce JSON responses with Kohana. The first thing to be aware of is that unless told otherwise, Kohana will always try to generate the view, and this will fail as a JSON response so first things first:
if ($this->request->is_ajax()) {
// Disable any rendering of the template so it just returns json.
$this->auto_render = FALSE;
}
You'll probably want to put this in the before() method, probably of the parent Controller so that it always runs before you are getting data from the DB.
My personal preference for something like this would be to set up a standard AJAX response array so that the data is always returned in a relatively standard format. Example:
// Standard ajax response array.
$this->ajax_response = array(
'success' => FALSE,
'error' => NULL,
'raw_data' => NULL,
'generated' => ''
);
Customise the above to match your required usage. You'll probably also want this in your before() method.
Now in your action methods, get the data from the DB and add it to the array.
public function action_foobar() {
// Get the primary key ID from the URL.
$var1 = $this->request->param('var1');
$data = ORM::factory('Model', $var1);
if ($data->loaded()) {
$this->ajax_response['success'] = TRUE;
$this->ajax_response['raw_data'] = $data;
} else {
$this->ajax_response['error'] = 'Data could not be found.';
}
}
You should then be able to request this data by calling a URL such as http://www.website.com/controller/foobar/42
The final piece of the puzzle is actually returning this data, as at the moment Kohana won't output anything because we have told it not to. In your after() method, do the following:
if ($this->request->is_ajax()) {
$this->request->headers('Content-Type', 'application/json');
$this->response->body(json_encode($this->ajax_response));
}
Then you're free to interpret that response however you see fit in the jQuery on your client-side application:
$.ajax({
type: "POST",
url: "http://www.website.com/controller/foobar/" + foobarId,
dataType: 'json',
success: function (data) {
if (!data.success) {
alert(data.error);
} else {
// Handle the returned data.
}
},
error: function (xhr, status, errorThrown) {
// Something went very wrong (response failed completely).
alert(errorThrown);
}
});
Good luck with building your app! I hope this helps to at least get you started.
I have a function as defined below:
public function subscribe($someQueue)
{
$callback = function($msg){
return $msg->body;
};
$this->channel->basic_consume( $someQueue, '', FALSE, TRUE, FALSE, FALSE, $callback);
while(count($this->channel->callbacks)) {
$this->channel->wait();
}
}
I'm using the following function:
Note: Following lines are in a different class file, hence creating object of the class that contains the above function.
$objRMQ = new RabbitMQ();
$msgBody = $objRMQ->subscribe("someQueue");
echo "message body returned from someMethod: ".$msgBody;
Basically, I want to return body of every message to the caller function that is published to the queue.
Current output:
message body returned from subscribe: NULL
Expected output:
holla, this is your message from queue
Since this question is old but still unanswered, I'll give a brief explanation. You've probably already figured out the answer by now, but this might help someone else searching in future.
The key concept here is "asynchronous execution".
When you subscribe to a channel using the basic_consume method, you are not asking for the callback to be executed once, immediately, but for it to be executed once a message becomes available, and then every time another message is available.
In the case of AMQPLib, you wait for new messages by repeatedly calling the wait() method; i.e. here:
while(count($this->channel->callbacks)) {
$this->channel->wait();
}
Thinking about this carefully, there are two mistakes in your code:
The line return $msg->body has nowhere to return to. The call will happen somewhere deep in the implementation of the wait() method, and you get no output from $this->channel->wait(), so have no way of doing anything with that returned value.
On the other side, when you call $objRMQ->subscribe("someQueue") from your other class, you are expecting it to return something, but that function has no return statement. The only return statement is inside the anonymous function you passed to basic_consume.
The solution is basically to do all your processing of the message - echo $msg->body, or whatever real processing you want to do - inside the callback. If you really want to gather data as messages come in, you could save it to some variable accessible outside the callback, but remember that you will at some point need to break out of the wait() loop in order to do anything with that data.
I'm working on a RESTful and am stuck on message gathering for returning to the user. Basically, depending on the options selected, a few classes will be included dynamically. I'll try to provide a real-world break down. We have a HTML-email-tempalte maker - depending on the template chosen a php script will be included. This script may have warnings and I need to pass them "upstream" so that the API can report them. So we have something like this ( -> = includes )
API -> HTMLGenerator -> (dynamically) template-script.php
I need the template-script to be able to report errors to the API controller so the API can report them to the API user. Not sure the best way/practice to accomplish this.
So far , my thoughts are maybe a singleton or session variable that the template-script can add messages to, then the API Controller can report them. Any thoughts?
Main API
REST create by POST to /v1/html basically just:
class API {
require($dynamic_script);
$errors = array('warnings'=>array('warning1',waring2'));
//set http header and return JSON
}
HTMLGenerator
class HTMLGenerator {
//basically some wrappers for junior / non-programmers
function addHeading($text) {
//Add a header and do some checks.
if(strlen($text) > $warnTooLong )
HTMLErrors::addWarning("Message");
}
}
Dynamic Script
$h = new HTMLGenerator();
$h->addHeader($text);
$h->addImage($imageUrl);
You need to use a custom error handler.
See this link - http://php.net/manual/en/function.set-error-handler.php
It allows us to handle a error that might be thrown to capture it and process it. So, when you capture it, you can pass this to the parent class and furthur upstream for further processing.
Global object would work, set_error_handler too, but these are just hacks. The cleanest option is to modify your app elements to do what they are suppose to do - return those messages.
These shouldn't be too hard to do:
function myOldFunction($param1, $param2)
{
// do something
}
modify this way:
function myOldFunction($param1, $param2, array &$messages = array())
{
// do something
$messages[] = 'hey mama, i\'m on stack overflow!';
}
usage:
$messages = array();
myOldFunction(1, 2, $messages);
print_r($messages);