I have a python script that parses a large set of data into an internal memory structure, and implements various fetch functions on the structure.
I want to built a simple web frontend for this script, with the condition that the data is only initialized/loaded once (since re-loading upon each fetch would consume too much time/resources). Essentially, the python handler needs to maintain its state between calls, so the data structure is preserved in memory.
Note: PHP's exec() or similar will not work, as this instantiates a new python handler per request. I have heard vague references to using mod_python for this purpose?
I have implemented a solution to a very similar problem. My solution was to use an xmlrpc server, specifically
twisted.web.xmlrpc
I have a method that allows for injection of new data, and have methods for retrieving the data.
Use a persistent server like CherryPy or Twisted Web. All requests will be served by the same process.
Related
Imagine I have a feature coded on nodejs because it implements a library which solves pretty well my problem and I want to add this feature on another PHP script that makes more stuff.
Is a good practice just call the nodejs script with shell_exec? Or should I code a standard communication protocol between them like HTTP requests? Am I coding too much instead of just run shell_exec?
The choice to call the script directly from the PHP code or via a http request is completely at your own discretion. However, you should measure the cost effect performance-wise.
If calling the script directly will not compromise the performance of the PHP code, then it is perfectly fine. You should also consider the long run effect of this. If the data you're processing will grow over time, then depending on you algorithm, and the Big-O effect of the data size, then this won't be an effective method.
The all round best approach will be to go the path of the micro-services concept. You could create an external API that will execute the script externally (with your data), and import the result back into the PHP process. The only drawback from this is the cost, especially if this will require you to acquire other CPU resource(s).
I am relatively new to node.js and socket.io. Currently I have a half finished private web project, which runs only with PHP with a MySQL database on the server side. I decided to bring it to a more advanced level using socket.io, for several features within the project.
So I read a lot about it and watched a whole bunch of tutorials. Also I found this and this during my research.
My question is, if that is still the common way to develop a web application?
More exactly: to use on one event (like a form submit) both an AJAX request and a socket.emit, for those events it is necessary/wanted.
The background of this thought is the following. I have a whole bunch of calculations running now in PHP. And the node.js server runs logically in JavaScript. So I can easily implement a node.js server without changing anything on my AJAX requests. Or rewrite everything I have so far, to js and use only a node.js server.
But this leads to 3 more questions:
Which runs possibly faster on the server side. A calculation scripted with PHP or JavaScript?
How to use transactions on a node.js server while using MySQL?
And how great is the influence by converting a PHP array to a JSON object, what you could avoid with the usage of just the node.js server where you don't need to convert anything.
JavaScript is executed on the client side so you are limited by the user's hardware whereas PHP is executed on your server. See this post for more info about performance comparaison.
I highly suggest you take a look at this pure node.js client that will perfectly do the job in your case.
PHP has many functions to use on JSON data (json_decode(), json_encode(), ...) but Node.js don't require JSON data to be converted. In the end, it really depend on your usage and how you plan to store and use that data
I have been looking at the node.js application vs php, I found many comparisons comparing these two server technique. Mostly people suggest that the javascript V8 engine is much faster than php in terms of running speed of a single file calculation.
I worked on some javascript code for Node.js, now I have this idea I don't know if it is correct.
In my opinion, Node.js runs a javascript application and listen on a port. so this javascript application is an application that is running on server computer. Therefore, the application code is all copied in the memory of the computer. stuff like global variables are declared and saved at the beginning when node.js execute this program. So any new request come in, the server can use these variables very efficiently.
In php, however, the php program execute *.php file based on request. Therefore, if some request is for www.xx.com/index.php, then the program will execute index.php, and in which, there may be stuff like
require("globalVariables.php");
then, php.exe would go there and declare these variables again. same idea for functions and other objects...
So am I correct in thinking that php may not be a good idea when there are many other libraries that need to be included?
I have searched for the comparison, but nobody have talked about this.
Thanks
You are comparing different things. PHP depends on Apache or nginx (for example) to serve scripts, while Node.js is a complete server itself.
That's a big difference, cause when you load a php page, Apache will spawn a thread and run the script there. In Node all requests are served by the Node.js unique thread.
So, php and Node.js are different things, but regarding your concern: yes, you can mantain a global context in Node that will be loaded in memory all the time. On the other hand PHP loads, runs and exits all the time. But that's not the typical use case, Node.js web applications have templates, that have to be loaded and parsed, database calls, files... the real difference is the way Node.js handles heavy tasks: a single thread for javascript, an event queue, and external threads for filesystem, network and all that slow stuff. Traditional servers spawn threads for each connection, that's a very different approach.
Here is my question: Consider Django or web2py in Python (as web frameworks) or Java WEB applications (being simple servlets apps or complex struts2/wicket/whatever frameworks). They share at least two things I like:
There's a Context environment or a way to access data out of the request or session contexts (i.e. global data, singletones, pools ... anything that can share in-memory values and behavior).
Classes are loaded/initialized ONCE. Perhaps i'm missing something but AFAIK in PHP a class is loaded and initialized in a PER REQUEST basis (so, in a regular class, if I (e.g.) modify a static value, this will live only in the current request, and even a simultaneous request hitting that value will get a different one).
Is there a way to get that in php? e.g. in Python/Django i can declare a regular class and that class can hold static data or be a true singleton (again: perhaps a pool or a kind of central queue manager), and will be the same object until the django server dies (note: modules in python are kept loaded in the python context when imported).
The fact that PHP's "context" lives on a per-request basis is pretty much core to how the language works with web servers.
If you want to get it working more like Java or other languages where the data doesn't get reset every request, you basically have two options:
Serialize data into a file, DB, whatever, and reload it on the next request
Instead of serving your pages through a web server, write the server using PHP
Serializing data into storage and reloading it on subsequent requests is the typical approach.
Writing a server in PHP itself, while possible, is not something I would recommend. Despite much effort, PHP still has sort of bad memory management, and you are very likely to encounter memory leaks in long-running PHP processes.
I'm assuming that for every page request, the webserver (eg. Apache) creates a new instance of a script in memory. Can these instances communicate with each other while running? and pass data too?
If you want to pass data between scripts in PHP I suggest using either memcached or a database. Or possibly APC.
If the scripts belong to the same session, they could theoretically communicate via the session but this would be effectively a one-way communication in most cases because only one script can access the session at any one time (session_start() locks the session until that script ends the session implicitly or explicitly).
I believe Martin and Cletus' suggestions are valid. My choice would be function of the end goal of the script.
How much data will you be throwing around? Can you handle the overhead of an external process?
What kind of data are you exchanging? Is it normalized? Or is it now worth normalizing?
Will you need to refer to that data later on? Or can it be discarded after being processed?
Will those scripts ever run on different servers?
Flat files, with a locking mechanism
Relational DB
Document DB (key/value store, whether persistent or not)
Shared memory (APC, or core functions)
Message queues (Active MQ and company)
I think you'll get the most value by externalizing the process as you can have more than one machine managing the messages/data and more than one producing/consuming them.
The model that PHP scripts operate off of doesn't really contain the notion of any kind of persistence in memory for those scripts, since generally they're designed to only run for the minimum of time required to serve the requested page. This would make it hard to have any meaningful use for stateful communication between those scripts, since typically once the page is served there's nothing more for the script to do. Thus usually any communication between PHP scripts is done more through manipulation of database entries and the like.
If you have some sort of continual processing that should be happening for which you'd want to be passing data around, you might want to look into other web application models such as servlets.
You should be able to do this with some shared memory, as described here: http://blog.taragana.com/index.php/archive/how-to-use-shared-memory-in-php/ (assuming you're not running on Windows)