Here is my question: Consider Django or web2py in Python (as web frameworks) or Java WEB applications (being simple servlets apps or complex struts2/wicket/whatever frameworks). They share at least two things I like:
There's a Context environment or a way to access data out of the request or session contexts (i.e. global data, singletones, pools ... anything that can share in-memory values and behavior).
Classes are loaded/initialized ONCE. Perhaps i'm missing something but AFAIK in PHP a class is loaded and initialized in a PER REQUEST basis (so, in a regular class, if I (e.g.) modify a static value, this will live only in the current request, and even a simultaneous request hitting that value will get a different one).
Is there a way to get that in php? e.g. in Python/Django i can declare a regular class and that class can hold static data or be a true singleton (again: perhaps a pool or a kind of central queue manager), and will be the same object until the django server dies (note: modules in python are kept loaded in the python context when imported).
The fact that PHP's "context" lives on a per-request basis is pretty much core to how the language works with web servers.
If you want to get it working more like Java or other languages where the data doesn't get reset every request, you basically have two options:
Serialize data into a file, DB, whatever, and reload it on the next request
Instead of serving your pages through a web server, write the server using PHP
Serializing data into storage and reloading it on subsequent requests is the typical approach.
Writing a server in PHP itself, while possible, is not something I would recommend. Despite much effort, PHP still has sort of bad memory management, and you are very likely to encounter memory leaks in long-running PHP processes.
Related
TL;DR: I'm not sure this topic has its place on StackOverflow, but basically it's just a topic of debate and thinking about making PHP apps like we would do with NodeJS for example (stateless request flow, asynchronous calls, etc.)
The situation
We know NodeJS can be used as both a web-server and web-app.
But for PHP, the internal web-server is not recommended for production (so says the documentation).
But, as Symfony full-stack is based on the Kernel which handles Request objects, it means we should be able to send lots of requests to the same kernel, only if we could "bootstrap" the php web-server (not the app) by creating a kernel before listening to HTTP requests. And our router would only create a Request object and make the kernel handle it.
But for this, a Symfony app has to be stateless, for example we need Doctrine to effectively clear its unit of work after a request, or maybe we would need to sort of isolate some components based on a request (By identifying a request with its unique PHP class reference id? Or by using other php processes?), and obviously, we would need more asynchronous things in PHP or in the way we use the internal web-server.
The main questions I sometimes ask myself, and now ask to the community
To clarify this, I have some questions about PHP:
Why exactly is the internal PHP webserver not recommended for production?
I mean, if we can configure how the server is run and its "router" file, we should be able to use it like any PHP server, yes or no?
How does it behaves internally? Is memory shared between two requests?
By using the router, it seems obvious to me that variables are not shared, else we could make nodejs-like apps, but it seems PHP is not capable of doing something like this.
Is it really possible to make a full-stateless application with Symfony?
e.g. I send two different requests to the same kernel object, in this case, is there any possibility that the two requests create a conflict in Symfony core components?
Actually, the idea of "Create a kernel -> start server -> on request, make the kernel handle it" behavior would be awesome, because it would be something quite similar to NodeJS, but actually, the PHP paradigm is not compatible with this because we would need each request to be handled asynchronously. But if a kernel and its container is stateless, then, there should be a way to do something like that, shouldn't it?
Some thoughts
I've heard about React PHP, Ratchet PHP for Websocket integration, Icicle, PHP-PM but never experienced them, it seems a bit too complex to me for now (I may lack some concepts about asynchronicity in apps, that's why my brain won't understand until I have some more answers :D ).
Is there any way that these libraries could be used as "wrappers" for our kernel request handling?
I mean, let's create this reactphp/icicle/whatever environment setup, create our kernel like we would do in any Symfony app, and run the app as web-server, and when a request is retrieved, we send it asynchrously to our kernel, and as long as the kernel has not sent the response, the client waits for it, even if the response is also sent asynchrously (from nested callbacks, etc., like in NodeJS).
This would make any existing Symfony app compatible with this paradigm, as long as the app is stateless, obviously. (if the app config changes based on a request, there's a paradigm issue in the app itself...)
Is it even a possible reality with PHP libraries rather than using PHP internal web-server in another way?
Why ask these questions?
Actually, it would be kind of a revolution if PHP could implement real asynchronous stuff internally, like Javascript has, but this would also has a big impact on performances in PHP, because of persistent data in our web-server, less bootstraping (require autoloader, instantiate kernel, get heavy things from cached files, resolve routing, etc.).
In my thoughts, only the $kernel->handleRaw($request); would consume CPU, the whole rest (container, parameters, services, etc.) would be already in the memory, or, for the case of services, "awaiting to be instantiated". Then, performance boost, I think.
And it may troll a bit the people who still think PHP is a very bad and slow language to use :D
For readers and responders ;)
If a core PHP contributor reads me, is there any way that internally PHP could be more asynchronous even with a specific new internal API based on functions or classes?
I'm not a pro of all of these concepts, and I hope really good experts are going to read this and answer me!
It could be a great advance in the PHP world if all of this was possible in any way.
Why exactly is the internal PHP webserver not recommended for
production? I mean, if we can configure how the server is run and its
"router" file, we should be able to use it like any PHP server, yes or
no?
Because it's not written to behave well under load, and there are no configuration options that let you handle HTTP request processing before it reaches PHP.
Basically, it lacks features if you compare it to nginx. It would be equal to comparing a skateboard to a Lamborghini.
It can get you from A to B but.. you get the gist.
How does it behaves internally? Is memory shared between two requests?
By using the router, it seems obvious to me that variables are not
shared, else we could make nodejs-like apps, but it seems PHP is not
capable of doing something like this.
Documentation states it's singlethreaded, so it appears that it would behave the same as if you wrote while(true) { // all your processing here }.
It's a playtoy designed to quickly check a few things if you can't be bothered to set up a proper web server before trying out your code.
Is it really possible to make a full-stateless application with
Symfony? e.g. I send two different requests to the same kernel object,
in this case, is there any possibility that the two requests create a
conflict in Symfony core components?
Why would it go to the same kernel object? Why not design your app in such a way that it's not relevant which object or even processing server gets the request? Why not design for redundancy and high availability from the get go? HTTP = stateless by default. Your task = make it irrelevant what processes the request. It's not difficult to do so, if you avoid coupling with the actual processing server (example: don't store sessions to local filesystem etc.)
Actually, the idea of "Create a kernel -> start server -> on request,
make the kernel handle it" behavior would be awesome, because it would
be something quite similar to NodeJS, but actually, the PHP paradigm
is not compatible with this because we would need each request to be
handled asynchronously. But if a kernel and its container is
stateless, then, there should be a way to do something like that,
shouldn't it?
Actually, nginx + php-fpm behave almost identical to node.js.
nginx uses a reactor to handle all connections on the same thread. Node.js does the exact same thing. What you do is create a closure / callback that is fed into Node's libraries and I/O is handled in a threaded environment. Multithreading is abstracted from you (related to I/O, not CPU). That's why you can experience that Node.js blocks when it's asked to do a CPU intensive task.
nginx implements the exact same concept, except this callback isn't a closure written in javascript. It's a callback that expects an answer from php-fpm during <timeout> seconds. Nginx takes care of async for you. What your task is is to write what you want in PHP. Now, if you're reading a huge file, then async code in your PHP would make sense, except it's not really needed.
With nginx and sending off requests for processing to a fastcgi worker, scaling becomes trivial. For example, let's assume that 1 PHP machine isn't enough to deal with the amount of requests you're dealing with. No problem, add more machines to nginx's pool.
This is taken from nginx docs:
upstream backend {
server backend1.example.com weight=5;
server backend2.example.com:8080;
server unix:/tmp/backend3;
server backup1.example.com:8080 backup;
server backup2.example.com:8080 backup;
}
server {
location / {
proxy_pass http://backend;
}
}
You define a pool of servers and then assign various weights / proxying options related to balancing how requests are handled.
However, the important part is that you can add more servers to cope with availability requirements.
This is the reason why nginx + php-fpm stack is appealing. Since nginx acts as a proxy, it can proxy requests to node.js as well, letting you handle web socket related operations in node.js (which, in turn, can perform an HTTP request to a PHP endpoint, allowing you to contain your entire app logic in PHP).
I know this answer might not be what you're after, but what I wanted to highlight is the way node.js works (conceptually) is identical to what nginx does when it comes to handling incoming request. You could make php work as node does, but there's no need for that.
Your questions can be summed up as this:
"Could PHP be more like Node?"
to which the answer is of course "Yes." But that leads us to another question:
"Should PHP be more like Node?"
and now the answer is not that obvious.
Of course in theory PHP could be made more like Node - even to a point to make it exactly the same. Just take the next version of Node and call it PHP 6.0 or something.
I would argue that it would be harmful to both Node and PHP. There is a diversity in the runtime environments for a reason. One of the variations is the concurrency model used in a given environment. Making one like the other would mean less choice for the programmer. And less choice is less freedom of expression.
PHP and Node were created in different times and for different reasons.
PHP was developed in 1995 and the name stood for Personal Home Page. The use case was to add some server-side dynamic features to HTML. We already had SSI and CGI at that point but people wanted to be able to inject right into the HTML - synchronously, as it wouldn't make much sense otherwise - results of database queries and other computations. It isn't a surprise how good it is at this job even today.
Node, on the other hand, was developed in 2009 - almost 15 years later - to create high performance network servers. So it shouldn't surprise us that writing such servers in Node is easy and that they have great performance characteristics. This is why Node was created in the first place. One of the choices it had to make was a 100% non-blocking environment of single-threaded, asynchronous event loops.
Now, single-threading concurrency is conceptually more difficult than multi-threading. But if you want performance for I/O-heavy operations then currently you have no other options. You will not be able to create 10,000 threads but you can easily handle 10,000 connections with Node in a single thread. There is a reason why nginx is single-threaded and why Redis is single threaded. And one common characteristic of nginx and Redis is amazing performance - but both of those were hard to write.
Now, as far as Node and PHP go, those technologies are so far from each other that it's hard to even comprehend how their fusion would look like. It reminds me the old April Fool's joke about unifying Perl and Python that so many people believed in.
PHP has its strengths and Node has it strengths. And just like it would be hard to imagine Node with blocking-I/O, it would be equally hard to imagine PHP with non-blocking I/O.
To summarize: it could be possible to make PHP like Node, but I wouldn't expect it to happen any time soon - if ever.
I come from a Java background, where the JVM is a long-running process, servers can be started by user code and application state is normally kept and managed by the framework of choice (e.g. Spring).
In the PHP world, things are stateless, as each script execution is volatile in the sense that it does not keep state (unless it uses an external medium, like a DB, in-memory cache, etc). The web server invokes the script via CGI (probably through php-fpm to optimise resources).
Is it correct that every single HTTP request incurs in the overhead of initialising the framework and middleware only for that request? It appears so when reading Laravel's Request Lifecycle, for example.
Doesn't this entail a lot of repetitive overhead for every single request that enters the system (e.g. detecting environment, initialising handlers, routes, ORM, logging, etc.)?
Or am I missing something? Do these frameworks indeed keep state in some manner?
I have a python script that parses a large set of data into an internal memory structure, and implements various fetch functions on the structure.
I want to built a simple web frontend for this script, with the condition that the data is only initialized/loaded once (since re-loading upon each fetch would consume too much time/resources). Essentially, the python handler needs to maintain its state between calls, so the data structure is preserved in memory.
Note: PHP's exec() or similar will not work, as this instantiates a new python handler per request. I have heard vague references to using mod_python for this purpose?
I have implemented a solution to a very similar problem. My solution was to use an xmlrpc server, specifically
twisted.web.xmlrpc
I have a method that allows for injection of new data, and have methods for retrieving the data.
Use a persistent server like CherryPy or Twisted Web. All requests will be served by the same process.
I'm assuming that for every page request, the webserver (eg. Apache) creates a new instance of a script in memory. Can these instances communicate with each other while running? and pass data too?
If you want to pass data between scripts in PHP I suggest using either memcached or a database. Or possibly APC.
If the scripts belong to the same session, they could theoretically communicate via the session but this would be effectively a one-way communication in most cases because only one script can access the session at any one time (session_start() locks the session until that script ends the session implicitly or explicitly).
I believe Martin and Cletus' suggestions are valid. My choice would be function of the end goal of the script.
How much data will you be throwing around? Can you handle the overhead of an external process?
What kind of data are you exchanging? Is it normalized? Or is it now worth normalizing?
Will you need to refer to that data later on? Or can it be discarded after being processed?
Will those scripts ever run on different servers?
Flat files, with a locking mechanism
Relational DB
Document DB (key/value store, whether persistent or not)
Shared memory (APC, or core functions)
Message queues (Active MQ and company)
I think you'll get the most value by externalizing the process as you can have more than one machine managing the messages/data and more than one producing/consuming them.
The model that PHP scripts operate off of doesn't really contain the notion of any kind of persistence in memory for those scripts, since generally they're designed to only run for the minimum of time required to serve the requested page. This would make it hard to have any meaningful use for stateful communication between those scripts, since typically once the page is served there's nothing more for the script to do. Thus usually any communication between PHP scripts is done more through manipulation of database entries and the like.
If you have some sort of continual processing that should be happening for which you'd want to be passing data around, you might want to look into other web application models such as servlets.
You should be able to do this with some shared memory, as described here: http://blog.taragana.com/index.php/archive/how-to-use-shared-memory-in-php/ (assuming you're not running on Windows)
I am new to php, but in other web technologies, you can share objects between page instances. For example, in java jsp pages you easily have on class that exist as static class for the whole server instance. How to do this in php?
I am not refering to sessions variables (at least I don't think so). This is more for the purpose of resource pooling (perhaps a socket to share, or database connections etc). So a whole class needs to be shared between subsequent loads, not just some primitive variables that I can store in the session.
I have also looked into doing php singleton classes but I believe that class is only shared within the same page and not across pages.
To make things even more clear, I'm looking for something that can help me share, say, a socket connected to a server for a connectSocket.php page such that all users who loads that page uses the same socket and does not open a new one.
This is a bit of a difficult answer, and might not be exactly what you are looking for.
PHP is built upon a 'shared-nothing' architecture. If you require some type of state across your application, you must do this through other means.
First I would recommend looking into the core of the problem.. Do you really need it? If you assume the PHP application could die (and lose state) is it ok to lose the data?
If you must maintain the state, even after the application dies or otherwise, you should assume probably the best place to put the data is in MySQL. PHP is intended as a thin layer around your business logic, so I can highly recommend this.
If you don't care about losing the data after a restart, the problem domain you're looking for is probably caching. I would recommend looking into memcached or if you're on a single machine, apc. APC will definitely work for you with Apache on a single machine, but you will still have to code your application assuming you might lose the data.
If you're worried your underlying datastore (MySQL) is too slow, but you still need to maintain the data after a restart, you should look into a combination of these 2 systems. You can always push and pull your data from the cache, but only when it updates send it over to Mysql.
If the data is purely user or session-bound, you probably want to just looking into the sessions system.
I've personally developed a reasonably large multi-tenant application, and although its a pretty complex application, I've never needed the true state you're looking for.
Update: Sorry, I did not read your note about sharing a socket. You will need a separate daemon to handle this, perhaps if you can explain your problem further, there might be other approaches. What type of socket is this?
There's a fundamental difference between web-served Java and web-served interpreted languages like PHP and Perl. In Java, your web server will have an operating environment that maintains state (ie. Tomcat). With interpreted languages, a request to your web server will generally spawn a new web server thread, which in turn loads a fresh operating environment for that thread, in this case, the PHP environment.
Therefore, in PHP, there is no concept of page instances. Every request to the web server is a fresh start. All the classes are re-loaded, so there is no concept of class sharing, nor is there a concept of resource pooling, unless it is implemented externally.
Sharing sockets between web requests therefore isn't really possible.
This is likely a partial answer but you can save an instance of a class into a Session variable and access it at another time.
Most of the PHP database libraries use connection pooling already. You call, for example, pg_connect as if you were requesting a new connection, but if the connection string is the same as a connection that already exists, you will get the established connection back instead. If you only care about pooling for database access, then you can just confirm that it exists in the db library you're using.
An other horroble solution may be to load the data of the object to any $_SESSION variable and then user it back into the object of the other page.
In fact, this is the solution I'm going to follow in my project, until I get some better one.
Regards!