What I have been able to grasp from reading the source and documentation from several PHP frameworks is that they generally don't persist, except for what you personally cache or throw into a $_SESSION var. Why is this? It seems a waste to essentially initialize the framework for every single request, would it not be better to at least serialize and store some core objects and variables to save processing and time?
At first I thought this was rather subjective and avoided asking, but everything I've read doesn't really speak about it at all, so there must be something obvious I'm missing.
The only real mention/discussion I've found of this is here which doesn't directly answer my question and some of which goes over my head a little.
Edit for Clarification: I am not asking about the inner workings of PHP, I know how persistence works (ie won't persist unless you make it through caching or session vars), I am asking why PHP frameworks don't do this for their core objects. Again it seems subjective to me, but as almost nothing I've read mentions it, and it seems to be fairly standard practice, I'd like to know what I'm missing.
Memory:
Most frameworks don't store these core mechanisms in $_SESSION due to memory concerns. Frameworks often generate variables / objects that can contain several megabytes of information. That may not sound like a lot, but scale that to a few thousand users and you've got a problem.
Data "Freshness"
The second issue with shoving framework components into memory is that they can become out of date very quickly. Instead of pulling an object out of memory, checking to see if it's outdated and then recreating it (if it's indeed outdated) is less efficient (most of the time) than just recreating it with every request.
I hope this clarifies things.
If you want data to persist between server requests then you need to use cookies/sessions or store your data in a database. This is just the way that it works. PHP cannot store data in itself for use between server requests.
Some frameworks may store core objects in a database or to a local file on disk, but it would depend on the framework.
Related
I am constructing a PHP framework from scratch (unfortunately I don't have any choice in this matter). The framework is required to rely heavily on object-oriented data, and therefore needs to have the ability to store large amounts of object-oriented data efficiently.
I am struggling with the second part.
I've been working on this for a few months. Initially I was introduced to the idea of an ORM, after trying a few pre-built libraries (Doctrine 2, Redbean etc) I liked the idea, but none of what I could find functioned the way that was required, so I set out to create my own ORM, of which turned out quite well. The only issue really is that it suffers in performance, and after spending some time trying to optimize it, I am now convinced that an ORM is not quite the solution to the problem. Although close, it just doesn't quite cut it.
I have briefly looked into other solutions, but due to my lack of experience in this area I am struggling to pin-point the solution.
Here are the requirements of the data storage engine:
Ultimately, it needs to be able to store key-value pairs
The "value" part can be a simple data type, but can also be an object, or an array of the same type of object.
The application defines the structure of each object (or the SCHEMA), sort of in the same way that a .wsdl file works, so the engine would need to like strict formats.
Objects can either have their instances re-used, or not. Meaning that if an object exists as a child object in multiple locations (across many objects) its values are the same everywhere that it is located (if it re-used). Otherwise, a new instance of the object exists for every existing object (not re-used).
There needs to be the ability to query the data efficiently, to make comparisons on any part of an object to find it. For example: find a customer where customer.address.postcode LIKE ('%XXX%')
Any suggestions would be greatly appreciated
EDIT
Thanks to those that have attempted to aid me so far in my somewhat crazy endeavour. To answer some questions that have so far been asked:
What solutions have you tried, and why did they not work?
ORM systems
I had tried a small number of pre-built ORM libraries for PHP. Including Doctrine 2 and Redbean. With Doctrine it was more to do with how you specified the SCHEMA of a model, in that you are required to do so in docblocks. I found this particularly awkward to use due to the requirements that I had, particularly because I knew of a number of ways this could be avoided. I did eventually manage to get Doctrine to work the way that I wanted, but this was after hacking away at the code. Again, this was fun, but it wasn't right.
Redbean actively required me to change the property names of objects. One of my requirements was to basically be able to plug in any sort of document-oriented object, and store it. So having to specifically name properties in order to do this was counter-intuitive. Again, I did play with Redbean for a bit to get it to work, which wasn't right.
It was after playing with a few more ORM systems that I felt I had the knowledge to make my own. Again, the ORM system that I made was good, in that it met the requirements precisely. It was massively let-down due to poor performance, specifically when dealing with large sets of data, but more so when dealing with largely complex models.
Storing objects in XML files
There was a very small time that I considered this, thinking that maybe my requirements meant that I was always going to end up with performance being a problem. So I set out designing a way to generate text-based storage and ultimately ended up creating a whole SCHEMA engine and a bunch of other interesting things. This turned out to be just a fun project in the end, I just couldn't get it to perform at all.
NoSQL
My most recent endeavours have pushed me down the route of systems such as MongoDB and a few other NoSQL systems that I didn't much get into like Cassandra.
MongoDB comes very close to being a tool I could use, however it would require that I add an additional layer because I do in-fact require a SCHEMA, since my objects always conform to a specific structure. I am slowly coming to terms with MongoDB possibly being the solution, however I want to make sure before I spend more time on this.
What exactly do you mean by efficient?
I'm not 100% talking about performance when I mention efficiency, although performance is most certainly an important factor that I am using to consider my options, I understand that going down this route rather than something like a relational database, performance is naturally going to be a problem.
I am more talking about using the right tools. I never like to have to hack away at someone's code to get things to work. To me, it feels as if I am pushing things down a road that the system wasn't designed to go down, and at some point in the future it will bite me in the a**.
So really, when I mention I am looking for something "efficient", I'm meaning tools that match the requirements as closely as possible, so that I am only using/extending the functionality, rather than re-writing it.
Here are some routes to look into. Your requirement for storing "objects" (quite a broad term when it comes to databases) makes me think of:
Storing data in databases in a serialised format, e.g. JSON. PostgreSQL these days has ways to reach into such a column to do search operations on it, so it is not as non-searchable as has been previously regarded (though I would expect it to be slower than querying correctly normalised data).
The requirement to store customer.address.postcode makes me think that you could store your data as a hierarchy, in which case there are several algorithms available to you. Look into nested sets. This is designed to work well with relational databases, without resorting to recursive SQL.
It's not an area of my expertise, but graph databases may be worth looking into.
On a side note, Doctrine is a great library from what I hear, but I suspect you need to work out what technology to use first. It is designed broadly to map onto a relational database, so if you can't express your problem cleanly in a raw RDBMS, Doctrine may not help.
(This could be an XY question, it's hard to tell. You've said you need Y, but if you can tell us that you want to achieve X, maybe the feedback you're getting would be more concrete - and take you in a better direction).
I currently have a AJAX heavy(almost everything) intranet webapp for a business. It is highly modularized(components and modules ala Joomla), with plenty of folders and files. ~ 80-100 different viewing pages (each very unique in it's own sense) on last count and will likely to increase in the near future.
I based around the design around commands and screens, the client request a command and sends the required data and receives the data that is displayed via javascript on the screen.
That said, there are generally two types of files, a display files with html, javascript, and a little php for templating. And also a php backend file with a single switch statement with actions such as, save, update and delete and maybe other function. Several pages/screens may use the same php backend.
Recently, I have been adding an server sided undo function that requires me to reuse some code. So, I took the chance to try out OOP but I notice that some functions are so simple, that creating a class, retrieving all the data then update all the related rows on the database seems like overkill for a simple action as speed is quite critical. Also I noticed there is only one class in an entire file.
So, what if the entire php is a class. So, between creating a class and methods, and using global variables and functions. Which is faster?
Writing object oriented PHP will not affect your performance at all. If you use extensions like Zend Optimizer, it can even work faster.
So, there's really no reason not to use the much cleaner and more easily maintainable object oriented paradigm in PHP.
Writing purely procedural code may even lead to a performance drop since optimizations can be much harder and small details that eat up execution time are more likely to occur in such a messy environment.
The question is misguided. The problem is not one of speed, it's one of code organization. Using only global functions and variables, and lots of them, it'll get harder and harder to avoid naming conflicts and keep everything organized. Classes help you package and abstract things. Execution speed is a secondary concern, and in most cases won't increase noticeably, if at all. Development speed though can increase significantly over time, since you'll have to deal less with conflicts.
Is that a good practice to store the objects of the domain model in the PHP session ?
This would enable to avoid to query and reconstruct every PHP object to each PHP request.
Is there a size limit ?
If you use OOP in PHP you probably want these objects to be reconstructed at every request for several reasons.
All in all there wouldn't be much of a performance difference between storing, loading and deserializing the objects from the session, as you suggested, or just reconstructing them either.
It would depend much on the kind of the classes you use too, of course. Constructing a class usually isn't very problematic in terms of performance unless there is a real lot of stuff needed to do so (Database queries, etc.)
Additionally, doing so could make your code less understandable and/or maintainable too.
And yes, there is a size limited defined by the maximum memory usage of PHP set in the php.ini
So this question is not really answerable unless you exactly know what the classes you want to store in the session are, what they do and how they interact.
For example, if you were having a class which abstracts a logged-in user on your site, storing it's object in the PHP session would probably a good idea. In most other cases, however it wouldn't.
I used to use procedural-style PHP. Later, I used to create some classes. Later, I learned Zend Framework and started to program in OOP style. Now my programs are based on my own framework (with elements of cms, but without any design in framework), which is built on the top of the Zend Framework.
Now it consists of lots classes. But the more I program, more I'm afraid. I'm afraid that my program will be slow because of them I'm afraid to add every another one class which can help me to develop but can slow the application.
All I know is that including lots of files slows application (using eAccelerator + gathering all the code in one file can speed up application 20 times!), but I have no idea if creating new classes and objects slows PHP by itself.
Does anyone have any information about it?
This bugs me. See...procedural code is not always spaghetti code, yet the OOP fanboys always presume that it is. I've written several procedural based web apps as well as an IRC services daemon in PHP. Amazingly, it seems to outperform most of the other ones that are out there and editing it is super easy. One of my friends who generally does OOP took a look at it and said "no code has the right to be this clean"
Conversely, I wrote my own PHP framework (out of boredom) and it was done in a purely OOP manner.
A good programmer can write great procedural code without the overhead classes bring. A bad programmer who uses OOP will always write crappy OOP code that slows things down.
There is no one right answer to which is better for PHP, but rather which is better for the exact scenario.
Here's good article discussing the issue. I also have seen some anecdotal bench-marks that will put OOP PHP overhead at 10-15%
Personally I think OOP is better choice since at the end it may perform better just because it probably was better designed and thought through. Procedural code tends to be messy and hard to maintain. So at the end - it has to be how critical is performance difference for your app vs. ability to maintain, extend and simply comprehend
The most important thing to remember is, design first, optimize later. A better design, which is more maintainable, is better than spaghetti code. Otherwise, you might as well write your web app in assembler. After you're done, you can profile (instead of guess), and optimize what seems slowest.
Yes, every include makes your program slower, but there is more to it than that.
If you decompose your program, over many files, there is a point where you're including/parsing/executing the least amount of code, vs the overhead of including all those files.
Furthermore, having lots of files with little code ain't so bad, because, as you said, using things like eAccelerator, or APC, is a trivial way to get a crap ton of performance back. At the same time you get, if you believe in them, all the wonderful benefits of having and Object Oriented code base.
Also, slow on a per request basis != not scalable.
Updated
As requested, PHP is still faster at straight up array manipulation than it is classes. I vaguely remember the doctrine ORM project, and someone comparing hydration of arrays versus objects, and the arrays came out faster. It's not an order of magnitude, it is noticable, however -- this is in french, but the code and results are completely understandable.. Just a note, that doctrine uses magic methods __get, and __set a lot, and these are also slower than an explicit variable access, part of doctrine's object hydration slowness could be attributed to that, so I would treat it as a worst case scenario. Lastly, even if you're using arrays, if you have to do a lot of moving around in memory, or tonnes of tests, such as isset, or functions like 'in_array' (it's order N), you'll screw the performance benefits. Also remember that objects are just arrays underneath, the interpreter just treats them as a special. I would, personally, favour better code than a small performance increase, you'll get more benefit from having smarter algorithms.
If your project contains many files and due to the nature of PHP's file access checking and restrictions, I'd recommend to turn on realpath_cache, bump up the configuration settings to reasonable numbers, and turn off open_basedir and safe_mode. Ensure to use PHP-FPM or SuExec to run the php process under a user id which is restricted to the document root to get back the security one usually gains from open_basedir and/or safe_mode.
Here are a few pointers why this is a performance gain:
https://bugs.php.net/bug.php?id=46965
http://nirlevy.blogspot.de/2009/01/slow-lstat-slow-php-slow-drupal.html
Also consider my comment on the answer from #Ólafur:
I found especially auto-loading to be the biggest slow down. PHP is extremely slow for directory lookup and file open access, the more PHP function you use during a custom auto-loader, the bigger the slow-down. You can help it a bit with turning off safe-mode (deprecated anyways) or even open-basedir (but I would not do that), but the biggest improvement comes from not using auto-loading and simply use "require_once" with complete fs pathes to require all dependencies per php file you use.
Using large frameworks for web apps that actually do not require so large number of classes for everything is probably the worst problem that many are not aware of. Strip it down at least not to include every bit of code, keep just what you need and throw the rest.
If you're using include_once() then you are causing an unnecessary slowdown, regardless of OOP design or not.
OOP will add an overhead to your code but I will bet that you will never notice it.
You may reconsider to rethink your classes structure and how do you implement them. If you said that OOP is slower you may have to redesign your classes and how do you implement them. A class is just a template of an object, any bad designed method affects all the objects of that class.
Use inheritance and polimorfism the most you can, this will effectively reduce the amount of behaviors and independent methods your classes need, but first off all you need to create a good inheritance map, abstracting your first or mother classes as much as you can.
It is not a problem about how many classes do you have, the problem is how many methods, properties or fields they have and how well are those methods structured. Inheritance reduces the amount of methods to design drammatically and the amount of code to be compiled too.
As several other people have pointed out, there is a mild overhead to OO PHP, but you can offset it by focusing your optimization effort on the core classes that your various other classes derive from. This is why C++ is becoming increasingly popular in the world of high-performance computing, traditionally the realm of C and Fortran.
Personally, I've never seen a PHP server that was CPU-constrained. Check your RAM use (you can optimize the core classes for this as well) and make sure you're not making unnecessary database calls, which are orders of magnitude more expensive than any extra CPU work you're doing.
If you design a huge OOP object hog, that does everything rather than doing functional decomposition to various classes, you will obviously fill up the memory with useless ballast code. Also, with a slow framework you will not make a simply hello World any fast. I noticed it is a kind trend (bad habit) that for one single facebook icon, people include a hole awesome font library and then next there is a search icon with fontello included. Each time they accomplish something unusual, they connect an entire framework. If you want to create a fast loading oop app use one framework only like zephir-phalcon or whatever you fancy and stick to it.
There are ways to limit the penalty from the include_once entries, and that's by having functions declared in the 'include_once' file that themselves have their code content in an 'include' statement. This will load your library of code, but only those functions actually being used will load code as it is needed. You take a second file system hit for the included code, but memory usages drop to practically nothing for the library itself, and only the code used by your program gets loaded. The hit from the second file system access can be mitigated by caching. When dealing with a large project of procedural based PHP, this provides low memory usage and fast processing. DO NOT do this with classes. This would be for a production instance, a development server will show all the penalty of hits since you don't want caching turned on.
I'm building a PHP site, but for now the only PHP I'm using is a half-dozen or so includes on certain pages. (I will probably use some database queries eventually.)
Are simple include() statements a concern for speed or scaling, as opposed to static HTML? What kinds of things tend to cause a site to bog down?
Certainly include() is slower than static pages. However, with modern systems you're not likely to see this as a bottleneck for a long time - if ever. The benefits of using includes to keep common parts of your site up to date outweigh the tiny performance hit, in my opinion (having different navigation on one page because you forgot to update it leads to a bad user experience, and thus bad feelings about your site/company/whatever).
Using caching will really not help either - caching code is going to be slower than just an include(). The only time caching will benefit you is if you're doing computationally-intensive calculations (very rare, on web pages), or grabbing data from a database.
Sounds like you are participating in a bit of premature optimization. If the application is not built, while performance concerns are good to be aware of, your primary concern should be getting the app written.
Includes are a fact of life. Don't worry about number, worry about keeping your code well organized (PEAR folder structure is a lovely thing, if you don't know what I'm talking about look at the structure of the Zend Framework class files).
Focus on getting the application written with a reasonable amount of abstraction. Group all of your DB calls into a class (or classes) so that you minimize code duplication (KISS principles and all) and when it comes time to refactor and optimize your queries they are centrally located. Also get started on some unit testing to prevent regression.
Once the application is up and running, don't ask us what is faster or better since it depends on each application what your bottleneck will be. It may turn out that even though you have lots of includes, your loops are eating up your time, or whatever. Use XDebug and profile your code once its up and running. Look for the segments of code that are eating up a disproportionate amount of time then refactor. If you focus too much now on the performance hit between include and include_once you'll end up chasing a ghost when those curl requests running in sync are eating your breakfast.
Though in the mean time, the best suggestions are look through the php.net manual and make sure if there's a built in function doing something you are trying to do, use it! PHP's C-based extensions will always be faster than any PHP code that you could write, and you'll be surprised how much of what you are trying to do is done already.
But again, I cannot stress this enough, premature optimization is BAD!!! Just get your application up off the ground with good levels of abstraction, profile it, then fix what actually is eating up your time rather than fixing what you think might eat up your time.
Strictly speaking, straight HTML will always serve faster than a server-side approach since the server doesn't have to do any interpretation of the code.
To answer the bigger question, there are a number of things that will cause your site to bog down; there's just no specific threshold for when your code is causing the problem vs. PHP. (keep in mind that many of Yahoo's sites are PHP-driven, so don't think that PHP can't scale).
One thing I've noticed is that the PHP-driven sites that are the slowest are the ones that include more than is necessary to display a specific page. OSCommerce (oscommerce.com) is one of the most popular PHP-driven shopping carts. It has a bad habit, however, of including all of their core functionality (just in case it's needed) on every single page. So even if you don't need to display an 'info box', the function is loaded.
On the other hand, there are many PHP frameworks out there (such as CakePHP, Symfony, and CodeIgniter) that take a 'load it as you need it' approach.
I would advise the following:
Don't include more functionality than you need for a specific page
Keep base functions separate (use an MVC approach when possible)
Use require_once instead of include if you think you'll have nested includes (e.g. page A includes file B which includes file C). This will avoid including the same file more than once. It will also stop the process if a file can't be found; thus helping your troubleshooting process ;)
Cache static pages as HTML if possible - to avoid having to reparse when things don't change
Nah includes are fine, nothing to worry about there.
You might want to think about tweaking your caching headers a bit at some point, but unless you're getting significant hits it should be no problem. Assuming this is all static data, you could even consider converting the whole site to static HTML (easiest way: write a script that grabs every page via the webserver and dumps it out in a matching dir structure)
Most web applications are limited by the speed of their database (or whatever their external storage is, but 9/10 times that'll be a database), the application code is rarely cause for concern, and it doesn't sound like you're doing anything you need to worry about yet.
Before you make any long-lasting decisions about how to structure the code for your site, I would recommend that you do some reading on the Model-View-Controller design pattern. While there are others this one appears to be gaining a great deal of ground in web development circles and certainly will be around for a while. You might want to take a look at some of the other design patterns suggested by Martin Fowler in his Patterns of Enterprise Application Architecture before making any final decisions about what sort of design will best fit your needs.
Depending on the size and scope of your project, you may want to go with a ready-made framework for PHP like Zend Framework or PHP On Trax or you may decide to build your own solution.
Specifically regarding the rendering of HTML content I would strongly recommend that you use some form of templating in order to keep your business logic separate from your display logic. I've found that this one simple rule in my development has saved me hours of work when one or the other needed to be changed. I've used http://www.smarty.net/">Smarty and I know that most of the frameworks out there either have a template system of their own or provide a plug-in architecture that allows you to use your own preferred method. As you look at possible solutions, I would recommend that you look for one that is capable of creating cached versions.
Lastly, if you're concerned about speed on the back-end then I would highly recommend that you look at ways to minimize your calls your back-end data store (whether it be a database or just system files). Try to avoid loading and rendering too much content (say a large report stored in a table that contains hundreds of records) all at once. If possible look for ways to make the user interface load smaller bits of data at a time.
And if you're specifically concerned about the actual load time of your html content and its CSS, Javascript or other dependencies I would recommend that you review these suggestions from the guys at Yahoo!.
To add on what JayTee mentioned - loading functionality when you need it. If you're not using any of the frameworks that do this automatically, you might want to look into the __autoload() functionality that was introduced in PHP5 - basically, your own logic can be invoked when you instantiate a particular class if it's not already loaded. This gives you a chance to include() a file that defines that class on-demand.
The biggest thing you can do to speed up your application is to use an Opcode cache, like APC. There's an excellent list and description available on Wikipedia.
As far as simple includes are concerned, be careful not to include too many files on each request as the disk I/O can cause your application not to scale well. A few dozen includes should be fine, but it's generally a good idea to package your most commonly included files into a single script so you only have one include. The cost in memory of having a few classes here and there you don't need loaded will be better than the cost of disk I/O for including hundreds of smaller files.