The question, simply put, is the one in the title. Is it possible?
So far, my experience with scripting languages is that, to increase performance, you need to cache everything and later just serve the generated HTML files.
That's ok for some use cases, but when you really need to generate a new page in realtime, it's just impossible.
Drupal can take up to 3 seconds (or more!) to render some web pages (PHP execution time, not DB). That's crazy. Completely crazy.
If many projects (like Facebook) are using PHP, obviously the problem is mine. But googling for this problem shows that it's common. Too common.
(Of course I installed APC for PHP. It certainly helps, but PHP is still ultra-slow).
Must I assume this is the reality for Drupal / PHP?
Thanks.
Short answer is no. But why would you not want to cache?
What do you mean by 'generate a new page in realtime'? Authenticated users (anyone logged in) can see new content right away. Anonymous users may have to wait a little bit (if you are using Boost, for example), BUT, you can always control it, or flush it when new content is added. You should cache as much as you can.
You can install Boost (static HTML files), Memcache, and enable Drupal cache. It's encouraged, especially the last one. You can also run nginx on the server.
You can also try using Pressflow, a drop-in replacement for Drupal that will give you better performance.
http://pressflow.org/
Its been discussed many times.. you can make Drupal extremely fast if you want to. Check out some of the 2bits articles:
http://2bits.com/contents/articles
Utilizing the available methods of caching will help you keep your hosting cost low, instead of throw more hardware on an unoptimized site.
As you say, Facebook uses PHP, and they clearly have reason to need good performance. Their solution was to write their own compiler for PHP called HipHop, which they released as open source. If you're worried about PHP's performance, you should give it a try as it will definitely improve things.
The downside is that it doesn't (yet) cover 100% of the PHP function set, so some PHP programs may not compile. I don't know where Drupal fits into this, but it would be worth trying it out - there's nothing to be lost by doing a test compilation; if its not going to work, you won't have lost anything.
On a similar vein, there is a project in the Drupal community to convert parts of the Drupal Core into a PHP Extension, meaning that some key Drupal functions are then built-in to the PHP runtime as compiled code. See the project page here. But note that this is still in a fairly early stage of development: it's still listed as experimental, and only covers a small number of functions. It might be worth keeping an eye on the project, though.
According to http://groups.drupal.org/node/34076, yes you can get a < 200ms response time with Drupal without caching.
The tips that I've received from some friends regarding Drupal load performance is to install less than 40 modules.
More than 40, especially if those contrib modules use too much hooks and memory, and the performance will be decreased.
Other tips:
remove imagecache ui and views ui on production site
if possible put htaccess on vhost.conf so that htaccess will only be called once on apahe start
use throttle module
use gzip for all html, css and js files
use cdn module and amazon server solution
use ajax for some parts or blocks of your site
last and if there is enough budget, migrate to oracle
Related
I'm new to WordPress, and I'm building some backend logic to it.
I want the admin to have as smooth experience with it as possible.
I want for him to be able to run the website with "a click of a button".
I'm used to Java and nodeJS environments, where I have life cycle,
where I can specify logic to happen when the server starts, but I'm having trouble to understand how it's done in WordPress(or PHP for that matter).
I want the website to check the database and see if it has needed tables for it's functioning, and if not, to create them and fill them with relevant data, as well as to check if the database is up-to-date (in case of a long crash),
and update if necessary.
Right now I'm thinking about running a Cron script to check it, every few minutes but it's heavy on resources. A better solution might be to run it on the first interaction with a user, but it seems not ideal, as it will slow him down.
Is there a life cycle in WordPress?
should I be worried about it crashing during important operation and then starting on it's own?
Can I specify logic for it to run on it's boot/restart?
I am not completely sure what you have in mind when you say "fill them with relevant data", but I would either recommend, leveraging WordPress' own logic or complete your task using other tools.
Just like any php script, it is stateless and it has an index.php as starting point. Then files are loaded in order and the contents of your request and the environment will depend where you end up.
This is just how php works and the key difference with JS is that JS executed on your computer, and php is a set of server side scripts that are compiled and will produce some sort of output that is sent back to your browser, just like when you call a REST api.
You might want to take a look at the following things:
wp-load.php: the file that will look for your constants defined in wp-config.php, when this file is not yet present it will redirect you to the "famous" wordpress site setup (after loading a bunch of stuff related to database connections and request data). You could follow the logic, but I would advise agains that. This due to the fact that the WordPress core is very old and this gives you an example of how php applications from the early 2000s used to look like and will most likely cause headaches.
Existing tools
Not only on server level, but also things like wp-cli or maybe a composer based solution like roots/bedrock or even roots/wordpress.
To answer your question about lifecycles directly
Yes, WordPress offers an old-timey hook system, but this is just during the request lifecycle for an active install, so this wouldn't be exactly what you seem to be looking for.
Finally, it is good to have some understanding of the internal workings of WordPress, but the whole reason that WordPress is easy to run and compatible with many setups, is just because they "strive for forever backwards compatibility" (which is also why they don't use semantic versioning). Which in turn means that the core is very outdated and unreadable, so I wouldn't bother trying to figure it out yourself.
And even more so I wouldn't want you to think that this is a fair representation of the PHP-world, since the initial release of WordPress, the language has completely evolved and most of the key components to it being a nice developer experience were still a long way ahead.
In short, I'd look for existing solutions which are built for your specific server setup and if that is not possible for some reason, try to find some sort of CLI tool in php, or other languages.
I am installing a pre-build php-based web application for a client. Unfortunately the application performs very slowly because it compiles lots of data.
Page load times go up to 40 sec.
I know about ob_caching but I don't want to mess with the application unless it is absolutely necessary.
Are there any tools/scripts/apache modules to cache the entire output of the application statically one the server and update it on a regular basis.
I am just looking for a middleware or something which build regular static html pages form the php application. (BTW: I tried eaccelerator, but it didn't improve the situation.)
I would appreciate any tips.
Thanks in advance.
eAccelerator should have made a measurable difference, so are you sure it was installed correctly? You should have seen an eaccelerator section in phpinfo() showing that the cache was full. You may also have ahd the cache set too small etc. Alternatively, try APC instead. If neither show any performance improvement, you may have a server issue.
In any case, 40 seconds is crazy slow for anything. Are you sure this is PHP and not poorly optimised SQL queries?
Looks like this should do the trick (but YMMV, depending on your application):
http://httpd.apache.org/docs/2.2/mod/mod_cache.html
I have made site on drupal
My site has 7500 users and approx (20 to 50 without logged in)(2 to 10 logged in) users are online (and this is not heavy traffic I think)
The site is on dedicated server. I have enabled setting in performance from drupal admin and also installed memcache and eaccelerator
I looked in query logs from using devel module. it is firing total 600 to 900 queries on each page
When I have installed patch of path.inc to reduce the queries of drupal_look_path(). It has reduced queries to around 400
I have also made some positive changes in mysql (my.cnf) file, but still there are many same queries run form user_load() function again and again
I have 60 to 70 modules enabled and all are use full. I can't remove the modules
Still the site is running slow it is taking approx 10 to 15 sec
Now I don't know why the site is running so slow
Is it because the drupal has the large php code ?
Is it because it is firing so many queries on each page?
Does the InnoDB engine improve the performance?
Please, any kind of suggestions are welcome
400 queries for each requests is a sucidie (but even 50+).
You should implement some html cacher. My website generally doens't even make the db connection. It just fires the html cached in a file.
Some additional things to look into:
Install a tool like Yslow/PageSpeed to see how much of those 10-15s are client and server time.
Instal XhProf (on a development site, not live) together with Devel to see which are the functions that use the most time. Look into these first. Edit, now with link: http://groups.drupal.org/node/82889
Using pressflow might help a bit, but since you are alrady using the path.inc patch, probably not so much.
You mentioned that you installed memcache. Did you also install the memcache module and configure the cache plugin to use memcache?
EDIT: Yes, switching to InnoDB can help. One of the main performance advantages of InnoDB is row-level locking (as opposed to table-level locking of MyISAM), which means that multiple INSERT/UPDATE queries against the same table won't block each other unless really necessary. However, InnoDB does not perform well at all out of the box, you really need to fine-tune your mysql configuration for your specific site. So this is a step that you should only take carefully and after testing and optimizing on a development site. There are various questions already on this site and elsewhere about InnoDB tuning...
Anything else than that is then site specific and depends on the modules you are using. But especially things like complex node_access setups and multiple languages (i18n!) tend to either cause slow queries and/or a lot of them.
Not all modules make use of the caching mechanisms you can switch on in the performance settings area. It would be worth trying to identify which ones are doing the most/slowest queries and attempting to get the developer(s) to improve them.
Alternatively, examine whether you could achieve things with fewer modules. Some modules do overlap somewhat in functionality, so you may be able to reorganise the way the site functions a bit.
Additionally, you need to look at whether your settings MySQL are allowing enough memory for these queries to be carried out. Most MySQL distributions come with different versions of my.ini labelled 'small', 'medium', 'huge' etc. Copy the 'huge' one to my.ini (back up the old one first) and restart the DB to see if maxing out all the cache sizes makes a difference. You may well have a bottle neck there, but it can be hard to work out what setting is causing it.
Same goes for PHP. Set memory_limit in php.ini to 500MB or something and see if it helps. Of course, you may not be able to do this, depending on your hosting arrangements, but it will eliminate one possible cause (or not) if you can.
Performance of your Drupal website also depends on how well your hosting platform is tuned for Drupal. Drupal requires special optimization of LAMP stack components. You can try Drupal-specific hosting companies http://www.drupalspecific.com to make your website run faster.
facing the drupal slowness issue myself. But have a very different issue than the others mentioned.
I disabled all the content also the drupal header for a drupal page of a specific content type.
Still the time taken by this page to load is above 20 secs!
I took help of YSlow and NET firebug panels.
Upon looking at them, noticed:
JS and CSS files inclusion individually takes 3 to 2 secs, and there are fair bit of inclusions happening, as a result it takes like 20 secs.
But i am not able to figure out, why the js and css inclusions are taking so much time. (this includes normal drupal core js and css files as well)
I recently had to work on a project where the previous developer modified the wp-admin directory. It seems like a bad idea to me, since Wordpress is constantly updated. Am I just not at that level of expertise with modifying Wordpress?
Being open source, I think it's a common thing for software like WordPress to be modified and extended at any point.
To modify or not to modify is a choice between trade-offs. New features can be encapsulated as modules, which may, perhaps, cause their functionality to be less integrated than desired. However, fully integrating changes may hinder easily updating the software as new versions are released.
It does require that someone be very familiar with the software to modify the software directly, but this isn't necessarily a bad idea.
On a side note, I think modifying WordPress is almost a necessity, especially if you want it to have a decent architecture or to actually be secure (ok, that was a jab, sue me).
Well, it is a bad idea only in that it means you are now responsible for maintaining an internal defacto fork ... every time WordPress releases an update, you have to do a three-way diff to merge your changes into the new "real" WordPress. (Three-way diff means you do a diff between your fork of the old version and the standard old version to build a patch set, then apply that patch set to the new version.) You should also be using a VCS yourself to keep yourself sane.
If you aren't up to this then you aren't up to it, there's nothing wrong with following the KISS principle and not mucking up the application code.
If you can write a plugin that does the same thing and does it just as efficiently, then you should do that so you don't have to maintain your own fork.
However, there are a lot of things WordPress is terrible at (efficiency, security) that you can ameliorate (sometimes without much work, just by disabling code you don't need) only by hacking the application code. WordPress is dirty legacy spaghetti code originally written by people with virtually zero knowledge of software or database design, and it does a lot of tremendously stupid things like querying the database on every request to see what it's own siteurl is, when this never changes -- there's nothing wrong with taking 5 minutes to change 2 lines of code so it doesn't do this any more.
I worked as tech lead on a then-top-20 Technorati-ranked blog and did a lot of work to scale WordPress on a single server and then onto a cluster (with separate servers for admin vs. public access). We had upstream reverse proxies (i.e. Varnish or Squid) acting as HTTP accelerators and an internal object/page fragment cache system that plugged into memcached with failover to filesystem caching using PEAR::Cache_Lite. We had to modify WordPress to do things like send sane, cache-friendly HTTP headers, to disable a lot of unnecessary SQL and processing.
I modified WP to run using MySQL's memory-only NDB cluster storage engine, which meant specifying indexes in a lot of queries (in the end we opted for a replicated cluster instead, however). In modifying it to run with separate servers for admin vs. public access, we locked down the public-side version so it ran with much reduced MySQL privileges allowing only reads (a third MySQL user got commenting privileges).
If you have a serious comment spam problem (i.e. 10K/hour), then you have to do something beyond plugins. Spam will DOS you because WordPress just initializing its core is something like half a second on a standalone P4 with no concurrency, and since WP is a code hairball there's no way to do anything without initializing the core first.
"WP-Cron" is braindead and should be disabled if you have access to an actual crontab to perform these functions. Not hard to do.
In short, I could go on forever listing reasons why you might want to make modifications.
Throughout this it was of course a goal for maintainability reasons to keep these modifications to a minimum and document them as clearly as possible, and we implemented many as plugins when it made sense.
On one blog/forum combination, we hacked together the signup procedure so that people filled in one form to sign up to both WordPress and phpBB at the same time. I'm sure there's a better way to do that with plugins, but it did have one unexpected benefit - it really confuses the spambots. Despite having several of them register each day, we've had about two spam posts in the life of the forum.
Not something I'd recommend, of course - it stops us from upgrading either software.
I tend to strongly advocate against modifying core code if at all possible, especially in a project that updates like WordPress does. If WordPress can't be made to do what you need it to with plugins and the like, you're probably better off with a more extensible/generic system like Drupal. Hacking a blogging-oriented CMS into something else might not be worth it.
In the older versions of WordPress (1.0 and even the early 2.0s), I wouldn't bat an eye to modifying WordPress itself.
However, WordPress' architecture has matured. Sidebars no longer need to be manually coded. Instead, you can port your theme to use widgets and just create widgets (what a godsend!). Don't like how something is displayed - just modify the theme! Don't like how WordPress handles something? Create a plug-in. I'm hard pressed to think of a reason to modify the WordPress code itself that cannot be handled via WordPress' contemporary modular components (widgets, plug-ins, themes) instead.
I'm the type of person to always get "under the hood" in open source apps like WordPress. However, nowadays, there's really no good reason to modify the core WordPress code.
What is the best way of implementing a cache for a PHP site? Obviously, there are some things that shouldn't be cached (for example search queries), but I want to find a good solution that will make sure that I avoid the 'digg effect'.
I know there is WP-Cache for WordPress, but I'm writing a custom solution that isn't built on WP. I'm interested in either writing my own cache (if it's simple enough), or you could point me to a nice, light framework. I don't know much Apache though, so if it was a PHP framework then it would be a better fit.
Thanks.
You can use output buffering to selectively save parts of your output (those you want to cache) and display them to the next user if it hasn't been long enough. This way you're still rendering other parts of the page on-the-fly (e.g., customizable boxes, personal information).
If a proxy cache is out of the question, and you're serving complete HTML files, you'll get the best performance by bypassing PHP altogether. Study how WP Super Cache works.
Uncached pages are copied to a cache folder with similar URL structure as your site. On later requests, mod_rewrite notes the existence of the cached file and serves it instead. other RewriteCond directives are used to make sure commenters/logged in users see live PHP requests, but the majority of visitors will be served by Apache directly.
The best way to go is to use a proxy cache (Squid, Varnish) and serve appropriate Cache-Control/Expires headers, along with ETags : see Mark Nottingham's Caching Tutorial for a full description of how caches work and how you can get the most performance out of a caching proxy.
Also check out memcached, and try to cache your database queries (or better yet, pre-rendered page fragments) in there.
I would recommend Memcached or APC. Both are in-memory caching solutions with dead-simple APIs and lots of libraries.
The trouble with those 2 is you need to install them on your web server or another server if it's Memcached.
APC
Pros:
Simple
Fast
Speeds up PHP execution also
Cons
Doesn't work for distributed systems, each machine stores its cache locally
Memcached
Pros:
Fast(ish)
Can be installed on a separate server for all web servers to use
Highly tested, developed at LiveJournal
Used by all the big guys (Facebook, Yahoo, Mozilla)
Cons:
Slower than APC
Possible network latency
Slightly more configuration
I wouldn't recommend writing your own, there are plenty out there. You could go with a disk-based cache if you can't install software on your webserver, but there are possible race issues to deal with. One request could be writing to the file while another is reading.
You actually could cache search queries, even for a few seconds to a minute. Unless your db is being updated more than a few times a second, some delay would be ok.
The PHP Smarty template engine (http://www.smarty.net) includes a fairly advanced caching system.
You can find details in the caching section of the Smarty manual: http://www.smarty.net/manual/en/caching.php
You seems to be looking for a PHP cache framework.
I recommend you the template system TinyButStrong that comes with a very good CacheSystem plugin.
It's simple, light, customizable (you can cache whatever part of the html file you want), very powerful ^^
Simple caching of pages, or parts of pages - the Pear::CacheLite class. I also use APC and memcache for different things, but the other answers I've seen so far are more for more complete, and complex systems. If you just need to save some effort rebuilding a part of a page - Cache_lite with a file-backed store is entirely sufficient, and very simple to implement.
Project Gazelle (an open source torrent site) provides a step by step guide on setting up Memcached on the site which you can easily use on any other website you might want to set up which will handle a lot of traffic.
Grab down the source and read the documentation.