I search a lot of magento performances articles about magento, I've tested varnish, complier and memcached ...etc. I no doubt with running those cache on Production servers. It runs like a rock, I am happy with that.
But I felt very painful on development magento modules. Whenever I edited lines of codes, I refresh the pages and see the update, it cost me 3~7 seconds on average. I am wondering if there is any order way to speed up on the development stage.
my development environment:
MBA 10.9
Vagrant 1.5 + Virtual Box + Ubuntu 12.04
Nginx + PHP-FPM + PHP5.7
disabled all caches while development <-- I known it.
Is there any tips?
A good start is to use a debugger for PHP, so you do not need to refresh the page so often, saves a lot of time :-) I really regret to have lived in pain for many years ignoring the existence of PHP debuggers.
And for magento speed itself, do not deactivate all caches but do it as you go and as you need it. So you can let the configuration cache ON for most of the time except when you adjust some config XML of your module, then clear the cache once and let it ON.
The same for translation cache. Let them on except you tuned some translation files or web service configurations, clear them if you need it once.
EAV collection cache can stay ON as well if you actually do not work on models.
Only cache you have to disable permanently is block cache for the most dev work.
Last but not least, use APC opcode cache, but ENABLE apc.stat=1
And the last one: Use the best hardware you can afford :-) and consider doing the most work on a powerfull, dedicated desktop maschine instead of virtual one which you can use as backup dev environment if you need to be mobile.
Related
How can I configure Magento so that:
1) it can handle 10000 customers at a time
2) it can tolerate sudden increase in load
I searched Google but most of articles are explaining to improve Magento load time.
Where should I set the cookies and session expiration time?
I want to achieve this by modifying .htaccess ,php.ini and Magento admin panel setting.
Other ways are also welcome.
I have done extensive testing on this and have found the single biggest thing to do to improve performance is the following.
(all the following links can be found at http://www.magentocommerce.com/)
Make sure Magento Caching is enabled (easy to overlook when you have
it turned off while developing)
Use some sort of full page caching such as
magento-connect/zoom-full-page-cache-1742.html
Use a CDN such as MaxCDN or AWS Cloudfront (using
magento-connect/6274.html makes this pretty easy)
The above make the big improvements. If you need more improvements installing varnish really does the biggest but can be a pain to use since it is normally setup to take over port 80. This makes managing and developing your site later a bit of a pain as you will generally need to disable it or have it bypass varnish to do any major development work.
Install varnish - magento-connect/pagecache-powered-by-varnish.html
or magento-connect/2984.html
Make sure you have APC and Memcached installed
Make sure you have gzip compression turned on.
Advance performance method (these are helpful if using AWS and you want to use multiple servers in different zones)
Install varnish on it's own server and direct the web-server port to
your Magento server. This reduces the number of hits your web-server
sees.
Install your magento database on it's own server. Magento has funny resource requirements memory and cpu wise. What may be good for Magento might not be as good for the web server etc or database. Splitting your database off should be fine if you are one the same local net (i.e. AWS same region). This method allows you to use multiple web servers.
Use a AWS with elastic IP and place webservers in different zones and multiple web servers with a single database server. Use multiple varnish servers infront of the web servers.
Some additional notes:
APC, Memcache, php, using ngix will only provide about a 10% improvement vs just using Magento Cache, full page cache and Varnish. Also before testing make sure you test your server setup using a stock magento install with dummy data. This will help you set a baseline performance to see if hardware changes or need or to identify if a module or other plugin might be causing big performance hits. Sites like loadimpact.com can be helpful.
If you have access to php.ini then I am going to take a wild guess and assume you also have access to your database configuration files (my.conf).
In case you are using MySQL adjusting the query_cache_size parameter can have a tremendous positive effect on Magento performance because its constantly polling a large amount of the same data that gets reused. The exact amount of memory you'll use depends entirely on your needs so you will have to adjust it accordingly.
FPC is the most important point that should be applied.This will reduce your Mysql load considerably.
Solr for search is the second thing that should be done.
Make sure your code is to Magento standards so that it effectively uses magento cache and FPC.
Above three will result in Max optimization.
Look here : Tweaking magento for performance for lots of additional points.
There are so many ways to optimize Magento. Some of the configuration are from Magento admin panel it self.
Go to: System->Configuration->CATALOG/Catalog->Frontend
Use Flat Catalog Category: Yes
Use Flat Catalog Product : Yes
Go to: System->Configuration->ADVANCED/Developer: Merge javascript and CSS
Install memcache on you server
Some of the Helping links:
http://magento2x.com/speed-up-magento/
http://support.metacdn.com/entries/25027521-Slow-Magento-Speed-up-Magento-with-our-Magento-Optimization-guide
Also you can go for Google and find some more terrific configuration for server.
one simple way is to install and config APC
APC
# in php.ini:
extension=apc.so
[apc]
apc.enabled = 1
apc.cache_by_default = On
apc.shm_segments = 1
apc.shm_size = 128M ; memory size when using as nosql
apc.ttl = 60
apc.user_ttl = 7200
apc.gc_ttl = 600
apc.num_files_hint = 0
apc.write_lock = On
;apc.stat = 0 ; enable detecting file stat, reload if changed
I recommend to red this "White Paper: Optimizing Magento for Peak Performance"
This white paper documents the exceptional performance that can be achieved by properly optimizing and configuring Magento Enterprise Edition. The resulting optimization not only can contribute to higher conversion rates and support for greater numbers of customers and orders per day, it can also lead to improved hardware efficiency and overall cost savings.
https://info.magento.com/Optimizing_Magento_for_Peak_Performance.html
I have problem with drupal 7 installation (I have it locally on my PC). The page loads
very slowly on every request, I have enabled Slow Queries Log and this is the output:
"Executed 626 queries in 300.88 ms. Queries exceeding 5 ms are highlighted. Page execution time was 15795.26 ms. Memory used at: devel_boot()=3.01 MB, devel_shutdown()=88.32 MB, PHP peak=92 MB."
Some more info:
The number of queries varies each request from 300-1000 !
15sec. and sometimes even more up to 70-90 seconds (but rarely) !
I have disabled the cron via "admin/config/system/cron" page.
Please help!
Start disabling custom modules you have one by one to see which is working slow.
Also you can check admin/reports/status for hints, what is wrong with your Drupal instance.
If you deleted folder with some module without disabling that module, drupal trying to find that lost folder.
Return deleted module's folder
Disable module
Delete folder again
First thing that matter for Drupal site performances is OS.
It is very important which OS you use on local machine. Drupal works better on UNIX based systems. For hosting Drupal site you should go with Linux platform. For developing go with Mac OS and go with Windows if you have no other choice.
Second thing is your hardware.
You should know that you must have decent hardware for good response of Drupal site on local machine. Average Drupal site using about 30-50 enabled modules and runs around 100-200 queries per page load. More complex site will rune slower and there are many ways to optimize performance of your Drupal site but that is whole another topic.
Third thing is type of modules you using.
Performance of your site is directly related with your enabled modules. Number of modules is important but it is also important what type are these modules. For instance some modules are used only in developing process so when you finish with your developing you should disable them. Math is simple. Using modules which are more complex will give you performance penalty.
Things you should do:
Check updates for your Drupal core and your modules
Update your core and modules with the latest releases
Run /update.php script
Disable developing modules and all other modules you don't need (most common modules which result some performance loss are: update manager, views UI, devel...)
Set up cron tasks only based on your needs
If you still have problems with your performances considering all above you should go on drupal.org support forums and ask for help with specific details of your problem
Hope this helps.
I need to speed up my magento installation, so I'm planning to put the content of 'var/' (or only var/cache and var/sessions) on a tmpfs.
I'm also buying a reserved instance on Amazon, so I would like to keep a sufficent amount of RAM. I want to enable memcached, PHP Apc, MySQL caching and HTTP caching.
I'm thinking of a Medium Reserved Instance with the following specs:
3.75 GB memory
2 EC2 Compute Unit (1 virtual core with 2 EC2 Compute Unit)
410 GB instance storage
32-bit or 64-bit platform
I/O Performance: Moderate
EBS-Optimized Available: No
API name: m1.medium
Will the RAM be enough to appy a good caching system?
Looking now (after 3 months) the var directory is 14gb, but I think cleaning it up each 5/7 days would be good too.
Do you have any suggestion for me?
P.S. the store will contain an average of 100/150 products.
I think moving /var to a tmpfs is probably not your biggest bottleneck and would likely be more trouble than its worth. Make sure Magento caching is enabled and you have APC enabled.
This post covers some general tips on increasing Magento performance:
Why is Magento so slow?
I would suggest looking into setting up a reverse proxy like Varnish.
Getting Varnish To Work on Magento
If you do plan on just using a tmpfs in memory I would suggest looking into Colin's improved over Zend_Cache_Backend_File
https://github.com/colinmollenhour/Cm_Cache_Backend_File
Also I would suggest looking into mytop to keep tabs of if you have any places you can optimize queries in the application itself or in my.cnf to help ease any DB bottlenecks.
http://jeremy.zawodny.com/mysql/mytop/
Session Digital has a good white paper (although somewhat dated) on optimizing Magento enterprise and the same can be applied to Community. Out of everything I've tried, Varnish, as mentioned in the White paper offered the most significant increase in response time.
http://www.sessiondigital.com/resources/tech/060611/Mag-Perf-WP-final.pdf
Hope this helps!
Firstly, +1 to all of the answers here.
if you're thinking about running /var/ out of tmpfs it's probably because you've heard of the lousy file IO on AWS or you have experienced issues with it yourself. However, the /var/ directory is the least of your concern - Zend / Magento's autoloaders are more taxing to IO. To mitigate that you want to run APC and the compilation (assuming you're not using persistent shopping cart).
As echoed by other commenters, anything that runs from cache or memory will circumvent PHP and thus the need to touch the disk and incur IO issues. Varnish is a bit of a brute-force approach and is a wonderful tool on massive sites that scale to millions of views; but I believe that Varnish's limitations with SSL and the lack of real documentation and support from our Magento community make it a better intellectual choice than an actual alternative.
When running Magento Community I prefer to run Tinybrick's Lightspeed on AWS on a Medium instance - which gives me the most bang-for-buck and is itself a full-page-cache. I get 200+ concurrent pages/second in this setup and I'm not running memcached or using compilation.
http://www.tinybrick.com/improve-magentos-slow-performance.html/
Be careful with running memcached in your AWS instance as well - I find that it can be impeded by a power-hungry Apache gone wild in the rare instance you haven't got a primed cache which causes Apache maxclients issues while it waits for cache response. If you could afford it I would rather run two micro Apache instances with a shared memcached session store and a load balancer in front of them - give some horsepower to the db on a separate box for them to share, though. But all setups are unique and your traffic/usage will dictate what you need.
I have run Magento in the AWS cloud for 3 years with great success - and I wish the same to you. Cheers.
I'm new to Symfony2 and I must admit I'm enjoying it. I started playing with the SonataAdmin but soon a major doubt popped out:
is it normal it takes almost 3 seconds to load a listing page (using an empty database)?? I know in production I should go for APC or memcache to speed up things but it looks strange to me it takes so much time.
I'm developing using a virtualmachine with Turnkey lamp (1GB ram).
My PC is fairly new: Intel i3 8Gb ram.
Please tell me what you think/experience.
Thanks.
In development environment, it is hard to measure performance because the framework and the bundles sometimes need to parse a lot of configuration files, introspect objects and perform time consuming task and cache the output.
In production, a lot of stuff is done upfront, i.e. when you are deploying to your web server. The work upfront is done to avoid parsing files, do time consuming task, etc. This is the reason why in production, you almost can't change anything without running php app/console clear:cache again after the modification have been done. Even changing a single Twig template requires a cache clear to update the output presented to the end user.
I did not test this bundle personally, but an admin generator bundle needs to check a lot of properties and objects to perform his task properly. This is indeed time consuming but this is required only in development mode. When in production, this introspection process is not necessary and the information is probably cached somewhere. This should give way better performance in production environment than in development environment.
Bottom line, I don't think this bundle suffer from a performance problem but this depends on your needs and objectives. The only thing I can be sure of: test it in production mode to see the speed it will give you in the end. Clear you cache for production mode and use app.php instead of app_dev.php. Also, check the documentation on performance that can be found on symfony.com.
php app/console cache:clear --env=prod --no-debug
Hope this helps.
Regards,
Matt
I have started having problems with my VPS in the way that it would faill to serve the pages on all the websites. It just showed a blank page, or offered to download the php file ( luckily the code was not in the download file :) ).
The server was still running, but this seemed to be a problem with PHP, since i could login into WHM.
If i did a apache restart, the sites would work again.
After some talks with the server support they told me this is a problem with the APC extension witch they considered to be old and not recommended for production servers. So they removed it for now, to see if the same kind of fails would continue to appear.
I haven't read anywhere that APC could have some problems or that its not always recommended to use, quite the contrary ... everywhere people are saying to always use it.
The APC extension was installed ssh and is the latest version.
Edit:
They also dont recomend MemCache and say that a more reliable extension would be eAccelerator
Um APC is current tech and almost a must for any performant PHP site.
Not only that but it will ship as standard in PHP 6 (rather than being an optional module like it is now).
I don't know what your issue is/was but it's not APC being outdated or old tech.
I run several servers myself and the only time I have ever had trouble with APC was when trying to run it concurrently with Zend Optimizer. They don't work together so if I must use Optimizer (like if some commercial, third-party code requires it) I run eAccelerator instead of APC. Effectively 6 of one, half-dozen of another when it comes to performance but I really doubt APC is the problem here.
Just to add, memcached is only going to benefit you greatly if you are running multiple servers which need to access a shared data cache. Memcached does not do opcode caching like APC/eAccelerator/Xcache/etc.
The problem is not to do with APC. If APC had a problem, it would either show up in your php log file or you simply wouldn't be able to access your website until you adjusted APC. The problem is more likely with apache itself. I've experience the same problem as you with blank pages before and it was to do with mod_security playing up and preventing pages from being sent that looked "suspicious". Also, memory usage in apache is good at killing the server under load. I've also had experience with a web host that had compiled apache with a memory leak so every X amount of requests (say 100,000) the server would crash! Most annoying.
Your web host doesn't sound the most competent out there as they are giving some bad advice, most likely based on ignorance.
APC should be used on production (with the mstat check turned off on production, but on for development). You can get more stats about your apc setup while it's working by loading the apc status file that comes with it and you get a nice page like this: http://drupal.org/files/images/APC%20Status-1.png
Memcache is very heavily used as it's also distributed! The use for such is as follows:
APC is the fastest as it works most closely to php, but only works on the same server executing the PHP itself so it's use is limited in that scope. Used primarily as an opcode cache.
Memcache is like a very fast database spread over many computers working as one unit. However, a powercut will wipe the lot!!! Hence why they are heavily used to remove preasure from the persistant database. Facebook and many other sites have hundreds of servers running memcache.
My advice would be to find a web host that understands PHP. Fighting web hosts is hard work about whos right and whos wrong... until you find a good one ;)
Sounds to me like they are pushing a product that they probably have referral kickbacks on.
I run my own servers (have for a while) and I've never had this problem, not any MAJOR problems with MemCache.