I'm a newbie to memcache. My host(hostgator) says they recommend on memcahce on VPS 5 & up. I'm on VPS 4 (Centos, 2 Ghz, 1.3 GB ram 60 GB HDD) & VPS 5 is Centos, 2.7Ghz, 1.8GB ram, 80GB hdd.
Does memcache require so much resources ? I thought that VPS4 should do but my host thinks other wise. Can anyone suggest why memcahced resource needs are so high ?
thanks
Anita
That depends on how much data you want to store using memcache. All data will be kept in RAM, so if you are going to keep there a lot of data you will also need a lot of RAM. Well.. if your VPS4 RAM is completely used by sites/apps you already have there - it is reasonable to use VPS 5 (or even a system with large amount of RAM). Of course, it will use some additional CPU resources, but it needs should not be very high. But if you have on your VPS something simple enough - than it looks like they just want to sell VPS 5.
Determine how much data you are going to need to cache? You just need to have an idea for what size the data objects are going to be and how many of them you think there will be.
You really just need to look at the memcached stats... probably shoot for a hit rate of 90%+ (depending on application needs), an eviction rate of 0, and and perhaps around 50% memory usage when the cache is fully populated to provide for any growth needs, spikes, etc.
Once you know how much that memory allocation is, you can size your server's memory properly.
Related
I recently saw stats on my vps dashboard. but I wonder why the level of use in RAM so much, when the number of visitors have not boming.
following product details:
OS : CentOS 7 64bit
CPU Cores count : 3
Total CPU(s) speed : 4800Mhz
Memory : 2 GB
The number of online visitors an average of 30 people. this takes up about 50% of the amount of Memory available. so it can be estimated if the number of online visitors reached 60 people, then the use of RAM is overloaded.
Is this at a reasonable level? or I need to set up a strategy to prevent the site from being down?
Additional information: I build a site NOT from wordpress or anything else. all suggestions and opinions are greatly awaited, thank you.
The subject is too broad.
Manage your cache
Make use of virtual memory
Use pointers and references when programming: Even if you are not using C++, you still can call large objects and files by reference.
kill unused processes
Uninstall unnecessary applications. This includes starting up applications if you know what you are doing.
Do not open many windows at once
prefer text and command utilities rather than graphical interfaces.
I am on a VPS server (Quad core with 768MB of RAM. I am running Apache and using APC as my default caching engine) . I get around 2000 visitors per day with a maximum of 100 concurrent visitors.
According to CakePHP DebugKit, my most memory intensive pages use around 16MB and the other pages average at around 10MB.
Is this memory usage normal?
Is it normal for my memory to bottleneck at 2000 visitors per page?
Should I consider upgrading my plan to 2GB RAM?
I also noticed that the View rendering is taking up most of the memory, around 70% on most pages.
I am monitoring my resource usage, when I have around 50 or more concurrent users, I am getting 0 free MB left.
Thank you
You should also check your other processes. From my experience, MySQL takes up more memory than anything else on any stack that I run. You should also implement better page caching so that PHP doesn't need to be touched when it isn't absolutely necessary. But Apache is also a memory hog that needs to be fine tuned. If you want to stick with Apache, then run Varnish in front of it.
Also, keep APC, but also add Memcached. It's much faster.
If your site has a spike-load that brings it to zero memory, then, if you can, consider launching extra instances of the server and doing a sort of round-robin (if the VPS is cloud-hosted and this is possible). If the load is constant, then definitely upgrade.
#burzum is completely right, however, that you should just switch to nginx. It's far, far better than Apache at this point. But, just to put you on the right track, quite a few people run nginx as a reverse proxy in front of Apache, and while that does speed up the server, it's entirely unnecessary because nginx can do pretty much anything you need it to do without Apache. Don't bother using Varnish in front of nginx either because nginx can act as its own reverse proxy.
Your best bet is to implement nginx with apcu (upgrade php to 5.5 if possible for better performance) and use memcached, and implement nginx's native microcaching abilities. If your site is heavy on read and light on write, then you might notice that everything is taken care of by nginx just retrieving a cached copy from memcache. While this helps with memory, it also helps with processing. My servers' CPUs usually have a 3-5% usage during peaks.
Is this memory usage normal?
Yes, it doesn't seem to be high for a normal CakePHP application.
Is it normal for my memory to bottleneck at 2000 visitors per page?
I guess yes, I'm not a "server guy".
Should I consider upgrading my plan to 2GB RAM?
I would try switching to Nginx first. Apache is a memory eating monster compared to Nginx, just search for a few banchmarks, a random one I've picked by a quick search is from this page.
Overall Nginx should provide you a much better performance. At some stage I would consider updating the memory but try Nginx first.
I need to speed up my magento installation, so I'm planning to put the content of 'var/' (or only var/cache and var/sessions) on a tmpfs.
I'm also buying a reserved instance on Amazon, so I would like to keep a sufficent amount of RAM. I want to enable memcached, PHP Apc, MySQL caching and HTTP caching.
I'm thinking of a Medium Reserved Instance with the following specs:
3.75 GB memory
2 EC2 Compute Unit (1 virtual core with 2 EC2 Compute Unit)
410 GB instance storage
32-bit or 64-bit platform
I/O Performance: Moderate
EBS-Optimized Available: No
API name: m1.medium
Will the RAM be enough to appy a good caching system?
Looking now (after 3 months) the var directory is 14gb, but I think cleaning it up each 5/7 days would be good too.
Do you have any suggestion for me?
P.S. the store will contain an average of 100/150 products.
I think moving /var to a tmpfs is probably not your biggest bottleneck and would likely be more trouble than its worth. Make sure Magento caching is enabled and you have APC enabled.
This post covers some general tips on increasing Magento performance:
Why is Magento so slow?
I would suggest looking into setting up a reverse proxy like Varnish.
Getting Varnish To Work on Magento
If you do plan on just using a tmpfs in memory I would suggest looking into Colin's improved over Zend_Cache_Backend_File
https://github.com/colinmollenhour/Cm_Cache_Backend_File
Also I would suggest looking into mytop to keep tabs of if you have any places you can optimize queries in the application itself or in my.cnf to help ease any DB bottlenecks.
http://jeremy.zawodny.com/mysql/mytop/
Session Digital has a good white paper (although somewhat dated) on optimizing Magento enterprise and the same can be applied to Community. Out of everything I've tried, Varnish, as mentioned in the White paper offered the most significant increase in response time.
http://www.sessiondigital.com/resources/tech/060611/Mag-Perf-WP-final.pdf
Hope this helps!
Firstly, +1 to all of the answers here.
if you're thinking about running /var/ out of tmpfs it's probably because you've heard of the lousy file IO on AWS or you have experienced issues with it yourself. However, the /var/ directory is the least of your concern - Zend / Magento's autoloaders are more taxing to IO. To mitigate that you want to run APC and the compilation (assuming you're not using persistent shopping cart).
As echoed by other commenters, anything that runs from cache or memory will circumvent PHP and thus the need to touch the disk and incur IO issues. Varnish is a bit of a brute-force approach and is a wonderful tool on massive sites that scale to millions of views; but I believe that Varnish's limitations with SSL and the lack of real documentation and support from our Magento community make it a better intellectual choice than an actual alternative.
When running Magento Community I prefer to run Tinybrick's Lightspeed on AWS on a Medium instance - which gives me the most bang-for-buck and is itself a full-page-cache. I get 200+ concurrent pages/second in this setup and I'm not running memcached or using compilation.
http://www.tinybrick.com/improve-magentos-slow-performance.html/
Be careful with running memcached in your AWS instance as well - I find that it can be impeded by a power-hungry Apache gone wild in the rare instance you haven't got a primed cache which causes Apache maxclients issues while it waits for cache response. If you could afford it I would rather run two micro Apache instances with a shared memcached session store and a load balancer in front of them - give some horsepower to the db on a separate box for them to share, though. But all setups are unique and your traffic/usage will dictate what you need.
I have run Magento in the AWS cloud for 3 years with great success - and I wish the same to you. Cheers.
I have an installation of 1.3.2.4 running two Store Views and 2,734 products. The site sees around 15,000 visits a month.
Apache and MySQL (mostly Apache) hovers at around 1.5 GB RAM usage most of the time and peaks over 3 GB. My questions is, considering the stats, is this normal? Seems like a lot.
If that memory usage is in fact abnormal, would an upgrade to 1.4.1.1 help?
If you consider your stores, then you are doing just fine. But regarding the traffic you're getting, it seems that you need to provide some extra features to Magento to let it fire up. For this, you can have some of the following:-
Install APC (Alternative PHP Cache) or XCache (or any other alternative) and configure the use of it in your Magento back-end. It dramatically increases the speed of Magento.
You can have Magento's cache stored in memory (tmpfs in Linux).
You can also tell Magento to save sessions into Memcache so that your sessions are in memory & distributed.
Check your Magento's Index Management section for any requirement of indexes, every month or bi-monthly. If you do find any indexing required, then do it immediately & clear the cache from your Cache Management.
Check your database every week or bi-monthly for any overhead in any of your database's tables. If you do find any overhead, then "optimize" those tables immediately.
Try reading some of these articles, to know more about these.
Also, upgrading to 1.4.1.1 will help you out in terms of features provided by Magento. But for performance, I think it will be best to wait for some more time, until Magento releases its version 2 in the market, in which some performance issues may be taken care of by Magento.
Hope it helps.
1.3.2.4 is a good stable release, upgrading to 1.4.0.1 is very painless and will give you the added benefit of split index management and much faster administration area (mass attribute update is fixed).
Don't be overly concerned about memory usage, depending on the number of Apache modules you have loaded, you should expect to see about 30MB per child. As long as your not swapping or encroaching your limits, you shouldn't have any real concerns over how much is being consumed. Disabling unused modules will help cut down memory - but to be honest, not by any noticeable margin.
You could always throw Nginx in front as a reverse proxy to serve static content requests and pipe PHP/dynamic reqs. back to Apache. That way you can keep the modular Apache build with .htaccess support and cut down your memory overheads significantly.
However, this could do with more information, such as the output from
free -m
To see how some of the memory is being allocated.
I'd probably suggest downloading tuning-primer.sh to run on your MySQL config. It will give a good (entry-level) indication of how efficient your memory allocation is.
Those stats look quite typical for Magento, if you consider a single hit/page load can use upwards of 64MB RAM.
Your Apache settings can also drastically effect the amount of RAM your system uses. Upgrading your Magento installation may give some small performance boost, but don't expect it to do much for memory consumption etc.
If your memory consumption is a real issue for you then you have several possible routes to reduce resource usage, such as:
Install Nginx as a reverse caching proxy to apache (apache is a hog and is poor serving static content).
Use Nginx + PHP Fast CGI and remove apache
Try using worker MPM module for apache, or Fast CGI.
Install caching proxy such as Varnish/Squid.
If you are stuck with apache you can tweek KeepAlive and other settings to allow you to reduce memory usage
Tweek MySQL settings, such as query caching to imporove resource usage / performance
I have found 1. to work very well in reducing cpu/memory usage as it will allow Nginx to serve static images etc without requiring apache to hog RAM trying to serve them.
The framework I am using is called SocialEngine.net v4, and it's completely written in Zend, so it's insanely super CPU intensive. SocialEngine is in PHP and uses MySQL.
I need to know what OS, what hardware you suggest (dual xeons, amd, how much ram, etc...) and how to optimize it properly to handle large amounts of traffic.
I only have 11k users right now, and it's running incredibly slow, I'm talking 7 second page load times.
The framework however does have memcached, and apc options for caching installed, but even with APC or Memcache on, it doesn't make a big enough difference...
I need to know what the best way to attack this is as far as optimizing mysql, inoodb tweaks, apache tweaks, any performance tweaks, what type of hardware, and amount of ram.
I have a very big marketing plan in place, and will probably start increasing traffic by 1,000+ signups per day... So traffic will start to rise very progressively. When I initially marketed, I did 50k uniques in 6 hours, 20k signups, and 500k pageviews... (server crashed, lost half my users... and haven't marketed since, because I been trying to rebuild)
You could start with xdebug to profile your application and find the bottleneck
Honestly? And this is just my opinion, instead of spending a small fortune on a single server - buy many small servers and load balance them. Mac Mini's are wonderful for this and are capable of running their standard OS X or Linux if you choose. You will get way more performance out of 10 small $500 machines than you will out of 1 $5000 machine.
You don't provide us with any information about your set up.
How many servers do you have? What services are they running?
When you say APC and Memcached is on, have you actually set them up to actually work?
How many connections does your Apache allow for?
What is your MySQL configuration like? Is the memory settings optimised? Most importantly are all your tables indexed properly? Have you checked your slow query log? Did you run EXPLAIN for your queries?
ZF wise, are you caching your table metadata? Are you caching tables that don't change so that you save network traffic? Have you checked the official ZF optimisation guide?
Also... Why did you assume that ZF is killing your CPU usage?