I'm currently working on a social web application using python/django. Recently I heard about the PHP's weakness on large scale projects, and how hippo-php helped Facebook to overcome this barrier. Considering a python social web application with lot of utilization, could you please tell me if a similar custom tool could help this python application? In what way? I mean which portion (or layer) of application need to be written for example in c++? I know that it's a general question but someone with relevant experience I think that could help me.
Thank you in advance.
The portion to rewrite in C++ is the portion that is too slow in Python. You need to figure out where your bottleneck is, which you can do by load testing or just waiting until users complain.
Of course, even rewriting in C++ might not help. Your bottleneck might be the database (move to a separate, faster DB server or use sharding) or disk, or memory, or anything. Find bottleneck, work out how to eliminate bottleneck, implement. With 'test' inbetween all those phases. General advice.
There's normally no magic bullet, and I imagine Facebook did a LOT of testing and analysis of their bottlenecks before they tried anything.
Don't try to scale too early! Of course you can try to be prepared but most times you can not really know where you need to scale and therefore spend a lot of time and money in wrong direction before you recognize it.
Start your webapp and see how it goes (agreeing with Spacedman here).
Though from my experience the language of your web app is less likely going to be the bottleneck. Most of the time it starts with the database. Many times it simply a wrong line of code (be it just a for loop) and many other times its something like forgetting to use sth. like memcached or task management. As said, find out where it is. In most cases its better to check something else before blaming the language speed for it (since its most likely not the problem!).
You can think about PostgreSQL as Oracle, so from what I've found on the internet (because I am also a beginner) here is the order of DBs from smaller projects, to biggest:
SQLite
MySql
PostgreSQL
Oracle
Related
I have a rather obscure question, which I'll probably get shot down for asking, but here goes.
I have just implemented New Relic web application monitoring for my site which is made up of roughly 70% legacy Perl code and 30% newer PHP code. New Relic gives some really nice information for slow SQL which I'd like to be able to analyse.
The problem is that New Relic does not provide Perl support, so I can only see 30% of my applications query statistics.
As a workaround, I had a pretty insane idea whereby I could potentially prepare my SQL in Perl as normal, then exec/shell out to a PHP script which executes the SQL using its built-in interface, then return the results back to perl somehow. (I have no idea if this is even possible). This way, I'd be able to see the other 70% of SQL statistics that I am currently unable to see.
I've heard about the PECL Perl extension but have yet to look into it in any detail. Does anyone know if/how I could accomplish something like this? Or suggest any alternative Perl/SQL monitoring tools I could use alongside new relic?
Regards,
ns
Thanks for your pointers.
I was blue-sky thinking when I asked this question. I have since realised that performance is likely to take a big hit, and it is an ugly solution to implement. I will instead try working with alternative tools alongside new relic.
Cheers! ns
I am writing a site in PHP and I am noticing that the page is taking 3-5 seconds to load (from a remote server), which is unacceptable. The software relies on around 12 classes to function correctly. I was wondering how much of a performance gain I would get if I rewrote most of the classes to just use regular php functions.
Thanks for any input.
Edit: I rely primarily on Redis, and using a simple MySQL query here and there.
Functions or classes should make little to no difference (totally negligible) : that's not what is making your website / application slow.
Hard to give you more information, as we don't know how your setup looks like, but you might want to take a look at the answer I posted to this question : it contains some interesting ideas, when it comes to performances of a PHP application.
BTW: 12 classes is really not a big number of classes, if I may...
Rewriting all the application in procedural programming is probably the worst thing you can ever do. Object oriented programming is not about performance gain but writing programmer-friendly and easily maintainable applications (among others).
You should not ever think about rewriting an OO application procedural. That's not the purpose. If you ever have bigger resource consumption using OO rather than procedural programming (and that is very unlikely) you should probably think about better scaling the app. Hardware nowadays is not that expensive.
On the other hand, your application has many possible bottlenecks and OO is probably not even on the list.
Did you check:
your Internet connection?
your server's Internet connection?
your ping loss to your server?
your server's configuration? (Apache/Nginx/Lighttpd or whatever it is)
your database server's configuration?
your database queries?
your server's load?
the timing for a connection to Redis?
your firewall's rules for the ports used by Redis?
your Redis' configuration? (maxclients, timeout)
If you answered NO to at least one question above, please do check that and if the problem persists, let me know!
The difference will probably not even be measurable.
Your problem most definitely isn't your code per se, but the way you access the database. Make sure your tables are appropriately indexed and you will see a substantial drop in page load times. You can use EXPLAIN SELECT ... to get further info on how a query actually runs and why it performs badly.
You wont find much of a difference, if anything at all between functions and classes.
Chances are, one or more of your classes is written inefficiently, or relies on something that is. Examples could include waiting on a remote server, image processing, some databases (such as overloaded databases) or countless other methods. You should try to profile your code to see where the problem lies.
Hi~ we are working on a small project which we want it to be public on line. Now our project runs well in the local, However when we just hoop it on line, the respond speed is so slow. we are all new to web development area, so Please provide some tips or suggestions .Thank you.
We use php, mysql as database and nginx as server.
Any advice is more than welcome. Thank you again.
A few tips:
Code Profiling
Use xdebug's profiling option. This will give you insight into areas of your code that may need optimization.
SQL Profiling
Most of my projects use PostgreSQL, so I haven't actually used this myself, but here's some documentation on profiling your MySQL queries. Again, it'll give you insight into areas of your SQL scripts that could use some optimization.
Investigate Cache Options
If you have a lot of database queries on your pages or a lot of PHP logic in your pages, it can be expensive to generate. There are a few different methods of cacheing and, while each method helps to solve load times, each aims to solve different specialized problems:
File based cache (article that explains the concept fairly well)
APC cache
Memcached
MySQL query cache
These would be the first steps that I'd take in optimizing one of my projects.
Edit:
TVK's suggestion is also good for nailing down client side performance issues.
If the problem is only occurring when you have to transfer the actual data, this addon by Google will be great for finding things to improve: http://code.google.com/speed/page-speed/
It will perform a bunch of algorithms to find things to optimise in your HTML as well as CSS and JavaScript.
We have been using PHP/MySQL for our web application which has been growing a lot, the database is around 4-5GB and one of the table is 2GB sometimes, hence slowing down whenever any queries to that table is called.
Should we just try to optimize, or are we using MySQL above its limit? Will switching our web app to .NET/SQL Server resolve the issues?
You're going to get a lot of very passionate responses to this.
PHP is, from a code and performance standpoint, very similar to classic ASP. ASP.NET v1 was , according to many, many benchmarks available via your favorite search engine, 3x-5x faster than classic ASP. Draw your own conclusions.
I feel that MSSQL is a superior database solution. If you're stuck with open source, at least look at Postgres. It's less popular but very powerful.
To answer your real question: performance is a function of your toolset and platform choice, but also of developer skill and project structure. I've seen far more projects that could benefit from some healthy refactoring and optimization than I have that are limited by the platform in which they are written. It is rarely worthwhile to rewrite a large application in a completely different language. Instead, I would focus on improving your existing codebase, and looking for ways to incrementally upgrade to a platform like ASP.NET.
Also keep in mind that switching will require you to jump to IIS Windows server and there will be more cost involved most likely. There are a lot of considerations here when thinking about a switch like this.
I say if the application calls for it, work it out.
You certainly aren't using MySQL above it's limit. But you should consider benchmarking your database queries on MSSQL to see if you notice a huge improvement.
There are many factors involved here, your code base, database optimisations & changes to table structure, server spec.... they all contribute independently.
Are there any particularly slow queries or is it running slow accross the application? Can any caching be implemented here? Do you have propper indexes?
I am thinking of using a PHP framework called CodeIgniter.
One of the things I am interested in is its speed. I have, however, no way to find out how fast it is, and would rather not simply take the word of their website for it. Does anybody know how I can determine its speed myself, or can someone tell me of a site that can?
Code Igniter also has some built-in benchmarking tools:
http://codeigniter.com/user_guide/general/profiling.html
Yes, the problem is you have to build your application to profile it.
At work we had a couple of projects written outside which we load-tested before putting them on our main boxes. We were quite surprised to find critical performance problems with both; one was written in CakePHP and the other was written using Drupal. I don't think this highlights a problem with any framework or CMS other than the need to do profiling and load-testing on any site which is going to get significant traffic. In both cases it was what the developer had done, rather than the characteristics of the software platform, that caused the problem. For example, there was a recursive function call the developer had created in the Cake project which instantiated the entire Cake object every recursion and this would have taken out the server had it gone live under load.
In my opinion performance should not be a deciding factor in choosing a framework; the objective differences are likely to be marginal and the way you use it is likely to cause far more performance problems than the inherent performance of the framework.
I believe that to scale any PHP application to run under load, you will need an opcode cache and you'll need to write in intelligent content caching using something like memcached or whatever built-in caching your framework supports.
If your site is database-driven I would be very surprised if your bottleneck would be the application framework. "Fast" as in faster development is what I would worry about rather than "fast" as in speedy handling of requests. Significant optimization is better done by caching strategies and optimizing your database access.
Besides database access your own code will be where most of the time for each request is spent (and even that is usually not significant compared to database access), the framework will likely not be affecting the time spent on a request, unless it is really badly written.
It way be better to look for a framework which has good caching support (which Code Igniter may have, I don't know), that will almost always save you more time than the few milliseconds you could shave off the request handling by using a slightly faster framework.
Have a look at the Zend Framework too, it has the benefit of being PHP 5, whereas Code Igniter is still PHP 4, as I understand it. That may be an issue when it comes to speed, but in favor of which framework I don't know. Zend has good caching support and a database profiler that can help you find where your bottlenecks are.
i'd recommend testing it for yourself. use xdebug's profiler to create a cachegrind compatible file and webgrind to visualize the file.
that way you end up with very reliable information.
CodeIgniter is plenty fast for most projects. Some have posted here and if you Google, you will find that it compares favorably to other frameworks with respect to speed.
I would agree with another poster that performance is usually not a big concern when it comes to framework choice. The major frameworks all have sufficient performance for most projects.
I maintain a site that gets slammed a few times a year. Last year the development team rewrote the entire site using Codeigniter and we have had much luck in terms of performance. Additionally, the time it took to perform the rewrite was minimal as this framework is quite easy to work with. CakePHP in my opinion is also a good choice if you find that you don't like Codeigniter.
For CodeIgniter and other PHP frameworks, PHP Quick Profiler is very handy for benchmarking and measuring speed especially for database queries. You must check this out:
php-quick-profiler
It's very easy to install and provides an awesome GUI for examine different benchmarking tests.