say if I have 10 tables. These tables are joined together on different conditions. Would it be better to go through all of these conditions with php and then execute what you end up with, or just pull all 10 tables and use the appropriate data. Which would be faster? Thanks.
It really depends on what your doing, something PHP is going to be light speeds faster and other MySQL is going to be. If there are going to be a lot of read/writes from the database I'd generally say MySQL is faster, but if you're doing anything with arrays of the data PHP is most likely going to be faster.
Almost certainly doing it in mysql would be faster. mysql is optimized for this kind of thing; php isn't. And doing it in mysql avoids having to serialize all of the data to transfer it between the two processes.
Beware premature optimization, though. Write it whatever way is easier and optimize later if its performance is a problem.
Related
So I've been developing an application that needs to compare a lot of data. The max (I'm assuming) at once will be about 28800 rows that need to be compared against another table separately.
So what I'm doing right now is making 28800 ajax calls to a database and it works, but it's obviously not the fastest procedure in the world.
Would it be faster to iterate over an array of about say, 80,000 elements 28800 times and just making 2 db calls?
I'm assuming it is not, but just figured I'd ask to be sure.
For a little background knowledge, check out an earlier question I posted:
How to avoid out of memory error in a browser due to too many ajax calls
Thank you!
Unless there is some very specific reason you are making that many calls it sounds like thre must be an easier way to retrieve all this data, have the database do some of the work, or rethink what is happening.
Can you write a database script that is passed some params and does the comparison?
Chances are the database can work faster than looping over all that data 28800 times as the database uses other methods for comparing and retrieving data. You are in a way comparing brute force looping to a database engine.
Can your write a query that does the comparison?
It sounds like you are pulling the data and then doing more work to it, can you pull the data you need with a single query?
The general rule that I haver found is that MySQL is massively fast on doing simple queries, especially when indexed, and abominably slow on complex ones involving multiple joins or nested sub queries.
This doesn't actually answer your question though, because its too vague. Mysql retrieval is critical with respect to what indexes are applied. It is also critical to the complexity of the query.
So there are no general rules. You must try both and see which is better.
Likewise javascript performance varies wildly between browsers.
A final comment: some things that are simple to define in procedural code, like checking the result of one query to formulate the shape of the next, are massively easier to code than to work out what exactly the optimal SQL syntax should be to combine them into a single query, and even then, you may find that the mysql server is building massive temporary tables to do it.
I'm building a PHP app which will get a lot of traffic. I have always learned that I should limit the number of SQL queries (right now it's about 15 per visitor). I could limit this to about half, using PHP to filter and sort data, but would this be faster? Alternatively, I could use "JOIN" in the query to reduce the number of queries, but is this really faster than executing multiple queries? Thanks!
If you have 15 queries per visitor, you most likely did something wrong, unless your application is pretty big.
Using PHP to sort and filter data instead of MySQL
Doing the sorting and filtering in PHP will not make your application faster, it will make it slower for sure, MySQL is optimized to be very fast given the right indexes, so you should certainly use it when you can.
Learn about database indexing and it will make your life easier and increase your application's performance. Here is a link :
http://www.mysqltutorial.org/mysql-create-drop-index.aspx
Joining versus multiple queries
You ask if using join's would be faster than executing multiple queries.
The answer is yes, it will always be faster to use join's given the right indexes. Which also comes back to the first topic, indexing.
However, keep in mind that joining is as efficient as you make it to be, it will be extremely efficient if done right, and extremely slow if done wrong, so you must know the basics to do it right.
You can look here for an introduction to join's :
http://blog.sqlauthority.com/2009/04/13/sql-server-introduction-to-joins-basic-of-joins/
I am writing a fairly simple webapp that pulls data from 3 tables in a mysql database. Because I don't need a ton of advanced filtering it seems theoretically faster to construct and then work within large multi-dimensional arrays instead of doing a mysql query whenever possible.
In theory I could just have one query from each table and build large arrays with the results, essentially never needing to query that table again. Is this a good practice, or is it better to just query for the data when it's needed? Or is there some kind of balance, and if so, what is it?
PHP arrays can be very fast, but it depends on how big are those tables, when the numbers get huge MySQL is going to be faster because, with the right indexes, it won't have to scan all the data, but just pick the ones you need.
I don't recommend you to try what you're suggesting, MySQL has a query cache, so repeated queries won't even hit the disk, so in a way, the optimization you're thinking about is already done.
Finally, as Chris said, never think about optimizations when they are not needed.
About good practices, a good practice is writing the simplest (and easy to read) code that does the job.
If in the end you'll decide to apply an optimization, profile the performance, you might be surprised, by unexpected results.
it depends ...
Try each solution with microtime function and you'll seethe results.
I think a MySQL Query cache can be a good solution. and if you've filtering on , you can create view.
If you can pull it off with a single query - go for it! In your case, I'd say that is a good practice. You might also consider having your data in a CSV or similar file, which would give you even better performance.
I absolutely concur with chris on optimizations: the LAMP stack is a good solution for 99% of web apps, without any need for optimization. ONLY optimize if you really run into a performance problem.
One more thought for your mental model of php + databases: you did not take into account that reading a lot of data from the database into php also takes time.
I've been doing a lot of calculating stuff nowadays. Usually I prefer to do these calculations in PHP rather than MySQL though I know PHP is not good at this. I thought MySQL may be worse. But I found some performance problem: some pages were loaded so slowly that 30 seconds' time limit is not enough for them! So I wonder where is the better place to do the calculations, and any principles for that? Suggestions would be appreciated.
Anything that can be done using a RDBMS (GROUPING, SUMMING, AVG) where the data can be filtered on the server side, should be done in the RDBMS.
If the calculation would be better suited in PHP then fine, go with that, but otherwise don't try to do in PHP what a RDBMS was made for. YOU WILL LOSE.
I would recommend doing any row level calculations using the RDBMS.
Not only are you going to benefit from better performance but it also makes your applications more portable if you need to switch to another scripting language, let's say PHP to Python, because you've already sorted, filtered and processed the data using your RBDMS.
It also helps separate your application logic, it has helped me keep my controllers cleaner and neater when working in an MVC environment.
i would say do calculations in languages that were created for that, like c++. But if you choose between mysql and php, php is better.
Just keep track of where your bottlenecks are. If your table gets locked up because you're trying to run some calculations, everyone else is in queue waiting to read/write the data in the selected tables and the queue will continue to grow.
MySQL is typically faster at processing your commands, but PHP should be able to handle simple problems without too much of a fuss. Of course, that does not mean you should be pinging your database multiple times for the same calculation over and over.
You might be better off caching your results if you can and have a cron job updating it once a day/hour (please don't do it every minute, your hosting provider will probably hate you).
Do as much filtering and merging as possible to bring the minimum amount of data into php. Once you have that minimum data set, then it depends on what you are doing, server load, and perhaps other factors.
If you can do something equally well in either, and the sql is not overly complex to write (and maintain) then do that. For simple math, sql is usually a good bet. For string manipulations where the strings will end up about the same length or grow, php is probably a good bet.
The most important thing is to request as little data as possible. The manipulation of the data, at least what sql can do, is secondary to retrieving and transferring the data.
Native MySQL functions are very quick. So do what makes sense in your queries.
If you have multiples servers (ie, a web server and a DB server), note DB servers are much more expensive then web servers, so if you have a lot of traffic or a very busy DB server do not do the 'extras' that can be handled just as easily/efficiently on a web server machine to help prevent slowdowns.
cmptrgeekken is right we would need some more information. BUT if you are needing to do calculations that pertain to database queries or doing operations on them, comparisons certian fields from the database, make the database do it. Doing special queries in SQL is cheape r(as far as time is concerned and it is optimized for that) But both PHP and MySQL are both server side it won't really matter where you are doing the calculations. But like I said before if they are operations on with database information, make a more complicated SQL query and use that.
Use PHP, don't lag up your MySQL doing endless calculations. If your talking about things like sorting its OK to use MySQL for stuff like that, SUM, AVG but don't overdo it.
Was curious to know which is faster - If i have an array of 25000 key-value pairs and a MySQL database of identical information, which would be faster to search through?
thanks a lot everyone!
The best way to answer this question is to perform a benchmark.
Although you should just try it out yourself, I'm going to assume that there's a proper index and conclude that the DB can do it faster than PHP due to being built to be all about that.
However, it might come down to network latencies, speed of parsing SQL vs PHP, or DB load and percentage of memory use.
My first thought would be it is faster searching with arrays. But, on the other side it really depends on several factors:
How is your databasa table designed (does it use indexes properly, etc)
How is your query built
Databases are generally pretty optimized for such searches.
What type of search you are doing on the array? There are several type of searches you could do. The slowest is a straight search where you go through each row an check for a value, a faster approach is a binary search.
I presumed that you are comparing a select statement executed directly on a database, and an array search in php. Not
One thing to keep in mind: If your search is CPU intensive on the database it might be worth doing it in PHP even if it's not as fast. It's usually easier to add web servers than database servers when scaling.
Test it - profiling something as simple as this should be trivial.
Also, remember that databases are designed to handle exactly this sort of task, so they'll naturally be good at it. Even a naive binary search would only have 17 compares for this, so 25k elements isn't a lot. The real problem is sorting, but that has been conquered to death over the past 60+ years.
It depends. In mysql you can use indexing, which will increase speed, but with php you don't need to send information through net(if mysql database on another server).
MySQL is built to efficiently sort and search through large amounts of data, such as this. This is especially true when you are searching for a key, since MySQL indexes the primary keys in a table.