I am writing a fairly simple webapp that pulls data from 3 tables in a mysql database. Because I don't need a ton of advanced filtering it seems theoretically faster to construct and then work within large multi-dimensional arrays instead of doing a mysql query whenever possible.
In theory I could just have one query from each table and build large arrays with the results, essentially never needing to query that table again. Is this a good practice, or is it better to just query for the data when it's needed? Or is there some kind of balance, and if so, what is it?
PHP arrays can be very fast, but it depends on how big are those tables, when the numbers get huge MySQL is going to be faster because, with the right indexes, it won't have to scan all the data, but just pick the ones you need.
I don't recommend you to try what you're suggesting, MySQL has a query cache, so repeated queries won't even hit the disk, so in a way, the optimization you're thinking about is already done.
Finally, as Chris said, never think about optimizations when they are not needed.
About good practices, a good practice is writing the simplest (and easy to read) code that does the job.
If in the end you'll decide to apply an optimization, profile the performance, you might be surprised, by unexpected results.
it depends ...
Try each solution with microtime function and you'll seethe results.
I think a MySQL Query cache can be a good solution. and if you've filtering on , you can create view.
If you can pull it off with a single query - go for it! In your case, I'd say that is a good practice. You might also consider having your data in a CSV or similar file, which would give you even better performance.
I absolutely concur with chris on optimizations: the LAMP stack is a good solution for 99% of web apps, without any need for optimization. ONLY optimize if you really run into a performance problem.
One more thought for your mental model of php + databases: you did not take into account that reading a lot of data from the database into php also takes time.
Related
I'm working on a website that is supposed to show the specifications of computers, for example: Cpu, cpu speed, ram and stuff like that. There are a lot of fields and I wonder whether the php's mysql_fetch_array would be faster or saving all the data in one varchar field and separating it using php? also wondering if there are any pros and cons to either of them?
I'm using php 5.3 and mySQL
Thanks in advance.
If I were you I'll use A database but Everything in separates field and tables (DB relational) with a Entity relationship Diagram + normalization. Doing that you will have scalability, Independence, avoid redundancy, beside if you add good INDEX (PK,FK and UK) you could have a good performance. Of course that will also depend of your queries and your PHP code. But that's a good practice to implement the right DB design plus security stuff.
The pros are A LOT... the cons, It will take you more time but it will worth it.
I doubt the speed difference is ever going matter. What will matter is whether or not you have a regular structure and the ability to freely query your data. Put it in the database & figure out how to speed things up later when/if it ever becomes an issue, rather than making poor design decisions up front because you think you might need more performance.
say if I have 10 tables. These tables are joined together on different conditions. Would it be better to go through all of these conditions with php and then execute what you end up with, or just pull all 10 tables and use the appropriate data. Which would be faster? Thanks.
It really depends on what your doing, something PHP is going to be light speeds faster and other MySQL is going to be. If there are going to be a lot of read/writes from the database I'd generally say MySQL is faster, but if you're doing anything with arrays of the data PHP is most likely going to be faster.
Almost certainly doing it in mysql would be faster. mysql is optimized for this kind of thing; php isn't. And doing it in mysql avoids having to serialize all of the data to transfer it between the two processes.
Beware premature optimization, though. Write it whatever way is easier and optimize later if its performance is a problem.
I'm a pretty new PHP developer yet have some good experience with T-SQL so I'm more than capable of building me some nice SQL queries. I'm building a PHP app and have ran into this issue on a number of occasions.
When I'm against a problem where I need to pull data from a MySQL database and process the output via PHP should I simply "*" out the whole result set and pick out what I need via PHP? IMO it makes more sense to put as much work in SQL as possible but I dunno.
In regards of performance and "best practice" what is generally best; rely on MySQL to do the big part of the work and let PHP pick up the output or let PHP do the majority of the work?
You should generally never apply the wild-card to grab all columns. It is far better practice to only retrieve the columns you are interested in using. This has implications on:
Performance: less overhead by only fetching and retrieving data that is required
Maintainability: it is much more obvious what your query is fetching. I.e., if you add additional columns to your structure you might be getting data you originally didn't expect.
If you only don't need all of the columns from the table, specify the ones you want explicitly. This is better performance-wise, not to mention that it's more readable.
SQL will almost always produce a better performance over PHP, but it really depends on the query being run. The more complex it is, the more likely SQL will be a better answer. SQL was built around 'getting you the right data quickly' whereas PHP was built around being a general purpose scripting language.
In general it's considered best practice to return specific columns. Performance is not a huge concern, unless you're dealing with tables that contain BLOBs (also consider if the table might have one in the future), but you might find it easier to deal with smaller return objects.
Of course, if you're in development mode, rather than maintenance mode, you might find SELECT * preferable if you're frequently adding or removing columns.
In my experience the access time to gather a * query or a specific field query is dwarfed by the over all request time, at least if this is done over the web.
IE in my testing with websites hosted both in the US and overseas I generally see that the difference between select fld,fld,fld and select * is negligible unless you have large table rows and lots of rows.
I personally think this is somewhat of a judgement call based on the size of your system, tables and if your tables and columns are in any kind of a state of flux. If I were you, not knowing the size of your system I'd go with select * but clearly it is not the "best practice" but I think it really needs to be on a specific system-by-system basis.
I'm currently a member of a team on a HUGE C#, WCF, WPF, SQL project and everywhere they went with the "best practices" and what we have is a huge mess. In some cases so much effort was spent to follow the best practice that the result has been far more work to maintain it. We have interfaces by the bazillions as well as wrappers, adapters, unity etc. Right now to make one small change it touches 15 - 20 files. I realize this is slightly off topic but don't blindly follow the best practice just because its there.
Okay, so I'm sure plenty of you have built crazy database intensive pages...
I am building a page that I'd like to pull all sorts of unrelated database information from. Here are some sample different queries for this one page:
article content and info
IF the author is a registered user, their info
UPDATE the article's view counter
retrieve comments on the article
retrieve information for the authors of the comments
if the reader of the article is signed in, query for info on them
etc...
I know these are basically going to be pretty lightning quick, and that I could combine some; but I wanted to make sure that this isn't abnormal?
How many fairly normal and un-heavy queries would you limit yourself to on a page?
As many as needed, but not more.
Really: don't worry about optimization (right now). Build it first, measure performance second, and IFF there is a performance problem somewhere, then start with optimization.
Otherwise, you risk spending a lot of time on optimizing something that doesn't need optimization.
I've had pages with 50 queries on them without a problem. A fast query to a non-large (ie, fits in main memory) table can happen in 1 millisecond or less, so you can do quite a few of those.
If a page loads in less than 200 ms, you will have a snappy site. A big chunk of that is being used by latency between your server and the browser, so I like to aim for < 100ms of time spent on the server. Do as many queries as you want in that time period.
The big bottleneck is probably going to be the amount of time you have to spend on the project, so optimize for that first :) Optimize the code later, if you have to. That being said, if you are going to write any code related to this problem, write something that makes it obvious how long your queries are taking. That way you can at least find out you have a problem.
I don't think there is any one correct answer to this. I'd say as long as the queries are fast, and the page follows a logical flow, there shouldn't be any arbitrary cap imposed on them. I've seen pages fly with a dozen queries, and I've seen them crawl with one.
Every query requires a round-trip to your database server, so the cost of many queries grows larger with the latency to it.
If it runs on the same host there will still be a slight speed penalty, not only because a socket is between your application but also because the server has to parse your query, build the response, check access and whatever else overhead you got with SQL servers.
So in general it's better to have less queries.
You should try to do as much as possible in SQL, though: don't get stuff as input for some algorithm in your client language when the same algorithm could be implemented without hassle in SQL itself. This will not only reduce the number of your queries but also help a great deal in selecting only the rows you need.
Piskvor's answer still applies in any case.
Wordpress, for instance, can pull up to 30 queries a page. There are several things you can use to stop MySQL pull down - one of them being memchache - but right now and, as you say, if it will be straightforward just make sure all data you pull is properly indexed in MySQL and don't worry much about the number of queries.
If you're using a Framework (CodeIgniter for example) you can generally pull data for the page creation times and check whats pulling your site down.
As other have said, there is no single number. Whenever possible please use SQL for what it was built for and retrieve sets of data together.
Generally an indication that you may be doing something wrong is when you have a SQL inside a loop.
When possible Use joins to retrieve data that belongs together versus sending several statements.
Always try to make sure your statements retrieve exactly what you need with no extra fields/rows.
If you need the queries, you should just use them.
What I always try to do, is to have them executed all at once at the same place, so that there is no need for different parts (if they're separated...) of the page to make database connections. I figure it´s more efficient to store everything in variables than have every part of a page connect to the database.
In my experience, it is better to make two queries and post-process the results than to make one that takes ten times longer to run that you don't have to post-process. That said, it is also better to not repeat queries if you already have the result, and there are many different ways this can be achieved.
But all of that is oriented around performance optimization. So unless you really know what you're doing (hint: most people in this situation don't), just make the queries you need for the data you need and refactor it later.
I think that you should be limiting yourself to as few queries as possible. Try and combine queries to mutlitask and save time.
Premature optimisation is a problem like people have mentioned before, but that's where you're crapping up your code to make it run 'fast'. But people take this 'maxim' too far.
If you want to design with scalability in mind, just make sure whatever you do to load data is sufficiently abstracted and calls are centralized, this will make it easier when you need to implement a shared memory cache, as you'll only have to change a few things in a few places.
Was curious to know which is faster - If i have an array of 25000 key-value pairs and a MySQL database of identical information, which would be faster to search through?
thanks a lot everyone!
The best way to answer this question is to perform a benchmark.
Although you should just try it out yourself, I'm going to assume that there's a proper index and conclude that the DB can do it faster than PHP due to being built to be all about that.
However, it might come down to network latencies, speed of parsing SQL vs PHP, or DB load and percentage of memory use.
My first thought would be it is faster searching with arrays. But, on the other side it really depends on several factors:
How is your databasa table designed (does it use indexes properly, etc)
How is your query built
Databases are generally pretty optimized for such searches.
What type of search you are doing on the array? There are several type of searches you could do. The slowest is a straight search where you go through each row an check for a value, a faster approach is a binary search.
I presumed that you are comparing a select statement executed directly on a database, and an array search in php. Not
One thing to keep in mind: If your search is CPU intensive on the database it might be worth doing it in PHP even if it's not as fast. It's usually easier to add web servers than database servers when scaling.
Test it - profiling something as simple as this should be trivial.
Also, remember that databases are designed to handle exactly this sort of task, so they'll naturally be good at it. Even a naive binary search would only have 17 compares for this, so 25k elements isn't a lot. The real problem is sorting, but that has been conquered to death over the past 60+ years.
It depends. In mysql you can use indexing, which will increase speed, but with php you don't need to send information through net(if mysql database on another server).
MySQL is built to efficiently sort and search through large amounts of data, such as this. This is especially true when you are searching for a key, since MySQL indexes the primary keys in a table.