I have several huge MYSQL tables (10gb+). Will the performance of PHP queries I am running on one of these tables be influenced by the presence of other huge tables in the same database? I am not running any queries on these other tables; they are neither directly nor indirectly linked to or referenced by my PHP query?
It deppends. You have to take in account the limited resources: memory and disc.
Are any other program or proccess accessing those tables?
Also you have to considered how your tables are fiscal stored on the database. Are they on the same Filegroup? Another problem is fragmentation: You can think that diferent tables are fisicaly split but it depends on how your database grow. You can have scenarios where two different tables that grow toghether on time, have their data mixed fisically on the disk and have impact on the performance. Hope my answer helps you.
Related
I have an optimization question.
The PHP web application, that I have recently started working with, has several large database tables in a MySQL database. The information in this tables should be accessible at all times for business purposes, which makes them grow really big eventually.
The tables are regularly written to and recent records are frequently selected.
Previous developers came up with a very weird practice of optimizing the system. They created separate database for storing recent records in order to keep tables compact and sync the tables once the record grows "old" (more than 24 hours old).
The application uses current date to pick the right database, when performing a SELECT query.
This is a very weird solution in my opinion. We had a big argument over that and I am looking to change this. However, before, I decided to ask:
1) Has someone ever came across anything similar before? I mean, separate database for recent records.
2) What are the most common practices to optimize databases for this particular case?
Any opinions are welcome, as there are many ways one can go at this point.
Try using INDEX:
CREATE INDEX
That improve the access, use and deploy of the information.
I believe this could help you RANGE Partitioning
The solution is to do a Partion to the table base on a date range.
By splitting a large table into smaller, individual tables, queries that access only a fraction of the data can run faster because there is less data to scan. Maintenance tasks, such as rebuilding indexes or backing up a table, can run more quickly.
The documentation of Mysql can be useful, check this out :
https://dev.mysql.com/doc/refman/5.5/en/partitioning-columns-range.html
I'm doing an RIA with JavaScript, MySQL and PHP on a Windows server.
I have 5,000 identically structured data sets I want to put in a database. 5 tables is enough for the data, all of which will be reasonably small except for one table that will have 300,000+ records for a typical data set.
Additionally, 500 users will get read only access to statistics compiled from those data sets. Those statistics are provided by PHP (no direct access is allowed). What's more, their access to data varies. Some users can only use one data set, others some, a few, all.
The results users see are relatively small; most requests return well under 100 rows, and the largest requests will be about 700 rows. All requests are through a JavaScript RIA which uses Ajax to connect to PHP which in turn connects to the data, does its thing and outputs JSON in response, which JavaScript then presents accordingly.
In thinking about how to structure this, three options present themselves:
Put the data sets in the same tables. That could easily give me 1,500,000,000 records in the largest table.
Use separate tables for each data set. That would limit the largest table size, but could mean 25,000 tables.
Forget the database and stick with the proprietary format.
I'm leaning towards #2 for a few reasons.
I'm concerned about issues in using very large tables (eg: query speeds, implementation limits, etc...).
Separate tables seem safer; they limit the impact of errors and structure changes.
Separate tables allow me to use MySQL's table level security rather than implementing my own row level security. This means less work and better protection; for instance, if a query is accidentally sent without row level security, users can get unauthorized data. Not so with table level security, as the database will reject the query out of hand.
Those are my thoughts, but I'd like yours. Do you think this is the right choice? If not, why not? What considerations have I missed? Should I consider other platforms if scale-ability is an issue?
1) I'm concerned about issues in using very large tables (eg: query speeds, implementation limits, etc...).
Whether the DBMS has to...
search through the large index of one table,
or search for the right table and then search through the smaller index of that table
...probably doesn't make much of a difference performance-wise. If anything, the second case has an undocumented component (the performance of locating the right table), so I'd be reluctant to trust it fully.
If you want to physically partition the data, MySQL supports that directly since version 5.1, so you don't have to emulate it via separate tables.
2) Separate tables seem safer; they limit the impact of errors and structure changes.
That's what backups are for.
3) Separate tables allow me to use MySQL's table level security rather than implementing my own row level security.
True enough, however similar effect can be achieved through views or stored procedures.
All in all, my instinct is to go with a single table, unless you know in advance that these data-sets differ enough structurally to warrant separate tables. BTW, I doubt you'd be able to do better with a proprietary format compared to a well-optimized database.
I'm setting up a MySQL database and I'm not sure of the best method to structure it:
I am setting up a system (PHP/MySQL based) where a few hundred people will be executing SELECT/UPDATE/SET/DELETE queries to a database (probably about 50 simultaneously). I imagine there are going to be a few thousand rows if they're all using the same database and table. I could split the data across a number of tables but then I would have to make sure they're all uniform AND I, as the administrator, will be running some SELECT DISTINCT queries via cron to update an administrative interface.
What's the best way to approach this? Can I have everybody sharing one database? one table? Will there be a problem when there are a few thousand rows? I imagine there is going to be a huge performance issue over time.
Any tips or suggestions are welcome!
MySQL/php can easily handle this as long as your server is powerful enough. MySQL loves RAM and will use as much as it can (within the limits you provide).
If you're going to have a lot of concurrent users then I would suggest looking at using innodb tables instead of MyISAM (the default in MySQL versions <5.5). Innodb locks individual rows when doing INSERT/UPDATE/DELETE etc, rather than locking the whole table like MyISAM does.
We use php/MySQL and would have 1000+ users on our site at the same time (our master db server does about 4k queries per second).
We have an existing PHP/MySQL app which doesn't have indexes configured correctly (monitoring shows that we do 85% table scans, ouch!)
What is a good process to follow to identify where we should be putting our indexes?
We're using PHP (Kohana using ORM for the DB access), and MySQL.
The answer likely depends on many things. For example, your strategy might be different if you want to optimize SELECTs at all costs or whether INSERTs are important to you as well. You might do well to read a MySQL Performance Tuning book or web site. There are several decent-to-great ones.
If you have a Slow Query Log, check it to see if there are particular queries that are causing problems. http://dev.mysql.com/doc/refman/5.5/en/slow-query-log.html
If you know the types of queries you'll be running or have identified problematic queries via the Slow Query Log or other mechanisms, you can then use the EXPLAIN command to get some stats on those queries. http://dev.mysql.com/doc/refman/5.5/en/explain.html
Once you have the output from EXPLAIN, you can use it to optimize. See http://dev.mysql.com/doc/refman/5.5/en/using-explain.html
Indexes are not just for the primary keys or the unique keys. If there are any columns in your table that you will search by, you should almost always index them.
I think this will help you on your database problems.
http://net.tutsplus.com/tutorials/other/top-20-mysql-best-practices/
In a project at work i have to improve performance. The data of the app is spread over many databases. I was told that is better for organizing the data. Whatever. Is there a performance penalty when i do a select over some tables spread on several databases instead of a select on those tables in one database?
depends on whether or not those databases are on the same physical server.
No, there shouldn't be a signifigant performance increase from spreading queries across different databases in mysql, assuming that the database are part of the same mysql install.
You'll do better to start with reducing the number of queries per page request, and zeroing in on individual queries that are taking a long time to complete.