I have Db of 100,000 users in MYSQL. In that DB i am having column ID,username,Fname,Lname, etc..
When www.example.com/Jim or www.example.com/123 (Where JIM is username and 123 is ID in the users table)
I am using MYSQL query : select * from users where ID = 123 OR username = Jim
I am executing above query in PHP.
Output of the above query is :
| ID | Username | fname | lname |
+----+----------+--------+---------+
|123 | jim | Jim | Jonson |
My Problem is its taking huge time to select username or ID in the DB.
I have used following query
SELECT * FROMusersUSE INDEX (UsersIndexId) where id=123
Is this right way to call Index ?
EXPLAIN SELECT * FROM `users` WHERE ID =327
OP
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE users Const PRIMARY,UsersIndexId PRIMARY 4 const
1
I sugest you take a look at this: How MySQL Uses Indexes
Quoting from the first paragraph:
Indexes are used to find rows with specific column values quickly.
Without an index, MySQL must begin with the first row and then read
through the entire table to find the relevant rows. The larger the
table, the more this costs. If the table has an index for the columns
in question, MySQL can quickly determine the position to seek to in
the middle of the data file without having to look at all the data. If
a table has 1,000 rows, this is at least 100 times faster than reading
sequentially.
That should help speed up your search.
(Edit: Updated the link to a newer version of the SQL docs)
PS: More specifically, column indexes might be what you want.
You can find more info about adding indexes here: Create Index Syntax
To complete #Kjartan answer, you can try the following :
ALTER TABLE users ADD INDEX id_i (`ID`);
ALTER TABLE users ADD INDEX username_i (`Username`);
Your queries should be faster.
Related
I am making a PHP backend API which executes a query on MySQL database. This is the query:
SELECT * FROM $TABLE_GAMES WHERE
($GAME_RECEIVERID = '$userId'OR $GAME_OTHERID = '$userId')
ORDER BY $GAME_ID LIMIT 1"
Essentially, I'm passing $userId as parameter, and getting row with smallest $GAME_ID value and it would return result in less than 100 ms for users that have around 30 000 matching rows in table. However, I have since added new users, that have around <100 matching rows, and query is painfully slow for them, taking around 20-30 seconds every time.
I'm puzzled to why the query is so much slower in situations where it is supposed to return low amount of rows, and extremely fast when returns huge amount of rows especially since I have ORDER BY.
I have read about parameter sniffing, but as far as I know, that's the SQL Server thing, and I'm using MySQL.
EDIT
Here is the SHOW CREATE statement:
CREATE TABLEgames(
IDint(11) NOT NULL AUTO_INCREMENT,
SenderIDint(11) NOT NULL,
ReceiverIDint(11) NOT NULL,
OtherIDint(11) NOT NULL,
Timestamptimestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (ID)
) ENGINE=MyISAM AUTO_INCREMENT=17275279 DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci
Here is the output of EXPLAIN
+----+-------------+-------+------+---------------+------+---------+-----+------+-------+
| id | select_type | table | type | possible_keys | key | key_len | ref | rows | extra |
+----+-------------+-------+------+---------------+------+---------+-----+------+-------+
| 1 | SIMPLE | games | NULL | index | NULL | PRIMARY | 4 | NULL | 1 |
+----+-------------+-------+------+---------------+------+---------+-----+------+-------+
id select_type table partitions type possible_keys key key_len ref rows filtered Extra
1 SIMPLE games NULL index NULL PRIMARY 4 NULL 1 19.00 Using where
I tried prepared statement, but still getting the same result.
Sorry for poor formatting, I'm still noob at this.
You need to use EXPLAIN to analyse the performance of the query.
i.e.
EXPLAIN SELECT * FROM $TABLE_GAMES WHERE
($GAME_RECEIVERID = '$userId'OR $GAME_OTHERID = '$userId')
ORDER BY $GAME_ID LIMIT 1"
The EXPLAIN would provide the information about the select query with execution plan.
It is great tool to identify the slowness in the query. Based on the obtained information you can create the Indexes for the columns used in WHERE clause .
CREATE INDEX index_name ON table_name (column_list)
This would definitely increase the performance of the query.
Your query is being slow because it cannot find a matching record fast enough. With users where a lot of rows match, chances of finding a record to return are much higher, all other things being equal.
That behavior appears when $GAME_RECEIVERID and $GAME_OTHERID aren't part of an index, prompting MySQL to use the index on $GAME_ID because of the ordering. However, since newer players have not played the early games, there are literally millions of rows that won't match, but have to be checked nonetheless.
Unfortunately, this is bound to get worse even for old users, as your database grows. Ideally, you will add indexes on $GAME_RECEIVERID and $GAME_OTHERID - something like:
ALTER TABLE games
ADD INDEX receiver (ReceiverID),
ADD INDEX other (OtherID)
PS: Altering a 17 million rows table is going to take a while, so make sure to do it during a maintenance window or similar if this is used in production.
Is this the query after the interpolation? That is, is this what MySQL will see?
SELECT * FROM GAMES
WHERE RECEIVERID = '123'
OR OTHERID = '123'
ORDER BY ID LIMIT 1
Then this will run fast, regardless:
SELECT *
FROM GAMES
WHERE ID = LEAST(
( SELECT MIN(ID) FROM GAMES WHERE RECEIVERID = '123' ),
( SELECT MIN(ID) FROM GAMES WHERE OTHERID = '123' )
);
But, you will need both of these:
INDEX(RECEIVERID, ID),
INDEX(OTHERID, ID)
Your version of the query is scanning the table until it finds a matching row. My version will
make two indexed lookups;
fetch the other columns for the one row.
It will be the same, fast, speed regardless of how many rows there are for USERID.
(Recommend switching to InnoDB.)
Let's say I got a website which, when visited, shows what is your lucky word today. The problem is that every word can be lucky for only one person so you need to be fast visiting the website. Below is a sample table with lucky words:
+---------------------+
| lucky_word |
+---------------------+
| cat |
| moon |
| piano |
| yellow |
| money |
+---------------------+
My question is: how can I prevent two (or more) users from accessing that table at one time. I assume that every user reads the first lucky_word from the existing table and the chosen word is deleted immediately so it won't be the next user's lucky word. For instance, I want to avoid cat to be shown to more than one visitor.
Should I solve this using an appropriate MySQL query or some lines in a PHP code or both?
You can use a locking read within a transaction; for example, using PDO:
$pdo = new PDO('mysql:charset=utf8;dbname='.DBNAME, USERNAME, PASSWORD);
$pdo->beginTransaction();
$word = $pdo->query('SELECT lucky_word FROM myTable LIMIT 1 FOR UPDATE')
->fetchColumn();
$pdo->prepare('DELETE FROM myTable WHERE lucky_word = ?')
->execute(array($word));
$pdo->commit();
In MySQL you can lock tables, to prevent other sessions reading and/or writing to the table. In the case of WRITE locks, the first session to request the lock will hold the table until it is released, and then the second session will get it until unlocked, and so forth. That way you can be sure that no two sessions are accessing or manipulating the same data at the same time.
Read all about it in the manual:
https://dev.mysql.com/doc/refman/5.6/en/lock-tables.html
How about adding a datestamp to the table updated when that particular word is used?
You could then use the following pseudo sql...
select word from words where lastdate <> [today];
update words set lastdate = today where word = [word];
A quickly method I used for a similar task:
1) create a table "unique_sequence" with just un field: id -> INT AUTOINCREMENT
CREATE TABLE `erd`.`unique_sequence` (
`id` INT NOT NULL AUTO_INCREMENT,
PRIMARY KEY (`id`));
2) when a user arrives on the site:
INSERT INTO unique_sequence VALUES();
SELECT word FROM lucky_word WHERE id = LAST_INSERT_ID();
As The ID that was generated by LAST_INSERT_ID() is maintained in the server on a per-connection basis it should be multi-user safe...
... and so we can be sure that every new user will get a unique ID that match the ones in lucky_word table.
I have a small problem with a php mysql query, I am looking for help.
I have a family tree table, where I am storing for each person his/her ancestors id separated by a comma. like so
id ancestors
10 1,3,4,5
So the person of id 10 is fathered by id 5 who is fathered by id 4 who is fathered by 3 etc...
Now I wish to select all the people who have id x in their ancestors, so the query will be something like:
select * from people where ancestors like '%x%'
Now this would work fine except, if id x is lets say 2, and a record has an ancestor id 32, this like query will retrieve 32 because 32 contains 2. And if I use '%,x,%' (include commas) the query will ignore the records whose ancestor x is on either edge(left or right) of the column. It will also ignore the records whose x is the only ancestor since no commas are present.
So in short, I need a like query that looks up an expression that either is surrounded by commas or not surrounded by anything. Or a query that gets the regular expression provided that no numbers are around. And I need it as efficient as possible (I suck at writing regular expressions)
Thank you.
Edit: Okay guys, help me come up with a better schema.
You are not storing your data in a proper way. Anyway, if you still want to use this schema you should use FIND_IN_SET instead of LIKE to avoid undesired results.
SELECT *
FROM mytable
WHERE FIND_IN_SET(2, ancestors) <> 0
You should consider redesigning your database structure. Add new table "ancestors" to database with columns:
id id_person ancestor
1 10 1
2 10 3
3 10 4
After -- use JOIN query with "WHERE IN" to choose right rows.
You're having this issue because of wrong design of database.First DBMS based db's aren't meant for this kind of data,graph based db's are more likely to fit for this kind of solution.
if it contain small amount of data you could use mysql but still the design is still wrong,if you only care about their 'father' then just add a column to person (or what ever you call it) table. if its null - has no father/unknown otherwise - contains (int) of his parent.
In case you need more then just 'father' relationship you could use a pivot table to contain two persons relationship but thats not a simple task to do.
There are a few established ways of storing hierarchical data in RDBMS. I've found this slideshow to be very helpful in the past:
Models for Hierarchical Design
Since the data deals with ancestry - and therefore you wouldn't expect it to change that often - a closure table could fit the bill.
Whatever model you choose, be sure to look around and see if someone else has already implemented it.
You could store your values as a JSON Array
id | ancestors
10 | {"1","3","4","5"}
and then query as follows:
$query = 'select * from people where ancestors like \'%"x"%\'';
Better is of course using a mapping table for your many-to-many relation
You can do this with regexp:
SELECT * FROM mytable WHERE name REGEXP ',?(x),?'
where x is your searched value
DROP TABLE IF EXISTS my_table;
CREATE TABLE my_table
(id INT NOT NULL AUTO_INCREMENT PRIMARY KEY
,ancestors VARCHAR(250) NOT NULL
);
INSERT INTO my_table VALUES(10,',1,3,4,5');
SELECT *
FROM my_table
WHERE CONCAT(ancestors,',') LIKE '%,5,%';
+----+-----------+
| id | ancestors |
+----+-----------+
| 10 | ,1,3,4,5 |
+----+-----------+
SELECT *
FROM my_table
WHERE CONCAT(ancestors,',') LIKE '%,4,%';
+----+-----------+
| id | ancestors |
+----+-----------+
| 10 | ,1,3,4,5 |
+----+-----------+
Trying to write statement where in single statement select all (*) and sum one column from the same database and the same table, depending on conditions.
Wrote such statement (based on this Multiple select statements in Single query)
SELECT ( SELECT SUM(Amount) FROM 2_1_journal), ( SELECT * FROM 2_1_journal WHERE TransactionPartnerName = ? )
I understand that SELECT SUM(Amount) FROM 2_1_journal will sum all values in column Amount (not based on codition).
But at first want to understand what is correct statement
With above statement get error SQLSTATE[21000]: Cardinality violation: 1241 Operand should contain 1 column(s)
Can not understand error message. From advice here MySQL - Operand should contain 1 column(s) understand that subquery SELECT * FROM 2_1_journal WHERE TransactionPartnerName = ? must select only one column?
Tried to change statement to this SELECT ( SELECT * FROM 2_1_journal WHERE TransactionPartnerName = ? ), ( SELECT SUM(Amount) FROM 2_1_journal), but get the same error...
What would be correct statement?
SELECT *, (SELECT SUM(Amount) FROM 2_1_journal)
FROM 2_1_journal
WHERE TransactionPartnerName = ?
This selects sums up Amount from the entire table and "appends" all rows where TransactionPartnerName is the parameter you bind in the client code.
If you want to limit the sum to the same criteria as the rows you select, just include it:
SELECT *, (SELECT SUM(Amount) FROM 2_1_journal WHERE TransactionPartnerName = ?)
FROM 2_1_journal
WHERE TransactionPartnerName = ?
A whole different thing: table names like 2_1_journal are strong indicators of a broken database design. If you can redo it, you should look into how to normalize the database properly. It is most likely pay back many times over.
With regard to normalization (added later):
Since the current design uses keys in table names (such as the 2 and 1 in 2_1_journal), I'll quickly illustrate how I think you can vastly improve that design. Lets say that the table 2_1_journal has the following data (I'm just guessing here because the tables haven't been described anywhere yet):
title | posted | content
------+------------+-----------------
Yes! | 2013-01-01 | This is just a test
2nd | 2013-01-02 | Another test
This stuff belongs to user 2 in company 1. But hey! If you look at the rows, the fact that this data belongs to user 2 in company 1 is nowhere to be found.
The problem is that this design violates one of the most basic principles of database design: don't use keys in object (here: table) names. A clear indication that something is very wrong is if you have to create new tables if something new is added. In this case, adding a new user or a new company requires adding new tables.
This issue is easilly fixed. Create one table named journal. Next, use the same columns, but add another two:
company | user | title | posted | content
--------+------+-------+------------+-----------------
1 | 2 | Yes! | 2013-01-01 | This is just a test
1 | 2 | 2nd | 2013-01-02 | Another test
Doing it like this means:
You never add or modify tables unless the application changes.
Doing joins across companies or users (and anything else that used to be part of the table naming scheme is now possible with a single, fairly simple select statement).
Enforcing integrity is easy - if you upgrade the application and want to change the tables, the changes doesn't have to be repeated for each company and user. More importantly, this lowers the risk of having the application get out of sync with the tables in the database (such as adding the field comments to all x_y_journal tables, but forgetting 5313_4324_journal causing the application to break only when user 5313 logs in. This is the kind of problem you don't want to deal with.
I am not writing this because it is a matter of personal taste. Databases are just designed to handle tables that are laid out as I describe above. The design where you use object keys as part of table names has a host of other problems associated with it that are very hard to deal with.
Let's say that I've got a table, like that (id is auto-increment):
id | col1 | col2
1 | 'msg'| 'msg'
2 | 'lol'| 'lol2'
3 | 'xxx'| 'x'
Now, I want to delete row number 2 and I get something like this
id | col1 | col2
1 | 'msg'| 'msg'
3 | 'xxx'| 'x'
The thing is, what I want to get is that:
id | col1 | col2
1 | 'msg'| 'msg'
2 | 'xxx'| 'x'
How can I do that in the EASIEST way (my knowledge about MySQL is very poor)?
You shouldn't do that.
Do not take an auto-incremented unique identifier as an ordinal number.
The word "unique" means that the identifier should be stuck to its row forever.
There is no connection between these numbers and enumerating.
Imagine you want to select records in alphabetical order. Where would your precious numbers go?
A database is not like an ordered list, as you probably think. It is not a flat file with rows stored in a predefined order. It has totally different ideology. Rows in the database do not have any order. And will be ordered only at select time, if it was explicitly set by ORDER BY clause.
Also, a database is supposed to do a search for you. So you can tell that with filtered rows or different ordering this auto-increment number will have absolutely nothing to do with the real rows positions.
If you want to enumerate the output - it's a presentation layer's job. Just add a counter on the PHP side.
And again: these numbers supposed to identify a certain record. If you change this number, you'd never find your record again.
Take this very site for example. Stack Overflow identifies its questions with such a number:
stackoverflow.com/questions/3132439/mysql-auto-decrementing-value
So, imagine you saved this page address to a bookmark. Now Jeff comes along and renumbers the whole database. You press your bookmark and land on the different question. Whole site would become a terrible mess.
Remember: Renumbering unique identifiers is evil!
I think there is no way to this directly. Maybe you can do "update" operation. But you must do it for all record after your deleted record. It is very bad solution for this.
Why using an auto-increment if you want to change it manually?
It is not good practice to change the value of an auto_increment column. However, if you are sure you want to, the following should help.
If you are only deleting a single record at a time, you could use a transaction:
START TRANSACTION;
DELETE FROM table1 WHERE id = 2;
UPDATE table1 SET id = id - 1 WHERE id > 2;
COMMIT;
However if you delete multiple records, you will have to drop the column and re-add it. It is probably not guaranteed to put the rows in the same order as previously.
ALTER TABLE table1 DROP id;
ALTER TABLE table1 ADD id INTEGER NOT NULL AUTO_INCREMENT;
Also, if you have data that relies on these IDs, you will need to make sure it is updated.
You can renumber the whole table like this:
SET #r := 0;
UPDATE mytable
SET id = (#r := #r + 1)
ORDER BY
id;