I'm relatively new to this stuff, so forgive if the question is dump. Suppose a table with fields: id, str_field. values of str_field is something like "12:17:1246:90". I want to get all rows where str_field contains e.g. "17". To do this I'll need to execute the following command
SELECT id FROM `table_name` WHERE INSTR(str_field, '17')>0
If there's a large number of rows in the table the query can be slow. Here's the question: if I'll index str_field will it increase the spead of query execution?
Thank you in advance!
PS. In other terms I'm asking: does index increase the spead on for the queries like
SELECT * FROM `table_name` WHERE str_field='value'
?
UPD str_field contains only numbers separated by colons.
If there's a large number of rows in the table the query can be slow. Here's the question: if I'll index str_field will it increase the spead of query execution?
Not much. If there are many other columns in your table then you can make a covering index on the columns you use: (id, str_field). This will be slightly faster because the index will be smaller than the original table and therefore can be read faster. However it will still require a full scan of the index (instead of a full scan of the entire table).
But other than that, you can improve the speed of the query by using a separate table with three columns to store the separate integers, using an approach called denormalization.
parent sortorder value
1 1 12
1 2 17
1 3 1246
1 4 90
Query like this:
SELECT parent AS id
FROM table_values
WHERE `value` = 17
You can then add an index on (value, parent) for this table, which will speed up the query. Note that the sortorder column is not required for this query. If you don't think you will ever need it then you don't need to include this column in your table.
You could create a fulltext index and use the special MATCH functionality: http://dev.mysql.com/doc/refman/5.0/en/fulltext-search.html
Still, searching for values instead of expressions is much faster...
Indexes can increase perfomance, that is true.
If you have a field which contains string data like 12:17:1246:90, you can use the query
SELECT id FROM `table_name` WHERE `str_field` LIKE '%17%';
% is the a joker character which means one or more characters.
Related
I have a performance issue when working with a huge table
I add index on column using this :
ALTER table add index column;
and on the text/blob column :
alter table add index (cat(200));
My table has about 6M rows and i am working with InnoDB engine (Mysql 5.5)
This query is very fast now that i add index on "order by" column:
SELECT * from table order by column DESC LIMIT 0,40
But when I add a WHERE clause on this query its very slow and it take about 10 seconds to load even with the column "cat" index like above. //index instead of indexed
SELECT * from table WHERE cat = 'electronic' order by column DESC LIMIT 0,40
the EXPLAIN of this slow query :
EXPLAIN SELECT * from table WHERE cat = 'electronic' order by 'id' DESC LIMIT 0,40
id : 1
select_type : SIMPLE
table : product
type : ref
possible_keys: cat
key: cat
Key_len: 203
ref: const
row : 1732184
extra: using where
The query working fine with small table with 50k rows but with 6M rows its slow. Why?
Do not use prefixing, such as cat(200); it usually makes the index unusable. I have never seen a case where the Optimizer, when faced with INDEX(a(10), b), gets past a and makes any use of b.
Change cat to be VARCHAR(255). That is probably more than sufficient for "categories".
The best index (if it is possible) is
INDEX(cat, `column`)
Note that cat is in the WHERE with =. It handles the entire WHERE, so the index can move on to the ORDER BY. Hence column can be used, too. More discussion of index making .
If cat must be TEXT, then the best you can do is
INDEX(`column`)
Then the Optimizer may decide to use it for avoiding a filesort. But if there are fewer than 40 (see LIMIT) 'electronic' rows, it will take an big scan and probably be slower than not using the index. So, I am not sure that it is even worth having INDEX(column).
For this query:
SELECT t.*
FROM table t
WHERE cat = 'electronic'
ORDER BY column DESC
LIMIT 0, 40;
The best index is a composite index on table(cat, column). You can use a prefix if column is too wide: table(cat, column(200)).
The best option is to index the table, if you dont know how to do it, you can check this doc
So, when you perform the query, the mysql will start searching on the indexed values, skipping a lot of useless data for that request.
I am using group by for like statement as i have database structure like this.
I want to get the count of workingzone groupby.but if i try to group by, then wrong output will appear as output will group by 99 and 99, as in figure.
My sql code is:
select count(organization),working_zone from `projects` where `district` = 12 and (`working_zone` = 99 or `working_zone` LIKE "99," or `working_zone` LIKE ",99") group by `organization`;
my desired result is:
count |working_zone
____6| 99
____3| 100
(99),(,99),(99,) should be grouped by doing sum and result should be 6.
You have an awful data structure -- although I wouldn't be surprised if the data is okay and you are really working off the result of a (reasonable) query. You should not be storing raw data in comma-delimited lists. Instead, use junction tables.
Why is having a separate row for each pair the SQLish way of storing data? Consider these reasons:
SQL has pretty based string functions (compared to other programming environments).
Data should be stored in its native type; don't store numbers as strings.
Foreign key relationships should be explicitly declared, and you can't declare a foreign key relationship using comma-delimited strings.
The primary SQL mechanism for optimizing queries are indexes, and comma-delimited lists preclude the use of indexes.
Sometimes, though, you are stuck with someone else's bad design decisions. If so, one solution uses a table of working zones:
select count(*), wz.working_zone
from projects p join
working_zones wz
on find_in_set(wz.working_zone, p.working_zone) > 0
where p.district; = 12 and
find_in_set(99, p.working_zone) > 0
group by wz.working_zone;
SELECT COUNT(organization),working_zone FROM table WHERE working_zone HAVING '99'
UNION ALL
SELECT COUNT(organization),SUBSTRING(working_zone,4) FROM table WHERE working_zone = '99,100'
Why my query fast when I run.
select count(*) as aggregate from `news` where `news`.`deleted_at` is null and `status` = '1'
But, slow when I run.
select count(*) as aggregate from `news` where `news`.`deleted_at` is null and `status` = '1' and `newscategory_id` = '17'
It is my table news structure image, have a look at here.
Sorry because my reputation is less than 8, so I can't attach image.
try adding an composite index on the three columns you are using for your select:
ALTER TABLE news ADD INDEX comp_index (deleted_at, status, newscategory_id);
and check it again.
probably use EXPLAIN to see if any indexes you have are used.
Indexes are used to find rows with specific column values quickly.
Without an index, MySQL must begin with the first row and then read
through the entire table to find the relevant rows. The larger the
table, the more this costs. If the table has an index for the columns
in question, MySQL can quickly determine the position to seek to in
the middle of the data file without having to look at all the data.
This is much faster than reading every row sequentially.
Try to add this in your DB:
CREATE INDEX newCategory_indx ON news (newscategory_id)
CREATE INDEX status_indx ON news (status)
This will give you quick result, as compared to previously generate (non-indexed column) result.
To know more about index and it's importance visit here
We have records with a count field on an unique id.
The columns are:
mainId = unique
mainIdCount = 1320 (this 'views' field gets a + 1 when the page is visited)
How can you insert all these mainIdCount's as seperate records in another table IN ANOTHER DBASE in one query?
Yes, I do mean 1320 times an insert with the same mainId! :-)
We actually have records that go over 10,000 times an id. It just has to be like this.
This is a weird one, but we do need the copies of all these (just) counts like this.
The most straightforward way to this is with a JOIN operation between your table, and another row source that provides a set of integers. We'd match each row from our original table to as many rows from the set of integer as needed to satisfy the desired result.
As a brief example of the pattern:
INSERT INTO newtable (mainId,n)
SELECT t.mainId
, r.n
FROM mytable t
JOIN ( SELECT 1 AS n
UNION ALL SELECT 2
UNION ALL SELECT 3
UNION ALL SELECT 4
UNION ALL SELECT 5
) r
WHERE r.n <= t.mainIdCount
If mytable contains row mainId=5 mainIdCount=4, we'd get back rows (5,1),(5,2),(5,3),(5,4)
Obviously, the rowsource r needs to be of sufficient size. The inline view I've demonstrated here would return a maximum of five rows. For larger sets, it would be beneficial to use a table rather than an inline view.
This leads to the followup question, "How do I generate a set of integers in MySQL",
e.g. Generating a range of numbers in MySQL
And getting that done is a bit tedious. We're looking forward to an eventual feature in MySQL that will make it much easier to return a bounded set of integer values; until then, having a pre-populated table is the most efficient approach.
I have got table with 300 000 rows. There is specially dedicated field (called order_number) in this table to story the number, which is later used to present the data from this table ordered by order_number field. What is the best and easy way to assign the random number/hash for each of the records in order to select the records ordered by this numbers? The number of rows in the table is not stable and may grow to 1 000 000, so the rand method should take it into the account.
Look at this tutorial on selecting random rows from a table.
If you don't want to use MySQL's built in RAND() function you could use something like this:
select max(id) from table;
$random_number = ...
select * from table where id > $random_number;
That should be a lot quicker.
UPDATE table SET order_number = sha2(id)
or
UPDATE table SET order_number = RAND(id)
sha2() is more random than RAND().
I know you've got enough answers but I would tell you how we did it in our company.
The first approach we use is with additional column for storing random number generated for each record/row. We have INDEX on this column, allowing us to order records by it.
id, name , ordering
1 , zlovic , 14
2 , silvas , 8
3 , jouzel , 59
SELECT * FROM table ORDER BY ordering ASC/DESC
POS: you have index and ordering is very fast
CONS: you will depend on new records to keep the randomization of the records
Second approach we have used is what Karl Roos gave an his answer. We retrieve the number of records in our database and using the > (greater) and some math we retrieve rows randomized. We are working with binary ids thus we need to keep autoincrement column to avoid random writings in InnoDB, sometimes we perform two or more queries to retrieve all of the data, keeping it randomized enough. (If you need 30 random items from 1,000,000 records you can run 3 simple SELECTs each for 10 items with different offset)
Hope this helps you. :)