What is the fastest way to create 899 rows in a table, using only the number. The column isn't autoincrement.
Currently I create a query like this:
$a1=range(100,999);
$a1=implode('),(',$a1);
$a1='INSERT INTO groups (val) VALUES('.$a1.')';
it gives a huge query like this:
INSERT INTO groups (val) VALUES(100),(101),(102),(103),(104),(105),(106),(107),(108),
(109),(110),(111),(112),(113),(114),(115),(116),(117),(118),(119),(120),(121),(122),
(123),(124),(125), etc etc etc....
I wonder if there is a faster and nicer way to do this?
I don't think you have a faster way of doing that. Look at MySQL documentation
The time required for inserting a row is determined by the following
factors, where the numbers indicate approximate proportions:
Connecting: (3)
Sending query to server: (2)
Parsing query: (2)
Inserting row: (1 × size of row)
Inserting indexes: (1 × number of indexes)
Closing: (1)
This does not take into consideration the initial overhead to open
tables, which is done once for each concurrently running query.
The size of the table slows down the insertion of indexes by log N,
assuming B-tree indexes.
You can use the following methods to speed up inserts:
If you are inserting many rows from the same client at the same time,
use INSERT statements with multiple VALUES lists to insert several
rows at a time. This is considerably faster (many times faster in some
cases) than using separate single-row INSERT statements. If you are
adding data to a nonempty table, you can tune the
bulk_insert_buffer_size variable to make data insertion even faster.
See Section 5.1.4, “Server System Variables”.
With one query you save the Connecting, Sending query to server , Closing, plus MySQL optimizing your query.
Also, if you're only inserting around 1000 rows with so little data, the insertion is very fast so i wouldn't be worried about performance in this case.
For a range of numbers a smaller query can be used if you want:-
INSERT INTO groups (val)
SELECT Hundreds.a * 100 + Tens.a * 10 + Units.a AS aNumber
FROM
(SELECT 0 AS a UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9) Hundreds,
(SELECT 0 AS a UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9) Tens,
(SELECT 0 AS a UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9) Units
HAVING aNumber BETWEEN 100 AND 999
Not sure this saves you anything much though.
Related
i have an database of 2000 tables and they seems increasing as each table represent a user.So structure of all tables is same.
i want to get top 5 values
e.g if we take column -> OS(operating system)
then
query should take top 5 operating systems names from each table with their count and in the end i should get the sum up result of all tables to get overall top 5 operating systems names in all tables of database..
how to do it ?
i can use PHP & MySQL only
i have fetched the same column from all tables but can't access the values stored in them..
SELECT `COLUMN_NAME` FROM INFORMATION_SCHEMA.columns WHERE TABLE_SCHEMA='database_name' AND COLUMN_NAME = 'os'
Your design is really something you should change, but this was an approach to do this in MySql. Test it in this Fiddle
create table test(os int);
insert into test SELECT 1 UNION SELECT 2 UNION SELECT 3;
create table test2(os int);
insert into test2 SELECT 11 UNION SELECT 12 UNION SELECT 13;
create table test3(os int);
insert into test3 SELECT 21 UNION SELECT 22 UNION SELECT 23;
create table testOtherColumn(Other int);
insert into testOtherColumn SELECT 101 UNION SELECT 102 UNION SELECT 103;
set #MyCmd=
concat(
(
select substr(group_concat(' UNION SELECT ',c.COLUMN_NAME,' FROM ',c.TABLE_NAME SEPARATOR ''),8)
from information_schema.columns as c
where c.COLUMN_NAME='os'
),';');
select #MyCmd;
PREPARE MyStatement FROM #MyCmd;
EXECUTE MyStatement;
DEALLOCATE PREPARE myStatement;
You will need some kind of sorting and LIMIT x and aggregats to achieve your goal. I just do not know enough of your tables structures...
I got a mysql query that selects all clicks for each hour of a day.
This query worked good till we have alot of click entries in our database. Now it needs sometimes several seconds (up to 9!) to request the datas...
The query is:
SELECT h.clickHour, COUNT(clicktime) AS c
FROM ( SELECT 0 AS clickHour
UNION ALL SELECT 1
UNION ALL SELECT 2
UNION ALL SELECT 3
UNION ALL SELECT 4
UNION ALL SELECT 5
UNION ALL SELECT 6
UNION ALL SELECT 7
UNION ALL SELECT 8
UNION ALL SELECT 9
UNION ALL SELECT 10
UNION ALL SELECT 11
UNION ALL SELECT 12
UNION ALL SELECT 13
UNION ALL SELECT 14
UNION ALL SELECT 15
UNION ALL SELECT 16
UNION ALL SELECT 17
UNION ALL SELECT 18
UNION ALL SELECT 19
UNION ALL SELECT 20
UNION ALL SELECT 21
UNION ALL SELECT 22
UNION ALL SELECT 23 ) AS h
INNER JOIN links l ON l.user_id = 1
LEFT OUTER
JOIN clicks
ON EXTRACT(HOUR FROM clicks.clicktime) = h.clickHour
AND DATE(clicks.clicktime) = '2014-09-21'
AND clicks.link_id = l.id
GROUP
BY h.clickHour
I got these unions because i need clicks for each hour also empty hours...
Please help!
Ok so we are talking about 0 to several thousand rows for the table clicks. The click time is saved as a timestamp and every click got a unique id. I see that the union thing is bad and i have to change it.
What i try now is to select all clicks of a day grouped by HOUR(clicktime):
But when i do so I get too many results like 10x then it should be.
I'd rewrite the query like this:
SELECT h.clickHour
, IFNULL(d.clickCount,0) AS c
FROM ( SELECT 0 AS clickHour UNION ALL SELECT 1 UNION ALL SELECT 2
UNION ALL SELECT 3 UNION ALL SELECT 4 UNION ALL SELECT 5
UNION ALL SELECT 6 UNION ALL SELECT 7 UNION ALL SELECT 8
UNION ALL SELECT 9 UNION ALL SELECT 10 UNION ALL SELECT 11
UNION ALL SELECT 12 UNION ALL SELECT 13 UNION ALL SELECT 14
UNION ALL SELECT 15 UNION ALL SELECT 16 UNION ALL SELECT 17
UNION ALL SELECT 18 UNION ALL SELECT 19 UNION ALL SELECT 20
UNION ALL SELECT 21 UNION ALL SELECT 22 UNION ALL SELECT 23
) h
LEFT
JOIN ( SELECT EXTRACT(HOUR FROM c.clicktime) AS clickHour
, SUM(1) AS clickCount
FROM clicks c
JOIN links l
ON l.user_id = 1
AND l.id = c.link_id
WHERE c.clicktime >= '2014-09-21'
AND c.clicktime < '2014-09-21' + INTERVAL 1 DAY
GROUP BY EXTRACT(HOUR FROM c.clicktime)
) d
ON d.clickHour = h.clickHour
The approach here is to get the inline view query d to return a maximum of 24 rows. This cranks through the clicks table to get the counts. W're going to defer the join operation to the fixed set of 24 rows until after we have calculated the hourly counts. (The join to h is there only to get rows with zero counts returned, which would otherwise just be "missing" rows.)
You can test the performance of the inline view query d, and of the entire query, I suspect there won't be much difference. The cost of materializing the inline view h isn't that much (there's some overhead, but it's very likely that will use the Memory storage engine; it's small enough and it should be simple integer datatype.) And that join operation of 24 rows to 24 rows won't be that expensive, even without any indexes available.
I suspect that the majority of time will be in materializing the derived table d.
We're going to want an index with a leading column of clickDate, so that we can use a more efficient index range scan operation, to avoid evaluating expressions for every flipping row in the table.
I changed this predicate: DATE(clickTime) = '2014-09-21' into a predicates that reference the bare column, this enables MySQL to consider an efficient range scan operation on the clickTime column, (to quickly eliminate a boatload of rows from consideration), rather than requiring that MySQL evaluate a function on every flipping row in the table.
Some performance gain may be obtained by making covering indexes available on the clicks and links tables (so that the query can be satisfied from the indexes, without a need to visit pages in the underlying table.)
At a minimum on the clicks table:
ON clicks (clickTime, link_id)
If id is unique (or primary key) on the links table, this index may not give any performance benefit:
ON links (id, user_id)
If a covering index used, the EXPLAIN output should show "Using index".
I don't see a way around the "Using filesort" operation, not without adding a column to clicks table that stores the clickTime truncated to the hour. With a column like that, and an appropriate index, it's possible that we could get the GROUP BY operation optimized using the index, avoiding the "Using filesort" operation.
Have you indexed?
Clicks table: clicktime, link_id
Links table: id, user_id
I have a little problem , I want to count same string in an array ,
for example
My table like this:
id | data
---------------------------
1 | #user1,#user2,#user3
2 | #user1,#user4
3 | #user1,#user5
4 | #user2,#user3
How can I count #user1,#user2,etc.. ?
You can use find_in_set to find data in comma separated field.
SELECT COUNT(*)
FROM some_table
WHERE FIND_IN_SET('#user2', data)
This will give you a count of the rows that contain this string.
Note that this does suggest a database design that is not normalised and as this function can't use indexes it is likely to perform badly compared to a properly normalised database (ie, split the strings off onto a different table, with one row per string per id).
EDIT - if you want a count of all the strings:-
SELECT sub1.aString, COUNT(*)
FROM
(
SELECT DISTINCT SUBSTRING_INDEX(SUBSTRING_INDEX(data, ',', 1 + units.i + 10 * tens.i), ',', -1) AS aString
FROM some_table,
(SELECT 0 AS i UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9) units,
(SELECT 0 AS i UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9) tens
) sub1
INNER JOIN some_table
ON FIND_IN_SET(sub1.aString, data)
GROUP BY sub1.aString
This uses a couple of sub queries to generate 100 rows for each row in you main table, each with a combination of the numbers 0 to 9 twice. From the combination it can calculate a number between 0 and 99 (can easily be expanded to add another sub query to go from 0 to 999, or more). It then uses SUBSTRING_INDEX with the generated number to split out the possible strings in data for each row. This will generate a LOT of duplicates, partly as the strings will likely be on many rows and partly because the last string on each row will be put out many times (ie, if there are 10 string, the last one will be put out 91 times due to the way SUBSTRING_INDEX is used). DISTINCT is used to remove these duplicates.
The result is then joined against your table using FIND_IN_SET, and COUNT / GROUP BY used to get all the counts of all the strings.
You can try somthing like this:-
SELECT COUNT(data)
FROM your_table
WHERE data LIKE '%#user1%'
I have 7 tables that I could UNION on (wwith a limit of 30)
OR
should I do 7 separate queries (with a limit of 30) and trace through them using PHP.
Which why is faster? More optimal? In the second way I would have to trace through part of the 7 queries simulataneously and find the top 30 I need.
What is your needs?
As #chris wrote before, this may help you:
Complex SQL (Maybe Outer Joins)
select * from (select ... from ... order ... limit 10 )
union all
select * from (select ... from ... order ... limit 10)
order by ... limit 10
As I know (checked on DB with 50 million rows) - its fater than not using the devired queries.
Before making decisions you need at least to run both kinds of queries with MySql's EXPLAIN and analyze results. Something like this:
EXPLAIN SELECT f1, f2, f3 FROM t1
UNION ALL
SELECT f1, f2, f3 FROM t2;
depends if each query produce unique results using UNION ALL is better you save server trips, and you can sort the result after the union is performed. e.g
select column1 alias1, column2 alias2, from table x where ...
UNION ALL
select column3 alias1, column2 alias2 from table y where ...
...
order by 1
Sorry by my English
I have a mysql table, each row of which can have an arbitrary number of comma-delimmited words. For example:
TABLE words
"test, dog, fun, yellow, quick, yellow"
"jogging, academic, fun, jogging, shoulder"
"shoulder, happy, flower, shoulder, shoulder"
I would like to remove the duplicate words in each row. So for example, the above would become:
TABLE words
"test, dog, fun, yellow, quick"
"jogging, academic, fun, shoulder"
"shoulder, happy, flower"
(Note that I only need to remove the duplicates in each row, by itself. I do not need to remove the duplicates between all rows.)
Any suggestions on the best way to accomplish this? Is there a better way than SELECTing and then UPDATEing through the table one row at a time?
Thanks, in advance, for your help.
This is better suited outside of SQL. It's not going to be pretty if you try to interrogate strings using a query. I recommend:
SELECTing each row
performing an $val = explode(', ',$column);
switch to $val = array_unique($val);, then
UPDATEing to the table with implode(', ',$val);`.
note: you can save yourself some time and do a strcmp($orig,$new) and only UPDATE if necessary.
I don't think there's better way than SELECTing and then UPDATEing through the table one row at a time. As I know, SQL just don't support manipulating string like that. You must take a string out to remove duplicate, then insert it again in the table.
Here is a pure mysql version you use a bunch of number for a CROSS JOIN for each word then you just DISTINCT concat the broken words.It would help if you have a primary or unique key in case the rows are identicals.
SELECT GROUP_CONCAT(DISTINCT SUBSTRING_INDEX(SUBSTRING_INDEX(t.col, ', ', x.cifre), ', ', -1)) AS words
FROM t
INNER JOIN
(
SELECT 1 + a.i + b.i * 10 cifre, b.i + a.i * 10 sute
FROM (SELECT 0 AS i UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9) a
CROSS JOIN (SELECT 0 AS i UNION SELECT 1 UNION SELECT 2 UNION SELECT 3 UNION SELECT 4 UNION SELECT 5 UNION SELECT 6 UNION SELECT 7 UNION SELECT 8 UNION SELECT 9) b
) x
ON (LENGTH(t.col) +1 - LENGTH(REPLACE(t.col, ', ', ''))) >= x.cifre
GROUP BY col
FIDDLE