This question already has answers here:
MySQL Integer vs DateTime index
(3 answers)
Closed 6 years ago.
I'm creating a table to store millions of records, that's it 86400seconds a day x 365days x 10years = 315,360,000 rows of records with only 3 columns,
with datetime, decimal, and smallint (only 3 fields) datetime as index.
I'm thinking of converting the datetime into INT unsigned (PHP time()) to reduce the storage. With the datetime, decimal and smallint, I'm having 2.5GB for 1 table. I've not tried to replace the DATETIME with INT.
The insertion to this table is 1 time job, and I'll have a lots of SELECT statement for analytical purpose, thus I'm changing the InnoDB to MyISAM.
Any thoughts or suggestion?
Indexes are one of the things used to get a faster search on a table:
http://dev.mysql.com/doc/refman/5.7/en/mysql-indexes.html
On the basic level, indexes are responsible for making the engine not iterate over the entire table when searching for something you asked for.
Some of the things stated in the link regarding the use of indexes:
To find the rows matching a WHERE clause quickly.
To eliminate rows from consideration. If there is a choice between
multiple indexes, MySQL normally uses the index that finds the
smallest number of rows (the most selective index).
See if that suits your needs
Related
I'm developing a web-based tool that can help analyze number intervals that occurs in a 6-digit lottery.
Let us focus on a certain number first. Say 7
The sql query I've done so far:
SELECT * FROM `l642` WHERE `1d`=7 OR `2d`=7 OR `3d`=7 OR `4d`=7 OR `5d`=7
OR `6d`=7 ORDER BY `draw_date` DESC LIMIT 2
This will pull the last two latest dates where number 7 is present
I'm thinking of using DATEDIFF but I'm confused on how to get the previous value to subtract it on the latest draw_date
My goal is to list the intervals of numbers 1-42 and I'll plan to accomplish it using PHP.
Looking forward to your help
A few ideas spring to mind.
(1) First, since you perfectly have your result set ordered, use PHP loop on the two rows getting $date1 =$row['draw_date']. Then fetch next/last row and set $date2 =$row['draw_date']. With these two you have
$diff=date_diff($date1,$date2);
as the difference in days.
(2)
A second way is to have mysql return datediff by including a rownumber in the resultset and doing a self-join with aliases say alias a for row1 and alias b for row2.
datediff(a.draw_date,b.drawdate).
How one goes about getting rownumber could be either:
(2a) rownumber found here: With MySQL, how can I generate a column containing the record index in a table?
(2b) worktable with id int auto_increment primary key column with select into from your shown LIMIT 2 query (and a truncate table worktable between iterations 1 to 42) to reset auto_increment to 0.
The entire thing could be wrapped with an outer table 1 to 42 where 42 rows are brought back with 2 columns (num, number_of_days), but that wasn't your question.
So considering how infrequent you are probably doing this, I would probably recommend not over-engineering it and would shoot for #1
This question already has an answer here:
Postgres constraint for unique datetime range
(1 answer)
Closed 7 years ago.
i would like to prevent an insert into my PgSQL table if the values for a time period (lets call them newStart and newEnd), if this would overlap with any other timeperiod in my rows.
So, the table has a bunch of row where each has a start and end date (oldStart and oldEnd).
Is it possible to do this inside the query, without an trigger, an without getting all the data from the db first end check it inside php (because i think this is not ne best/fastest way to do).
Please correct me i think wrong.
I found the PgSQL overlap function, but building a query was not possible for me! How can i get more in touch with advanced database things? Just by doing?
Use an exclusion constraint.
More elegant with actual range types instead of start/end:
Preventing adjacent/overlapping entries with EXCLUDE in PostgreSQL
But you can also use an expression in the constraint. Basically:
CREATE TABLE tbl (
tbl_id serial PRIMARY KEY
, starts_at timestamp
, ends_at timestamp
, EXCLUDE USING gist (tsrange(starts_at, ends_at) WITH &&) -- no overlapping
);
More details:
Postgres constraint for unique datetime range
How can I get more in touch with advanced database things?
Study answers here, read the excellent manual, experiment with newly found techniques.
This question already has answers here:
MySQL PRIMARY KEYs: UUID / GUID vs BIGINT (timestamp+random)
(4 answers)
Closed 8 years ago.
I am developing a GPS device based application in Cakephp 2.0 and mysql (InnoDB). Each device sends data every minute to the Db and I need to make it scalable to a very large number of devices simultaneously to the server.
I have not used BIGINT auto increment as primary key because there is a limit and max value beyond which the limit to BIGINT will be reached and the system will fall apart, even if far away.
I created the primary key as char(36) and generated UUID from php and started storing the data. Primarily because the limit and uniqueness of the primary key will never stop and the design will never fail.
Uniqueness is the only reason for me and nothing else.
Problems:
The system is in pilot testing mode and time to insert the data has increased to very large extent. Refer to http://kccoder.com/mysql/uuid-vs-int-insert-performance/ This is exactly happening in my case where by time, the time to insert the data is increasing and i expect that it might get worse in the coming days as the number of data keeps increasing. There are around 2,00,000 data in the table now.\
The primary key being char(36), i assume the performance is getting effected in inserts, select statements and joins.
My idea is to replace the primary key UUID column with a varchar(50) column and have Device ID / IMEI Number + timestamp stored as primary key and they will always be unique. But on the downside, it's again a varchar field and performance issues on long run.
What is the best option for me in the long run?
Just use BIGINT UNSIGNED.
An unsigned INT64 is something like 4.000.000.000 times 4.000.000.000. So assuming you have one device for each of the less than 8 billion people on the planet logging once per second, that leaves you with 2 billion seconds or more than 63 years. I assume, that in 63 years an INT128 is normal (or small).
I am also quite sure, that you will run into very different classes of trouble long before you reach 2^64 rows in a single table.
I want to create a MyISAM log table with Unix timestamp values. I will have various SELECT statements every once in while using a "from" date to "to" date and the table will grow quite large.
I don't know which would be the fastest between an Indexed INT or Indexed TIMESTAMP (they have the same storage space I believe).
I checked on Stackoverflow already but the answers are somewhat vague, from new members or they say one or the other.
It shouldn't make a difference. Timestamps are represented internally as integers, so comparing and indexing them will be essentially the same.
This question already has answers here:
What are the optimum varchar sizes for MySQL?
(2 answers)
Closed 9 years ago.
I am designing a database which will to store JSON strings of various sizes. I'm considering using different tables (each with two columns, 'id' and 'data') to store different string sizes (tinytext - bigtext). In this case each table would be searched, starting with the table containing the smallest string sizes.
I'm also considering using a single table with a single string size and using multiple rows to store large JSON strings.
..or I could just create a table with a large VARCHAR size and save myself some development time.
There are two points that I am designing around:
In some cases, mysql stores small pieces of data "in row" which helps performance. What does this mean and how can I take advantage of this?
In some cases, mysql processes VARCHAR as its largest possible size. When does this happen and how can I avoid it?
From the database point of view there is no particular "good" length for varchar. However try to keep maximum row size under 8kb, including non-clustered indexes. Then you will avoid MySQL storing data out of row, which hampers performance.
use 255
Why historically do people use 255 not 256 for database field magnitudes?
Although, as a side note, if you are working with PHP and trying to insert strings in excess of 1000 characters, you will need to truncate to your max col size on the PHP side before inserting, or you will hit an error.