I am querying data from a database, and the query clause varies with the day.
In other words, I want to get data in one-day unit by counting seconds in unix timestamps.
So, what I did for one-day-query is,
SELECT IDnum, Timestamp FROM table_name
WHERE CONVERT(Timestamp, UNSIGNED INTEGER) >= ($start_tim-86400*($i+1))
AND CONVERT(Timestamp, UNSIGNED INTEGER) < ($start_time-86400*$i)
because Timestamp attribute in table is varchar(32), I used CONVERT() in where clause to convert it to int, and compare with int type of 86400.
This query clause works so far, but it takes at least 5 mins in that I set ini_set('max_execution_time', 300). And after 5 mins, it showed
"Fatal error: Maximum execution time of 420 seconds exceeded in C:\www\LineChartOne\generateJSONdata.php on line 90", and still not complete.
My question is why it need so much time and, because I used function call in the "where clause"? Or something wrong with the "where clause", otherwise it should not run so slowly.
I'm guessing that the CONVERT() is super inefficient. Use UNIX_TIMESTAMP() or FROM_UNIXTIME() or date functions like DATE_SUB() or DATE_DIFF() to compare the dates.
If it still takes forever then either there's something wrong and you don't have tens of millions of rows then it is likely that:
Your PHP code is inefficient.
There's more to this query than what you're showing.
Your web and or mySQL servers are under heavy load from other users.
Edit:
Sweet jesus, the code in your comment makes my brain itch. Also I just now realized that your Timestamp column is a VARCHAR and not a TIMESTAMP. Bad user. Do not store numbers or dates as strings. Go sit in the corner.
Option 1: Make it an INT. [works with your existing code]
Option 2: Make it a DATETIME. [generally more flexible]
Things your are not getting:
FROM_UNIXTIME() converts an integer timestamp to a DATETIME type. If you feed it a string it will do an implicit conversion. [conversions are slow]
UNIX_TIMESTAMP() takes a date string, DATE, or DATETIME and returns an integer.
The code you gave UNIX_TIMESTAMP(FROM_UNIXTIME(Timestamp)) converts the string to an int to a date back to an int. The documentation is here, read up on it.
The vast majority of the reason why your query takes so long is because mySQL has to convert all of the values in Timestamp from VARCHAR to INTEGER. Change the column type and you can do simple integer comparisons which will be worlds faster.
A much smaller performance gain can be had by pre-computing the expression $start_time-86400*($i+1) in PHP. mySQL should optimize to evaluate only once, but the optimizer can do funky things when your queries become more complex and it might start computing this for each row.
The mysql query parser is struggling with converting those columns, you need to use the correct data types for the values you are storing, ie: int for int, varchar for strings, datetime etc....
ini_set('max_execution_time', 300); //300 seconds = 5 minutes
Place this at the top of your PHP script and let your script loose!
Taken from Increase PHP Script Execution Time Limit Using ini_set()
Related
I have to work with a postgres database 9.4 and i want to work with timestamp.
The first 'problem' is that when i create a timestamp column on postgres, i don't know what it does internally but when i query it returns '2010-10-30 00:00:00'
For me a timestamp is something like this 12569537329 (unix timestamp).
I say that because is a integer or a float, it's way easier for computer to deal comparing to string, and each country has his own time format, with unix timestamp is a number and end of story.
Querying from php the result is a string, so i have to make a bunch juggling and because of time zone, day light saving and other things something might could gone wrong.
I searched a lot of and can't find a way to work with unix timestamp on postgresql.
Can someone explain if there a way, or the right way to work and get as close as possible to unix timestamp.
UPDATE
One thing that i found that it gonna help me and it take a long time to discover that is possible on postgresql is change the Interval Output.
pg manual
In php the date interval for month is 'month' for pg is 'mon' on php it will understand mon as monday.
I think that if you have to juggle too much you are doing it wrong.
Gladly postgres let us to change that behavior for session or permanently.
So setting intervalstyle to iso_8601 it will work as php iso_8601 and will output P1M
Just convert the unix time_t to/from a timestamp, and use that in postgres:
CREATE TABLE omg
( seq SERIAL NOT NULL PRIMARY KEY
, stampthing TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP
);
INSERT INTO omg(stampthing) SELECT now();
INSERT INTO omg(stampthing) VALUES
('epoch'::timestamptz )
, ('epoch'::timestamptz + 12569537329 * '1 second'::interval)
;
SELECT stampthing
, DATE_PART('epoch',stampthing) AS original
FROM omg;
Output:
CREATE TABLE
INSERT 0 1
INSERT 0 2
stampthing | original
-------------------------------+------------------
2016-01-20 16:08:12.646172+01 | 1453302492.64617
1970-01-01 01:00:00+01 | 0
2368-04-24 20:08:49+02 | 12569537329
(3 rows)
If you just query a timestamp column in postgres, you'll get a formatted date. If you prefer the unix timestamp integer you can either cast it when you return it using a syntax like
select extract(epoch from '2011-11-15 00:00:00+01'::timestamp with time zone)::integer;
If you do this a lot, you may make a view with the extract.
or you can define your timestamp column as an integer and store your data using the extract()
I have mysql column time(3) and it's storing good value of time..
but then I want to sum two times it converts to bad time format;
I have two records:
id | time
---|-----------
1 | 00:00:15.490
2 | 00:02:14.900
So in real I shoud get: 00:02:30.390
but I get 230.390
is anyway to get correct answer from Mysql?
P.S. I am using php for functions but dont want to use it, unless there is other way.
Need to sum times with MILLISECONDS
for now I am using query SELECT SUM(time) AS total_time FROM times WHERE 1
Provided your table definition is something like this:
create table test (
id integer,
`time` time(3) -- important to specify precision
);
You can do this:
select time(sum(`time`))
from test;
note: requires mysql 5.6+
edit
Actually, time is the wrong function to use, as it doesn't have many smarts.
use sec_to_time instead, ie:
select sec_to_time(sum(`time`))
from test;
time extracts a time value, sec_to_time calculates a time value -- ie, time(70) returns NULL because there's no valid time that has 70 seconds, where as sec_to_time will correctly return '00:01:10' for the same input
edit
Turns out i'm still wrong. Lets try treating the milliseconds separately to the rest of the time:
select sec_to_time(sum(time_to_sec(`time`)) + sum(microsecond(`time`))/1000000)
from test;
Wrap your outputted result with the time function. So:
time(sum(`time`))
where time is the time function and 'time' is your summed column.
while fetching data from MySQL for today's date, what should be preferred why
1) "select * from table1 where created_dt Like CURDATE()";
2) "select * from table1 where created_dt Like ".date(Y-m-d H:i:s);
will it make any difference in execution time of the script, If i use MySql function instead of php function or vice versa
will it make any difference in execution time of the script
Definitely NO.
The difference would be pretty negligable, though generating the date in SQL would probably be marginally faster.
You should not be using 'like' though, since that is for partial string comparisons, and you are using a date field (which, internally, will be an integer). You are also missing two sets of quotes for your php date example.
But most importantly, if you are using a datetime format for created_dt, and then attempting to match by date using date(created_dt) or any kind of string comparison, you will not be taking advantage of indexing, and are likely to cause a full table scan. You'd be better using:
select * from table1 where created_dt between CURDATE() and date_add(CURDATE(), interval 1 day);
Which can take advantage of a btree index on created_dt to make your query perform considerably faster and more efficiently.
It would be best to use option 1 - date value computed on the database server because:
MySQL doesn't need to convert the string value from the query to a date, since CURDATE retrieves a DATE value
the PHP date function parses the parameter and formates the date and concatenates the result to the query string, which is surely a more complex task then the mysql CURDATE function which has no parameters and simply retrieves the date
This will be done with PHP.
I basically want to get the number of rows that were inserted 30 minutes ago.
I have a time field on my table which is type TIMESTAMP and on update it's set to CURRENT_TIMESTAMP.
The date is stored in this format:
2011-05-27 04:29:17
My query is supposed to look something like this, however i just can't do it
SELECT COUNT(*) FROM mytable WHERE UNIX_TIMESTAMP(time) < '.time().'-1800
Where time() is PHP's function that fetches the UNIX time.
What it should basically do is print me the number of rows inserted from now to 30 minutes ago, but i just can't seem to make it work.
Can somebody help?
Small edit:
Another problem i am seeing is that php's function time() displays the unix time which is UTC. The time stored in mysql is probably GMT i.e whatever my computer's time/timezone is set to.
You can easily get rows stored from now to 30 mins ago by simply using:
SELECT count(*) FROM mytable WHERE `time` >= DATE_SUB(UTC_TIMESTAMP, INTERVAL 30 minute)
Usage of UTC_TIMESTAMP is just an example if you're storing your date/time data as UTC_TIMESTAMP(), you can probably use NOW() if necessary, depends on what you're storing really.
**EDIT**
Removed bad pointers and fixed example :)
Do you really need your computer's timezone to be different than UTC? why not just set it to UTC & save yourself the confusion? If that doesn't work, just use dateadd() on mysql to convert your mysql timestamp to UTC when checking?
My suggestion would be to write a small function to convert the mysql timestamp to your PHP timestamp format & load it into mysql. Then all you need to do is to call tmstamp(time_stamp) instead of time_stamp in your query. You can do the reverse too i.e. Convert PHP's "30 minutes ago" timestamp to mysql format and rerun your query (probably easier).
Usually it's just a formatting issue. It's not standardized across programs.
This question already has answers here:
Should I use the datetime or timestamp data type in MySQL?
(40 answers)
Closed 9 years ago.
Probably many coders want to ask this question. it is What's the adventages of each one of those MySQL time formats. and which one you will prefer to use it in your apps.
For me i use Unix timestamp because maybe i find it easy to convert & order records with it, and also because i never tried the DATETIME thing. but anyways i'm ready to change my mind if anyone tells me i'm wrong.
Thanks
Timestamp (both PHP ones and MySQL's ones) are stored using 32 bits (i.e. 4 bytes) integers ; which means they are limited to a date range that goes from 1970 to 2038.
DATETIME don't have that limitation -- but are stored using more bytes (8 bytes, if I'm not mistaken)
After, between storing timestamps as seen by PHP, or timestamps as seen by MySQL :
using PHP timestamps means manipulations are easier from PHP -- see Date/Time Functions
using MySQL's timestamps means manipulations are easier from MySQL -- see 11.6. Date and Time Functions
And, for more informations between MySQL's TIMESTAMP and DATETIME datatypes, see 10.3.1. The DATETIME, DATE, and TIMESTAMP Types
As others have said, timestamps can represent a smaller range of datetimes (from 1970 to 2038). However, timestamps measure the number of seconds since the Unix Epoch (1970-01-01 00:00:00 UTC), thereby making them independent of time zone, whereas DATETIME stores a date and time without a time zone. In other words, timestamps unambiguously reference a particular point in time, whereas the exact point in time a DATETIME refers to requires a time zone (which is not stored in a DATETIME field). To see why this can matter, consider what happens if we change our time zone.
Let's say we want to store the datetime 2010-03-27 12:00 UTC. If we store this and retrieve it using a timestamp or DATETIME, then there usually appears to be no difference. However, if the server now changes so that the local time zone is UTC+01, then we get two different results if we pull out the datetime.
If we'd set the field to a DATETIME, it would report the datetime as 2010-03-27 12:00, despite the change in time zone. If we'd set the field to a timestamp, the date would be reported as 2010-03-27 11:00. This isn't a problem with either datatype -- it's just a result of the fact that they store slightly different information.
That really depends. I'll give you 2 examples where one overcome the other:
Timestamp is better than DATETIME when you want to store users session in the database and the session creation time (in Timestamp format) is used for fast row retrieval (with index).
E.g. table may look like this:
[session_create_time AS Timestamp][IP_address AS 32bit Int][etc...]
Having an index on the first two columns can really speed up your queries. If you had a DATETIME value type for the session_create_time field, then it could be taken much more time. Take into account that session queries are executed each time a user request a page, so efficiency is crucial.
DATETIME is better than Timestamp when you want to store a user's date of birth or some historic events that require flexible time range.
Unless digitizing records prior to January 1, 1970, I like the UNIX epoch. Its just a matter of preference, whole unsigned numbers are simpler to deal with when using multiple languages.
Just keep in mind, the epoch starts at January 1, 1970. A lot of companies had been in business for decades, if not longer, prior to that.