I need to store all the sessions in my db, I want to be able to tell if the users latest session has expired.
I tried this but as you can imagine all the sessions that where less the < current time showed.
"SELECT userid FROM session WHERE expiretime < '".date()."'"
basiclly i need to group the userid I know that.
Then I need it to only limit 1 per userid
Then I need to make sure it is the last session id
CORRECT WAY TO DO THIS!
Ok I figured it out to do this you need to use IN
SELECT * FROM `user_sessions` WHERE id IN (SELECT max(id) as id FROM `user_sessions` GROUP BY userid)
There is a rule.
To do something, one has to know what are they doing.
That's extremely handy principle, always helps me.
For example, if I want to compare some date stored in my database, I have to know what format it is. Is it a unix timestamp or datetime or some custom format?
Next thing I'd have to know is what is PHP date() function output when called without parameters and if it match database format. And if it's just an error message - than how to make it match with database format (I'd read a documentation page for this).
Sounds sensible? I hope so.
Next thing I would research is how to perform date calculations in mysql. It seems I will need some comparison using mysql function DATE_SUB()
Honestly, are you sure you want your custom session storage? A regular one already doing all this job for you.
SELECT * FROM `user_sessions` WHERE id IN (SELECT max(id) as id FROM `user_sessions` GROUP BY userid)
Try using the MySQL NOW() function.
"SELECT userid FROM session WHERE expiretime < NOW()"
Related
These 2 querys take too long to produce a result (sometimes 1 min or even sometime end up on some error) and put really heavy load on the server:
("SELECT SUM(`rate`) AS `today_earned` FROM `".PREFIX."traffic_stats` WHERE `userid` = ?i AND from_unixtime(created) > CURRENT_DATE ORDER BY created DESC", $user->data->userid)
("SELECT COUNT(`userid`) AS `total_clicks` FROM `".PREFIX."traffic_stats` WHERE `userid` = ?i", $user->data->userid)
The table has about 4 million rows.
This is the table structure:
I have one index on traffic_id:
If you select anything from traffic_stats table it will take forever, however inserting to this table is normal.
Is it possible to reduce the time spent on executing this query? I use PDO and I am new to all this.
ORDER BY will take a lot of time and since you only need aggregate data (adding numbers or counting numbers is commutative), the ORDER BY will do a lot of useless sorting, costing you time and server power.
You will need to make sure that your indexing is right, you will probably need an index for user_id and for (user_id, created).
Is user_id numeric? If not, then you might consider converting it into numeric type, int for example.
These are improving your query and structure. But let's improve the concept as well. Are insertions and modifications very frequent? Do you absolutely need real-time data, or you can do with quasi-realtime data as well?
If insertions/modifications are not very frequent, or you can do with older data, or the problem is causing huge trouble, then you could do this by running periodically a cron job which would calculate these values and cache them. The application would read them from the cache.
I'm not sure why you accepted an answer, when you really didn't get to the heart of your problem.
I also want to clarify that this is a mysql question, and the fact that you are using PDO or PHP for that matter is not important.
People advised you to utilize EXPLAIN. I would go one further and tell you that you need to use EXPLAIN EXTENDED possibly with the format=json option to get a full picture of what is going on. Looking at your screen shot of the explain, what should jump out at you is that the query looked at over 1m rows to get an answer. This is why your queries are taking so long!
At the end of the day, if you have properly indexed your tables, your goal should be in a large table like this, to have number of rows examined be fairly close to the final result set.
So let's look at the 2nd query, which is quite simple:
("SELECT COUNT(`userid`) AS `total_clicks` FROM `".PREFIX."traffic_stats` WHERE `userid` = ?i", $user->data->userid)
In this case the only thing that is really important is that you have an index on traffic_stats.userid.
I would recommend, that, if you are uncertain at this point, drop all indexes other than the original primary key (traffic_id) index, and start with only an index on the userid column. Run your query. What is the result, and how long does it take? Look at the EXPLAIN EXTENDED. Given the simplicity of the query, you should see that only the index is being used and the rows should match the result.
Now to your first query:
("SELECT SUM(`rate`) AS `today_earned` FROM `".PREFIX."traffic_stats` WHERE `userid` = ?i AND from_unixtime(created) > CURRENT_DATE ORDER BY created DESC", $user->data->userid)
Looking at the WHERE clause there are these criteria:
userid =
from_unixtime(created) > CURRENT_DATE
You already have an index on userid. Despite the advice given previously, it is not necessarily correct to have an index on userid, created, and in your case it is of no value whatsoever.
The reason for this is that you are utilizing a mysql function from_unixtime(created) to transform the raw value of the created column.
Whenever you do this, an index can't be used. You would not have any concerns in doing a comparison with the CURRENT_DATE if you were using the native TIMESTAMP type but in this case, to handle the mismatch, you simply need to convert CURRENT_DATE rather than the created column.
You can do this by passing CURRENT_DATE as a parameter to UNIX_TIMESTAMP.
mysql> select UNIX_TIMESTAMP(), UNIX_TIMESTAMP(CURRENT_DATE);
+------------------+------------------------------+
| UNIX_TIMESTAMP() | UNIX_TIMESTAMP(CURRENT_DATE) |
+------------------+------------------------------+
| 1490059767 | 1490054400 |
+------------------+------------------------------+
1 row in set (0.00 sec)
As you can see from this quick example, UNIX_TIMESTAMP by itself is going to be the current time, but CURRENT_DATE is essentially the start of day, which is apparently what you are looking for.
I'm willing to bet that the number of rows for the current date are going to be fewer in number than the total rows for a user over the history of the system, so this is why you would not want an index on user, created as previously advised in the accepted answer. You might benefit from an index on created, userid.
My advice would be to start with an individual index on each of the columns separately.
("SELECT SUM(`rate`) AS `today_earned` FROM `".PREFIX."traffic_stats` WHERE `userid` = ?i AND created > UNIX_TIMESTAMP(CURRENT_DATE)", $user->data->userid)
And with your re-written query, again assuming that the result set is relatively small, you should see a clean EXPLAIN with rows matching your final result set.
As for whether or not you should apply an ORDER BY, this shouldn't be something you eliminate for performance reasons, but rather because it isn't relevant to your desired result. If you need or want the results ordered by user, then leave it. Unless you are producing a large result set, it shouldn't be a major problem.
In the case of that particular query, since you are doing a SUM(), there is no value of ORDERING the data, because you are only going to get one row back, so in that case I agree with Lajos, but there are many times when you might be utilizing a GROUP BY, and in that case, you might want the final results ordered.
I wrote a small script which uses the concept of long polling.
It works as follows:
jQuery sends the request with some parameters (say lastId) to php
PHP gets the latest id from database and compares with the lastId.
If the lastId is smaller than the newly fetched Id, then it kills the
script and echoes the new records.
From jQuery, i display this output.
I have taken care of all security checks. The problem is when a record is deleted or updated, there is no way to know this.
The nearest solution i can get is to count the number of rows and match it with some saved row count variable. But then, if i have 1000 records, i have to echo out all the 1000 records which can be a big performance issue.
The CRUD functionality of this application is completely separated and runs in a different server. So i dont get to know which record was deleted.
I don't need any help coding wise, but i am looking for some suggestion to make this work while updating and deleting.
Please note, websockets(my fav) and node.js is not an option for me.
Instead of using a certain ID from your table, you could also check when the table itself was modified the last time.
SQL:
SELECT UPDATE_TIME
FROM information_schema.tables
WHERE TABLE_SCHEMA = 'yourdb'
AND TABLE_NAME = 'yourtable';
If successful, the statement should return something like
UPDATE_TIME
2014-04-02 11:12:15
Then use the resulting timestamp instead of the lastid. I am using a very similar technique to display and auto-refresh logs, works like a charm.
You have to adjust the statement to your needs, and replace yourdb and yourtable with the values needed for your application. It also requires you to have access to information_schema.tables, so check if this is available, too.
Two alternative solutions:
If the solution described above is too imprecise for your purpose (it might lead to issues when the table is changed multiple times per second), you might combine that timestamp with your current mechanism with lastid to cover new inserts.
Another way would be to implement a table, in which the current state is logged. This is where your ajax requests check the current state. Then generade triggers in your data tables, which update this table.
You can get the highest ID by
SELECT id FROM table ORDER BY id DESC LIMIT 1
but this is not reliable in my opinion, because you can have ID's of 1, 2, 3, 7 and you insert a new row having the ID 5.
Keep in mind: the highest ID, is not necessarily the most recent row.
The current auto increment value can be obtained by
SELECT AUTO_INCREMENT FROM information_schema.tables
WHERE TABLE_SCHEMA = 'yourdb'
AND TABLE_NAME = 'yourtable';
Maybe a timestamp + microtime is an option for you?
im triyng to figure out an smart way to solve this problem. In my MySql server i have this simplified table:
As you can see, this is an statistic table, im interested on ploting how many visits (profile_visit under the type colunm) recived independently of the ip by each day from the begining of the data, that means that i have to query something like WHERE type=profile_visit AND user_url=xxx. But, this gives me a bunch of rows representing each visit made.
The question is, how can i use this raw data retrived from the query to obtain an array with the total visits by day (i dont care about time)?
Im using PHP, is a good idea to make the adaptation using a php script or it can be done using just MySQL querys?
If i reach the array with the total visits by day i can just simple adapt the format i need by:
$result=mysql_query("SELECT * FROM ".table_stats." WHERE user_url='xxx' AND type='profile_visit'");
echo "data.addRows([";
while($row = mysql_fetch_array($result, MYSQL_ASSOC)) {
$salida = $salida . "['".$row['date']."', $row['total']],";
}
$salida = rtrim($salida, ",");
echo $salida . "]);";
Thanks for any help and orientation about this.
You can easily do this directly from SQL that will run faster than retrieving the information from the DB and then processing it with php. The query should look like:
SELECT datetime, COUNT(id_stat) as numVisits WHERE type="type_profile" AND user_url = "xxx" GROUP BY DATE(DATE_SUB(datetime, INTERVAL 1 DAY))
This will return the number of visits (numVisits) grouped by day, and the lowest datetime recorded that day.
I do not know if you want to display the information just showing the day. If so, you will need to use php to modify the string provided by the DB.
Using your example the result of the query is:
datetime | numVisits
2011-11-10 12:05:44 | 9
2011-11-12 20:06:06 | 3
...
Is this what your after?
SELECT `tbl`.`ip`,COUNT(*) AS `visits` FROM `tbl` WHERE `tbl`.`type` = 'profile_visit' GROUP BY `tbl`.`ip` ORDER BY `visits`
This will return two columns, one with the IP and the other with the respective number of visits (all of type 'profile_vist').
UPDATE
Sorry, I read your question to quickly and missed the time parameter, you'll need something along the line:
SELECT
`tbl`.`ip`,
DATE(`tbl`.`date`) AS `date`,
COUNT(*) AS `visits`
FROM `tbl`
WHERE `tbl`.`type` = 'profile_visit'
AND DATE(`tbl`.`date`) = 'date'
GROUP BY `tbl`.`ip`
ORDER BY `visits`
This will give you a summary on the specific date. If you don't need the IP, remove it from the SELECT-list and GROUP BY(DATE(tbl.date)) instead. To optimize, consider using DATE instead of DATETIME to avoid casting between the two (or adding an additional column).
I have an online highscores made with php + mysql but it currently shows the All Time highscores, I want to add Daily/Weekly/Monthly to that and I was wondering what would be the best way todo that?
My current thought is to add 3 new tables and then have the data inserted into each of them, and then having a cron which would run at the appropriate times to delete the data from each of the tables.
Is there any better way I could do this?
Another thing, I want to have it so the page would be highscores.php?t=all t=daily, etc. How would I make it so that the page changed the query depending on that value?
Thanks.
Use one table and add a column with the date of the highscore. Then have three different queries for each timespan, e.g.
SELECT ... FROM highscores WHERE date>"05-12-2011";
If you want to have a generic version without the need to have a fixed date, use this one:
SELECT ...
FROM highscores
WHERE date >= curdate() - INTERVAL DAYOFWEEK(curdate())+6 DAY;
I am stuck with the following query, which won't order by it's date. Any help, or insight into what I am doing wrong will be much appreciated. The query is supposed to get an entry by thread_id, and then show the newest post in the thread, much like with a forum post, which it does fine. But when I try to order the results from newest to oldest using ORDER BY clause, it seems to ignore it.
$query = "SELECT *
FROM messages
WHERE (thread_id, received)
IN (SELECT thread_id, MAX(received)
FROM messages
WHERE receiver='$user' OR sender='$user'
AND is_hidden_receiver!='1'
GROUP BY thread_id)
ORDER BY received DESC";
Cheers, Lea
You were using the PHP time() function to generate a value to be inserted into an INT(11) column. I'm a little mystified as to why this was sorting incorrectly. I will update this answer if I figure out how to explain it concisely.
This feature is built into MySQL, it is the TIMESTAMP column type. You should probably read up on it a bit more before being happy with this solution. It has some interesting properties, depending on how to define your table, a column of type TIMESTAMP can act either as a creation timestamp or a modification timestamp.
Is the problem that it really isn't sorting by "received", or are you just getting different results than you expect? It could be order of operations on the where clause--I'm not sure if AND or OR takes precedence. Maybe try changing this:
receiver='$user' OR sender='$user' AND is_hidden_receiver!='1'
to whichever one of these you are wanting:
(receiver='$user' OR sender='$user') AND is_hidden_receiver!='1'
receiver='$user' OR (sender='$user' AND is_hidden_receiver!='1')