I have a table with 100+ entries, I want to display fixed number of entries from this table randomly.
I am using the query below.
SELECT * FROM Table1 WHERE active='1' AND id NOT IN
(SELECT ad_id FROM Table1_logs WHERE uid='$username') ORDER BY RAND() LIMIT 5
Table1 contain all the entries, Table1_logs contains the entries which have been utilized by users today.
Problem:
I need to pick 5 entries daily for each users and should not exceed that amount.
when a user utilizes one entry and it is saved in logs, query picks again 5 entries and it remains 5 all the time.
What I want to acheieve:
When a user utilizes one entry, it count should decrease. He should be able to see only 5 entries daily.
You should add a new field to table1 "utilizationCount" . When a link is utilized, you increase the value by one. And in the query, you should order by this field: " order by utilizationCount asc limit 5" .
Related
Lets start by saying that I cant use INDEXING as I need the INSERT, DELETE and UPDATE for this table to be super fast, which they are.
I have a page that displays a summary of order units collected in a database table. To populate the table an order number is created and then individual units associated with that order are scanned into the table to recored which units are associated with each order.
For the purposes of this example the table has the following columns.
id, UID, order, originator, receiver, datetime
The individual unit quantities can be in the 1000's per order and the entire table is growing to hundreds of thousands of units.
The summary page displays the number of units per order and the first and last unit number for each order. I limit the number of orders to be displayed to the last 30 order numbers.
For example:
Order 10 has 200 units. first UID 1510 last UID 1756
Order 11 has 300 units. first UID 1922 last UID 2831
..........
..........
Currently the response time for the query is about 3 seconds as the code performs the following:
Look up the last 30 orders by by id and sort by order number
While looking at each order number in the array
-- Count the number of database rows that have that order number
-- Select the first UID from all the rows as first
-- Select the last UID from all the rows as last
Display the result
I've determined the majority of the time is taken by the Count of the number of units in each order ~1.8 seconds and then determining the first and last numbers in each order ~1 second.
I am really interested in if there is a way to speed up these queries without INDEXING. Here is the code with the queries.
First request selects the last 30 orders processed selected by id and grouped by order number. This gives the last 30 unique order numbers.
$result = mysqli_query($con, "SELECT order, ANY_VALUE(receiver) AS receiver, ANY_VALUE(originator) AS originator, ANY_VALUE(id) AS id
FROM scandb
GROUP BY order
ORDER BY id
DESC LIMIT 30");
While fetching the last 30 order numbers count the number of units and the first and last UID for each order.
while($row=mysqli_fetch_array($result)){
$count = mysqli_fetch_array(mysqli_query($con, "SELECT order, COUNT(*) as count FROM scandb WHERE order ='".$row['order']."' "));
$firstLast = mysqli_fetch_array(mysqli_query($con, "SELECT (SELECT UID FROM scandb WHERE orderNumber ='".$row['order']."' ORDER BY UID LIMIT 1) as 'first', (SELECT UID FROM barcode WHERE order ='".$row['order']."' ORDER BY UID DESC LIMIT 1) as 'last'"));
echo "<td align= center>".$count['count']."</td>";
echo "<td align= center>".$firstLast['first']."</td>";
echo "<td align= center>".$firstLast['last']."</td>";
}
With 100K lines in the database this whole query is taking about 3 seconds. The majority of the time is in the $count and $firstlast queries. I'd like to know if there is a more efficient way to get this same data in a faster time without Indexing the table. Any special tricks that anyone has would be greatly appreciated.
Design your database with caution
This first tip may seems obvious, but the fact is that most database problems come from badly-designed table structure.
For example, I have seen people storing information such as client info and payment info in the same database column. For both the database system and developers who will have to work on it, this is not a good thing.
When creating a database, always put information on various tables, use clear naming standards and make use of primary keys.
Know what you should optimize
If you want to optimize a specific query, it is extremely useful to be able to get an in-depth look at the result of a query. Using the EXPLAIN statement, you will get lots of useful info on the result produced by a specific query, as shown in the example below:
EXPLAIN SELECT * FROM ref_table,other_table WHERE ref_table.key_column=other_table.column;
Don’t select what you don’t need
A very common way to get the desired data is to use the * symbol, which will get all fields from the desired table:
SELECT * FROM wp_posts;
Instead, you should definitely select only the desired fields as shown in the example below. On a very small site with, let’s say, one visitor per minute, that wouldn’t make a difference. But on a site such as Cats Who Code, it saves a lot of work for the database.
SELECT title, excerpt, author FROM wp_posts;
Avoid queries in loops
When using SQL along with a programming language such as PHP, it can be tempting to use SQL queries inside a loop. But doing so is like hammering your database with queries.
This example illustrates the whole “queries in loops” problem:
foreach ($display_order as $id => $ordinal) {
$sql = "UPDATE categories SET display_order = $ordinal WHERE id = $id";
mysql_query($sql);
}
Here is what you should do instead:
UPDATE categories
SET display_order = CASE id
WHEN 1 THEN 3
WHEN 2 THEN 4
WHEN 3 THEN 5
END
WHERE id IN (1,2,3)
Use join instead of subqueries
As a programmer, subqueries are something that you can be tempted to use and abuse. Subqueries, as show below, can be very useful:
SELECT a.id,
(SELECT MAX(created)
FROM posts
WHERE author_id = a.id)
AS latest_post FROM authors a
Although subqueries are useful, they often can be replaced by a join, which is definitely faster to execute.
SELECT a.id, MAX(p.created) AS latest_post
FROM authors a
INNER JOIN posts p
ON (a.id = p.author_id)
GROUP BY a.id
Source: http://20bits.com/articles/10-tips-for-optimizing-mysql-queries-that-dont-suck/
I have a pretty simple query,
$query3 = $db->query("SELECT * FROM mybb_ranks WHERE id='1' ORDER by points DESC");
And what it'll return is a database of people who are registered and ranked. Since players points can be randomly changing due to matches, we determine the rank # by assigning it once the value is fetched in a way like this:
$i = 1;
while($row = mysqli_fetch_array($query5))
{
echo "$row[player]'s rank is $i";
$i++;
}
As you can see, it orders the player's by their points, but determines the rank # from a variable that adds after displaying every rank. However, in each of the user profiles, I would like to display their rank as well. This is a bit more difficult, because I need a certain way to count the amount of rows the query has to go through in order to get to the certain player. So for instance, if PlayerB is ranked at #5, I would need a way to display that on his own profile. For this to happen, I imagine the query would need to be altered to be able to count each individual row (4 rows) before it reaches the certain player on the 5th row. I was wondering, how would I go about this?
Try this:
UPDATE mybb_ranks
LEFT JOIN (
SELECT
player,
#rank:=#rank+1 as current_rank
FROM mybb_ranks
WHERE id='1'
ORDER BY points DESC
) AS t
ON mybb_ranks.player = t.player
SET mybb_ranks.rank = t.current_rank
That means you have to create additional column rank in your mybb_ranks table.
This query will update the rank of user each time you generate your ranks list.
So when you need to show user's rank in his profile page you just request it from the table:
SELECT rank
FROM mybb_ranks
WHERE player = :player_id
if you want it more dynamic, you can run this UPDATE query every time when you generate your player profile page, right before SELECT rank.
I'm building a chat for a turn-based game (using PHP and MySQL) where I only want to save the 30 most recent messages in the MySQL database.
Currently, I'm doing 3 queries to delete the oldest message when the count exceeds 30:
Insert the new chat message as a new row:
INSERT INTO chatMessages (userId,matchId,message,timestamp) VALUES (".$params["userId"].",".$params["matchId"].",'".$params["message"]."',NOW())
Check if there are too many messages in the database, and get the oldest one's Id.
SELECT COUNT(id) AS count, id AS oldestId FROM chatMessages WHERE matchId = ".$params["matchId"]." ORDER BY id ASC LIMIT 1
Remove the oldest message
DELETE FROM chatMessages WHERE id = ".$oldestId
Is there any way to do this in 2, or even 1 single query? We have quite a lot of traffic on our servers, so performance is key.
You can reduce the 2nd and 3rd query to one:
delete from chatMessages
where id not in (select id from chatMessages
order by id desc
limit 30)
You could put that in an insert trigger. That way oldest records would get deleted automatically after an insert.
Ok, I have 3 tables;
'bu_blogs' contains blogs which have a unique blog_id.
'bu_sites' contains sites which have a unique site_id.
'bu_blogs_done' contains id, blog_id and site_id. A new row is added to this table every time a site_id is submitted to a blog_id.
What I want to do is SELECT 2 random rows from 'bu_blogs' where a field in 'bu_blogs_done' for the particular blog_id and site_id does not exist, i.e it haven't been submitted to that blog_id yet.
Thanks
Stian
If your table isn't too big (e.g. approx 100 rows), you can use something like this simple example for the random part:
SELECT * FROM bu_blogs ORDER BY RAND() LIMIT 2
Then it's just a case of adding a WHERE clause to filter out the ones that exist in bu_blogs_done.
I have a mysql database. How do I select the most recent 10 records? Im not storing timestamps. But the most the recent records are the ones at the bottom rite? Also. How so I get the next ten , the next ten and so on on clicking a button. Kinda like a bunch of forum posts. The recent ones show up first.
I believe you have an auto increment column as a primary key you can use this column and to order by desc
select * from table order by id desc limit 10
otherwise you have a very poor database design
If you have an AUTO_INCREMENT column you can order by that in descending order then limit by 10.
But I suggest you store timestamps and order by that instead so you know you're sorting your records according to date, and not some other value that coincides with date of insertion.
In addition to what #BoltClock mentioned, prequerying the maximum ID might help the engine with what other records are retrieved... ie: if you have a million records, and most recent 10, I don't know if it will still try to query out the million, order them, and THEN dump it.. I would try something like
select STRAIGHT_JOIN
YT.*
from
( select max( IDColumn ) as MaxOnFile
from YourTable ) PreQuery,
YourTable YT
where
YT.IDColumn >= PreQuery.MaxOnFile -10
order by
YT.IDColumn DESC
limit 10
However, if for some reason, records are allowed to be deleted, you may opt to subtract a little farther back than the -10... but at least this way, the system won't even TRY to process all the other records...