I have a custom php blog where I have categories and posts.
In a side widget I have listed all my category titles and next to each of them I'd like to show how many posts they contain.
How can I do that?
My categories table has 2 columns: cat_id and cat_title
My posts table has a post_category_id column.
As your question is extremely simple I won't give you the code, but give you the answer so you have to implement it. For solving your problem you have two options:
1. To use WHERE statement
You can read about it here.
2. To use Inner Join
You can read about it here.
The first one is easier, and as far as I know, it's also faster.
Counting the results:
After filtering the results and getting it through PHP, you can get the rows count with int $mysqli_result->num_rows; (OO style) or int mysqli_num_rows ( mysqli_result $result ) (procedural style) (documentation).
Why not use:
$count = count($mysqli_result_array);
It's trivial enough to count the number of posts per category in SQL.
select post_category_id, count(*) from posts group by post_category_id;
HOWEVER, this query requires a full table scan of your posts, so it is much too slow to do every time your page is accessed. There are several solutions to this problem.
One solution is to add a post_count column to your categories table, and periodically run a query to update your post-counts.
update categories
set categories.post_count = i.cnt
from (
select post_category_id, count(*) cnt
from posts
group by post_category_id
) i
where categories.cat_id = i.post_category_id
Another solution is to add a post_count column to your categories table, and create a trigger for your posts table which automatically updates the counts every time a post is added (or removed). This solution has advantages (the counts are always up to date, and are incrementally updated) and disadvantages (every posting now creates lock contention over the post counts). Trigger code can vary depending on exactly which database you are using, but the insert trigger should look something like this:
CREATE TRIGGER post_count_trigger ON posts
AFTER INSERT
BEGIN
update categories set post_count=post_count+1 where cat_id=post_categori_id
END
Related
Let's assume a user is following thousands of other people,
These people send news regularly, and in his/her page, our user wants to see the recent news (paginated) from these people only.
What is the most efficient way to do this?
This is what I'm doing currently:
Create a table called following in database, each follow is added here, id, user_id, following_user_id
Get a list of user's following_user_ids
fetch all news WHERE user_id (news poster id) is IN(...following_user_ids...)
For example if our user's id is 1:
SELECT `following_user_id` FROM `following` WHERE `user_id` = 1; /* This is used in the IN() below */
SELECT * FROM `news` WHERE `user_id` IN (4,11,7,...following_user_ids....) ORDER BY `id` DESC limit 50 offset 0
/* Of course the `user_id` is indexed in the `news` table */
But if the user is following thousands of people and the news table is huge, I'm assuming the IN (... thousands of IDs ...) will be very slow?
So, is there a more efficient way to do this?
EDIT:
In case any one also has this issue, just stick with the IN method, it is a lot faster than JOIN in my case.
select
news.*
from
news
join following on news.user_id=following.following_user_id
where
following.user_id=1
Pagination
OFFSET has a problem. As he pages forward/backward and others are inserting new rows, he will miss stories or see the same story twice on consecutive pages.
The solution is to "remember where you left off". More: http://mysql.rjweb.org/doc.php/pagination
JOIN
The JOIN approach is cleaner, but not necessarily faster. In either case, the end result is a large list of stories, of which he is only interested in a page's worth. Shoveling the rest around is costly.
The fix for this is to find only the ids of the stories while finding the page's worth. Then look up (via another JOIN) the rest of the data for each story.
Prebuilt list
Still, if there are thousands of followed people (or millions of followers, in the case of Trump), it gets quite costly. There is a technique for making the SELECT faster at the cost of INSERTs needing to run around and store information.
Have a new 3-column table: (1) follower_id, (2) timestamp, (3) story_id. Whenever a story is posted, one row per follower is added to this table. When a follower wants the latest stories, it is sitting right in this table (or at least the ids are).
More: http://mysql.rjweb.org/doc.php/lists
You can limit your search by using the 'LIMIT' function, that will need to be updated everytime the user want more information:
LIMIT [offset,] row_count;
Putting it in your example would be something like this, saving this select in a temporary table variable:
SELECT * FROM `following_user_ids` ORDER BY `id` DESC limit rowcount offset offset_variable;
If you want to put in the example of a social media, you can update the limit everytime the user asks for more posts, so that the user will be able to see the posts of several of the he follows.
I'm not very experienced with more advanced MySQL query stuff.. (mostly basic queries, return and parse response..etc)
However.. I am not clear on the correct approach when I need multiple things (responses) from the database.. Is there a way to get these things from the single query? or do I need to do a new query for each time?
Background:
I use PDO to do a SELECT statement
ie:
$getAllVideos_sql = "SELECT * as FROM $tableName WHERE active IS NOT NULL OR active != 'no' ORDER BY topic, speaker_last, title;";
$getAllVideos_stmt = $conn->prepare($getAllVideos_sql);
$getAllVideos_stmt->execute();
$getAllVideos_stmt->setFetchMode(PDO::FETCH_ASSOC);
$results = $getAllVideos_stmt->fetch(PDO::FETCH_ASSOC);
//parse as I see fit
This gives me my 'chunk of data' that I can pick apart and display as I want.
However.. I want to also be able to give some stats (totals)
For the total (distinct) 'topics'.. as well as total count for the 'titles' (should all be unique by default)
Do I need to do another query, prepare, execute, setFetchMode, fetch all over again?
Is this the proper way to do this? Or is there a way to crib off the initial commands that are already in play?
To be clear, I'm not really looking for a query... I'm looking to understand the proper way one does this.. when they need several pieces of data like I do? multiple queries and executions..etc?
Or maybe it can and -should- be done in one snippet? With an adjustment to the query itself to return sub select/queries info?
this isnt the correct syntax, because it only returns 1 record..(but the total topic count seems to be correct, even though I only get 1 record returned)
SELECT *, count(DISTINCT topic)as totalTopics, count(DISTINCT title)as totalTitles FROM $tableName;
Maybe this the more proper approach? Try to include these totals/details in the main query to pick out?
Hope this makes sense.
Thanks
I don't think you're going to get anything very clean that'll do this, however something like this might work:
SELECT * from $Table t
INNER JOIN (
SELECT COUNT(DISTINCT Topic) as TotalTopics FROM $Table
) s ON 1 = 1
INNER JOIN (
SELECT COUNT(DISTINCT Title) as TotalTitles FROM $Table
) f ON 1 = 1
WHERE ( Active IS NOT NULL ) AND Active != 'no'
Especially with web applications, many people are regularly doing counts or other aggregations somewhere along the way. Sometimes if it is a global context such as all topics for all users, having some stored aggregates helps rather than requerying all record counts every time.
Example. If you have a table with a list of "topics", have a column in there for uniqueTitleCount. Then, based on a trigger, when a new title is added to a topic, the count is automatically updated by adding 1. You can pre-populate this column by doing a correlated update to said "topics" table, then once the trigger is set, you can just have that column.
This also works as I see many times that people want "the most recent". If your system has auto-increment IDs in the tables, similarly, have the most recent ID created for a given topic, or even most recent for a given title/document/thread so you don't have to keep doing something like.
select documentID, other_stuff
from sometable
where documentID in ( select max( documentID )
from sometable
where the title = 'something' )
Use where these make sense then your optimization pull-downs get easier to handle. You could even have a counter per document "title" and even a most recent posting date so they can quickly be sorted based on interest, frequency of activity, whatever.
My PHP application has the function product_fetch([parameters]) which returns 'Product' objects which information are stored in database.
In my admin area, there is a page called "Featured Products" which allows me to select 10 products to be displayed in the main page.
Now comes the problem: I made 10 select/combobox, each allows me to select one product, out of 400. So in order to make all the options, a query has to be made: SELECT * FROM products
Question: Is it correct to make such a query, even though there's hundreds of rows?
The solution you proposed is certainly do-able, and 400 rows is really meek compared to the upper limits of what MySQL is capable of handling. What is more concerning is the user experience here. Granted this will only affect you, but I would design myself something a little nicer than a bunch of <select>s. My idea is to start with just one textbox that autocompletes the names of your products. This can be accomplished if the product title has a fulltext index. Then your autocomplete script could use this query:
SELECT * FROM Products WHERE MATCH(title) AGAINST ('contents of textbox' IN BOOLEAN MODE);
There are plenty of jQuery plugins like Autocomplete that will handle the JS side (querying the server for autocomplete results). The one I just mentioned adds a term GET parameter which you could easily grab and throw into the query:
// You might want to only select the relevant columns
$stmt = $pdo->prepare('SELECT * FROM Products WHERE MATCH(title) AGAINST (:search IN BOOLEAN MODE)');
$stmt->execute(array(':search' => $_GET['term']);
// Output JSON results
echo json_encode($stmt->fetchall());
Once you type in (or click on an autocomplete result) in the one textbox, another should appear below it and the focus should go to it. From there you can type another product, and continue until you reach 10. Each textbox (unless there is only one) should have a delete link next to it that removes that product from the featured listing.
I'll look for a site that implements this kind of functionality so you can better understand what I'm talking about. Basically here, what you're searching for is achieved. You don't have to have 10 select boxes with 400 options and you don't need a SELECT * FROM Products.
You would be much better of specifying which produts you want in the query and only returning those if you have no intention of using any of the others at all.
You can do this using many methods. A simple one would be using an ID field in an in statement like this:
select col1, col2 from products where id in(1,4,12,5)
It might seem to make little difference, but what if your procuts table had a hundred thousand rows in it?
You could also have a flag in the table to say that the items are featured which would let you use something like this:
select col1, col2 from products where featured='Y'
Or you could even have a table that only has featured items (even just their ID) and join it to your main listing like this:
select
a.col1,
a.col2
from
products a
join featured b
on a.id=b.id
If you want to pick through your whole table, you can even use a simple limit clause that picks up a certain number of rows from the table and can be reused to get the next set:
select col1, col2 from products limit 10;
// Will pick the first 10 rows of data.
select col1, col2 from procuts limit 30,10;
// Will pick rows 31-40 (skipping first 30, then returning next 10)
The short version is though, no matter how you do it, pulling back a whole table of data to pick through it within PHP is a bad thing and to be avoided at all costs. It makes the database work much harder, uses more network between the database and PHP (even if they are on the same machine, it is transferring a lot more data between the two of them) and it will by default make PHP use a lot more resources to process the information.
SELECT * FROM tbl LIMIT $page,10;
The above query will select 10 entries from offset $page
Fetching all rows is a bad idea as you are using 10 only anyway. When your table expands to millions of rows, you will see a noticeable difference.
There is nothing fundamentally wrong with selecting all rows if that's what you mean to do. Hundreds of rows also shouldn't be a problem for the query in terms of performance. However, thousands or millions might if your database grows that big.
Think about the user - can they scroll through hundreds of products? Probably. If not, then maybe it's the UI design at fault, not the query.
I am trying to make a social networking site in PHP/MySQL. I am currently developing status update and comments on status's system. I am trying to show all status of mine and comments on certain status. For doing that I have two tables: comment and user_status.
I have used this MySQL query,
SELECT * FROM user_status LEFT JOIN
comment ON id_status = comment.status_id
WHERE sender_id = '$id2'
OR receive_id = '$id2'
/* $id2 is my id */
I have successfully showed status and one comment. But the problem is, when the number of comments are more than one, then the status shows more than one times. How much same status will be showed depends on how much comments available on certain status. But I would like to be able to display same status only one time, and display more than one comments (if available) on certain status.
This isn't so much a PHP problem as it is confusion about how SQL joins work.
It sounds as if what you really want is not so much a join but a distinct set of records from two tables. Until your SQL skils develop a little more, consider simplifying things by making two queries -- one each for comment and user_status. Also consider requesting just the specific fields you're interested rather than using SELECT *.
Here is a visual explanation of different SQL joins, in case you want to pursue this with a single query.
I assume that you are not displaying the raw results from the query, but rather are piping them to an html page to display. Display only the most recent status in a textbox. then display thin a table or list an ordered list of the comments.
Your query is correct.
I know i am writing query's wrong and when we get a lot of traffic, our database gets hit HARD and the page slows to a grind...
I think I need to write queries based on CREATE VIEW from the last 30 days from the CURDATE ?? But not sure where to begin or if this will be MORE efficient query for the database?
Anyways, here is a sample query I have written..
$query_Recordset6 = "SELECT `date`, title, category, url, comments
FROM cute_news
WHERE category LIKE '%45%'
ORDER BY `date` DESC";
Any help or suggestions would be great! I have about 11 queries like this, but I am confident if I could get help on one of these, then I can implement them to the rest!!
Putting a wildcard on the left side of a value comparison:
LIKE '%xyz'
...means that an index can not be used, even if one exists. Might want to consider using Full Text Searching (FTS), which means adding full text indexing.
Normalizing the data would be another step to consider - categories should likely be in a separate table.
SELECT `date`, title, category, url, comments
FROM cute_news
WHERE category LIKE '%45%'
ORDER BY `date` DESC
The LIKE '%45%' means a full table scan will need to be performed. Are you perhaps storing a list of categories in the column? If so creating a new table storing category and news_article_id will allow an index to be used to retrieve the matching records much more efficiently.
OK, time for psychic debugging.
In my mind's eye, I see that query performance would be improved considerably through database normalization, specifically by splitting the category multi-valued column into a a separate table that has two columns: the primary key for cute_news and the category ID.
This would also allow you to directly link said table to the categories table without having to parse it first.
Or, as Chris Date said: "Every row-and-column intersection contains exactly one value from the applicable domain (and nothing else)."
Anything with LIKE '%XXX%' is going to be slow. Its a slow operation.
For something like categories, you might want to separate categories out into another table and use a foreign key in the cute_news table. That way you can have category_id, and use that in the query which will be MUCH faster.
Also, I'm not quite sure why you're talking about using CREATE VIEW. Views will not really help you for speed. Not unless its a materialized view, which MySQL doesn't suppose natively.
If your database is getting hit hard, the solution isn't to make a view (the view is still basically the same amount of work for the database to do), the solution is to cache the results.
This is especially applicable since, from what it sounds like, your data only needs to be refreshed once every 30 days.
I'd guess that your category column is a list of category values like "12,34,45,78" ?
This is not good relational database design. One reason it's not good is as you've discovered: it's incredibly slow to search for a substring that might appear in the middle of that list.
Some people have suggested using fulltext search instead of the LIKE predicate with wildcards, but in this case it's simpler to create another table so you can list one category value per row, with a reference back to your cute_news table:
CREATE TABLE cute_news_category (
news_id INT NOT NULL,
category INT NOT NULL,
PRIMARY KEY (news_id, category),
FOREIGN KEY (news_id) REFERENCES cute_news(news_id)
) ENGINE=InnoDB;
Then you can query and it'll go a lot faster:
SELECT n.`date`, n.title, c.category, n.url, n.comments
FROM cute_news n
JOIN cute_news_category c ON (n.news_id = c.news_id)
WHERE c.category = 45
ORDER BY n.`date` DESC
Any answer is a guess, show:
- the relevant SHOW CREATE TABLE outputs
- the EXPLAIN output from your common queries.
And Bill Karwin's comment certainly applies.
After all this & optimizing, sampling the data into a table with only the last 30 days could still be desired, in which case you're better of running a daily cronjob to do just that.