want to create a facebook type newsfeed that basically lists out different events by their timestamp. These events reside in different tables. I have some code that lets me order a result set by timestamp from the different tables. My problem is I don't know the right way to combine the results from queries of different tables into one result set. I have a feeling that UNION might be the way to do it, however, UNION seems to require that the result sets have similar numbers of columns, datatypes etc. that seems a bit hard to enforce.
Some posts have suggested the right way to do this would be to crate a separate newsfeed table but that sounds cumbersome.
This is what I have so far. Newsfeed draws different events such as comments, photo posted etc.
table comments
id|comment|timestamp
photos
id|name|timestamp
$sql = "SELECT p.name,p.timestamp from `photos` p
JOIN
SELECT c.commentc.timestamp from `comments` c
ORDER GREATEST(IFNULL(p.timestamp,0), IFNULL(c.timestamp,0))"
You can use UNION and just use null in the position of non-matching columns
SELECT id, comment, null as name, timestamp FROM comments
UNION
SELECT id, null as comment, name, timestamp FROM photos;
SQLFiddle example - http://sqlfiddle.com/#!2/f9f40f/1
Related
We need to grab the last and newest 20 entries from different tables. However, the GROUP BY statement skips records because we are working with LEFT JOIN on tables.
All these records are linked to unique persons in another table. We store these person's id's in an array for more queries later.
We have a few tables (in which all those person id's are stored) and we want to get them sorted and grouped.
The tables are like this:
SELECT lastRecord+personID FROM t1
SELECT lastRecord+personID FROM t2
SELECT lastRecord+personID FROM t3
SELECT lastRecord+personID FROM t4
WHERE t5.Essential_Column_Name = '1'
GROUP BY personID
ORDER BY 'all the latest entries'
LIMIT 20
With that, the relevance of all the latest entries should be equal.
We do have a timestamp column as well. Perhaps that might work better.
Any input is highly appreciated!
For people looking for an answer on this; this is the right post, answer and update to this Q:
UNION mysql gives weird numbered results
With thanks to all for the ideas and providing the paths to the right solution.
I'm having some problems retrieving data from two tables and then listing them. I'd like to list the user's feed posts and their likes activity all in one.
Feeds - Table for users posts
Likes - Table for users likes (So when a use likes a post, a record is added to likes (Table likes contains data which contains the feeds ID of the post liked)
What I'm attempting to make: List BOTH feeds and user's Like activity in an ACTIVITY WALL.
So it should output like (ordered by timestamp desc):
"THIS IS A POST by user A"
Shows that user C liked user B's post
"THIS IS A POST by user B"
"THIS IS A POST by user L"
Shows that user A liked user F's post
"THIS IS A POST by user F"
-and it goes on-
My current SQL:
SELECT * FROM feeds,likes WHERE feeds.deleted!=0 or likes.deleted!=0 ORDER BY feeds.timestamp, likes.timestamp
However, my problem is I have no idea how to link both tables, since the IDs in my 'feeds' differ from those in 'likes'
To combine the two sets, you can use a UNION ALL set operator.
Something like this:
SELECT f.timestamp AS `timestamp`
, 'feed' AS `src`
, f.feed_id AS `id`
, f.feed_content AS `content`
FROM feeds f
WHERE f.deleted!=0
UNION ALL
SELECT l.timestamp AS `timestamp`
, 'like' AS `src`
, l.like_id AS `id`
, l.note AS `content`
FROM likes l
WHERE l.deleted!=0
ORDER BY 1 DESC
Note the the queries (on either side of the UNION ALL operator) need to match, in terms of the number of columns returned, and the datatype of each column.
To accommodate differences, such as extra columns returned from one table, but not from the other, you can add literal expressions in place of the "missing" columns.
The return of the extra src column is one way we can use to distinguish which query a row was returned by. It's not mandatory to return such a column, but it's something I often find useful. (The src column could be removed from each query, if it's not useful for your use case.)
Note that it's also possible to combine the results from more than two queries in this way, we'd just add another UNION ALL and another query.
The column names in the combined resultset are determined from the first query. The column names and aliases in the second query are ignored.
The ORDER BY applies to the entire set, and follows the last select.
Query should be linked via postID
F=feeds table, L=likes table, U1=usertable linked to owned feeds, U2=usertable linked to likes table
SELECT F.postTitle+' posted by '+ U1.username,'liked by'+U2.username
FROM likes L
LEFT JOIN feeds F on (F.postID=L.postID)
LEFT JOIN users U1 on (U1.userID=F.userID)
LEFT JOIN users U2 on (U2.userID=L.userID)
ORDER BY L.date,L.postID DESC
When you write SELECT * FROM feeds,likes... you are implicitly CROSS JOINing both tables. The engine will combine every record in each table with every record in the other. That is far from what you want.
I don't think you should be "linking" both tables, either. What you need, roughly speaking, is to get every post and every like, and then order that big set according to timestamps.
It sounds more like a UNION between two queries, and an ORDER BY applied to the whole UNION. UNIONs are never easy on the eye, by the way...
The thing with UNIONs is that both sub-queries need to return the same amount of columns. Not knowing exactly which columns you have, I'll show you one possible solution:
SELECT activity, timestamp FROM (
( SELECT CONCAT(u.name,' posted ',f.content) as activity, timestamp
FROM user u
JOIN feed f on (f.user_id=u.id)
WHERE f.deleted!=0
) UNION
( SELECT CONCAT(u.name, ' liked a post by ',u2.name) as activity, timestamp
FROM user u
JOIN likes l on (l.user_id=u.id)
JOIN feed f on (l.feed_id=f.id)
JOIN user u2 on (f.user_id=u2.id)
WHERE l.deleted!=0
)
) as whole_result
ORDER by timestamp desc
You should, of course, modify this to match your structure.
Hope this helps!
I think, it's better to use 3rd table, say, "actions", and insert to it real actions. Then just select rows from this table, joined to "posts" & "users" table.
When user posts articles, o likes an article, insert corresponding row to "actions" table.
actions table:
|id|action_name|user_id|post_id| date |
1 posted 3 3 5/7/2014
2 liked 5 3 5/7/2014
3 liked 4 3 6/7/2014
4 posted 5 6 7/7/2014
5 liked 3 6 7/7/2014
SELECT user_name a, post_title b, action_name c FROM actions c LEFT JOIN users a ON a.id=c.user_id LEFT JOIN posts b ON b.id = c.post_id ORDER BY c.date DESC LIMIT 10
Then, in loop, choose how to display this data, according to "action_name".
In such way you can expand your wall for other activities, +use indexes for better database performance.
I have this SQL query here that grabs the 5 latest news posts. I want to make it so it also grabs the total likes and total news comments in the same query. But the query I made seems to be a little slow when working with large amounts of data so I am trying to see if I can find a better solution. Here it is below:
SELECT *,
`id` as `newscode`,
(SELECT COUNT(*) FROM `likes` WHERE `type`="newspost" AND `code`=`newscode`) as `total_likes`,
(SELECT COUNT(*) FROM `news_comments` WHERE `post_id`=`newscode`) as `total_comments`
FROM `news` ORDER BY `id` DESC LIMIT 5
Here is a SQLFiddle as well: http://sqlfiddle.com/#!2/d3ecbf/1
I would recommend adding a total_likes and total_comments fields to the news table which gets incremented/decremented whenever a like and/or comment is added or removed.
Your likes and news_comments tables should be used for historical purposes only.
This strenuous counting should not be performed every time a page is loaded because that is a complete waste of resources.
You could rewrite this using joins, MySQL has known issues with subqueries, especially when dealing with large data sets:
SELECT n.*,
`id` as `newscode`,
COALESCE(l.TotalLikes, 0) AS `total_likes`,
COALESCE(c.TotalComments, 0) AS `total_comments`
FROM `news` n
LEFT JOIN
( SELECT Code, COUNT(*) AS TotalLikes
FROM `likes`
WHERE `type` = "newspost"
GROUP BY Code
) AS l
ON l.`code` = n.`id`
LEFT JOIN
( SELECT post_id, COUNT(*) AS TotalComments
FROM `news_comments`
GROUP BY post_id
) AS c
ON c.`post_id` = n.`id`
ORDER BY n.`id` DESC LIMIT 5;
The reason is that when you use a join as above, MySQL will materialise the results of the subquery when it is first needed, e.g at the start of this query, mySQL will put the results of:
SELECT post_id, COUNT(*) AS TotalComments
FROM `news_comments`
GROUP BY post_id
into an in memory table and hash post_id for faster lookups. Then for each row in news it only has to look up TotalComments from this hashed table, when you use a correlated subquery it will execute the query once for each row in news, which when news is large will result in a large number of executions. If the initial result set is small you may not see a performance benefit and it may be worse.
Examples on SQL Fiddle
Finally, you may want to index the relevant fields in news_comments and likes. For this particular query I think the following indexes will help:
CREATE INDEX IX_Likes_Code_Type ON Likes (Code, Type);
CREATE INDEX IX_newcomments_post_id ON news_comments (post_id);
Although you may need to split the first index into two:
CREATE INDEX IX_Likes_Code ON Likes (Code);
CREATE INDEX IX_Likes_Type ON Likes (Type);
First check for helping indexes on columns id, post_id and type,code.
I assume this is T-SQL, as that is what I am most familiar with.
First I would check indexes. If that looks good, then I'd check statement. Take a look at your query map to see how it's populating your result.
SQL works backward, so it starts with your last AND statement and goes from there. It'll group them all by code, and then type, and finally give you a count.
Right now, you're grabbing everything with certain codes, regardless of date. When you stated that you want the latest, I assume there is a date column somewhere.
In order to speed things up, add another AND to your WHERE and account for the date. Either last 24 hours, last week, whatever.
I have 3 tables, all with some kind of articles: articles, columns and reports. All tables have a column called date, which contains the Unix timestamp when the record was added.
Now, I want to get the 50 most recently added records of the articles, columns and reports table and list them on a webpage. Because an article and a column can have the same unique ID, it is also necessary to know which result came from which table, for linking to the complete article (www.webpage.com/article/12 or www.webpage.com/column/12 for instance).
What is the best way to achieve this?
Frankly, the best way is probably to pull the elements into different variables. Anyway, here's an alternative if you want to do it all at once:
SELECT a.date, a.title, 'ARTICLE' category
FROM articles a
WHERE date > :date
UNION ALL
SELECT date, title, 'COLUMN' category
FROM columns
WHERE date > :date
UNION ALL
SELECT date, title, 'REPORT' category
FROM reports
WHERE date > :date
Something like that, anyway. The idea is that you can include a flag referencing the source table in each sub-select statement.
I'm trying to solve a MySQL problem, I have two tables:
CATEGORIES (COLUMNS: _id, _name)
POSTS (COLUMNS: _id, _category, _title, _text)
_category field in POSTS is LONGTEXT and can have multiple CATEGORIES _ID's separated only by , using implode(",") PHP function.
I try to list with PHP the 10 most popular categories, and to display in () the posts in them, but without luck.
I'm not very familar with MySQL, I only know how to use SELECT FROM WHERE ORDER LIMIT, INSERT & UPDATE so I will be very happy if someone can give me a good solution. I tried to use IN() but IN() needs the _category field of POSTS to be like this '1','2','3','4', now its 1,2,3,4 without the quotes, so if anyone know how I can transform this field into list without FIELD TYPE SET, I will be pretty happy.
You may want to change your relation model to the following:
Table CATEGORIES with columns:
_id
_name
Table POSTS with columns:
_id
_title
_text
Table POSSESS with columns:
post_id (FOREIGN KEY)
category_id (FOREIGN KEY)
A tuple in POSSESS relation (table) means the post_id is in the category_id category.
the key word for this is "many-to-many" relations, if possible refactor your scheme like Mark Baker wrote.
Using the model that Dyin suggested, you would then use something like this to list the top 10 categories by popularity (assuming that the more posts a category has, the more popular it is):
SELECT
c.*, # get all values for rows in categories
count(p.post_id) AS post_count # here we are counting the posts for each category using a field alias for the count
FROM (
categories AS c, # we are aliasing the tables also to shorten the typing a bit
possess AS p # you could also use aliases to join the same table multiple times
)
WHERE
c.id = p.category_id # link the categories and the possess tables
GROUP BY c.id # without this, the query would just count all posts, this way the result set is separated into groups by category
ORDER BY post_count DESC
LIMIT 10
Given what you said about your experience with SQL, this query might seem a bit over the top for now, but I think you could use as a starting point for learning more, as always, google is your friend. Start by researching how to link tables using foreign keys and joins.
I've used this:
SELECT *, (SELECT COUNT(*) FROM offer WHERE FIND_IN_SET(type._id, offer._type)) AS _count FROM type ORDER BY _count DESC LIMIT 0, 10
Works fine for now, its table type (columns: _id, _name) and offer (columns: .., .., _types,