Now I've read on this fantabulous site about how to check if the timestamp that is in your database is before now(), and that answers part of my question.
But the remainder of my question still eludes me:
How can you not only check for timestamps BEFORE now(), but also put a cap on how many before now are queried?
I have a database that is acting as a guestbook, and the output is a flash piece that looks like a post-it board. you fill out a form, and the submission immediately "tacks" itself to the post-it board. The problem is that when you first load the page, it will load every single post starting from id-0 to id-5,000,000 and so on and so forth.
I would like to put a cap on how many are loaded so the query looks at the following things:
What's the current timestamp?
Go back in time (for example) 10 entries ago
Post from THAT point on
The fields I have in my database are: id, comments, timestamp
EDIT
I'm looking at some of the answers, and I would like to ask about the LIMIT. If I put a limit on the queries, will I still be able to query posts that are PAST now?
Example: Say there are two people viewing the site at the same time. One visitor posts a comment into the database. I want person 2 to still be able to the see that comment pop up on his end.
The flash post-it area runs off a php script that queries the database and exports it into an XML format that Flash can read. Every 5 seconds, the flash re-queries the database to check for more posts.
If I put the Limit on the query, will I still be able to grab NEW entries on the fly? I just want the LIMIT to generate a starting point offset from ground zero
I think what you are looking for is called Limit
You just put it at the end of your statement and the query will return the amount of results you wanted
YOUR QUERY LIMIT 0,10
This will return 10 first results
SELECT * FROM
(SELECT ... WHERE dt<NOW() ORDER BY dt DESC LIMIT 10) a
ORDER BY a.dt ASC
or
SELECT ... WHERE dt<NOW() ORDER BY dt DESC LIMIT 10
check which is the more suitable for you.
Related
I have two queries ultimately I think they will be in the same context of the other but in all. I have a user database that I want to pull out for tracking records based on hour. Example registrations per hour. But in this registrations per hour I want to have the query to dump results by hour increments (or weeks, or months) ie: 1,000 regitrations in november, 1,014 in december and so on, or similar for weeks hours.
I also have a similar query where I want to generate a list of states with the counts next to them of how many users I have per state.
My issue is, I'm thinking I think to one dimensionally currently cause the best idea I can think of at the moment is making in the case of the states 50 queries, but I know thats insane, and there has to be an easier way thats less intense. So thats what Im hoping someone from here can help me with, by giving me a general idea. Cause I don't know which is the best course of action for this currently.. be it using distinct, group_by or something else.
Experiment a bit and see if that doesn't help you focus on the question a bit more.
Try selecting from your registrations per hour table and appending the time buckets you are interested in to the select list.
like this:
select userid, regid, date_time, week(date_time), year(date_time), day(date_time)
from registraions;
you can then roll up and count things in that table by using group by and an aggregate function like this:
select count(distinct userid), year(date_time)
from registraions
group by year(date_time)
Read about about date time functions:
MySQL Date Time Functions
Read about aggregate functions"
MySQL Group By
I'm running a contest where I count up all the daily comments on a webpage per unique commenter each day. At the end of the month, the person with the most comment-votes wins. I have logic written for this that worked perfectly ... until today.
When a page has more than 900 or so comments, the vote-counter stops going up. I figured this had to do with pagination, so I altered my FQL query to use LIMIT and OFFSET to parse 100 comments at a time and combine the results for my vote-counting function, but when running my queries (even by hand, putting the https://graph.facebook.com/comments url in the address bar) it craps out somewhere between 500 and 600 comments, so the last 400 or so comments on the webpage never even return!
Since this method is failing is there a more reliable way to get the comment data?
UPDATE: I'm supplying some code, but I've found the same problems persist in FQL as in the Graph API. Here's my graph API call:
https://graph.facebook.com/comments/?ids=http://www.burlesquebitch.com/article.php?id=538&limit=500
If you crank the limit to over ~600 where more than 900 comments exist (or add an offset that would put the results past that range) it begins to fail. The earlier comments disappear. When I count up the results they're somewhere between 500-600.
https://graph.facebook.com/comments/?ids=http://www.burlesquebitch.com/article.php?id=538&limit=1000
Then I finally got fql working:
$fql_query_result = file_get_contents("https://graph.facebook.com/fql?q=SELECT+object_id,+id,+text,+time+,+fromid+FROM+comment+WHERE+object_id='366760226746431'+LIMIT+1000");
$data = json_decode($fql_query_result,true);
It also fails, but more like around 800-900. In any case the bad behavior is the same. I even did some experiments where I limited the query by time hoping that by isolated my results to a date range I would keep myself within it's failing threshold:
$fql_query_result = file_get_contents("https://graph.facebook.com/fql?q=SELECT+object_id,+id,+text,+time,+fromid+FROM+comment+WHERE+object_id='366760226746431'+AND+time>1365984000+AND+time<1366416000+LIMIT+500");
$data = json_decode($fql_query_result,true);
But no dice. If I flip the < to > in my time comparison the entire query fails and returns nothing. Idealy I would want to get the paging information, as suggested in the answer below, but I can't SELECT it in my query... if I try and get anything that ends in _cursor the query returns empty.
I am forced to conclude that whatever horrendous bug lives in the Graph API code its extended into the FQL code as well. I know of no other way to get data from a facebook object this size.
You should try out cursor pagination, it's recommended as explained at https://developers.facebook.com/docs/reference/api/pagination/
Returned results under cursor paging more consistently match the limit
requested, even after hiding any records for which you do not have
permissions to view (eg. if you request 10 records, but do not have
permissions to see 3 of those records, 3 additional records will be
pulled transparently, so that a full 10 records are pulled).
Example with post_id_cursor:
SELECT text, post_id, post_id_cursor FROM comment WHERE post_id='22707976849_10151395520781850' ORDER BY time DESC limit 50
You get the post_id_cursor of the last comment, then navigate next page with >post_id_cursor symbol
SELECT text, post_id, post_id_cursor FROM comment WHERE post_id='22707976849_10151395520781850' AND post_id_cursor>'Mjg3NA==' ORDER BY time DESC limit 50
Example with object_id_cursor is same:
SELECT text, post_id, object_id_cursor FROM comment WHERE object_id='10151395520696850' ORDER BY time DESC limit 50
SELECT text, post_id, time, object_id_cursor FROM comment WHERE object_id='10151395520696850' AND object_id_cursor>'Mjg3NA==' ORDER BY time DESC limit 50
Make sure you enabled "July 2013 Breaking Changes:" field at your app advanced settings,
https://developers.facebook.com/apps/YOUR_APP_ID/advanced. More info at https://developers.facebook.com/roadmap
I am trying to implement the pagination in php. I am using the Mysql as back end database. I am trying to implement the pagination logic.
I would be having lots of record. But the user will see only 10 at a time.
Now to show the first page, i do a
SELECT * from USERS LIMIT 10.
Now to get the next 10 and the subsequent 10 records i am not able to write a query. Please help me fetch the in between records to support pagination logic. Also provide if any other suggestions for pagination.
You should use the OFFSET option.
SELECT * FROM Users LIMIT 10 OFFSET 10 (or 20, or 30);
That way you just pass the start position in the request when you hit next (or the page number) and you'll retrieve the records you want.
MySQL's limit feature can take two arguments:
select * from USERS limit 10,10
The above would retrieve 10 rows starting at row 10. Bear in mind that the MySQL row offset is 0 based, not 1. The first argument is the starting row, the second is the page size.
Also, if your page size is consistent, all you need to do is pass in the current page (default to zero). That would then allow you to specify the start row as a page * size.
I'm making a digg-like website that is going to have a homepage with different categories. I want to display the most popular submissions.
Our rating system is simply "likes", like "I like this" and whatnot. We basically want to display the submissions with the highest number of "likes" per time. We want to have three categories: all-time popularity, last week, and last day.
Does anybody know of a way to help? I have no idea how to go about doing this and making it efficient. I thought that we could use some sort of cron-job to run every 10 minutes and pull in the number of likes per the last 10 minutes...but I've been told that's pretty inefficient?
Help?
Thanks!
Typically Digg and Reddit-like sites go by the date of the submission and not the times of the votes. This way all it takes is a simple SQL query to find the top submissions for X time period. Here's a pseudo-query to find the 10 most popular links from the past 24 hours using this method:
select * from submissions
where (current_time - post_time) < 86400
order by score desc limit 10
Basically, this query says to find all the submissions where the number of seconds between now and the time it was posted is less than 86400, which is 24 hours in UNIX time.
If you really want to measure popularity within X time interval, you'll need to store the post and time for every vote in another table:
create table votes (
post foreign key references submissions(id),
time datetime,
vote integer); -- +1 for upvote, -1 for downvote
Then you can generate a list of the most popular posts between X and Y times like so:
select sum(vote), post from votes
where X < time and time < Y
group by post
order by sum(vote) desc limit 10;
From here you're just a hop, skip, and inner join away from getting the post data tied to the returned ids.
Do you have a decent DB setup? Can we please hear about your CREATE TABLE details and indices? Assuming a sane setup, the DB should be able to pull the counts you require fast enough to suit your needs! For example (net of indices and keys, that somewhat depend on what DB engine you're using), given two tables:
CREATE TABLE submissions (subid INT, when DATETIME, etc etc)
CREATE TABLE likes (subid INT, when DATETIME, etc etc)
you can get the top 33 all-time popular submissions as
SELECT *, COUNT(likes.subid) AS score
FROM submissions
JOIN likes USING(subid)
GROUP BY submissions.subid
ORDER BY COUNT(likes.subid) DESC
LIMIT 33
and those voted for within a certain time range as
SELECT *, COUNT(likes.subid) AS score
FROM submissions
JOIN likes USING(subid)
WHERE likes.when BETWEEN initial_time AND final_time
GROUP BY submissions.subid
ORDER BY COUNT(likes.subid) DESC
LIMIT 33
If you were storing "votes" (positive or negative) in likes, instead of just counting each entry there as +1, you could simply use SUM(likes.vote) instead of the COUNTs.
For stable list like alltime, lastweek, because they are not supposed to change really fast so that I think you should save the list in your cache with expiration time is around 1 days or longer.
If you concern about correct count in real time, you can check at every page view by comparing the page with lowest page in the cache.
All you need to do is to care for synchronizing between the cache and actual database.
thethanghn
Queries where the order is some function of the current time can become real performance problems. Things get much simpler if you can bucket by calendar time and update scores for each bucket as people vote.
To complete nobody_'s answer I would suggest you read up on the documentation (if you are using MySQL of course).
I mean what the most efficient way to get information about the quantity of your page's items and make sql query with LIMIT that you need. or I should get all items and then crop array with php functions?
now I do 2 queries: first to count all items and second to get items that I need with LIMIT.
OK, I'll be more concrete. For example I need to show a question on my page and 20 answers to this question. At the bottom there shold be page control: links to the next, prev page and so on. I want to show proper number of links (number of answers/20) and when I go to any link I want to recieve proper answers (for example 41 to 60 on the 3d page). So what's the best way to get number of items (answers) to show proper number of links and to get proper answers for each link?
I guess your'e trying to say you want to know how many items/answers there is in the query but only read up to 20 items at at time, for pagination.
Firstly: You really should look for a pagination package; lots and lots of people have had the same problem before and there probably exists both free/opensource and proprietary solutions for your programming language and framework. (If you say what language you are using I'm sure someone can reccomend a solution for you.)
Anyway, I know I like to know how things work, so this is how it usually does:
As far as I know the pagination code calculates the pages by doing one query using select count(*) from tblX where something divide this number with the items-per-page number and use ceiling (e.g. 4.1 => 5).
For listing the results per page a new query is required; don't worry the count query is terribly much faster than getting every result discarding the ones you don't need DO NOT DO THAT (that's the recipie for becoming the top story on this page). Something like select * from tblX where something limit Y offset Z where Y is the number of items per page, and Z is the the (requested_page - 1)*Y; page 1 will have offset 0, page 2 have offset 20 (if thats what Y are) etc..
But do not try to implement this manually, it's unneccesary, tedious and error prone, much better to use your time customizing a readymade solution.
I'm assuming you want a count of the number of rows you'll be reading so as to do some pagination or similar? I don't understand your need for the LIMIT in the context of your question. However, if you just want a count of how many rows have been found, use one of the following.
You select the count of all rows such as:
select count(*) as counted, name, address
from contact
Or found rows:
SELECT SQL_CALC_FOUND_ROWS, name, address
from contact
This may be mysql specific I'm not sure.
Update:
For pagination you would do something like the following - (Psuedocode)
$rows = array($result)
$num_rows = sql_calc_found_rows
$per_page = 20
$pages = ceil($num_rows / $per_page)
page
$rows_this_page = array()
$rows_this_page = get_values($rows, (min index)$page_number * $per_page - $per_page, (max index)$page_number * $per_page - 1)