I'm thinking about a webchat working with PHP, JQuery and MySQL but I'm in some trouble about how the queries should be send to the db.
Structure:
MySQL receives all data from chat and saves it like in a table "conversation" with columns from/to.
Jquery do the queries and verifies if there is any new message for the current user.
Problem:
How to do the queries with Jquery in a way to not overload the db server (in consideration to have a lot of users)? I think to query the db with a timer in Jquery, so within some secs. a new query will be done and it will be repeated for all users at same time.
The best way to do it (according to me) is to have a 1 second interval in jquery. Once a second you call a php file (fetch_new.php for example).
In your database, you have two tables:
users
messages
When a user client makes a call to fetch_new.php (through JavaScript), you update the users table and set a column (last_sync) to the current time (preferably in microtime, that way the user will never get two of the same messages sent to them). The new message(s) get appended to the conversation.
When you execute your query, it should look something like this:
SELECT * FROM `messages` WHERE `to` = $user_id AND `timestamp` > $last_sync
It is perfectly fine to send many questions per second to MySQL as long as it can handle many connections (enough RAM and CPU). Keep an eye on the "Status" tab in PHPMyAdmin to see how much CPU and RAM is being used.
In the chat applications I've built I've added some "intelligence" features. If a user hasn't received any messages in the last (let's say) 20 seconds, I increase the interval to make a request to fetch_new.php every 2 seconds instead of every second. If over a minute has passed, I increase it to 3 seconds and so on. This way you won't slow down your application by having a lot of idle users.
my approach would be something like:
$(document).ready(function()
{
setTimeout(function()
{
// do something here
}, 2000);
});
The above code will execute your function every 2 seconds. Don't know about efficiency though. So perhaps another approach would work better.
Related
For some reasons (that I think it is not the point of my question, but if it help, ask me and I can describe why), I need to check MySQL tables continuously for new records. If any new records come, I want to do some related actions that are not important now.
Question is, how I should continuously check the database to make sure I am using the lowest resources and getting the results, close to the realtime.
For now, I have this:
$new_record_come = false;
while(! $new_record_come) {
$sql = "SELECT id FROM Notificatins WHERE insert_date > (NOW() - INTERVAL 5 SECONDS)";
$result = $conn->query($sql);
if ($result)
{
//doing some related actions...
$new_record_come = true;
}
else
{
sleep(5); //5 seconds delay
}
}
But I am worry that if I get thousands of users, it will make the server down, even if the server is a high price one!
Do you have any advice to make it better in performance or even change the way completely or even change the type of query or any other suggestion?
Polling a database is costly, so you're right to be wary of that solution.
If you need to scale this application up to handle thousands of concurrent users, you probably should consider additional technology that complements the RDBMS.
For this, I'd suggest using a message queue. After an app inserts a new notification to the database, the app will also post an item to a topic on the message queue. Typically the primary key (id) is the item you post.
Meanwhile, other apps are listening to the topic. They don't need to do polling. The way message queues work is that the client just waits until there's a new item in the queue. The wait will return the item.
A comment suggested using a trigger to invoke a PHP script. This won't work, because triggers execute while the transaction that spawned them is not yet committed. So if the trigger runs a PHP script, which probably needs to read the record from the database. But an uncommitted record is not visible to any other database session, so the PHP script can never read the data that it was notified about.
Another angle (much simpler than message queue I think):
I once implemented this on a website by letting the clients poll AND compare it to their latest id they received.
For example: You have a table with primary key, and want to watch if new items are added.
But you don't want to set up a database connection and query the table if there is nothing new in it.
Let's say the primary key is named 'postid'.
I had a file containing the latest postid.
I updated it with each new entry in tblposts, so it contains alsways the latest postid.
The polling scripts on the clientside simply retrieved that file (do not use PHP, just let Apache serve it, much faster: name it lastpostid.txt or something).
Client compares to its internal latest postid. If it is bigger, the client requests the ones after the last one. This step DOES include a query.
Advantage is that you only query the database when something new is in, and you can also tell the PHP script what your latest postid was, so PHP can only fetch the later ones.
(Not sure if this will work in your situation becuase it assumes an increasing number meaning 'newer'.)
This might not be possible with your current system design but how about instead of using triggers or a heartbeat to poll the database continuously that you go where the updates, etc happen and from there execute other code? This way, you can avoid polling the database continuously and code will fire ONLY IF somebody initiates a request?
I've doubt regarding speed and latency for show real time data.
Let's assume that I want to show read time data to users by fire ajax requests at every second that get data from MySql table by simple collection query.
For that currently these two options are bubbling in my mind
MySql / Amazon Aurora
File system
Among these options which would be better? Or any other solution?
As I checked practically, if we open one page in browser then ajax requests gives response in less than 500ms using PHP, MySql, Nginx stack.
But if we open more pages then same ajax requests gives response in more than 1 second that should be less than 500ms for every visitors.
So in this case if visitors increase then ajax requests gives very poor response.
I also checked with Node.js+MySql but same result.
Is it good to create json files for records and fetch data from file? Or any other solution?
Indeed, you have to use database to store actual data but you can easily add memory cache (it could be internal dictionary or separate component) to track actual updates.
Than your typical ajax request will look something like:
Memcache, do we have anything new for user 123?
Last update was 10 minutes ago
aha, so nothing new, let's return null;
When you write data:
Put data into database
Update lastupdated time for clients in memcache
Actual key might be different - e.g. chat room id. Idea is to read database only when updates actually happened.
Level 2:
You will burn you webserver and also client internet with high number of calls. You can do something:
DateTime start = DateTime.Now;
while(Now.Subtract(30 seconds) < start)
{
if (hasUpdates) return updates;
sleep(100);
}
Than client will call server 1 time per 30 seconds.
Client will get response immediately when server notices new data.
How would I add an area that shows which users are online until they leave the page or log out?
I thought about making a table that keeps the users session until they leave the page, but how would I be able to tell when they left? I only need it to be updated maybe every 5 seconds. I want to stay away from cron jobs just in case of windows, (windows doesn't support cron does it?)
Thanks for any help
I did similar thing in one of our applications some time ago. Bind an Ajax call to some event like click or mousemove. In that call write the timestamp to the active_users table. Write another function to query this table and select users with timestamp older than say 5 min. Call this via Ajax as well in specific time intervals or on broeser event.
There are many possible solutions to it and you should probably post some code for your implementation if you want code back.
As an overall design question, you would probably need a means for storing how many users are online (more on detecting that below) either in memory (backend service, in memory DB like redis or memcached) or on disk (regular DB).
As for detecting when users are online, it can be a fairly difficult problem, but assuming that exact up to the second numbers don't matter that much, you could increment the number (in the DB or memory from above) when you serve the page (PHP) or when the user loads the page (Javascript).
As for detecting when a user has navigated away, you could use an onunload handler in Javascript such that it notifies the server when the user navigates away. One caveat to watch out for is that this message could get lost or not be sent (ie. JS off, browser crash, etc.) and your count could become wildly inaccurate.
Alternatively, the client can ping the server with a heartbeat every [predefined interval] and the server would know they are online (decrementing the counter when the user stops sending the heartbeat, assuming they have navigated away)
Based on these high level solutions, they could also be combined to form a hybrid solution, using heartbeats to detect that users are still online and an onunload handler to notify the server right away on close/navigation.
Finally, just as a note about cron jobs, Windows doesn't have cron per se but can accomplish a similar goal using Scheduled Tasks.
There are various options. Here is the one I prefer, but your mileage may vary.
Use JavaScript to ask your users to poll/bump your server once in a while and use this request to store which users are online. An example of the JavaScript part using jQuery:
var ajaxCall;
var livedataTimer;
function poll(timeout){
livedataTimer = setTimeout(function(){
if (ajaxCall != null){
ajaxCall.abort();
}
if (timeout == 5){
timeout = 10000;
}
ajaxCall = $.ajax({ url: '/ajax/bump.php', type: 'POST', success: function(data){
poll(timeout);
}, dataType: "json"});
}, timeout);
return livedataTimer;
}
An example of the /ajax/bump.php file:
<?php
session_start(); ## to know which user is online
## connect to your DB somewhere
mysqli_query($connection,"
INSERT INTO `online-status`
(`user`, `time`)
VALUES
(".(int)$_SESSION['user']['id'].", UNIX_TIMESTAMP())
");
return TRUE; ## return TRUE so that ajax knows the file has completed
?>
Now to get the list of online users, you can lookup your online-status table with a condition, for example, "where time is not older than 20 seconds."
It's a good practice to INSERT the data into a separate database, not UPDATE the timestamps in your userlist database. These tend to be quite slow.
I used the time(); function to insert a value into the users table every time a user visited a page, and pulled it out and and used this function,
echo '<h3>Online Users</h3>';
$query = "SELECT * FROM users";
$data = mysqli_query($dbc, $query);
while ($row = mysqli_fetch_array($data)){
if(time() - 10 < $row['active']){
echo ''.$row['username'].'';
}
}
Is there a way to query for a set time from a PHP script?
I want to write a PHP script that takes an id and then queries the MySQL database to see if there is a match. In the case where another user may have not yet uploaded their match, so I am aiming to query until I find a match or until 5 seconds have passed, which I will then return 0.
In pseudocode this is what I was thinking but it doesn't seem like a good method since I've read looping queries isn't good practice.
$id_in = 123;
time_c = time();
time_stop = time + 5; //seconds
while(time_c < time_stop){
time_c = time()
$result = mysql_query('SELECT * WHERE id=$id_in');
}
It sounds like your requirement is to poll some table until a row with a particular ID shows up. You'll need a query like this to do that:
SELECT some-column, another column FROM some-table WHERE id=$id_in
(Pro tip: don't use SELECT * in software.)
It seems that you want to poll for five seconds and then give up. So let's work through this.
One choice is to simply sleep(5), then poll the table using your query. The advantage of this is that it's very simple.
Another choice is what you have. This will make your php program hammer away at the table as fast as it can, over and over, until the poll succeeds or until your five seconds run out. The advantage of this approach is that your php program won't be asleep when the other program hits the table. In other words, it will pick up the change to the table with minimum latency. This choice, however, has an enormous disadvantage. By hammering away at the table as fast as you can, you'll tie up resources on the MySQL server. This is generally wasteful. It will prevent your application from scaling up efficiently (what if you have ten thousand users all doing this?) Specifically, it may slow down the other program trying to hit the table, so it can't get the update done in five seconds.
There's middle ground, however. Try doing a half-second wait
usleep(500000);
right before each time you poll the table. That won't waste MySQL resources as badly. Even if your php program is asleep when the other program hits the table, it won't be asleep for long.
There is no need to do simple polling and sleeping. I don't know your exact requirements, but in general your question asks for GET_LOCK() or PHP's Semaphore support.
Assuming your uploading process starts with
SELECT GET_LOCK("id-123", 0);
Your Query thread can then wait on that lock:
SELECT GET_LOCK("id-123", 5) as locked, entities.*
FROM entities WHERE id = 123;
You might eventually find that a TRIGGER is the thing what you were looking for.
Hi I want to create a script which detects if certain rows in a mysql table no longer are empty.
I guess i need to run a php script every 5th second which checks certail table rows. If any of them are empty it should continue updating. If not it should redirect to another php page which shows the updated/filled tables.
Any ideas on how to do this?
It is kind of like the game "who wants to be a millionaire" where x persons gets the same question which they need to answer. Answering them will store their answers in certain table rows. When these rows are not empty the main page needs to show all answers and how long time they have spent answering them.
I just need the updste each 5th second script which returns "true" or "false" (if any of the rows are empty).
Cheers,
Mads
Polling table every 5 seconds sounds very wrong.
You should rethink you workflow on how you react to external events.
If you can afford changing your database to PostgreSQL, it offers good solution for this problem. Namely, you create trigger on table of interest, and inside that trigger use NOTIFY.
Another connection that wants to react to table changes, will use LISTEN to register for notifications and then simply go to sleep using select() on connection handle. Then, once trigger has fired, this connection will immediately receive this event and can react on it right away.
This approach allows to use exactly 0 CPU time on both server and client and yet react on changes immediately.
A way to do this :
Set a JavaScript timer when the player answers the question (displaying a 'waiting for other players to answer' message for example) and triggering an asynchronous query to a php script (let's say waiting.php) every 5 seconds. If waiting.php returns 0, nothing happens. In waiting.php returns 1, player is redirected on the page witch will display all the answers and times
In waiting.php, write something like that :
echo (current(mysql_fetch_row(mysqli_query('SELECT count(*) from responses WHERE questionId = '.$_GET['QuestionId'].' AND answer_field IS NULL'))) > 0 ? '0' : '1');