Hi I want to create a script which detects if certain rows in a mysql table no longer are empty.
I guess i need to run a php script every 5th second which checks certail table rows. If any of them are empty it should continue updating. If not it should redirect to another php page which shows the updated/filled tables.
Any ideas on how to do this?
It is kind of like the game "who wants to be a millionaire" where x persons gets the same question which they need to answer. Answering them will store their answers in certain table rows. When these rows are not empty the main page needs to show all answers and how long time they have spent answering them.
I just need the updste each 5th second script which returns "true" or "false" (if any of the rows are empty).
Cheers,
Mads
Polling table every 5 seconds sounds very wrong.
You should rethink you workflow on how you react to external events.
If you can afford changing your database to PostgreSQL, it offers good solution for this problem. Namely, you create trigger on table of interest, and inside that trigger use NOTIFY.
Another connection that wants to react to table changes, will use LISTEN to register for notifications and then simply go to sleep using select() on connection handle. Then, once trigger has fired, this connection will immediately receive this event and can react on it right away.
This approach allows to use exactly 0 CPU time on both server and client and yet react on changes immediately.
A way to do this :
Set a JavaScript timer when the player answers the question (displaying a 'waiting for other players to answer' message for example) and triggering an asynchronous query to a php script (let's say waiting.php) every 5 seconds. If waiting.php returns 0, nothing happens. In waiting.php returns 1, player is redirected on the page witch will display all the answers and times
In waiting.php, write something like that :
echo (current(mysql_fetch_row(mysqli_query('SELECT count(*) from responses WHERE questionId = '.$_GET['QuestionId'].' AND answer_field IS NULL'))) > 0 ? '0' : '1');
Related
For some reasons (that I think it is not the point of my question, but if it help, ask me and I can describe why), I need to check MySQL tables continuously for new records. If any new records come, I want to do some related actions that are not important now.
Question is, how I should continuously check the database to make sure I am using the lowest resources and getting the results, close to the realtime.
For now, I have this:
$new_record_come = false;
while(! $new_record_come) {
$sql = "SELECT id FROM Notificatins WHERE insert_date > (NOW() - INTERVAL 5 SECONDS)";
$result = $conn->query($sql);
if ($result)
{
//doing some related actions...
$new_record_come = true;
}
else
{
sleep(5); //5 seconds delay
}
}
But I am worry that if I get thousands of users, it will make the server down, even if the server is a high price one!
Do you have any advice to make it better in performance or even change the way completely or even change the type of query or any other suggestion?
Polling a database is costly, so you're right to be wary of that solution.
If you need to scale this application up to handle thousands of concurrent users, you probably should consider additional technology that complements the RDBMS.
For this, I'd suggest using a message queue. After an app inserts a new notification to the database, the app will also post an item to a topic on the message queue. Typically the primary key (id) is the item you post.
Meanwhile, other apps are listening to the topic. They don't need to do polling. The way message queues work is that the client just waits until there's a new item in the queue. The wait will return the item.
A comment suggested using a trigger to invoke a PHP script. This won't work, because triggers execute while the transaction that spawned them is not yet committed. So if the trigger runs a PHP script, which probably needs to read the record from the database. But an uncommitted record is not visible to any other database session, so the PHP script can never read the data that it was notified about.
Another angle (much simpler than message queue I think):
I once implemented this on a website by letting the clients poll AND compare it to their latest id they received.
For example: You have a table with primary key, and want to watch if new items are added.
But you don't want to set up a database connection and query the table if there is nothing new in it.
Let's say the primary key is named 'postid'.
I had a file containing the latest postid.
I updated it with each new entry in tblposts, so it contains alsways the latest postid.
The polling scripts on the clientside simply retrieved that file (do not use PHP, just let Apache serve it, much faster: name it lastpostid.txt or something).
Client compares to its internal latest postid. If it is bigger, the client requests the ones after the last one. This step DOES include a query.
Advantage is that you only query the database when something new is in, and you can also tell the PHP script what your latest postid was, so PHP can only fetch the later ones.
(Not sure if this will work in your situation becuase it assumes an increasing number meaning 'newer'.)
This might not be possible with your current system design but how about instead of using triggers or a heartbeat to poll the database continuously that you go where the updates, etc happen and from there execute other code? This way, you can avoid polling the database continuously and code will fire ONLY IF somebody initiates a request?
User keys in search parameters, then we make a request to a data provider and redirect user to a loading page. The response from the data provider hits a callback url, in which case we parse the results and store about 200 rows into the db. Meanwhile the loading page uses ajax to query the db every second and when the results are all there we display the results to the user.
The issue is that insert into the mysql db is too slow. We know the response back from the data provider comes back within seconds, but the processing of the script and inserting of rows into the db is very slow. We do use multirow insert.
Any suggestions to improve? FYI, the code is hugely long... that's why not displaying right now.
There are multitude of factors affecting your insertions:
1) slow hardware and bad server speeds.
Sol : Contact your server administrator
2) Use something other than InnoDB
3) Use a surrogate key , other than your primary key that is numeric and sequential along with your natural primary key.
OR
4) Try this https://stackoverflow.com/a/2223062/3391466.
Suggestion: Instead of running the code on one page and having the user wait the whole process, why not have the php page store the instructions in a php queue? The instructions would then be executed by a separate php script (for instance a Cron Job) and the user wouldn't have to wait for the whole process to take place.
However, in this situation it would be ideal to let the user know that the changes made can take a bit of time to update.
Cron jobs are very easy to implement. In CPanel there is an option for Cron Jobs where you specify which script you want to run and in which intervals. You can let your script know to run once every 1 minute (or more or less depending on how much demand there is). From there your script would check the queue and could keep on running until the queue is empty again.
Let me know if that helped!
I'm working on a basic lamp(willing to change) website , and I currently need a way to run some function on the server that runs for several hours per user, and every X hours it needs to query the mysql database to see if the value for that user has been updated, if it hasn't it need it to insert a new record in the database...I also should mention that the 'every X hours' can change per user too, and the total runtime of the function per user can also vary.
So basically I need a function that runs continuously on the server for few hours per user. What is the best way to do this? I want the site to be able to support many users (like 10000 +).
I'm willing to try new technologies for every aspect of the site, I'm still in the design phase and I was looking for some input.
I've looked at cron but not really sure how well it would work when dealing with so many users...
edit: Here is a typical scenario of events;
User presses button on the website and closes the browser.
Server starts a timer from when they pressed the button, now
the server will check if that user has pressed a different button within a given time frame (time frame can change per user), say within 30 minutes. If they didn't press the other button then the server needs to automatically insert a new record in the database.
The script will need to continue running, checking every 30 mins for say the next 5 hours.
Thank you!
Cron would work as well as you can code the page it will run. It's not a cron limitation.
The question is ambiguous btw. Maybe explaining your full scenario would help.
Meanwhile, my suggestion would be to set up a scrip that allows you to manually check what you need to check.
You definitely need the DB to be InnoDB optimized with proper indexes to be able to support 1000 plus users.
To alleviate the number of calls to the database, a common practice is to run scripts only on what you are interested (so in the case of users you would only select those who have logged on in say the past 3 hours)
That's achievable in 2 ways, a simple select statement, or by adding entries to a specific table on the login page, and remove them after the automated script has finished running.
All of this is pure theory without understanding exactly what you need to do though.
You are telling what/how you want to do, but not why you want to do it. Maybe letting us know why could lead to a different how ;)
However, what you can do is still use cron (or anything similar). The trick is to have
a last_interaction timestamp column
a maximum_interval column
a daily_runtime column
in your users database. Not optimized but you are in the design phase so you shouldn't pay too much attention to the performance aspect (except is explicitly required).
I'm thinking about a webchat working with PHP, JQuery and MySQL but I'm in some trouble about how the queries should be send to the db.
Structure:
MySQL receives all data from chat and saves it like in a table "conversation" with columns from/to.
Jquery do the queries and verifies if there is any new message for the current user.
Problem:
How to do the queries with Jquery in a way to not overload the db server (in consideration to have a lot of users)? I think to query the db with a timer in Jquery, so within some secs. a new query will be done and it will be repeated for all users at same time.
The best way to do it (according to me) is to have a 1 second interval in jquery. Once a second you call a php file (fetch_new.php for example).
In your database, you have two tables:
users
messages
When a user client makes a call to fetch_new.php (through JavaScript), you update the users table and set a column (last_sync) to the current time (preferably in microtime, that way the user will never get two of the same messages sent to them). The new message(s) get appended to the conversation.
When you execute your query, it should look something like this:
SELECT * FROM `messages` WHERE `to` = $user_id AND `timestamp` > $last_sync
It is perfectly fine to send many questions per second to MySQL as long as it can handle many connections (enough RAM and CPU). Keep an eye on the "Status" tab in PHPMyAdmin to see how much CPU and RAM is being used.
In the chat applications I've built I've added some "intelligence" features. If a user hasn't received any messages in the last (let's say) 20 seconds, I increase the interval to make a request to fetch_new.php every 2 seconds instead of every second. If over a minute has passed, I increase it to 3 seconds and so on. This way you won't slow down your application by having a lot of idle users.
my approach would be something like:
$(document).ready(function()
{
setTimeout(function()
{
// do something here
}, 2000);
});
The above code will execute your function every 2 seconds. Don't know about efficiency though. So perhaps another approach would work better.
So long story short, im trying to build a chat application....well its already built kind of but, ive been trying to implement AJAX into it. Essentially so that as the messages come through from the db when a user posts it, that it reflects on the page automatically without page refresh.
on the javascript side the only event i could think of that would run without user input is the javascript setInterval() function that i have set to repeat # every 5 seconds for now.
on the PHP side which has the messages (echoed out), i have a statement that only displays the
last 10 messages.
The problem is that, every 5 seconds, it spits out the same 10 messages. over and over...broken record kinda lol. anyways so what i dont know/cant figure out how to implement, is a filter kind of, or SOMEthing that says "
hey only show me the last 10 messages and update ONLY when there is
new content.
"
So what i need is something( i assume javascript since its the one refreshing the page) that says only display the latest x posts only when its new content. I was thinking maybe an if/else that compares one value to another and then if false/true, run the refresh...but
what do i compare to???
as you can see, im stuck.
i dont post the code because its alot...dont want to put two chunks of code here but essentially the PHP reads rows from a DB section, and posts the latest 10 posts from the db via echo.
and the Js has AJAx commands that gets those messages and posts them on the page.
And again, both work except for the re-posting the same 10 messages on ajax /js refresh.
any tips,links, LOGIC...anything to help me get through this hump i gladly and humbly appreciate.
P.S - if you need to see code. let me know and ill post it but i think its pretty clear.
Thanks in advanced.
so webchat clients do this thing where they make a request to the server and the server lets it hang, as if its taking a long time to process, but instead, its waiting to see if someone else posts a chat or what have you, the moment it does, it replies to the user, if the timeout max is reached, it just allows the connection to actually timeout, at which point the javascript causes it to fire again. this eliminates the nasty, make a call every 5 sec feature for starters.
Keep a last returned timestamp in your javascript, and when you need to update the clients, send that along, and use that as your sql conditions:
select * from chat_logs WHERE datetime > $postedValueFromClient # AND YOUR OTHER CONDITIONS
Give that query results back to your clients, and update the last timestamp as well.
I can't help you further since I don't have any code, but that's one way of implementing it.
I'd fetch 1 line at a time and remember the ID of the last line that was received, also make a column in your table for instance read and set it to 1 or 0, if the message was receive set to 1 and only look for messages that have read set to 0.
MySQL statement:
SELECT * FROM `table` WHERE `id` > `lastID` AND `read` = 0;