I run a procedure that takes about 20 minutes to complete, I just wonder if PHP can keep the connection active until the process finishes.
To be clear, this page will have a button which when you press it will call a php page to run a sql query, in the main page I just wait for the Http request to be complete to send a success message.
For queries that are set to take up some time, you should move some automation requests into the mix, preferably cronjobs if you have access to a linux server.
With cronjobs, you can create enteries in the database for specific values, linked to the user. The cronjob will kick in lets say, every 5 minutes to execute a query if a pending query has finished. This will minimize the fact the user will need to sit on the page until completion. Because you should know, the second the user navigates away from the active page; all active connections, queries etc will stop.
Once the script has complete, make a custom message at the end to send to the user letting them know that their process has been completed.
You should also know, PHP works down the script, from line 1 to the end; so if your hang is on line 40 for example; the script will sit on that line until the query has completed then carry on processing.
Edit: This is for example purposes only to point you in the direction that i'm getting at, and should not be used as you see it. This is merely a markup example
<?php
if (isset($_POST['ButtonToExecute']))
{
// Query to update a table in your database which an external PHP script will work with
// Example Table Structure:
/*
Username
State
*/
if ($state == 0)
{
// Then update state to 1 with username to the username for the query to take place on
}
else
{
// Warn user that their process will take longer to complete as their already is an active query in process
// but add them to a secondry pipeline which will be picked up on the next cronjob interval
}
}
?>
On your cronjob, might have:
<?php
if ($state=='1')
{
// Execute your script
// After query execution update state to 2
// if state == 2 then insert custom message/email to send
}
?>
Edit your crontab:
yourpreferrededitor /etc/crontab
^ Where yourpreferrededitor means your text editor, whether nano or other.
and your cronjob line:
* * * * * root /usr/bin/php /var/www/cron.php
^ This is taken from a current cronjob I have constantly running set for every minute of every day
A Cronjob will give the user the ability to navigate away from the page, because as I mentioned above, the second the user navigates away from the script.. All on going processes stop. The cronjob will carry on running throughout without no user interaction needed (apart from they make the initial request)
You could do it in an ordinary php script if you set the timeout limit and ignore user abort but this is a bad idea because your user will have no feedback if the script worked.
At the company I work at we had the same problem
What we did:
Run the query in a separate script.
Start the script WITHOUT waiting for results
Split the query into multiple parts (using limit and offset)
Have the query script write messages to a temp file
Implement a status screen fetching the current state from the log via ajax
BTW an example for all the wise guys asking why it takes so long:
using a transitive hull can speed up your application a lot if you have to deal with a tree with millions of nodes but building it can take hours.
You could try:
set_time_limit(0);
You may want to take a look at Set Time Limit
In your PHP ini you can set a max-execution time for scripts, however you NEVER want to have a user sit on a page loading screen for that long.
One thing that I could suggest would to e increase your max execution time to something around 30 minutes, and then call this in javascript after a page has already been rendered, so the user will not be left in the dark not knowing what the script is doing.
Related
Trying to make a dice-roll function in my telegram bot.
How it works right now:
When a user sends "roll" bot replies with sendDice method and sends another message with result like "you rolled 5, you won and blah-blah .."
> how it looks <
The problem is — the second message should not appear instantly, ideally after dice-roll animation is finished.
My first and obvious try on that was to add "sleep(3)" before sending the second message, and it worked fine, until I realized it completely delays the execution of my script for those 3 seconds. (if two users rolled at the same time, one of the users has to wait until another guy's roll will be finished). So it's not cool
What can I use? :c
The easiest option is to add the "task" to the "queue". The queue can be a table in the database with timestamps and chat id, when and to whom to send a message. Start another process, for example, which is started by cron, and it works for one minute. During that minute, he goes to the database and checks to see if there is something that needs to be sent now.
Crontab config
Open crontab
sudo crontab -e
Add next string
* * * * * php /path/to/cron.php >> /path/to/log/file/for/debug.log 2>&1
Cron run your script every 1 minute.
Cron.php "live" 60 second
cron.php:
$now = time();
$expectedTime = $now + 60;
while (true) {
Worker::run();
if ($expectedTime < time()) {
die(0);
}
}
Where Worker::run() your method, which get records from db, check timestamp and send message
From Wikipedia:
In computing, [..] fork is an operation whereby a process creates a
copy of itself.
When your PHP script runs, you can create multiple processes that interact with each other. Those processes run concurrently and asynchronous. This way you can have one process waiting to send the message, while the rest of the script continues to run.
Instead of starting another process, you could also start another thread. The technical difference between the two is explained here:
Forking vs Threading
PHP offers Process Control Extensions for both forking and threading. You might want to check out the example in the PHP documentation for pcntl_fork().
Depending on your needs, you might want to use a framework designed to handle concurrency throughout your application. If that is the case, I would recommend amphp.
Everything I google tells me this should not be happening, however it is.
I'm building a migration tool to build a 'master database'. I have an admin panel, only accessible to a few select people. There is a merge button that starts an AJAX call to run the migration php function. I'm not positive how long this script takes considering I'm still developing it but none the less I'm expecting a minimum of 20 minutes once pushed to production and populated with the production database. I do NOT need a lecture on best practices telling me not to do it via a GUI. This will become a cron as well, however I want to be able to induce it manually, if the admin desires.
So here's my process. The migration function immediately closes the session session_write_close() allowing me to run multiple php scripts simultaneously. I do this because I start a setInterval that checks to see a session variable. This is my 'progress' which is just an int on what loop iteration I'm on. In my migration script I open sessions, add 1 to that int, and close the sessions again. I do this at the end of each loop.
By doing this I have successfully created a progress for my AJAX. Now I noticed something. If I start my migration, then close out of my tab - or refresh. Once I reload the page my progress continues to grow. This tells me that the migration script is still executing in the background.
If I close 100% out of my browser, or clear my sessions I no longer see progress go up. This however is not because the script stops. This is because my progress indication relies on sessions and once I clear my sessions or close out my browser my session cookie changes. However I know the script is still running because I can query the database manually and see that entries are being added.
NOW to my question:
I do NOT want this. If my browser closes, if I press refresh, if I loose connection, etc I want the script to be TERMINATED. I want it to stop mid process.
I tried ignore_user_abort(false); however I'm pretty sure this is specific to command line and made no difference for me.
I want it to be terminated because I'm building a 'progress resume' function where we can choose where to resume the migration progress again.
Any suggestions?
UPDATE:
I didn't want to go this route but some solution I just though of is I could have another session variable. And it's my 'last time client was validated' which could be a timestamp. In my javascript, on the client side, every like 30 seconds I could hit a php script to 'update last time client was validated'. And in my migration function at the beginning of each loop I could check to make sure that timestamp isn't like 60 seconds old for example. If it IS 60 seconds old, or older, I do a die thus stopping my script. This would locally mean 'if there is no client updating this timestamp then we can assume the user closed out of his browser/tab/refreshed'. And as for the function I can ignore this check if in command line (cron). Not the ideal solution but it is my plan B
I am, and did, go with the solution to ping from the client to indicate if the client is still alive or not.
So essentially this is what I did:
From the client, in javascript, I set up a setInterval to run every 1.5 seconds and that hits a php script via AJAX. This php script updates a session variable with the current timestamp (this could easily be a database value if you needed to, however I didn't want the overhead of another query).
$_SESSION['migration_listsync_clientLastPing_'.$decodedJson['progressKey']] = time();
Then, inside my migration function I run a check to see if the 'timestamp' is over 10 seconds old, and if it is I die - thus killing the script.
if(isset($_SESSION['migration_listsync_clientLastPing_'.$progressKey])){
$calc = time() - $_SESSION['migration_listsync_clientLastPing_'.$progressKey];
if($calc > 10){
die();
}
}
I added a 'progressKey' param which is a random number from 1-100 that is generated when the function is called. This number is generated in javascript and passed into both of my AJAX calls. This way if the user refreshes the page and then immediately pressed the button again we won't have 2 instances of the function running. The 'old' instance will die after a few seconds and the new instance will take over.
This isn't an ideal solution however it is an effective one.
I need some advice on the best way to do this. I am developing a web application that monitors a data stream and adds that to a database. It then checks it against a ruleset for specific criteria. If one of these criteria is met, it needs to run a specific query on that same entry x minutes later. (x is set by user config).
I was thinking a way to do it would be to have it create cron jobs but first, I don't know how to do that, secondly, I was wondering if there is a better way.
Entry received -> Matches criteria -> Wait X minutes -> Execute another query.
I need it to do this without pausing the script because lots of data would be coming in so if it waited for 10 minutes it wouldn't ever process all the data.
Thanks!
If there are many similar tasks like this, you can make a table for tasks and populate it when needed.
Then, cron script that runs every minute can select all entries from this table with execution time less or equal than current time and run them.
Don't create multiple cron jobs.
Store the users configuration as a timestamp (in the future), and have one cron job that runs frequently (every 1 minute is the maximum) and will query the list of user configurations to see if another query needs to be executed.
Store the last execution time of the cron job and have it query the list -
SELECT * FROM `pending_queries` WHERE `execution_ts` <= {$last_execution_ts}
A word of warning
Should some of the queries take a longer time to execute, you run the risk of potentially overlapping the execution time of one or more queries. To solve this, as soon as you extract data on a pending query, you should mark it with an is_processing flag as some other field in the row. So your query now becomes-
SELECT * FROM `pending_queries` WHERE
`execution_ts` <= {$last_Execution_ts} AND `is_processing`=0
Finally, after processing a query, you should delete it from the pending_queries table.
First and easiest way:
Learn cronjob.
Second way:
If you want to wait for 10 minutes without stopping the script execution, the user will have to wait 10 mins.
If you want to do this operation without bothering the user, you can create a child process that will handle this operation. The process won't stop but interaction with the user will be end. In the child process you can pause the script for 10 mins.
How to create a child process: http://php.net/manual/en/function.pcntl-fork.php
How to pause the process: http://php.net/manual/en/function.sleep.php
I solved similar problem for myself using JS/JQuery and PHP. Here is kind of pseudo code for it:
pingTimer = 0;
function pingTimerIncrement() {
//your criterias to be checked here either using plain
//JS or AJAX to do it on server side
if (criterias met) {
pingTimer += 1; //increase by 1 minute
if (pingTimer == YOUR_USER_CONFIG_MINUTES) {
//code to update your database using ajax
}
}
}
$(function() {
var timerInterval = setInterval("pingTimerIncrement()", 6000); //1 minute
});
the other way if you dont want to use JS/JQuery then use cronjob
I'm making a PHP site, and I would like to have a script (when you click a button) which adds some info to my MySQL database (I can do this part by myself) and it executes a script 5 minutes later. Maybe it's not difficult, but it's hard to google stuff like this.
Sleep is a VERY bad idea. Client browser would have to wait 5 minutes to finish request!!!
In my opinion it's not possible to do it like you want to.
You should create another script which queries database and checks if there is new data (and on successful fetch does the job). This script should be run by cron every N minutes.
Pretty tough one.
I'd go for something like this:
your original script adds a record to the database, containing its time of execution,
another script contains the action that needs to be taken 5 minutes later - but launches it only if the db record mentioned above contains a timestamp of at least 5 minues ago (hope that's clear enough, I'm having trouble phrasing this)
set crontab to execute the second script every X minutes (perhaps 2).
It won't be 5 minutes EXACTLY, but rather something between 5 and 7 (in case you choose to launch the script every 2 minutes). Would that do?
You could implement a queue in your database, where you add "commands" to be executed, and also store when to execute this command. Then have a cron job that runs every minute and checks said queue to see if it's time to execute a certain command.
If you're on a unix box:
exec("echo 'php script.php' | at now +5 minutes");
Which will schedule the php script.php command to run after 5 minutes.
I'm making a browser-based game and I want it to if someone wants to build a building it takes * minutes and then finishes.
Considering this is your actual goal, I recommend just saving the original building with a timestamp.
I know you tagged your question with PHP, but I don't want to include all the overhead of handling mysql queries in PHP, especially since I don't know how you prefer to execute the queries or what framework you're suing, so here's some pseudocode to handle this "building buildings" task:
build.php
building_type_id = sanitize(POST['id'])
user_id = current_user['id']
query('INSERT INTO buildings (user_id, building_type_id, created_at)
VALUES (' + user_id + ', ' + building_type_id + ', CURRENT_TIME)');
my_buildings.php
user_id = current_user['id']
completed_buildings = query('SELECT * FROM buildings b
LEFT OUTER JOIN building_types t ON b.building_type_id = t.id
WHERE DATE_ADD(b.created_at, INTERVAL t.construction_time SECOND) < NOW();')
under_construction = query('SELECT * FROM buildings b
LEFT OUTER JOIN building_types t ON b.building_type_id = t.id
WHERE DATE_ADD(b.created_at, INTERVAL t.construction_time SECOND) > NOW();')
Hope this helps!
IMHO the best way is: On button click save the job to run in the db with the time it should run. Write a small daemon, fetches every 10/5/2 seconds new jobs which should be executed and executes them.
EDIT: Btw the idea using cron for checking for new jobs to execute, is better, but only if you have a small website and you don't need to do load balancing for the jobs.
The way I would do this is to run a cron job between the two scripts.
the first script sets a value in a database table.
the cron job executes the second script. every minute or what not.
the second script checks for the database value set by script 1 to decide whether to run entirely or not.
I would suggest doing the timer in Javascript rather than PHP.
Put a timestamp in the user's $_SESSION to indicate when they started the event, and then have Javascript call back to the browser after five minutes.
PHP would still need to know the start time (to prevent the user from hacking the game by tweaking the Javascript time-out), but it wouldn't need to actually do any count-down timing or sleeping or anything like that itself.
You could fork the process and in the child fork, do a sleep for 5 minutes before executing your second script. I've tested this and it appears the child process will still execute even after the parent has finished. Something like
//initial code
$pid = pcntl_fork(); //fork the process
if ($pid==0) // if in the child
{
exec("sleep 300; php second_process.php"); //sleep for 5 minutes and execute second script
return; // or exit
}
// rest of initial script...
The "return;" is important as the rest of the script will execute a 2nd time (i.e. in the child) unless it's there.
Someone asked about the purpose of this and your answer was:
"I'm making a browser-based game and I want it to if someone wants to build a building it takes * minutes and then finishes"
You don't actually need to time an execution for this. You can do it all in one run by storing buildStartedAt and buildFinishedAt as part of the building-schema.
Now maybe you want the building to have a nice animation when it finishes, then you just do all of that on the frontend but make sure nothing meaningful can be done with the building if the timestamp is before the buildFinishedAt time in order to a void cheating by potential hackers.
Are you looking for that?
sleep php.net
Objects:
Object Sale
n Object User will participate on Object Sale
Each User can create one Object Agent
Each Agent will generate n Object Licitation (from time to time and with some conditions) for the object Sale
Goal:
Create a function that when a Sale has only 5 seconds left to end, will check all the Agents that there are for this Sale and post a licitation from the Agent that hasn't made a licitation on this sale the longest
This is something that needs to be continuously running since this agents would be in charge to replace users on posting licitations allowing them to be away during auctions.
My question
Updated (My question wasn't clear enough):
At index page of the sales I have a script with javascript that calculates the time left on a sale, so I can know when a sale should call the function to check for agents and place their bids, but my newbie question is: if I make a call for the function at this page, will this only work if the user has the page open? Or if he closes the function won't be called anymore? Because I need this to still work even the user closes webpage.
DaMacc already answered that this doesn't work
Updated (05/01/2010)
I've already created this function. The function will find all the sales that have 5 seconds left to end, then it will search all the agents and then place a bid from the agent that hasn't made a licitation on the selected sale the longest. Now i need this function to be called every second. I was going to use cron but cron has 1-minute boundaries. And I need this function to run on the server and not depend on the user that owns the agent.
Any suggestions?
PS: there are some auctions websites that have this kind of bidagents i'm trying to do, I could reffer one to use as example... But i'm not sure i'm allowed to do that here... :S
Thanks.
Create your normal function to do whatever it is you need to do and then use something like cron to set it up as a task to run every X amount of time.
Edit to expand on comments
In that case you are probably better off combining a few solutions. As mentioned in this question, I would recommend that you use cron to call a script every minute and in that script you run your process in a loop.
Some things to consider:
How long will each execution take?
Will you need to time_sleep_until at the end of each loop or will your script take longer than 1 second to run, in which case you will need to be calling the script multiple times from cron.
Be sure to keep track of how long your script has been running for as you don't want to have the situation where every minute you are taking up more and more resources as the script called the previous minute hasn't finished yet.
At the start of the script, take note of the current time, then at the start of each loop, check whether a minute has passed since the start of the script, if it has, exit the script (as another script will have taken over now thanks to cron).
Hope this helps. Let me know if this doesn't make a lot of sense and I'll try to clear it up a bit.
I wonder if this is what you are looking for:
function execInBackground($cmd) {
if (substr(php_uname(), 0, 7) == "Windows") {
pclose(popen("start /B ". $cmd, "r"));
} else {
exec($cmd . " > /dev/null &");
}
}
You can then start a php process in the background using the following method:
execInBackground("php path/to/file.php " . $param01 . " " . $param02);
// where $param01 and $param02 are optional values that you may want to
// send to the page. Equivalent to GET parameters of URL.
// You can include as many parameter values as you want.
//Example:
execInBackground("php automation/bidChecker.php daniel 53.25");
// automation/bidChecker.php is the file to be executed
// daniel can be the username
// 53.25 can be the bid value
And in the PHP file that runs in the background, you can access the parameter values using the following method:
$param01 = $argv[1]; // assigns daniel as value
$param02 = $argv[2]; // assigns 53.25 as value
This process can be started from within a script run when the user does something. It will also keep on running even if the user leaves the page and until you programatically break the operation.
I really don't know if this is what you are looking for. If so, you got it now.
Use a Queue, like Zend_Queue: http://framework.zend.com/manual/en/zend.queue.html . Just run an infinite loop (or two) and dispatch messages sent from your web application
Have you looked into using Javascript at all for this? If you have certain events that trigger the need to run this check, it may be the way to go.
You could write a js function included on every page that uses the the setInterval/clearInterval javascript functions to send an AJAX request to your server every few seconds and could send a different response back to the browser based on whether the conditions were met or not. (bandwidth may be an issue with this though)
I would recommend looking into jQuery and using it's AJAX functions for this.