I'm willing to set up a long polling Ajax call to check for orders in my e-commerce web app. There is a specificity in this application in the way that customers are able to place order in the future. As such, in the admin panel, we have past orders and futures orders (that can be 2 months or 20 minutes in the future).
Basically, I want the admin user in the back-end to be warned as soon as a future order comes to an end (the future date reaches the current time). To proceed, I make the user admin doing an Ajax call (as soon as they are connected to the admin) to the server to check for futures orders to arrive. This Ajax call is a long polling request as the call waits for the server to deliver result. If server has nothing to offer, the request keeps pending until there is an order to show.
Ajax request
(function poll() {
setTimeout(function() {
$.ajax({
url: '{{ path('commande_check') }}',
method: 'post',
success: function(r) {
if(r.ids) alert('New order!'); // I've simplified this part of the code to make it clean, admin are actually warned through Node.JS server
},
error: function() {},
complete: poll
});
}, 5000);
})();
{{ path('commande_check') }} (edited from Edit2)
public function checkAction(Request $request)
{
if($request->isXmlHttpRequest())
{
$response = new Response();
$em = $this->getDoctrine()->getManager();
$ids = array();
while(!$ids)
{
$ids = $em->getRepository('PaymentBundle:Commande')->findNewestOrders(new \DateTime());
if($ids)
break;
else
time_sleep_until(time() + self::SECONDS_TO_SLEEP);
}
if($ids)
{
return new JsonResponse(array(
'ids' => $ids
));
}
$response->setStatusCode(404);
return $response;
}
$response = new Response();
$response->setStatusCode(405);
return $response;
}
findNewestOrder() method
public function findNewestOrders(\DateTime $datetime)
{
$query = $this->createQueryBuilder('c')
->select('c.id')
->leftJoin('Kt\PaymentBundle\Entity\Paiement', 'p', \Doctrine\ORM\Query\Expr\Join::WITH, 'p.id = c.paiement')
->andWhere('p.etat = 0')
->where("DATE_FORMAT(c.date, '%Y-%m-%d %H:%i') = :date")
->setParameter('date', $datetime->format('Y-m-d H:i'))
->andWhere('c.kbis IS NULL')
->andWhere('c.notif = 0')
->getQuery();
return $query->getArrayResult();
}
My problem is the alert sometimes never get shown whereas the record in the DB gets updated. The weirdest things is it sometimes happens even when I've leaved the page making the Ajax call like if it keeps running in the background. I think the problem comes from the time_sleep_until() function. I tried with sleep(self::SECOND_TO_SLEEP) but the problem was the same.
Any help would by gladly appreciated. Thanks!
Edit 1
I sense there is something to do with connection_status() function as the while loop appears to continue even if the user has switched page causing the field notif to be updated in the background.
Edit 2
As per my answer, I've managed to overcome this situation but the problem still remains. The admin does get the notification properly. However, I do know the Ajax call still keeps going on as the request has been made.
My problem is now: could this result in a server resources overload?
I'm willing to start a bounty on this one as I'm eager to know the best solution to achieve what I want.
I think I got it all wrong.
The intent of long-polling Ajax is not that there is only one connection that stays opened such as websockets (as I thought it did). One would have to make several requests but much less than regular polling.
Regular polling
The intent for Ajax regular polling is one makes a request to the server every 2 or 3 seconds to have a semblance of real-time notification. These would result in many Ajax calls during one minute.
Long polling
As the server is waiting for new data to be passed on to the browser, one would need to make only a minimal number of requests per minute. As I'm checking in the database for new order every minute, using long polling can make me lower the number of requests per minute to 1.
In my case
In consequence, the specificity of the application makes the use of Ajax long-polling unnecessary. As soon as a MySQL query has been made for a specific minute, there is no need for the query to run again in the same minute. That means I can do regular polling with an interval of 60000 ms. There's also no need to use sleep() nor time_sleep_until().
Here's how I ended up doing it:
JS polling function
(function poll() {
$.ajax({
url: '{{ path('commande_check') }}',
method: 'post',
success: function(r) {
if(r.ids)
alert('New orders');
},
error: function() {},
complete: function() {
setTimeout(poll, 60000);
}
});
})();
{{ path('commande_check') }}
public function checkAction(Request $request)
{
if($request->isXmlHttpRequest())
{
$em = $this->getDoctrine()->getManager();
$ids = $em->getRepository('PaymentBundle:Commande')->findNewestOrders(new \DateTime());
if($ids)
{
return new JsonResponse(array(
'ids' => $ids
));
}
$response = new Response();
$response->setStatusCode(404);
return $response;
}
$response = new Response();
$response->setStatusCode(405);
return $response;
}
As such, I end up with one request per minute that will check for new orders.
Thanks #SteveChilds, #CayceK and #KevinB for their kind advice.
In general for this problem it is kinda rough to say. We don't have a lot of information as to exactly what your other functions do. Like findNewestOrders...
We can assume that it pulls all new orders that have yet to be fulfilled by the admin and therefore will be displayed. However, if it is looking only for orders that are exactly equal they will never be filled.
Theoretically this will run forever if no new order ever is filed. You have no time limit on this so it is possible that the server feels like you have a case in which while will never be false and executes an exceeded execution time.
As per your comment
time_sleep_until
Returns TRUE on success or FALSE on failure.
The only way it would ever fail is if the function itself failed or some server side issue caused a failure return. As you never officially visit the page and no act of leaving your ajax'd page submits a failure response it should never really fail.
I think it might be more wise to look into doing a CRON job for this and have a database of incomplete orders that you query instead. The CRON can run every minute and populate the database. The run on the server would not be that great as it would most likely take no more than 30 seconds any way.
Long-polling may be a great idea for many functions, but I'm not wholly confident it is in the case. I would seriously recommend setInterval as the load on the server and client would not be that great in a 30 seconds call every minute or two. That is your call in the end.
I personally would check frequently rather than have one request which runs for a long time - its not really ideal to have long running processes like this as they tie up server connections and really, its just bad practice. Plus the browser may well time the connection out, which is why you may not be seeing the responses you expect.
Have you tried changing the ajax call so it calls in say, every 60 seconds (or however often you want), checks for new orders since the last time it was polled (simply keep a track of this in the page / HTML5 local storage so it persists across pages and pass it in the ajax request as a parameter) and then simply returns an indication of yes there have been new orders, or no there hasn't?
You can then display a message if there have been new orders.
I have finally managed to overcome this bug but without digging deeply in the problem.
I have separated the code that updates the notif field from the code that fetch new orders. In that way, the while loop still goes on but cannot update the field.
The field is therefore updated on success of the first ajax call by making a new ajax request to update the field. Therefore, the admin always receives the notification.
I just have to enquiry on a memory/thread level to see what consumption of resources this loop uses.
As no solution has been found despites my workaround for the initial bug, I won't accept my answer as the problem remains still.
Many thanks for all the help on that question.
Related
What is the best way to keep checking a condition until it is true? I know there is a while statement in PHP but was not sure if there is a better packaged way in Laravel.
Basically I am transcoding a video through AWS. I want the frontend to keep saying "Uploading" until I know the video is transcoded and saved in AWS and all info is in the database. The videos will be short, but still transcoding is not instant so if I do:
if ($job['Status'] == 'complete') {
$submission = new Submission();
$submission->email = $request->input('email');
$submission->original = config('filesystems.disks.s3.url') . $original_key;
$submission->save();
return response()->json([
'submission' => $submission,
'message' => 'Upload Successful. Good luck!!!!',
'job' => $job
]);
}
This if statement will be false right away. But if I checked again every few seconds it will pass after a little bit. Is there a pulse type function to continue to run that if statement every X amount of time until it passes? Using a while seems to be hitting max_execution_time the limit.
Does this have to be on the PHP side?
If JS works, I suppose one way would be to have a bool state in your JS to show "Uploading" with setTimeout() to check if the video is uploaded every x seconds. When it passes you can set the state to false, which would make "Uploading" disappear.
Also, don't forget to stop the setTimeout function when it passes.
In my opinion, there could be 02 solutions:
1) If I were you, I'd prefer to handle it in Frontend side with AJAX. This is a pseudo code to explain my idea
function uploadMyVideo() { // to be called once we confirm the upload
var jqxhr = $.ajax( "/upload/url" )
.done(function() {
// call another ajax to set the status = SUCCESS
})
.fail(function() {
// call another ajax to set the status = FAIL
})
.always(function() {
// in case you need it
});
}
2) You can use Laravel scheduler to call periodically (eg: every minute) a job which checks for completed status, and run the logic inside it. (it's a cron job with laravel way)
I have a web app created in Laravel that takes credit card payments.
Every day a scheduled command that I created runs to take "today's" payments (basically it submits one http request for each pending payment to the payment gateway).
Now I need to allow to trigger this payment submission process via a button in a dashboard.
The command takes a random long time to process (depending on the number of payments to process), so call it from the controller I think is not an option.
I'm thinking of just refactor it: move all the command code to a "middleman" class so I could call this class on both the command and the controller.
PaymentsSubmissionHelper::submit()
PaymentsSubmissionCommand: PaymentsSubmissionHelper::submit()
PaymentsSubmissionController: PaymentsSubmissionHelper::submit()
However, the command shows a progress bar and the estimated time to process and I will need to show a progress bar in the html interface as well. In the web interface I will need to make ajax requests to the server to get the current progress but in the command this progress is tracked in a completely different way using:
$bar = $this->output->createProgressBar($totalPayments);
$bar->setFormat(' %current%/%max% [%bar%] %percent:3s%% %elapsed:6s%/%estimated:-6s% %message%');
and for each processed payment:
$bar->advance();
How can I create keep track of the progress on both the command and the controller?
Any help will be appreciated.
Thanks in advance!
As already pointed out in another answer, Laravel's queued event listeners are the way to handle long-running processes on the front end. You shouldn't need to refactor your console command at all.
As to showing progress on the front end, one simple solution would be to set up some AJAX polling. Ever few seconds have AJAX fire off a request to a controller method which simply looks at today's payments, calculates how many are processed (presumably you have some kind of status field which will show you whether or not the running job has handled it yet), and return a number representing the percentage that are done. The AJAX success handler would then update your progress tracker on the page.
// Check status every 2s
var timer = setInterval(function() {
pollStatus();
}, 2000);
pollStatus = function() {
$.ajax({
url: 'somewhere/jobStatus',
success: function(resp) {
$('#progress').html(resp . '%');
if (resp === 100) {
// We've reached 100%, no need to keep polling now
clearInterval(timer);
}
}
});
}
It might be wise to somehow make sure polls don't overrun, and maybe you'd want to tweak the frequency of polling.
I would suggest using queued event listeners in this use case. You would dispatch an event in your controller and have a listener which could trigger the command. By queueing the listener you avoid a long response time. No need to refactor the command itself!
Regarding a progress bar, you could have a static progess bar that updates on page load where you would read out the status from your DB and display it similarly to how Amazon displays how far along your order is at any moment.
For a real time updated progress bar, I suggest implementing web sockets. Socket.io seems great.
As you are using progress bar and advancing it, you will do same in ajax but the progress logic will be different off-course.
The common part in both the cases is handling each card payment. So I will say create separate class or service which takes card payment instance e.g. PaymentProcess, processes it and returns if successful or failed.
Then in command you can do (psuedocode) :
public function handle()
{
$pendingPayments = Payment::where('status', 'pending');
$bar = $this->output->createProgressBar($pendingPayments->count());
$pendingPayments->chunk(10, function($payments) use($bar){
$payments->each(function($payment) use ($bar){
$process = (new PaymentProcess($payment))->process();
$bar->advance();
});
});
$bar->finish();
}
Now if you trigger this from frontend, the ajax response should give you an id of current process stored somewhere. Then you will keep sending another ajx requests in an interval of lets say 1 second and get the current progress until it reaches to 100%. (If you are using XMLHttpRequest2 then the logic will differ)
For that you can create another table to store progresses and then keep updating it.
Now similarly you can use the PaymentProcess inside controller. :
public function processPendingPayments(Request $request)
{
// Authorize request
$this->authorize('processPendingPayments', Payment::class);
$pendingPayments = Payment::where('status', 'pending');
// Create a progress entry
$progress = PaymentProgress::create([
'reference' => str_random('6')
'total' => $pendingPayments->count(),
'completed' => 0
]);
$pendingPayments->chunk(10, function($payments) use($bar){
$payments->each(function($payment) use ($bar){
$process = (new PaymentProcess($payment))->process();
// Update a progress entry
$progress->update([
'completed' => $progress->completed + 1;
]);
});
});
return response()->json([
'progress_reference' => $progress->reference
], 200);
}
Now another endpoint to get the progress
public function getProgress(Request $request)
{
// Authorize request
$this->authorize('getProgress', Payment::class);
$request->validate([
'reference' => 'required|exists:payment_process,reference'
]);
$progress = PaymentProcess::where('reference', $request->reference)->first();
$percentage = $progress->completed / $progress->total * 100;
return response()->json(compact('percentage'), 200);
}
Thank you for reading.
I have an input field that sends its contents in an XMLHttpRequest to a php script. The script queries the database with the POST data from the field and returns the results.
Because the XMLHttpRequest is invoked using onkeyup, typing in a lengthy value sends several calls in a short period. To combat this I wrote some code that creates a timestamp, loads it into the session, sleeps, then rechecks the timestamp. if the timestamp has increased, it means a subsequent call was made and the script should abort. Otherwise the script executes. Here is the code.
$micro = microtime(true);
$_SESSION['micro'] = $micro;
usleep(500000); // half a second
if ($micro < floatval($_SESSION['micro']))
{
// later call has been made, abort
echo 'abort';
exit;
}
else
{
// okay to execute
}
The code appears to work as expected at first. If I add or remove a character or two from the input field the result appears quickly.
However if I type a good 12 characters as fast as I can there is a large delay, sometimes 2 or 3 seconds long.
I am working on localhost, so there is no connection issues. The query is also really small, grabbing one column containing a single word from a specific row.
I have also set XMLHttpRequest to be asynchronous, so that should also be fine.
xmlhttp.open("POST","/test/",true);
If I remove the flood prevention code, typing in the field returns results instantly - no matter how much and how quickly I type.
It's almost as if usleep() keeps stacking itself or something.
I came up with this code on my own, best I could do at my level. No idea why it isn't behaving as expected.
Help is greatly appreciated, thanks!
When you open a session using session_start(), PHP locks the session file so any subsequent requests for the same session while another request has it open will be blocked until the session closes (you were exactly right with the "stacking" you suspected was happening).
You can call session_write_close() to close the session and release the lock but this probably won't help in this situation.
What's happening is each time the key is pressed, a request gets issued and each one is backed up while the previous one finishes, once the session is released one of the other requests opens the session and sleeps, and this keeps happening until they've all finished.
Instead, I'd create a global variable in Javascript that indicates whether or not a request is in progress. If one is, then don't send another request.
Something like this:
<script>
var requesting = false;
$('#input').on('keyup', function() {
if (requesting) return ;
requesting = true;
$.ajax({
url: "/url"
}).done(function() {
requesting = false;
});
}
</script>
drew010's answer explained my problem perfectly (Thanks!). But their code example, from what I gather by how it was explained (I didn't try it), does the opposite of what I need. If the user types "hello" the h will get sent but the ello might not unless the result makes it back in time. (Sorry if this was a wrong assumption)
This was the solution I came up with myself.
<input type="text" onkeyup="textget(this.value)" />
<script>
var patience;
function ajax(query)
{
// XMLHttpRequest etc
}
function textget(input)
{
clearTimeout(patience);
patience = setTimeout(function(){ajax(input)},500);
}
</script>
when a key is pressed in the input field, it passes its current value to the textget function.
the textget function clears an existing timer if any and starts a new one.
when the timer finishes counting down, it passes the value further to the ajax function to perform the XMLHttpRequest.
because the timer is reset every time the textget function is called, if a new call is made before the timer finishes (0.5 seconds), the previous call will be lost and is replaced by the new one.
Hope this helps someone.
I have php script which can take quite a lot of time (up to 3-5 minutes), so I would like to notify user how is it going.
I read this question and decided to use session for keeping information about work progress.
So, I have the following instructions in php:
public function longScript()
{
$generatingProgressSession = new Zend_Session_Namespace('generating_progress');
$generatingProgressSession->unsetAll();
....
$generatingProgressSession->total = $productsNumber;
...
$processedProducts = 0;
foreach($models as $model){
//Do some processing
$processedProducts++;
$generatingProgressSession->processed = $processedProducts;
}
}
And I have simple script for taking data from session (number of total and processed items) which return them in json format.
So, here is js code for calling long script:
$.ajax({
url: 'pathToLongScript',
data: {fileId: fileId, format: 'json'},
dataType: 'json',
success: function(data){
if(data.success){
if(typeof successCallback == "function")
successCallback(data);
}
}
});
//Start checking progress functionality
var checkingGenerationProgress = setInterval(function(){
$.ajax({
url: 'pathToCheckingStatusFunction',
data: {format: 'json'},
success: function(data){
console.log("Processed "+data.processed+" items of "+data.total);
if(data.processed == data.total){
clearInterval(checkingGenerationProgress);
}
}
});
}, 10000)
So, long scripted is called via ajax. Then after 10 seconds checking script is called one time, after 20 second - second time etc.
The problem is that none of requests to checking script is completed until main long script is complete. So, what does it mean? That long script consumes too many resources and server can not process any other request? Or I have some wrong ajax parameters?
See image:
-----------UPD
Here is a php function for checking status:
public function checkGenerationProgressAction()
{
$generatingProgressSession = new Zend_Session_Namespace('generating_progress');
$this->view->total = $generatingProgressSession->total;
$this->view->processed = $generatingProgressSession->processed;
}
I'm using ZF1 ActionContext helper here, so result of this function is json object {'total':'somevalue','processed':'another value'}
I'd
exec ('nohup php ...');
the file and send it to background. You can set points the long running script is inserting a single value in DB to show it's progress. Now you can go and check every ten or whatever seconds if a new value has been added and inform the user. Even might be possible to inform the user when he is on another page within your project, depending on your environment.
Yes, it's possible that the long scripts hogs the entire server and any other requests made in that time are waiting to get their turn. Also i would recommend you to not run the check script every 10 seconds no matter if the previous check has finished or not but instead let the check script trigger itself after it has been completed.
Taking for example your image with the requests pending, instead of having 3 checking request running at the same time you can chain them so that at any one time only one checking request is run.
You can do this by replacing your setInterval() function with a setTimeout() function and re-initialize the setTimeout() after the AJAX check request is completed
Most likely, the following calls are not completing due to session locking. When one thread has a session file open, no other PHP threads can open that same file, as it is read/write locked until the previous thread lets go of it.
Either that, or your Server OR Browser is limiting concurrent requests, and therefore waiting for this one to complete.
My solution would be to either fork or break the long-running script off somehow. Perhaps a call to exec to another script with the requisite parameters, or any way you think would work. Break the long-running script into a separate thread and return from the current one, notifying the user that the execution has begun.
The second part would be to log the progress of the script somewhere. A database, Memcache, or a file would work. Simply set a value in a pre-determined location that the follow-up calls can check on.
Not that "pre-determined" should not be the same for everyone. It should be a location that only the user's session and the worker know.
Can you paste the PHP of "pathToCheckingStatusFunction" here?
Also, I notice that the "pathToCheckingStatusFunction" ajax function doesn't have a dataType: "json". This could be causing a problem. Are you using the $_POST['format'] anywhere?
I also recommend chaining the checks into after the first check has completed. If you need help with that, I can post a solution.
Edit, add possible solution:
I'm not sure that using Zend_namespace is the right approach. I would recommend using session_start() and session_name(). Call the variables out of $_SESSION.
Example File 1:
session_name('test');
session_start();
$_SESSION['percent'] = 0;
...stuff...
$_SESSION['percent'] = 90;
Example File 2(get percent):
session_name('test');
session_start();
echo $_SESSION['percent'];
I am trying to make a chat room on my website, I am using php and mysql to store the messages and all the info. How could I automatically refresh the page every time someone updates the database? example:
If I am on my site, the messages show up on my screen but I can only see more recent messages after I refresh the page. Is there a way to make it real-time?
Also I do not know much javascript/ajax/jquery or any of that. Any help is appreciated!
There will be low amount of traffic on my site. Probably around 10-15 people at a time, if that even.
Your best bet is to make an AJAX request every sec or so and see if there are new messages.
You probably do not want to be reloading the page every time. My recommendation, and there are many ways to do this, is to make a ajax call every so often and check/pull the new information from the database.
I would research AJAX and do a tutorial.
This would be accomplished through ajax by calling a function and updating the div. I would not suggest making people refresh a page everytime they send a message it would get ugly. Another option would be using HTML5 web workers
http://msdn.microsoft.com/en-us/hh549259.aspx
You are going to need to learn AJAX in order to make this work well, and jQuery is probably the easiest way to do it. If we can assume that the DIV you want to update has the ID PonyRides, you would want to do:
$("#PonyRides").ajax({url: "/chat.php?getupdates=true"});
This will get the contents of chat.php and stick it into the #PonyRides DIV. This assumes that chat.php will get the contents of the database and format them into HTML.
The remaining challenge is to make it update whenever your database does, but the simplest way is just to reload the whole chat regardless of whether an update has been made or not.
That will impact performance, but if you have less than a hundred chatters you'll probably be fine. If you have more than that, you'd do well to sense inactivity and decrease the checking period, or only send updates instead of the whole chat. Those are more complicated topics, though, and you can build them in as needed once you get these basic concepts down.
To do this, simply wrap the ajax() call in an interval like so:
setInterval(function(){ //the following code runs repeatedly
$("#PonyRides").ajax({url: "/chat.php?getupdates=true"}); //update our chat div
},5000); //repeat every five seconds
The other, awful method would be to load chat in an iFrame, set to reload periodically using the meta refresh technique. This would be dreadful, and can only be recommended if you are trying for some reason to support incredibly old browsers.
You can use AJAX request to update the values
<script type='text/javascript'>
// function for making an object for making AJAX request
function getXMLHTTPRequest() {
try {
req = new XMLHttpRequest();
} catch(err1) {
try {
req = new ActiveXObject("Msxml2.XMLHTTP");
} catch (err2) {
try {
req = new ActiveXObject("Microsoft.XMLHTTP");
} catch (err3) {
req = false;
}
}
}
return req;
}
var http899 = getXMLHTTPRequest();
function searchFabIndia() {
var myurl = "http://my2nddomain.com/yebhi.php";
myRand = parseInt(Math.random()*999999999999999);
var modurl = myurl+"?rand="+myRand;
http899.open("GET", modurl, true);
http899.onreadystatechange = useHttpResponse899;
http899.send(null);
}
function useHttpResponse899() {
if (http899.readyState == 4) {
if(http899.status == 200) {
// do all processings with the obtained values / response here
// after doing the stuff, call fn again after 30 s say
setTimeout("searchFabIndia()", 30000);
}
}
}
</script>
<body onload='searchFabIndia();'>
I would suggest making an AJAX request to a file on your server which will update the database. If the update to the database is successful then return the message which was updated. Back on the client side you wait for the response and if you get one then append the message to the end of the content. This way you're loading all the messages every time (which would be expensive), you're only loading new messages.
There must be something similar to SignalR(.net) for php. It lets you add code when an event occurs, I think that is what you are looking for.