I have constructed a PHP file which scrapes a web page (using cURL) to obtain some data, and outputs it to the screen in JSON format.
The target website involves some redirects which temporarily outputs data to my PHP file. Once the redirects have completed successfully, the JSON is presented as expected. The problem that I am encountering is that when I try to access the JSON using jQuery's $.ajax() method, it sometimes returns the incorrect data, because it isn't waiting for the redirects to complete.
My question is if it's possible to tell the AJAX request to wait a certain number of seconds before returning the data, thus allowing time for the redirects in the PHP script to execute successfully?
Please note that there is no cleaner solution for the page scrape, the redirects are essential and have to be outputted to the screen for the scraping to complete.
There's always timeout in the settings.
jQuery docs:
timeout Number
Set a timeout (in milliseconds) for the request. This will
override any global timeout set with $.ajaxSetup().
The timeout period starts at the point the $.ajax call is made;
if several other requests are in progress and the browser
has no connections available, it is possible for a request
to time out before it can be sent. In jQuery 1.4.x and below,
the XMLHttpRequest object will be in an invalid state if
the request times out; accessing any object members may
throw an exception. In Firefox 3.0+ only, script and JSONP
requests cannot be cancelled by a timeout; the script will
run even if it arrives after the timeout period.
You should use promise() in jQuery.
You could always store the result of your ajax call and then wait for the redirects to finsih, i.e.:
$.ajax({
success: function(e)
{
var wait = setTimeout(function(){ doSomethingWithData(e.data); }, 5000); //5 sec
}
})
Alternatively, you could set up an Interval to check if something happened (redirect finished) every x amount of ms. I'm assuming your redirects are letting you know they completed?
http://examples.hmp.is.it/ajaxProgressUpdater/
$i=0;
while (true)
{
if (self::$driver->executeScript("return $.active == 0")) {
break;
}
if($i == 20) {
break;
}
$i++;`enter code here`
echo $i;
usleep(10000);
}
Related
The apache server I am using to develop my system will not respond to request while the scripts that control the polling of messages is being run. This only happends on a domain level meaning that I can send an http request to any other apps hosted localy and get a response. When I do eventually get a response from this its about a minute later.
Here is the Js
window.fetch_messages = function ()
{
var last_message = $("div.message:last").attr('data-ai_id');
var last_message_status = $("p.message_status:last").text();
var project_id = getParameterByName('project-id');
$.ajax({
url:'/project_messages',
type:'POST',
data:{ project_id:project_id, latest_message:last_message, status:last_message_status },
timeout:50000,
async: true,
success:new_messages, // This upon completion also resends the request
error:function(data){ console.log(data); setTimeout(fetch_messages(),50000); }
});
}; // When On the page that uses this I call this function to start polling
Here is the server side code
do
{
// Check for status change
$status_change = $this->mentor_model->query_status($this->project_id, $this->last_message_id, $this->last_message_status, $_SESSION['user']);
// Check for new messages
$messages = $this->mentor_model->query_messages($this->project_id, $this->last_message_id);
// If there is a status update or new message.
if($messages || $status_change)
break;
usleep(1000000);
}
while(empty($messages) && empty($status_change));
echo json_encode(array("messages"=>$messages, "status"=>$status_change));
exit;
While this action is being run The server takes a long time to handle any request weather it be a GET, POST or another AJax request. Iv also tried changing both code sets to no avail as long as its long polling, the server will take a long time to handle.
Do I have this wrong or is there some apache setting I'm suppose to change. Using xamp on windows 8.1 also tried wamp with no change
Thanks to steven for this. Ansewer taken straight from the source of php manual page
for session_write_close();
You can have interesting fun debugging anything with sleep() in it if
you have a session still active. For example, a page that makes an
ajax request, where the ajax request polls a server-side event (and
may not return immediately).
If the ajax function doesn't do session_write_close(), then your outer
page will appear to hang, and opening other pages in new tabs will
also stall.
I have php script which can take quite a lot of time (up to 3-5 minutes), so I would like to notify user how is it going.
I read this question and decided to use session for keeping information about work progress.
So, I have the following instructions in php:
public function longScript()
{
$generatingProgressSession = new Zend_Session_Namespace('generating_progress');
$generatingProgressSession->unsetAll();
....
$generatingProgressSession->total = $productsNumber;
...
$processedProducts = 0;
foreach($models as $model){
//Do some processing
$processedProducts++;
$generatingProgressSession->processed = $processedProducts;
}
}
And I have simple script for taking data from session (number of total and processed items) which return them in json format.
So, here is js code for calling long script:
$.ajax({
url: 'pathToLongScript',
data: {fileId: fileId, format: 'json'},
dataType: 'json',
success: function(data){
if(data.success){
if(typeof successCallback == "function")
successCallback(data);
}
}
});
//Start checking progress functionality
var checkingGenerationProgress = setInterval(function(){
$.ajax({
url: 'pathToCheckingStatusFunction',
data: {format: 'json'},
success: function(data){
console.log("Processed "+data.processed+" items of "+data.total);
if(data.processed == data.total){
clearInterval(checkingGenerationProgress);
}
}
});
}, 10000)
So, long scripted is called via ajax. Then after 10 seconds checking script is called one time, after 20 second - second time etc.
The problem is that none of requests to checking script is completed until main long script is complete. So, what does it mean? That long script consumes too many resources and server can not process any other request? Or I have some wrong ajax parameters?
See image:
-----------UPD
Here is a php function for checking status:
public function checkGenerationProgressAction()
{
$generatingProgressSession = new Zend_Session_Namespace('generating_progress');
$this->view->total = $generatingProgressSession->total;
$this->view->processed = $generatingProgressSession->processed;
}
I'm using ZF1 ActionContext helper here, so result of this function is json object {'total':'somevalue','processed':'another value'}
I'd
exec ('nohup php ...');
the file and send it to background. You can set points the long running script is inserting a single value in DB to show it's progress. Now you can go and check every ten or whatever seconds if a new value has been added and inform the user. Even might be possible to inform the user when he is on another page within your project, depending on your environment.
Yes, it's possible that the long scripts hogs the entire server and any other requests made in that time are waiting to get their turn. Also i would recommend you to not run the check script every 10 seconds no matter if the previous check has finished or not but instead let the check script trigger itself after it has been completed.
Taking for example your image with the requests pending, instead of having 3 checking request running at the same time you can chain them so that at any one time only one checking request is run.
You can do this by replacing your setInterval() function with a setTimeout() function and re-initialize the setTimeout() after the AJAX check request is completed
Most likely, the following calls are not completing due to session locking. When one thread has a session file open, no other PHP threads can open that same file, as it is read/write locked until the previous thread lets go of it.
Either that, or your Server OR Browser is limiting concurrent requests, and therefore waiting for this one to complete.
My solution would be to either fork or break the long-running script off somehow. Perhaps a call to exec to another script with the requisite parameters, or any way you think would work. Break the long-running script into a separate thread and return from the current one, notifying the user that the execution has begun.
The second part would be to log the progress of the script somewhere. A database, Memcache, or a file would work. Simply set a value in a pre-determined location that the follow-up calls can check on.
Not that "pre-determined" should not be the same for everyone. It should be a location that only the user's session and the worker know.
Can you paste the PHP of "pathToCheckingStatusFunction" here?
Also, I notice that the "pathToCheckingStatusFunction" ajax function doesn't have a dataType: "json". This could be causing a problem. Are you using the $_POST['format'] anywhere?
I also recommend chaining the checks into after the first check has completed. If you need help with that, I can post a solution.
Edit, add possible solution:
I'm not sure that using Zend_namespace is the right approach. I would recommend using session_start() and session_name(). Call the variables out of $_SESSION.
Example File 1:
session_name('test');
session_start();
$_SESSION['percent'] = 0;
...stuff...
$_SESSION['percent'] = 90;
Example File 2(get percent):
session_name('test');
session_start();
echo $_SESSION['percent'];
I have a bunch of AJAX requests, the first is a
$.post("go.php", { url: pastedURL, key: offskey, mo: 'inward', id: exID});
This request, in go.php includes a system command, which despite nohup and & at the end, doesn't stop whirling on Firebug - waiting for response.
Unfortunately, the next AJAX request
$.post("requestor.php", { request: 'getProgress', key: offskey}, function(data) { console.log(data.response); }, "json");
Doesn't run, it whirls round in firebug (i'm guessing until go.php has finished) - it overloads everything eventually (this is on a timer to check every few seconds).
So I guess the question is, is there an AJAX method which simply throws data and walks away, instead of waiting for response... or someway I can perform another request whilst the other is waiting.
Hope someone knows what I mean.
Check for async: false, as parameter for $.ajax() method in jQuery.
You need it to be set async: true.
Turns out the PHP request on the other end would need this in order to proceed with waiting:-
"> /dev/null 2>/dev/null &"
source
There are a limited number of connections used by the browser. The number varies by browser, but for older ones like IE7 it can be as little as 2 connections per server. Once you fill them with pending requests you have to wait for those to complete before you can make any additional requests to that server.
(this is on a timer to check every few seconds)
It sounds like you may be making additional requestor.php requests using setTimeout or setInterval before the previous ones have finished? If so, don't do that. Use a full $.ajax (perhaps with a timeout as well) and in the success/error callback only then make another request.
I have a File which process many other files and may take upto 30mins to process. I do an AJAX request to the file in the front end. The file outputs to another temporary file regarding the percentage of completion. And when it finishes it outputs a success message as its own XML output (not to the tmp file).
The problem i am having is, when the processing time is small .. say max upto 3mins, the AJAX request (made through jQuery) stays alive. But a time out occurs when the processing takes longer time (above 4mins). And the AJAX connection is cut. How do i prevent it and make it stay alive till the browser is closed?
You won't be able to do that. Unless it is a comet server, that can keep the connection alive at the server side and when there is any update to the data, it pushes out the contents.
In your case, the only way i can think of is doing this:
function ajax_call () {
$.ajax({
url : 'get_file_processing_output.html',
success : function (response) {
check your response, if file processing is not finished, then call ajax_call() again
if it is finished, then just do whatever you need.
},
timeout : function () {
time out then directly call ajax_call() again, maybe with a time interval would be better
}
})
}
I have a success call back above in ajax, because i feel you should response something from your server side to tell the client that the processing is not yet done.
I have finished a facebook app that uses php and ajax to get statisicts about friends and find best match. The app is using tabs and is working fine, however sometimes when I click on one tab to send ajax request and loading icon shows I do not get any response back and the loading icon keeps showing. Is there a way that I can count how many seconds an ajax call is taking so I redirect the user to an altertanive html page after x number of seconds?
To achieve what you'd like to obtain, you could call (after X seconds, using setTimeout()) a function that checks if the ajax response has been loaded.
For example:
var responseLoaded = false;
function checkIt(){
if(!responseLoaded)
window.location = "%http://alternative-page%";
}
setTimeout(checkIt,10000) //after 10 seconds
responseLoaded could be a global variable that can be set to true at the end of the ajax response.
Or, alternatively, you could check if some DOM element or JS var is present (i.e. created by AJAX call) at that time (when the function has been called through setTimeout)
Then, You could use
Date.getTime()
To obtain the current timestamp. You can compare the timestamp at the beginning with the one at the end of the ajax response to see the total time it takes. (just if you want to check the average ajax loading time)
Another option would be to set the timeout parameter of the of the xmlhttprequest object. In jQuery it's the timeout option in the options object.
$.ajax ({
timeout: 1000,
success: successCallback,
error: function (req, error, errorStatus) {
if (error == 'timeout') {
//send them to other page....
}
}
});
You would probably want to put in some more error handlers though... also, since the xmlhttpobject2 has on timeout function callback, this process might be more streamlined now...
Don't quote me exactly on the syntax either. I'm writing this on my phone from memory....