I suspect this question will seem too... silly, but I'm trying to get my head around a nice solution and I'm kinda stuck.
So, here's my situation :
I'm using Ajax to perform a series of tasks. Actually, I'm queuing (or parallely at times, doesn't really matter to me) 1 or more requests.
Show the progress as a percentage (1 out of X tasks performed)
When finished, show the final result.
What I'm trying to do :
Instead of having 3-4 different tasks running (= 3-4 different PHP scripts called asynchronously via Ajax), I would like to have just 1 (= 1 script) - in other words, combine the X scripts into one. (That's easy).
Issues I'm facing :
How could I still report the percentage complete (1 out of X tasks)?
Any ideas?
I would have updated a key in Memcached each time a task is complete. And then you would let your ajax-call simply get the value from your memcached key.
http://php.net/manual/en/book.memcached.php
If you have the request dropped into a database by your original ajax, then kick off the script, you will still have time (assuming the request takes some time to complete) for subsequent tasks to be dropped into the same database to be picked up by the script that is still running.
As for reporting it, perhaps run a quick jquery to see how many items are in the queue? Alternately have the task.php file update the database (possibly another table even) to say how many jobs it has completed and how many are currently still in the queue.
If you don't need to send much data to the PHP script, you can use a "long poll" approach. With this, you don't use AJAX but insert a script tag like this:
<script src="my_php_script?task1=x¶m_t1_1=42&task2=y"></script>
The PHP file can then send back a JavaScript command like
updatePercent(12);
after each task is done. The commands should be executed by the browser whenever they come in. Be sure to call flush after every task.
Looking into Comet may give you other ideas on how to handle the Client-Server connection.
You can manage queue - before sending AJAX request - put the task in queue (could be object or array). Run AJAX asynchronously with complete function which will remove a job from queue when it's done.
You can update progress together with removing job from queue or handle it separately using setTimeout() which will check how many task there are in queue and how many were put in it in total: % = (submitted_tasks - items_in_queue) / submitted_tasks * 100
Related
I have an html which run a php several times (about 20).
Something like
for(i=0;i<20; i++){
$.get('script.php?id='+i,function(){...});
}
Every time the script runs, it get some content through different websites (every time a different one), so each script takes from 1 to 10 seconds to complete and give a response.
I want to run all the script simultaneously to be faster, but 2 things makes it slow: the first is on the html page, it seems that ajax requested are queued after first five, at least chrome developer said me that... (this can be fixed easily I think, I've not bothered yet to find a solution); the second thing is on php side: even if the first 5 scripts are triggered together, they are run sequentially, not even in order. I 've put some
echo microtime(true);
around the script to get wherever the script is slow and what I found out (a bit surprised) is that the time on the beginning of the script (the one which should be run at almost the same time on all script) is different: the difference is very consistent, also 10 seconds, like if the second script wait the first to end before begin. How can I have all the script be running together at the same time? Thank you.
I very frankly advise that you should not attempt anything "multi-threaded" here, "nor particularly 'fancy.'"
Design your application to have some kind of queue (it can be a simple array) of "things that need to be done." Then, issue "some n" number of initial AJAX requests: certainly no more than 3 to 5.
Now, wait for notification that each request has succeeded or failed. After somehow recording the status of the request, de-queue another request and start it.
In this way, "n requests, but no more than n requests," will be active at any one time, and yes, they will each take a different amount of time, "but nobody cares."
"JavaScript is not multi-threaded," and we have no need for it. Everything is done by events, which are handled one at a time, and that happens to work beautifully.
Your program runs until the queue is empty and all of the outstanding AJAX requests have been completed (or, have failed).
There's absolutely no advantage in "getting too fancy" when the requests that you are performing might each take several seconds to complete. There's also no particular advantage in trying to queue-up too many TCP/IP network requests. Predictable Simplicity, in this case, will (IMHO) rule the day.
I want to accomplish the following behavior in php:
1 - Script gets called with parameters
2- I Intiate a thread for a long running operation
3 - Script should return control to the caller
4- Thread executes till its finished
Is this behavior possible? What i am seeing now, is that the script wont return until the thread has finished executing, which makes sense as the execution of the thread would probably die if the script stops executing , but is there no way to stop blocking the client so they can go on about their business? Am i stuck using some exec() call to get this behavior? Is there a way to get this done with threading only? Id like to avoid using exec if possible..
So if someone calls my script from a browser, it should just return immidiatly, and the long running process should keep executing until its done.
Thanks
Daniel
Yes, its possible. Call your php script via AJAX, and and create multiple instances of the ajax function dynamically. See attached screenshot. When I compared results of running a single function versus 24 instances, my data was processed about 15x faster. I am trying to populate a MySQL table with about 30 million records, and each record involves calculating distance in miles from city center, based on lat/lng. So yes, its no walk in the park. As you can see, I am averaging about See this:
multi threads http://gaysugardaddyfinder.com/screen2.PNG
multi threads http://gaysugardaddyfinder.com/screen.png
This may be a glorious hack or what not - but it sure worked great for me.
My server is a Xeon 72 Core setup with 64 GB RAM.
Here's what I'm trying to accomplish in high-level pseudocode:
query db for a list of names (~100)
for each name (using php) {
query a 3rd party site for xml based on the name
parse/trim the data received
update my db with this data
Wait 15 seconds (the 3rd party site has restrictions and I can only make 4 queries / minute)
}
So this was running fine. The whole script took ~25 minutes (99% of the time was spent waiting 15 seconds after every iteration). My web host then made a change so that scripts will timeout after 70 seconds (understandable). This completely breaks my script.
I assume I need to use cronjobs or command line to accomplish this. I only understand the basic us of cronjobs. Any high level advice on how to split up this work in a cronjob? I am not sure how a cronjob could parse through a dynamic list.
cron itself has no idea of your list and what is done already, but you can use two kinds of cron-jobs.
The first cron-job - that runs for example once a day - could add your 100 items to a job queue.
The second cron-job - that runs for example once every minute in a certain period - can check if there are items in the queue, execute one (or a few) and remove it from the queue.
Note that both cron-jobs are just triggers to start a php script in this case and you have two different scripts, one to set the queue and one to process part of a queue so almost everything is still done in php.
In short, there is not much that is different. Instead of executing the script via modphp or fcgi, you are going to execute it via command line php /path/to/script.php.
Because this is a different environment than http, some things obviously don't work. Sessions, cookies, get and post variables. Output gets send to stdout instead of the browser.
You can pass arguments to your script by using $argv.
I'm looking for a way to run a php script multiple times from a browser. Here's the scenario:
I'm building a mySQL table from a series of large files ranging anywhere from 100 megs to 2 gigs. On average, there will be around 150,000 records in the table.
I'm doing so right now by having a javascript function that does an AJAX call to the PHP script. On success, the function sets a timeout to run itself and trigger the AJAX call to run the second hundred.
My thinking behind this was to give the function a second to close out before it runs itself again.
This isn't working so well. The whole function itself works, but performance-wise it is quite slow.
When I wasn't doing 100 records at a time and not wasn't using javascript, just PHP, I could get about 15,000 records into the table before it would time out. Right now it takes about 10 minutes for it to do the same number of records.
I know that the continuous running javascript is bleeding memory and performance like crazy and was just wondering if anyone had any ideas on how to accomplish running a PHP script over and over from a browser. Crons are not an option at this point.
Its called (async) work/job queues, seems you need to explore Gearman
Couldn't you just have the PHP script itself repeat the function multiple times? If the problem is that the function sometimes fails or times out, could you could catch the exception within your script? Or do you have an unavoidable and totally fatal error that really necessitates using an external minder?
I ran into a similar situation... my solution was to use an ajax queue. Essentially you feed a series of ajax calls into a queue which runs them sequentially, starting the next after the previous has returned from the server as successful.
Setting a timeout can run into a situation where the next ajax call is made before the server completed the last. This is the likely cause of your performance issue. I don't really like javascript timeouts myself just for the resource overuse alone.
Google "Ajax Queue" for code that you find useful, or I can post mine, which is jQuery.
configure a cronjob to run your script every minute
hello i have some problems with my php ajax script
i'm using PHP/mysql
i have a field in my accounts table that will save the time for the last request from a user, i will use that to kick the idle user out of the chat. and i will make a php function that will delete all the rows that its time field more than the time limit, but where should i use this method is it okay to fire it every time a new request sent to my index.php ? i think that will make a huge load on the server,is n't it ? do you have a better solution?
thanks
There are two viable solutions:
either create a small PHP script that makes this deletion in an infinite loop (and of course sleeps for a specified amount of time before doing it again), and then start it via PHP CLI,
or create one that makes the deletion only once, then exits, and call it from cron (if you're using a UNIXish server) or Task Scheduler (on Windows).
The second one is simpler, but its drawback is that you can't make the interval between the deletions shorter than 60 seconds.
A solution could be to fire the deletion function just once every few requests.
Using rand() you could give it a 1 in 100 (for example) change of running the function, so that about one page request in a 100 will clean up the expired data.