If a user stops a page from loading completely in the middle of a transaction and then refreshes the page, will transaction apply twice?
Short answer is - you have to check. Transactions fail if even one query doesn't execute correctly. However, here's the real issue - if the user aborts the loading, that doesn't mean the server will stop executing the code. Aborting the page load just means that the user decided not to receive the data back. Your web server doesn't have to comply, nor does PHP - they might not even be aware of the fact that user closed the pipe on their end. Now we enter a whole different play area - PHP can be embedded into servers or it can act as a standalone server (known as php-fpm). If PHP is embedded into a server such as Apache via mod_php - Apache decides whether it will interrupt the thread or process that's executing the PHP process. It can still decide to execute the whole thing, collect the result and then it will try to write the result back to the socket - only to realize that the user isn't there any more.
As you can see, the complexity of your problem is not as simple as "what if user aborts" - it's not up to the user to control program execution on your web server.
Now, let's assume that the web server is willing to play ball and that it will somehow tell PHP to abort execution - in that case, transaction won't execute, it will fail because commit won't be issued.
If the server isn't willing to play ball or it simply hasn't got the means to abort the PHP process - then yes, you will end up with two transactions executed, or more, depending what your users do.
Related
We have a client server architecture with Angular on client side and Apache2 PHP PDO and MySQL on the server side. server side exposing an API to clients that gives them data to show.
Some observations :
some API calls can take very long to compute and return response.
server side seem to handle a single request per client at any given time (im seeing only one coresponding query thats being executed in mysql), that limit comes either from apache or from mysql since front-end sending requests in parallel for sure.
front end cancels requests that are not relevant anymore (data being fetched will not be visible)
seems like requests canceled by front end are not canceled in server side and continues to run anyway, i think even if they are queued they will still run when their turn arrives (even though they were cancelled on client side)
Need help to understand :
what exactly is the cause of not having all requests (or at least X>1 requests) run on parallel? can it be changed?
What configurations should i change in either apache or mysql to overcome this?
is there a way to make apache drop cancelled requests? at least those that are still queued and not started?
Thanks!
EDIT
Following #Markus AO comment (Thanks Markus!!!) this was session blocking related... wish i knew about that before!
OP has a number of tangled problems on the table. However I feel these are worthwhile concerns (having wrestled with them myself), so let's take this apart. For great justice; main screen turn on:
Solving Concurrent Request Problems
There are several possible problems and solutions with concurrent connections in a (L)AMP stack. Before looking at tuning Apache and MySQL, however, let me gloss a common "mystery" issue that creates concurrence problems; namely, a necessary evil called "PHP Session Locking".
PHP Session Blocking & Concurrent Requests
In a nutshell: When you use sessions in your application, after calling session_start(), PHP locks the session file stored at your session.save_path directory. This file lock will remain in place until the script ends, or session_write_close() is called. Result: Any subsequent calls by the same user will be queued, rather than concurrently processed, to ensure there's no session data corruption. (Imagine parallel scripts writing into the same $_SESSION!)
An easy way to demonstrate this is to create a long-running script; then call it in your browser; and then open a new tab, and call it again (or in fact, call any script sharing the same session cookie/ID). You'll see that the second call won't execute until the first one is concluded. This is a common cause of strange AJAX lags, especially with parallel AJAX requests from a single page. Processing will be consecutive instead of concurrent. Then, 10 calls at 0.3 sec each will take a total of 3 sec to conclude, and so on. We don't want that, do we!
You can remedy request blocking caused by PHP session lock by ensuring that:
Scripts using sessions should call session_write_close() once done storing session data. The session lock will be immediately released.
Scripts that don't require sessions shouldn't start sessions to begin with.
Scripts that need to only read session data: Using session_start() with ['read_and_close' => true] option will give you a read-only (non-persistent) $_SESSION variable without session locking. (Available since PHP 7.)
Options 1 and 3 will leave you with read access for the $_SESSION variable and release/avoid the session lock. Any changes made to $_SESSION after the session is closed will be silently discarded; no warnings/errors are displayed.
The session lock request blocking issue is only consequential for a single user (using the same session). It has no impact on multi-user concurrence. For further reading, please see:
SO: Session (Auto)-Start, Performance & Session Locking
SO: PHP & Sessions: Is there any way to disable PHP session locking?
In-Depth: PHP Session Locking: How To Prevent Sessions Blocking in PHP requests.
Apache & MySQL Concurrent Requests
Once upon a time, before realizing PHP was the culprit behind blocking/queuing my concurrent calls, I spent a small aeon in tweaking Apache and MySQL and wondering, what happen?
Apache 2.4 supports 150 concurrent requests by default; any further requests will queue up. There are several settings under the MPM/Multi-Processing Module that you can tune to support the desired level of concurrent connections. Please see:
MPM Docs
Worker Docs
Overview at Oxpedia
MySQL has options for max_connections (default 151) and max_user_connections (default unlimited). If your application sends a lot of concurrent requests per user, you'll want to ensure the global max connections is high enough to ensure a handful of users don't hog the entire DBMS.
Obviously, you'll further want to tune these settings in light of your server CPU/RAM specs. (The calculations for which are beyond this answer.) Your concurrency issues probably aren't caused by too many open TCP sockets, but hey, you never know...
Canceling Requests to Apache/PHP/MySQL
We don't have much to go on as far as your application's specific wiring, but I understand from the comments that as it stands, a user can cancel a request at the front-end, but no back-end action is taken. (Ie. any back-end response is simply ignored/discarded.)
"Is there a way to make Apache drop cancelled requests?" I'm assuming that your front-end sends the requests directly and without delay to Apache; and onward to PHP > MySQL > PHP > Apache. In that case, no, you can't really have Apache cancel the request that it's already received; or you could hit "stop", but chances are PHP and MySQL are already munching it away...
Holding a "Cancel Window"
However, you could program a "cancel window" lag into your front-end, where requests are only passed on to Apache after e.g. a 0.5-second sleep waiting for a possible cancel. This may or may not have a negative impact on the UX; may be worth implementing to save server resources if a significant portion of requests are canceled. This assumes an UI with Javascript. If you're getting direct HTTP calls to API, you could have a "sleepy proxy receiver" instead.
Using a "Cancel Controller"
How would one cancel PHP/MySQL processes? This is obviously only feasible/doable if calls to your API result in a processing time of any significant duration. If the back-end takes 0.28 sec to process, and user cancels after 0.3 seconds, then there isn't much left to cancel, is there.
However, if you do have scripts that may run for longer, say into a couple of seconds. You could always find relevant break-points in your code, where you have a "not-canceled" check or a kill/rollback routine. Basically, you'd have the following flow:
Front-end sends request with unique ID to main script
PHP script begins the long march for building a response
On cancel: Front-end re-sends the ID to a light-weight cancel controller
Cancel controller logs ID to temporary file/database/wherever
PHP checks at break-points if there's a cancel request for current process
On cancel, PHP executes a kill/rollback routine instead of further processing
This sort of "cancel watch" will obviously create some overhead, and as such you may want to only incorporate this into heavier scripts, to ensure you actually save some processing time in the big picture. Further, you'd only want at most a couple of breakpoints at significant junctions. For read requests, you could just kill the process; but for write requests, you'd probably want to have a graceful rollback to ensure data integrity in your system.
You can also cancel/kill a long-running MySQL thread, already initiated by PHP, with mysqli::kill. For this to make sense, you'd want to run it as MYSQLI_ASYNC, so PHP's around to pull the plug. PDO doesn't seem to have a native equivalent for either async queries or kill. Came across $pdo->query('KILL CONNECTION_ID()'); and PHP Asynchronous MySQL Query (see answer for PDO). Haven't tested these myself. Also see: Kill MySQL query on user abort
PHP Connection Handling
As an alternative to a controller that passes the cancel signal "from the side", you could look into PHP Connection Handling and poll for aborted connection status at your cancel check-points with connection_aborted(). (See "MySQL kill" link above for a code example.)
A CONNECTION_ABORTED state follows if a user clicks the "stop" button in their browser. PHP has a ignore_user_abort() setting, default "Off", which should abort a script on user-abort. (In my experience though, if I have a rogue script and session lock is on, I can't do anything until it times out, even when I hit "stop" in the browser. Go figure.)
If you have "ignore user abort" on false, ie. the PHP script terminates on user abort, be aware that this will be a wholly uncontrolled termination, unless you have register_shutdown_function() implemented. Even so, you'd have to flag check-points in your code for your shutdown function to be able to "rewind the clock" from the termination point onward. Also note this caveat:
PHP will not detect that the user has aborted the connection until an attempt is made to send information to the client. Simply using an echo statement does not guarantee that information is sent, see flush(). ~ PHP Manual on ignore_user_abort
I have no experience with implementing "user abort" over AJAX/JS. For a starting point, see: Abort AJAX Request and Cancel an HTTP fetch() request. Not sure how/if they register with PHP. If you decide to travel down this road, please return and update us with your code / research!
I am writing a website with php and there is a part of code need huge amount of time to execute.
Since I don't use thread, when I run that code, the whole server is blocked by it. But it's OK.
Hovever, even though I closed that web page, it still executes and blocked my server. I cannot access any page of my website until the process completed.
Since the execution time is very long, so that I set a very long set_time_limit() for it but I don't set ignore_user_abort so that I supposes that it should not run after user abort. Or is it the problem of curl(the code does many curl job)?
Can someone tell me that why the php script cannot stop when the user close the connection? Or there are some way to assure the script can be stopped when user abort?
Thanks.
Closing the browser doesn't tell the server to stop doing something. It doesn't tell the server anything.
Long-running processes don't belong in web applications. Generally you would want some background task to handle the process. Either the web application would spawn this task (this seems like a workable approach) or would in some way queue the processing of this task where a background worker would see that queue (such as a database table polled every X minutes by a daemon process).
The goal is to not block the UI while the task is running. Even if the user were to leave the browser open, the browser itself may "give up" after a while or something else could sever the user from the UX while waiting for too long. Let the user invoke the process, but separate the invocation of the process from the execution of the process so the user can return to the application interface.
From the PHP Manual:
PHP will not detect that the user has aborted the connection until an
attempt is made to send information to the client.
Thus, even using ignore user abort, you must try to interact with the client again inside the script to ensure it aborts correctly. Note that on the page in question there are some additional notes about what constitutes 'sending information' (for example, an Echo doesn't qualify by itself, apparently).
Further Reading:
http://php.net/manual/en/function.ignore-user-abort.php
I recently executed a mysql query via chrome and closed it out. How exactly does a browser stop a PHP script using the stop button? I thought PHP was a server-side language and could not be controlled through a client.
*UPDATE*I'm now aware of SHOW PROCESSLIST, but this only shows you which threads are running.Is there a SQL command I can use to view a executed query with great detail?
A client (Chrome) has nothing to do with the execution of scripts (PHP) on the server, which in turn have no control over database processes (MySQL query).
Look at your servers process list to see what's going on in general (Apache processes).
Or even better: use SHOW PROCESSLIST; on the MySQL console to find the long running query. You may quit it by using KILL ###ID_OF_QUERY###;.
No, you don't need to keep it open. If you exit a running car, does the car turn off? No.
Sorry, that came off a little snotty, but it wasn't intended too.
The browser, in your case Chrome, is not actually running the actual code. The server is. Thus, once the instruction is executed, closing the browser no longer matters as the request has been given to the server.
two functions are essential for executing time consuming php scripts.
it has nothing to do with the browser (as other users already pointed out)
lookup ignore_user_abort and set_time_limit
The script will continue to execute regardless of browser closure. You can free up your browser by sending the response and allowing the php process to continue on.
ignore_user_abort(true);
$response = "Processing!";
header("Connection: close");
header("Content-Length: " . mb_strlen($response));
echo $response;
flush();
// Insert your lengthy query here
The Answer is it depends, as others mentioned you can check what is running on the mysql server by using the show processlist;
If it is a single query that takes a long time, the it will most likely carry on running after the browser has closed. PHP will have sent the request to the Database and will in effect be sat waiting for it to complete, in turn the browser will be waiting for the webserver to finish building the page/resource that is on that url
so the request is: browser <-> web server (<-> php ) <-> mysql in an ideal world if the user cancels the request everything would tidy itself up nicely, but that in my experience sadly is not the case, if one of the chain decides not to wait, the process that it is waiting for doesn't necessarily know until it tries to send the response back and fails
Come on guys, this is PHP 101. Quoted from the manual:
You can decide whether or not you want a client disconnect to cause
your script to be aborted. Sometimes it is handy to always have your
scripts run to completion even if there is no remote browser receiving
the output. The default behaviour is however for your script to be
aborted when the remote client disconnects.
Execution will stop at the next tickable event after the connection flag is set to ABORTED - which will be detected when PHP attempts to flush output to the client
The current MySQL query will finish executing (as the next event that PHP has control over doesn't occur until after the query has completed), but your script would not make it past that, unless you explicitly set ignore_user_abort. It's always important to account for this when writing code.
There are two ways around this
Set ignore_user_abort to true for the invocation of your script
Do not print anything back to the client until after all of your processing is complete - since a connection closed status won't be detected until output is flushed
Suppose a page takes a long time to generate, some large report for example, and the user closes the browser, or maybe they press refresh, does the PHP engine stop generating the page from the original request?
And if not, what can one do to cope with users refreshing a page a lot that causes an expensive report to be generated.
I have tried this and it seems that it does not stop any running query on the database. But that could be an engine problem, not PHP.
Extra info:
IIS7
MS SQL Server via ODBC
When you send a request to the server, it is executed on the server without any communication with the browser until information is sent back to the browser. When PHP tries to send data back to the browser, it will fail and therefore the script will exit.
However, if you have a lot of code executing before any headers are sent, this will continue to execute until the headers are sent and a failed response is received.
PHP knows when a connection has been closed when it tries to output some data (and fails). echo, print, flush, etc. Aside from this, no, it doesn't; everything else is happening on the server end.
There is little in the way of passing back information about the browser state once a request has been made (or in your case, in progress)
To know if a user is still connected to your site, you will need to implement a long poll / comet or perhaps a web socket.
Alternatively - you may want to run the long query initiated via an ajax call - while keeping the main browser respsonsive (not white screened). This allows you to detect if the browser is closed during the long query with a Javascript event onbeforeunload() to notify your backend that the user has left. (I'm not sure how you would interupt a query in progress from another HTTP request though)
PHP have two functions to control this. set_time_limit(num) able to increase the limit before a page execution "dies". If you don't expand that limit, a page running "too long" will die. Bad for a long process. Also you need ignore_user_abort(TRUE) so the server don't close the PHP process if the server detect that the page has ben closed in the client side.
You may also need to check for memory leaks if you are writing something that use much memory and run for several hours.
When you send a request to the server the server will go away and perform the appropriate actions. IIS/SQL Server does not know if the browser has been closed (and it is not IIS/SQL Server's responsibility to understand this) so it will execute the commands (as told to do so by the PHP engine until it has finished or until the engine kills any transactions). Since your report could be dynamic, IIS will not cache page requests, SQL Server however can cache the last previously ran queries therefore you will see some performance gain from the database backend.
I've written in PHP a script that takes a long time to execute [Image processing for thousands of pictures]. It's a meter of hours - maybe 5.
After 15 minutes of processing, I get the error:
ERROR
The requested URL could not be retrieved
The following error was encountered while trying to retrieve the URL: The URL which I clicked
Read Timeout
The system returned: [No Error]
A Timeout occurred while waiting to read data from the network. The network or server may be down or congested. Please retry your request.
Your cache administrator is webmaster.
What I need is to enable that script to run for much longer.
Now, here are all the technical info:
I'm writing in PHP and using the Zend Framework. I'm using Firefox. The long script that is processed is done after clicking a link. Obviously, since the script is not over I see the web page on which the link was and the web browser writes "waiting for ...".
After 15 minutes the error occurs.
I tried to make changes to Firefox threw about:config but without any success. I don't know, but the changes might be needed somewhere else.
So, any ideas?
Thanks ahead.
set_time_limit(0) will only affect the server-side running of the script. The error you're receiving is purely browser-side. You have to send SOMETHING to keep the browser from deciding the connection's dead - even a single character of output (followed by a flush() to make sure it actually get sent out over the wire) will do. Maybe once every image that's processed, or on a fixed time interval (if last char sent more than 5 minutes ago, output another one).
If you don't want any intermediate output, you could do ignore_user_abort(TRUE), which will allow the script to keep running even if the connection gets shut down from the client side.
If the process runs for hours then you should probably look into batch processing. So you just store a request for image processing (in a file, database or whatever works for you) instead of starting the image processing. This request is then picked up by a scheduled (cron) process running on the server, which will do the actual processing (this can be a PHP script, which calls set_time_limit(0)). And when processing is finished you could signal the user (by mail or any other way that works for you) that the processing is finished.
use set_time_limit
documentation here
http://nl.php.net/manual/en/function.set-time-limit.php
If you can split your work in batches, after processing X images display the page with some javascript (or META redirects) on it to open the link http://server/controller/action/nextbatch/next_batch_id.
Rinse and repeat.
batching the entire process also has the added benefit that once something goes wrong, you don't have to start out the entire thing anew.
If you're running on a server of your own and can get out of safe_mode, then you could also fork background processes to do the actual heavy lifting, independent of your browser view of things. If you're in a multicore or multiprocessor environment, you can even schedule more than one running process at any time.
We've done something like that for large computation scripts; synchronization of the processes happened over a shared database---but luckily enough, they processes were so independent that the only thing we needed to see was their completion or termination.