I have a quite heavy SQL search query that takes a few minutes to complete, called from a PHP script that's called by an ajax request.
Problem is, users often click the search button many times from impatience, and each time it creates a new ajax request, and a new PHP script execution. But the old ones continue executing, even though the connection has been dropped. This causes the SQL server load to constantly be at 100% CPU usage.
So I tested my theory that script execution continues even after the browser tab is closed. I used 2 types of queries, an ad hoc query and a stored procedure execution, both methods do the same thing, inserting the numbers 1-9 into a table, with a 2 second delay between each number.
Ad hoc query:
for($i = 1; $i < 10; $i++){
$sql = "INSERT INTO t (i) VALUES(?)";
$res = pdoQuery($sql, array($i));
if($res === false){
echo $pdo->getErrorMessage();
http_response_code(500);
exit();
}
sleep(2);
}
SP call:
$sql = "EXEC sp_Slow";
$res = pdoQuery($sql);
if($res === false){
echo $pdo->getErrorMessage();
http_response_code(500);
exit();
}
How I tested: using buttons that trigger ajax calls to each script, I tested them, by clicking the button and immediately closing the tab. And then monitoring the data in the table. And just as I suspected, new data gets inserted every 2 seconds. (This also happens if I directly open the script in the browser and closing the tab, instead of requesting it through ajax)
I need a way to completely kill both PHP and SQL execution whenever the user disconnects, transactions are not important because it's just a select operation.
You can change this behaviour using php.ini directive or at runtime with ignore_user_abort() function.
Here's what I did, from the comment by #Robbie Toyota, thanks!
if(!empty($_SESSION['SearchSPID'])){
$sql = "KILL " . $_SESSION['SearchSPID'];
$res = pdoQuery($sql);
if($res === false){
exit('Query error: failed to kill existing spid:' . $_SESSION['SearchSPID']);
}
$_SESSION['SearchSPID'] = null;
}
$sql = "SELECT ##spid AS SPID";
$res = pdoQuery($sql);
$spid = $res->row_assoc()['SPID'];
$_SESSION['SearchSPID'] = $spid;
// do long query here
$_SESSION['SearchSPID'] = null;
Of course using this method you have to be careful about session file locking, which if happens will make this whole thing pointless, because then the requests will be sequential and not parallel
Related
I appear to be having some difficulties with handling running multiple database calls, especially in regards to large datasets being returned. It appears that PHP only lets you have one database call running at a time, per session. This normally isn't an issue, as the database calls tend to be so small it doesn't lock anything up, but the large ones cause this waiting issue.
I discovered this issue when I fixed an unrelated issue, and discovered that if you click a button to query the database via an AJAX call, then try to refresh the website, it won't start loading the website until that database call is done, as the page does have an internal function to make a database call. Conversely, if I were to start the database query, then load up a pure html webpage stating "Hello World", it loads instantly. Based on this, Apache isn't having an issue serving, it's something to do with database connections.
To point, I've isolated code that's possibly relevant, as I can't figure out why I'm only able to have one active call at a time. In short, is there a way to have multiple database calls running per user at a time, or will a user have to wait?
db_connect.php:
<?php
$user = 'TEST';
include_once 'config.php'; //Intialize constants for the connection
$conn = oci_connect(USER, PASSWORD, '//'.HOST.':1630/'.DATABASE);
oci_set_client_identifier($conn, $user); //Identify who's making these calls.
?>
events.php: (if I refresh this after clicking the ajax button to do the same fetch, it won't load until that AJAX call is finished. Doesn't matter if I have code to abort the call, the database is still running that database query.)
<?php
session_start();
include 'db_connect.php';
include 'database/event_defs.php';
?>
<html>
<!-- boilerplate nonsense -->
<body>
<table>
<?php
$dataset = get_event_list($conn, $_SESSION['username']); //Returns 1000 records, could take a while to fully retrieve it.
foreach($dataset as $key => $val) {
//Make multiple rows happen here.
}
?>
</table>
<button onclick="do_ajax_call('get_event_list');">Make DB Call</button>
</body>
</html>
database/event_defs.php: (Probably the most relevant part).
<?php
function get_event_list($conn, $user) {
$l_result = array();
$sql = 'BEGIN ...(:c_usr, :c_rslt); END'; //name is irrelevant.
if($stmt = oci_parse($conn, $sql)) {
$l_results = oci_new_cursor($conn);
oci_bind_by_name($stmt,':c_usr',$user);
oci_bind_by_name($stmt,':c_rslt',$lresults,-1,OCI_B_CURSOR);
if(oci_execute($conn)) {
oci_execute($l_results); //Problem line, seems to stall out here for a while and won't let the user query again until this call finishes.
while($r = oci_fetch_array($l_results, OCI_ASSOC) {
$l_result[] = $r;
}
} else {
return 'bad statement';
}
} else {
return 'unable to connect';
}
return $l_result;
}
?>
Version information:
PHP 5.4.45
Oracle 11g
Apache 2.2.15
As MonkeyZeus monkey has already pointed out in the comments to your question, the second request is most likely only blocked by the session mechanism.
Since it looks like you don't need anything but the username from the session, just grab that value and finish the session mechanism.
<?php
session_start();
// check $_SESSION['username'] here if necessary
$username = $_SESSION['username'];
// no need to keep the session mecahnism "alive"
session_abort(); // and since nothing has been written to _SESSION, abort() should do.
require 'db_connect.php';
require 'database/event_defs.php';
?>
<html>
<!-- boilerplate nonsense -->
<body>
<table>
<?php
$dataset = get_event_list($conn, $username); //Returns 1000 records, could take a while to fully retrieve it.
foreach($dataset as $key => $val) {
//Make multiple rows happen here.
}
?>
It's PHP session blocking mechanism.
You need to call session_write_close() when you don't need session any more.
May be after this string:
$dataset = get_event_list($conn, $_SESSION['username']);
After calling session_write_close() you can't use $_SESSION.
I have an sqlite database that I query from PHP periodically. The query is always the same and it returns me a string. Once the string changes in the database the loop ends.
The following code is working, but I am pretty sure this is not the optimal way to do this...
class MyDB extends SQLite3
{
function __construct()
{
$this->open('db.sqlite');
}
}
$loop = True;
while ($loop == True) {
sleep(10);
$db = new MyDB();
if (!$db) {
echo $db->lastErrorMsg();
} else {
echo "Opened database successfully\n";
}
$sql = 'SELECT status from t_jobs WHERE name=' . $file_name;
$ret = $db->query($sql);
$state = $ret->fetchArray(SQLITE3_ASSOC);
$output = (string)$state['status'];
if (strcmp($output, 'FINISHED') == 0) {
$loop = False;
}
echo $output;
$db->close();
}
If you want to get an output immediately and a kind of interface, I think The best solution for your problem might be to use HTTP long polling. This way, it will not hold the connection for hours if the job is not done:
you will need to code a javascript snippet (in another html or php page) that runs an ajax call to your current php code.
Your web server (and so, your php code) will keep the connection opened for a while until the job is done or a time limit is reached (say 20-30 seconds)
if the job is not done, the javascript will make another ajax call and everything will start again, keeping the connection, etc... until you get the expected output status...
BEWARE : this solution will not work on any hosting provider
You will need to set the max_execution_time to a higher value than the default one see php doc for this.
I think you can find many things on http long polling with php/javascript on google / stack overflow...
I have code as below where I am redirecting the user at the end but sometimes it seems that the output variable is not getting set or insert query is not getting executed . So is there any way I can wait for all the operations to be completed before the die statement is executed.
Thanks
$diff=levenshtein(strtolower($str1),strtolower($str2));
if($diff<=2)
{
$output="ok";
}
else
{
$output="Not ok";
}
$query="INSERT INTO `table` (`sn`, `output`) VALUES (NULL,'$output')";
$result=mysqli_query($con,$query);
header('Location: http://www.domain.com/');
die;
There may be several reasons why it doesn't work.
The first one and most obvious is that is enough to have only one simple warning on your page (variable X is not set) and this will broke your header() command. It would say that headers have been already set.
The second problem is that sometimes the server is so busy answering other request and your $con variable will be a Boolean false instead of a reliable database connector, therefore your query could fail because of this too.
I encountered both situations in real life and I solved them by ensuring that every variable is set before use it and by checking if the query was executed successfully.
You can check if a variable is set by using this code:
if (!isset($variable)) $variable = "set me here";
You can check if your query was succesfully executed by adding this:
if (!$result) {
//do something here in case your query failed
}
I'm having trouble getting a table to lock in MySQL. I've tried testing for concurrent requests by running two scripts at the same time on different browsers.
Script 1:
mysql_query("lock tables id_numbers write");
$sql = "select number from id_numbers";
$result = mysql_query($sql);
if ($record = mysql_fetch_array($result))
{
$id = $record['number'];
}
sleep(30);
$id++;
$sql = "update id_numbers set number = '$id'";
$result = mysql_query($sql);
mysql_query("unlock tables");
Script 2:
$sql = "select number from id_numbers";
$result = mysql_query($sql);
if ($record = mysql_fetch_array($result))
{
$id = $record['number'];
}
echo $id;
I start Script 1 first, then start Script 2. Theoretically, Script 2 should wait 30+ seconds until Script 1 unlocks the table, and then output the updated ID. But it immediately outputs the original ID, so the lock is obviously not taking effect. What could be going wrong?
BTW, I know I shouldn't still be using mysql_*, but I'm stuck with it for now.
EDIT: I've discovered that the lock does happen on the live site, but not on my dev site. They are both on shared hosts, so I'm assuming there's some setting that's different between the two. Any idea what that could be?
Using the code you have here, I get exactly the behavior you expect: script 2 will block until script 1 issues the unlock tables.
Your problem must lie elsewhere. Things to check:
Does the table actually exist on the database? (Check your mysql_query() return values and use mysql_error() if you get any FALSE returns.)
Are both scripts connecting to the same database?
Are you sure the table is not a temporary (thus connection-local) table?
Could script 1's mysql connection (or the script itself) be timing out during the sleep(30), thus releasing the lock earlier than you expect?
I'm trying to increment +1 impression every time an ad is displayed on my site, however the variable increments +2 to +3 arbitrarily. I've removed everything that's working correctly and I made a page with only this code in it:
<?php
require "connect_to_mydb.php";
echo 'Hello***** '.$testVariable=$testVariable+1;
mysql_query("UPDATE `imageAds` SET `test`=`test`+1 WHERE `id`='1'");
?>
Every time the page is refreshed the, test increments arbitrarily either +2 or +3 and my page displays Hello***** 1 (Just to show its not looping). Access is restricted to this page so it's not other users refreshing the page.
Also, id and test are int(11) in the DB.
My DB required connection has nothing in it that would interfere.
Edit
Here is an updated code:
<?php
require "connect_to_mydb.php";
mysql_query("UPDATE `imageAds` SET `test`=`test`+1 WHERE `id`='1'");
$sql = mysql_query("SELECT * FROM imageAds WHERE id='1' LIMIT 1");
$check = mysql_num_rows($sql);
if($check > 0){
$row = mysql_fetch_array($sql);
echo $row['test'];
}
?>
Increments by +2 everytime
Edit
This is whats in connect_to_mydb.php
<?php
$db_host = "*************************";
$db_username = "*********";
$db_pass = "**********";
$db_name = "**************";
mysql_connect("$db_host","$db_username","$db_pass") or die ("could not connect to mysql");
mysql_select_db("$db_name") or die ("no database");
?>
Either there's a bug in MySQL's implementation of UPDATE, or you're doing something wrong in some code you haven't posted.
Hint: It's very unlikely to be a bug in MySQL. Other people would have noticed it.
From what you've shown, it looks like your page is being loaded multiple times.
This attempt to prove that the code is only being called once doesn't prove anything:
echo 'Hello***** '.$testVariable=$testVariable+1;
This will always print the same thing (Hello***** 1) even if you open this page multiple times because the value of $testVariable is not preserved across seperate requests.
This +2/+3 error is occurring only with Chrome and my Mobile Android browser and the code is solid. I looked to see if there is any issue with Chrome sending more than one http request (thx user1058351) and there is which is documented here:
http://code.google.com/p/chromium/issues/detail?id=39402
So since this way was unreliable I just completed a work around that is solid. Instead of including a PHP file that updates the amount of ad impressions on reload, I now have it so when the page loads, an AJAX request is sent to a separate PHP file which updates the ad stats and returns the appropriate data. The key I think is to send it through the JS code so only one http request can be sent to increment the data.
Thank you to all who responded especially user1058351 and Mark Byers (not a bug in MYSQL but possibly appears to be a bug in Chrome).