I have a form in PHP that saves some data into a database. However, after a successful insert, the PHP function below executes to arrange every entry by a "visible" ID column (called correlativo in this case).
function ordenar_ids()
{
$sql = "SELECT DISTINCT(DATE_FORMAT(fecha, '%Y')) FROM emergencias ORDER BY fecha";
$results = mysql_query($sql);
$anios = [];
while ($row = mysql_fetch_array($results))
array_push($anios, $row[0]);
foreach($anios as $anio)
{
$sql = "SELECT id_emergencia FROM emergencias WHERE fecha LIKE '".$anio."%' ORDER BY fecha ASC;";
$results = mysql_query($sql);
$correlativo = 1;
while ($row = mysql_fetch_array($results))
{
$sql = "UPDATE emergencias SET correlativo = $correlativo WHERE id_emergencia = '".$row[0]."';";
mysql_query($sql);
$correlativo++;
}
}
return 1;
}
Initial (local) tests were successful, however when we tested this in a production environment we noticed this process is taking way too long (about 15 seconds on average) which is detrimental to the user experience.
Is it possible to run this function in the background after the user has submitted the form? If it's not possible, how can we work around this problem?
Additional Data:
The user doesn't need a return value of this function, nor it's needed for the following actions that happens after.
You can't easily run a mysql query in the background. But xou could start a new php script in background: http://php.net/manual/de/function.exec.php#86329
But better would be to improve the script performance itself. I think, you can improve it by using one "update ... select" query, avoiding LIKE % or using mysql triggers.
Related
My problem is simple. On my website I'm loading several results from MySQL tables inside a while loop in PHP and for some reason the execution time varies from reasonably short (0.13s) or to confusingly long (11s) and I have no idea why. Here is a short version of the code:
<?php
$sql =
"SELECT * FROM test_users, image_uploads
WHERE test_users.APPROVAL = 'granted'
AND test_users.NAME = image_uploads.OWNER
".$checkmember."
".$checkselected."
ORDER BY " . $sortingstring . " LIMIT 0, 27
";
$result = mysqli_query($mysqli, $sql);
$data = "";
$c = 0;
$start = microtime(true);
while($value = mysqli_fetch_array($result)) {
$files_key = $value["KEY"];
$file_hidden = "no";
$inner_query = "SELECT * FROM my_table WHERE KEY = '".$files_key."' AND HIDDEN = '".$file_hidden."'";
$inner_result = mysqli_query($mysqli, $inner_query);
while ($row = mysqli_fetch_array($inner_result)) {
// getting all variables with row[n]
}
$sql = "SELECT * FROM some_other_table WHERE THM=? AND MEMBER=?";
$fstmt = $mysqli->prepare($sql);
$fstmt->bind_param("ss", $value['THM'], 'username');
$fstmt->execute();
$fstmt->store_result();
if($fstmt->num_rows > 0) {
$part0 = 'some elaborate string';
} else {
$part0 = 'some different string';
}
$fstmt->close();
// generate a document using the gathered data
include "../data.php"; // produces $partsMerged
// save to data string
$data .= $partsMerged;
$c++;
}
$time_elapsed_secs = substr(microtime(true) - $start, 0, 5);
// takes sometimes only 0.13 seconds
// and other times up to 11 seconds and more
?>
I was wondering where the problem could be.
Does it have to do with my db connection or is my code flawed? I haven't had this problem at the beginning when I first implemented it but since a few months it's behaving strangely. Sometimes it loads very fast other times as I said it takes 11 seconds or even more.
How can I fix this?
There's a few ways to debug this.
Firstly, any dynamic variables that form part of your query (e.g. $checkmember) - we have no way of knowing here whether these are the same or different each time you're executing the query. If they're different then each time you are executing a different query! So it goes without saying it may take longer depending on what query is being run.
Regardless of the answer, try running the SQL through the MySQL command line and see how long that query takes.
If it's similar (i.e. not an 11 second range) then the answer is it's nothing to do with the actual query itself.
You need to say whether the environment you're running this in is a web server, e.g. accessing the PHP script via a browser, or executing the script via a command line.
There isn't enough information to answer your question. But you need to at least establish some of these things first.
The rule of thumb is that if your raw SQL executes on a MySQL command line in a similar amount of time on subsequent attempts, the problem area is elsewhere (e.g. connection to a web server via a browser). This can be monitored in the Network tab of your browser.
So here is my scenario...
The bug_tracker table is in one server and task_traker is in another.
I want to show a combined result but can't since there are in two separate databases remotely.
So I am calling the task tracker first and then getting the bug details per iteration.
$task = oci_parse($task_conn, "select * from task_table where ....");
oci_execute($task);
while ($task_row = oci_fetch_array($task, OCI_ASSOC+OCI_RETURN_NULLS)) {
$bug = oci_parse($bug_conn, "select * from bug_table where id = " . $task_row['BUGID'] );
oci_execute($bug);
while ($task_row = oci_fetch_array($task, OCI_ASSOC+OCI_RETURN_NULLS)) {
... //output
}
... //output
}
But this entire process is making it very slow... since there are large number of records and columns.
Is there any way to make it even slightly faster? Note: I don't have access so can't setup oracle db links.
You could improve it using the IN statement:
<?php
$task = oci_parse($task_conn, "select * from task_table where ....");
oci_execute($task);
while ($task_row = oci_fetch_array($task, OCI_ASSOC+OCI_RETURN_NULLS)) {
$bugs[] = $task_row['BUGID'];
$users[] = $task_row['USER'];
$status[] = $task_row['TASK_STATUS'];
}
$bug = oci_parse($bug_conn, "select * from bug_table where id IN (" . implode(',', $bugs) . ");" );
oci_execute($bug);
while ($task_row = oci_fetch_array($task, OCI_ASSOC+OCI_RETURN_NULLS)) {
// ...
}
?>
On a sidenote, why are you not using PDO? I believe using it will already give you a performance boost.
PHP is not meant for this kind of operation, neither should you try to write your own join function.
One proper way of solving this issue is to dump the data from both databases into a local database, and there do the join.
You do not need anything fancy for the local database, an SQLite3 is probably enough.
Just dump the data from each database into a CSV files using a bash script that you put into cron. After the dump, (re)create each table in your SQLite3, and load the CSVs into these tables. After this you can do a join once and push the result into a new table which you then are free to query.
This is what in the datawarehouse world is often referred to as an ETL process, just in this case, very very simplified.
i'm trying to make a long mysql query and process and update the row founded:
$query = 'SELECT tvshows.id_show, tvshows.actors FROM tvshows where tvshows.actors is not NULL';
$result = mysql_query($query);
$total = mysql_num_rows($result);
echo $total;
while ($db_row = mysql_fetch_assoc($result))
{
//process row
}
but after 60 second give me a timeout request, i have try to insert these in my php code:
set_time_limit(400);
but it's the same, how i can do?
EDIT:
only the query:
$query = 'SELECT tvshows.id_show, tvshows.actors FROM tvshows where tvshows.actors is not NULL';
takes 2-3 second to perform, so i think the problem is when in php i iterate all the result to insert to row or update it, so i think the problem is in the php, how i can change the timeout?
EDIT:
here is the complete code, i don't think is a problem here in the code...
$query = 'SELECT tvshows.id_show, tvshows.actors FROM tvshows where tvshows.actors is not NULL';
$result = mysql_query($query);
$total = mysql_num_rows($result);
echo $total;
while ($db_row = mysql_fetch_assoc($result)) {
//print $db_row['id_show']."-".$db_row['actors']."<BR>";
$explode = explode("|", $db_row['actors']);
foreach ($explode as $value) {
if ($value != "") {
$checkactor = mysql_query(sprintf("SELECT id_actor,name FROM actors WHERE name = '%s'",mysql_real_escape_string($value))) or die(mysql_error());
if (mysql_num_rows($checkactor) != 0) {
$actorrow = mysql_fetch_row($checkactor);
$checkrole = mysql_query(sprintf("SELECT id_show,id_actor FROM actor_role WHERE id_show = %d AND id_actor = %d",$db_row['id_show'],$actorrow[0])) or die(mysql_error());
if (mysql_num_rows($checkrole) == 0) {
$insertactorrole = mysql_query(sprintf("INSERT INTO actor_role (id_show, id_actor) VALUES (%d, %d)",$db_row['id_show'],$actorrow[0])) or die(mysql_error());
}
} else {
$insertactor = mysql_query(sprintf("INSERT INTO actors (name) VALUES ('%s')",mysql_real_escape_string($value))) or die(mysql_error());
$insertactorrole = mysql_query(sprintf("INSERT INTO actor_role (id_show, id_actor, role) VALUES (%d, %d,'')",$db_row['id_show'],mysql_insert_id())) or die(mysql_error());
}
}
}
}
Should definitely try what #rid suggested, and to execute the query on the server and see the results/duration to debug - if the query is not a simple one, construct it as you would in your PHP script, and only echo the SQL command, don't have to execute it, and just copy that in to the server MySQL command line or whichever tool you use.
If you have shell access, use the top command after running the above script again, and see if the MySQL demon server is spiking in resources to see if it really is the cause.
Can you also try a simpler query in place of the longer one? Like just a simple SELECT count(*) FROM tvshows and see if that also takes a long time to return a value?
Hope these suggestions help.
There are so many problems with your code.
Don't store multiple values in a single column. Your actors column is pipe-delimited text. This is a big no-no.
Use JOINs instead of additional queries. You can (or could, if the above weren't true) get all of this data in a single query.
All of your code can be done in a single query on the server. As I see it, it takes no input from the user and produces no output. It just updates a table. Why do this in PHP? Learn about INSERT...SELECT....
Here are some resources to get you started (from Googling, but hopefully they'll be good enough):
http://www.sitepoint.com/understanding-sql-joins-mysql-database/
http://dev.mysql.com/doc/refman/5.1/en/join.html
http://dev.mysql.com/doc/refman/5.1/en/insert-select.html
What is Normalisation (or Normalization)?
Let me know if you have any further questions.
The following code makes the web loading for like 10 minutes and I dont know why!
function chatheartbeat(){
include("config.php");
$useradn = $_SESSION['displayname'];
$query = "select * from chat where (userbdn = '".$useradn."' AND isread = 1) order by id ASC";
$result = mysql_query($query , $link);
$num_rows = mysql_num_rows($result);
if ($num_rows >= 1) {
$items = array();
$i='1';
while($chat = mysql_fetch_array($result)){
$items[$i]['from']=$chat['useradn'];
$items[$i]['msg']=$chat['msg'];
$items[$i]['timee']=date("H:i" ,$chat['timee']);
$i++;
}
$query = "update chat set isread = 0 where userbdn = '".$useradn."' and isread = 1";
mysql_query($query , $link);
header('Content-type: application/json');
echo json_encode($items);
exit;
}else{
sleep(2);
chatheartbeat();
}
}
Any suggestions?
The function will never return until there are some results from the first select.
I would suggest that you return after n runs through (e.g. 5), even if there are no results. The client can then re-issue the ajax call to poll again.
Also, it's not a great idea to do this in a recursive manner. I would suggest doing this in a for loop instead.
You are calling your function recursively and as the session variable does not change between the function calls, if it goes to the else part the first time, it will go there every time, creating a never ending loop.
By the way, if you want to use polling, you should set a timer in the client part (javascript) and not use a recursive function on the server side.
There are a few things you want to look at when performing any long polling techniques.
You need to exit after some predefined time period. Waiting on the server until you have something to respond with will lead you to execution timeouts.
Know what is going on with your SESSION data. PHP by default will use file based sessions and locks the file during the course of the request (unless you intervine)
What this code does is taking links from the db and compare it to a keyword, if it compares then KeywordCounter++, and in every time LinkCounter++
I want to type LinkCounter after every link it goes through but in the code I wrote it only shows me after the loop ends (after all the links crosses). How can I see the LinkCounter every time a link is checked?
How will I be able to see live the counter jumps?
<?php //holdes the db connection include('Connect.php');
$KeyWord = 'googoo';
$LinkCounter = "0";
$KeywordCounter = "0";
$query = mysql_query("SELECT * FROM doalgo where Pass != '0'") or die(mysql_error());
while ($info = mysql_fetch_array($query)) {
$id = $info['id'];
$link = $info['link'];
$num_rows = mysql_num_rows($query);
mysql_query("UPDATE doalgo SET Pass = '1' WHERE id = '$id'");
$CurrentFile = file_get_contents($link);
if (!strpos($CurrentFile, $KeyWord)) {
//nothing
} else {
mysql_query("UPDATE doalgo SET Posted = '1' WHERE id = '$id'");
$KeywordCounter++;
}
$LinkCounter++;
if ($id == $num_rows) {
die();
}
}
echo "<br />KeywordCounter: ".$KeywordCounter;
echo "<br />LinkCounter: ".$LinkCounter;
? >
Its better you calculate the average speed of update (for example number of updates per hour) and send just a single integer to the browser every 1 hour.
using jquery you can change the value shown to user with that speed.
If I understand your question correctly, you want the web page to display immediately, then constantly update the LinkCounter display as the SQL queries progress?
If this is a correct understanding, to do this requires AJAX. Your server has to send constant updates to the web browser every time $LinkCounter is updated, then the JavaScript running in the browser will update the display with that information. Obviously, it's a much more complicated thing to do than what your script currently does. It's an entirely different design pattern.
If this is truly something you want to learn to do, there are many books on the subject of AJAX, or google can help you, too.