After read many post about "delete massive amount of rows", I tried several answer for my problem but not work like hope.
Explanation : I've been commendited to transcript a desktop application to a web app. The application have to main goal a update of several sql tables.
When I was working on it, I've been informed to change the truncate method to a delete method. So now we don't do a Truncate and Insert, but a Replace/Insert and Delete all rows not replace/insert into the table.
Here some code :
$codeliste = "";
$req2 = "SELECT projection.article.ean, projection.article.artic
FROM projection.article
WHERE projection.article.annule is null and projection.article.ean is not null";
$req_res2 = mssql_query($req2);
if (mssql_num_rows($req_res2) > 0) {
$nbc = 0;
while ($result = mssql_fetch_row($req_res2)) {
if(trim($result[0]) <> "") {
if($codeliste == "") $codeliste = "'".$result[0]."'";
else $codeliste .= ",'". $result[0]."'";
$req = "REPLACE INTO fean ( bar_code , article ) VALUES ('". trim($result[0]) ."','". trim($result[1]) ."')";
if($connexion->exec($req)) {
$nbc++;
}else{
echo "<br/>Fail Replace fean";
}
}
}
echo "<br/>Exported Ean : ".$nbc."<br/>";
}else{
echo "Fail request at projection.article of ean";
}
if($codeliste <> "") {
$req = "DELETE FROM fean WHERE bar_code NOT IN (".$codeliste .")";
if($connexion->exec($req)) {
echo "Success delete Ean not export";
}else{
echo "Fail at DELETE fean";
}
}else{
echo "codeliste is empty";
}
More Information : I'm not allowed to truncate tables, I can't change the configuration of php.ini and conf.ini, I'm working on a delete up to 50.000~100.000 rows.
With the code above I obtain "SQLSTATE[HY000]: General error: 2006 MySQL server has gone away"
After that I tried tempory table, I obtain for result a loading page who never end (to be sure I waited 1h and no result from the page).
Then I tried to delete 5.000 rows by 5.000 rows with a loop, no more success.
Thanks in advance.
EDIT : I do some research and try to change the way I was deleting for a other way to the NOT IN in the code above, like CBroe suggest.
But I've got one more time "Fatal error: Uncaught exception 'PDOException' with message 'SQLSTATE[HY000]: General error: 2006 MySQL server has gone away'" when I load my page.
Have you another idea of way to make this page Delete without gone away ?
EDIT2 : I've had some change in my code in the objectif to send less request to the database mysql.
But I've been inform that my request have cause some lag issue because my delete request stay on sleep state in mysql. To be honest i don't know why, if someone could explain me why and maybe how I can avoid this problem in futur I would be grateful.
Here my code now : my goal was to reduce the number of request sql I send to the database.
$codeliste = array();$codesuppr = array();
$req2 = "SELECT projection.article.ean, projection.article.artic
FROM projection.article
WHERE projection.article.annule is null and projection.article.ean is not null";
$req_res2 = mssql_query($req2);
if (mssql_num_rows($req_res2) > 0) {
$nbc = 0;
while ($result = mssql_fetch_row($req_res2)) {
if(trim($result[0]) <> "") {
array_push($codeliste, trim($result[0]));
$req = "REPLACE INTO fean ( bar_code , article ) VALUES ('". trim($result[0]) ."','". trim($result[1]) ."')";
if($connexion->exec($req)) {
$nbc++;
}else{
echo "<br/>Fail Replace fean";
}
}
}
echo "<br/>Exported Ean : ".$nbc."<br/>";
}else{
echo "Fail request at projection.article of ean"
}
if (!empty($codeliste)) {
$req = "SELECT bar_code FROM fean";
$resultats = $connexion->query($req);
foreach($resultats AS $resultat) {
if (!in_array(trim($resultat), $codeliste)) {
array_push($codesuppr, trim($resultat));
}
}
if (!empty($codesuppr)) {
$total= 0;
$nbc = 0;
$codeliste = array();
foreach($codesuppr AS $code_barre) {
if(count($codeliste) < 500 && ($total + count($codeliste)) < count($codesuppr)) {
array_push($codeliste, $bar_code);
}else{
array_push($codeliste, $bar_code);
$req = "DELETE FROM fean WHERE bar_code IN ('".implode("','",$codeliste)."')";
if($connexion->exec($req)) {
$nbc++;
}else{
echo "Fail at DELETE fean";
}
$total .= count($codeliste);
$codeliste = array();
}
}
echo "Success delete Ean not export";
}
}else{
echo "codeliste is empty<br/>";
}
EDIT3 : After looking mysql, it's seems Replace send too many request at mysql who send back : Uncaught exception 'PDOException' with message 'SQLSTATE[HY000]: General error: 2006 MySQL server has gone away', and finish in sleep command.
Related
I would like to ask if how can the SQL rearrange the database simultaneously after there is an new input.
Sample data:
I want to follow the Hybrid FCFS + Priority where 1:1 ratio, 1 regular and 1 priority simultaneously.
The identifier of Priority here is when the Priority = 1 of that input.
Here the sample PHP script code but I am stucked here, I don't know if I will modify the code in PHP or it is directly statemented in SQL database
function save_queueSAMPLE(){
extract($_POST);
if($priority == 1){
}
else{
$sql = "INSERT INTO `queue_listSAMPLE` (`customer_name`,`priority`) VALUES('{$customer_name}','{$priority}')";
$save = $this->query($sql);
if($save) {
$resp['status'] = 'success';
$resp['id'] = $this->query("SELECT last_insert_rowid()")->fetchArray()[0];
} else {
$resp['status'] = 'failed';
$resp['msg'] = "An error occured. Error: ".$this->lastErrorMsg();
}
return json_encode($resp);
}
}
I'm creating PHP API for my Android application which will be used to scan QR codes. Part of that API is checking if scanned code is valid and can be scanned in a certain moment.
Whole checking part is a stored procedure in MariaDB database which is just executed by PHP script. Execution part in PHP is looking like this:
$sql = "CALL someProcedure('qr_code', #out); ";
$sql .= "SELECT #out AS `out`;";
if($conn->multi_query($sql)) {
while ($conn->more_results()) {
$conn->next_result();
}
$rs = $conn->store_result();
$row = $rs->fetch_assoc();
$odp = $row['out'];
if (!empty($rs)) {
$response['success'] = 1;
$response['message'] = $odp;
echo json_encode($response);
$rs->free();
} else {
$response['success'] = 0;
$response['message'] = mysqli_error($conn);
echo json_encode($response);
}
There are 4 results of that stored procedure:
scanned code doesn't exist,
scanned code is not an package (it is an articles code)
scanned code cannot be delivered just yet
scanning was successful
Now, when there is something wrong with the code, PHP part executes without a problem but if scanning is successful I would get a timeout (doesn't matter if it's default 30 seconds or 5 minutes).
The reason there is a timeout (I think) is that when scanning is successful there are some loops executed in Stored Procedure which may be returned in resultsets and choke PHP script ALTHOUGH when I execute that Stored Procedure in DBeaver (with exact same query as in PHP) there is no problem.
So, my question is what I can do about it? Removing the while loop in PHP script above makes the script execute without a problem (but I can't get the out parameter value.
Why not do two queries instead of one multi-query?
This is a respond to a comment by Rick James
Here's modified code:
$sql = "CALL ".$storedProcedure."(".$columns."); ";
$sql2 = "SELECT #out AS `out`;";
if($conn->query($sql)) {
if($rs = $conn->query($sql2)) {
$row = $rs->fetch_assoc();
$odp = $row;
if (!empty($odp)) {
$response['success'] = 1;
$response['message'] = $odp;
echo json_encode($response);
$rs->free();
} else {
$response['success'] = 0;
$response['message'] = mysqli_error($conn);
echo json_encode($response);
}
} else {
$response['success'] = 0;
$response['message'] = mysqli_error($conn);
echo json_encode($response);
}
And mysql error while executing the script:
{"success":0,"message":"Commands out of sync; you can't run this command now"}
The problem is when executing the second query.
I finally got it. All I had to do is to store results in every while loop iteration.
Here's how it looks now.
$sql = "CALL someProcedure('qr_code', #out); ";
$sql .= "SELECT #out AS `out`;";
if($conn->multi_query($sql)) {
while ($conn->more_results()) {
$rs = $conn->store_result();
$conn->next_result();
}
$rs = $conn->store_result();
$row = $rs->fetch_assoc();
$odp = $row['out'];
if (!empty($rs)) {
$response['success'] = 1;
$response['message'] = $odp;
echo json_encode($response);
$rs->free();
} else {
$response['success'] = 0;
$response['message'] = mysqli_error($conn);
echo json_encode($response);
}
The data is not inserting into another table, here's the code below :
if (isset($_POST))
{
$job = $_POST['jobtitle'];
$dur = $_POST['duration'];
$deg = $_POST['requireddegree'];
$exp = $_POST['experiance'];
$sal = $_POST['salary'];
$mark = $_POST['marks'];
if ( !empty($job) && !empty($dur) && !empty($deg) && !empty($exp) && !empty($sal) && !empty($mark))
{
$dur = mysql_real_escape_string($dur);
$deg= mysql_real_escape_string($deg);
$exp = mysql_real_escape_string($exp);
$sal = mysql_real_escape_string($sal);
$mark = mysql_real_escape_string($mark);
$job = mysql_real_escape_string($job);
$query="INSERT INTO jobposting (duration,degree,experiance,salary,marks,Jobtitle) VALUES ('".$dur."','".$deg."','".$exp."','".$sal."','".$mark."','".$job."') ";
if ($query_run= mysql_query($query))
{
header('location : Main.html');
}
else
{
echo ' Data not Inserted! ';
}
}
With this it gives me server error or there was an error in CGI script.But when I write the variables in this form '$dur' instead of '".$dur." then the else conditon runs after insert query and displays data is not inserted.
However, i have written the same logic while inserting data in my another table and it inserts successfully.But there I put '$dur'.
I can't find the problem.Will be glad for your suggestions :)
I can't seem to find any other error by seeing this code expect for
$query="INSERT INTO jobposting (duration,degree,experiance,salary,marks,Jobtitle) VALUES ('$dur','$deg','$exp','$sal','$mark','$job') ";
//Use ".$job." only for stuff like '".md5($_POST['password'])."' otherwise this creates problem some times.
// Adding this always helps
if(!mysqli_query($con,$query))
{
die('error'.mysqli_error($con));
}
// in $con = $con=mysqli_connect("localhost","root","");
else
{
if ($query_run= mysql_query($query))
{
header('location : Main.html');
}
else
{
echo ' Data not Inserted! ';
}
}
I think by making these changes and making sure that your db name and other basic stuff are correct then you should be good to go otherwise, specify your exact error.
We'll get to the point...
I have a simple form (2 of them) that relies off the previous filled out.
The intention of these forms are to sign, post to db, validate email. After the user validates their email their permission will change to be able to see the next form.
These forms work great, and everything is functional in exception to this last bit.
I am having difficulty with the form applying the values to the db table when there is existing user.
What I would like to do is only have it update the keys for that user where users session-ed API key =$API AND form_ica_initials is NULL in the roster table. If it does then will INSERT INTO
Here is what I have cleaned up. (originally wrote for the first phase of the forms to be filled out, trying to tweak to work for last half of forms)
if (empty($_POST['initials'])) { $error[] = 'You must enter your initials in every box below.'; }
else { $initials = $_POST['initials']; }
$API = $_SESSION['API'];
if (empty($error)) {
$query_verify_form = "SELECT * FROM roster WHERE API ='$API'";
$result_verify_form = mysqli_query($dbc, $query_verify_form);
if (!$result_verify_form) {
echo ' Database Error Occured ';
}
if (mysqli_num_rows($result_verify_form) == 0) {
$form_icaauth = md5(uniqid(rand(), true));
error_reporting(E_ALL);
$query_insert_user = "UPDATE `roster`
(
`fullname`, `form_ica_initials`, `form_icaauth`,`form_ica_ip`
)
VALUES (
'$fullname', '$initials', '$form_icaauth','$DOCSIGNEDBYIP'
)
";
$result_insert_user = mysqli_query($dbc, $query_insert_user);
if (!$result_insert_user) {
echo 'Query Failed ';
}
if (mysqli_affected_rows($dbc) == 1) {
...
echo '<br><center><div class="success">...</div>';
}
else {
echo '<center><div class="error">...</div></center>';
}
}
else {
echo '<center><div class="warning" >...</div></center>';
}
}
else {
echo '<center><div class="info"> <ol>';
foreach ($error as $key => $values) {
echo ' <li>' . $values . '</li>';
}
echo '</ol></div></center>';
}
mysqli_close($dbc); //Close the DB Connection
}
If I change the if (mysqli_num_rows($result_verify_form) == 0) { to ==1 It will post the values to the table by creating a new record, and not update the existing users fields as specified. However, by doing that it will circumvent the errors that I have structured.
I know my way around PHP a bit... but having difficultly with this one
I was able to get it to work with the following.
if (empty($error)) {
$query_verify_form = "SELECT * FROM roster WHERE API='$API' AND form_ica_initials IS NULL";
$result_verify_form = mysqli_query($dbc, $query_verify_form);
if (mysqli_num_rows($result_verify_form) == 1) {
$form_icaauth = md5(uniqid(rand(), true));
error_reporting(E_ALL);
$query_insert_user = "UPDATE roster SET fullname='$fullname', form_ica_initials='$initials', API='$API', form_icaauth='$form_icaauth', form_ica_ip='$DOCSIGNEDBYIP'";
$result_insert_user = mysqli_query($dbc, $query_insert_user);
if (!$result_insert_user) {
echo '<center><div class="error">Query Failed </div></center>';
}
First I had to change if (mysqli_num_rows($result_verify_form) == 1) from 0 to 1 to return Yes we've found that record.
I then had to change the INSERT INTO ... VALUES to UPDATE ... SET. I added also added AND form_ica_initials IS NULL to validate that the user hasn't completed this form yet. IF they have, then we'd prompt with a message to check their email. If they havent then we'd run the UPDATE
I'm currently struggling with a page that allows a user to complete one of two options. They can either update an existing item in the SQL database or they can delete it. When the customer deletes an option everything runs perfectly well, however whenever a customer updated an item it displays the Query failed statement from the delete function before applying the update.
It seems obvious to me that the problem must be in my IF statement and that the DeleteButton function isn't exiting if the $deleteno variable isn't set. Any help would be appreciated. Excuse the horribly messy code PHP isn't a language I am familiar with. (I have not included the connect information for privacy reasons)
function DeleteButton(){
#mysqli_select_db($con , $sql_db);
//Checks if connection is successful
if(!$con){
echo"<p>Database connection failure</p>";
} else {
if(isset($_POST["deleteID"])) {
$deleteno = $_POST["deleteID"];
}
if(!isset($deleteno)) {
$sql = "delete from orders where orderID = $deleteno;";
$result = #mysqli_query($con,$sql);
if((!$result)) {
echo "<p>Query failed please enter a valid ID </p>";
} else {
echo "<p>Order $deleteno succesfully deleted</p>";
unset($deleteno);
}
}
}
}
That is the code for the delete button and the following code is for the UpdateButton minus the connection information (which works fine).
if(isset($_POST["updateID"])) {
$updateno = $_POST["updateID"];
}
if(isset($_POST["updatestatus"])) {
if($_POST["updatestatus"] == "Fulfilled") {
$updatestatus = "Fulfilled";
} elseif ($_POST["updatestatus"] == "Paid") {
$updatestatus = "Paid";
}
}
if(isset($updateno) && isset($updatestatus)) {
$sql ="update orders set orderstatus='$updatestatus' where orderID=$updateno;";
$result = #mysqli_query($con,$sql);
if(!$result) {
echo "<p>Query failed please enter a valid ID</p>";
} else {
echo "<p>Order: $updateno succesfully updated!</p>";
}
}
Once again these are incomplete functions as I have omitted the connection sections.
if(!isset($deleteno)) {
$sql = "delete from orders where orderID = $deleteno;";
Are you sure you want to execute that block if $deleteno is NOT set?
P.S. You shouldn't rely on $_POST['deleteId'] being a number. Please read about SQL injections, how to avoid them and also about using prepared statements.
I've update your code, but you need to write cleaner code ( spaces, indents, etc ) this won't only help you to learn but to find your errors easily.
<?php
function DeleteButton()
{
#mysqli_select_db($con , $sql_db);
/*
Checks if connection is successful
*/
if(!$con){
echo"<p>Database connection failure</p>";
} else {
/*
Check if $_POST["deleteID"] exists, is not empty and it is numeric.
*/
if(isset($_POST["deleteID"]) && ! empty($_POST["deleteID"]) && ctype_digit(empty($_POST["deleteID"]))
$deleteno = $_POST["deleteID"];
$sql = "delete from orders where orderID='$deleteno'";
$result = #mysqli_query($con,$sql);
if(!$result){
echo "<p>Query failed please enter a valid ID </p>"
} else {
echo "<p>Order $deleteno succesfully deleted</p>";
unset($deleteno);
}
} else {
echo "<p>Please enter a valid ID </p>" ;
}
}
}
/*
Part 2:
===========================================================================
Check if $_POST["updateID"] exists, is not empty and it is numeric.
Check if $_POST["updatestatus"] exists, is not empty and equal to Paid or Fullfilled
*/
if( isset($_POST["updateID"]) &&
! empty($_POST["updateID"]) &&
ctype_digit(empty($_POST["updateID"]) &&
isset($_POST["updatestatus"]) &&
! empty($_POST["updatestatus"]) &&
( $_POST["updatestatus"] == "Fulfilled" || $_POST["updatestatus"] == "Paid" ) )
{
$updateno = $_POST["updateID"];
$updatestatus = $_POST["updatestatus"];
$sql ="update orders set orderstatus='$updatestatus' where orderID=$updateno;";
$result = #mysqli_query($con,$sql);
if(!$result){
echo "<p>Query failed please enter a valid ID</p>";
} else {
echo "<p>Order: $updateno succesfully updated!</p>";
}
}
There is an error in MySQL Syntax
$sql = "delete from orders where orderID = $deleteno;";
$deleteno after orderID must be inside single quotes.
change it to this $sql = "delete from orders where orderID = '$deleteno';";