I have a function that collects data and puts it into an object. It then converts this object into a json string, and places it into a table. It takes about 30 seconds to do it.
Now, this json string is fairly big (roughly 36mb or so).
But when I execute the query, nothing gets put into the table, and I don't get any errors.
So I did some debugging, and the code now looks like this:
function __construct($total, $allActions, $employees, $branches, $companies, $departments, $lastUpdated, $update = false)
{
echo "Made it to constructor. Update: ".json_encode($update);
global $conn;
$this->Total = $total;
$this->AllActions = $allActions;
$this->Employees = $employees;
$this->Branches = $branches;
$this->Companies = $companies;
$this->Departments = $departments;
$this->LastUpdated = $lastUpdated;
if($update) {
try {
$query = $conn->prepare("INSERT INTO actionplansummarydata_new (json, last_updated) VALUES (?, ?)");
echo "Prepared query.";
if($query->bind_param('ss', json_encode($this), $this->LastUpdated->format("Y-m-d H:i:s"))) {
echo "Bound parameters.";
if ($query->execute()) {
echo "Executed query.";
} else {
echo "Error inserting summary: " . $query->error;
}
} else {
echo "Error binding query: " . $query->error;
}
} catch(Exception $ex) {
echo "Exception: ".$ex->getMessage();
}
}
}
Now, when this is executed, this is the response I get on the page:
Made it to constructor. Update: truePrepared query.Bound parameters.
But I have an if statement here:
if ($query->execute()) {
echo "Executed query.";
} else {
echo "Error inserting summary: " . $query->error;
}
Neither of these gets echoed to the page, nor do I get any exceptions.
I'm completely baffled.
The server is MariaDB 10.4, the json field in the database is longtext, so it should be able to store it.
If your error reporting is at E_ALL, you should not be facing a silent fail with the query execution... In any case, first of all, some steps in troubleshooting the point of failure.
Move json_encode($this) out of the bind, assign to a variable, and var_dump to see what you're actually feeding the binder. So too with $this->LastUpdated->format; presuming it's a DateTime object, but who knows. Easier to debug when fail-points are outside conditional statements.
Can you confirm whether it works as expected if you insert strings of dummy data instead? E.g.:
if($query->bind_param('ss', '{}', '2020-05-19 01:01:01')); {
One problem here is with the fact that bind_param binds variables by reference, not by value. Therefore, you can't feed it with functions. You have to assign the values to variables and only then bind them (because a function doesn't return a usable reference). Meaning, change to:
$json = json_encode($this);
$updated = $this->LastUpdated->format("Y-m-d H:i:s");
if($query->bind_param('ss', $json, $updated)) {
You can assign the bind result (true/false) into a variable, then var_dump. (We'd rather not var_dump from inside the conditional statement, right?) The bind would fail === false for reasons above. Likewise, var_dump the prepared query to ensure it succeeds, tracing each step.
A fatal error might be triggered by the database engine acting up, quite possible with the size of your insert query. You may want to look up MariaDB error logs for more clues, in case you're hitting a limit somewhere. I'd guess with max_allowed_packet, default 16MB, you're well over it. (Also see mysqli_stmt::send_long_data for sending long data in blocks.)
In other issues, json is a reserved word in MySQL 8.0 (not listed as such for Maria 10.4?!). You may want to rename the column to ensure it doesn't become a problem (even if it works here).
As far as how this scenario is possible, what you describe (no output from the if/else) only makes sense if there's a fatal error triggered by the $query->execute() that's never displayed. Unless there's a bug in PHP error reporting with certain MySQL failures. Do triple-check your PHP error reporting configuration, including anywhere in your code/.htaccess that might override php.ini.
Check the setting of max_allowed_packet; this may be stopping you. Set it to 1G.
If that does not work, there may be timeout issues, or some other size limitation.
For large text strings, I sometimes like to compress them, and stored the string into a MEDIUMBLOB.
what data type that you are using to store the 36MB of JSON string ? you probably need to change the data type to LONGTEXT that can store up to 4GB of data or JSON data type that can store up to 1GB of data
or
you can store it individually as columns in a table and change it back to JSON on your code when needed
Try to check if prepared
if($query = $conn->prepare("INSERT INTO actionplansummarydata_new (json, last_updated) VALUES (?, ?)")){
$query->bind_param('ss', json_encode($this), $this->LastUpdated->format("Y-m-d H:i:s"));
if($query->execute()){
// code here
}
}
Related
The PHP code I have inserts the HTML form data from the previous page into the database and in the same SQL statement return the PostID back from the inserted data. The PostID column is AUTO_INCREMENTING. I have been researching this problem for a week or two now and have found no significant solutions.
<?php
include("dbconnect.php");
mysql_select_db("astral_database", $con);
session_start();
$username = $_SESSION['username'];
$forumtext = $_POST["forumtext"];
$forumsubject = $_POST["forumsubject"];
$postquery = 'INSERT INTO Forums (Creator, Subject, Content) VALUES ("$username", "$forumsubject", "$forumtext"); SELECT LAST_INSERT_ID()';
$result = mysql_query($postquery, $con);
if (!$con) {
echo "<b>If you are seeing this, please send the information below to astraldevgroup#gmail.com</b><br>Error (331: dbconnect experienced fatal errors while attempting to connect)";
die();
}
if ($username == null) {
echo "<b>If you are seeing this, please send the information below to astraldevgroup#gmail.com</b><br>Error (332: Username was not specified while attempting to send request)";
die();
}
if ($result != null) {
echo "last id: " . $result;
$fhandle = fopen("recentposts.txt", "r+");
$contents = file_get_contents("recentposts.txt");
fwrite($fhandle, json_encode(array("postid" => $result, "creator" => $username, "subject" => $forumsubject, "activity" => time())) . "\n" . $contents);
fclose($fhandle);
mysql_close($con);
header("location: http://astraldevgroup.com/forums");
die();
} else {
die("<b>If you are seeing this, please send the information below to astraldevgroup#gmail.com</b><br>Error (330: Unhandled exception occured while posting forum to website.)<br>");
echo mysql_error();
}
mysql_close($con);
?>
First off, the mysql_query doesn't return anything from the SELECT statement. I haven't found anything that will properly run both the SELECT statement and the INSERT statement in the same query. If I try running them in two different statements, it still doesn't return anything. I tried running the following statement in the SQL console and it ran perfectly fine without errors.
INSERT INTO Forums (Creator, Subject, Content) VALUES ("Admin", "Test forum 15", "This is a forum that should give me the post id."); SELECT LAST_INSERT_ID();
The mysql_query function does not run multiple statements
Reference: http://php.net/manual/en/function.mysql-query.php
mysql_query() sends a unique query (multiple queries are not supported) to the currently active database on the server ...
That's one reason your call to mysql_query isn't returning a resultset.
The most obvious workaround is to not try to run the SELECT in the same query. You could use a call to the mysql_insert_id instead.
Reference: PHP: mysql_insert_id http://php.net/manual/en/function.mysql-insert-id.php
Answers to some of questions you didn't ask:
Yes, your example code is vulnerable to SQL Injection.
Yes, the mysql_ interface has been deprecated for a long time.
Yes, you should being using either PDO or mysqli interfaces instead of the deprecated mysql_ functions.
FOLLOWUP
Re-visiting my answer, looking again at the question, and the example code.
I previously indicated that the code was vulnerable to SQL Injection, because potentially unsafe values are included in the SQL text. And that's what it looked like on a quick review.
But looking at it again, that isn't strictly true, because variable substitution isn't really happening, because the string literal is enclosed in single quotes. Consider what the output from:
$foo = "bar";
echo '$foo';
echo '"$foo"';
Then consider what is assigned to $postquery by this line of code:
$postquery = 'INSERT ... VALUES ("$username", "$forumsubject", "$forumtext")';
Fixing that so that $username is considered to be a reference to a variable, rather than literal characters (to get the value assigned to $username variable incorporated into the SQL text) that would introduce the SQL Injection vulnerability.
Prepared statements with bind placeholders are really not that hard.
$result will never be null. It's either a result handle, or a boolean false. Since you're testing for the wrong value, you'll never see the false that mysql_query() returned to tell you that the query failed.
As others have pointed out, you can NOT issue multiple queries in a single query() call - it's a cheap basic defense against one form of SQL injection attacks in the PHP mysql driver. However, the rest of your code IS vulnerable other forms of injection attacks, so... better start reading: http://bobby-tables.com
Plus, on the logic side, why are you testing for a null username AFTER you try to insert that very same username into the DB? You should be testing/validating those values BEFORE you run the query.
I have a checkbox list and I want to insert each item checked into the database. Since it is a checkbox, I only have one value to pass to the database, but I am also passing a variable (primary key) grabbed from a previous database insertion.
My problem is that I can't get it to post to the database. I have tried multiple ways to do this through posts here and nothing seems to work. I am at a loss. I am also fairly new to PHP and mysqli, so I am sure there are better ways to do what I am trying to do, so please help me out.
The table consists of three columns (school_id, graduate_id, schoolName). The school_id is the AI primary key and graduate_id will be populated with the variable $graduateID grabbed from the previous query.
Here is what I have:
if (isset($_POST['school'])) {
$school=$_POST['school'];
$schoolQuery="";
foreach($school as $value) {
if(!$schoolQuery) {
$schoolQuery="INSERT INTO schoolReunion (graduate_id, schoolName) VALUES ($graduateID, '$value')";
} else {
$schoolQuery .= ", ($graduateID, '$value')";
mysqli_store_result($schoolQuery);
}
}
$schoolQuery .=";";
if (mysqli_multi_query ($schoolQuery)) {
echo "Files have been updated successfully.";
} else {
echo "Error: " . $schoolQuery . "<br/>" . mysqli_error($dbc);
}
}
When the error code prints out, it looks fine, but it just won't post anything and gives me no errors. Can you help me?
#Maximus & #Eko Junaidi Salam, while you may be right, I don't see any evidence that there are issues with the front-end/form.
#jeroen, you are totally correct.
CarR is dealing with a single concatenated query, so mysqli_query is suitable.
I recommend wrapping the user-supplied variable in mysqli_real_escape_string() to cover the query vulnerability.
mysqli_store_result() is to be used on queries that return a result set; mysqli_affected_rows() is the function that yields a measure of success with INSERT/UPDATE/DELETE type queries.
Lastly, you are right to correct MrTechie. I don't think he realized the query concatenation.
Assuming the form is delivering the necessary values, I'll suggest a new query build section:
$schoolQuery="";
foreach($school as $value){
if(!$schoolQuery) {
$schoolQuery="INSERT INTO schoolReunion (graduate_id,schoolName) VALUES ";
}else{
$schoolQuery.=",";
}
$schoolQuery.="('$graduateID','".mysqli_real_escape_string($dbc,$value)."')";
}
$schoolQuery.=";";
$graduateID IS NOT escaped because I am assuming it comes from a safe place; $value IS escaped because it comes from user input.
Now to deliver the built query. Your code has syntax and logic errors, so try this:
if($schoolResult=mysqli_query($dbc,$schoolQuery)){
$total_rows=mysqli_affected_rows($dbc);
echo $total_rows," file",($total_rows!=1?"s have":" has")," been added.";
// if($total_rows<1){echo "Query Logic Error, # $schoolQuery";}
}else{
// echo "Query Syntax Error # $schoolQuery<br>".mysqli_error($dbc);
}
Uncomment the error lines while you are testing, then re-comment or delete for production.
This should sufficiently fix all the issues that are apparent in your snippet of code. Beyond that are the assumed issues...
Since you mention that you are employing a previous query to declare $graduateID, I will assume you have acquired the value via a SELECT query. Be sure to check that you have used
mysqli_free_result($graduateResult); // I assumed this variable name
to avoid any conflicts with subsequent queries. This is sometimes overlooked.
mysqli_multi_query() is best used when you are dealing with queries that are dependent on a prerequisite query. It seems your $schoolQuery is dependent on the success of what I will call "$graduateQuery". If you wish, you could use mysqli_multi_query() to run $graduateQuery then $schoolQuery. To help you with this implementation, I would need to see more of your code. If you wish to go down that road, it might be best to message me directly or start a new post (after you've had a try at it yourself.) Here is half of your job done: Strict Standards: mysqli_next_result() error with mysqli_multi_query
So I'm making a application where I want a json string to display(for now). I call a stored procedure with php(which works) and that stored procedure returns a row.
The problem is that the record/row that is returned has to be a json string. For that I use php:
$verbinding = pg_connect("host=**** port=**** dbname=*** user=******* password=****");
$query = 'SELECT to_json(function_name(parameter))';
$contests = pg_query($query);
if(json_decode($contests)) {
echo 'succeeded';
} else {
echo 'failed';
}
Now my code always returns failed and I don't know why or how can I fix this. Thank you in advance for your help!
This:
$contests = pg_query($query);
json_decode($contests)
is nonsensical. If you replace json_decode with a print statement you will see why.
Per the manual the return value of pg_query is:
A query result resource on success or FALSE on failure.
and if you look at the examples, you will see some basic usage, where you will see that you need to use pg_fetch_row or similar to access the results.
This makes sense given that a query result can have multiple rows and multiple values per row.
The "See also" section of the documentation link above points you at the functions you need.
In this case I suggest using pg_fetch_result (see the sidebar).
While you're at it some basic error handling would be wise.
I also strongly suspect you're failing to use bind parameters, exposing yourself to SQL injection. Use pg_query_params.
Here's a starting point for you. It's untested, and I don't really use PHP, but it should give you an idea where you're going wrong:
$query = 'SELECT to_json(function_name($1))';
$contests = pg_query_params($query, array($somevariable))
or die(pg_last_error());
$json = pg_fetch_result($contests, 0, 0);
if(json_decode($json)) {
# ....
}
Now, seriously, go read the manual.
I'm trying to loop data from a api and then post these values to a MySQL db.
something like this:
$values = json_decode(file_get_contents("my-json-file"));
$SQL = new mysqli(SQL_HOST, SQL_USER, SQL_PASS, DB_NAME);
$SQL->autocommit(FALSE);
foreach($values As $item)
{
$query = "INSERT INTO my_table VALUES ('".$item->value1."', '".$item->value2.";)";
$SQL->query($query);
if(!$SQL->commit())
{
echo "ERROR ON INSERT: [" . $query . "]<hr/>";
}
}
$SQL->close();
Since the loop is too fast, the SQL can't catch up. (Yea!)
I would then need something like this:
foreach($values As $item)
{
/**** STOP/PAUSE LOOP ****/
$query = "INSERT INTO my_table VALUES ('".$item->value1."', '".$item->value2.";");
$SQL->query($query);
if($SQL->commit())
{
/**** START THE LOOP AGAIN ****/
}
else
{
echo "ERROR ON INSERT: [" . $query . "]<hr/>";
}
}
Or how should I do this the right way?
EDIT: It inserts random posts every time.
EDIT 2: This is just example code. It does escape and all that, and yes the semi colon is wrong here but since so many commented on it i will not change it. This was not the problem in the real case.
I tried to run it on another server and there it worked. The problem was fixed by restarting MAMP.
Firstly, your idea that the loop runs too fast for MySQL to keep up is completely totally wrong. The $SQL->query() call will wait for the MySQL to return a response before proceeding, so the loop won't run any faster than MySQL is responding.
Now onto the actual problem.... your query:
$query = "INSERT INTO my_table VALUES ('".$item->value1."', '".$item->value2.";)";
There's a semi-colon in there at the end, after value2 which is invalid. I guess you intended to type a quote mark there? The semi-colon will be causing all your queries to fail and throw errors.
This may be the cause of your problem but you haven't got any error checking in there, so you won't know. Add some error checking to your code after calling the query; even if the query is right, it's still possible to get errors, and your code should check for them. See the examples on this manual page: http://www.php.net/manual/en/mysqli-stmt.error.php
Finally, since you're using the mysqli API, it's worth mentioning that your code would be a lot better and probably more secure if you used prepared statements. See the examples in PHP manual here: http://www.php.net/manual/en/mysqli-stmt.bind-param.php
[EDIT]
Another possible reason your query is failing is that you're not escaping the input values. If any of the input values contains a quote character (or any other character that is illegal in SQL) then the query will fail. In addition, this problem makes your code vulnerable to a SQL injection hacking attack.
You need to escape your input using $SQL->real_escape_string() OR by changing your query to use prepared statements (as recommended above).
Your query is inside the loop, which means that the loop will wait until your query finished executing before it continue, php code is processed in order...
Has #phpalix said, PHP goes in order, and waits for the previous action to finish.
I think you SQL is wrong. Try replacing your INSERT with this:
$query = "INSERT INTO my_table VALUES ('".$item->value1."', '".$item->value2."');";
And don't forget to run at least mysql_real_escape_string for each variable, for security measures.
As many of the answers and comments say, it does not continue until the SQL is done. The problem was in my local apache/mysql server. It was fixed by restarting it. Yes, stupid post.
From PHP, I am calling a SQL stored procedure. The stored procedure checks the database, and either inserts the data if it's new, or modifies data that already exists.
I'm getting an error that one of my parameters is of an incorrect data type. I can't really control this because of where the data is coming from. I am calling the stored procedure hundreds, or thousands of times looping through an array of data, and I only get this error for a couple of records out of the batch.
What is the proper way to handle this error in the stored procedure? If any of the parameters have the incorrect data type, I just want to skip that record and move on to the next record. Each call comes from a foreach loop.
The database is reporting the error and PHP is displaying it. I do not have experience with error handling.
Example PHP Code:
foreach($item_array as $item) {
$id = $item['id'];
$color = $item['color'];
$con = connect()
$query = 'EXECUTE PROCEDURE sp_update_db(:id, :color);'
$params = array(':id' => $id, ':color' => $color);
$stmt = prepare($con, $query);
$result = execute($stmt, $params);
close($con);
}
Running the code I get "Warning: SQL error: [stored procedure a paramater was of the incorrect datatype]".
Looks to me that your proc is not even executed as a result of the wrong parameter being passed; you can only handle this on your php code by catching the exception.
If the proc is in fact being called, but simply failling inside the procedure due to some sort of data type mismatch, you can use
BEGIN TRY
-- your proc statements here
END TRY
BEGIN CATCH
END CATCH
Documentation here.
UPDATE
Since you said that you are calling the proc once per each record that you need to process, you need to catch the error on the PHP side. You can use try/catch blocks on PHP. See here.
Basically, you'd need to have the try/catch block inside your foreach loop enclosing only the part that calls the stored procedure; however, I would just have an if statement before the stored procedure call that makes sure all the parameters that will be passed to the proc are of the expected type and lenght. For example, if stored procedure expectes parameter #a of type int, I would prevent the proc from being called at all if the parameter that is about to be passed is not a number or an empty string.
UPDATE 2
Based on sample php (Warning: I am not a PHP coder), seems like this will work:
try{
$result = execute($stmt, $params);
}
catch (Exception $e) {
}
But again, if you know the data types expected by the proc, why not have an if instead of the try/catch?