I have an Array named $items that looks like the following:
array (
0 => '{"id":"1", "qty":"1", "price":"12.51"}',
1 => '{"id":"2", "qty":"2", "price":"25.02"}',
)
I'm trying to build a mySQL INSERT statement which includes the data in $items as follows:
$sql = 'INSERT INTO vals (id, items, timestamp)
VALUES (' . $id . ', "' . json_encode($items) . '", "' . date('Y-m-d H:i:s') . '")';
However, the INSERT into mySQL is failing due to json_encode() adding double-quotes around the Array Elements:
INSERT INTO
vals
(
id,
items,
timestamp
)
VALUES
(
1,
"[
"{\"id\":\"1\", \"qty\":\"1\", \"price\":\"12.51\"}",
"{\"id\":\"2\", \"qty\":\"2\", \"price\":\"25.02\"}"
]",
"2015-11-26 20:31:02"
)
It's the double-quotes before/after the curly-braces "{ ...data... }" that are the problem.
Is there a way to convert the Array to a String that will elimnate these extra quotes?
Thanks much for any guidance!
EDIT:
From the examples below, I'm trying to use mysqli prepared statements.
I'm executing the following:
$stmt->bind_param("i", (int) $id)
and am getting this error:
ERROR: exception 'Symfony\Component\Debug\Exception\FatalErrorException'
with message 'Call to a member function bind_param() on a non-object'
I didn't get an error executing the following:
$stmt = $mysqli->prepare($sql)
so I'm thinking $stmt should be okay to call bind_param() on.
I looked at the PHP docs and don't believe I need to do anything else with $stmt. Does anyone see something I'm missing?
You either need to escape your json_encode'd string for use in the query, or use a prepared statement and bind the value to a parameter.
Your quotes around the individual array items are actually required as your individual array items are in fact strings... so all you need is to escape the entire value...
So either:
$sql = 'INSERT INTO vals (id, items, timestamp)
VALUES (' . $id . ', "' . mysql_real_escape_string(json_encode($items)) . '", "' . date('Y-m-d H:i:s') . '")';
or a better way of doing this:
$json = json_encode($items);
$sql = 'INSERT INTO vals (id, items, timestamp) VALUES (?, ?, ?)';
/* Prepared statement, stage 1: prepare */
if (!($stmt = $mysqli->prepare($sql))) {
echo "Prepare failed: (" . $mysqli->errno . ") " . $mysqli->error;
}
/* Prepared statement, stage 2: bind and execute */
if (!$stmt->bind_param("i", $id)) {
echo "Binding parameters failed: (" . $stmt->errno . ") " . $stmt->error;
}
if (!$stmt->bind_param("s", $json)) {
echo "Binding parameters failed: (" . $stmt->errno . ") " . $stmt->error;
}
if (!$stmt->bind_param("s", date('Y-m-d H:i:s'))) {
echo "Binding parameters failed: (" . $stmt->errno . ") " . $stmt->error;
}
if (!$stmt->execute()) {
echo "Execute failed: (" . $stmt->errno . ") " . $stmt->error;
}
EDIT
Also, #JoshJ is correct about your values being JSON strings... re-encoding a Json string will double escape everything. If this is not your intended use (i.e. retrieving the values as strings), then you should in fact try decoding each string to an associative array using:
foreach ($items as $key => $item) {
$items[$key] = json_decode($item, true);
}
This will give you an $items value of:
[
0 => [
"id" => "1",
"qty" => "1",
"price" => :"12.51"
],
1 => [
"id" => "2",
"qty" => "2",
"price" => "25.02"
]
]
There are other options of course for treating numeric string s as numbers in the output which you may also want to investigate...
See: http://php.net/manual/en/function.json-decode.php
EDIT 2
In Symfony, you may need to use the PDO bindParam() function syntax.
$sql = 'INSERT INTO vals (id, items, timestamp) VALUES (:id, :items, :datetime)';
$stmt = $pdoconnection->prepare($sql);
$stmt->bindParam(':id', $id);
etc...
See: http://php.net/manual/en/pdostatement.bindparam.php and http://php.net/manual/en/class.pdostatement.php
Note that from PHP 5.3.3 on, there's a flag for auto-converting numbers, while options parameter was added since PHP 5.3.0:
$arr = array( 'row_id' => '1', 'name' => 'George' );
echo json_encode( $arr, JSON_NUMERIC_CHECK ); // {"row_id":1,"name":"George"}
The simplest and safest answer for this is to switch to prepared statements. Documentation for that can be found here
If that doesnt work right with you, you can use simple built in methods for escaping characters.
If you are using MYSQL, you can make use of the mysql_real_escape_string function — It escapes special characters in a string for use in an SQL statement.
Be carefull, as this extension (MYSQL) was deprecated in PHP 5.5.0, and it was removed in PHP 7.0.0.
for MYSQLi, use mysqli::real_escape_string
A use case for your scenarial, i assume you are using the mysql extension:
$sql = 'INSERT INTO vals (id, items, timestamp) VALUES (' . $id . ', "' .mysql_real_escape_string(json_encode($items)). '", "' . date('Y-m-d H:i:s') . '")';
You can even use php's addslaches() function documented here. What it does is that it:
returns a string with backslashes before characters that need to be escaped. These characters are single quote ('), double quote ("), backslash () and NUL (the NULL byte).
Related
I have data that should be escaped inside a JSON formatted string, so I'm using PDO's named parameters and PDO::Prepare to bind them.
Because JSON with it's apostrophes causes errors in the MySQL query, i have to use single quotes around it - although this causes the PDO::Prepare to ignore the named parameters inside the JSON, so it fails with SQLSTATE[HY093]: Invalid parameter number: parameter was not defined.
Any ideas how to work around this?
function send($_data) {
global $_SESSION;
global $dbApi;
#These are temporary debug variables:
$_SESSION['room_id'] = 1;
$id = 124;
$json = '"' . $id . '": {"user_id": ":email","data": ":data"}';
$query = "UPDATE `room_index` " .
"SET `data` = JSON_ARRAY_INSERT(`data`, '$[0]', '" . $json . "') " .
"WHERE `id` = :room_id";
$dbApi->query($query, array(':email' => $_SESSION['email'],
':data' => $_data,
':room_id' => $_SESSION['room_id']));
}
To explain the code a bit, :email ($_SESSION['email']) doesn't have to be a parameter, but it's cleaner this way. The main issue is :data ($_data) - that is user input straight from a textarea via JS.
$dbApi is a class with a proper query function, that looks like this:
function query($_query, $_params = array()) {
global $_DB; # <- Database connection object
$query = $_DB->prepare($_query);
if (! $query)
echo $_DB->errorInfo();
try {
$query->execute($_params);
} catch (PDOException $e) {
die( $e->getMessage() );
}
return $query;
}
There are 2 issues with the code.
1. JSON_INSERT is more appropriate.
As I'm inserting a named object into another object (the top level document), the JSON_INSERT offers such a syntax straight away
2. Using JSON_OBJECT instead of manually writing the JSON syntax
As my main issue was, that PDO doesn't replace single, or double quoted parameters, the solution was using JSON_OBJECT, which doesn't require double quotes as they are automatically generated later (my assumption) - but after PDO replaces the variables and also places single quotes around them.
New, tested code outputting valid JSON:
#Temporary, to avoid other unrelated issues
$_SESSION['room_id'] = 1;
$_SESSION['email'] = 'email#email.com';
$id = 123;
$json = 'JSON_OBJECT("user_id", :email, "data", :data)';
$query = "UPDATE `room_index` " .
"SET `data` = JSON_INSERT(`data`, :id, $json) " .
"WHERE `id` = :room_id";
$dbApi->query($query, [':id' => "$." . $id,
':email' => $_SESSION['email'],
':data' => $_data,
':room_id' => $_SESSION['room_id']]);
Hi i been trying to inserting array's into MySql database
The problem i am having is that i have different datatypes and sometime data can be a 0 value, having () curly brackets, percentage value with % sign. I would like to know a way use some already built php function that can deal with this issues.
So here is what i have done so far:
$t = array('country_code' => $data->country_code,
'Name' => $data->Name,
'money' => $data->money,
'chanceToDie' => $data->death,
'age' => $cb->age)
/* FORMAT EXAMPLE
country_code = Africa (AF)
name = jack
chanceToDie = 5.5
age = 62
*/
$columns = implode(", ",array_keys($t));
//Tried
$values = implode(", ",array_values($t)); //Dont work
$values = "'".implode("', '",array_values($t))."'"; //Dont work
$sql = "INSERT INTO table ($columns) VALUES ($values)";
You need to quote each individual value and use array_values() instead of array_keys():
$values = '"' . implode('", "', array_values($t)) . '"';
However, this leaves you with an sql injection problem so you should really use a prepared statement.
In PDO you could use something like (assuming you control the keys and they are safe to use):
$values = ':' . implode(', :', array_keys($t));
// generates: ... VALUES(:country_code, :Name, :money, // etc
Now you can prepare and execute your query using the array to bind the values to the placeholders. See for example http://php.net/manual/en/pdo.prepared-statements.php (the 6th example).
Try to use the advantage of PDO prepared queries - it is more safe and convinient.
Your code may look like this:
$col_names = array_keys($t);
// filter column names before inserting to sql to prevent sql injection
array_filter($col_names, function($v){return preg_relace("#\W#", "_", $v);});
// generate placeholders list: ?,?,?,?
$placeholders = implode(',', array_fill(0, count(t), "?"));
$values = array_values($t);
$q = $pdo->prepare('insert into (' . implode(",", $col_names) . ') values (' . $placeholders . ')');
$q->execute($values);
PDO will deal with data types and correctly replace every placeholder with the corresponding value.
I am trying to query large amounts of data to a server, here is my code for that:
$queryString = "";
$connect = mysqli_connect("localhost", "username", "password", "database");
$loopLength = 20;
$currentGroup = 1;
$currentLoopAmount = 0;
$queryAmount = 5;
for($v = 0; $v < ceil($loopLength / $queryAmount); $v++){
//echo "Looping Main: " . $v . "<br>";
//echo $loopLength - (($currentGroup - 1) * 10) . "<br>";
if($loopLength - (($currentGroup - 1) * $queryAmount) >= $queryAmount){
$currentLoopAmount = $queryAmount;
}
else{
$currentLoopAmount = $loopLength - (($currentGroup - 1) * $queryAmount);
}
//echo $currentLoopAmount;
$queryString = "";
for($q = (($currentGroup - 1) * $queryAmount); $q < $currentLoopAmount + (($currentGroup - 1) * $queryAmount); $q++){
//echo " Looping Sub: " . $q . "<br>";
$tempVariable = grabPageData($URLs[$q], $q);
$queryString .= $tempVariable;
if($q < $loopLength-1){
$queryString .= ",";
}
else{
$queryString .= ";";
}
}
echo $queryString;
$query = "INSERT INTO PublicNoticesTable (url, county, paperco, date, notice, id) VALUES " . $queryString;
$result = mysqli_query($connect, $query);
if($result){
echo "Success";
}
else{
echo "Failed : " . mysqli_error($connect) . "<br>";
}
$currentGroup += 1;
}
The $loopLength variable is dynamic and can be in the thousands or potentially hundred thousands. I designed this function to divide that massive number into a batch of smaller queries as I couldn't upload all the data at one time on my shared hosting service through GoDaddy. The $queryAmount variable represents how big the smaller queries are.
Here is an example of one of the value sets that gets inserted into the table:
It is the data from a public notice that my code retrieved in the grabPageData() function.
('http://www.publicnoticeads.com/az/search/view.asp?T=PN&id=37/7292017_24266919.htm','Pima','Green Valley News and Sun','2017/07/30',' ___________________________ARIZONA SUPERIOR COURT, PIMA COUNTYIn the Matter of the Estate of:JOSEPH T, DILLARD, SR.,Deceased.DOB: 10/09/1931No. PB20170865NOTICE TO CREDITORS(FOR PUBLICATION)NOTICE IS HEREBY GIVEN that DANA ANN DILLARD CALL has been appointed Personal Representative of this Estate. All persons having claims against the Estate are required to present their claimswithin four months after the date of the firat publication of this notice or the claims will be forever barred. Claims must be presented by delivering or mailing a written statement of the claim to the Personal Representative at the Law Offices of Michael W. Murray, 257 North Stone Avenue, Tucson, Arizona 85701.DATED this 17th day of July, 2017./S/ Micahel W. MurrayAttorney for the Personal RepresentativePub: Green Valley News & SunDate: July 23, 30, August 6, 2017 Public Notice ID: 24266919',' 24266919'),
To attain this data, I run it through a function that crawls the page and grabs it. Then I put the webpage html code through this function:
function cleanData($data){
$data = strip_tags($data);
//$data = preg_replace("/[^a-zA-Z0-9]/", "", $data);
//$data = mysql_real_escape_string($data);
return $data;
}
Which gives me the content without tags as you see above. Here's the problem.
The function executes and everything seems just dandy. Then the function (depending on the $queryAmount variable which I don't keep over 10 for problem's sake) outputs, as you can see it would in the function something like...
Failed : You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '' at line 1
The weird part is that when I have large amounts of data like say the $loopLength variable is like 116. The result will output, "Failed : (error)Failed : (Error)Fai...(Error)Success. So it's only actually querying the last set of data??? Not sure.
I am not sure how to fix this and I want a fresh eye. Can somebody help me please. I have been working on this problem for... several hours trying to find solution.
Sorry for making this question a pain in the butt :(
EDIT:
I changed the code from previously to use mysql prepared statements and what not... See below:
$grabDataResults = [
"url" => "",
"county" => "",
"paperco" => "",
"date" => "",
"notice" => "",
"id" => "",
];
$connect = mysqli_connect("localhost", "bwt_admin", "Thebeast1398", "NewCoDatabase");
if($stmt = mysqli_prepare($connect, "INSERT INTO PublicNoticesTable (url, county, paperco, date, notice, id) VALUES (?, ?, ?, ?, ?, ?)")){
mysqli_stmt_bind_param($stmt, 'ssssss', $grabDataResults["url"], $grabDataResults["county"], $grabDataResults["paperco"], $grabDataResults["date"], $grabDataResults["notice"], $grabDataResults["id"]);
$loopLength = 1;
for($v = 0; $v < $loopLength; $v++){
$grabDataResults = grabPageData($URLs[$v], $v);
mysqli_stmt_execute($stmt);
printf("%d Row inserted.\n", mysqli_stmt_affected_rows($stmt));
printf("Error:\n", mysqli_stmt_error($stmt));
echo "(" . $grabDataResults["url"] . "," . $grabDataResults["county"] . "," . $grabDataResults["paperco"] . "," . $grabDataResults["date"] . "," . $grabDataResults["notice"] . "," . $grabDataResults["id"] . ")";
}
mysqli_stmt_close($stmt);
mysqli_close($connect);
}
Unfortunately this is what I get from the output:
1 Row inserted. 0 Error:
No error actually prints out and the row is inserted. However when I navigate to my database, and look at the values that have been stored.. They are all empty. The echo statement outputs this:
(http://www.publicnoticeads.com/az/search/view.asp?T=PN&id=31/7292017_24266963.htm,Yuma,Sun (Yuma), The,2017/07/30,, 24266963)
So I know that all of the variables contain something except for the $notice variable which gets destroyed by my cleanData() function for some reason.
You need to bind the data after fetching it and before executing it...
$loopLength = 1;
for($v = 0; $v < $loopLength; $v++){
$grabDataResults = grabPageData($URLs[$v], $v);
mysqli_stmt_bind_param($stmt, 'ssssss', $grabDataResults["url"],
$grabDataResults["county"], $grabDataResults["paperco"],
$grabDataResults["date"], $grabDataResults["notice"],
$grabDataResults["id"]);
mysqli_stmt_execute($stmt);
printf("%d Row inserted.\n", mysqli_stmt_affected_rows($stmt));
printf("Error:\n", mysqli_stmt_error($stmt));
echo "(" . $grabDataResults["url"] . "," . $grabDataResults["county"] . "," . $grabDataResults["paperco"] . "," . $grabDataResults["date"] . "," . $grabDataResults["notice"] . "," . $grabDataResults["id"] . ")";
}
You code is correct. Just you need to add () arround querystring
Akso you need to remove ; from query string end. SO remove following else condition
else{
$queryString .= ";";
}
change you query like :
$query = "INSERT INTO PublicNoticesTable (url, county, paperco, date, notice, id) VALUES (" . $queryString . ")";
Also it advisable to use prepared statements to prevent from sql injections
The main error I can see on your query, are the query itself. You're using a INSERT INTO with separated fields and values. But you forget to use the pharentesis on values.
Remember, the use of INSERT INTO are as follows:
First option:
INSERT INTO table field1 = value1, field2 = value2;
Second option:
INSERT INTO table (field1, field2) VALUES (value1, value2);
Also, remember to escape every field and value for avoid more errors: Example:
First option:
INSERT INTO `table` `field1` = 'value1', `field2` = 'value2';
Second option:
INSERT INTO `table` (`field1`, `field2`) VALUES ('value1', 'value2');
If you're using mysqli driver, for more security, you can use prepared statements, to get your values automatically escaped. In that case, the syntax of the query are as follows:
First option:
INSERT INTO `table` `field1` = ?, `field2` = ?;
Second option:
INSERT INTO `table` (`field1`, `field2`) VALUES (?, ?);
Also, instead of using mysqli_query(), you need to use mysqli_prepare(), mysqli_bind_param() and mysqli_execute(). You can check more data about they syntax here: http://php.net/manual/en/mysqli.prepare.php
At least, you can use mysqli_real_escape_string() function for getting your input properly escaped and checked. You can check documentation of this function here: http://php.net/manual/en/mysqli.real-escape-string.php
I am trying to create a custom MVC framework in that I have a function which takes in array and triggers insert queries.
function insertFromArray($array, $table) {
foreach ($array as $key => $value) {
$fields[] = "`" . $key . "`";
$values[] = "'" . $value . "'";
}
$sql = "INSERT INTO `$table` (" . implode(",", $fields) . ") VALUES (" . implode(",", $values) . ")";
mysql_query($sql);
return (mysql_error()) ? mysql_error() : mysql_insert_id();
}
But when I get some mysql function instead of a string or integer my function does not work say for a filed name 'dateTime' I get value = NOW() in that case my query would be
INSERT INTO `$table` (`field1`, `field2`, `dateTime`) VALUES ('value1','value2','NOW()')
And this would throw error I want some generic solution so that my queries would run in either case if it is a string, number or if it is any mysql function.
I tried creating an array of all the functions and started with comparisons like if the starting of the $value matches any function from the array of mysql functions but I do not think that is a good way to do it, is there any better way to do that
$values[] = "'" . $value . "'";
The above line treats all values as strings and this causes the issue. Do not treat all values as strings and enclose them by single quotes.
You could even make the value part an array itself with 2 elements: one for the value and one for the type of the value (string, numeric, function, etc.) and would treat ech of them differently while creating the insert statement.
And pls use mysql or PDO instead of php's old mysql extension.
You could use an array to store your SQL keywords/function names. Then in your query string, you test to see if the current value is an SQL word or not. If it is, don't quote it, otherwise, you quote it.
Here is an example:
function is_sql_term($term)
{
$sql_terms = array('NOW()', 'DATE()', ...);
return in_array($term, $sql_terms) || preg_match("/\(.*\)/", $term);
}
Then you test as follows:
$values[] = is_sql_term($value) ? $value : "'" . $value . "'";
When inserting (or updating) a record in mysql using prepared statements and binding of values, I ran into a problem trying to insert the word child’s which contains the extended ascii character 145 (right single quotation mark). All data after the "d" is truncated. This is true for inserting any of the MS Word odd characters. My table is encoded as utf8_general_ci and I'm utf-8 all the way through in my code. The field is of type "text."
Of course I can escape them or do a str_replace() to remove them (which is what I am choosing to do), but I'd like to understand why I can't insert them directly since I always thought prepared statements handled this.
Sample insert code - incomplete, but it shows the essential
$q = "INSERT INTO mytable (userid, title, descr) VALUES (?,?,?)";
if (!($stmt = $this->mysqli->prepare($q))) {
error_log("You have a problem...");
return false;
}
/* bind parameters for markers */
$stmt->bind_param("iss", $userid, $title, $descr);
/* execute query */
$stmt->execute();
if ($stmt->errno) {
$e = new Exception;
error_log("\r\nSql Error: " . $q . ' Sql error #: ' . $stmt->errno . ' - ' . $stmt->error . "\r\n" . $e->getTraceAsString());
return false;
} else {
$lastinsertedid = $this->mysqli->insert_id;
/* close statement */
$stmt->close();
return $lastinsertedid;
}
This should not be an issue, binding the param as a string should escape any quotes as needed.
Try this to see if you can see what the error is.
$stmt = $this->mysqli->prepare($q);
if (!is_object($stmt))
error_log("You have a problem...");
var_dump($this->mysqli);
}
P.S. Don't forget $stmt->close() after $stmt->execute() as you will run into out of sync issues when attempting multiple prepared statements
I can offer some tips on syntax clean up too, take your updated question:
$q = "INSERT INTO mytable (userid, title, descr) VALUES (?,?,?)";
$stmt = $this->mysql->prepare($q);
if (!is_object($stmt))
error_log("You have a problem...");
var_dump($this->mysqli);
}
/* bind parameters for markers */
$stmt->bind_param("iss", $userid, $title, $descr);
/* execute query */
$stmt->execute();
if ($stmt->errno) {
$stmt->close(); // Still need to close here
$e = new Exception;
error_log("\r\nSql Error: " . $q . ' Sql error #: ' . $stmt->errno . ' - ' . $stmt->error . "\r\n" . $e->getTraceAsString());
return false;
}
/* close statement */
$stmt->close();
return $this->mysqli->insert_id;
The encoding for ’ was not utf8. That is the only problem. Your code may be utf8 throughout, but the data was not.
The fact that child’s truncated after the d is the symptom non-utf8 bytes being fed into otherwise good code.
Find out what application or code is generating the ’, either figure out what encoding it is generating or figure out whether it can be made to generate utf8. In the former case, we need to change one setting, in the other case, no code change need be made.
HEX('’') is E282AC in utf8.