I am using an API that returns search results as json. Then, I need to write this to a MYSQL table. I've done this successfully before, but this time the situation is different, and I think that's because of the structure of the resulting array: the key names are dynamic in that, if no data exists for a particular key, the key is not listed in the array. Here is an example vardump of the array:
array
0 =>
array
'title' => string 'Funny but not funny' (length=19)
'body' => string 'by Daniel Doi-yesterday while eating at Curry House...
'url' => string 'http://danieldoi.com/2012/11/20/funny-but-not-funny/'
'source_site_name' => string 'WordPress.com' (length=13)
'source_site_url' => string 'http://www.wordpress.com' (length=24)
'query_topic' => string 'thanksgiving' (length=12)
'query_string' => string 'blogs=on&topic=thanksgiving&output=json' (length=39)
1 =>
array
'title' => string 'Travel Easy this Holiday Season...' (length=34)
'body' => string 'Give yourself a few gifts and get this holiday season off...
'url' => string 'http://facadebeauty.wordpress.com/2012/11/20
'date_published' => string 'Tue, 20 Nov 2012 18:22:35 +0000' (length=31)
'date_published_stamp' => string '1353435755' (length=10)
Notice how the order/inclusion of the keys is subject to change.
My proposed solution was to use the array keys as column names, turning them into a variable to use in a query statement, but this isn't working for me. Here's my attempt:
$jsonString = file_get_contents("http://search-query-URL&output=json");
$array = json_decode($jsonString, true);
// database connection code snipped out here
$table = "results";
foreach($array as $arr_value) {
foreach ($arr_value as $value) {
$colName = key($arr_value);
$colValue = ($value);
$insert="INSERT INTO $table ($colName) VALUES ('$colValue')";
mysql_query($insert) OR die(mysql_error());
next($arr_value);
}
}
Any suggestions on where to look next? Thank you!
UPDATE 11/27:
Here I am trying to adapt David's suggestion. I'm receiving the following error: "Database Connection Error (1110) Column 'title' specified twice on query."
Here's my code as it stands now:
$mysqli = mysqli_connect("localhost");
mysqli_select_db($mysqli, "mydatabase");
foreach ($array as $column) {
foreach ($column as $key => $value) {
$cols[] = $key;
$vals[] = mysqli_real_escape_string($mysqli, $value);
}
}
$colnames = "`".implode("`, `", $cols)."`";
$colvals = "'".implode("', '", $vals)."'";
$mysql = mysqli_query($mysqli, "INSERT INTO $table ($colnames) VALUES ($colvals)") or die('Database Connection Error ('.mysqli_errno($mysqli).') '.mysqli_error($mysqli). " on query: INSERT INTO $table ($colnames) VALUES ($colvals)");
mysqli_close($mysqli);
if ($mysql)
return TRUE;
else return FALSE;
Final Upate - WORKING!
It's working. Here's what we've got:
$mysqli = mysqli_connect("localhost");
mysqli_select_db($mysqli, "mydatabase");
foreach ($array as $column) {
foreach ($column as $key => $value) {
$cols[] = $key;
$vals[] = mysqli_real_escape_string($mysqli, $value);
}
$colnames = "`".implode("`, `", $cols)."`";
$colvals = "'".implode("', '", $vals)."'";
$mysql = mysqli_query($mysqli, "INSERT INTO $table ($colnames) VALUES ($colvals)") or die('Database Connection Error ('.mysqli_errno($mysqli).') '.mysqli_error($mysqli). " on query: INSERT INTO $table ($colnames) VALUES ($colvals)");
unset($cols, $vals);
}
mysqli_close($mysqli);
if ($mysql)
return TRUE;
else return FALSE;
I actually have just this function sitting around because I use it all the time. What this does: pool all the associative pairs in your array for insertion into the table, and puts them in as a single insert. To be clear: this isn't what you're doing above, as it looks like you're trying to insert each value in its own insert, which would probably give you way more rows than you want.
function mysqli_insert($table, $assoc) {
$mysqli = mysqli_connect(PUT YOUR DB CREDENTIALS HERE);
mysqli_select_db($mysqli, DATABASE NAME HERE);
foreach ($assoc as $column => $value) {
$cols[] = $column;
$vals[] = mysqli_real_escape_string($mysqli, $value);
}
$colnames = "`".implode("`, `", $cols)."`";
$colvals = "'".implode("', '", $vals)."'";
$mysql = mysqli_query($mysqli, "INSERT INTO $table ($colnames) VALUES ($colvals)") or die('Database Connection Error ('.mysqli_errno($mysqli).') '.mysqli_error($mysqli). " on query: INSERT INTO $table ($colnames) VALUES ($colvals)");
mysqli_close($mysqli);
if ($mysql)
return TRUE;
else return FALSE;
}
As MarcB says above, there is a risk that your target table won't have a column that your results show. But we are sanitizing the insertion (with mysqli_real_escape_string) so we shouldn't have issues with injections vulnerabilities.
Here's one-liner:
$query = 'INSERT INTO '.$table.'(`'.implode('`, `', array_keys($array)).'`) VALUES("'.implode('", "', $array).'")';
I noticed an answer being posted above, but since I already wrote this, thought I'd post it. Note: I hope you trust where you get your data from, otherwise this opens up all sorts of problems. I've also added comments throughout the code, hopefully helps to understand how it's being done.
$jsonString = file_get_contents("http://search-query-URL&output=json");
$array = json_decode($jsonString, true);
// database connection code snipped out here
$table = "results";
foreach($array as $sub_array) {
// First, grab all keys and values in separate arrays
$array_keys = array_keys($sub_array);
$array_values = array_values($sub_array);
// Build a list of keys which we'll insert as string in the sql query
$sql_key_list = array();
foreach ($array_keys as $key){
$sql_key_list[] = "`$key`";
}
$sql_key_list = implode(',', $sql_key_list);
// Build a list of values which we'll insert as string in the sql query
$sql_value_list = array();
foreach ($array_values as $value){
$value = mysql_real_escape_string($value);
$sql_value_list[] = "'$value'";
}
$sql_value_list = implode(',', $sql_value_list);
// Build the query
$insert = "INSERT INTO $table ($sql_key_list) VALUES ($sql_value_list)";
mysql_query($insert) or die(mysql_error());
}
Related
public function add_employee($input)
{
$key_array = null;
$value_array = null;
$bind_array = null;
foreach ($input as $column => $value) {
if ($value) {
#$bind_array => ?, ?, ?;
#$value_array => [$value1, $value2, $value3];
#$key_array => column1, column2, column3;
}
}
$sql = "INSERT INTO ol_employee ($key_array) VALUES ($bind_array)";
$this->db->query($sql, $value_array);
}
Refer to comment in the function, how to achieve that output?
the idea is, from the input POST i get which over 27 fields, i just want to fill in into the $sql query i prepared as you can see. I don't think writing each table column manually is a good way.
im using Codeigniter 4 php framework + postgresql.
According to CodeIgniter 4 documentation, you can do this inside your loop for each employee:
$data = [
'title' => $title,
'name' => $name,
'date' => $date
];
$db->table('mytable')->insert($data);
You can use array and implode function first make array of key_array, value_array and bind_array then use implode() and use in sql like following
public function add_employee($input)
{
$key_array = array();
$value_array = array();
$bind_array = array();
foreach ($input as $column => $value) {
if ($value) {
$bind_array[] = '?';
$value_array[] = $value;
$key_array[] = $column;
}
}
$binds = implode(",",$bind_array);
$keys = implode(",",$key_array);
$values = implode(",",$value_array);
$sql = "INSERT INTO ol_employee ($keys) VALUES ($binds)";
$this->db->query($sql,$values);
}
by the insight of Alex Granados, this is the query i use:
public function add_employee($input)
{
foreach ($input as $column => $value) {
if ($value) {
$data[$column] = $value;
}
}
$this->db->table('ol_employee')->insert($data);
}
this will eliminate null field and regardless how many field, still working good.
as long the input name field from form is same as db column. Else, need to do some changes on that.
Thanks guys.
I'm getting null values after I run the DBEscape($data) function that is for SQL injection protection. Can someone help?
My inputs are all multiple arrays, ex: name="quote[][dt_flight]", name="quote[][acft]", etc.
Method is POST.
function DBEscape($data){
$link = DBConect();
if(!is_array($data)){
$data = mysqli_real_escape_string($link,$data);
}
else {
$arr = $data;
foreach ($arr as $key => $value){
$key = mysqli_real_escape_string($link, $key);
$value = mysqli_real_escape_string($link, $value);
$data[$key] = $value;
}
}
DBClose($link);
return $data;
}
function DBCreate($table, array $data, $insertId = false){
$table = DB_PREFIX.'_'.$table;
$data = DBEscape($data);
var_dump($data);
$fields = implode(", ", array_keys($data));
$values = "'".implode("', '", $data)."'";
$query = "INSERT INTO {$table} ({$fields}) VALUES ({$values});";
var_dump($query);
return DBExecute($query, $insertId);
}
if(isset($_POST["quote"]) && is_array($_POST["quote"])){
foreach($_POST["quote"]["dt_flight"] as $key => $text_field){
$last_id = DBCreate('quote',$_POST['quote'],true);
$i++;
}
}
The connection works since it is inserting the rows into the tables. I used vardump before and after the DBEscape to figure out that it is deleting the values, the keys are fine.
PS: The proposed answer is for a single variable not an array.
As you can see in your var_dump-result, the data you sent to DBCreate and thus to DBEscape looks like
array(
'dt_flight' => array(0 => '2018-06-13'),
'acft' => array(0 => 'VQ-BFD',
// and so on
)
Therfore the data you sent to
// $value = array(0 => '2018-06-13') here
$value = mysqli_real_escape_string($link, $value);
And well, mysqli_real_escape_string doesn't like arrays very much, thus will return NULL and thus inserting empty data in your table.
You most likely want to resolve this error within your foreach($_POST["quote"]["dt_flight"]) loop, since I suppose you sent multiple flight-data:
foreach($_POST["quote"]["dt_flight"] as $key => $text_field) {
// $key would be 0, for $_POST["quote"]["dt_flight"][0] = '2018-06-13'
$keyData = [];
foreach($_POST["quote"] as $field => $allFieldValues) {
// Walk over every field, and add the value for the same $key
if (is_array($data) && isset($allFieldValues[$key])) {
// Would add for example $keyData['acft'] = $_POST['quote']['acft'][0] = 'VQ-BFD';
$keyData[$field] = $allFieldValues[$key];
}
}
var_dump($keyData);
// Would look like array(
// 'dt-flight' => '2018-06-13',
// 'acft' => 'VQ-BFD',
// and so on
// )
$last_id = DBCreate('quote',$keyData,true);
$i++;
}
Although this is not part of your question, I really suggest you also take care of my comment on your question about mysqli_real_escape_string not being a safe way to escape column-names (or table-names and so on). For example with following solution:
function DBCreate($table, array $data, $insertId = false) {
// For each table the known columns
$columns = array( 'quote' => array('dt_flight', 'acft', '...') );
// Verify valid table given
if (!isset($columns[$table])) {
throw new InvalidArgumentException('No such table: ' . $table);
}
// Remove everything from data where the key is not in $columns[$table]
// = Remove everything where the column-name is non-existing or even an attempt to hack your system
$data = array_intersect_key($data, array_fill_keys($columns[$table], null));
if (!count($data)) {
throw new InvalidArgumentException('No (valid) data given at all');
}
// Next, continue with your implementation
}
I have a PHP array of the column names in my SQL table. I also have an array of the values I want to assign to these columns. How do I put this in an SQL query. At present im writing out each column title like so:
$query = "INSERT INTO `first_page_data`(`a`, `b`, `c`, `d`, `e`, `f`, `g`, `h`)
VALUES ('$1','$2','$3','$4','$5','$6','$7','$8')";
But there must be a way of just using the arrays?
As an extra, is there a way of defining key/value pairs to keep the two pairs of data together, and then using these to insert into the database? how is this formatted in the SQL query?
Here's another similar solution.
Code:
<?php
function mysql_insert_array($table, $data, $exclude = array()) {
$fields = $values = array();
if( !is_array($exclude) ) $exclude = array($exclude);
foreach( array_keys($data) as $key ) {
if( !in_array($key, $exclude) ) {
$fields[] = "`$key`";
$values[] = "'" . mysql_real_escape_string($data[$key]) . "'";
}
}
$fields = implode(",", $fields);
$values = implode(",", $values);
if( mysql_query("INSERT INTO `$table` ($fields) VALUES ($values)") ) {
return array( "mysql_error" => false,
"mysql_insert_id" => mysql_insert_id(),
"mysql_affected_rows" => mysql_affected_rows(),
"mysql_info" => mysql_info()
);
} else {
return array( "mysql_error" => mysql_error() );
}
}
?>
Example usage:
<?php
// Open database here
// Let's pretend these values were passed by a form
$_POST['name'] = "Bob Marley";
$_POST['country'] = "Jamaica";
$_POST['music'] = "Reggae";
$_POST['submit'] = "Submit";
// Insert all the values of $_POST into the database table `artists`, except
// for $_POST['submit']. Remember, field names are determined by array keys!
$result = mysql_insert_array("artists", $_POST, "submit");
// Results
if( $result['mysql_error'] ) {
echo "Query Failed: " . $result['mysql_error'];
} else {
echo "Query Succeeded! <br />";
echo "<pre>";
print_r($result);
echo "</pre>";
}
// Close database
?>
Source: Inserting An Array into a MySQL Database Table
//Insert ( var , Array )
function insert($table, $inserts) {
$values = array_map('mysql_real_escape_string', array_values($inserts));
$keys = array_keys($inserts);
return mysql_query('INSERT INTO `'.$table.'` (`'.implode('`,`', $keys).'`) VALUES (\''.implode('\',\'', $values).'\')');
}
/* Samples
insert('first_page_data', array(
'a' => 'Just Persian Gulf',
'b' => 'DB9',
'c' => '2009'
));
*/
it's good And Rapid!
Using PHP:
Getting your values into an array as $key => $value depends on the situation, but manually it would happen like so:
$array = array(`a` => '$1',`b` => '$2', ...and so on); //I am assuming that $ is not a variable indicator since it is inside single quotes.
There are a variety of array functions that can help you if you have existing arrays that you would rather manipulate to create the final array.
Once you have it, however:
$query = 'INSTERT INTO `first_page_data` (';
foreach ($array as $key => $value) {
$query .= '`' . $key . '`';
}
$query .= ') VALUES (';
foreach ($array as $value) {
$query .= '`' . $value . '`';
}
$query .= ')';
The code runs a foreach on the array twice, once to get the key and append it to the appropriate part of the string, and the other to add the corresponding values.
Try serialize() before the INSERT and unserialize() to get the array after a SELECT.
You need only one field to insert all the data.
http://ca1.php.net/manual/fr/function.serialize.php
http://ca1.php.net/manual/fr/function.unserialize.php
# Insert this array
$arr = array("sounds" => "one", "sound" => "two", "big" => "blue");
function addQuotes($str){
return "'$str'";
}
# Surround values by quotes
foreach ($arr as $key => &$value) {
$value = addQuotes($value);
}
# Build the column
$columns = implode(",", array_keys($arr));
# Build the values
$values = implode(",", array_values($arr));
# Build the insert query
$query = "INSERT INTO table (".$columns.") VALUES (".$values.")";
echo $query;
// returns
INSERT INTO table (sounds,sound,big) VALUES ('one','two','blue')
I am trying to build a class that insert array of values into oracle database, I am also trying to make the class scalable (works with all forms of the application).
the reason am doing this is to reduce code repeating for the application am working on, because it has a lot of forms, and some of them has a lot of variables (50+).. therefore I need to make a class capable of inserting values into table, where the number of values is dynamic (depending on the form).
so far what I did is:
class ArrayQuery {
public $conn;
public $table;
public function __construct($conn){
$this->conn = $conn;
}
public function setTableName($table)
{
$this->table = $table;
}
public function InsertArray(array $args){
$keys = array_keys($args);
$values = array_values($args);
// need help here //
}
}
The construct function will get database connection, by calling the $conn object from the db_connection class (e.g $conn = new db_connection(); $query = new ArrayQuery($conn);)
setTableName is obviously for defining the desired table.
now the ArrayQuery, what I did in my form page, I named the keys of the array the same as the name of columns in table, for example array args['id']= $_post['id']. so basically array_key = column name. I did that in order to get columns name without manually setting them.
now I have 2 arrays: $keys (holds the name of columns), and $values (holds the data to be insert).
I cant figure out how to bind variables into keys? knowing that the number of variables is dynamic ?
any help will be much appreciated
UPDATE:
Here is how the $args array are set
$args = array(
'id' => $_POST['id'],
'firstname' => $_POST['firstname'],
'lastname' => $_POST['lastname'],
'email' => $_POsT['email'],
)
answer:
thanks to #calculon for putting me on the right track, with little modifications to work in my scenario, for future reference the answer to my case is:
$i=0; $col=''; $val='';
foreach ($args as $key => $value) {
if($i==0){
$col .= $key;
$val .= ':'.$key;
}
else{
$col .= ', '.$key;
$val .= ', :'.$key;
}
$i++;
}
$sql = 'INSERT INTO '.$table.' ('.$col.') VALUES ('.$val.') ';
$stmt = oci_parse($this->conn, $sql);
foreach ($args as $key => $value) {
oci_bind_by_name($stmt, $key, $args[$key]) ;
}
oci_execute($stmt);
hope it will be helpful, thanks again calculon.
Approximately such algorithm for inserting one row with an unknown number of columns I used in my project, for multiple rows should be added one more cycle
$i=0; $keys=''; $vals='';
foreach ($args as $key => $value) {
if($i==0){
$keys .= ''.$key;
$vals .= ':'.$key;
}
$keys .= ', '.$key;
$vals .= ', :'.$key;
$i++;
}
$sql = 'INSERT INTO '.$table.' ('.$keys.') VALUES ('.$vals.') ';
$stmt = oci_parse($conn, $sql);
foreach ($args as $key => $value) {
oci_bind_by_name($stmt, $key, $args[$key]) ;
}
oci_execute($stmt);
In trying to create a simple PHP PDO update function that if the field is not found would insert it, I created this little snippet.
function updateorcreate($table,$name,$value){
global $sodb;
$pro = $sodb->prepare("UPDATE `$table` SET value = :value WHERE field = :name");
if(!$pro){
$pro = $sodb->prepare("INSERT INTO `$table` (field,value) VALUES (:name,:value)");
}
$pro->execute(array(':name'=>$name,':value'=>$value));
}
It does not detect though if the update function is going to work with if(!$pro); How would we make this one work.
You are assigning $pro to the prepare, not the execute statement.
Having said that, if you are using mysql you can use the insert... on duplicate key update syntax.
insert into $table (field, value) values (:name, :value) on duplicate key update value=:value2
You can't use the same bound param twice, but you can set two bound params to the same value.
Edit: This mysql syntax will only work where a key (primary or another unique) is present and would cause an insert to fail.
If it's mysql-only you could try INSERT INTO ... ON DUPLICATE KEY UPDATE
http://dev.mysql.com/doc/refman/5.0/en/insert-on-duplicate.html
You will first need to execute it.
Apart from that, this is a dodgy way of doing this. It would be better to start a transaction, do a SELECT and then determine what to do (INSERT or UPDATE). Just checking whether the UPDATE query succeeded doesn't suffice, it succeeds when no row is found too.
try,
PDO::exec()
returns 1 if inserted.
2 if the row has been updated.
for prepared statements,
PDOStatement::execute()
You can try,
PDOStement::rowCount()
The following are PHP PDO helper functions for INSERT and UPDATE
INSERT function:
function basicInsertQuery($tableName,$values = array()){
/*
//
USAGE INSERT FUNCTÄ°ON
$values = [
"column" => $value,
];
$result = basicInsertQuery("bulk_operations",$values);
*/
try {
global $pdo;
foreach ($values as $field => $v)
$vals[] = ':' . $field;
$ins = implode(',', $vals);
$fields = implode(',', array_keys($values));
$sql = "INSERT INTO $tableName ($fields) VALUES ($vals)";
$rows = $pdo->prepare($sql);
foreach ($values as $k => $vl)
{
$rows->bindValue(':' . $k, $l);
}
$result = $rows->execute();
return $result;
} catch (\Throwable $th) {
return $th;
}
}
UPDATE function:
function basicUpdateQuery($tableName, $values = array(), $where = array()) {
/*
*USAGE UPDATE FUNCTÄ°ON
$valueArr = [ column => "value", ];
$whereArr = [ column => "value", ];
$result = basicUpdateQuery("bulk_operations",$valueArr, $whereArr);
*/
try {
global $pdo;
//set value
foreach ($values as $field => $v)
$ins[] = $field. '= :' . $field;
$ins = implode(',', $ins);
//where value
foreach ($where as $fieldw => $vw)
$inswhere[] = $fieldw. '= :' . $fieldw;
$inswhere = implode(' && ', $inswhere);
$sql = "UPDATE $tableName SET $ins WHERE $inswhere";
$rows = $pdo->prepare($sql);
foreach ($values as $f => $v){
$rows->bindValue(':' . $f, $v);
}
foreach ($where as $k => $l){
$rows->bindValue(':' . $k, $l);
}
$result = $rows->execute();
return $result;
} catch (\Throwable $th) {
return $th;
}
}