$connection->createCommand()->batchInsert('user', ['name', 'age'], [
[$names, $ages],
])->execute();
I know that i can do a batch insert using the code above in Yii 2. But how can i prevent duplicate entry using batchInsert? For example, if I have a duplicate name, i dont want to insert it in the db
Two possible options:
1) Use unique constraint for name column in your database table.
That way you just try to execute query and catch exception:
try {
// Your query goes here
} catch (\yii\db\Exception) {
// Handle error
}
\yii\db\Exception is more common database operations related exception, you can use more specific \yii\db\IntegrityException.
2) Exclude duplicates from array in PHP before feeding data and executing query. Depending on how array is constructed you can do it:
during building this array in foreach for example by checking if item with same name already exists in formed array, if yes - append element to it, otherwise - skip.
or afterwards using for example array_filter function.
I recommend first approach, because even you decide to handle duplicates in PHP, according data structure and additional protection in database won't be superfluos.
Related
I have an external database that I am trying to access from within a Drupal page, I have successfully queried the database and output data to the page using fetchAssoc(), however this only returns the first row in the database. I would like to return all rows into an array for processing, so I'm attempting to use fetchAllAssoc(), this however results in an exception. The database has the following SQL fields:
id, model, manufacturer, url, date_modified
My test code is as follows:
<?php
db_set_active('product_db');
$query = db_select('product', 'p')->fields('p');
$sqlresults = $query->execute()->fetchAllAssoc('id');
foreach($sqlresults as $sqlresult)
{
printf($sqlresult);
}
db_set_active();
?>
I'm thinking that it is the key field 'id' that I am specifying with fetchAllAssoc() that is the problem, as fetchAssoc() prints values correctly. All documentation I have found seems to say that you pass a database field as the key but I have also passed a numeric value with no success.
Many thanks in advance for any advice, I'm sure I'm just missing something stupid.
I think it should work in this way, but within the foreach you want to print the $sqlresult variable as a string, but it is an object (it causes the error).
printf function needs a string as the first parameter, see:
http://php.net/manual/en/function.printf.php
Use for instance var_dump instead:
var_dump($sqlresult);
I have a strange situation.
Suppose I have a very simple function in php (I used Yii but the problem is general) which is called inside a transaction statement:
public function checkAndInsert($someKey)
{
$data = MyModel::model()->find(array('someKey'=>$someKey)); // search a record in the DB.If it does not exist, insert
if ( $data == null)
{
$data->someCol = 'newOne';
$data->save();
}
else
{
$data->someCol = 'test';
$data->save();
}
}
...
// $db is the instance variable used for operation on the DB
$db->transaction();
$this->checkAdnInsert();
$db->commit();
That said, if I run the script containing this function by staring many processes, I will have duplicate values in the DB. For example, if I have $someKey='pippo', and I run the script by starting 2 processes, I will have two (or more) records with column "someCol" = "newOne". This happens randomly, not always.
Is the code wrong? Should I put some constraint in DB in form of KEYs?
I also read this post about adding UNIQUE indexes to TokuDB which says that UNIQUE KEY "kills" write performance...
The approach you have is wrong. It's wrong because you delegate the authority for integrity/uniqueness check to PHP, but it's the database that's responsible for that.
In other words, you don't have to check whether something exists and then insert. That's bad because there's always some slight ping involved between PHP and MySQL and as you already saw - you can get false results for your checks.
If you need unique values for certain column or combination of columns, you add a UNIQUE constraint. After that you simply insert. If the record exists, insert fails and you can deal with it via Exception. Not only is it faster, it's also easier for you because your code can become a one-liner which is much easier to maintain or understand.
i'm relatively new to coding and I need a little help. I'm basically trying to loop through an entry in a mySQL database and push any new entry into an array , so that it only comes up once in my array.
// SQL query
$response = $bdd->query('SELECT serie_bd FROM inventaire_bd');
//creating array to group all elements from the db so that they do not repeat
$serie_bd_groupe=array();
while($data_collected = $response->fetch())
{
if(array_key_exists($data_collected,$serie_bd_groupe)==false)
{
array_push($data_collected,$serie_bd_groupe);
}
}
Will this work? - it seems like the loop will just stay stuck after it comes accross an entry a second time because the if statement wont execute itself.
Also in the future, are their any php equivalent to jsfiddle.net so i can test code syntaxically?
Thank you for your time
Your array keys will be default integers, so you don't want to check those. Instead of this:
if(array_key_exists($data_collected,$serie_bd_groupe)==false)
you should do this:
if(!(in_array($data_collected,$serie_bd_groupe)))
http://php.net/manual/en/function.in-array.php
On the other hand, if you're expecting your collected data to be the array key rather than value, you'd do something like this, instead of your array_push:
$serie_bd_groupe[$data_collected] = 1;
then your key check would work.
If you are looking for UNIQUE values (serie_bd) from your database, update your query to include "DISTINCT" like this:
$bdd->query('SELECT DISTINCT serie_bd FROM inventaire_bd');
On the other hand, I think you are looking for http://phpfiddle.org/
I am developing an application in PHP for which I need to implement a big file handler.
Reading and writing the file is not a problem, but checking the content of the file is a problem.
I built a recursive function which checks whether or not a variable is already used in the same document.
private function val_id($id){
if(!isset($this->id)){
$this->id = array();
}
if(in_array($id, $this->id)){
return $this->val_id($id+1);
}else{
$this->id[] = $id;
return $id;
}
}
When in_array($id,$this->id) returns FALSE, the $id will be added to $this->id (array which contains all used ids) and returns a valid id.
When this returns TRUE, it returns the same function with parameter $id+1
Since we are talking about over 300000 records a time, PHP won't not to be able to store such big arrays. It seems to quit writing lines in the documents I generate when this array gets too big. But I don't receive any error messages like that.
Since the generated documents are SQL files with multiple rows INSERT another solution could be to check if the id already exists in the database. Can MySQL catch these exceptions and try these entries again with adding 1 to id? How?
How do you think I need to solve this problem?
Kind regards,
Wouter
make error messages to appear.
increase memory_limit
instead of values store the parameter in the key - so you'll be able to use isset($array[$this->id]) instead of in_array()
Use INSERT IGNORE to disable duplicate key check in mysql and remove your key check in php. Your statement could look like this.
INSERT IGNORE INTO tbl_name SET key1 = 1, col1 = 'value1'
If you want to add 1 to the id always you could use ON DUPLICATE KEY to increment your key by one:
INSERT INTO table (a,b,c) VALUES (1,2,3)
ON DUPLICATE KEY UPDATE c=c+1;
Why should 30.000 records be a problem? Each record in a standard PHP array takes 144 bytes, for 30.000 that would mean 4218,75 kByte. No big deal.
Otherwise, Your Common Sense's idea with the array-key is worth a thought, because it's faster.
I have to retrieve the history of a user and I have 4 tables whose data depend on each other.I can retrieve the data using loops,but I instead used the "where IN ()" clause and I implode the output of the previous query.However,if the list I provide to "where IN()" is empty it return an error.Is it that IN() cannot be empty?
When imploding an array for the IN clause, i do one of two things
1: Check if you even need to run the query at all
if(!empty($some_array)) {
//run mysql query
}
else {
// if you need to do something if the array is empty, such as error or set some defaults, do it here
}
2: A value in the array initiliser which is not ever in the database (for example, if im selecting based on a auto incrememnt id, i use zero as a default array value to stop any issues with empty data sets, as zero will never be in my id column).
$some_array = array(0);
You can add an empty value to the start, such as IN (0,your values here)