want to insert or update nearly 10000 records. It takes too long to insert in the for loop. Trying for the aternative way it only inserts the values but on duplicate_key entry shows the errors.
$var=0;
if($method == 1)
{
// $sql5 is the current way trying but shows error
$sql5='insert into stu_regnum_hall(hall_name,row,column,register_number,subject_code,exam_date,session,unique_key) values';
for ($i = 0; $i <=$arr_length; $i++)
{
for ($j = 1; $j <= 5; $j++)
{
for($k=0;$k<=4;$k++)
{
if(isset($show[$var+$k]))
{
if($k==0)
{
$hall_name=$arr1[$i];
$row=$j;
}
$column=$k+1;
$reg_num=$show[$var+$k];
$sub_code=$newArr12[$var+$k];
$unique_key=$reg_num.$date.$session;
//current way to insert This is taking too long time to store nearly 10000 records
Yii::$app->db->createCommand('insert into stu_regnum_hall(hall_name,row,column,register_number,subject_code,exam_date,session,unique_key) values ("'.$hall_name.'","'.$row.'","'.$column.'","'.$reg_num.'","'.$sub_code.'","'.$date.'","'.$session.'","'.$unique_key.'") ON DUPLICATE KEY UPDATE hall_name ="'.$hall_name.'",row="'.$row.'",column="'.$column.'",register_number="'.$reg_num.'",subject_code="'.$sub_code.'",exam_date="'.$date.'",session="'.$session.'",unique_key="'.$unique_key.'"')->query();
//concatenation of $sql5
$sql5.='("'.$hall_name.'","'.$row.'","'.$column.'","'.$reg_num.'","'.$sub_code.'","'.$date.'","'.$session.'","'.$unique_key.'") ON DUPLICATE KEY UPDATE hall_name= "'.$hall_name.'",row="'.$row.'",column="'.$column.'",register_number="'.$reg_num.'",subject_code="'.$sub_code.'",exam_date="'.$date.'",session="'.$session.'",unique_ key="'.$unique_key.'",';
}
else
{
break;
}
}
$var=$var+5;
}
}
$sql5.='("");';
// This is the required answer but getting error
Yii::$app->db->createCommand($sql5)->query();
}
the $sql5 query should work in this
Try Yii2 batchInsert with a little custom use INSERT IGNORE, REPLACE or ON DUPLICATE KEY UPDATE. It will insert very fast.
$sql = Yii::$app->db->queryBuilder->batchInsert('your_table_name', ['column1', 'column2','column3',...], $dataArray);
// For IGNORE on Duplicate
//$sql = 'INSERT IGNORE' . mb_substr($sql, strlen('INSERT'));
// For Replace on Duplicate
//$sql = 'REPLACE' . mb_substr($sql, strlen('INSERT'));
// For Update on Duplicate
$sql .= ' ON DUPLICATE KEY UPDATE `column1` = VALUES(`column1`), `column2` = VALUES(`column2`), ...';
Yii::$app->db->createCommand($sql)->execute();
Refer: http://www.yiiframework.com/doc-2.0/yii-db-querybuilder.html#batchInsert()-detail
Related
I'd like to pass multiple variables in a foreach loop to add the value from $array_sma[] into my database. But so far I can only insert the value from $short_smas, while I'd also like to insert the values from $mid_smas. I have tried nested foreach but it's multiplying the values.
$period = array(1,2,3,4,5,6,7,8,9,10,11,12,13,14,15);
$sma = array(6,9);
foreach ($sma as $range) {
$sum = array_sum(array_slice($period, 0, $range));
$result = array($range - 1 => $sum / $range);
for ($i = $range, $n = count($period); $i != $n; ++$i) {
$result[$i] = $result[$i - 1] + ($period[$i] - $period[$i - $range]) / $range;
}
$array_sma[] = $result;
}
list($short_smas,$mid_smas)=$array_sma;
foreach ($short_smas as $short_sma) {
$sql = "INSERT INTO sma (short_sma)
VALUES ('$short_sma') ";
if ($con->query($sql) === TRUE) {
echo "New record created successfully<br><br>";
} else {
echo "Error: " . $sql . "<br>" . $con->error;
}
}
The code in my question works fine i.e. the value from the first sub array ($short_smas) of $array_sma[] gets inserted into the column short_sma of my msql database. The problem I have is when I try to insert the second sub array $mid_smas (see list()) from $array_sma[] in my second column of my database call mid_sma.
I think this is closed to what I want to achieve but still nothing gets inserted in the DB, source: php+mysql: insert a php array into mysql
I don't have any mysql syntax error.
$array_sma[] = $result;
$sql = "INSERT INTO sma (short_sma, mid_sma) VALUES ";
foreach ($array_sma as $item) {
$sql .= "('".$item[0]."','".$item[1]."'),";
}
$sql = rtrim($sql,",");
Main problem is that $short_smas and $mid_smas have different size. Moreover they are associative arrays so either you pick unique keys from both and will allow for empty values for keys that have only one value available or you pick only keys present in both arrays. Code below provides first solution.
// first lets pick unique keys from both arrays
$uniqe_keys = array_unique(array_merge(array_keys($short_smas), array_keys($mid_smas)));
// alternatively we can only pick those present in both
// $intersect_keys = array_intersect(array_keys($short_smas),array_keys($mid_smas));
// now lets build sql in loop as Marcelo Agimóvel sugested
// firs we need base command:
$sql = "INSERT INTO sma (short_sma, mid_sma) VALUES ";
// now we add value pairs to coma separated list of values to
// insert using keys from prepared keys array
foreach ($uniqe_keys as $key) {
$mid_sma = array_key_exists($key, $mid_smas)?$mid_smas[$key]:"";
$short_sma = array_key_exists($key, $short_smas)?$short_smas[$key]:"";
// here we build coma separated list of value pairs to insert
$sql .= "('$short_sma', '$mid_sma'),";
}
$sql = rtrim($sql, ",");
// with data provided in question $sql should have string:
// INSERT INTO sma (short_sma, mid_sma) VALUES, ('3.5', ''), ('4.5', ''), ('5.5', ''), ('6.5', '5'), ('7.5', '6'), ('8.5', '7'), ('9.5', '8'), ('10.5', '9'), ('11.5', '10'), ('12.5', '11')
// now we execute just one sql command
if ($con->query($sql) === TRUE) {
echo "New records created successfully<br><br>";
} else {
echo "Error: " . $sql . "<br>" . $con->error;
}
// don't forget to close connection
Marcelo Agimóvel also suggested that instead of multiple inserts like this:
INSERT INTO tbl_name (a,b,c) VALUES (1,2,3);
its better to use single:
INSERT INTO tbl_name
(a,b,c)
VALUES
(1,2,3),
(4,5,6),
(7,8,9);
That's why I append value pairs to $sql in foreach loop and execute query outside loop.
Also its worth mentioning that instead of executing straight sql its better to use prepared statements as they are less prone to sql injection.
So, I got a series of data that I need to insert into a table. Right now I am using a for loop to iterate through each entry and save the model one by one. But that doesn't seem like a good way to do it, moreover using transaction would be an issue. What's a better way to do it to improve performance and also so I can use transaction.Here's the code I am currently using.
foreach ($sheetData as $data)
{
$newRecord = new Main;
$newRecord->id = $data['A'];
$newRecord->name = $data['B'];
$newRecord->unit = $data['C'];
$newRecord->save();
}
If you can skip validation, you can generate a simple sql insert and execute it once. like:
$count = 0;
$sql = '';
foreach ($sheetData as $data)
{
if(!$count)
$sql .= 'INSERT INTO tbl_main (id ,name ,unit) Values ('.$data['A'].','$data['B']','$data['C']') ';
else
$sql .= ' , ('.$data['A'].','$data['B']','$data['C']')';
$count++;
}
Yii::app()->db->createCommand($sql)->execute();
I got something like this.
It's working, but have to do a mysql query like 40 times is not the best thing.
Can somebody help me get it in to one query?
$string_unique = 'Unique string';
$number = count($something);
for ($i=0; $i < $number; $i++)
{
if(!empty($some_input)) {
$string_one = 'Something_one' . $i;
$string_two = 'Something_two' . $i;
mysql_query("INSERT INTO `table` (`unique_string`, `string_one`, `string_two`) VALUES('$unique_string', '$string_one', '$string_two') ON DUPLICATE KEY UPDATE `string_one` = '$string_one', `string_two` = '$string_two'") or die(mysql_error());
} else {
$string_one = '';
$string_two = '';
mysql_query("INSERT INTO `table` (`unique_string`, `string_one`, `string_two`) VALUES('$unique_string', '$string_one', '$string_two') ON DUPLICATE KEY UPDATE `string_one` = '$string_one', `string_two` = '$string_two'") or die(mysql_error());
}
You can generate a single query in that loop and then execute it only once:
$query = 'INSERT INTO `table` (`unique_string`, `string_one`, `string_two`) VALUES ';
$queryValues = array();
$string_unique = 'Unique string';
$number = count($something);
for ($i=0; $i < $number; $i++)
{
if(!empty($some_input)) {
$string_one = 'Something_one' . $i;
$string_two = 'Something_two' . $i;
$queryValues[] = sprintf('("%s", "%s", "%s")', $unique_string, $string_one, $string_two);
} else {
// I can't understand what are you using this part for... Anyway.
$queryValues[] = sprintf('("%s", "", "")', $unique_string);
}
}
$query .= implode(', ', $queryValues);
// And here is the unique key updating.
// I don't know whole the structure, so I just guess you have the PRIMARY `id` row
// What you did in your ON DUPLICATE KEY was unnecessary.
$query .= ' ON DUPLICATE KEY UPDATE `id`=`id`';
// Don't use mysql_* functions in new code.
// See: http://stackoverflow.com/questions/12859942/why-shouldnt-i-use-mysql-functions-in-php
mysql_query($query);
You're a allowed to add multiple rows of data in an insert statement in MySQL. That way, you can build one big insert statement in the loop, and execute it in one go after the loop. That is way faster and much more efficient.
You can insert multi row with sql,
here an example
insert into table1 (First,Last) values ("Fred","Smith"),
("John","Smith"),
("Michael","Smith"),
("Robert","Smith");
I think I've got a tiny mistake in my code so when I try to add two records to MySQL database it adds them both, but at the moment its only adding the second row that should be inputted. SO for example I have two RefTitle fields, two RefSurname fields and so on!
Some PHP Code:
<?php
if(empty($err)) {
for($i = 0; $i < 2; $i++)
{
$RefTitle = $_POST['RefTitle'][$i];
$RefSurname = $_POST['RefSurname'][$i];
$RefForenames = $_POST['RefForenames'][$i];
$RefInstitute = $_POST['RefInstitute'][$i];
$RefEmail = $_POST['RefEmail'][$i];
$RefTelephone = $_POST['RefTelephone'][$i];
$EmailOK = $_POST['EmailOK'][$i];
$sql_insert = "INSERT into `referees`
(`RefTitle`,`RefSurname`,`RefForenames`,`RefInstitute`, `RefEmail`,
`RefTelephone`,`EmailOK`)
VALUES
('$RefTitle','$RefSurname','$RefForenames','$RefInstitute','$RefEmail',
'$RefTelephone','$EmailOK'
)
"; ?>
I have the [] after each name field in my html form.
Thanks
Here's what I think is happening.
In the loop, you're adding the current query to a string, overwriting the previous query. After the loop ends, you run mysql_query but this will only run on the last query generated by the loop.
You have two ways to fix this:
Execute the query within the loop on every iteration. Pseudo code:
for ($i = 1 to 100)
{
$query = //build your query;
mysql_query($query);
}
Alternatively, (and the better way) is to use a value list in your insert statement.
An insert with a list of values looks like this
INSERT INTO table VALUES (1,2,3), (4,5,6), (7,8,9)
So, (pseudo code again)
$query = "INSERT INTO table VALUES ";
for ($i = 1 to 100)
{
$query .= " (";
$query .= "" //comma seperate the values of the current iteration
$query .= "),";
}
//after loop ends, remove the trailing extraneous comma
// execute the entire query at once
mysql_query($query);
I have two arrays with anywhere from 1 to 5 set values. I want to insert these values into a table with two columns.
Here's my current query, given to me in another SO question:
INSERT INTO table_name (country, redirect)
VALUES ('$country[1]', '$redirect[1]'),
('$country[2]', '$redirect[2]'),
('$country[3]', '$redirect[3]'),
('$country[4]', '$redirect[4]'),
('$country[5]', '$redirect[5]')
ON DUPLICATE KEY UPDATE redirect=VALUES(redirect)
I'm a little concerned however with what happens if some of these array values aren't set, as I believe the above assumes there's 5 sets of values (10 values in total), which definitely isn't certain. If a value is null/0 does it automatically skip it?
Would something like this work better, would it be a lot more taxing on resources?
for($i = 0, $size = $sizeof($country); $i <= size; $i++) {
$query = "INSERT INTO table_name (country, redirect) VALUES ('$country[$i]', '$redirect[$i]) ON DUPLICATE KEY UPDATE redirect='$redirect[$i]'";
$result = mysql_query($query);
}
Questions highlighted in bold ;). Any answers would be very much appreciated :) :)!!
Do something like this:
$vals = array()
foreach($country as $key => $country_val) {
if (empty($country_val) || empty($redirect[$key])) {
continue;
}
$vals[] = "('" . mysql_real_escape_string($country_val) . "','" . mysql_real_escape_string($redirect[$key]) . "')";
}
$val_string = implode(',', $vals);
$sql = "INSERT INTO .... VALUES $val_string";
That'll built up the values section dynamically, skipping any that aren't set. Note, however, that there is a length limit to mysql query strings, set by the max_allowed_packet setting. If you're building a "huge" query, you'll have to split it into multiple smaller ones if it exceeds this limit.
If you are asking whether php will automatically skip inserting your values into the query if it is null or 0, the answer is no. Why dont you loop through the countries, if they have a matching redirect then include that portion of the insert statement.. something like this: (not tested, just showing an example). It's one query, all values. You can also incorporate some checking or default to null if they do not exist.
$query = "INSERT INTO table_name (country, redirect) VALUES ";
for($i = 0, $size = $sizeof($country); $i <= size; $i++) {
if(array_key_exists($i, $country && array_key_exists($i, $redirect)
if($i + 1 != $size){
$query .= "('".$country[$i]."', '".$redirect[$i]).",";
} else $query .= "('".$country[$i]."', '".$redirect[$i].")";
}
}
$query .= " ON DUPLICATE KEY UPDATE redirect=VALUES(redirect);"
$result = mysql_query($query);