I try to upload my data to mysql Its working but it takes 35sec, too many sec.
What do I need to change in my code that it will work faster than 35 sec?
I use php to write my code and SQL query to send the data to my table that called "words" .
At my table in the database I have 4 columns ('word', 'num', 'hit', 'instoplist').
What I can do to fix this problem?
Thanks
This is my code:
<?php
function removeStopWordsFromArray($words)
{
.......
insert($words);
}
function insert($myWords)
{
global $conn;
foreach ($myWords as $key => $value) {
$word = $value['word'];
$number = $value['document'];
$hit = $value['hit'];
$stop = $value['stopList'];
$sql = "INSERT INTO words (word,num,hit,instoplist) VALUES ('$word', '$number', '$hit','$stop')";
if($conn->query($sql)!== TRUE)
{
echo "error". $conn->error;
}
}
fclose($fp);
}
$temp = pareseDocs();
removeStopWordsFromArray($temp);
?>
For every data you are running a query in DB. But the correct way in your case is to insert data in batches. You can write the code is following way:
$sql = "INSERT INTO words (word,num,hit,instoplist) VALUES";
foreach ($myWords as $key => $value) {
$word = $value['word'];
$number = $value['document'];
$hit = $value['hit'];
$stop = $value['stopList'];
$sql .= "('$word', '$number', '$hit','$stop'),";
}
$sql = rtrim($sql,',') //to remove last comma
if($conn->query($sql)!== TRUE)
{
echo "error". $conn->error;
}
This will run only single query in DB. Hence will be faster.
You can try this query outside of loop pass only one query:
INSERT IGNORE INTO MyTable ( Column1, Column2 ) VALUES
( Value1, Value2 ), ( Value1, Value2 )
IGNORE FOR HANDLING ERRORS
Your problem is that you are making each query separately in a for loop.
Take a look at https://stackoverflow.com/a/452934/4988637 to find out more on how to insert mutliple rows in a single query.
If you change your method to one single query, you should find your program's run-time to be drastically shortened.
In SQL Server 2008 you can insert multiple rows using a single SQL
INSERT statement.
INSERT INTO MyTable ( Column1, Column2 ) VALUES
( Value1, Value2 ), ( Value1, Value2 )
For reference to this have a look at MOC Course 2778A - Writing SQL
Queries in SQL Server 2008.
In your case, you could modify your code to look something like the following.
$sql = "INSERT INTO words (word, num, hit, instoplist) VALUES ";
foreach($myWords as $key => $value) {
$word = $value['word'];
$number = $value['document'];
$hit = $value['hit'];
$stop = $value['stopList'];
$sql .= "('$word', '$number', '$hit','$stop'),";
}
$sql = rtrim($sql, ',');
if($conn->query($sql) !== true) {
echo "error".$conn->error;
}
Related
When I enter data to mysql it shows as below, I want to take all the data to a one row. but since I'm entering data using an array I only have access to two elements at once. So how can I enter data to a single line? below is a snip of my code
$array = array("AAPL","GOOG","YHOO","FB","MSFT","NKE","SBUX");
foreach ($array as $s) {
$sql = "INSERT INTO portfolio ({$s} , {$sss}) VALUES ('{$amount}', '{$value}')";
}
Snip from MYSQL table
Your array is not of data values but column names. Your insert needs to be more like;
$sql = "INSERT INTO portfolio ("
foreach ($array as $s) {
$sql = $sql + " $s, ";
}
$sql = $sql + ") values (";
foreach ($amount as $s) {
$sql = $sql + " $s, ";
};
$sql = $sql + ");"
However, you should be using mysqli, binding parameters not building strings as in this example. Also, your table as shown has only one value per stock, no value and amount, only room for one value.
I have an array $members that contains some ID(maximum 6 in number) from the table users. Using the following code, I loop through each index of $members, search for the details and store them in another array.
foreach($members as $key=>$value){
$res = mysql_query("SELECT id,name,email FROM users WHERE id='$value'");
if ($res === false) {
echo mysql_error();
die;
}
$row = mysql_fetch_assoc($res);
if($row['id'])
{
$members_name[]=$row['name'];//array for name
}
}
Now I want to insert the ID & names that are stored in the array into another TABLE register in the following format:
(The left side are the rows in my TABLE register)
mem_0_id-->$members[0]
mem_0_name-->$members_name[0]
mem_1_id-->$members[1]
mem_1_name-->$members_name[1]
mem_2_id-->$members[2]
mem_2_name-->$members_name[2]
mem_3_id-->$members[3]
mem_3_name-->$members_name[3]
mem_4_id-->$members[4]
mem_4_name-->$members_name[4]
How can I insert in this way? using just a single INSERT statement?
haven't tried this, but here is my answer anyway :)
$query = "INSERT INTO register(id, name) VALUES ($members[0], $members_name[0])";
for($i=1; $i<count($members); $i++)
{
$query .= ", ($members[$i], $members_name[$i])";
}
then try to execute the query..
Do you do anything else with the array, or are you just retrieving it from one table in order to insert it into another?
If so then you can do the whole thing like this.
$memberIds = implode(',', $members); // comma separated list of member ids
$query = "insert into register (id, name) select id, name from users where id in ($memberIds)";
mysql_query($query); // this will select and insert in one go
If you do need to keep the array in memory, then it's still a good idea to get it all out at once
$memberIds = implode(',', $members); // comma separated list of member ids
$query = "select id, name from users where id in ($memberIds)";
$res = mysql_query($query);
while ($row = mysql_fetch_assoc($res)) {
$memberData[] = $row;
}
That's because running a query inside a loop is very bad for performance. Every time you run a query there is an overhead, so getting all the data at once means you pay this overhead once rather than multiple times.
Then you can build a statement to insert multiple rows:
$sql = "insert into register (id, name) values ";
$sql .= "(" . $memberData[0]['id'] . "," . $memberData[0]['name'] . ")";
for($i = 1; $i < count($memberData); $i++) {
$sql .= ",(" . $memberData[$i]['id'] . ",'" . $memberData[$i]['name'] . "')";
}
mysql_query($sql);
It's a bit nasty because of the commas and quotes but if I've done it correctly then if you do
echo $sql;
you should get something like
insert into register (id, name) values (1, 'john'), (2, 'jane'), (3, 'alice');
You can see that the first way, where you select and insert in one statment, is a lot nicer and easier so if you don't do anything else with the array then I highly recommend doing it that way.
I have an array that grabs checkbox data and posts certain information into the database if the checkbox data isn't a copy of something already in the database. What I would like to know is how can I create a code that scans through the database and finds data that wasn't part of my checkbox data and delete it from the database.
Okay, for example let's say I have values 1,2,3,4 in my database, but in my checkboxes I only get back 1,2,4. I would like a code that scans my database and deletes that value(s) (in this case 3) from the database.
Here is my current code:
foreach($_POST['publish'] as $index => $val){
$matches = mysql_query("SELECT * FROM `$blog_table` WHERE `postID` = '$val");
if (mysql_num_rows($matches) > 0)
{
// do nothing
} else {
$query3 = "insert into `$blog_table`
(`postID`)values
('$val')";
mysql_query($query3);
}
}
Here would be the code I would use with escaped input
if (!empty($_POST['publish']))
{
$inserts = array();
$deletes = array();
foreach ($_POST['publish'] as $val)
{
$deletes[] = intval($val);
$inserts[] = '('.intval($val).')';
}
$delete = "DELETE FROM `".$blog_table."` WHERE postID NOT IN (".implode(',',$deletes).");";
$insert = "INSERT INTO `".$blog_table."` (`postID`) VALUES ".implode(',',$inserts)."";
}
you should use query like this:
delete from table where id NOT in (3)
in php like:
$query = "DELETE FROM `$blog_table` WHERE `postID` NOT IN (" . implode(',', $array) . ")";
For the MySQL query, you can use NOT IN:
DELETE FROM tablename
WHERE col1 NOT IN (1, 2, 4)
i have an array of company name, i insert each company name as separate record.below is the code
<input type="text" name="company_name[]">
$company_name = $_POST['company_name'];
if($company_name)
{
foreach($company_name as $company)
{
$mycompany[] = $company;
}
}
$val="('".implode("'), ('",$mycompany)."')";
$sql = "INSERT INTO `table`
(`company_name`) VALUES ".$val."";
The above query look like this and it successfully inserted 2 records in table.
INSERT INTO `table` (`company_name`) VALUES ('DELL'), ('IBM')
Now the problem is that with every company_name there is a company_code which i want to insert with each record and there is 3rd values lets suppose order_num which i also want to insert but the order_num should be same in all records,i need the query below
INSERT INTO `table` (`order_num`,`company_name`,`company_code`) VALUES ('123','DELL','axc89'), ('123','IBM','bxc90')
Okay, at last you endeavored to produce something understandable
1) This this code is absolutely useless:
$company_name = $_POST['company_name'];
if($company_name)
{
foreach($company_name as $company)
{
$mycompany[] = $company;
}
}
as $mycompany being an exact duplicate of $company_name
2) To get your "very complex" query
foreach($_POST['company_name'] as $key => $value)
{
$name = mysql_real_escape_string($value);
$code = mysql_real_escape_string($_POST['company_code'][$key]);
$mycompany[] = "(123,'$name','$code')";
}
$sql = "INSERT INTO `table` (order_num,company_name,company_code) VALUES ";
$sql .= implode(",",$mycompany);
INSERT INTO `table` (`order_num`,`company_name`,`company_code`) VALUES ('123','DELL','axc89');
INSERT INTO `table` (`order_num`,`company_name`,`company_code`) VALUES ('123','IBM','bxc90');
You can use an array
$arrray_to_be_inserted=array
(
[0]=>array('123','DELL','axc89'),
[1]=>array('123','IBM','bxc90')
);
Instead of passing value to a new variable in foreach , try to access as $key and $value, try this and see if it can help you
<?php
$company_name = array("DEL","IBM");
$company_code = array("1","2");
$order_num = array("4","5");
if($company_name)
{
foreach($company_name as $key => $value)
{
$mycompany[] = $value;
$mycompanycode[] = $company_code[$key];
$myordernum[] = $order_num[$key];
}
}
?>
Just make sure names and their values are corresponding to each other .
Ok so whilst I have been working on my PHP and MySQL skills I am new to inserting data into multiple tables in one go. I have done some reading around the subject and I appreciate the basics and the importance of normalised data etc and as such the need to put the information into various tables.
I have pieced the code below together from some of my previous work as well as the tutorial offered at http://www.desilva.biz/mysql/insertid.html . The issue I currently face is that the tutorial I learnt the code from for inputting into various tables was not based around an array of data and whilst I have got it to almost work I cannot use my fuelrecords_id because where I have to call it in my current code it has not yet been defined. Therefore to make my code work at present I just have to use a comma for the column.
Finally I would like to perfect a way to make the if statements work correctly with the array data so if a 0 or blank is submitted as part of the array a new row is not inserted with just 0's in my database tables for that respective row of data
<?php
$wedrf=trim($_POST['FR_WE']);
list($d, $m, $y) = explode('/', $wedrf);
$mk=mktime(0, 0, 0, $m, $d, $y);
$wed_refor=strftime('%Y-%m-%d',$mk);
$con = mysql_connect("ip","username","password");
if (!$con)
{
die('Could not connect: ' . mysql_error());
}
mysql_select_db("jbsrint", $con);
//New Code
$row_data = array();
foreach($_POST['VEH_LIST_REG'] as $row=>$VEH_LIST_REG) {
$WEDATE=$wed_refor;
$VEH_LIST_REG=mysql_real_escape_string($VEH_LIST_REG);
$FR_DIE_L=mysql_real_escape_string($_POST['FR_DIE_L'][$row]);
$FR_DIE_C=mysql_real_escape_string($_POST['FR_DIE_C'][$row]);
$FR_PET_L=mysql_real_escape_string($_POST['FR_PET_L'][$row]);
$FR_PET_C=mysql_real_escape_string($_POST['FR_PET_C'][$row]);
$FR_OIL_L=mysql_real_escape_string($_POST['FR_OIL_L'][$row]);
$FR_OIL_C=mysql_real_escape_string($_POST['FR_OIL_C'][$row]);
$row_data[] = "(',', '$VEH_LIST_REG', '$WEDATE')";
$row_data1[] = "(',','$FR_DIE_L', '$FR_DIE_C', ',')";
$row_data2[] = "(',', '$FR_PET_L', '$FR_PET_C', ',')";
$row_data3[] = "(',', '$FR_OIL_L', '$FR_OIL_C', '$FR_OIL_C')";
}
if (!empty($row_data)) {
$query = 'INSERT INTO fuelrecords(FR_ID, VEH_LIST_REG, FR_WE) VALUES '.implode(',', $row_data);
#$result = mysql_query( $query);
# get fuelrecord id
$fuelrecords_ID = mysql_insert_id();
# if the user submitted diesel information
if( isset($FR_DIE_L) )
{
# and insert the diesel details
$sql = "INSERT INTO fuelrecords_die(FRD_ID,FR_DIE_L,FR_DIE_C,fuelrecords_ID) VALUES ".implode(',', $row_data1);
$result = mysql_query( $sql);
}
# if the user submitted petrol information
if( isset($FR_PET_L) )
{
# and insert the diesel details
$sql = "INSERT INTO fuelrecords_pet(FRP_ID,FR_PET_L,FR_PET_C,fuelrecords_ID) VALUES ".implode(',', $row_data2);
$result = mysql_query( $sql);
}
# if the user submitted oil information
if( isset($FR_OIL_L) )
{
# and insert the oil details
$sql = "INSERT INTO fuelrecords_oil(FRO_ID,FR_OIL_L,FR_OIL_C,fuelrecords_ID) VALUES ".implode(',', $row_data3);
$result = mysql_query( $sql);
}
if (mysql_query($query))
echo '<font color=\"FFFFFF\" size=\"3px\">Successful inserted</font>';
else
echo '<font color=\"FFFFFF\" size=\"3px\">Insert failed</font>';
}
?>
<?php
mysql_close($con)
?>
Tables are as follows:
fuelrecords
FR_ID (Auto increment)
VEH_LIST_REG
FR_WE
fuelrecords_die
FRD_ID (AUTO INCREMENT)
FR_DIE_L
FR_DIE_C
fuelrecords_ID (foreign ID from fuelrecords)
fuelrecords_pet
FRP_ID (AUTO INCREMENT)
FR_PET_L
FR_PET_C
fuelrecords_ID (foreign ID from fuelrecords)
fuelrecords_oil
FRO_ID (AUTO INCREMENT)
FR_OIL_L
FR_OIL_C
fuelrecords_ID (foreign ID from fuelrecords)
Basically the purpose is to log vehicle fuel usage and cost. As there wont always be data for petrol, diesel and oil hence the separate tables so only needed dat is logged.
Hope this clarifies
A always all help and assistance is much appreciated.
If I understand your code correctly you have 4 tables: fuelrecords, fuelrecords_die, fuelrecords_pet, fuelrecords_oil.
The 3 tables fuelrecords_die, fuelrecords_pet, fuelrecords_oil each have a foreign key fuelrecords_id to fuelrecords.fr_id.
You now want to insert multiple tuples into fuelrecords and, if additional data is provided, multiple tuples into the other 3 tables. I assume the fuelrecords.fr_id column is a auto incrementing primary key.
To insert multiple tuples into fuelrecords and to have them each have a new fr_id, you just don't pass a value for the column fr_id. This is equivalent to passing NULL as value. MySQL will then automatically insert unique consecutive numbers for each tuple.
After that you can call mysql_insert_id() to get the first inserted id. Using mysql_affected_rows() you can get the number of inserted tuples. This is enough information to get the id for all lastly inserted tuples. The first is mysql_insert_id()+0 the second is mysql_insert_id()+1, ..., the last is mysql_insert_id()+(mysql_affected_rows()-1).
In the next step you iterate over your input data again and insert the fuelrecords_id into each of the tuples for the other 3 tables, using the above mentioned method. If $i is the index of your input data $_POST['FR_DIE_L'][$i] (starting at $i==0), the fuelrecords_id will be mysql_insert_id()+$i. You are only allowed to iterate to mysql_insert_id()+mysql_affected_rows()-1, but you will probably have the same count of POST-data anyways.
A much simpler but slightly less efficient way is to just do one insert into fuelrecords and then one insert into the other 3 tables for each single POST-data object. You won't have to calculate the fuelrecords_id as mysql_insert_id() will give you the correct id after each insert.
<?php
$wedrf=trim($_POST['FR_WE']);
list($d, $m, $y) = explode('/', $wedrf);
$mk=mktime(0, 0, 0, $m, $d, $y);
$wed_refor=strftime('%Y-%m-%d',$mk);
$row_data = array();
// shorthand for mysql_real_escape_string
function esc($value) {
return mysql_real_escape_string($value);
}
// all tuples for fuelrecords to be inserted
foreach($_POST['VEH_LIST_REG'] as $row => $VEH_LIST_REG) {
$row_data[] = "(NULL, '".esc($VEH_LIST_REG)."', '".esc($wed_refor)."')";
}
if (!empty($row_data)) {
$query = 'INSERT INTO fuelrecords(FR_ID, VEH_LIST_REG, FR_WE) VALUES '.implode(',', $row_data);
$result = mysql_query($query);
# get first fuelrecord id
$first_fuelrecords_id = mysql_insert_id();
// all tuples for the other 3 tables. insert only if data is givin.
$die_data = array();
$pet_data = array();
$oil_data = array();
foreach($_POST['VEH_LIST_REG'] as $row => $VEH_LIST_REG) {
// calculate the right fuelrecords_id for this tuple
$fuelrecords_id = (int)($first_fuelrecords_id + $row);
// insert for fuelrecords_die
if (isset($_POST['FR_DIE_L'][$row]))
{
$die_data[] = "(NULL, ".$fuelrecords_id.", '".esc($_POST['FR_DIE_L'][$row])."', '".esc($_POST['FR_DIE_C'][$row])."')";
}
// insert for fuelrecords_pet
if (isset($_POST['FR_PET_L'][$row]))
{
$pet_data[] = "(NULL, ".$fuelrecords_id.", '".esc($_POST['FR_PET_L'][$row])."', '".esc($_POST['FR_PET_C'][$row])."')";
}
// insert for fuelrecords_oil
if (isset($_POST['FR_OIL_L'][$row]))
{
$oil_data[] = "(NULL, ".$fuelrecords_id.", '".esc($_POST['FR_OIL_L'][$row])."', '".esc($_POST['FR_OIL_C'][$row])."')";
}
}
// insert the tuples into fuelrecords_die
if (!empty($die_data))
{
$sql = "INSERT INTO fuelrecords_die(FRD_ID, fuelrecords_ID, FR_DIE_L, FR_DIE_C) VALUES ".implode(',', $die_data);
$result = mysql_query( $sql);
}
// insert the tuples into fuelrecords_pet
if (!empty($pet_data))
{
$sql = "INSERT INTO fuelrecords_pet(FRP_ID, fuelrecords_ID, FR_PET_L, FR_PET_C) VALUES ".implode(',', $pet_data);
$result = mysql_query( $sql);
}
// insert the tuples into fuelrecords_oil
if (!empty($oil_data))
{
$sql = "INSERT INTO fuelrecords_oil(FRO_ID, fuelrecords_ID, FR_OIL_L, FR_OIL_C) VALUES ".implode(',', $oil_data);
$result = mysql_query( $sql);
}
}
?>
A small off-topic addition: Try not to use upper case variable names. Upper case identifiers are usually preserved for constants:
define("MY_SHORT_PI", 3.14159265);
define("MY_CONST", "foobar");
$my_variable = "bat";
echo "I am a constant ".MY_SHORT_PI;
echo "Me too ".MY_CONST;
echo "I am a variable ".$my_variable;
This won't have any effect on the PHP interpreter. It's just a common notation to make your code readable for others. There are many style guides out there like the one from PEAR.
Second Example (see comments)
<?php
$wedrf=trim($_POST['FR_WE']);
list($d, $m, $y) = explode('/', $wedrf);
$mk=mktime(0, 0, 0, $m, $d, $y);
$wed_refor=strftime('%Y-%m-%d',$mk);
// VALUES strings for fuelrecords
$row_data = array();
// temporary storage for just _L and _C values
$die_data_tmp = array();
$pet_data_tmp = array();
$oil_data_tmp = array();
// VALUES strings for the three tables
$die_data = array();
$pet_data = array();
$oil_data = array();
// shorthand for mysql_real_escape_string
function esc($value) {
return mysql_real_escape_string($value);
}
// all tuples for fuelrecords to be inserted
foreach($_POST['VEH_LIST_REG'] as $row => $VEH_LIST_REG) {
// check if diesel values are greater than 0
if (0 < (int)$_POST['FR_DIE_L'][$row] && 0 < (int)$_POST['FR_DIE_C'][$row])
$die_data_tmp[$row] = array($_POST['FR_DIE_L'][$row], $_POST['FR_DIE_C'][$row]);
// check if petrolium values are greater than 0
if (0 < (int)$_POST['FR_PET_L'][$row] && 0 < (int)$_POST['FR_PET_C'][$row])
$pet_data_tmp[$row] = array($_POST['FR_PET_L'][$row], $_POST['FR_PET_C'][$row]);
// check if oil values are greater than 0
if (0 < (int)$_POST['FR_OIL_L'][$row] && 0 < (int)$_POST['FR_OIL_C'][$row])
$oil_data_tmp[$row] = array($_POST['FR_OIL_L'][$row], $_POST['FR_OIL_C'][$row]);
// check if at least one of the 3 tables will get tuples. if not just continue
// with the next and don't assign this fuelrecord tuple to $row_data
if (! isset($die_data_tmp[$row]) && ! isset($pet_data_tmp[$row]) && ! isset($oil_data_tmp[$row]))
continue;
// all values are at least 1, so add this tuple to our inserts
$row_data[$row] = "(NULL, '".esc($VEH_LIST_REG)."', '".esc($wed_refor)."')";
}
if (!empty($row_data)) {
$query = 'INSERT INTO fuelrecords(FR_ID, VEH_LIST_REG, FR_WE) VALUES '.implode(',', $row_data);
$result = mysql_query($query);
# get first fuelrecord id
$current_fuelrecords_id = mysql_insert_id();
// all tuples for the other 3 tables. insert only if data is givin.
foreach($row_data as $row => $VEH_LIST_REG) {
// insert for fuelrecords_die
if (isset($die_data_tmp[$row]))
{
$die_data[] = "(NULL, ".$current_fuelrecords_id.", '".esc($die_data_tmp[$row][0])."', '".esc($die_data_tmp[$row][1])."')";
}
// insert for fuelrecords_pet
if (isset($pet_data_tmp[$row]))
{
$pet_data[] = "(NULL, ".$current_fuelrecords_id.", '".esc($pet_data_tmp[$row][0])."', '".esc($pet_data_tmp[$row][1])."')";
}
// insert for fuelrecords_oil
if (isset($oil_data_tmp[$row]))
{
$oil_data[] = "(NULL, ".$current_fuelrecords_id.", '".esc($oil_data_tmp[$row][0])."', '".esc($oil_data_tmp[$row][1])."')";
}
// increment the fuelrecords_id for the next tuple.
++$current_fuelrecords_id;
}
// insert the tuples into fuelrecords_die
if (!empty($die_data))
{
$sql = "INSERT INTO fuelrecords_die(FRD_ID, fuelrecords_ID, FR_DIE_L, FR_DIE_C) VALUES ".implode(',', $die_data);
$result = mysql_query( $sql);
}
// insert the tuples into fuelrecords_pet
if (!empty($pet_data))
{
$sql = "INSERT INTO fuelrecords_pet(FRP_ID, fuelrecords_ID, FR_PET_L, FR_PET_C) VALUES ".implode(',', $pet_data);
$result = mysql_query( $sql);
}
// insert the tuples into fuelrecords_oil
if (!empty($oil_data))
{
$sql = "INSERT INTO fuelrecords_oil(FRO_ID, fuelrecords_ID, FR_OIL_L, FR_OIL_C) VALUES ".implode(',', $oil_data);
$result = mysql_query( $sql);
}
}
?>