Merely due to the sake of being efficient I would like to hear some advises from the PHP expers here. (Actually this question goes out all developers that mess with database connections on a regular basis)
Assume you have one personId and this id has multiple numbers. You have a table with columns pID,nums
From the php side you retrieve these numbers in an array. Now my question comes to place. Do you do something like;
for($i=0;$i<count($arr);$i++)
{
//Call the insert query over and over again
}
Or there is obviously a better solution?
Essentially you want to do a bulk insert - it's faster to execute one composite statement, than lots of individual inserts, so try:
$data is your array
$sql = array();
foreach( $data as $row ) {
$sql[] = '("'.mysql_real_escape_string($row['field1']).'")';
}
mysql_query('INSERT INTO table (field) VALUES '.implode(',', $sql));
If your data set is really large, you can look at dumping your values into a file, and then executing "LOAD DATA INFILE" (will only work if MySQL runs on the same host as your script).
Alternatively, you can use one insert query:
INSERT INTO table(pid,nums) VALUES (1,2), (1,3), (1,4), ... -- etc
with multiple values.
Related
I have array like this:
$array = array("AAA,http://aaa.com,bbb,http://bbb.com,ccc,http://ccc.com");
How can I take this array and insert into database like this:
No name url
1. AAA. Http://aaa.com
2. BBB. Http://bbb.com
3. CCC. Http://ccc.com
Using PHP and MySQL.
Thank you.
Assembling one INSERT statement with multiple rows is much faster in MySQL than one INSERT statement per row.
That said, it sounds like you might be running into string-handling problems in PHP, which is really an algorithm problem, not a language one. Basically, when working with large strings, you want to minimize unnecessary copying. Primarily, this means you want to avoid concatenation. The fastest and most memory efficient way to build a large string, such as for inserting hundreds of rows at one, is to take advantage of the implode() function and array assignment.
$sql = array();
foreach( $data as $row ) {
$sql[] = '("'.mysql_real_escape_string($row['text']).'", '.$row['category_id'].')';
}
mysql_query('INSERT INTO table (text, category) VALUES '.implode(',', $sql));
The advantage of this approach is that you don't copy and re-copy the SQL statement you've so far assembled with each concatenation; instead, PHP does this once in the implode() statement. This is a big win.
If you have lots of columns to put together, and one or more are very long, you could also build an inner loop to do the same thing and use implode() to assign the values clause to the outer array.
please try and good luck
I have a form where the user is supposed to input data (name, streetname, streetnumber) and then i want to check if that data exists in the table (name, streetname+streetnumber) so i can use that row instead of making a new one.
Is there an easy way to do this or will i have to fetch the data into an array and search that somehow?
I have this code:
$sql = "INSERT INTO 725G54_proj_address VALUES (NULL,'$streetname','$streetnumber')";
$result1 = mysql_query($sql,$con);
$address=mysql_insert_id();
echo "ID of last inserted record is: " . $address;
echo "<br>";
$sql = "INSERT INTO 725G54_proj_contact VALUES ('Null','$name','$telephonenumber')";
$result2 = mysql_query($sql,$con);
$contact = mysql_insert_id();
echo "ID of last inserted record is: " . $contact;
$sql = "INSERT INTO 725G54_proj_LivesAt VALUES ('$contact','$address')";
$result3 = mysql_query($sql,$con);`enter code here`
What i would like to is is find out if example Mainstreet 31 allready exists in proj_address and if it does, use its autoincremented id for the insert into proj_Lives_at.
Edit:
I decided to go with sending queries to check if the data existed.
I load that into a variable and check if it is empty, but there is some problem so that "if(empty)" doesn't work as it should. the other way around ("if(!empty)") works however.
the query and loading look like this:
$sql2="SELECT 725G54_proj_address.Address_ID, GROUP_CONCAT( cast( concat( 725G54_proj_address.Street_Name, ' ', 725G54_proj_address.Street_Number ) AS char ) SEPARATOR ', ') AS addresses
FROM 725G54_proj_address
WHERE 725G54_proj_address.Street_Name = '$streetname' AND 725G54_proj_address.Street_Number= '$streetnumber'
GROUP BY 725G54_proj_address.Address_ID
ORDER BY 725G54_proj_address.Address_ID ASC";
$result2=mysql_query($sql2);
Edit 2:
I have tried switching to using count() instead of empty() and it found the variable as populated even when it shouldnt.
It sounds like you are creating a many-to-one relationship (potentially) and should therefore be storing that data in a separate lookup table. What I suggest in that case is to first try to insert that row using an "INSERT IGNORE" statement
(http://dev.mysql.com/doc/refman/5.5/en/insert.html) and then use mysql_affected_rows() to see if it made a change. If it didn't then you know there's a duplicate row, which you can select, take the ID of, and then you're done.
As somebody mentioned in one of the comments above, you can make that into a stored process.
Assuming you've created proper keys on your table (i.e., a unique key across (name, streetname, streetnumber), or something similar), then MySQL INSERT...ON DUPLICATE KEY UPDATE, or possibly REPLACE INTO, might be useful here. The former will convert your INSERT statement into an UPDATE if the key to be inserted already exists in the table, and the latter will act as regular INSERT unless the key already exists, in which case it will first DELETE the existing row before INSERTing the new data.
Of course, it's not exactly an onerous burden simply to SELECT COUNT(*) AS n_existing FROM table WHERE..., and then inspect n_existing in the result set to see whether a row already exists with the same key you're preparing to INSERT, and there is an argument to be made from the principle of least astonishment that it would be better to SELECT before INSERTing than to use abstruse SQL statements which may not have been encountered before by anyone else who's going to be maintaining your code. Still, the option exists, should you prefer it.
This is what i am doing now : - in PHP
foreach($array as $value)
{
$query = select abc from tblname where colname =" .$value.
// fire query
}
then i create array of these values and display accordingly.
The PROBLEM: -
I have applied foreach, which fires the query every time it encounters a value in the array.
result, if i have 10 values in my array it fires 10 queries. and uses network 10 times, result slow output.
What i want -
I want to give the array to a stored procedure which shall give me a resultset which will have the outputs corresponding to all the elements in the array.
I know this can be done but do not know how.
the mysql doesnot take arrays as datatype.
the result shall be that network shall be used only once, despit of any number of values in the array.
LIKE -
StoredProcedure(inputMysqlARRAY) // not possible, need a workaroung
{
// fire simple select(same) query for each value.
}
then call this stored procedure from PHP and input array. // need workaround.
You just have to be smarter about your calls. For instance, keeping cached DB objects around and that sort of thing.
Without knowing more about your code (your question is fairly garbled), it seems that if your query is something like this:
$query = "select abc from tblname where colname =" .$value; // run 10 times.
You really just need to write smarter code:
$values = array(); // Now, populate this array.
// When you're done, run the query:
$query = 'select abc from tblname where colname IN (\''.implode('\',\'', $values).'\')';
Generally, we refer to this as Dynamic SQL and is the underpinning for how things are typically done today. A stored procedure (or, based on how I read your question, stored function) is useful at times, but is somewhat antiquated as a first-order methodology for interfacing with SQL. The DB guys still sometimes swear by it, but I think that even they are fairly well in consensus that smarter queries are always better.
Hey, I have a table with "id", "name", and "weight" columns. Weight is an unsigned small int.
I have a page that displays items ordered by "weight ASC". It'll use drag-n-drop, and once the order is changed, will pass out a comma-separated string of ids (in the new order).
Let's say there's 10 items in that table. Here's what I have so far:
Sample input:
5,6,2,9,10,4,8,1,3,7
Sample PHP handler (error handlers & security stuff excluded):
<?php
$weight = 0;
$id_array = explode(',', $id_string);
foreach ($id_array as $key => $val)
{
mysql_query("UPDATE tbl SET weight = '$weight' where id = '$val' LIMIT 1");
$weight++;
}
?>
When I make a change to column order, will my script need to make 10 separate UPDATE queries, or is there a better way?
You could create a temporary table with the new data in it (i.e., id and weight are the columns), then update the table with this data.
create temporary table t (id int, weight float);
insert into t(id, weight) values (1, 1.0), (2, 27), etc
update tbl inner join t on t.id = tbl.id
set tbl.weight = t.weight;
So, you have one create statement, one insert statement, and one update statement.
You can only specify one where clause in a single query -- which means, in your case, that you can only update one row at a time.
With 10 items, I don't know if I would go through that kind of troubles (it means re-writing some code -- even if that's not that hard), but, for more, a solution would be to :
delete all the rows
inserts them all back
doing all that in a transaction, of course.
The nice point is that you can do several inserts in a single query ; don't know for 10 items, but for 25 or 50, it might be quite nice.
Here is an example, from the insert page of the MySQL manual (quoting) :
INSERT statements that use VALUES
syntax can insert multiple rows. To do
this, include multiple lists of column
values, each enclosed within
parentheses and separated by commas.
Example:
INSERT INTO tbl_name (a,b,c) VALUES(1,2,3),(4,5,6),(7,8,9);
Of course, you should probably not insert "too many" items in a single insert query -- an insert per 50 items might be OK, though (to find the "right" number of items, you'll have to benchmark, I suppose ^^ )
Yes, you would need to do 10 updates. There are ways to batch up multiple queries in a single call to mysql_query, but it's probably best to avoid that.
If it's performance you are worried about, make sure you try it first before worrying about that. I suspect that doing 10 (or even 20 or 30) updates will be plenty fast.
10 updates is the simplest way conceptually. if you've got a bazillion rows that need to be updated, then you might have to try something different, such as creating a temporary table and using a JOIN in your UPDATE statement or a subquery with a row constructor.
Store the records in a temp table with batch insert and delete the records from the tbl and then from temp table do batch insert in tbl
Say I have an array of strings in a php array called $foo with a few hundred entries, and I have a MySQL table 'people' that has a field named 'name' with a few thousand entries. What is an efficient way to find out which strings in $foo aren't a 'name' in an entry in 'people' without submitting a query for every string in $foo?
So I want to find out what strings in $foo have not already been entered in 'people.'
Note that it is clear that all of the data will have to be on one box at one point. The goal would be doing this at the same time minimizing the number of queries and the amount of php processing.
I'd put your $foo data in another table and do a LEFT OUTER JOIN with your names table. Otherwise, there aren't a lot of great ways to do this that don't involve iteration at some point.
The best I can come up with without using a temporary table is:
$list = join(",", $foo);
// fetch all rows of the result of
// "SELECT name FROM people WHERE name IN($list)"
// into an array $result
$missing_names = array_diff($foo, $result);
Note that if $foo contains user input it would have to be escaped first.
What about the following:
Get the list of names that are already in the db, using something like:
SELECT name FROM people WHERE name IN (imploded list of names)
Insert each item from the return of array_diff()
If you want to do it completely in SQL:
Create a temp table with every name in the PHP array.
Perform a query to populate a second temp table that will only include the new names.
Do an INSERT ... SELECT from the second temp table into the people table.
Neither will be terribly fast, although the second option might be slightly faster.
CREATE TEMPORARY TABLE PhpArray (name varchar(50));
-- you can probably do this more efficiently
INSERT INTO PhpArray VALUES ($foo[0]), ($foo[1]), ...;
SELECT People.*
FROM People
LEFT OUTER JOIN PhpArray USING (name)
WHERE PhpArray.name IS NULL;
For a few hundred entries, just use array_diff() or array_diff_assoc()
$query = 'SELECT name FROM table WHERE name != '.implode(' OR name != '. $foo);
Yeash, that doesn't look like it would scale well at all.
I'm not sure there is a more efficient way to do this other than to submit all the strings to the database.
Basically there are two options: get a list of all the strings in MySQL and pull them into PHP and do the comparisons, or send the list of all the strings to the MySQL server and let it do the comparisons. MySQL is going to do the comparisons much faster than PHP, unless the list in the database is a great deal smaller than the list in PHP.
You can either create a temporary table, but either way your pushing all the data to the database.