I have two tables.
first datatable structure has nine columns but the important three are:
code | name | value
2D3 | name test | 0.12
the second table has the same three columns.
Now I want to update all rows of the first table with the values of table two where code AND name are the same as in table two.
So my current idea is to do a select of all rows of table 1 with the code and name columns, than check if a row with the same code and name exists in table 2, if yes get that value and do a UPDATE query in table 1 with value of table 2.
The problem is that the two tables are hugh and I am sure that I am not using the fastest method.
anyone an idea for the fastest method to do this? Thank you very much!
EDIT: the query:
$getall = "SELECT code, name, value FROM table2";
$query = mysqli_query($conn, $getall );
while($result = mysqli_fetch_array($query))
{
$UpdateAll = "UPDATE table1 SET value = '".mysqli_real_escape_string($conn,$result["value"])."' WHERE name = '".mysqli_real_escape_string($conn,$result["name"])."' AND code = '".mysqli_real_escape_string($conn,$result["code"])."'";
$result2 = mysqli_query($conn, $UpdateAll); }
You speak of two databases but really mean two tables, don't you? In this case, you can do it with one update using a join, like this:
update table1 t1
inner join table2 t2 on t2.code = t1.code and t2.name = t1.name
set t1.value = t2.value;
We have two tables with same structure (MSSQL)
We want to compare
(ID) 122 from table1
with
highest ID 122 from table2
If there is one or more differences, we want to show the name of the column along with the data it holds from both tables.
I do not want to list all the columns out, as there is over 150
Something like (psuedo): -
Select * from table1
Select * from table2
Compare table1 (column by column) with table2 (column by column)
Show changes : -
'Name' WAS 'John' NOW 'James' CHANGED 3/11/2016 3.45pm
'Mobilenum" WAS '02373643743' NOW '0983783738' CHANGED 4/11/2016 12.46pm
Do not want to try triggers...need php code....
We should not need to give any column names as it should run though them all dynamically...
Why do you not want to list the columns - Is this table going to change regularly?
You can still create the SQL dynamically - but store it as a static string.
Lots of examples for this (hint use information_schema.columns)
If the tables are not huge then I would p[rob write a bunch of Tests (1-per column) and UNION All them together - Then only return the Data where diff = true
Eg
Select * from
(
Select case when ISNULL(table1.columnA, -999) <> ISNULL(table2.columnA, -999) then 1 else 0 end as diff, table1.columna as Old_Val, table2.columnA as new_val
from table1 join table2 on ...
UNION ALL
/* etc etc * 150 ... */
) as x
where x.diff = 1
I am stuck to update one column of table by comparing with another table in php/Mysql. I have tried to speed up the process by indexing the table columns, optimizing the query etc but unable to speed up the process.
In my php based application there is two table (table A and table B) , I want to update one column of table A by comparing with table B (with two column - name & sku).
Previously above process has taken max 15 mints to update 28k products. But now both table (table A and table B) have 60k rows. Now it's taking more than two hours. I have used below query
mysql_query("UPDATE tableA a
JOIN tableB b ON a.product_code_sku = b.sku
SET a.is_existing_product = '1'") or die(mysql_error());
mysql_query("UPDATE tableA a
JOIN tableB b ON a.product_name = b.product_name
SET a.is_existing_product = '1'") or die(mysql_error());
Above query was very slow after that I have changed the updating process like below
$query_result = mysql_query("SELECT t1.`id`,t2.`product_id` FROM `tableA` t1,
`tableB` t2 where (t1.product_code_sku = t2.sku
or t1.product_name = t2.product_name)") or die (mysql_error());
while($result_row = mysql_fetch_array($query_result))
{
mysql_query("UPDATE `tableA` SET is_existing_product = '1'
where id = '".$result_row['id']."' ") or die (mysql_error());
}
But all of my efforts are in vain.
Please advice me how to make the process faster.
Your first update query and the second update query is doing two different thing. The second query is slower because you are using a OR for comparison.
You can consider to create a temporary table to compare and insert, the update back to tableA.
First and all, you should examine the execution for the two join queries, like
desc select a.id
from tableA a
join tableB b ON a.product_code_sku = b.sku;
If this is the reason why the update is slow, you should optimize the query.
Otherwise, you can try the below:
For instance (assuming ID the primary key),
// make sure the columns are in the same data type
create table tmp_sku (
id .. // just the primary key, make sure is using the same data type as in tableA
);
// do a insert into this temporary table
insert into tmp_sku select a.id
from tableA a
join tableB b ON a.product_code_sku = b.sku;
// now we have list of matches,
// then do a insert .. duplicate key update
// by comparing the primary id
insert into tableA (id, is_existing_product)
select tmp_sku.id, 1 from tmp_sku
on duplicate key set is_existing_product = 1;
// repeat for the product name
truncate tmp_sku;
insert into tmp_sku
select a.id
from tableA a
join tableB b ON a.product_name = b.product_name;
// repeat the duplicate .. update
insert into tableA (id, is_existing_product)
select tmp_sku.id, 1 from tmp_sku
on duplicate key set is_existing_product = 1;
Any idea how to copy: name, content from rows where language_id = 1 to rows where language_id = 2?
How should SQL command look like?
I want to achive:
http://dev.mysql.com/doc/refman/5.0/en/insert-select.html is what you need to do
assuming it is the productid that you want to update from lang1 to lang 2
update a set
a.name = b.name,
a.content = b.content
from tablea a
join tablea b on a.productid = b.productid
where a.language_id = 2
and b.language_id = 1
ofcourse this will do it for every row in the table so if you want to restrict it then make sure to restrict it by the productids
Did you mean copying all language_id=1 rows to language_id=2 ones?
My knowledge of MySQL syntax is very poor, so I dare not give you all the codez, but at least you may find the following approach useful:
Create a temp table with the structure like this:
product_id int,
name (varchar?)
content (varchar?)
That is, include product_id and all the columns you need to copy.
Populate the temp table with the language_id=1 data. Probably like this:
INSERT INTO temp_table
SELECT product_id, name, content
FROM orig_table
WHERE language_id = 1
Update those rows in the original table where language_id=2 with the corresponding data in the temp table. It may look like this:
UPDATE orig_table
SET
name = temp_table.name,
content = temp_table.content
FROM temp_table
WHERE orig_table.product_id = temp_table.product_id
AND orig_table.language_id = 2
Insert the rows from the temp table into the original table, where the products don't have language_id=2. Something like this:
INSERT INTO orig_table (product_id, language_id, name, content)
SELECT product_id, 2, name, content
FROM temp_table
WHERE NOT EXISTS (
SELECT 1 FROM orig_table
WHERE product_id = temp_table.product.id
AND language_id = 2
)
If you didn't mean to change the already existing language_id=2 data, then step #3 should be omitted and you might further want to modify step #2 in such a way that it selected language_id=1 data only for the products lacking language_id=2.
Is there anyway I can erase all the duplicate entries from a certain table (users)? Here is a sample of the type of entries I have. I must say the table users consists of 3 fields, ID, user, and pass.
mysql_query("DELETE FROM users WHERE ???") or die(mysql_error());
randomtest
randomtest
randomtest
nextfile
baby
randomtest
dog
anothertest
randomtest
baby
nextfile
dog
anothertest
randomtest
randomtest
I want to be able to find the duplicate entries, and then delete all of the duplicates, and leave one.
You can solve it with only one query.
If your table has the following structure:
CREATE TABLE `users` (
`id` int(10) unsigned NOT NULL auto_increment,
`username` varchar(45) NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=MyISAM AUTO_INCREMENT=8 DEFAULT CHARSET=latin1;
you could do something like that (this will delete all duplicate users based on username with and ID greater than the smaller ID for that username):
DELETE users
FROM users INNER JOIN
(SELECT MIN(id) as id, username FROM users GROUP BY username) AS t
ON users.username = t.username AND users.id > t.id
It works and I've already use something similar to delete duplicates.
You can do it with three sqls:
create table tmp as select distinct name from users;
drop table users;
alter table tmp rename users;
This delete script (SQL Server syntax) should work:
DELETE FROM Users
WHERE ID NOT IN (
SELECT MIN(ID)
FROM Users
GROUP BY User
)
I assume that you have a structure like the following:
users
-----------------
| id | username |
-----------------
| 1 | joe |
| 2 | bob |
| 3 | jane |
| 4 | bob |
| 5 | bob |
| 6 | jane |
-----------------
Doing the magic with temporary is required since MySQL cannot use a sub-select in delete query that uses the delete's target table.
CREATE TEMPORARY TABLE IF NOT EXISTS users_to_delete (id INTEGER);
INSERT INTO users_to_delete (id)
SELECT MIN(u1.id) as id
FROM users u1
INNER JOIN users u2 ON u1.username = u2.username
GROUP BY u1.username;
DELETE FROM users WHERE id NOT IN (SELECT id FROM users_to_delete);
I know the query is a bit hairy but it does the work, even if the users table has more than 2 columns.
You need to be a bit careful of how the data in your table is used. If this really is a users table, there is likely other tables with FKs pointing to the ID column. In which case you need to update those tables to use ID you have selected to keep.
If it's just a standalone table (no table reference it)
CREATE TEMPORARY TABLE Tmp (ID int);
INSERT INTO Tmp SELECT ID FROM USERS GROUP BY User;
DELETE FROM Users WHERE ID NOT IN (SELECT ID FROM Tmp);
Users table linked from other tables
Create the temporary tables including a link table that holds all the old id's and the respective new ids which other tables should reference instead.
CREATE TEMPORARY TABLE Keep (ID int, User varchar(45));
CREATE TEMPORARY TABLE Remove (OldID int, NewID int);
INSERT INTO Keep SELECT ID, User FROM USERS GROUP BY User;
INSERT INTO Remove SELECT u1.ID, u2.ID FROM Users u1 INNER JOIN Keep u2 ON u2.User = u1.User WHERE u1.ID NOT IN (SELECT ID FROM Users GROUP BY User);
Go through any tables which reference your users table and update their FK column (likely called UserID) to point to the New unique ID which you have selected, like so...
UPDATE MYTABLE t INNER JOIN Remove r ON t.UserID = r.OldID
SET t.UserID = r.NewID;
Finally go back to your users table and remove the no longer referenced duplicates:
DELETE FROM Users WHERE ID NOT IN (SELECT ID FROM Keep);
Clean up those Tmp tables:
DROP TABLE KEEP;
DROP TABLE REMOVE;
A very simple solution would be to set an UNIQUE index on the table's column you wish to have unique values. Note that you subsequently cannot insert the same key twice.
Edit: My mistake, I hadn't read that last line: "I want to be able to find the duplicate entries".
I would get all the results, put them in an array of IDs and VALUES. Use a PHP function to work out the dupes, log all the IDs in an array, and use those values to delete the records.
I don't know your db schema, but the simplest solution seems to be to do SELECT DISTINCT on that table, keep the result in a variable (i.e. array), delete all records from the table and then reinsert the list returne by SELECT DISTINCT previously.
The temporary table is an excellent solution, but I'd like to provide a SELECT query that grabs duplicate rows from the table as an alternative:
SELECT * FROM `users` LEFT JOIN (
SELECT `name`, COUNT(`name`) AS `count`
FROM `users` GROUP BY `name`
) AS `grouped`
WHERE `grouped`.`name` = `users`.`name`
AND `grouped`.`count`>1
Select your 3 columns as per your table structure and apply condition as per your requirements.
SELECT user.userId,user.username user.password FROM user As user
GROUP BY user.userId, user.username
HAVING (COUNT(user.username) > 1));
Every answer above and/or below didn't work for me, therefore I decided to write my own little script. It's not the best, but it gets the job done.
Comments are included throughout, but this script is customized for my needs, and I hope the idea helps you.
I basically wrote the database contents to a temp file, called the temp file, applied the function to the called file to remove the duplicates, truncated the table, and then input the data right back into the SQL. Sounds like a lot, I know.
If you're confused as to what $setprofile is, it's a session that's created upon logging into my script (to establish a profile), and is cleared upon logging out.
<?php
// session and includes, you know the drill.
session_start();
include_once('connect/config.php');
// create a temp file with session id and current date
$datefile = date("m-j-Y");
$file = "temp/$setprofile-$datefile.txt";
$f = fopen($file, 'w'); // Open in write mode
// call the user and pass via SQL and write them to $file
$sql = mysql_query("SELECT * FROM _$setprofile ORDER BY user DESC");
while($row = mysql_fetch_array($sql))
{
$user = $row['user'];
$pass = $row['pass'];
$accounts = "$user:$pass "; // the white space right here is important, it defines the separator for the dupe check function
fwrite($f, $accounts);
}
fclose($f);
// **** Dupe Function **** //
// removes duplicate substrings between the seperator
function uniqueStrs($seperator, $str) {
// convert string to an array using ' ' as the seperator
$str_arr = explode($seperator, $str);
// remove duplicate array values
$result = array_unique($str_arr);
// convert array back to string, using ' ' to glue it back
$unique_str = implode(' ', $result);
// return the unique string
return $unique_str;
}
// **** END Dupe Function **** //
// call the list we made earlier, so we can use the function above to remove dupes
$str = file_get_contents($file);
// seperator
$seperator = ' ';
// use the function to save a unique string
$new_str = uniqueStrs($seperator, $str);
// empty the table
mysql_query("TRUNCATE TABLE _$setprofile") or die(mysql_error());
// prep for SQL by replacing test:test with ('test','test'), etc.
// this isn't a sufficient way of converting, as i said, it works for me.
$patterns = array("/([^\s:]+):([^\s:]+)/", "/\s++\(/");
$replacements = array("('$1', '$2')", ", (");
// insert the values into your table, and presto! no more dupes.
$sql = 'INSERT INTO `_'.$setprofile.'` (`user`, `pass`) VALUES ' . preg_replace($patterns, $replacements, $new_str) . ';';
$product = mysql_query($sql) or die(mysql_error()); // put $new_str here so it will replace new list with SQL formatting
// if all goes well.... OR wrong? :)
if($product){ echo "Completed!";
} else {
echo "Failed!";
}
unlink($file); // delete the temp file/list we made earlier
?>
This will work:
create table tmp like users;
insert into tmp select distinct name from users;
drop table users;
alter table tmp rename users;
If you have a Unique ID / Primary key on the table then:
DELETE FROM MyTable AS T1
WHERE MyID <
(
SELECT MAX(MyID)
FROM MyTable AS T2
WHERE T2.Col1 = T1.Col1
AND T2.Col2 = T1.Col2
... repeat for all columns to consider duplicates ...
)
if you don't have a Unique Key select all distinct values into a temporary table, delete all original rows, and copy back from temporary table - but this will be problematic if you have Foreign Keys referring to this table