Efficient way to check and insert data into MySQL database node - php

First Way:
Fetch all the records from the database and create a result array.
Loop the new inserting array and check the unique id is exist or not with the above result array then inserting to the DB.
Second Way:
Loop the new inserting array and check the unique id is exist or not in the database then insert into the database.
Note: Inserting data will be very less compared to the database table data.
Please suggest any best way to do it.


An implemetation of #NigelRen idea
insert into destTable (id, coln,...)
select st.id,st.coln,...
from srcTable st
left join destTable dt on st.id=dt.id
where dt.id is null
where id is the relation one-to-one between the two tables

Related

How to record data derived from COUNT DISTINCT query?

how to record data (into the database) derived from COUNT DISTINCT query?
For example,
Table1
AA
BB
CC
BB
BB
Then the COUNT DISTINCT query outputs the following data 131 successively based on table1.
How to automatically create a table for 131 output?

You seem to be looking for MySQL's CREATE TABLE ... SELECT Statement:
create table mydistincttable as
select ... -- your actual query goes here
MySQL creates a new table with one column for each column returned by your query, (with equivalent datatypes) and fills it with the results of the query. The names of the columns of the created table follow those returned by the query.
As you would expect, that the created table has no indexes, constraints, or the-like - because such information cannot be inferred from a query. If you want such features, you need to add them manually after the table is created.

that can be done with mysql VIEW
CREATE VIEW view_name AS your_query ;

MySQL migrate data from a table with a set field

I have a table with a set column that contains 2 languages and another column on this table containing an order.
Is it possible migrate my table and to create a record in a new table for each set item with a MySQL query?
Directly migrating records works with records that have a set column without a language combination. But if a record would contain a set of languages I am left with one record with an empty column for language. I would preferably want 2 new records inserted into my fresh table.
One for each language in the set.
INSERT INTO newtable (
newtable.oldtable_id,
newtable.language,
newtable.order
)
SELECT oldtable.id, oldtable.languages, oldtable.order
FROM oldtable
WHERE oldtable.order IS NOT NULL;
I doubt it is possible but I would still like to know if somebody might have a workaround for this since I need to migrate the data somehow.

Create a helper table with one field that lists all languages that can be in the set column. The table can be a temporary one that you drop after the migration. Then create an insert ... select ... statement in which you join the source table on the helper table using MySQL's find_in_set() function:
INSERT INTO newtable (
newtable.oldtable_id,
newtable.language,
newtable.order
)
SELECT oldtable.id, helpertable.language, oldtable.order
FROM oldtable inner join helpertable ON find_in_set(helpertable.language, oldtable.languages)>0;

php mysql - insert items if not exist

I have a PHP array and I want to insert from this array the items that are not already in MySQL table. After searching i could do this item by item using the following code
INSERT INTO `rtusers` (`Status`, `MSISDN`)
SELECT * FROM (SELECT 0, '966111111111') AS tmp
WHERE NOT EXISTS (
SELECT `MSISDN` FROM rtusers
WHERE MSISDN = '966111111111' ) LIMIT 1;
but the problem is i have hundreds of items in the array. if i used this code as a loop this will make hundreds of hits to the database. Does anybody have an easier solution?

You should probably load the array directly into a temporary table of some sort. You can do this with individual insert statements or using load data infile or some other bulk load mechanism.
Then to ignore the record, define a unique index on rtusers(mission) and use:
insert into rtusers(status, mission)
select status, mission
from rtusers_staging
on duplicate key update mission = values(mission);
The on duplicate key part doesn't do anything. It just ignores any duplicate records.

Create a table with new numbers names new_users then execute this query:
INSERT INTO `rtusers` (`Status`, `MSISDN`)
SELECT Status, MSISDN from new_users
WHERE MSISDN NOT IN (SELECT MSISDN FROM rtusers);

transforming large imports of data

I have a question on the best way to import a long csv table and then transform into two tables for efficiency.
This is a very scaled down version but for this purpose suits:
each row is a unique record but the first three columns consist of very consistant data over a large amount of rows.
I figured the best way to manage this data was to build two tables:
The first being an auto increment id field and a group by of the first three columns.
This gives a nice compact table of the main groupings of my data.
The second table was to be every row but instead of holding all the repeated data I hold only the variable data columns d, e and f along with the autoincrement id field i generate when importing into the first table.
My question is really how do I get the id from the first table - is my only way to requery that table to find the id and then do the insert into the second table?
a,b,c,d,e,f
09/02/2013,A1,1,18503112043123,11,2.1219
09/02/2013,A1,1,44102576116476,73,14.0817
09/02/2013,A1,1,66918345446536,134,25.8486
09/02/2013,A1,2,62009507978229,10,1.929
09/02/2013,A1,2,92278593945574,55,10.6095
09/02/2013,B1,1,50474606002324,90,17.361
09/02/2013,B1,1,59697581427675,7,1.3503
09/02/2013,B1,1,86298530583467,51,9.8379
09/02/2013,B1,2,34885481077847,80,15.432
09/02/2013,B1,2,25479347211047,164,31.6356
09/02/2013,B1,3,56270556524425,6,1.1574
09/02/2013,C1,1,57680166803098,24,4.6296
09/02/2013,C1,1,72778510788287,77,14.8533
09/02/2013,C1,1,26084111080146,140,27.006
09/02/2013,C1,1,31435464483578,361,65.5937
09/02/2013,C1,2,29457756254473,492,89.3964
09/02/2013,C1,2,68414218104066,293,53.2381
EDIT
I have two queries in mind:
1: My parent table which has an auto increment
insert into parent_table
select null,a,b,c
from table
group by a,b,c
My child table which is all my rows of data but includes corresponding auto increment id from the parent table.
I dont understand how to pull the id back again without doing a query back to the parent table as i input the data into the child table

You can use PDO::lastInsertId or mysqli::$insert_id to retrieve the
auto generated id used in the last query.
Just do the insert and then fetch the id
$sth = $pdo->prepare("insert into first_table (a, b, c) values (?, ?, ?)");
$sth->execute(array('2013-02-09', 'A1', 1));
$id = $pdo->lastInsertId();
There is also the MySQL LAST_INSERT_ID(). You could test
insert into second_table (first_table_id, d, e, f) values (LAST_INSERT_ID(), ...)
but I have never tried this myself.

How do I insert data while updating?

There are two tables called table1 and table2
both table has got same index. when updating table1 and table2 if table2 hasnt got a data and table1 has got the same then it should insert that data to table2. so that both tables got the data and is updated. how can I do this??

create a trigger on table 1 to query for the new row in table 2. if not found, insert.

showing code would help, but one option is to use REPLACE() it will add if needed to replace otherwise, if you really need two identical tables you should look in to replication and stored procedures

Categories