I have to write a script that updates some mysql tables. For this purpose I am provided with a .dbf file that contains the up-to-date data. So what I am doing is:
Convert .dbf file to .sql file (using this script by xtranophilist )
Extract mysql statements from .sql file and execute them (Creating mysql table "temp" and filling it with data)
Get data from freshly created table ("temp") where column tablenr = '1' and check for each row if row exists in other table ("data_side1")
1.and 2. is working so far, now I am wondering how to do 3:
How do you check if some row exists in some table in MySQL via PHP? And what is the best way to do so?
You can use this:
SELECT EXISTS(SELECT 1 FROM data_side1 WHERE ...)
For more details please check http://dev.mysql.com/doc/refman/5.7/en/exists-and-not-exists-subqueries.html
Related
I have a scheduled job that will parse the following CSV file, and if the server is already in the table execute an update, insert if not. (The server name is the key).
The problem is that the CSV may contain lines with the same server but with a different application. In those cases, I must do an update, but keep the previous value of the app.
How can I keep the previous value before updating?
server_name;ip_address;domain;application
s1;10.10.10.4;dom1;app1
s1;10.10.10.4;dom1;app2
s2;10.15.69.8;dom5;app10
s3;10.15.69.39;dom7;app5
My code is like ($tab contains what I have on the table)
while(($line = fgetcsv($lines))!== false){
if (in_array($line[0],$tab)){ //update query}
else { //insert query }
}
You say this is a scheduled job. Because this is a repeating activity it would be worth your while to build some infrastructure: specifically an external table to provide a SQL interface for the CSV file.
External tables are highly neat: they are defined using CREATE TABLE but instead of a storage clause we specify a source file (pretty similar to SQL*Loader controlfile syntax). Then we can query the data in the CSV file without loading it into a staging table. Find out more.
As for the problem of inserting and updating you should check out the MERGE statement. This handles INSERT and UPDATE in a single statement. Note that you can drive the MERGE using an external table:
merge into your_table
using ( select * from external_table ) ext
on ( your_table.server_name = ext.server_name )
when not matched
then insert values ( ext.server_name, ....)
when matched
then update set your_table.ip_address = ext.ip_address ....
We can apply conditional filters to the INSERT and UPDATE clauses. Find out more.
Import you csv rows value in a temporary parking table and then use an insert select for the rows not in your final destination table
insert into your_final_dest (server_name,ip_address,domain,application)
select server_name,ip_address,domain,application
from your_csv_parking
where server_name not in ( select server_name from your_final_dest )
adn use update for others
UPDATE your_final_dest
SET application = (SELECT application
FROM your_csv_parking
WHERE your_final_dest.server = your_csv_parking.server)
WHERE EXISTS (SELECT your_csv_parking.application
FROM your_csv_parking
WHERE your_final_dest.server = your_csv_parking.server);
Using MySQL I can run the query:
SHOW CREATE TABLE employee;
And it will return the create table statement for the specified table. This is useful if you have a table already created, and want to create the same table on another database.
Is it possible to get the insert statement for an already existing set of rows, so that if a blank db with the tables created is set up, the sql can be pasted into the query box & all the tables will be populated exactly the same as the original?
Thanks
Example of data populated in the table, retrieving the sql for this.
Check the following link for exporting DB from phpmyadmin
https://mediatemple.net/community/products/dv/204403864/export-and-import-mysql-databases
I am using phpMyAdmin to load a CSV file into a MySQL table but, does the table have to be created before the load?
I have read that if the workbench is used ( Import a CSV file into MySQL workbench into a new table dynamically ) the latest version creates the table on the fly but, is this possible with phpMyAdmin.
Here is the code that doesn't work
load data local infile '/Users/...'
into table skin_cutaneous_melanoma
fields terminated by ','
lines terminated by '\n'
Error is:
#1146 - Table hospital.skin_cutaneous_melanoma' doesn't exist
Sure, phpMyAdmin can do this.
From the main page, look for the Import tab. This is, of course, how you expect to import any file including if you already have the database and table created, but if not phpMyAdmin creates a temporary database and table for you using generic names and best guesses as to the data types for your columns. Note that this is probably not the ideal structure, but is the best guess based on the existing data in your database. For best results, you'll probably want to put the desired column names at the top of your CSV file and select the corresponding option during import, or rename them after import (likewise with the database and table names) -- but it is absolutely possible to do the import to a new table.
my Current MySQL table employee_data has 13k rows with 17 columns. The data in the table came from a CSV file Employees.csv. after importing my csv data I added a new column 'password' (so its not in the csv file) Password is edited and accessed via a web portal. I now have an updated csv file and I want to update my main table with that data but I don't want to lose my password info.
Should I import my new CSV file into a temp table in my database and some how compare them? I am not sure where to start and I am open to recommendations.
I am now realizing I should have kept my password info in a separate table. Doh!
I guess I could created a php file that compares each row based on the employee_id field but with 13k rows I am afraid it would time out possibly.
I would do it like this :
Create a temp table using CREATE TABLE new_tbl LIKE orig_tbl; syntax
use LOAD DATA INFILE to import the data from the CSV into the table
Use UPDATE to update the primary table using a primary key / unique column (perhaps employee_id)
I have worked with tables containing 120 million lines and imported CSV files containing 30 million lines into it - this is the method I use all of the time - much more efficient than anything in PHP (and thats my server side language of choice)
Try other tools other than php based ones phpMyAdmin
MySQL workbench is a great tool,
based on you connection it will take a while to query the database with your data.
There are workarounds with php timeout limit,
set_time_limit();
i have a .csv file containing a table data.I want to dump the total csv file into db using a query in mysql using column name as key...
rather than doing it manually...like using "insert into" query...
if any other language like php or python program make this work...then also ok..
can some one suggest me....pls
you can do this fairly easily with navicat software.
more info here
load data local infile ...
LOAD DATA LOCAL INFILE '/tmp/your_file.csv' INTO TABLE your_table;
You need to ensure the mysql user has enough privileges to perform load