I have the following 1 db table in Database 1 and 1db table in Database 2, now the stucture of both tables are exactly the same. Now what happens is table 1 (DB1) gets new rows added daily, I need to update the table 1 (DB 1) new rows in table 1 (DB 2) so that these 2 tables remain the same. A cron will trigger a php script on midnight to do this task. What is the best way to do this and how using PHP/mysql?
You might care to have a look at replication (see http://dev.mysql.com/doc/refman/5.4/en/replication-configuration.html). That's the 'proper' way to do it; it isn't to be trifled with, though, and for small tables the above solutions are probably better (and certainly easier).
This might help you out, its what i do on my database for a similar kinda thing
$dropSQL = "DROP TABLE IF EXISTS `$targetTable`";
$createSQL = "CREATE TABLE `$targetTable` SELECT * FROM `$activeTable`";
$primaryKeySQL = "ALTER TABLE `$targetTable` ADD PRIMARY KEY(`id`)";
$autoIncSQL = "ALTER TABLE `$targetTable` CHANGE `id` `id` INT( 60 ) NOT NULL AUTO_INCREMENT";
mysql_query($dropSQL);
mysql_query($createSQL);
mysql_query($primaryKeySQL);
mysql_query($autoIncSQL);
obviously you will have to modify the taget and active table variables. Dropping the table will lose the primary key when you do this, oh well .. easy enough to add back in
I would recommend replication as has already been suggested. However, another option is to use mysqldump to grab the rows you need and send them to the other table.
mysqldump -uUSER -pPASSWORD -hHOST --compact -t --where="date=\"CURRENT_DATE\"" DB1 TABLE | mysql -uUSER -pPASSWORD -hHOST -D DB2
Replace USER, HOST, and PASSWORD with login info for your database. You can use different information for each part of the command if DB1 and DB2 have different access information. DB1 and DB2 are the names of your databases, and TABLE is the name of the table.
You can also modify the --where option to grab only the rows which need to updated. Hopefully you have some query you can use. As mentioned previously, if the table has a primary key, you could grab the last key which DB2 has using a command something like
KEY=`echo "SELECT MAX(KEY_COLUMN) FROM TABLE;" mysql -uUSER -pPASSWORD -hHOST -D DB2`
for a bash shell script (then use this value in the WHERE clause above). Depending on how your primary key is generated, this may be a bad idea since rows may be added in holes in the keyspace if they exist.
This solution will also work if rows are changed as long as you have a query which can select these rows. Just add the --replace option to the mysqldump command. In your situation, it would be best to add some type of value such as date updated which you can compare by.
Related
I have table on MySQL server and would like to get the SQL that would re-create the table.
How do I get the query to recreate the SQL table?
MySQL supports SHOW CREATE TABLE to return the SQL that was used to create a table.
From their docs:
mysql> SHOW CREATE TABLE t;
CREATE TABLE t (
id INT(11) default NULL auto_increment,
s char(60) default NULL,
PRIMARY KEY (id)
) ENGINE=MyISAM
This can be useful if you have just a connection to the DB, versus a CLI on the server. If you have a CLI, use mysqldump like liamgriffiths recommends.
mysqldump can do it - http://dev.mysql.com/doc/refman/5.1/en/mysqldump.html
You can use it like this:
mysqldump [databasename] [tablename] > mysqltablesql.sql
If you are using phpMyAdmin, select a table then click the 'Export' tab. One of the output types is SQL. Selecting this will generate a SQL script to create the table and insert all of the data that is currently in the table. Make sure under the dump data option to select 'Structure and Data'
mysqldump [OPTIONS] database [tables]>backup-file.sql
If you don't give any tables or use the --databases or --all-databases, the whole database(s) will be dumped.
You can get a list of the options your version of mysqldump supports by executing mysqldump --help.
Note that if you run mysqldump without --quick or --opt, mysqldump will load the whole result set into memory before dumping the result. This will probably be a problem if you are dumping a big database.
If you have access to phpMyAdmin, you can get this code through the Export tab on the table that you require.
Select SQL then make sure you export "Structure" ("CREATE table" code) and "Data" ("INSERT" code) with Export type set to "INSERT".
I'd go with #liamgriffiths proposal of mysqldump, but you could also try create table <new-table-name> like <old-table-name> followed by insert into <new-table-name> select * from <old-table-name>
I am new with PHP development and just wondering if theres a existing function on PHP than duplicate the copy command on phpmyadmin, i know that the query sequence is below, but this is like a long query/code since the table has alot of columns. i mean if phpmyadmin has this feature maybe its calling a build in function?
SELECT * FROM table where id = X
INSERT INTO table (XXX)VALUES(XXX)
Where the information is based from the SELECT query
Note: The id is primary and auto increment.
Here is the copy command on phpmyadmin
i mean if phpmyadmin has this feature maybe its calling a build in function?
There is no built-in functionality in MySQL to duplicate a row other than an INSERT statement of the form: INSERT INTO tableName ( columns-specification ) SELECT columns-specification FROM tableName WHERE primaryKeyColumns = primaryKeyValue.
The problem is you need to know the names of the columns beforehand, you also need to exclude auto_increment columns, as well as primary-key columns, and know how to come up with "smart defaults" for non-auto_increment primary key columns, especially composite keys. You'll also need to consider if any triggers should be executed too - and how to handle any constraints and indexes that may be designed to prevent duplicate values that a "copy" operation might introduce.
You can still do it in PHP, or even pure-MySQL (inside a sproc, using Dynamic SQL) but you'll need to query information_schema to get metadata about your database - which may be more trouble than it's worth.
I have a mysql table having over a million rows with user information. I want to add a column 'password' in that table, randomly generate password and update all the records (using PHP).
What is the fastest way to perform this task?
Thanks in advance.
A possible scenario
Generate all passwords into a file with your php script
Create temp table and use LOAD DATA INFILE (which happens to be the fastest method to import data from file) to load data from the password file.
Alter your table and add password column
Use UPDATE with join to update password column in your original table from temp table
Drop temp table
Keep in mind that the operation for altering the table might take very long time, I would recommend to sepearate this two steps.
1) Alter the table, here I would recommend the following steps (pseudocode):
create new_table like old_table
alter new_table add column password
insert into new_table (columns) select * from old_table
rename old_table to old_table_bck, new_table to old_table
drop old_table_bck
At this point you have your original table with the new column.
2) Now after you changed your structure you can populate the new column 'password' with your php. If you are using InnoDB as storage engine time doesn´t matter since you are not locking the table with your updates. If you do this within transactions I would suggest to break the update process down to smaller transactions instead of inserting one large bulk.
If you can´t take (minimal) downtime, i would suggest that you look at pt-online-schema-change, we use this tool on major databases where we can´t effort any downtime in order to make schema changes while database are running. It roughly performs above steps (1) and additionally ensures with the help of triggers that data inserted on the original table while performing the alter process is also written to the altered table.
I wanted to import a dump sql file via ssh like:
mysql -hlocalhost -uUSER -pPASS DATABASE < dump.sql
But it actually overwrites and removes the old database. The dump is tagged with "DROP TABLE IF EXISTS" but why does it remove the old entries? This dump was supposed to add NEW (unique) entries into the database but it removes old database entries.
Is there a way to avoid this?
Thanks.
There are various options you can pass to MySQLDump so that the script will only contain what you need (e.g. omit the create table statements and just insert data): http://dev.mysql.com/doc/refman/5.1/en/mysqldump.html
I think you're interested in --no-create-info and --no-create-db in particular.
If you need to work with the dump you already have, just remove the offending "DROP TABLE" lines from it manually.
The dump is tagged with "DROP TABLE IF EXISTS" but why does it remove the old entries?
That's exactly what DROP TABLE IF EXISTS does; if the table is there, it gets dropped (removed) together with all the rows inside of it.
When creating the dump you could add --skip-add-drop-table option.
How to reset the database after 3 hrs & make it behave as a new database through php script
Possibly the easiest way would be to have a cron job that executes every three hours and calls mysql with "clean" database set up. The crontab set up would be something along the lines of:
* */03 * * * mysql -u XXX -pXXX < clean_database.sql
However, the "clean_database.sql" file would need to use "DROP TABLE IF EXISTS ..." for each of the tables you want to reset. That said, you can simply use mysqldump with a "known good" version of the database to create this file. (You'll need to add a "use <database name>;" statement at the top that said.)
The easiest way is to drop the database and recreate it using your create scripts. If you don't have create scripts you can get them by making a dump of your database.
To delete the data in each table without dropping the tables you can use the TRUNCATE TABLE tablename command on each table.
If you don't have permission to use TRUNCATE you can use DELETE FROM tablename without a WHERE clause.
Note that if you have foreign key constraints you may have to run the statements in a specific order to avoid violating these constraints.
To get a list of all tables you can use SHOW TABLES.
steps to do:
connect to database server
select database
mysql_query("SHOW TABLES");
read in array or object
foreach($tables as $tableName) of the item mysql_query("TRUNCATE TABLE $tableName")
I hope the principle is clean to you ;-)
mysql_query('DROP DATABASE yourdatabase');
mysql_query('CREATE DATABASE yourdatabase');
mysql_query('CREATE TABLE yourdatabase.sometable ...'); // etc.
This will drop the database, and create it anew. You can then use the CREATE TABLE syntax to recreate the tables - note that as this script has significant powers, you should consider creating a special mySQL user for it, one that's not used during normal operations.