Is there an easy way to SELECT a row and INSERT it into another table that has exactly the same columns? It needs to reserve the id and be able to move it back later on.
STEPS:
Move entire row to an identical temporary table (with a different name of-course) while using the same id.
Delete the row from old table (I know this, dont worry ;))
Move the row back from the temporary table into the original table, while using the same original id.
Delete row from temporary table.
I know how to do this, but there are probably much better, cleaner and faster ways to achieve this in one or two queries. My main concern is that when the table gets refactored, the query should still do its job without skipping newer columns.
I hope someone can come with a good suggestion :)
Assuming the tables have identical columns, you should be able to just do this:
INSERT INTO newtable SELECT * FROM oldtable WHERE id = somevalue
Here are two alternate ideas I can think of:
Add a BIT or BOOL column to the table that can be updated in order to 'flag' the record one way or another so that you can filter on it in your queries.
Create a table that just lists the ID of the record(s) that you want to flag. Enter a new ID into this field when you want it flagged (replaces the act of deleting and moving to the 2nd table) and remove it when you wish to 'unflag' it (replaces the act of deleting it from the 2nd table and re-adding it to the original table.
Aside from the initial schema change required by option 1, both options will work regardless of any changes to the schema of the original table. If you are trying to avoid maintaining the second table, I'd consider these.
If you do wish to copy to the holding table and delete from the original table, and later copy back to the original table, and delete from the holding table, be sure to wrap those operations into a transaction so that you don't end up with duplicated data or data loss in the event of things not going as planned.
You use CREATE TABLE ... SELECT to create a table and then work with it. This will create a table with columns from SELECT in your case the original table. Then you can delete the rows from original table, move the rows back to the original table using INSERT INTO ... SELECT followed by dropping the temporary table. This way you don't need to maintain two tables and the temporary table will always get the latest columns from the original table.
See mysql manual. Following example copied from manual.
mysql> SELECT * FROM foo;
+---+
| n |
+---+
| 1 |
+---+
mysql> CREATE TABLE bar (m INT) SELECT n FROM foo;
Query OK, 1 row affected (0.02 sec)
Records: 1 Duplicates: 0 Warnings: 0
mysql> SELECT * FROM bar;
+------+---+
| m | n |
+------+---+
| NULL | 1 |
+------+---+
1 row in set (0.00 sec)
So you need something like this
CREATE TABLE bar SELECT * FROM foo WHERE fooColumn = 'whatever';
DELETE FROM foo where fooColumn <> 'whatever';
INSERT INTO foo SELECT * FROM bar;
DROP TABLE bar
if you want to avoid creating and dropping tables too often, just create the temporary table once.
INSERT INTO bar SELECT * FROM foo WHERE fooColumn = 'whatever';
DELETE FROM foo where fooColumn <> 'whatever';
INSERT INTO foo SELECT * FROM bar;
But make sure you keep the temporary and the original table in sync all the time i.e. if you add or drop a column in the original table, make sure that is replicated in the temporary table immediately as INSERT INTO ... SELECT only copies data not schema.
You can do this :
INSERT INTO tbltemp (Select * from table1 where someid=somevalue)
Or you could move the entire table into other one :
INSERT INTO tbltemp (select * from table1);
What ever you do, be careful that the record/records which you need to move/copy must not include any primary key values which are already there in target table.
Related
So I'm creating a system that will be pulling 50-150 records at a time from a table and display them to the user, and I'm trying to keep a view count for each record.
I figured the most efficient way would be to create a MEMORY table that I use an INSERT INTO to pull the IDs of the rows into and then have a cron function that runs regularly to aggregate the view ID counts and clears out the memory table, updating the original one with the latest view counts. This avoids constantly updating the table that'll likely be getting accessed the most, so I'm not locking 150 rows at a time with each query(or the whole table if I'm using MyISAM).
Basically, the method explained here.
However, I would of course like to do this at the same time as I pull the records information for viewing, and I'd like to avoid running a second, separate query just to get the same set of data for its counts.
Is there any way to SELECT a dataset, return that dataset, and simultaneously insert a single column from that dataset into another table?
It looks like PostgreSQL might have something similar to what I want with the RETURNING keyword, but I'm using MySQL.
First of all, I would not add a counter column to the Main table. I would create a separate Audit table that would hold ID of the item from the Main table plus at least timestamp when that ID was requested. In essence, Audit table would store a history of requests. In this approach you can easily generate much more interesting reports. You can always calculate grand totals per item and also you can calculate summaries by day, week, month, etc per item or across all items. Depending on the volume of data you can periodically delete Audit entries older than some threshold (a month, a year, etc).
Also, you can easily store more information in Audit table as needed, for example, user ID to calculate stats per user.
To populate Audit table "automatically" I would create a stored procedure. The client code would call this stored procedure instead of performing the original SELECT. Stored procedure would return exactly the same result as original SELECT does, but would also add necessary details to the Audit table transparently to the client code.
So, let's assume that Audit table looks like this:
CREATE TABLE AuditTable
(
ID int
IDENTITY -- SQL Server
SERIAL -- Postgres
AUTO_INCREMENT -- MySQL
NOT NULL,
ItemID int NOT NULL,
RequestDateTime datetime NOT NULL
)
and your main SELECT looks like this:
SELECT ItemID, Col1, Col2, ...
FROM MainTable
WHERE <complex criteria>
To perform both INSERT and SELECT in one statement in SQL Server I'd use OUTPUT clause, in Postgres - RETURNING clause, in MySQL - ??? I don't think it has anything like this. So, MySQL procedure would have several separate statements.
MySQL
At first do your SELECT and insert results into a temporary (possibly memory) table. Then copy item IDs from temporary table into Audit table. Then SELECT from temporary table to return result to the client.
CREATE TEMPORARY TABLE TempTable
(
ItemID int NOT NULL,
Col1 ...,
Col2 ...,
...
)
ENGINE = MEMORY
SELECT ItemID, Col1, Col2, ...
FROM MainTable
WHERE <complex criteria>
;
INSERT INTO AuditTable (ItemID, RequestDateTime)
SELECT ItemID, NOW()
FROM TempTable;
SELECT ItemID, Col1, Col2, ...
FROM TempTable
ORDER BY ...;
SQL Server (just to tease you. this single statement does both INSERT and SELECT)
MERGE INTO AuditTable
USING
(
SELECT ItemID, Col1, Col2, ...
FROM MainTable
WHERE <complex criteria>
) AS Src
ON 1 = 0
WHEN NOT MATCHED BY TARGET THEN
INSERT
(ItemID, RequestDateTime)
VALUES
(Src.ItemID, GETDATE())
OUTPUT
Src.ItemID, Src.Col1, Src.Col2, ...
;
You can leave Audit table as it is, or you can set up cron to summarize it periodically. It really depends on the volume of data. In our system we store individual rows for a week, plus we summarize stats per hour and keep it for 6 weeks, plus we keep daily summary for 18 months. But, important part, all these summaries are separate Audit tables, we don't keep auditing information in the Main table, so we don't need to update it.
Joe Celko explained it very well in SQL Style Habits: Attack of the Skeuomorphs:
Now go to any SQL Forum text search the postings. You will find
thousands of postings with DDL that include columns named createdby,
createddate, modifiedby and modifieddate with that particular
meta data on the end of the row declaration. It is the old mag tape
header label written in a new language! Deja Vu!
The header records appeared only once on a tape. But these meta data
values appear over and over on every row in the table. One of the main
reasons for using databases (not just SQL) was to remove redundancy
from the data; this just adds more redundancy. But now think about
what happens to the audit trail when a row is deleted? What happens to
the audit trail when a row is updated? The trail is destroyed. The
audit data should be separated from the schema. Would you put the log
file on the same disk drive as the database? Would an accountant let
the same person approve and receive a payment?
You're kind of asking if MySQL supports a SELECT trigger. It doesn't. You'll need to do this as two queries, however you can stick those inside a stored procedure - then you can pass in the range you're fetching, have it both return the results AND do the INSERT into the other table.
Updated answer with skeleton example for stored procedure:
DELIMITER $$
CREATE PROCEDURE `FetchRows`(IN StartID INT, IN EndID INT)
BEGIN
UPDATE Blah SET ViewCount = ViewCount+1 WHERE id >= StartID AND id <= EndID;
# ^ Assumes counts are stored in the same table. If they're in a seperate table, do an INSERT INTO ... ON DUPLICATE KEY UPDATE ViewCount = ViewCount+1 instead.
SELECT * FROM Blah WHERE id >= StartID AND id <= EndID;
END$$
DELIMITER ;
Situation:
Old scripts added rows to a table without deleting existing rows.
Need to discover "duplicate" rows (based on matching two fields).
For each set of duplicate rows, sort by ids and return all but the newest one (highest id).
Each row has an associated external file, so can't simply delete the older rows - need to return a list of all the older rows, which will then be processed by a php script.
Example:
TABLE mytable:
ID A B Filename
1 10 abc aa.png
2 11 dddd bb.xml
3 10 abc cc.png
4 10 dddd dd.png
5 10 abc ee.xml
6 11 dddd ff.xml
Rows with IDs 1 & 3 & 5 are duplicates (both A and B match).
Similarly, 2 & 6 are duplicates. Return list (1, 2, 3) - these are the "older" rows that need to be processed.
Even better: return a set of records, containing 'ID' and 'Filename' for those rows.
My primary question is an SQL query that does this, though it would also be useful to me to see how to use the result of that query in php.
There are existing stackoverflow posts related to deleting duplicate rows, but the ones I found delete the rows directly. This won't work for me, as I need to have the external php script delete the corresponding external files:
Deleting Duplicate Rows from MySql Table
How to delete duplicate records in mysql database?
How to delete all the duplicate records in PHP/Mysql
IMPORTANT: The other posts which I quote don't bother to distinguish newer from older; they are about removing fully duplicate records, but that is not my situation. I have records which are partially duplicates; that is, several records match the specified criteria, but there is important information in other fields, hence I have to know which is newest (highest id) for each value of criteria; those are the ones to keep.
I would try this "make sure you test the code before to apply it on production data"
Assuming you have lots of data, I would create temporary table of the data that you want to keep so you can perform the operation fast.
-- Generate a list of the IDs to keep
CREATE TEMPORARY TABLE keepers (KEY(ID)) ENGINE = MEMORY
SELECT A, B MIN(ID) AS ID
FROM table
GROUP BY A, B;
-- Delete the records that you do not wish to keep
DELETE FROM table
WHERE NOT EXISTS (SELECT 1 FROM keepers WHERE ID = table.ID);
If the DELETE query does not work "return an error" about the sub query, you can try this instead of the DELETE query.
CREATE TEMPORARY TABLE deleteme (KEY(ID)) ENGINE = MEMORY
SELECT ID FROM table
WHERE NOT EXISTS (SELECT 1 FROM keepers WHERE ID = table.ID);
DELETE t.* FROM table AS t
INNER JOIN deleteme AS d ON d.ID = t.ID;
To get the data:
Select the records you want to keep (inner query) and join back on itself (outer query) keeping all records and using the dummyfield to find the to be deleted records.
CREATE TEMPORARY TABLE delete_these AS
SELECT *
FROM table a
LEFT JOIN (SELECT MAX(id) as non_deletion_id, 1 AS dummyfield,
FROM table a
GROUP BY your two fields) b ON non_deletion_id=a.id
WHERE dummyfield IS NULL;
I'm using PHP to insert groups of records into a MySQL DB.
Whenever I insert a group of records, I want to give that group a unique set ID that is incremented by 1 for each group of records in the DB.
Currently, I'm checking the latest set ID in the DB and incrementing it by 1 for each new set of records.
The thing that scares me though is what happens if I query the DB to get the latest set ID, and before I can insert a new set of records with that set ID + 1, another insert occurs on the table thus taking the set ID I was about to use?
While fairly unlikely, something like that could greatly sacrifice the integrity of the data.
What can I do to prevent such a thing from happening? Is there any way to temporarily lock the DB table so that no other inserts can occur until I have performed a SELECT/INSERT combo?
Locking the table is one option, but that approach impacts concurrency.
The approach I would recommend is that you use a separate table with AUTO_INCREMENT column, and use a separate INSERT into that table, and a SELECT LAST_INSERT_ID() to retrieve the auto_increment value.
And then use that value as the group identifier for the group of rows you insert into your original table.
The basic approach is:
LOCK TABLE foo WRITE;
SELECT MAX(id) + 1 FROM foo
INSERT ...
INSERT ...
UNLOCK TABLES;
Locking the table prevents any other process from changing the table until you explicitly unlock it.
Having said that, seriously consider just using a table with an AUTO_INCREMENT column. MySQL will do the work of maintaining unique keys wholly automatically, and then you can simply refer to those keys from your existing table.
I have 2 tables. suppose a & b
a has id, name, roll. b has id,group,name
This name column data are not same. How can I select and uniquely identify them?
I know about
SELECT a.id,a.name,a.group FROM a,b ............
I know this. But this is an example. I am working with huge amount of data with 20-30 columns in each table. So I don't want to write the column names I need to select rather I want to write the names that I want to exclude.
Like
SELECT * Except b.name............
OR is there any way to uniquely identify after join. Like
.......... a,b WHERE a.name as name1
Please don't ask why those column names are same. I admit it was a mistake. But it's already implemented and heavily used. So finding another way. Is there any simple way to exclude a column while merging them?
Well, you can't write the names you wish to exclude. That is not how SQL works.
However, if writing out 20-30 column names is that much of a burden, you can use information_schema.columns. I write it that way, because 20-30 column names is not particularly large and writing them out is probably less effort than writing the question.
But, back to the solution. It looks something like this:
select concat(c.column_name, ' as ', 'a_', column_name, ', ')
from information_schema.columns c
where table_name = 'a' ;
You might want to include the table schema as well.
As an IDEA, what you can do is, if you want to avoid columns of specific table & your statements have multiple table, you can try following,
Suppose you have 20 columns in table a & 5 columns in table b, you want to avoid col2,col3 & col4 of table b. Standard method is that you should write name of all columns of table a & required columns of table b. But you can avoid to write long list of 20 columns of table by writing a.* & then type required columns of table b. Please see below statement.
Select a.*,b.col1,b.col4,b.col5 from a,b
But if you require to exclude some columns from both table, then I think there is no other way than writing all required column names from both table.
There is no way to exclude a column in SQL SELECT Statement, you can only select a column. You can give alias name to columns while selecting them like below, so that you can identity columns using those alias names.
SELECT a.id as [column1],a.name as [column2],a.group as [column3] FROM a,b ............
There is no way to exclude a specific column but you can avoid to write all columns name and easy your job by below steps-
Step1: Execute below query-
SELECT a.*,b.* FROM a,b ............limit 1;
Step2: Export it into csv format with headings.
Step3: Copyp first (heading) row from csv.
Step4: Delete columns, those are not required and use other columns in your query.
There's only one waY i could see-
first create a temorary table
CREATE TEMPORARY TABLE IF NOT EXISTS mytable
(id int(11) NOT NULL, PRIMARY KEY (id)) ENGINE=MyISAM;
then put your column in temporary table-
SELECT * INTO mytable
FROM YourTable
/* Drop the cloumns that are not needed */
ALTER TABLE mytable
DROP COLUMN ColumnToDrop
/* Get results and drop temp table */
SELECT * FROM #TempTable
DROP TABLE #TempTable
Let's say that I've got a table, like that (id is auto-increment):
id | col1 | col2
1 | 'msg'| 'msg'
2 | 'lol'| 'lol2'
3 | 'xxx'| 'x'
Now, I want to delete row number 2 and I get something like this
id | col1 | col2
1 | 'msg'| 'msg'
3 | 'xxx'| 'x'
The thing is, what I want to get is that:
id | col1 | col2
1 | 'msg'| 'msg'
2 | 'xxx'| 'x'
How can I do that in the EASIEST way (my knowledge about MySQL is very poor)?
You shouldn't do that.
Do not take an auto-incremented unique identifier as an ordinal number.
The word "unique" means that the identifier should be stuck to its row forever.
There is no connection between these numbers and enumerating.
Imagine you want to select records in alphabetical order. Where would your precious numbers go?
A database is not like an ordered list, as you probably think. It is not a flat file with rows stored in a predefined order. It has totally different ideology. Rows in the database do not have any order. And will be ordered only at select time, if it was explicitly set by ORDER BY clause.
Also, a database is supposed to do a search for you. So you can tell that with filtered rows or different ordering this auto-increment number will have absolutely nothing to do with the real rows positions.
If you want to enumerate the output - it's a presentation layer's job. Just add a counter on the PHP side.
And again: these numbers supposed to identify a certain record. If you change this number, you'd never find your record again.
Take this very site for example. Stack Overflow identifies its questions with such a number:
stackoverflow.com/questions/3132439/mysql-auto-decrementing-value
So, imagine you saved this page address to a bookmark. Now Jeff comes along and renumbers the whole database. You press your bookmark and land on the different question. Whole site would become a terrible mess.
Remember: Renumbering unique identifiers is evil!
I think there is no way to this directly. Maybe you can do "update" operation. But you must do it for all record after your deleted record. It is very bad solution for this.
Why using an auto-increment if you want to change it manually?
It is not good practice to change the value of an auto_increment column. However, if you are sure you want to, the following should help.
If you are only deleting a single record at a time, you could use a transaction:
START TRANSACTION;
DELETE FROM table1 WHERE id = 2;
UPDATE table1 SET id = id - 1 WHERE id > 2;
COMMIT;
However if you delete multiple records, you will have to drop the column and re-add it. It is probably not guaranteed to put the rows in the same order as previously.
ALTER TABLE table1 DROP id;
ALTER TABLE table1 ADD id INTEGER NOT NULL AUTO_INCREMENT;
Also, if you have data that relies on these IDs, you will need to make sure it is updated.
You can renumber the whole table like this:
SET #r := 0;
UPDATE mytable
SET id = (#r := #r + 1)
ORDER BY
id;