I have this query in php. It's an insert select copying from table2, but I need to get the IDs of the newly created rows and store them into an array. Here is my code:
$sql = "INSERT INTO table1 SELECT distinct * from table2";
$db->query($sql);
I could revert the flow starting with a select on table2 and making all single inserts but it would slow down the script on a big table. Ideas?
You could lock the table, insert the rows, and get the ID of the last item inserted, and then unlock; that way you know that the IDs will be contiguous as no other concurrent user could have changed them. Locking and unlocking is something you want to use with caution though.
An alternative approach could be to use one of the columns in the table - either an 'updated' datetime column, or an insert-id column (for which you put in a value that will be the same across all of your rows.)
That way you can do a subsequent SELECT of the IDs back out of the database matching either the updated time or your chosen insert ID.
Im trying to create a table which only will have the serial and 9 int foreign keys.
Is there a way to insert the last id from other tables to this table?
i been trying with RETURNING, nextval()-1, currval(),ordering results by timestamp but nothing seems to work, i just cant get the ids values from the other tables inserted into the table.
even tried this simple set of queries for each table to update one by one of the table columns
$lst="SELECT id from table1 ORDER BY timestamp DESC limit 1";
$lst_res=pg_query($conn,$lst);
$sql_eq_bat_up="UPDATE table9 set table1_id='$lst_res[id]'";
$do_up_tbl1=pg_query($conn,$sql_eq_bat_up);
i been doing some research but had no luck, im using postgreSQL 9.5 and php for this, i hope you can help me thanks.
So I'm creating a system that will be pulling 50-150 records at a time from a table and display them to the user, and I'm trying to keep a view count for each record.
I figured the most efficient way would be to create a MEMORY table that I use an INSERT INTO to pull the IDs of the rows into and then have a cron function that runs regularly to aggregate the view ID counts and clears out the memory table, updating the original one with the latest view counts. This avoids constantly updating the table that'll likely be getting accessed the most, so I'm not locking 150 rows at a time with each query(or the whole table if I'm using MyISAM).
Basically, the method explained here.
However, I would of course like to do this at the same time as I pull the records information for viewing, and I'd like to avoid running a second, separate query just to get the same set of data for its counts.
Is there any way to SELECT a dataset, return that dataset, and simultaneously insert a single column from that dataset into another table?
It looks like PostgreSQL might have something similar to what I want with the RETURNING keyword, but I'm using MySQL.
First of all, I would not add a counter column to the Main table. I would create a separate Audit table that would hold ID of the item from the Main table plus at least timestamp when that ID was requested. In essence, Audit table would store a history of requests. In this approach you can easily generate much more interesting reports. You can always calculate grand totals per item and also you can calculate summaries by day, week, month, etc per item or across all items. Depending on the volume of data you can periodically delete Audit entries older than some threshold (a month, a year, etc).
Also, you can easily store more information in Audit table as needed, for example, user ID to calculate stats per user.
To populate Audit table "automatically" I would create a stored procedure. The client code would call this stored procedure instead of performing the original SELECT. Stored procedure would return exactly the same result as original SELECT does, but would also add necessary details to the Audit table transparently to the client code.
So, let's assume that Audit table looks like this:
CREATE TABLE AuditTable
(
ID int
IDENTITY -- SQL Server
SERIAL -- Postgres
AUTO_INCREMENT -- MySQL
NOT NULL,
ItemID int NOT NULL,
RequestDateTime datetime NOT NULL
)
and your main SELECT looks like this:
SELECT ItemID, Col1, Col2, ...
FROM MainTable
WHERE <complex criteria>
To perform both INSERT and SELECT in one statement in SQL Server I'd use OUTPUT clause, in Postgres - RETURNING clause, in MySQL - ??? I don't think it has anything like this. So, MySQL procedure would have several separate statements.
MySQL
At first do your SELECT and insert results into a temporary (possibly memory) table. Then copy item IDs from temporary table into Audit table. Then SELECT from temporary table to return result to the client.
CREATE TEMPORARY TABLE TempTable
(
ItemID int NOT NULL,
Col1 ...,
Col2 ...,
...
)
ENGINE = MEMORY
SELECT ItemID, Col1, Col2, ...
FROM MainTable
WHERE <complex criteria>
;
INSERT INTO AuditTable (ItemID, RequestDateTime)
SELECT ItemID, NOW()
FROM TempTable;
SELECT ItemID, Col1, Col2, ...
FROM TempTable
ORDER BY ...;
SQL Server (just to tease you. this single statement does both INSERT and SELECT)
MERGE INTO AuditTable
USING
(
SELECT ItemID, Col1, Col2, ...
FROM MainTable
WHERE <complex criteria>
) AS Src
ON 1 = 0
WHEN NOT MATCHED BY TARGET THEN
INSERT
(ItemID, RequestDateTime)
VALUES
(Src.ItemID, GETDATE())
OUTPUT
Src.ItemID, Src.Col1, Src.Col2, ...
;
You can leave Audit table as it is, or you can set up cron to summarize it periodically. It really depends on the volume of data. In our system we store individual rows for a week, plus we summarize stats per hour and keep it for 6 weeks, plus we keep daily summary for 18 months. But, important part, all these summaries are separate Audit tables, we don't keep auditing information in the Main table, so we don't need to update it.
Joe Celko explained it very well in SQL Style Habits: Attack of the Skeuomorphs:
Now go to any SQL Forum text search the postings. You will find
thousands of postings with DDL that include columns named createdby,
createddate, modifiedby and modifieddate with that particular
meta data on the end of the row declaration. It is the old mag tape
header label written in a new language! Deja Vu!
The header records appeared only once on a tape. But these meta data
values appear over and over on every row in the table. One of the main
reasons for using databases (not just SQL) was to remove redundancy
from the data; this just adds more redundancy. But now think about
what happens to the audit trail when a row is deleted? What happens to
the audit trail when a row is updated? The trail is destroyed. The
audit data should be separated from the schema. Would you put the log
file on the same disk drive as the database? Would an accountant let
the same person approve and receive a payment?
You're kind of asking if MySQL supports a SELECT trigger. It doesn't. You'll need to do this as two queries, however you can stick those inside a stored procedure - then you can pass in the range you're fetching, have it both return the results AND do the INSERT into the other table.
Updated answer with skeleton example for stored procedure:
DELIMITER $$
CREATE PROCEDURE `FetchRows`(IN StartID INT, IN EndID INT)
BEGIN
UPDATE Blah SET ViewCount = ViewCount+1 WHERE id >= StartID AND id <= EndID;
# ^ Assumes counts are stored in the same table. If they're in a seperate table, do an INSERT INTO ... ON DUPLICATE KEY UPDATE ViewCount = ViewCount+1 instead.
SELECT * FROM Blah WHERE id >= StartID AND id <= EndID;
END$$
DELIMITER ;
I have a product_group table with the following fields: group_id, product_id, order. The table will be queried against a lot: a single-form view will make it possible to insert new records and/or update existing ones with one submit.
I'm trying to figure out optimal solution to cover the following 3 cases:
User tries to insert an existing row: do nothing. Here a unique index of the 3 columns can be useful.
User changes only the order column: perform an update.
User inserts a completely new set of values: perform an insert.
Is there a way to put all of this together in one MySQL query? If not, what would be the best approach here? The goal is to limit database queries as much as possible.
Does this do what you want?
insert into product_group(group_id, product_id, `order`)
values (#group_id, #product_id, #order)
on duplicate key update `order` = values(`order`);
Along with a unique index on group_id, product_id:
create unique index idx_product_group_2 on product_group(group_id, product_id)
This handles your three cases:
Because the value assignment is a no-op if the values are the same.
The order column will be updated if the other two have the same value.
A new row that has a different group_id or product_id will be inserted.
As a note, order is a lousy name for a column, because it is a SQL key word.
I'm using PHP to insert groups of records into a MySQL DB.
Whenever I insert a group of records, I want to give that group a unique set ID that is incremented by 1 for each group of records in the DB.
Currently, I'm checking the latest set ID in the DB and incrementing it by 1 for each new set of records.
The thing that scares me though is what happens if I query the DB to get the latest set ID, and before I can insert a new set of records with that set ID + 1, another insert occurs on the table thus taking the set ID I was about to use?
While fairly unlikely, something like that could greatly sacrifice the integrity of the data.
What can I do to prevent such a thing from happening? Is there any way to temporarily lock the DB table so that no other inserts can occur until I have performed a SELECT/INSERT combo?
Locking the table is one option, but that approach impacts concurrency.
The approach I would recommend is that you use a separate table with AUTO_INCREMENT column, and use a separate INSERT into that table, and a SELECT LAST_INSERT_ID() to retrieve the auto_increment value.
And then use that value as the group identifier for the group of rows you insert into your original table.
The basic approach is:
LOCK TABLE foo WRITE;
SELECT MAX(id) + 1 FROM foo
INSERT ...
INSERT ...
UNLOCK TABLES;
Locking the table prevents any other process from changing the table until you explicitly unlock it.
Having said that, seriously consider just using a table with an AUTO_INCREMENT column. MySQL will do the work of maintaining unique keys wholly automatically, and then you can simply refer to those keys from your existing table.