MySQL fixing autoincrement gaps in two tables - php

Question

I have two tables like so;
id_image foo bar
1 3 5
2 8 1
3 17 88
7 14 23
8 12 9
id_image bar foo
1 2 3
1 5 6
2 18 11
2 10 12
3 8 21
3 17 81
7 29 50
7 1 14
8 10 26
8 27 34
There is a gap in the autoincremented id_image in the first table. In the second table, the id_image refers to the id_image in the first table, and there's two of each ID in there.
Notice: This table is theoretical. I have no idea where the gap is exactly, or whether or not there are even multiple gaps. All I know is that the first value is 1 and the last value is higher than the total row count.
Now, I'd like to fix this gap.
Before you say that the gaps don't matter and if they do, it's bad database design, let me tell you; I agree with you.
However, what I'm dealing with is a (hopelessly rear end backwards) third-party open source system to which I need to import a huge amount of existing data that doesn't have cross-referenceable IDs into multiple tables. The only way I can make sure that the same data gets a matching ID in every table throughout the system is to input it sequentially, and that means I can't have gaps.
So what I do now need to do is;
Fix the gap in the id_image column in the first table, so that the last value matches with the row count.
Edit the id_image column in the second table so that its value corresponds to the same row is corresponded to before the gap fix.
How would I begin to do this? I understand that this might be outside the capabilities of the MySQL query language, so PHP answers are also acceptable. Thanks! :)

Answer 1

ALTER TABLE table2
ADD FOREIGN KEY FK_IMAGE (id_image)
REFERENCES table1 (id_image)
ON DELETE CASCADE
ON UPDATE CASCADE;
SET #currentRow = 0;
UPDATE table1 INNER JOIN (
SELECT #currentRow := #currentRow + 1 AS id_image_new, id_image AS id_image_old
FROM table1
ORDER BY id_image ASC) t on t.id_image_old = table1.id_image
SET table1.id_image = t.id_image_new;
ALTER TABLE table1 AUTO_INCREMENT = 1;
The FK will automatically update ids of your 2nd table accordingly.
I'm not sure at all but in some older versions of mysql, update a table that you are referencing within a subquery of the update could crash. If so, just create a 2nd table and fill it up (inserts), then delete the old one and rename the new one.

Answer 2

The basic idea here is to find all of the gaps first to determine how much you need to decrement each id. Then, you have to iterate through both tables and apply the decrement. (You'll need to add: host, db, user, pass, and the actual table names)
try {
$pdo = new PDO('mysql:host=HOST;dbname=DB', 'user', 'pass');
$pdo->beginTransaction();
// Iterate through all id's in the first table
$stmt = $pdo->exec('SELECT image_id FROM TableOne ORDER BY image_id ASC');
$stmt->bindColumn('image_id', $id);
if(!$stmt->fetch(PDO::FETCH_BOUND)) {
throw Exception('No rows in table');
}
$lastId = $id;
$gaps = array();
// Find all the gaps
while($stmt->fetch(PDO::FETCH_BOUND)) {
if($id != ($lastId + 1)) {
$gaps[] = $id;
}
$lastId = $id;
}
if(!isset($gaps[0])) {
throw new Exception('No gaps found');
}
// For each gap, update the range from the last gap to that gap by subtracting
// the number of gaps there has been from the id
$lastGap = $gaps[0];
for($i = 1; $i < count($gaps); $i++) {
$stmt = $pdo->prepare('UPDATE TableOne SET image_id = image_id - :i WHERE image_id BETWEEN :lastGap AND :gap');
$stmt->execute(array(
':i' => $i,
':lastGap' => $lastGap,
':gap' => $gaps[$i]
));
$stmt = $pdo->prepare('UPDATE TableTwo SET image_id = image_id - :i WHERE image_id BETWEEN :lastGap AND :gap');
$stmt->execute(array(
':i' => $i,
':lastGap' => $lastGap,
':gap' => $gaps[$i]
));
$lastGap = $gaps[$i];
}
// Finally, fix the gap between the last found gap and the end of the table
$stmt = $pdo->prepare('UPDATE TableOne SET image_id = image_id - :i WHERE image_id BETWEEN :lastGap AND :gap');
$stmt->execute(array(
':i' => $i,
':lastGap' => $lastGap,
':gap' => $gaps[$i]
));
$stmt = $pdo->prepare('UPDATE TableTwo SET image_id = image_id - :i WHERE image_id BETWEEN :lastGap AND :lastId');
$stmt->execute(array(
':i' => $i,
':lastGap' => $lastGap,
':lastId' => $lastId
));
// Verify everything is correct
$stmt = $pdo->exec('SELECT image_id FROM TableOne ORDER BY image_id ASC');
$stmt->bindColumn('image_id', $id);
if(!$stmt->fetch(PDO::FETCH_BOUND)) {
throw new Exception('No rows'); // Should never be thrown
}
$lastId = $id;
while($stmt->fetch(PDO::FETCH_BOUND)) {
if($id != ($lastId + 1)) {
throw new Exception('There was an error between ids ' . $lastId . ' and '. $id);
}
$lastId = $id;
}
$stmt = $pdo->exec('SELECT image_id FROM TableTwo ORDER BY image_id ASC');
$stmt->bindColumn('image_id', $id);
if(!$stmt->fetch(PDO::FETCH_BOUND)) {
throw new Exception('No rows in table two'); // Shouldn't hit this
}
$lastId = $id;
$ids = array($id);
while($stmt->fetch(PDO::FETCH_BOUND)) {
$ids[] = $id;
if(count($ids) == 2) {
if($ids[0] !== $ids[1]) {
throw new Exception('Table two error on ids ');
}
if($ids[0] !== $lastId) {
throw new Exception('Table two error on id gapfix');
}
$lastId = $ids[0];
$ids = array();
}
}
$pdo->commit();
} catch(Exception $e) {
$pdo->rollBack();
var_dump($e);
}
Important: You might want to throw this in a file and run via the CLI: php -f gapfix.php and include a query before $pdo->commit() that returns a list of all the ids so you can verify the operation worked as expected. If it didn't, you can roll it back as if nothing happened. The code now checks for itself if the first table is in the right order. It doesn't however check the second table yet. All checking has been implemented!

Answer 3

Painful this.
Create a table like the first, no identity on Id_Image and an extra int column called rownumber
Use the pseudo row_number trick to populate it, some like
Insert into NewTable
Select id_image,foo,bar,#RowNumber := #RowNumber + 1 order by id_image.
If you have a foreign key to the second table drop it, then it's a simple update with a join. Drop the old table1 , rename the new one, add the identity and reseed, and put your foreign key back if you had one.
You do realise you are going to have to keep doing this crap?
There's probably a fun way of doing all this in one go, if you have cascading updates on, but pay careful attention to the execution plan. The RowNumber trick only works if things are done in Id_Image order. If Mysql decides there's a more efficient way of doing the query....