How to Update 1 Million of rows quickly in MYSQL [duplicate] - php

Question

This question already has answers here:
Best way to INSERT many values in mysqli?
(4 answers)
Closed 1 year ago.
I have only 1 table with 1 million rows that i need to update. The data what I need to update is in a file.txt and its only 1 value to update in 1 million rows.
I use PHP to read file and make the Update.
The problem is the time. Is so slow. Any idea to improve and make more fast?
What I have is:
UPDATE productos SET urlRedirect = getChangeProducto WHERE url LIKE 'getUrlProducto'
TXT file, 2 values separated by (;) first are getChangeProducto and second getUrlProducto:
https://www.example.com/guess/8595651?o=34&t=direct&f=102.301&ord=111;https://example.com/p/guess-jeans-116-175-cm-327554`
PHP FILE:
$mysqli = get_dbc();
if ($mysqli->connect_errno) {
echo "Failed to connect to MySQL: (".$mysqli->connect_errno.") ".$mysqli->connect_error;
exit();
}
$mysqli->set_charset("utf8");
if (!$mysqli) exit(mysqli_connect_error());
$handle = fopen("fileToUpdate.txt", "r");
$total_lines = 0;
if ($handle) {
$line = fgets($handle);
while ($line !== false) {
$url_to_delete = trim($line);
$trozos = explode(";", $url_to_delete);
$total_lines++;
if ($stmt = $mysqli->prepare("UPDATE productos SET urlRedirect = ? WHERE url = ? ")) {
$stmt->bind_param('ss', $trozos[1],$trozos[0]);
$stmt->execute(); // Execute the prepared query.
}
$stmt->close();
$line = fgets($handle);
}
fclose($handle);
}
// Output — Total Lines: 6305.
echo "Total Lines: $total_lines.\n";

Answer 1

I also encountered this problem before. That was around 2009 or 2010 where I need to update multiple entries fast because of the request timeout. Some solution I found was to extend the request timeout configuration of the server but another solution I found was the create a temporary table on the database server and relate the two tables but unfortunately, I cannot share with you the link because that was a long time ago.
So here what I did as long as I could recall:
Create a temporary table with similar structure to your CSV bearing the primary key to which relates to the table you like to perform the update. I know MySQL has the temporary table capability.
From your CSV, upload data by insert the record at once (by batch or multiple values on INSERT statement). This will be fast
Once you have inserted all the record, relate the temporary to your table on UPDATE statement.
Example: UPDATE orig_table ot, temp_table tt SET ot.URLRedirect=tt.URLRedirect WHERE ot.ID=tt.ID
Destroy your temporary table.
The UPDATE statement will set the value of orig_table to the new value from temp_table provided that they relate through their ID.