I have a very large log table from which I want to move rows between specific dates, putting them into the same table structure on an other database. I don't want to copy everything, only specific date rows, so that the table on the main database stays small-ish. So I have to SELECT the data I want and only move (and delete) that.
Keep in mind that there is a lot of data, and I don't want to copy it all with a mysqldump. I want to create a php file or function and I will add a crone job which will run after few days and move all specific data to other database from main database.
Use something like this:
$sql = $db->query("SELECT * FROM old_table LIMIT 100");
while($row = $sql->fetch_assoc()){
$values = "";
foreach($row as $v)
$values .= "'" . $db->real_escape_string($v) . "', ";
$values = rtrim($values, ", ");
$db->query("INSERT INTO new_table (" . implode(",", array_keys($row) . ") VALUES ($values)");
$db->query("DELETE FROM old_table WHERE `ID` = {$row->ID} LIMIT 1");
}
For two databases, use this instead:
$db = new MySQLi("hostname", "user", "password", "database");
$db2 = new MySQLi("hostname", "user", "password", "database");
$sql = $db->query("SELECT * FROM old_table LIMIT 100");
while($row = $sql->fetch_assoc()){
$values = "";
foreach($row as $v)
$values .= "'" . $db->real_escape_string($v) . "', ";
$values = rtrim($values, ", ");
$db2->query("INSERT INTO new_table (" . implode(",", array_keys($row) . ") VALUES ($values)");
$db->query("DELETE FROM old_table WHERE `ID` = {$row->ID} LIMIT 1");
}
Related
I am trying to update the store column by adding new item (string) to the end of the column,
But what happens is the new item is added twice at the end of the column, This is the code:
$query = "SELECT * FROM users";
$result = $conn->query($query);
while($row = $result->fetch_assoc()){
$item = 'item_name';
$store = $row['store'];
$newstore = $store . '|' . $item;
echo 'newstore : ' . $newstore . '<br>'; // It looks normal : store|item
$sql = "UPDATE users SET store='" . $newstore . "' WHERE username='" . $row['username'] . "'";
$conn->query($sql);
}
In in the database I find: store|item|item
Rather than reading the entire table and looping through it with PHP, run just a single UPDATE query to concatenate the extra data onto the column.:
$item = 'item_name';
$query = "UPDATE users SET store=concat(store,'|','$item')";
$result = $conn->query($query);
Note: this form is potentially open to SQL injection if you can't trust the value in $item. You'd do better to use a prepared query if that's the case.
I am receiving an internal server error for my PHP script timing out. I am taking 8000 rows from mySQL database and breaking each row into 3 different tables.
I did some research on stack and ran into this as a solution:
set_time_limit(0);
ignore_user_abort(1);
Now, I am just not receiving the error anymore in chrome, but still not adding to the database.
Here is a little bit of pseudo of what I am doing:
$db = new myConnectDB();
$query = "SELECT * FROM tableName";
$result = $db->query($query);
$numRows = $result->num_rows;
//beerName', '$style' , '$brewery', '$abv','$ibu','$breweryID', '$icon', '$iconBrew' , '$lat', '$long' , '$beerRating')";
for ($i=0; $i < 8000 ; $i++) {
//fetch associate
$row = $result->fetch_assoc();
//get all 13 variables from rom
$db2 = new myConnectDB();
//insert some variables into 1st table
mysqli_close($db2);
//todo: insert BeerTable / check and
$db3 = new myConnectDB();
//insert variables into second table
mysqli_close($db3);
//todo: insert BreweryTable / check and update
$db4 = new myConnectDB();
//insert variables into third table.
mysqli_close($db4);
}
echo $i;
This is bad practice to establish and break connection inside a loop. Try to refactor your code this way:
$db = new myConnectDB();
$db2 = new myConnectDB();
$db3 = new myConnectDB();
$db4 = new myConnectDB();
$query = "SELECT * FROM tableName";
$result = $db->query($query);
$numRows = $result->num_rows;
$insertQuery1 = "INSERT INTO tbl1 (col1, col2, col3) VALUES";
$insertQuery2 = "INSERT INTO tbl2 (col1, col2, col3) VALUES";
$insertQuery3 = "INSERT INTO tbl3 (col1, col2, col3) VALUES";
for ($i=0; $i < 8000 ; $i++) {
//fetch associate
$row = $result->fetch_assoc();
//get all 13 variables from rom
//insert some variables into 1st table
$insertQuery1 .= '(' . $val1 . ',' . $val2 . ',' . $val3 ')';
//todo: insert BeerTable / check and
//insert variables into second table
$insertQuery2 .= '(' . $val4 . ',' . $val5 . ',' . $val6 ')';
//todo: insert BreweryTable / check and update
//insert variables into third table.
$insertQuery1 .= '(' . $val7 . ',' . $val8 . ',' . $val9 ')';
}
$db1->query($insertQuery1);
$db2->query($insertQuery2);
$db3->query($insertQuery3);
mysqli_close($db2);
mysqli_close($db3);
mysqli_close($db4);
In this concept you won't query 3 inserts for each row. You collect 3 big insert queries and execute them after the loop. So this code would work faster.
I'm new to PHP, and I do not it's syntax and principles very well.
I have such code:
function exportFromTransbase($table_name) {
//$odbc_query = "SELECT * FROM " . $table_name. " WHERE ((CDS_CTM subrange(248 cast integer) = 1) AND (CDS_LNG_ID = 16))";
$odbc_query = "SELECT * FROM " . $table_name. "";
$data = odbc_exec($this->odbc_id, $odbc_query);
odbc_longreadlen($data, 10485760);
$oufile=fopen($table_name.".sql", 'w') or die("error writing file");
$q1='INSERT INTO `' . substr($table_name, 4) . '` VALUES';
fwrite($oufile, $q1);
while($row = odbc_fetch_array($data))
{
foreach($row as $key => $value) {
$keys[] = "`" . $key . "`";
if ($value == ""){
$value = 'NULL';
$values[] = "" . mysql_real_escape_string($value) . "";
}
else{
$values[] = "'" . mysql_real_escape_string($value) . "'";
}
//echo "\n \n ololo ".$value;
}
$mysql_query = "\n (".implode(",", $values).")," ;
fwrite($oufile, $mysql_query);
//mysql_query($mysql_query);
set_time_limit(0);
unset($keys);
unset($values);
unset($row);
}
$stat = fstat($oufile);
ftruncate($oufile, $stat['size']-1);
fseek($oufile, 0, SEEK_END);
fwrite($oufile, ";".$r);
//} while ($r < 5 );
fclose($oufile);
if ($mysql_query){
print "Ýêñïîðò äàííûõ èç òàáëèöû " . $table_name . " çàâåðøåí!";
//strtolower(substr($table_name, 4))
}
}
what and where i need to custom, so that i export all table fields except one, for example called Size, i must insert in db is this field nulls....
Also if it is easy, how to split my sql query in batches of 5000 rows? so insert (5000 rows) then insert another 5000....
But first i need to export all fields, except one...
To the best of my knowledge, it can't be possible.
You can use simply:
SELECT field1, field2, field3, field4 FROM table
See Select all columns except one in MySQL?
And if you have more fields in your table then you can use * only.
At the time of insert you can make a condition which ignores the field which you don't want in new table.
Can you use
"select * from {table} where {sth} != {condition}"
Then you can fetch all data except one. The one is excrpted follow your condition.
Meanwhile,if you want to insert by 5000, you can read 5000 lines one time and insert into.
I have 3 tables:
Users - uID (INT AUTO_INCREMENT), name (VARCHAR)
Movies - mID (IN AUTO_INCREMENT), title (VARCHAR)
Watched - uID (INT), mID (INT)
I'm writing a php function that constructs a query which adds records of movies watched by a particular person. Here's what I've come up with so far:
function set_watched($name, $movies){
$sql = "SET #userid = (SELECT uID FROM users WHERE name = '$name' LIMIT 1); ";
$sql .= "INSERT INTO watched (uID, mID) VALUES ('";
foreach ($movies as $index => $movie){
}
}
My question:
Is there a way to combine the #userid variable with the results of a SELECT mID FROM MOVIES WHERE title = $movie OR title = $movie [generated with foreach]?
I don't want to generate separate SELECT statements for every movie title. Perhaps I don't even have to use the #userid variable at all?
Try something like this:
$sql = "INSERT INTO watched (uID, mID)
SELECT User.uID, Movies.mID
FROM (SELECT uID FROM Users WHERE Users.name = '$name' LIMIT 1) AS User, Movies
WHERE ";
foreach ($movies as $index => $movie){
$sql .= "Movies.title = '$movie' OR ";
}
$sql = substr($sql, 0, -4) . ";";
I prefer using arrays and imploding them for this sort of an application. Also, I wouldn't try and force these two things into one query. I would either:
Modify the function parameters to accept uID as its input, instead of name
Change the logic to two queries.
Besides, PHP's mysql_query function doesn't support multiple queries, so if you're using the standard mysql functions, you can't execute two queries with one call to mysql_query.
Running with case #2, you can use something like this (untested, of course):
$sql = 'SELECT uID FROM users WHERE name = "' . $name. '" LIMIT 1';
$result = mysql_query( $sql);
$row = mysql_fetch_row( $result);
mysql_free_result( $result);
$values_array = array();
foreach ($movies as $index => $movie)
{
$values_array[] = '( "' . $row['uID'] . '", "' . $movie . '")';
}
$sql = 'INSERT INTO watched (uID, mID) VALUES ' . implode( ', ', $values_array);
$result = mysql_query( $sql);
I've got a SQL query within a foreach loop. Sometimes there can be many, and I mean a lot of queries to do, depending on several criteria, up to 78 queries potentially.
Now, I know that premature optimisation is root cause of all evil, but I don't want to see 78 queries - it's just not healthy.
Here's the code:
$crumbs = explode(",", $user['data']['depts']);
foreach ($crumbs as &$value) {
$data = $db->query("SELECT id FROM tbl_depts WHERE id = '" . $value . "'");
$crumb = $data->fetch_assoc();
$dsn = $db->query("SELECT msg, datetime FROM tbl_motd WHERE deptid = '" . $value . "'");
$motd = $dsn->fetch_assoc();
if ($motd['msg'] != "") {
<?php echo $motd['msg']; ?>
}
}
Can I make it any better?
Use IN MySQL operator to search over a set of values for id:
$ids = '"' . implode('", "',$crumbs) . '"';
$query1 = "SELECT id FROM tbl_depts WHERE id IN (" . $ids . ")";
$query2 = "SELECT msg, datetime FROM tbl_motd WHERE deptid IN (" . $ids . ")";
And so you will not need to retrieve all data you need using foreach loop, so you will have only 2 queries instead of 78.
Example: I have a table named table with 10 records which ids are: 1,2,3,4,5,6,7,8,9,10 (auto-incremented). I know I need records with ids 1,5,8. My query will be:
$sql = "SELECT * FROM `table` WHERE id in (1,5,8);";
And I don't understand why do you need to use & operator in foreach loop if you don't modify the $crubms arrays values.
I think this is want you want.
SELECT msg, datetime
FROM tbl_depts td
INNER JOIN tbl_motd tm ON td.id = tm.deptid