Moving rows from one database to another using mysqli - php

My question is pretty straight-forward, and I assume that people need to do this pretty often; yet, after hours of searching online, nothing especially enlightening has come up. I am aware of a thread on here about doing the same thing using PDO, but I want to stick to mysqli to be consistent across my whole site.
So, I need to move one or several rows from one database to another (and NOT from one table to another within a single database).
The code I have so far is sure to get some laughs, as this task proved to be considerably beyond my ability. Anyway, here goes:
To begin with, I have two mysqli_connect... not sure if that is encouraged, although it has not caused me problems so far.
$connect_MAIN = mysqli_connect($servername_MAIN, $username_MAIN, $password_MAIN, $dbname_MAIN);
$connect_TEMP = mysqli_connect($servername_TEMP, $username_TEMP, $password_TEMP, $dbname_TEMP);
Here is the meat in the sandwich in terms of code:
///Prepare this in advance because the 'IN' values are taken from a $_GET
$sql_select = "SELECT * FROM Concert WHERE id IN (1, 2, 3, 4)";
///Connect to source DB
$result = $connect_TEMP->query($sql_select);
if ($result->num_rows != 0) {
///Connect to destination DB
$stmt = $connect_MAIN->prepare('INSERT INTO Concert (venue_id, date, ensemble_id, info, title, repertoire, time) VALUES (?, ?, ?, ?, ?, ?, ?)');
$venue_id = $date = $ensemble_id = $info = $title = $repertoire = $time = null;
$stmt->bind_param("isissss", $venue_id, $date, $ensemble_id, $info, $title, $repertoire, $time);
while($row = $result->fetch_assoc()) {
$venue_id = $row["venue_id"];
$date = $row["date"];
$ensemble_id = $row["ensemble_id"];
$info = $row["info"];
$title = $row["title"];
$repertoire = $row["repertoire"];
$time = $row["time"];
if ($stmt->execute() === TRUE) {
if ($show_once == 1) {
echo "Info successfully submitted.";
$show_once = 0;
}
} else {
echo "Hmm, something went wrong..." . $connect_MAIN->error;
}
}
}
$stmt->close();
mysqli_close($connect_TEMP);
}
So, I know that this is a bit of a jumble... as I said, I am not sure if it is possible to connect to DB2 in the middle of a while loop returning values from DB1. Feel free to mock, but only if you have something useful to suggest!

Use a parametrized query instead of concatenating strings.
///Prepare this in advance because the 'IN' values are taken from a $_GET
$sql_select = "SELECT * FROM Concert WHERE id IN (1, 2, 3, 4)";
///Connect to source DB
$result = $connect_TEMP->query($sql_select);
if ($result->num_rows != 0) {
///Connect to destination DB
$stmt = $connect_MAIN->prepare('INSERT INTO Concert (venue, date, info, time) VALUES (?, ?, ?, ?)');
$venue = $date = $info = $time = null;
$stmt->bind_param("ssss", $venue, $date, $info, $time);
while($row = $result->fetch_assoc()) {
$venue = $row["venue"];
$date = $row["date"];
$info = $row["info"];
$time = $row["time"];
if ($stmt->execute()) {
///This is just to stop message from appearing multiple times.
if ($show_once == 1) {
echo "Info successfully submitted.";
$show_once = 0;
}
} else {
echo "Hmm, something went wrong..." . $stmt->error;
}
}
$stmt->close();
mysqli_close($connect_TEMP);
}

Yes, you may establish a second DB connection at any point in your script. There's nothing preventing it nor there's a reason to discourage it. It's far more common than you'd think (for different reasons). However, considering the overhead that actually connecting to the database carries, I would advice you to not establish a connection on each iteration (this is the best practice, performance-wise)
What I would do:
First, connect to the source database, get the data you need
Not much to say here, you already did and it looks OK. Check, as you did, that there's at least one result before moving on. If the resultset is empty, don't go any further (exit)
If there was at least one row in the resultset, establish a connection to the destination database
Do this before starting to loop to avoid the performance penalty of establishing multiple connections. If you're looking at hundreds/thousands or more rows, you'll really notice the difference. Once the connection is established, move on
Loop through the source data and prepare the insert statements
You have a choice here. Either a) loop through the whole resultset and prepare one single insert with multiple rows which you'll insert all at once at the end of the loop or b) create a single insert (single row) on each iteration, run the insert and then move on to the next iteration.
There's reasons for and against both strategies. Choose the one that suits you better.
In algorythmical terms, your code is pretty much ready (just connect to the destination DB once before you start looping). You may benefit from using a parameterized query with bindings instead of writing the full insert string, but other than that you're pretty much there.

If you are running your own MySql server, you may do this in a simpler way using the Federated engine.
The federated table behaves like a normal table, but it stores the data in a remote database. With this engine you can do someting like this:
INSERT INTO remoteTable (values) SELECT * FROM localTable

Related

PHP & Mysql Syntax incorrect?

I am having an issue with this not running at all when the information is submitted, I believe that I have the syntax wrong.
if ($_POST['note'] != $player->note_text) {
$message = 'Admin '.$user.' has added the note ('.$_POST['note'].') to '.$player->name.'('.$pid.')';
logIt($user, $message, $dbcon);
$note = $_POST['note'];
$note = '"'.$note.'"';
$UpdateN = "INSERT INTO notes (uid, staff_name, name, alias, note_text, warning) VALUES ('$_POST[hidden]', '$user', '$player->name', '$player->aliases', '$note','$_POST[warn]')";
$UpdateN2 = "INSERT INTO players WHERE `playerid` = $_POST[hidden] (warning) VALUES ('$_POST[warn]')";
mysqli_query($dbcon, $UpdateN, $UpdateN2);
The new line I added which seems to have broken it is '$UpdateN2'.
I am not sure if the new line has broken the statement, since I am new to PHP and mysqli any pointers to forums or websites that I can learn and understand this in a more detailed way I would appreciate.
Edit: I have since moved from using mysqli to PDO, I strongly suggestion that anyone willing to start using MYSQL commands with PHP to have a look at this: http://wiki.hashphp.org/PDO_Tutorial_for_MySQL_Developers
I started reading this and testing things out, I find this much more easier but also it looks a lot cleaner and understandable when reading it back after a few days have passed.
In keeping with the mysqli procedural style in the original, using prepared statements with bind placeholders, something like this:
$UpdateN = 'INSERT INTO notes (uid, staff_name, name, alias, note_text, warning)'
. ' VALUES ( ?, ? , ? , ? , ? , ? )';
$UpdateN2 = 'UPDATE players SET warning = ? WHERE playerid = ? ';
if( $sth = mysqli_prepare($dbcon,$UpdateN) ) {
mysqli_stmt_bind_param($sth,'ssssss'
,$_POST['hidden']
,$user
,$player->name
,$player->aliases
,$_POST['note']
,$_POST['warn']
);
if( mysqli_stmt_execute($sth) ) {
// statement execution successful
} else {
printf("Error: %s\n",mysqli_stmt_error($sth));
}
} else {
printf("Error: %s\n",mysqli_error($dbcon));
}
if( $sth2 = mysqli_prepare($dbcon,$UpdateN2) ) {
mysqli_stmt_bind_param($sth2,'ss'
,$_POST['warn']
,$_POST['hidden']
);
if( mysqli_stmt_execute($sth2) ) {
// statement execution successful
} else {
printf("Error: %s\n",mysqli_stmt_error($sth2));
}
} else {
printf("Error: %s\n",mysqli_error($dbcon));
}
If it was me, I'd just make two separate calls to mysqli_query, one for each of your separate queries. PHP has historically been very wary of permitting multiple queries in a single function call, because of possible sql injection vulnerabilities.
On a related note, you need to sanitize your input. $_POST should never, ever, ever appear directly in a mysql query string, because your post data might -actually be something like ')"; DROP TABLE users;'.
Finally, you're using a WHERE clause on an insert. That's probably breaking your query. You should take a couple of days and research how and why that's breaking your query, and how and why to write proper sql queries. Doing so will add value to your company, because your company is less likely to suffer a catastrophic data breach, and it will add value to you, because you'll be a better SQL coder.
Edit: and in the time it took me to write this, three different people made each of those points. LOL.

Multiple row inserts as fast as possible

I've seen multiple threads discussing this but there always has been totally different conclusion in the answers. Especially I wonder whether it is really necessary to create a own prepared statement (with the right amount of placeholders) in order to insert it as single query. I expected that when I use beginTransaction and endTransaction before and after my for loop, that pdo/php waits with the transaction until all data is collected and it will send these data's as a single query once the server hits the line endTransaction.
How would I need to rewrite such a for loop insert with multiple inserts in order to reach the best performance (it has between 1 and 300 rows usually but it also could reach 2000 rows).
for($i=0; $i<$baseCount; $i++)
{
$thLevel = $bases[$i]["ThLevel"];
$gold = $bases[$i]["Gold"];
$elixir = $bases[$i]["Elixir"];
$darkElixir = $bases[$i]["DarkElixir"];
$dateFound = $elixir = $bases[$i]["TimeFound"];
$query = $db->prepare("INSERT INTO bot_attacks_searchresults (attack_id, available_gold, available_elixir, available_dark_elixir, date_found, opponent_townhall_level)
VALUES (:attack_id, :available_gold, :available_elixir, :available_dark_elixir, :date_found, :opponent_townhall_level)");
$query->bindValue(':attack_id', $attackId);
$query->bindValue(':available_gold', $gold);
$query->bindValue(':available_elixir', $elixir);
$query->bindValue(':available_dark_elixir', $darkElixir);
$query->bindValue(':date_found', $dateFound);
$query->bindValue(':opponent_townhall_level', $thLevel);
$query->execute();
}
Prepare the statement once. MySQL lexes it once, so any subsequent call to the query will be quick since it's already lexed and juts needs parameters.
Start the transaction before the loop. This is done so your hard drive can write down all the rows in one input output operation. The default mode is that 1 insert query = 1 I/O of the hdd.
Create the loop, bind your parameters there and call the $query->execute();
Exit the loop and commit() the transaction.
Full code:
$db->beginTransaction();
$query = $db->prepare("INSERT INTO bot_attacks_searchresults (attack_id, available_gold, available_elixir, available_dark_elixir, date_found, opponent_townhall_level)
VALUES (:attack_id, :available_gold, :available_elixir, :available_dark_elixir, :date_found, :opponent_townhall_level)");
for($i = 0; $i < $baseCount; $i++)
{
$thLevel = $bases[$i]["ThLevel"];
$gold = $bases[$i]["Gold"];
$elixir = $bases[$i]["Elixir"];
$darkElixir = $bases[$i]["DarkElixir"];
$dateFound = $elixir = $bases[$i]["TimeFound"];
$query->bindValue(':attack_id', $attackId);
$query->bindValue(':available_gold', $gold);
$query->bindValue(':available_elixir', $elixir);
$query->bindValue(':available_dark_elixir', $darkElixir);
$query->bindValue(':date_found', $dateFound);
$query->bindValue(':opponent_townhall_level', $thLevel);
$query->execute();
}
$db->commit();
Here's a very crude proof of concept:
<?php
$values = array();
for($i=0;$i<10;$i++)
{
$values[] = "($i)";
}
$values = implode($values,',');
$query = "INSERT INTO my_table VALUES $values";
echo $query;
?>
outputs INSERT INTO my_table VALUES (0),(1),(2),(3),(4),(5),(6),(7),(8),(9)
You would need to restructure this slightly to work with prepare (PHP is not my forte), but the principle is the same; i.e. you build the query inside the loop, but execute it only once.

PHP sql database, inserting UNIQUE values

Im working with sql database (phpMyadmin) and with programming language PHP.
I've got a table with data e.g:
T1:
145
138.8
110.6
1759.4
1400.9
1891.4
1755.4
1760.4
T2:
141.9
110.6
1400.9
1891.4
1758.4
My question >
Is there existing sql syntax, that is able to average values which are lying in ONE THRESHOLD (like > 1755.4,1760.4,1759.4 OR 145,138.8) and INSERT THEM AS UNIQUE VALUES to another table ?
What I need is create T2(contain UNIQUE values) from T1, and INSERTING them only if they didn't exist in T2.
I was trying, to looping throw 2 arrays (1starray = T1 and 2ndarray = T2 ) too. And compare data in array1 and array2, and trying to insert values which can be unique.
for($z = 0; $z<$numberApp; $z++) {
while($j<$numberAbs-1){
$Abs=$arrayabs[$j];
$state_change = (abs($Abs - abs($previous_Abs)));
if ($state_change>100){
$instantChange = True;
echo "<br>"." better than tolerance: " . $instantChange;
}
else {
$instantChange = False;
echo "<br>"." lower than tolerance" . $instantChange;
}
//if($state_change = (abs($some_item - $previous_item)<= $tolerance)){
if($Abs != $previous_Abs){
echo "<br>" ." some_item : " .$Abs." previous_item : " .$previous_Abs;
}
//$lastAbs=$arrayabs[$j];
//echo " next_item : " .$lastAbs;
if ($instantChange == True and $onChange == False){
$Abs2 [] = $podiel;
$finalPower = $podiel;
}
if ($instantChange){
$D=0;
echo "ABSOLUT >>> " .$lastAbs;
$percentual1 = ($power_app[$z] * 0.9);
$percentual2 = ($power_app[$z] * 1.1);
if(($lastAbs<$percentual1 || $lastAbs>$percentual2)){
$user = 'root';
$pass = 'password';
$db = 'vypis';
//if($N>0){
$db = new mysqli('localhost', $user, $pass, $db) or die();
$sql = "INSERT INTO vypis.nilm_app2(power) VALUES ('$lastAbs')";
if(mysqli_query($db, $sql)){
echo "Records added successfully.";
} else{
echo "ERROR: Could not able to execute $sql. " . mysqli_error($db);
}
}
//$power_app[$z++];
}
Thank you for everyone, who going to help, and sorry for my bad english.
The math here sounds complex enough that you're probably better off doing the complex stuff (averaging the values and determining whether a value is unique) in PHP, then finishing off with a couple simple MySQL statements.
This can probably be done in pure MySQL with a bit of trickery, but frequently it isn't worth the time and complexity it creates. (Imagine you'll have to debug that monster SQL query six months from now!)
By contrast, it's trivial to take a handful of values in PHP and average them; it's also trivial to run a quick MySQL query to determine whether a value is unique. So if you're unsure how to mash these together into one big SQL statement, start out by just handling them in the PHP code as separate steps! Then later on, if performance issues come up, you can think about how to combine them (but at least by that point you already have something that works).

Seemingly identical sql queries in php, but one inserts an extra row

I generate the below query in two ways, but use the same function to insert into the database:
INSERT INTO person VALUES('','john', 'smith','new york', 'NY', '123456');
The below method results in CORRECT inserts, with no extra blank row in the sql database
foreach($_POST as $item)
$statement .= "'$item', ";
$size = count($statement);
$statement = substr($statement, 0, $size-3);
$statement .= ");";
The code below should be generating an identical query to the one above (they echo identically), but when I use it, an extra blank row (with an id) is inserted into the database, after the correct row with data. so two rows are inserted each time.
$mytest = "INSERT INTO person VALUES('','$_POST[name]', '$_POST[address]','$_POST[city]', '$_POST[state]', '$_POST[zip]');";
Because I need to run validations on posted items from the form, and need to do some manipulations before storing it into the database, I need to be able to use the second query method.
I can't understand how the two could be different. I'm using the exact same functions to connect and insert into the database, so the problem can't be there.
below is my insert function for reference:
function do_insertion($query) {
$db = get_db_connection();
if(!($result = mysqli_query($db, $query))) {
#die('SQL ERROR: '. mysqli_error($db));
write_error_page(mysqli_error($db));
} #end if
}
Thank you for any insite/help on this.
Using your $_POST directly in your query is opening you up to a lot of bad things, it's just bad practice. You should at least do something to clean your data before going to your database.
The $_POST variable often times can contain additional values depending on the browser, form submit. Have you tried doing a null/empty check in your foreach?
!~ Pseudo Code DO NOT USE IN PRODUCTION ~!
foreach($_POST as $item)
{
if(isset($item) && $item != "")
{
$statement .= "'$item', ";
$size = count($statement);
$statement = substr($statement, 0, $size-3);
$statement .= ");";
}
}
Please read #tadman's comment about using bind_param and protecting yourself against SQL injection. For the sake of answering your question it's likely your $_POST contains empty data that is being put into your query and resulting in the added row.
as #yycdev stated, you are in risk of SQL injection. Start by reading this and rewrite your code by proper use of protecting your database. SQL injection is not fun and will produce many bugs.

Insert query with special character in php

Here is my code
<?php
$con = mysql_connect('localhost','test','test');
mysql_select_db('test',$con);
require_once("xml2json.php");
$testXmlFile = 'mytest.xml';
$xmlStringContents = file_get_contents($testXmlFile);
$jsonContents = "";
$jsonContents = xml2json::transformXmlStringToJson($xmlStringContents);
$obj =json_decode($jsonContents);
$rows = array();
foreach($obj->rss->channel->item as $item) {
echo $item->title."\n";
echo $item->description."\n";
$rows[] = "('".mysql_real_escape_string($item->title)."','".mysql_real_escape_string($item->description)."')";
}
$del_horoscope = "delete from jos_horoscope";
mysql_query($del_horoscope);
$query = mysql_real_escape_string("INSERT INTO `jos_horoscope` (`title`,`description`) VALUES ".implode(', ',$rows));
mysql_query($query);
if (!mysql_query($query)) echo 'Oh no! Something went wrong with the query: '.mysql_error();
?>
I am not able to insert the title and description in DB. It always says
Oh no! Something went wrong with the query: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '\'What\'s special today for you\',\'Your birth on the 25th day of the month (7 e' at line 1
Some of the Data which I am trying to insert is
What's special today for you
Your birth on the 25th day of the month (7 energy) modifies
your life path by giving you some special interest in technical,
scientific, or other complex and often hard to understand subjects.
You may become something of a perfectionist and a stickler for
details. Your thinking is logical and intuitive, rational and responsible.
Your feelings may run deep, but you are not very likely to let them
show. This birthday makes you a more private person, more
introspective and perhaps more inflexible.
Aries Horoscope for Thursday, April 25, 2013
Although most of us when we make a decision we don`t have to worry what it will impact for the world we live in, but when you make a decision we have to remember that it does impact our society in so many ways. Our choice has to be an ethical one, regardless of the outcome. Your emotions are brought into more conscious focus these few days. You may find that you can more easily communicate your feelings at this time.
The First line is title and rest is Description till Aries Horoscope . After that another title starts . Please help me . I tried many options but its not working for me
Here is my table structure
Please help me .
You are escaping your query multiple times, here is the correct way:
foreach($obj->rss->channel->item as $item) {
$rows[] = "('".mysql_real_escape_string($item->title)."','".mysql_real_escape_string($item->description)."')";
}
Now you don't need to escape it again:
$query = "INSERT INTO `jos_horoscope` (`title`,`description`) VALUES ".implode(', ',$rows);
mysql_query($query);
First of all, you don't want to escape the actual INSERT string. Just the values you are going to INSERT.
$query = mysql_real_escape_string("INSERT INTO `jos_horoscope` (`title`,`description`) VALUES ".implode(', ',$rows));
Is what you are currently doing. Just escape each of the values you are going to insert. Which leads us to the next problem. Your $rows[] array you are building inside the foreach loop isn't formatted properly if the foreach runs multiple times. Your insert should be inside of the foreach. And just scrap the array.
foreach($obj->rss->channel->item as $item) {
echo $item->title."\n";
echo $item->description."\n";
$title = mysql_real_escape_string($item->title);
$description = mysql_real_escape_string($item->description);
$query = "INSERT INTO `jos_horoscope` (`title`,`description`) VALUES ('$title', '$description')";
mysql_query($query);
}
Then I'm not really sure what the delete query is for.

Categories