How to compare tables from different mySQL databases with PHP? - php

I'm new to PHP as well as the field and have been tasked with finding inconsistencies amongst tables from different mySQL databases. Every table should be the same between databases (column names, column count), with the exception of the actual data held in it, which they are not. I will need to identify the offending columns, the table they are in and, the database they are in. Below is my code thus far:
<?php
chdir(<path>);
include(<file>);
include(<file>);
$db = new db($hostname, $user, $pass, $commondatabase);
// Get baseline DB columns
//$db->select_db('<baseDB>');
//$btq = "SHOW TABLES";
//$btr = $db->query($btq);
//while($trows = $db->fetch_assoc($btr) {
// $bcq = "SHOW COLUMNS FROM ".$btr[<key>];
// $bcr = "$db->query($bcq);
// $basecolumns[] = $bcr;
//}
$dbq = "SELECT * FROM <commonDB>.<DBnames> ORDER BY <DBnamesID>";
$dbr = $db->query($dbq);
while($dbrow = $db->fetch_assoc($dbr)) {
echo "\n";
print_r($dbrow['dbname']);
echo "\n";
$db->select_db($dbrow['dbname']);
$tq = "SHOW TABLES";
$tr = $db->query($tq);
while($table = $db->fetch_assoc($tr)) {
/*print_r($dbtables);*/
$cq = "SHOW COLUMNS FROM ".$table['Tables_in_'.$dbrow['dbname']];
$cr = $db->query($cq);
$dbcols = [];
while($col = $db->fetch_assoc($cr)) {
/*print_r($col);*/ $dbcols = $col;
// Do check against baseline here
//if($dbcols != $basecolumns) {
// $badcolumns[] = $dbcols;
//}
}
}
}
$db->close();
?>
Currently it will loop down to the columns, but the output is not visually pleasing nor very manageable. My first concern is to get this working to where I can loop down to the columns and actually get them in their own arrays to be checked against the baseline and I've hit a wall, so any direction would be much appreciated. Currently each column is being assigned to its own array versus an array of all the column names for the database and table the loop is on. TIA!

Here is what I came up with. Not sure if it's the most DRY way to go about it, but it works and achieves the results I was looking for. Thank you to Barmar for sending me down the right path.
<?php
chdir(<path>);
include(<file>);
include(<file>);
$db = new db($hostname,$user,$pass,$commondatabase);
$db->select_db(<database>);
// Get tables from baseline database
$q="select distinct <column-name> from <table-name> where <where-param> ORDER BY <order-by-param>";
$r=$db->query($q);
while($tables=$db->fetch_assoc($r)) {
$table = $tables[<key>];
$x = array();
$x["$table"]=array();
// Get table columns from baseline database
$q="select <column-name> from <table-name> where <where-param> and <where-param> ORDER BY <order-by-param>";
$r2=$db->query($q);
while ($columns=$db->fetch_assoc($r2)) {
$x["$table"][]=$columns[<key>];
}
// Check other databases for this table
$q="select * from $commondatabase.<table-with-database-names> ";
$q.="where <where-param> order by <order-by-param>";
$r2=$db->query($q);
while ($databases=$db->fetch_assoc($r2)) {
$y = array();
$y["$table"]=array();
// Get table columns in $databases[<key>]
$q="select <column-name> from <table-name> where <where-param> and <where-param> ORDER BY <order-by-param>";
$r3=$db->query($q);
while ($columns=$db->fetch_assoc($r3)) {
$y["$table"][]=$columns[<key>];
}
// Check against baseline
$z1 = array_diff($x["$table"],$y["$table"]);
$z2 = array_diff($y["$table"],$x["$table"]);
// Echo out comparison results
if (empty($z1) && empty($z2)) {
//echo "all good.\n";
} else {
echo "Difference found in {$databases[<key>]}.{$tables[<key>]}:";
if (!empty($z1)) print_r($z1);
if (!empty($z2)) print_r($z2);
}
}
}
$db->close();
?>

Related

How to dynamically display links in web pages using PHP

Good day # all.
I've this code snippet which's aim is to display the Exam this user is qualified to take based on the courses registered for. It would display the Exam Name, Date Available, Passing Grade and either Take Exam link if he/she hasn't written or View Result if he/she has written previously.
/*Connection String */
global $con;
$user_id = $_SESSION['user_id']; //user id
$courses = parse_course($user_id); //parse course gets the list of registered courses (Course Codes) in an array
foreach ($courses as $list)
{
$written = false;
$list = parse_course_id($list); //parse_course_id gets the id for each course
$ers = mysqli_query($con, "Select * from exams where course_id = '$list'");
while ($erows = mysqli_fetch_assoc($ers)) {
$trs = mysqli_query($con, "Select * from result_data where user_id = '$user_id'");
while ($trows = mysqli_fetch_assoc($trs)) {
if ($trows['user_id'] == $user_id && $trows['exam_id'] == $erows['exam_id'])
$written = true;
else
$written = false;
}
if($written)
{
echo "<tr><td>".$erows['exam_name']."</td><td>".$erows['exam_from']." To ".$erows['exam_to']."</td><td>".$erows['passing_grade']."%</td><td>".'View Result '."</td></tr>";
$written = false;
}
else
{
echo "<tr><td>".$erows['exam_name']."</td><td>".$erows['exam_from']." To ".$erows['exam_to']."</td><td>".$erows['passing_grade']."%</td><td>".'Take Exam '."</td></tr>";
$written = false;
}
}
}
But It only displays one View Result entry even if I've taken more than one exam. It shows the recent entry. Please what am I missing?
Untested, but here's how I would do it.
I've assumed $user_id is an integer. I'm a bit worried about it being used in SQL without any sanitization. I can't guarantee anything else you're doing is secure either because I can't see your other code. Please read: http://php.net/manual/en/security.database.sql-injection.php
(Oh I see someone already commented on that - don't take it lightly!)
Anyway, my approach would be to collect the user's written exam IDs into an array first. Then loop through the available exams and check each exam id to see if it's in the array we made earlier.
I wouldn't bother looking into the join advice unless you find this is performing poorly. In many systems it would be common to have 3 functions in this situation, one that generates $users_written_exam_ids ones that pulls up something like $all_available_exams and then this code which compares the two. But because people are seeing both queries here together there is a strong temptation to optimize it, which is cool but you probably just want it to work :)
<?php
global $con;
// Get the user id. Pass through intval() so no SQL injection is possible.
$user_id = intval($_SESSION['user_id']);
// Parse course gets the list of registered courses (Course Codes) in an array
$courses = parse_course($user_id);
foreach ($courses as $list)
{
// Gets the id for each course
$list = parse_course_id($list);
$users_written_exam_ids = array();
$trs = mysqli_query($con, "SELECT exam_id FROM result_data WHERE user_id = '$user_id'");
while ($trows = mysqli_fetch_assoc($trs))
{
$users_written_exam_ids[] = $trows['exam_id'];
}
$ers = mysqli_query($con, "SELECT * FROM exams WHERE course_id = '$list'");
while ($erows = mysqli_fetch_assoc($ers)) {
echo '<tr><td>' . $erows['exam_name'] . '</td><td>' . $erows['exam_from']
. ' To ' . $erows['exam_to'] . '</td><td>' . $erows['passing_grade']
. '%</td><td>';
if (in_array($erows['exam_id'], $users_written_exam_ids))
{
echo 'View Result';
}
else
{
echo 'Take Exam';
}
echo '</td></tr>';
}
}

PHP: Fastest way to loop through a array / stringbuiler or something else

We have a PHP script that loops through many XML / CSV files from different websites. Right now we manage to build a good XML / CSV parser script.
The PHP script we wrote is looping though some BIG XML or CSV files. In these XML or CVS files contains Barcodes from different products.
Right now before the script starts I fill an array with the Product ID + Barcode from the MySQL like this:
function Barcodes_Array() {
$sql = "SELECT ProductId, Barcode FROM Products WHERE (Barcode <> '') ";
$res = mysql_query($sql);
while ($rijen = mysql_fetch_assoc($res)) {
$GLOBALS['arrBarcodes'][] = $rijen;
}
}
Each time we loop through the XML (or CSV) files we have to check if the Barcode exists in the array and return the Product ID.
For searching in the function:
$ProductId = SearchBarcodeProduct($EanNr, 'Barcode');
And yet the function:
function SearchBarcodeProduct($elem, $field)
{
$top = sizeof($GLOBALS['arrBarcodes']) - 1;
$bottom = 0;
$ProductId = 0;
while($bottom <= $top)
{
if($GLOBALS['arrBarcodes'][$bottom][$field] == $elem) {
return $GLOBALS['arrBarcodes'][$bottom]['ProductId'];
}
else {
if (is_array($GLOBALS['arrBarcodes'][$bottom][$field])) {
if (in_multiarray($elem, ($GLOBALS['arrBarcodes'][$bottom][$field]))) {
return $GLOBALS['arrBarcodes'][$bottom]['ProductId'];
}
}
}
$bottom++;
}
return $ProductId;
}
We fill in the array because it took forever each time we ask the MySQL Products Table.
My Question is now:
It still takes a VERY long time each time looping through the array of the barcodes. Is there a faster way for any other solutions maybe a different way then a array?
Can someone help please i am working like weeks on this stupid :) thing!
Why do you need 2 functions?
Try just one
function itemBarcode($id) {
$id = intval($id);
$sql = "SELECT ProductId, Barcode FROM Products WHERE ProductId = $id Barcode <> '') ";
$res = mysql_query($sql);
if ($row = mysql_fetch_assoc($res)) {
return $row['barcode'];
} else {
return 0;
}
}
Update if you need to search by barcode you can create another function:
function itemProduct($barcode) {
$sql = "SELECT ProductId, Barcode FROM Products WHERE Barcode = $barcode ";
$res = mysql_query($sql);
if ($row = mysql_fetch_assoc($res)) {
return $row['ProductId'];
} else {
return 0;
}
}
Sounds like you are missing an index on your Barcode column in your database.. A single row lookup using a presumably unique single indexed column should be blisteringly fast.
CREATE INDEX Barcode_Index ON Products (Barcode)
Then simply:
SELECT ProductId FROM Products WHERE Barcode = *INPUT*
You could also make the index UNIQUE if you NULL the Barcode where they currently = '' if there are more than one of these.
Another option is keying the array you have with the Barcode:
while ($rijen = mysql_fetch_assoc($res)) {
$GLOBALS['arrBarcodes'][$rijen['Barcode']] = $rijen;
}
or even just:
while ($rijen = mysql_fetch_assoc($res)) {
$GLOBALS['arrBarcodes'][$rijen['Barcode']] = $rijen['ProductId'];
}
Then you can do a straight look up:
$ProductId = isset($GLOBALS['arrBarcodes'][$Barcode])
?$GLOBALS['arrBarcodes'][$Barcode]['ProductId']
:0;
or:
$ProductId = isset($GLOBALS['arrBarcodes'][$Barcode])
?$GLOBALS['arrBarcodes'][$Barcode]
:0;
N.B Please read the warnings in the comments about use of $GLOBALS and mysql_query.
If you need it, store the barcodes array in an object or variable instead.
PDO is pretty handy, and I think it can also key your returned array for you on fetch.

Grouping Mysql PHP Query Results

I have a basic PHP page which searches a MySQL Database of retro ZX Spectrum magazine articles.
The address is [www.retroresource.co.uk][1]
The PHP Results Page is fine, in that it produced the results, but I would like to be able to group the results by 'fgames.fgname'
So for example if I searched for Dizzy
Dizzy (as a header)
Table with rows that have fgames.fgname = Dizzy
Spindizzy (as a header)
Table with rows that have fgames.fgname = Spindizzy
Treasure Island Dizzy (as a header)
Table with rows that have fgames.fgname = Treasure Island Dizzy
etc etc
I have googled and looked in a few of my text books, but am unable to see a solution. Any help much appreciated.
Peter
Amended Code after assistance (No results though)
<?php
error_reporting(E_ALL);
ini_set('display_errors', TRUE);
ini_set('display_startup_errors', TRUE);
require( '../connect_db.php' ) ;
$query = $_GET['query'];
$min_length = 4;
// you can set minimum length of the query if you want
$query = strip_tags($query);
$query = mysqli_real_escape_string ($dbc,$query);
$q = "SELECT fgames.fgname, fgames.fgprorg, fgames.fgprbud, fgames.fgratng, fgames.fgprdsk, ftypes.fttname, frefs.fryymm, frefs.frpage, frefs.frissue, fmagzne.fmname, frefcde.ffname
FROM fgames, ftypes,frefs, fmagzne,frefcde
WHERE ftypes.fttype = fgames.fgtype
and fgames.fglink = frefs.frlink2
and frefs.frentry = frefcde.ffentry
and frefs.frmag = fmagzne.fmmag
and fgames.fgname LIKE '%".$query."%'
order by fgames.fgname ASC" ;
$r = mysqli_query( $dbc , $q ) ;
$current_group = '';
while ( $row = mysqli_fetch_array( $r , MYSQLI_ASSOC ) )
{
if($row['fgames.fgname']) !== $current_group)
{
if($current_group !== '')
{
echo '</table>';
}
$current_group = $row['fgames.fgname'];
echo '<table><tr><th>FGNAME</th><th>FGPRORG</th><th>FGRBUD</th><th>FGRATNG</th><th>FGRDSK</th>
<th>FTTTNAME</th><th>FRYYMM</th><th>FRPAGE</th><th>FRISSUE</th><th>FMNAME</th><th>FFNAME</th></tr>';
}
echo '<tr><td>'.$row['fgname'].'</td><td>'.$row['fgprorg'].'</td><td> '.$row['fgprbud'].' </td><td>'.$row['fgratng'].' </td> <td>'.$row['fgprdsk'].' </td><td>'.$row['fttname'].'</td><td> '.$row['fryymm'].'</td><td> '.$row['frpage'].'</td><td> '.$row['frissue'].' </td> <td>'.$row['fmname'].' </td><td>'.$row['ffname'].'</td></tr>';
}
echo '</table>';
else
{
echo '<p>' . mysqli_error( $dbc ) . '</p>' ;
}
# Close the connection.
mysqli_close( $dbc ) ;
?>
There are two basic approaches.
Either load the query data into a two dimensional array and output with nested loop like this:
$array_for_output = array();
while($row = /* your query fetch mechanism here */) {
$array_for_output[$row['field_you_are_grouping_on']] = $row;
}
foreach($array_for_output as $table_name => $table_data) {
// output table header here
foreach($table_data as $table_row) {
// output individual row
}
// output table closure here
}
Or the other approach is to query using an ORDER BY clause on the field you are trying to group by and detect changes in the current group in single loop
$query = "
SELECT ...
FROM ...
WHERE ...
ORDER BY field_you_are_grouping_on, any_secondary_ordering_field_here ASC";
// execute query here
$current_group = '';
while ($row = /* your query fetch mechanism here */) {
if($row['field_you_are_grouping_on']) !== $current_group) {
// we have started a new group
if($current_group !== '') {
// this is not first table
// so output closure to previous table here
}
$current_group = $row['field_you_are_grouping_on'];
// output new table header here
}
// output row data here
}
// output closure to final table here
The second approach is usually preferable from a memory management standpoint (you don't have to store entire result set in memory at any point), but some beginning developers may not find this not as intuitive as the first method. The first method also may be more appropriate if you want to take the full result set and inject it into some sort of template that is used to render the display.

Very slow PHP - MySQL script

I am new at using PHP-MySQL. I have two MySQL tables:
Concreteness: A table that contains concreteness scores for 80K words
Brian: A table with 1 million rows, each containing one or two words.
I have a small PHP script that takes each row in "Brian", parses it, looks for the scores in "Concreteness" and records it in "Brian."
I have been running this script with several other tables that had 300-400k rows with each hundreds of words. "Brian" is different because it has 1 million rows with 1 or 2 words per row. For some reason, my script is SUPER slow with Brian.
Here is the actual script:
<?php
include "functions.php";
set_time_limit(0); // NOTE: no time limit
if (!$conn)
die('Not connected : ' . mysql_error());
$remove = array('{J}','{/J}','{N}','{/N}','{V}','{/V}','{RB}','{/RB}'); // tags to remove
$db = 'LCM';
mysql_select_db($db);
$resultconcreteness = mysql_query('SELECT `word`, `score` FROM `concreteness`') or die(mysql_error());
$array = array(); // NOTE: init score cache
while($row = mysql_fetch_assoc($resultconcreteness))
$array[strtolower($row['word'])] = $row['score']; // NOTE: php array as hashmap
mysql_free_result($resultconcreteness);
$data = mysql_query('SELECT `key`, `tagged` FROM `brian`') or die(mysql_error()); // NOTE: single query instead of multiple
while ($row = mysql_fetch_assoc($data)) {
$key = $row['key'];
$tagged = $row['tagged'];
$weight = $count = 0;
$speech = explode(' ', $tagged);
foreach ($speech as $word) {
if (preg_match('/({V}|{J}|{N}|{RB})/', $word, $matches)) {
$weight += $array[strtolower(str_replace($remove, '', $word))]; // NOTE: quick access to word's score
if(empty($array[strtolower(str_replace($remove, '', $word))])){}else{$count++;}
}
}
mysql_query('UPDATE `brian` SET `weight`='.$weight.', `count`='.$count.' WHERE `key`='.$key, $conn) or die(mysql_error());
// Print out the contents of the entry
Print "<b>Key:</b> ".$info['key'] . " <br>";
}
mysql_free_result($data);
?>
I guess the real problem is the 1 million mysql update statements you fire to the database. Consider bundling the update statements (and also remove the print):
$i=0;
while ($row = mysql_fetch_assoc($data)) {
// ... left out the obvious part
$sql .= "'UPDATE `brian` SET `weight`='.$weight.', `count`='.$count.' WHERE `key`='.$key;";
$i++;
if ($i%1000 == 0) {
mysql_query($sql) or die(mysql_error());
$i=0;
$sql = "";
}
}
// remember to save the last few updates
mysql_query($sql) or die(mysql_error());

PHP/MySQL: Copy Table and Data from one Database to another

Anoter question: What is the best way to copy a whole table from one Database without using something like this:
CREATE TABLE DB2.USER SELECT * FROM DB1.USER;
This does not work because I can't use data from two Databases at the same time. So I decided to make this with php. There I had to cache all Data and then I'd create a table in the other DB.
But now - what would be the fastest way to cache the data? I guess there are closely everytime less than 1000 records per table.
Thanks for your input
Export it to a .sql file and import to the new database.
In phpMyAdmin click the database you want to export, click export, check save as file, then go.
Then upload it to the new database that you want to duplicate the data in.
In a strict PHP sense, the only way I could think to do it would to be use of SHOW TABLES and describe {$table} to return the all the field names and structures of the records, parse that out, create your create table statements, then loop through each table and create insert statements.
I'm afraid the best I can do for you is a sort of prototype code that I would imagine would be incredibly server intensive, which is why I recommend you go an alternate route.
Something like:
<?php
// connect to first DB
$tables = mysql_query("SHOW TABLES") or die(mysql_error());
while ($row = mysql_fetch_assoc($tables)) {
foreach($row as $value) {
$aTables[] = $value;
}
}
$i = 0;
foreach ($aTables as $table) {
$desc = mysql_query("describe " . $table);
while ($row = mysql_fetch_assoc($desc)) {
$aFields[$i][] = array($row["Field"],$row["Type"],$row["Null"],$row["Key"],$row["Default"],$row["Extra"]);
}
$i++;
}
// connect to second DB
for ($i = 0; $i < count($aTables); $i++) {
// Loop through tables, fields, and rows for create table/insert statements
$query = 'CREATE TABLE IF NOT EXISTS {$aTables[$i]} (
//loop through fields {
{$aFields[$i][$j][0]} {$aFields[$i][$j][1]} {$aFields[$i][$j][2]} {$aFields[$i][$j][3]} {$aFields[$i][$j][4]} {$aFields[$i][$j][5]},
{$aFields[$i][$j][0]} {$aFields[$i][$j][1]} {$aFields[$i][$j][2]} {$aFields[$i][$j][3]} {$aFields[$i][$j][4]} {$aFields[$i][$j][5]},
{$aFields[$i][$j][0]} {$aFields[$i][$j][1]} {$aFields[$i][$j][2]} {$aFields[$i][$j][3]} {$aFields[$i][$j][4]} {$aFields[$i][$j][5]},
{$aFields[$i][$j][0]} {$aFields[$i][$j][1]} {$aFields[$i][$j][2]} {$aFields[$i][$j][3]} {$aFields[$i][$j][4]} {$aFields[$i][$j][5]},
etc...
}
)';
//loop through data
$query .= 'INSERT INTO {$aTables[$i]} VALUES';
$result = mysql_query("SELECT * FROM " . $aTables[$i]);
while ($row = mysql_fetch_assoc($result)) {
$query .= '(';
foreach ($aFields[$i][0] as $field) {
$query .= '"{$row[$field]}",';
}
$query .= '),';
}
mysql_query($query);
}
?>
This is based off of this script which may come in handy for reference.
Hopefully that's something to get you started, but I would suggest you look for a non PHP alternative.
This is the way I do it, not original with me:
http://homepage.mac.com/kelleherk/iblog/C711669388/E2080464668/index.html
$servername = "localhost";
$username = "root";
$password = "*******";
$dbname = "dbname";
$sqlfile = "/path/backupsqlfile.sql";
$command='mysql -h' .$servername .' -u' .$username .' -p' .$password .' ' .$dbname .' < ' .$sqlfile;
exec($command);

Categories