I am having a problem with getting an sql query to interpolate as I would want, and would be grateful for some help please.
Within the manual page for pg_query_params,there is a code example for pg_query() passing a variable using curly braces. This appeared to be exactly what I need for my task. So, my code is as follows:
$fh = fopen('/home/www/KPI-Summary.sql',"r")
or die("Problem opening SQL file.\n");
$dbh = pg_connect("$connect")
or die('Could not connect: ' . pg_last_error());
$j = 0;
while (($line = fgets($fh)) !== false) {
$tmp[$j] = array(); // Initialise temporary storage.
$result = pg_query($dbh, $line); // process the line read.
if (!$result) { echo "Error: query did not execute"; }
...
while ($row = pg_fetch_row($result)) { // Read sql result.
$tmp[$j][2][] = $row;
}
$j++;
}
fclose($fh);
The sql file contains several queries, one per line, like this:
SELECT count(*) from table WHERE value=0 AND msgid='{$arg[1]}';
However, currently, my variable is not being replaced by the contents -- and therefore although the query runs OK, it is returning zero rows. What do I need to do in order to get the expected result? (Note: each sql line varies, and the query parameters are not constant -- hence using variable(s) within the sql file.)
OK. I have a solution (although it might not be the correct approach).
This works -- but it needs polish I think. Suggestions regarding a better regexp would be very much appreciated.
$bar = 'VALUE-A'; // Can we replace simple variable names?
$arg[1] = 'VALUE-B'; // What about an array, such as $arg[1]?
function interpolate($matches){
global $bar;
global $arg;
if ($matches[2]) {
$i = isset(${$matches[1]}[$matches[2]]) ? ${$matches[1]}[$matches[2]] : 'UNDEF';
} else {
$i = isset(${$matches[1]}) ? ${$matches[1]} : 'UNDEF';
}
return $i;
}
$fh = fopen('/home/www/file.sql',"r") or die("Failed.\n");
while (($line = fgets($fh)) !== false) {
...
$line = preg_replace_callback('|\{\$([a-z]+)\[*(\d*)\]*}|i', "interpolate", $line);
echo $line; // and continue with rest of code as above.
}
fclose($fh);
(Of course, the solution suggests that the question title is completely wrong. Is there any way to edit this?)
did you use pg_escape_string?
$arg[1] = pg_escape_string($arg[1]);
$line="SELECT count(*) from table WHERE value=0 AND msgid='{$arg[1]}';";
Related
I'm tryin to insert datas (160,000+ rows) using INSERT INTO and PHP PDO but i have a bug.
When I launch the PHP script, i see more than the exact number of lines in my CSV inserted in my database.
Can someone say me if my loop is not correct or something ?
Here the code I have :
$bdd = new PDO('mysql:host=<myhost>;dbname=<mydb>', '<user>', '<pswd>');
// I clean the table
$req = $bdd->prepare("TRUNCATE TABLE lbppan_ticket_reglements;");
$req->execute();
// I read and import line by line the CSV file
$handle = fopen('<pathToMyCsvFile>', "r");
while (($data = fgetcsv($handle, 0, ',')) !== FALSE) {
$reqImport =
"INSERT INTO lbppan_ticket_reglements
(<my31Columns>)
VALUES
('$data[0]','$data[1]','$data[2]','$data[3]','$data[4]','$data[5]','$data[6]','$data[7]','$data[8]',
'$data[9]','$data[10]','$data[11]','$data[12]','$data[13]','$data[14]','$data[15]','$data[16]',
'$data[17]','$data[18]','$data[19]','$data[20]','$data[21]','$data[22]','$data[23]','$data[24]',
'$data[25]','$data[26]','$data[27]','$data[28]','$data[29]','$data[30]')";
$req = $bdd->prepare($reqImport);
$req->execute();
}
fclose($handle);
The script works a little because datas are in the table but i dunno why it bugs and inserts more datas. I think maybe, due to the file size (18 Mo) maybe the script crash and attempts to relaunch inserting same rows again.
I can't use LOAD DATA on the server I'm using.
Thanks for your help.
This is not an answer but adding this much into comments is quite tricky.
Start by upping the maximum execution time
If that does not solve your issue, start working your way through the code line by line and handle every exception you can think of. For example, you are truncating the table BUT you say you have loads more data after execution, could the truncate be failing?
try {
$req = $bdd->prepare("TRUNCATE TABLE lbppan_ticket_reglements;");
$req->execute();
} catch (\Exception $e) {
exit($e->getMessage()); // Die immediately for ease of reading
}
Not the most graceful of try/catches but it will allow you to easily spot a problem. You can also apply this to the proceeding query...
try {
$req = $bdd->prepare($reqImport);
$req->execute();
} catch (\Exception $e) {
exit($e->getMessage());
}
and also stick in some diagnostics, are you inserting 160k rows? You could optionally echo out $i on each loop and see if you can spot any breaks or abnormalities.
$i = 0;
while (($data = fgetcsv($handle, 0, ',')) !== FALSE) {
// ... your stuff
$i++;
}
echo "Rows inserted " . $i . "\n\n";
Going beyond that you can the loop print out the SQL content for you to look at manually, perhaps its doing something weird and fruity.
Hope that helps.
Assuming $data[0] is the unique identifier then you can try this to spot the offending row(s):
$i = 0;
while (($data = fgetcsv($handle, 0, ',')) !== FALSE) {
echo 'Row #'.++$i.' - '.$data[0];
}
Since you are not using prepared statements, it is very possible that one of the $data array items are causing a double-insert or some other unknown issue.
i got a function in PHP to read table from ODBC (to IBM AS400) and write it to a text file on daily basis. it works fine until it reach more than 1GB++. Then it just stop to some rows and didn't write completely.
function write_data_to_txt($table_new, $query)
{
global $path_data;
global $odbc_db, $date2;
if(!($odbc_rs = odbc_exec($odbc_db,$query))) die("Error executing query $query");
$num_cols = odbc_num_fields($odbc_rs);
$path_folder = $path_data.$table_new."/";
if (!file_exists($path_folder)) mkdir ($path_folder,0777);
$filename1 = $path_folder. $table_new. "_" . $date2 . ".txt";
$comma = "|";
$newline = chr(13).chr(10);
$handle = fopen($filename1, "w+");
if (is_writable($filename1)) {
$ctr=0;
while(odbc_fetch_row($odbc_rs))
{
//function for writing all field
// for($i=1; $i<=$num_cols; $i++)
// {
// $data = odbc_result($odbc_rs, $i);
// if (!fwrite($handle, $data) || !fwrite($handle, $comma)) {
// print "Cannot write to file ($filename1)";
// exit;
// }
//}
//end of function writing all field
$data = odbc_result($odbc_rs, 1);
fwrite($handle,$ctr.$comma.$data.$newline);
$ctr++;
}
echo "Write Success. Row = $ctr <br><br>";
}
else
{
echo "Write Failed<br><br>";
}
fclose($handle);
}
no errors, just success message but it should be 3,690,498 rows (and still increase) but i just got roughly 3,670,009 rows
My query is ordinary select like :
select field1 , field2, field3 , field4, fieldetc from table1
What i try and what i assume :
I think it was fwrite limitation so i try not to write all field (just write $ctr and 1st record) but it still stuck in same row.. so i assume its not about fwrite exceed limit..
I try to reduce field i select and it can works completely!! so i assume it have some limitation on odbc.
I try to use same odbc datasource with SQL Server and try to select all field and it give me complete rows. So i assume its not odbc limitation.
Even i try on 64 bits machine but it even worse, it just return roughly 3,145,812 rows.. So i assume it's not about 32/64 bit infrastructure.
I try to increase memory_limit in php ini to 1024mb but it didnt work also..
Is there anyone know if i need to set something in my PHP to odbc connection??
I am really trying to wrap my head around this and failing miserably. What I want to do it build a MySQL query based on the URL parameters passed by the URL. I am trying to create a re usable dynamic script that can do what it needs to do based on the URL parameter.
This is what I have come up with, and it appears that it does what it is supposed to do (no errors or anything) but nothing actually gets inserted in the database. I know somewhere I have made a dumb mistake (or thought something out wrong) so hopefully one of you guys can point me in the right direction.
Thanks!
//List all possible variables you can expect the script to receive.
$expectedVars = array('name', 'email', 'score', 'age', 'date');
// This is used for the second part of the query (WHERE, VALUES, ETC)
$fields = array('uName','uEmail','uScore','uAge','uDate');
// Make sure some fields are actually populated....
foreach ($expectedVars as $Var)
{
if (!empty($_GET[$Var]))
{
$fields[] = sprintf("'%s' = '%s'", $Var, mysql_real_escape_string($_GET[$Var]));
}
}
if (count($fields) > 0)
{
// Construct the WHERE Clause
$whereClause = "VALUES " . implode(",",$fields);
//Create the SQL query itself
$sql = ("INSERT INTO $mysql_table ($fields) . $whereClause ");
echo "1"; //It worked
mysql_close($con);
}
else
{
// Return 0 if query failed.
echo "0";
}
?>
You missed mysql_query($sql):
if(!mysql_query($sql)){
//die(mysql_error());
}
Please consider to use PDO or My SQLi using parametrize query because mysl_* function depreciated.
Your SQL is all wrong. You're using the field = value syntax for an INSERT, then you're concatenating an array as if it were a string ($fields), and you're missing a couple of parentheses around the values.
a couple of things: i've found for php <-> mysql its important to see what's going into mysql and experiement directly with those queries in phpmyadmin when i get stuck.
1 - in my code I output mysql_error() when the query fails or when a debug flag is set. this usually explains the sql issue in a way that can point me to a misspelled field name etc...
2 - this way i can feed that mysql query directly into phpmyadmin and tweak it until it gives me the results i want. (while i'm there i can also use explain to see if i need to optimize the table)
specifics in your code. unlike C languages sprintf is implied. here's how i'd write your code:
// List all possible variables you can expect the script to receive.
$expectedvars = array('name', 'email', 'score', 'age', 'date');
// This is used for the second part of the query (WHERE, VALUES, ETC)
// $fields = array('uName','uEmail','uScore','uAge','uDate');
$fields = array();
// Set only the variables that were populated ...
foreach ($expectedvars as $var) {
if (!empty($_GET[$var])) {
$name = "u" + ucwords($var); // convert var into mysql field names
$fields[] = "{$name} = " . mysql_real_escape_string($_GET[$var]);
}
}
// only set those fields which are passed in, let the rest use the mysql default
if (count($fields) > 0) {
// Create the SQL query itself
$sql = "INSERT INTO {$mysql_table} SET " . implode("," , $fields);
$ret = mysql_query($sql);
if (!$ret) {
var_dump('query_failed: ', $sql, $ret);
echo "0"; // Query failed
} else {
echo "1"; // It worked
}
} else {
// Return 0 if nothing to do
echo "0";
}
mysql_close($con);
I am passing a string to this file -txtname- (string separated by spaces) and saparate each word and then pass it to the function subtoken() that should fetch the corresponding words from the database, having two attributes-rootwords and example,but subtoken() function executes only once and exits.
$counter=0;
$string = $_REQUEST['txtname'];
$token = strtok($string, ' ');
while ($token != false)
{echo $counter;
subtoken($token,$counter);
$counter++;
$token = strtok(' ');
}
function subtoken($fname,$counter)
{
$row ="";
$result="";
$result = mysql_query('SELECT * FROM hindi WHERE rootwords LIKE \'%:'.$fname.':%\'' );
while($row = mysql_fetch_array($result))
{
$temp=$row['rootwords'];
$token2 = strtok($temp, ':');
echo $token2 ;
while($token2!=false)
{
echo $token2."<br/>" ;
$token2=strtok(':');
}
}
}
mysql_close($con);
The double usage of strtok will prevent the "main" loop to properly process all tokens from the original $string. You simply can't have more than one "open" strtok use at the same time.
Original suspect was your query, that it just doesn't select anything. Try printing the SQL statement, then executing that statement directly (e.g. via phpmyadmin)
// insert this line:
echo(
'SELECT * FROM hindi '.
'WHERE rootwords LIKE \'%:'.$fname.':%\'');
// just before the actual query execution
$result = mysql_query(
'SELECT * FROM hindi '.
'WHERE rootwords LIKE \'%:'.$fname.':%\'' );
From my experience, echo'ing as much data as possible early on while debugging is one of the best tools to easily spot errors.
See also: http://nl2.php.net/mysql_fetch_array
mysql_fetch_array needs a second parameter to work correctly.
Replace mysql_fetch_array with mysql_fetch_row. Or mysql_fetch_assoc if you want to reference to the columns by name.
May be you have some error/warning? Is error_reporting set to E_ALL and display_errors to "On"?
It seems that strtok() has only one pointer. So, when you ended with it in function, you need to reinitialise $token = strtok($string, ' ');? dut I am not sure. I think it would be better if you used explode() function.
there have been hundreds if not thousands of posts concerning the use of PHP's eval(); to run code from a database. Through all my searching I have not found an answer to my question (explained shortly).
Firstly I'll introduce you to my application.
I have three records of valid code stored in a database:
eg:
['code1']
$num1 = 1;
$num2 = 3;
$num3 = $num1+$num2; //4
['code2']
$num4 = $num3; //4
$num5 = 5;
$num6 = $num4+$num5; //9
['code3']
$num7 = $num4; //4
$num8 = $num6; //9
$num9 = $num7+$num8; //13
echo $num9; //13
Next I have a function to call and run a record:
eg:
function runCode($codeName) {
// assume db connection is established
$result = mysql_query("SELECT `code` FROM CodeStore WHERE `name` = '".$codeName."'");
if ($result) {
// Fetch one row
$row = mysql_fetch_assoc($result);
if (!$row) {
die('No rows returned.');
} else {
return eval($row['code']);
}
} else {
die('Invalid query: '.mysql_error());
}
}
Now, what needs to happen is to call the three above snippets, one after each other, and have the variables inside ($numX) available for use between each other.
eg:
runCode('code1');
runCode('code2');
runCode('code3');
The above call of the three snippets from the db should echo '13', it does not. And there is my question:
How can I make these variables available outside the eval'd code?
You're not fetching the results from the db. mysql_query() does not return all the SELECTed rows.
function runCode($codeName) {
// assume db connection is established
$result = mysql_query("SELECT `code` FROM CodeStore WHERE `name` = '".$codeName."'");
if ($result) {
// Fetch one row
$row = mysql_fetch_assoc($result);
if (!$row)
die('No rows returned.');
$valid = eval($row['code']);
if($valid) {
return $valid;
} else {
die('Error executing: '.$codeName);
}
} else {
die('Invalid query: '.mysql_error());
}
}
See the manual for mysql_fetch_assoc.
How can I make these variables
available outside the eval'd code?
That's a bit difficult, there's no simple way to do this.
The eval'd pieces of code don't run in the same scope, so you'll have to somehow save the current symbol table and restore it afterwards.
If keeping the variables is enough for you, you might try get_defined_vars (and probably a combo of compact and extract or a roll your own replacement for those functions). However this functions returns the name of all defined vars (include superglobals like $_GET etc.).
If you build a clever diff algorithm, you could probably compare the variables which were defined before and after the eval'd code and find out which variables are new. Good luck with that :).
In addition to svens's answer, I think you're not checking the return value from eval() correctly. From the PHP manual:
eval() returns NULL unless return is called in the evaluated code, in which case the value passed to return is returned. If there is a parse error in the evaluated code, eval() returns FALSE and execution of the following code continues normally.
You should only treat it as a failure if FALSE === eval($code). Even then, you could get in trouble if your code returns FALSE.
Variables from DB are local in function. Make them global.
So you may use global keyword to change variables scope:
<?php
error_reporting (E_ALL);
$txt = array(
'$num1 = 1;
$num2 = 3;
global $num3; // make it global, will be used later
$num3 = $num1+$num2;
echo "num3=" . $num3 . "<br>"; //4',
'global $num3; // need to use value from this var
global $num4; // make it global, will be used later
$num4 = $num3; //4
$num5 = 5;
global $num6; // make it global, will be used later
$num6 = $num3+$num5;
echo "num6=" . $num6 . "<br>"; //9',
'global $num4; // need to use value from this var
global $num6; // need to use value from this var
$num7 = $num4; //4
$num8 = $num6; //9
global $num9; // make it global, will be used later (maybe)
$num9 = $num7+$num8; //13
echo "num9=" . $num9 . "<br>"; //13'
);
function runCode($codeName) { // just for example
eval($codeName);
}
runCode($txt[0]);
runCode($txt[1]);
runCode($txt[2]);
?>
This solution demands changing existing code in a DB. It may be difficult.
So, here is a different algorithm. At first, join all code chunks together. Then run eval function and pass joined code to it. For example:
function composeCode($codeName) {
// assume db connection is established
$result = mysql_query("SELECT `code` FROM CodeStore WHERE `name` = '".$codeName."'");
if ($result) {
// Fetch one row
$row = mysql_fetch_assoc($result);
if (!$row) {
die('No rows returned.');
} else {
return $row['code']; // CHANGED ORIGINAL CODE
}
} else {
die('Invalid query: '.mysql_error());
}
}
$code = '';
$code .= composeCode('code1');
$code .= composeCode('code2');
$code .= composeCode('code3');
eval($code);
I would definitely go with the "pass an array of code names to runCode()" way.
The benefits you get:
You can get all the snippets with a single SELECT ... WHERE name IN ('code1', 'code2', ...) query
You share the variables between the snippets while keeping them in the scope of the runCode() function call (so that they get destroyed when the function execution terminates)
Then, you can operate in two ways:
eval() them one-by-one while fetching rows from the database result
implode() them with newline character to get a single code snippet to eval() at once.