We dynamically generate a sql file which contains one big insert query. This file will be run periodically from a PHP app with the following command:
mysql --force -u foo -pbar demo < demo-file.sql
There was no output from the command except when an error happened as the content of the file was only one insert command. Now we decided to change the file that it will contain multiple insert queries instead of one big query. Since then the output of the command is
0
0
0
Our PHP app fails because it assumes now that there is an error happening because the output is not empty. So my question is, can I write an sql file that there is no output generated with multiple queries? I try not to touch the PHP app.
I know there are better designs but the code is historically grown :-)
[UPDATE 1]
Basically the app does
$response = shell_exec('mysql --force -u foo -pbar demo < demo-file.sql');
if (empty($response)) {
echo 'OK';
} else {
echo 'error: '.$response;
}
[UPDATE 2]
The sql file contains something like
insert into;
select sleep(0);
insert into;
select sleep(0);
insert into;
select sleep(0);
Those zeros come from the sleep lines. Get rid of them, you don't need them.
Also, for performance sake, if you are using a schema that supports transactions, you should add:
START TRANSACTION
INSERT...
INSERT...
INSERT...
...
COMMIT
Related
I ran into an odd issue earlier today which had me stumped for a while: I was doing precisely one transaction (confirmed umpteen different ways), and yet I was adding two elements to my SQLite3 database.
Finally twigged as to the possible cause, and tracked it down.
Here's a very minimal testcase which I've written to be as simple and straightforward as possible:
<?php
function dostuff($f) {
#unlink($f);
$db = new SQLite3($f);
$db->exec('CREATE TABLE test (a TEXT)');
$s = $db->prepare('INSERT INTO test (a) VALUES (:a)');
$s->bindValue(':a', 'OHI');
return $s->execute();
}
$f = '__test1__.sqlite3';
dostuff($f);
system('sqlite3 '.$f.' .dump');
print "\n---------------------\n\n";
$f = '__test2__.sqlite3';
$r = dostuff($f);
while (($row = #$r->fetchArray(SQLITE3_ASSOC)) !== false) {
print "LOOP EXECUTED!\n";
}
system('sqlite3 '.$f.' .dump');
?>
Firstly, dostuff creates a new database, which is removed if it exists. Then I create a test table, set up a prepared transaction, run the prepared transaction, and then the function returns the SQLite3Result object from $s->execute().
(I made a function for this code because it gets run twice, to keep the SLOC down.)
Here's what I understand is going on:
I run dostuff() with a first test file.
I run dostuff() with a second test file, but then I grab the SQLite3Result that the function returns, and then iterate over that as though I've just run a SELECT statement and I need to grab the results.
In both cases I dump the resulting database that's been created using system(sqlite3 .dump), as this is a simple, effective way to dump DBs with well-known, predictable results, and I wanted to avoid writing my own potentially buggy code.
This is the output I get on my machine, with PHP 7.0.5 and SQLite3 3.12.0 (PHP module version 0.7):
PRAGMA foreign_keys=OFF;
BEGIN TRANSACTION;
CREATE TABLE test (a TEXT);
INSERT INTO "test" VALUES('OHI');
COMMIT;
---------------------
PRAGMA foreign_keys=OFF;
BEGIN TRANSACTION;
CREATE TABLE test (a TEXT);
INSERT INTO "test" VALUES('OHI');
INSERT INTO "test" VALUES('OHI');
COMMIT;
Note that my code contains a loud "LOOP EXECUTED" line inside the fetchArray() loop, but this isn't showing up. The only output that is being presented is the output of the system() calls and the print statement displaying the dashes.
I crashed into this as I was building a DB helper class that would run statements for me and handle retrieving the results (my queries return very very small result sets so gathering all of them simplifies everything with no cost).
I've now written prepared-statement query() and exec() functions that do and do not try to fetch a result set, respectively - now I just have to remember to use the right ones in the right places!
TL;DR: INSERT + fetchArray() runs the query twice.
My question: Is this a bug? Or is it known behavior in SQLite3?
Im trying to get the following to work within php:
<?php
$qry="CREATE TEMPORARY TABLE IF NOT EXISTS `coursesearch` AS (
SELECT `TREKKER_ID`, `TREKKER_COMPANY`, `TREKKER_FULLDISPLAYNAME`, `TREKKER_POSTCODE`,
`TREKKER_ACTIVE`, `TREKKER_SUBSCRIBOPN`, `tbl_trials`.*,
`EROUTE_RIDERID`, `EROUTE_FKID`, `EROUTE_EXPIRYDATE`
FROM `tbl_trekks`,
`tbl_trials`,
`tbl_scores`
WHERE `TREKKER_POSTCODE`" . $pcode .
" AND `tbl_trials`.`TRIALID` = `tbl_trekks`.`TREKKER_ID`";
if( $checkscores===true )
{
$qry.=" OR (`tbl_scores`.`EROUTE_RIDERID` = `tbl_trekks`.`TREKKER_ID`
AND `tbl_scores`.`EROUTE_FKID` <> 0)";
}
$qry.= "); SELECT * FROM `coursesearch`;";
if( !is_array($arr = $dbOb->DB_runQuery("slt", $qry )) ) { return 0; }
else {
print_r( $arr );
}
?>
Ive tested it directly in the phpMyAdmin console and this works (Obviously without the php specific code and tags) the result set is displayed in the temp table 'coursesearch'. However running the code in php gives this standard error message :
Error: DB Class: 1064: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'SELECT * FROM coursesearch' at line 14.
I understand from researching in google and the mysql site that temporary tables dissappear once the script finishes but surely i'm outputting the resultset before the script finishes. Ive tried using aliases and prefixing all fields with their respective tables names, also tried without backticks in PHP - but the problem persists on the same line,
near 'SELECT * FROM coursesearch' at line 14
Have i misunderstood what temporary tables can achieve here??
Either use PHP multi_query() in mysqli or separate these into two separate queries (run separately)
If that means adding such functionality to your $dbOb-> to run a DB_runMultiQuery (that you have to write) instead of calling $dbOb->DB_runQuery, then that is what you have to do.
PHP Manual page on Multi Query
Executes one or multiple queries which are concatenated by a
semicolon.
A PHP application on the server is saving a certain document with a sequential number into a MySQL database. How to obtaion that sequential number to a command line prompt that initiates the local doocument scanner?
ex:
c:\myscan ask_for_current_seq_nmbr.pdf
myscan is something written in c that takes care of the PC stuff. Only the name of file is unknown.
Some code (from the query PHP file)
$query = "SELECT last_seq FROM seq_table WHERE cat = 1";
$result = mysql_query($query, $link) or die('ERROR: '. mysql_error().'<br />ON LINE: '.__LINE__);
while($row = mysql_fetch_assoc($result)) {
echo $row['last_seq'];
}
!!! NOTE !!!
I am fetching a page from a remote server. ex. www.site.com/query.php?q=SELECT * FROM...
And that selection results in the last used sequential number which I would like to use in my command prompt.
!! UPDATE !!
We HAVE to go through a PHP file on the remote server to avoid having to use Remoote MySQL which has to be enabled on an IP basis.
You can call processes that run on the commandline with various function from PHP from the exec familyDocs.
If you're having problems building the actual command string, you can do with:
$cmd = sprintf('c:\myscan %d.pdf', $sequential_number);
As you write that the script is already writing it into the db with the $sequential_number I assume you have it already.
In case the database generates the number, then probably as the primary key. See mysql_insert_idDocs for obtaining the id.
Okay judging by the backslash and the C:\ I am guess you're using windows.
You are going to have to combine the following:
http://dev.mysql.com/doc/refman/5.5/en/mysql.html
How to store the result of a command expression in a variable using bat scripts?
and then to access the content of the variable you created use the %VARIABLE_NAME% syntax.
You should have flag in your mysql table like is_processed with value = 0 or 1.
When scan starts it runs query:
SELECT * FROM TABLE where is_processed = 0 order by sec_number
After processing you should run query:
UPDATE TABLE set is_processed = 1 where sec_number = 'sec_processed_number';
I'm downloading large sets of data via an XML Query through PHP with the following scenario:
- Query for records 1-1000, download all parts (1000 parts has roughly 4.5 megs of text), then store those in memory while i query the next 1001 - 2000, store in mem (up to potentially 400k)
I'm wondering if it would be better to write these entries to a text field, rather than storing them in memory and once the complete download is done trying to insert them all up into the DB or to try and write them to the DB as they come in.
Any suggestions would be greatly appreciated.
Cheers
You can run a query like this:
INSERT INTO table (id, text)
VALUES (null, 'foo'), (null, 'bar'), ..., (null, 'value no 1000');
Doing this you'll do the thing in one shoot, and the parser will be called once. The best you can do, is running something like this with the MySQL's Benchmark function, running 1000 times a query that inserts 1000 records, or 1000000 of inserts of one record.
(Sorry about the prev. answer, I've misunderstood the question).
I think write them to database as soon as you receive them. This will save memory and u don't have to execute a 400 times slower query at the end. You will need mechanism to deal with any problems that may occur in this process like a disconnection after 399K results.
In my experience it would be better to download everything in a temporary area and then, when you are sure that everything went well, to move the data (or the files) in place.
As you are using a database you may want to dump everything into a table, something like this code:
$error=false;
while ( ($row = getNextRow($db)) && !error ) {
$sql = "insert into temptable(key, value) values ($row[0], $row[1])";
if (mysql_query ($sql) ) {
echo '#';
} else {
$error=true;
}
}
if (!error) {
$sql = "insert into myTable (select * from temptable)";
if (mysql_query($sql) {
echo 'Finished';
} else {
echo 'Error';
}
}
Alternatively, if you know the table well, you can add a "new" flag field for newly inserted lines and update everything when you are finished.
I'm trying to make an "at" job work at a given time, for testing purposes I'm using $time but this will be a datetime that I get from a separate MySQL query. In the command line I go like this:
echo "mysql -e 'UPDATE admin SET row1=row2 WHERE id=1;'" | at now
And I get a "job 36 at 2010-10-28 15:05". in PHP I tried going like this:
exec("\"mysql -e 'UPDATE admin SET row1=row2 WHERE id=1'\" | at $time");
But the query doesn't run. Worse, I have no idea what is happening.
echo exec('whoami');
returns "daemon" though. How can I echo whatever response I'm getting from the exec command? ideally I guess it would say "job 36 at 2010-10-28 15:05" or something similar.
Also, I have a .my.cnf file in my dir that specifies the db, login and password to use, does the daemon need to have one also for these to work?
[from the answers I can tell I wasn't clear about what I am trying to do. I need to
A. Run a mySQL query to get a date/time and an id
B. Schedule an update to take place to rows that match the id at the date/time
I'd already done "A" and was using "1" for the id and "now" for the time while testing. I'll look into PDO.
I'm not that familiar with the at command, but it seems to help run commands at a certain time.
If you're trying to write a scheduled script in PHP there are better ways to do it. Just write a CLI script and use the cron to schedule it. As Svisstack notes, if you're running MySQL queries use an in-built function such as PDO rather than system commands.
If you're just running systems commands, I'm not sure why you're using PHP ;)
Are you perhaps in safe_mode? If yes then your | is getting escaped automatically. Per the manual:
With safe mode enabled, the command
string is escaped with
escapeshellcmd(). Thus, echo y | echo
x becomes echo y \| echo x.
You can get more information by using the other parameters for the exec command. Try running this and see the output.
$output = array();
$return = false;
$last_line = exec("\"mysql -e 'UPDATE admin SET row1=row2 WHERE id=1'\" | at now", $output, $return);
var_dump($last_line);
var_dump($output);
var_dump($return);
Also, it looks like when you ran it at the command line, you echo'ed the MySQL command, but when you put it in the PHP code, it's not doing the echo anymore. I'm not sure if that makes a difference since I'm not to familiar with at, but I thought that I could offer some troubleshooting help.
$last_line = exec("echo \"mysql -e 'UPDATE admin SET row1=row2 WHERE id=1'\" | at now", $output, $return);
A missing echo in your second statement / exec? (and I'd rather use popen / proc_open the at $time, and fwrite the command for at to execute (after which you close the input stream, then the program. Use atq to verify wether it worked, and be aware the current use may be disallowed at jobs (normally found in files like /etc/at.allow or /etc/at.deny)