So I am backing up my DB with the following code:
$rows = $this->_getDb()->fetchAll("SELECT * FROM $table");
foreach ($rows AS $row)
{
foreach ($row AS &$value)
{
if (!is_numeric($value))
{
$value = "'".addcslashes($value, "'\\")."'";
$value = str_replace("\n", '\n', $value);
}
}
if (!$count)
{
$output .= "INSERT INTO `$table` (" .implode(', ', $columns). ") VALUES";
}
$count++;
$output .= "\n(" .implode(', ', $row). "),";
if ($count >= $limit)
{
$count = 0;
$output = preg_replace('#,$#', ";\n\n", $output);
}
}
$output = preg_replace('#,$#', ";\n\n", $output);
This seems to be working great... But I'm comparing my output to what I get from a PHPMyAdmin and I'm noticing some slight differences. With my code, because of how I am using addcslashes, if a string contains a ' it will escape it with \'. However, in the output from PHPMyAdmin, it will instead replace a single ' with two single quotes '' in a row.
Is there really a difference? Please look at the way I am escaping non-numeric fields and tell me if there is a better way.
The method of two single-quotes in a row '' is more standard SQL, so it is more portable.
Even MySQL supports a SQL_MODE called NO_BACKSLASH_ESCAPES that would make your backslash-treated strings invalid if you try to import the backup to a server with that mode enabled.
But I have to add comments to make it clear that your method of backing up your database is full of other problems.
You don't handle NULLs:
if (!is_numeric($value))
This does not handle column names that need to be delimited because they may conflict with SQL reserved words:
$output .= "INSERT INTO `$table` (" .implode(', ', $columns). ") VALUES";
If you have millions of rows in any of your tabes, trying to store the entire content of that table in the form of a series of INSERT statements in a single string will exceed PHP's max memory limit:
$output .= "\n(" .implode(', ', $row). "),";
You only show part of your script, but it may also not properly handle binary strings, character sets, triggers, stored routines, views, table options, etc.
You are really reinventing the wheel.
I've seen past questions where people attempt to do what you're doing, but it's difficult to get right:
PHP Database Dump Script - are there any issues?
Why is my database backup script not working in php?
It will be a much better use of your time to just use shell_exec() from PHP to call mysqldump or mydumper.
Related
I have a table with hundreds of columns. The table structure is out of my control (controlled by a third party). The table also has horrendous field names with spaces, single quotes, etc. and so do the table values. The table is updated once per hour via cron. The cron job truncates and rebuilds the table each time. I also keep an archive table of that table, that I use a REPLACE INTO statement to update or insert as required.
My challenge - I prefer not to have to explicitly define all 350 field names and values, and do so again in my REPLACE INTO statement as this will take a very long time and will require maintenance if the table changes. I would much rather use arrays. Here is what is not working but hopefully gives an idea of the goal (I realize this is deprecated MySQL but it is what it is for a variety of reasons):
$listings = mysql_query("SELECT * FROM current.table");
while ($listing = mysql_fetch_assoc($listings)){
//prepare variables
$fields = array_keys($listing);
$fields = implode('`, `', $fields);
$fields = "`$fields`";
$values = array_values($listing);
$values = implode("`, `", $values);
$values = "`$values`";
mysql_query('REPLACE INTO archive.table ($fields) VALUES ($values)');
}
Posting as a community wiki, no rep should come from this since it did solve the OP's question (as per suggested in comments).
"Aha! Erroneous single quotes on the mysql_query statement was the culprit. I also did mysql_real_escape_string on $values and used single quotes instead of ticks. Worked like a charm. Thank you! Final answer: – Tavish"
Use mysql_error() on the queries. What you posted seems legit code, however values needs to be quoted ' and escaped for possible injection, not ticked.
While using double quotes " for the second query's encapsulation.
mysql_query("REPLACE INTO archive.table ($fields) VALUES ($values)");
As well as other suggestions given.
OP's final code (taken from comments):
while ($listing = mysql_fetch_assoc($listings)){
$fields = array_keys($listing);
$fields = implode(', ', $fields);
$fields = "$fields";
$values = array_values($listing);
$values = implode(", ", $values);
$values = mysql_real_escape_string($values);
$values = str_replace("`","'",$values);
$values = "'$values'";
mysql_query("REPLACE INTO archive.table ($fields) VALUES ($values)");
}
This question already has answers here:
Google-like Search Engine in PHP/mySQL [closed]
(9 answers)
Closed 1 year ago.
I currently have a search option on my PHP+MYSQL website.
The MYSQL query is currently something like "SELECT pageurl WHERE name LIKE '%$query%'.
The reason I posted here is because I noticed that if the name of one of my products is "Blue Bike" and someone looks for "Bike Blue", no results are returned.
I am looking for a solution to this because I know that if I type on google same word, something appears.
I was thinking to create a PHP function to mix up all the words from the query if the query is having 4 or fewer words, generating around 24 queries.
Is there an easier solution to this?
Thanks for your time
As to not let this go without a working answer:
<?php
$search = 'this is my search';
$searchSplit = explode(' ', $search);
$searchQueryItems = array();
foreach ($searchSplit as $searchTerm) {
/*
* NOTE: Check out the DB connections escaping part
* below for the one you should use.
*/
$searchQueryItems[] = "name LIKE '%" . mysqli_real_escape_string($searchTerm) . "%'";
}
$query = 'SELECT pageurl FROM names' . (!empty($searchQueryItems) ? ' WHERE ' . implode(' AND ', $searchQueryItems) : '');
?>
DB connections escaping
mysqli_:
Keep using mysqli_real_escape_string or use $mysqli->real_escape_string($searchTerm).
mysql_:
if you use mysql_ you should use mysql_real_escape_string($searchTerm) (and think about changing as it's deprecated).
PDO:
If you use PDO, you should use trim($pdo->quote($searchTerm), "'").
use full text search instead of like
full text search based on indexed text and is very faster and beter than using like.
see this article for more information about full text search
What you are looking for is fulltext search.
Try Sphinx, it is very fast and integrates well with MySQL.
Sphinx website
I wrote a function that approaches Google's operation taking into account the double quotes for the elements to search as a whole block. It does NOT take into account the - or * instructions.
table: MySQL table to consider
cols: array of column to parse
searchParams: search to process. For example: red mustang "Florida 90210"
function naturalQueryConstructor($table, $cols, $searchParams) {
// Basic processing and controls
$searchParams = strip_tags($searchParams);
if( (!$table) or (!is_array($cols)) or (!$searchParams) ) {
return NULL;
}
// Start query
$query = "SELECT * FROM $table WHERE ";
// Explode search criteria taking into account the double quotes
$searchParams = str_getcsv($searchParams, ' ');
// Query writing
foreach($searchParams as $param) {
if(strpos($param, ' ') or (strlen($param)<4)) {
// Elements with space were between double quotes and must be processed with LIKE.
// Also for the elements with less than 4 characters. (red and "Florida 90210")
$query .= "(";
// Add each column
foreach($cols as $col) {
if($col) {
$query .= $col." LIKE '%".$param."%' OR ";
}
}
// Remove last ' OR ' sequence
$query = substr($query, 0, strlen($query)-4);
// Following criteria will added with an AND
$query .= ") AND ";
} else {
// Other criteria processed with MATCH AGAINST (mustang)
$query .= "(MATCH (";
foreach($cols as $col) {
if($col) {
$query .= $col.",";
}
}
// Remove the last ,
$query = substr($query, 0, strlen($query)-1);
// Following criteria will added with an AND
$query .= ") AGAINST ('".$param."' IN NATURAL LANGUAGE MODE)) AND ";
}
}
// Remove last ' AND ' sequence
$query = substr($query, 0, strlen($query)-5);
return $query;
}
Thanks to the stackoverflow community where I found parts of this function!
To have a google like search you'd need many database and index nodes, crazy algorithms.. now you come up with a SELECT LIKE ... lol :D
MySQL is slow in searching, you'd need fulltext and index set properly (MyISAM or Aria Engine). Combinations or different entities to search for are almost not implementable properly AND fast.
I'd suggest to setup an Elasticsearch server which is based on Apache's Lucene.
This searchs very fast and is easy to maintain. And you would not have to care about SQL injection and can still use the mysql server fast.
Elasticsearch (or other Lucene based search engines like SolR) can easily be installed on any server because they are written in Java.
Good documentation:
http://www.elasticsearch.org/guide/en/elasticsearch/client/php-api/current/
I would do an explode first:
$queryArray = explode(" ", $query);
and then generate the SQL query something like:
for ($i=0; $i< count($queryArray); $i++) {
$filter += " LIKE '%" + $queryArray[$i] + "%' AND" ;
}
$filter = rtrim ($filter, " AND");
$sql = "SELECT pageurl FROM ... WHERE name " + $filter
(note: haven't tested/run this code)
I ran into the following question while writing a PHP script. I need to store the first two integers from an array of variable lenght into a database table, remove them and repeat this until the array is empty. I could do it with a while loop, but I read that you should avoid writing SQL statements inside a loop because of the performance hit.
A simpliefied example:
while(count($array) > 0){
if ($sql = $db_connect->prepare("INSERT INTO table (number1, number2) VALUES (?,?)")){
$sql->bind_param('ii',$array[0],$array[1]);
$sql->execute();
$sql->close();
}
array_shift($array);
array_shift($array);
}
Is this the best way, and if not, what's a better approach?
You can do something like this, which is way faster aswell:
Psuedo code:
$stack = array();
while(count($array) > 0){
array_push($stack, "(" . $array[0] . ", " . $array[1] . ")");
array_shift($array);
array_shift($array);
}
if ($sql = $db_connect->prepare("INSERT INTO table (number1, number2)
VALUES " . implode(',', $stack))){
$sql->execute();
$sql->close();
}
The only issue here is that it's not a "MySQL Safe" insert, you will need to fix that!
This will generate and Array that holds the values. Within 1 query it will insert all values at once, where you need less MySQL time.
Whether you run them one by one or in an array, an INSERT statement is not going to make a noticeable performance hit, from my experience.
The database connection is only opened once, so it is not a huge issue. I guess if you are doing some insane amount of queries, it could be.
I think as long as your loop condition is safe ( will break in time ) and you got something from it .. it's ok
You would be better off writing a bulk insert statement, less hits on mysql
$sql = "INSERT INTO table(number1, number2) VALUES";
$params = array();
foreach( $array as $item ) {
$sql .= "(?,?),\n";
$params[] = $item;
}
$sql = rtrim( $sql, ",\n" ) . ';';
$sql = $db_connect->prepare( $sql );
foreach( $params as $param ) {
$sql->bind_param( 'ii', $param[ 0 ], $param[ 1 ] );
}
$sql->execute();
$sql->close();
In ColdFusion you can put your loop inside the query instead of the other way around. I'm not a php programmer but my general belief is that most things that can be done in language a can also be done in language b. This code shows the concept. You should be able to figure out a php version.
<cfquery>
insert into mytable
(field1, field2)
select null, null
from SomeSmallTable
where 1=2
<cfloop from="1' to="#arrayLen(myArray)#" index="i">
select <cfqueryparam value="myArray[i][1]
, <cfqueryparam value="myArray[i][]
from SomeSmallTable
</cfloop>
</cfquery>
When I've looked at this approach myself, I've found it to be faster than query inside loop with oracle and sql server. I found it to be slower with redbrick.
There is a limitation with this approach. Sql server has a maximum number of parameters it will accept and a maximum query length. Other db engines might as well, I've just not discovered them yet.
In PHP, I want to insert into a database using data contained in a associative array of field/value pairs.
Example:
$_fields = array('field1'=>'value1','field2'=>'value2','field3'=>'value3');
The resulting SQL insert should look as follows:
INSERT INTO table (field1,field2,field3) VALUES ('value1','value2','value3');
I have come up with the following PHP one-liner:
mysql_query("INSERT INTO table (".implode(',',array_keys($_fields)).") VALUES (".implode(',',array_values($_fields)).")");
It separates the keys and values of the the associative array and implodes to generate a comma-separated string . The problem is that it does not escape or quote the values that were inserted into the database. To illustrate the danger, Imagine if $_fields contained the following:
$_fields = array('field1'=>"naustyvalue); drop table members; --");
The following SQL would be generated:
INSERT INTO table (field1) VALUES (naustyvalue); drop table members; --;
Luckily, multiple queries are not supported, nevertheless quoting and escaping are essential to prevent SQL injection vulnerabilities.
How do you write your PHP Mysql Inserts?
Note: PDO or mysqli prepared queries aren't currently an option for me because the codebase already uses mysql extensively - a change is planned but it'd take alot of resources to convert?
The only thing i would change would be to use sprintf for readability purposes
$sql = sprintf(
'INSERT INTO table (%s) VALUES ("%s")',
implode(',',array_keys($_fields)),
implode('","',array_values($_fields))
);
mysql_query($sql);
and make sure the values are escaped.
Nothing wrong with that. I do the same.
But make sure you mysql_escape() and quote the values you stick in the query, otherwise you're looking at SQL injection vulnerability.
Alternately, you could use parametrized queries, in which case you can practically pass the array in itself, instead of building a query string.
The best practice is either to use an ORM (Doctrine 2.0), an ActiveRecord implementation (Doctrine 1.0, RedBean), or a TableGateway pattern implementation (Zend_Db_Table, Propel). These tools will make your life a lot easier, and handle a lot of the heavy lifting for you, and can help protect you from SQL injections.
Other than that, there's nothing inherently wrong with what you're doing, you just might want to abstract it away into a class or a function, so that you can repeat the functionality in different places.
Using the sprintf trick mentioned by Galen in a previous answer, I have come up with the following code:
$escapedfieldValues = array_map(create_function('$e', 'return mysql_real_escape_string(((get_magic_quotes_gpc()) ? stripslashes($e) : $e));'), array_values($_fields));
$sql = sprintf('INSERT INTO table (%s) VALUES ("%s")', implode(',',array_keys($_fields)), implode('"," ',$escapedfieldValues));
mysql_query($sql);
It generates a escaped and quoted insert. It also copes independent of whether magic_quotes_gpc is on or off. The code could be nicer if I used new PHP v5.3.0 anonymous functions but I need it to run on older PHP installations.
This code is a bit longer that the original (and slower) but it is more secure.
I use this to retrieve the VALUES part of the INSERT.
But it might be an absurd way to do things. Comments/suggestions are welcome.
function arrayToSqlValues($array)
{
$sql = "";
foreach($array as $val)
{
//adding value
if($val === NULL)
$sql .= "NULL";
else
/*
useless piece of code see comments
if($val === FALSE)
$sql .= "FALSE";
else
*/
$sql .= "'" . addslashes($val) . "'";
$sql .= ", ";
};
return "VALUES(" . rtrim($sql, " ,") . ")";
}
There is a problem with NULL (in the accepted answer) values being converted to empty string "". So this is fix, NULL becomes NULL without quotes:
function implode_sql_values($vals)
{
$s = '';
foreach ($vals as $v)
$s .= ','.(($v===NULL)?'NULL':'"'.mysql_real_escape_string($v).'"');
return substr($s, 1);
}
Usage:
implode_sql_values(array_values( array('id'=>1, 'nick'=>'bla', 'fbid'=>NULL) ));
// =='"1","bla",NULL'
If you want to enhance your approach and add the possibility for input validation and sanitation, you might want to do this:
function insertarray($table, $arr){
foreach($arr as $k => $v){
$col[] = sanitize($k);
$val[] = "'".sanitize($v)."'";
}
query('INSERT INTO '.sanitize($table).' ('.implode(', ', $col).') VALUES ('.implode(', ', $val).')' );
}
I need to do a dump of a table on a remote server, but I can't access the server directly. The only access I have is through PHP scripts.
Is there some way in which MySQL will return an
INSERT INTO `table_name` (`field1`, `field2`) VALUES ('a', 'b'), ('c', 'd')
statement, like what mysqldump will return?
I don't have access to phpMyAdmin, and I preferably don't want to use exec, system or passthru.
See this question for another export method
1) can you run mysqldump from exec or passthru
2) take a look at this: http://www.php-mysql-tutorial.com/perform-mysql-backup-php.php
If you can use php-scripts on the server i would recommend phpmyadmin. Then you can do this from the web-interface.
You should check out PHPMyAdmin, it is a php based MySQL administration tool. It supports backups and recovery for the database as well as a 'GUI' to the database server. It works very well.
I'm pretty sure phpMyAdmin will do this for you.
This
select 'insert into table table_name (field1, field2) values'
|| table_name.field1 || ', ' || table_field2 || ');'
from table_name
should get you started. Replace || with the string concatenation operator for your db flavour. If field1 or field2 are strings you will have to come up with some trick for quoting/escaping.
Here is one approach generating a lot of separate query statements. You can also use implode to more efficiently combine the strings, but this is easier to read for starters and derived from this you can come up with a million other approaches.
$results = mysql_query("SELECT * FROM `table_name`");
while($row = mysql_fetch_assoc($results)) {
$query = "INSERT INTO `table_name` ";
$fields = '(';
$values = '(';
foreach($row as $field=>$value) {
$fields .= "'".$field."',";
$values .= "'".mysql_escape_string($value)."',";
}
//drop the last comma off
$fields = substr($fields,0,-1);
$values = substr($values,0,-1);
$query .= $fields . " VALUES " . $values;
//your final result
echo $query;
}
See if that gets you started