SQL full text search with PHP and PDO - php

I'm trying to write a simple, full text search with PHP and PDO. I'm not quite sure what the best method is to search a DB via SQL and PDO. I found this this script, but it's old MySQL extension. I wrote this function witch should count the search matches, but the SQL is not working. The incoming search string look like this: 23+more+people
function checkSearchResult ($searchterm) {
//globals
global $lang; global $dbh_pdo; global $db_prefix;
$searchterm = trim($searchterm);
$searchterm = explode('+', $searchterm);
foreach ($searchterm as $value) {
$sql = "SELECT COUNT(*), MATCH (article_title_".$lang.", article_text_".$lang.") AGINST (':queryString') AS score FROM ".$db_prefix."_base WHERE MATCH (article_title_".$lang.", article_text_".$lang.") AGAINST ('+:queryString')";
$sth = $dbh_pdo->prepare($sql);
$sql_data = array('queryString' => $value);
$sth->execute($sql_data);
echo $sth->queryString;
$row = $sth->fetchColumn();
if ($row < 1) {
$sql = "SELECT * FROM article_title_".$lang." LIKE :queryString OR aricle_text_".$lang." LIKE :queryString";
$sth = $dbh_pdo->prepare($sql);
$sql_data = array('queryString' => $value);
$sth->execute($sql_data);
$row = $sth->fetchColumn();
}
}
//$row stays empty - no idea what is wrong
if ($row > 1) {
return true;
}
else {
return false;
}
}

When you prepare the $sql_data array, you need to prefix the parameter name with a colon:
array('queryString' => $value);
should be:
array(':queryString' => $value);
In your first SELECT, you have AGINST instead of AGAINST.
Your second SELECT appears to be missing a table name after FROM, and a WHERE clause. The LIKE parameters are also not correctly formatted. It should be something like:
sql = "SELECT * FROM ".$db_prefix."_base WHERE article_title_".$lang." LIKE '%:queryString%' OR aricle_text_".$lang." LIKE '%:queryString%'";
Update 1 >>
For both SELECT statements, you need unique identifiers for each parameter, and the LIKE wildcards should be placed in the value, not the statement. So your second statement should look like this:
sql = "SELECT * FROM ".$db_prefix."_base WHERE article_title_".$lang." LIKE :queryString OR aricle_text_".$lang." LIKE :queryString2";
Note queryString1 and queryString2, without quotes or % wildcards. You then need to update your array too:
$sql_data = array(':queryString1' => "%$value%", ':queryString2' => "%$value%");
See the Parameters section of PDOStatement->execute for details on using multiple parameters with the same value. Because of this, I tend to use question marks as placeholders, instead of named parameters. I find it simpler and neater, but it's a matter of choice. For example:
sql = "SELECT * FROM ".$db_prefix."_base WHERE article_title_".$lang." LIKE ? OR aricle_text_".$lang." LIKE ?";
$sql_data = array("%$value%", "%$value%");
<< End of Update 1
I'm not sure what the second SELECT is for, as I would have thought that if the first SELECT didn't find the query value, the second wouldn't find it either. But I've not done much with MySQL full text searches, so I might be missing something.
Anyway, you really need to check the SQL, and any errors, carefully. You can get error information by printing the results of PDOStatement->errorCode:
$sth->execute($sql_data);
$arr = $sth->errorInfo();
print_r($arr);
Update 2 >>
Another point worth mentioning: make sure that when you interpolate variables into your SQL statement, that you only use trusted data. That is, don't allow user supplied data to be used for table or column names. It's great that you are using prepared statements, but these only protect parameters, not SQL keywords, table names and column names. So:
"SELECT * FROM ".$db_prefix."_base"
...is using a variable as part of the table name. Make very sure that this variable contains trusted data. If it comes from user input, check it against a whitelist first.
<< End of Update 1
You should read the MySQL Full-Text Search Functions, and the String Comparison Functions. You need to learn how to construct basic SQL statements, or else writing even a simple search engine will prove extremely difficult.
There are plenty of PDO examples on the PHP site too. You could start with the documentation for PDOStatement->execute, which contains some examples of how to use the function.
If you have access to the MySQL CLI, or even PHPMyAdmin, you can try out your SQL without all the PHP confusing things. If you are going to be doing any database development work as part of your PHP application, you will find being able to test SQL independently of PHP a great help.

Related

Php search Splitting criteria type

I have a php search form with two fields. One for $code another for '$name'.The user uses one or the other, not both.
The submit sends via $_POST.
In the receiving php file I have:
SELECT * FROM list WHERE code = '$code' OR name = '$name' ORDER BY code"
Everything works fine, however I would like that $code is an exact search while $name is wild.
When I try:
SELECT * FROM list WHERE code = '$code' OR name = '%$name%' ORDER BY code
Only $code works while $name gives nothing. I have tried multiple ways. Changing = to LIKE, putting in parentheses etc. But only one way or the other works.
Is there a way I can do this? Or do I have to take another approach?
Thanks
If you only want to accept one or the other, then only add the one you want to test.
Also, when making wild card searches in MySQL, you use LIKE instead of =. We also don't want to add that condition if the value is empty since it would become LIKE '%%', which would match everything.
You should also use parameterized prepared statements instead of injection data directly into your queries.
I've used PDO in my example since it's the easiest database API to use and you didn't mention which you're using. The same can be done with mysqli with some tweaks.
I'm using $pdo as if it contains the PDO instance (database connection) in the below code:
// This will contain the where condition to use
$condition = '';
// This is where we add the values we're matching against
// (this is specifically so we can use prepared statements)
$params = [];
if (!empty($_POST['code'])) {
// We have a value, let's match with code then
$condition = "code = ?";
$params[] = $_POST['code'];
} else if (!empty($_POST['name'])){
// We have a value, let's match with name then
$condition = "name LIKE ?";
// We need to add the wild cards to the value
$params[] = '%' . $_POST['name'] . '%';
}
// Variable to store the results in, if we get any
$results = [];
if ($condition != '') {
// We have a condition, let's prepare the query
$stmt = $pdo->prepare("SELECT * FROM list WHERE " . $condition);
// Let's execute the prepared statement and send in the value:
$stmt->execute($params);
// Get the results as associative arrays
$results = $stmt->fetchAll(PDO::FETCH_ASSOC);
}
The variable $results will now contain the values based on the conditions, or an empty array if no values were passed.
Notice
I haven't tested this exact code IRL, but the logic should be sound.

PHP PDO spaces in the prepare placeholder [duplicate]

I'm trying to implement a pretty basic search engine for my database where the user may include different kinds of information. The search itself consists of a couple of a union selects where the results are always merged into 3 columns.
The returning data however is being fetched from different tables.
Each query uses $term for matchmaking, and I've bound it to ":term" as a prepared parameter.
Now, the manual says:
You must include a unique parameter marker for each value you wish to pass in to the statement when you call PDOStatement::execute(). You cannot use a named parameter marker of the same name twice in a prepared statement.
I figured that instead of replacing each :term parameter with :termX (x for term = n++) there must be a be a better solution?
Or do I just have to bind X number of :termX?
Edit Posting my solution to this:
$query = "SELECT ... FROM table WHERE name LIKE :term OR number LIKE :term";
$term = "hello world";
$termX = 0;
$query = preg_replace_callback("/\:term/", function ($matches) use (&$termX) { $termX++; return $matches[0] . ($termX - 1); }, $query);
$pdo->prepare($query);
for ($i = 0; $i < $termX; $i++)
$pdo->bindValue(":term$i", "%$term%", PDO::PARAM_STR);
Alright, here is a sample. I don't have time for sqlfiddle but I will add one later if it is necessary.
(
SELECT
t1.`name` AS resultText
FROM table1 AS t1
WHERE
t1.parent = :userID
AND
(
t1.`name` LIKE :term
OR
t1.`number` LIKE :term
AND
t1.`status` = :flagStatus
)
)
UNION
(
SELECT
t2.`name` AS resultText
FROM table2 AS t2
WHERE
t2.parent = :userParentID
AND
(
t2.`name` LIKE :term
OR
t2.`ticket` LIKE :term
AND
t1.`state` = :flagTicket
)
)
I have ran over the same problem a couple of times now and I think i have found a pretty simple and good solution. In case i want to use parameters multiple times, I just store them to a MySQL User-Defined Variable.
This makes the code much more readable and you don't need any additional functions in PHP:
$sql = "SET #term = :term";
try
{
$stmt = $dbh->prepare($sql);
$stmt->bindValue(":term", "%$term%", PDO::PARAM_STR);
$stmt->execute();
}
catch(PDOException $e)
{
// error handling
}
$sql = "SELECT ... FROM table WHERE name LIKE #term OR number LIKE #term";
try
{
$stmt = $dbh->prepare($sql);
$stmt->execute();
$stmt->fetchAll();
}
catch(PDOException $e)
{
//error handling
}
The only downside might be that you need to do an additional MySQL query - but imho it's totally worth it.
Since User-Defined Variables are session-bound in MySQL there is also no need to worry about the variable #term causing side-effects in multi-user environments.
I created two functions to solve the problem by renaming double used terms. One for renaming the SQL and one for renaming the bindings.
/**
* Changes double bindings to seperate ones appended with numbers in bindings array
* example: :term will become :term_1, :term_2, .. when used multiple times.
*
* #param string $pstrSql
* #param array $paBindings
* #return array
*/
private function prepareParamtersForMultipleBindings($pstrSql, array $paBindings = array())
{
foreach($paBindings as $lstrBinding => $lmValue)
{
// $lnTermCount= substr_count($pstrSql, ':'.$lstrBinding);
preg_match_all("/:".$lstrBinding."\b/", $pstrSql, $laMatches);
$lnTermCount= (isset($laMatches[0])) ? count($laMatches[0]) : 0;
if($lnTermCount > 1)
{
for($lnIndex = 1; $lnIndex <= $lnTermCount; $lnIndex++)
{
$paBindings[$lstrBinding.'_'.$lnIndex] = $lmValue;
}
unset($paBindings[$lstrBinding]);
}
}
return $paBindings;
}
/**
* Changes double bindings to seperate ones appended with numbers in SQL string
* example: :term will become :term_1, :term_2, .. when used multiple times.
*
* #param string $pstrSql
* #param array $paBindings
* #return string
*/
private function prepareSqlForMultipleBindings($pstrSql, array $paBindings = array())
{
foreach($paBindings as $lstrBinding => $lmValue)
{
// $lnTermCount= substr_count($pstrSql, ':'.$lstrBinding);
preg_match_all("/:".$lstrBinding."\b/", $pstrSql, $laMatches);
$lnTermCount= (isset($laMatches[0])) ? count($laMatches[0]) : 0;
if($lnTermCount > 1)
{
$lnCount= 0;
$pstrSql= preg_replace_callback('(:'.$lstrBinding.'\b)', function($paMatches) use (&$lnCount) {
$lnCount++;
return sprintf("%s_%d", $paMatches[0], $lnCount);
} , $pstrSql, $lnLimit = -1, $lnCount);
}
}
return $pstrSql;
}
Example of usage:
$lstrSqlQuery= $this->prepareSqlForMultipleBindings($pstrSqlQuery, $paParameters);
$laParameters= $this->prepareParamtersForMultipleBindings($pstrSqlQuery, $paParameters);
$this->prepare($lstrSqlQuery)->execute($laParameters);
Explanation about the variable naming:
p: parameter, l: local in function
str: string, n: numeric, a: array, m: mixed
I don't know if it's changed since the question was posted, but checking the manual now, it says:
You cannot use a named parameter marker of the same name more than once in a prepared statement, unless emulation mode is on.
http://php.net/manual/en/pdo.prepare.php -- (Emphasis mine.)
So, technically, allowing emulated prepares by using $PDO_obj->setAttribute( PDO::ATTR_EMULATE_PREPARES, true ); will work too; though it may not be a good idea (as discussed in this answer, turning off emulated prepared statements is one way to protect from certain injection attacks; though some have written to the contrary that it makes no difference to security whether prepares are emulated or not. (I don't know, but I don't think that the latter had the former-mentioned attack in mind.)
I'm adding this answer for the sake of completeness; as I turned emulate_prepares off on the site I'm working on, and it caused search to break, as it was using a similar query (SELECT ... FROM tbl WHERE (Field1 LIKE :term OR Field2 LIKE :term) ...), and it was working fine, until I explicitly set PDO::ATTR_EMULATE_PREPARES to false, then it started failing.
(PHP 5.4.38, MySQL 5.1.73 FWIW)
This question is what tipped me off that you can't use a named parameter twice in the same query (which seems counterintuitive to me, but oh well). (Somehow I missed that in the manual even though I looked at that page many times.)
It's possible only if you enable prepared statement emulation. You can do it by setting PDO::ATTR_EMULATE_PREPARES to true.
A working solution:
$pdo->setAttribute(PDO::ATTR_EMULATE_PREPARES, TRUE);
$query = "SELECT * FROM table WHERE name LIKE :term OR number LIKE :term";
$term = "hello world";
$stmt = $pdo->prepare($query);
$stmt->execute(array('term' => "%$term%"));
$data = $stmt->fetchAll();
User defined variables its one way to go and use a the same variable multiple times on binding values to the queries and yeah that works well.
//Setting this doesn't work at all, I tested it myself
$pdo->setAttribute(PDO::ATTR_EMULATE_PREPARES, TRUE);
I didn't wanted to use user defined variables at all like one of the solutions posted here. I didn't wanted also to do param renaming like the other solution posted here. So here it's my solution that works without using user defined variables and without renaming anything in your query with less code and it doesn't care about how many times the param is used in the query. I use this on all my project and it's works well.
//Example values
var $query = "select * from test_table where param_name_1 = :parameter and param_name_2 = :parameter";
var param_name = ":parameter";
var param_value = "value";
//Wrap these lines of codes in a function as needed sending 3 params $query, $param_name and $param_value.
//You can also use an array as I do!
//Lets check if the param is defined in the query
if (strpos($query, $param_name) !== false)
{
//Get the number of times the param appears in the query
$ocurrences = substr_count($query, $param_name);
//Loop the number of times the param is defined and bind the param value as many times needed
for ($i = 0; $i < $ocurrences; $i++)
{
//Let's bind the value to the param
$statement->bindValue($param_name, $param_value);
}
}
And here is a simple working solution!
Hope this helps someone in the near future.

prevent sql injection on query with variable (and large) number of columns

I have a sql query that is generated using php. It returns the surrogate key of any record that has fields matching the search term as well as any record that has related records in other tables matching the search term.
I join the tables into one then use a separate function to retrieve a list of the columns contained in the tables (I want to allow additions to tables without re-writing php code to lower ongoing maintenance).
Then use this code
foreach ($col_array as $cur_col) {
foreach ($search_terms_array as $term_searching) {
$qry_string.="UPPER(";
$qry_string.=$cur_col;
$qry_string.=") like '%";
$qry_string.=strtoupper($term_searching);
$qry_string.="%' or ";
}
}
To generate the rest of the query string
select tbl_sub_model.sub_model_sk from tbl_sub_model inner join [about 10 other tables]
where [much code removed] or UPPER(tbl_model.image_id) like '%HONDA%' or
UPPER(tbl_model.image_id) like '%ACCORD%' or UPPER(tbl_badge.sub_model_sk) like '%HONDA%'
or UPPER(tbl_badge.sub_model_sk) like '%ACCORD%' or UPPER(tbl_badge.badge) like '%HONDA%'
or UPPER(tbl_badge.badge) like '%ACCORD%' group by tbl_sub_model.sub_model_sk
It does what I want it to do however it is vulnerable to sql injection. I have been replacing my mysql_* code with pdo to prevent that but how I'm going to secure this one is beyond me.
So my question is, how do I search all these tables in a secure fashion?
Here is a solution that asks the database to uppercase the search terms and also to adorn them with '%' wildcards:
$parameters = array();
$conditions = array();
foreach ($col_array as $cur_col) {
foreach ($search_terms_array as $term_searching) {
$conditions[] = "UPPER( $cur_col ) LIKE CONCAT('%', UPPER(?), '%')";
$parameters[] = $term_searching;
}
}
$STH = $DBH->prepare('SELECT fields FROM tbl WHERE ' . implode(' OR ', $conditions));
$STH->execute($parameters);
Notes:
We let MySQL call UPPER() on the user's search term, rather than having PHP call strtoupper()
That should limit possible hilarious/confounding mismatched character set issues. All your normalization happens in one place, and as close as possible to the moment of use.
CONCAT() is MySQL-specific
However, as you tagged the question [mysql], that's probably not an issue.
This query, like your original query, will defy indexing.
Try something like this using an array to hold parameters. Notice % is added before and after term as LIKE %?% does not work in query string.PHP Manual
//Create array to hold $term_searching
$data = array();
foreach ($col_array as $cur_col) {
foreach ($search_terms_array as $term_searching) {
$item = "%".strtoupper($term_searching)."%";//LIKE %?% does not work
array_push($data,$item)
$qry_string.="UPPER(";
$qry_string.=$cur_col;
$qry_string.=") LIKE ? OR";
}
}
$qry_string = substr($qry_string, 0, -3);//Added to remove last OR
$STH = $DBH->prepare("SELECT fields FROM table WHERE ". $qry_string);//prepare added
$STH->execute($data);
EDIT
$qry_string = substr($qry_string, 0, -3) added to remove last occurrence of OR and prepare added to $STH = $DBH->prepare("SElECT fields FROM table WHERE". $qry_string)

How can I make this query injection-proof? (PHP)

The Gist
I want to perform an SQL query that depends on a variable number of parameters in my GET without being vulnerable to SQL injection.
The Parameters
My URL can be formed like this:
https://www.example.com/index.php?param1=blah1,param2=blah2,param3=a,b,c
or like this:
https://www.example.com/index.php?param1=blah1,param2=blah2,param3=a,b,c,d,e,f,g
In other words, param3 can have a variable number of comma-delimited parameters a,b,c,etc.
The White-list
I check to make sure that all parameters in a,b,c,etc. are in an approved white-list before I perform the query.
// $valid_params is an array of pre-approved parameters.
$arr = explode(',', clean($_GET['param3']));
$params = Array();
foreach($arr as $param){
if(in_array($param, $valid_params)){
array_push($params, $param);
}
}
The Query
I set up my database connection like this (with MySQL):
$db_connection = new PDO("mysql:host={$DB_HOST};dbname={$DB_NAME}",$DB_USER,$DB_PASS);
$db_connection->setAttribute(PDO::ATTR_EMULATE_PREPARES, false);
$db_connection->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
And I want to perform a query like this (except safely):
$comma_separated_params = implode(',',$params);
$result = $db_connection->query("SELECT {$comma_separated_params} FROM some_table");
The Goal
Does anyone know how I could do this safely and efficiently?
Depending on your concern for overhead, you could just SELECT * and then filter the array in PHP - if the parameter is never sent to the database then there is no room for injection.
However it's not exactly the most elegant solution. Here's how I'd do it:
$comma_separated_params =
implode(
",",
array_map(
function($a) {return "`".$a."`";},
array_intersect(
explode(",",$_GET['param3']),
$valid_params
)
)
)
);
That one-line-wonder (with newlines added for clarity) will take the $_GET['param3'] variable, split it on commas, intersect it with your valid parameters (instead of your foreach loop), wraps each element in backticks (see note below) and finally glues them together with commas.
See, backticks allow you to use literally any string as a field name. Usually it's to allow keywords as names, but it can also allow for column names with spaces, and so on. The only character that has meaning within the backticks are backslashes and backticks - which it is safe to assume are not present since they'd have to be in your list of $valid_params to get this far.
Whitelisting is the way to go here. If you only allow things in that you've already specifically defined you should be okay. As for how efficient, this is all relative. The version you're using will perform well for relatively small lists, such as those with under 100 columns, so I wouldn't worry.
Bonus points for using PDO.
There's a chance your definition of 'allowed' columns and what's actually in the database might diverge. A more relaxed specification might be to grab the fields using SHOW FIELDS for the table in question and only allow those.
If you are only allowing a specific list of predefined values to be passed in param 3, and you are comparing the input values against them, I don;t think you have any injection exposure, as you then have full control over the value that ultimately go into your $comma_seperated_params variable.
This needs some work to finish, but with parameter binding it would look like this:
$binding = array();
$selects = array();
foreach ( $params as $value ) {
$binding[] = ':' . $value;
$selects = '?';
}
$select = implode(',', $select);
$result = $db_connection->prepare("SELECT $select FROM some_table");
foreach ( $binding as $key => $bind ) {
$result->bindParam($key, $bind, PDO::PARAM_STR);
}
$result->execute();
PDO::prepare will help you. This is exactly is recommended by experts. Never use mysql_real_escape_string (string). Always go for prepared statements.

PHP mysql - ...AND column='anything'...?

Is there any way to check if a column is "anything"? The reason is that i have a searchfunction that get's an ID from the URL, and then it passes it through the sql algorithm and shows the result. But if that URL "function" (?) isn't filled in, it just searches for:
...AND column=''...
and that doesn't return any results at all. I've tried using a "%", but that doesn't do anything.
Any ideas?
Here's the query:
mysql_query("SELECT * FROM filer
WHERE real_name LIKE '%$searchString%'
AND public='1' AND ikon='$tab'
OR filinfo LIKE '%$searchString%'
AND public='1'
AND ikon='$tab'
ORDER BY rank DESC, kommentarer DESC");
The problem is "ikon=''"...
and ikon like '%' would check for the column containing "anything". Note that like can also be used for comparing to literal strings with no wildcards, so, if you change that portion of SQL to use like then you could pre-set the variable to '%' and be all set.
However, as someone else mentioned below, beware of SQL injection attacks. I always strongly suggest that people use mysqli and prepared queries instead of relying on mysql_real_escape_string().
You can dynamically create your query, e.g.:
$query = "SELECT * FROM table WHERE foo='bar'";
if(isset($_GET['id'])) {
$query .= " AND column='" . mysql_real_escape_string($_GET['id']) . "'";
}
Update: Updated code to be closer to the OP's question.
Try using this:
AND ('$tab' = '' OR ikon = '$tab')
If the empty string is given then the condition will always succeed.
Alternatively, from PHP you could build two different queries depending on whether $id is empty or not.
Run your query if search string is provided by wrapping it in if-else condition:
$id = (int) $_GET['id'];
if ($id)
{
// run query
}
else
{
// echo oops
}
There is noway to check if a column is "anything"
The way to include all values into query result is exclude this field from the query.
But you can always build a query dynamically.
Just a small example:
$w=array();
if (!empty($_GET['rooms'])) $w[]="rooms='".mysql_real_escape_string($_GET['rooms'])."'";
if (!empty($_GET['space'])) $w[]="space='".mysql_real_escape_string($_GET['space'])."'";
if (!empty($_GET['max_price'])) $w[]="price < '".mysql_real_escape_string($_GET['max_price'])."'";
if (count($w)) $where="WHERE ".implode(' AND ',$w); else $where='';
$query="select * from table $where";
For your query it's very easy:
$ikon="";
if ($id) $ikon = "AND ikon='$tab'";
mysql_query("SELECT * FROM filer
WHERE (real_name LIKE '%$searchString%'
OR filinfo LIKE '%$searchString%')
AND public='1'
$ikon
ORDER BY rank DESC, kommentarer DESC");
I hope you have all your strings already escaped
I take it that you are adding the values in from variables. The variable is coming and you need to do something with it - too late to hardcode a 'OR 1 = 1' section in there. You need to understand that LIKE isn't what it sounds like (partial matching only) - it does exact matches too. There is no need for 'field = anything' as:
{field LIKE '%'} will give you everything
{field LIKE 'specific_value'} will ONLY give you that value - it is not partial matching like it sounds like it would be.
Using 'specific_value%' or '%specific_value' will start doing partial matching. Therefore LIKE should do all you need for when you have a variable incoming that may be a '%' to get everything or a specific value that you want to match exactly. This is how search filtering behaviour would usually happen I expect.

Categories