I have website where users can search posts by entering keywords,
I am using Sphinx search for full text search, everyhting is working as expected.
But when i enter/input some special charaters in search query the search dosnt complete and throws error.
e.g.
keyword i search for :
hello)
my query for sphinxql :
SELECT id FROM index1 WHERE MATCH('hello)')
error i get :
index index1: syntax error, unexpected ')' near ')'
my php code looks like this
<?php
$sphinxql = mysqli_connect($sphinxql_host.':'.$sphinxql_port,'','') or die('ERROR');
$q = urldecode($_GET['q']);
$sphinxql_query = "SELECT id FROM $sphinx_index WHERE MATCH('".$q."') ";
?>
How can i escape user input and make sure the query wont brake and return the result set ?
You should use SQL escaping, to avoid SQL injection.
http://php.net/manual/en/mysqli.real-escape-string.php
$sphinxql_query = ".... MATCH('".mysqli_real_escape_string($sphinxql,$q)."') ";
... BUT you may want to ALSO, escape extended syntax.
See the FIRST THREE POSTS (after that it delves into misunderstanding) in this thread in the sphinx forum
http://sphinxsearch.com/forum/view.html?id=13619
For a simple solution.
The function in that thread, can be used to make your query work. It will escape the ) and stop it being taken as a operator.
BUT, it also means you WONT be able to use any search operators - because it blindly escapes them ALL. (which is the confusion later in the thread)
If you want to be able to use some or all operators, need to use more advanced escaping. (which I dont have a good solution for)
Edit: actully lets go the whole hog...
<?php
//Escapes all the Extended syntax, so can accept anything the user throws at us.
function EscapeString ( $string ) {
$from = array ( '\\', '(',')','|','-','!','#','~','"','&', '/', '^', '$', '=' );
$to = array ( '\\\\', '\(','\)','\|','\-','\!','\#','\~','\"', '\&', '\/', '\^', '\$', '\=' );
return str_replace ( $from, $to, $string );
}
if ($allow_full_extended_syntax) {
$q = $_GET['q'];
// the user is responsible for providing valid query.
} elseif ($allow_partical_extended_syntax) {
$q = InteligentEscape($_GET['q']);
//I don't have this function, it would need to be created.
} else {
$q = EscapeString($_GET['q']);
// escapes ALL extended syntax. NO operators allowed
}
$sphinxql_query = ".... MATCH('".mysqli_real_escape_string($sphinxql,$q)."') ";
Then it sounds like you want both $allow_full_extended_syntax and $allow_partical_extended_syntax set to false. Which means no operators will work, because they will be fully escaped.
The EscapeString function needs to escape the < character as well. Also see escapeString function in PECL shpinx for reference.
Related
In our web application, we have a class which emulates prepared statements for correct escaping of SQL parameters.
Now, I now it would be much better to just use PDO, but the app is very old and the refactoring could be quite long, so for the time being I just wanted to fix a bug I found.
Consider this piece of code which uses our classes:
$s = Q()->statement("SELECT * FROM DUAL WHERE 1 = :a AND 2 = :b AND 3 = :c");
$s->bindValue('c', ':b');
$s->bindValue('b', ':a');
$s->bindValue('a', ':c');
var_dump($s->prepared);
the first line creates the statement, then some values are bound, then I dump the prepared statement.
The result of this is the following:
SELECT * FROM DUAL WHERE 1 = ':c' AND 2 = '':c'' AND 3 = ''':c'''
and this happens because the parameters are substituted one at a time from the last to the first.
I also tried doing the replacement in a single function call, using str_replace() with array parameters, but to no avail.
So I would like to know if there is a way to make this operation somehow "atomic" so that if a placeholder value is another valid placeholder it does not get replaced.
EDIT:
Here is the method of my class which does the replacement:
protected function prepare() {
if (!$this->db) {
trigger_error (__METHOD__ . ': no Connection available to properly quote the value', E_USER_WARNING);
return false;
}
$this->prepared = str_replace(
array_map(array($this, 'getPlaceholderName'), array_keys($this->params)),
array_map(array($this->db, 'quote'), array_values($this->params)),
$this->original
);
return true;
}
You probably want a single call to good old strtr() whith the array signature:
string strtr ( string $str , array $replace_pairs )
Of course, plain string replacement is just a hack and can never replace a proper SQL parser, but I guess you already know that.
This is the code you want to use:
$queryString = strtr($queryString, array(":a" => "b",":b"=>"c", ":c" => "a"));
So let me give you some information, i have a blog system which is backed to a database, the database holds the title of the articles. Now i have tried to create a "related news" feature, and it is rather basic, it just takes the title of the existing page and splits it up like so (Note that $row_object_title is just the title pulled from the database):
$row_object_title_lower = strtolower($row_object_title);
$keywords = explode(" ",$row_object_title_lower);
I then run it through my function:
exclude_conjuctions($keywords);
code for that function(looks for certain words and removes it from the array:
function exclude_conjuctions($array){
global $keywords_new;
$keywords_new = $array;
$conjuctions = array("here","to","and","but","or","nor","for");
$counter = count($keywords_new);
foreach($conjuctions as $conjuction){
for($i=0;$i <= $counter;$i++){
if ($keywords_new[$i] == $conjuction){
unset($keywords_new[$i]);
}
}
}
return $keywords_new;
}
So now i will build my query to retreive all articles that have the keywords in the title:
$sql = '';
foreach ($keywords_new AS $keyword)
{
if ($sql != '')
$sql .= ' OR ';
$sql .= "object_title LIKE '%$keyword%'";
}
$squery = 'SELECT object_title FROM example_table WHERE '.$sql;
NOW. It seems to be working okay, but there are times when it returns a title which does not have the same words as the current article, so i investigated it and it seems it picks up parts of the word and returns it, which is of course not what we want, if you are confused take a look at this image:
http://puu.sh/7UhhW.jpg
Note how i search for "dow" and those letters are found in both the current title and the retrieved titles. Of course i need it to only return related articles that have the full words in the title, not part of the words. What am i doing wrong guys? maybe my MySQL query needs to be changed? maybe there is a better solution? would love some help.
This is a problem as you can imagine.
Thanks for the help in advance.
Try doing LIKE '% {$keyword} %'
Also your query is vulnerable for SQL Injections.
How can I prevent SQL injection in PHP?
EDIT : A better way to do this would be using a Regular Expression:
REGEXP '[[:<:]]{$keyword}[[:>:]]'
Instead of LIKE...
Try using the === operator instead of the == operator to compare strings. A good reason why can be found here
Also, you are wrapping your query with % on each side. That says to return all matches that CONTAIN those strings. Thus 'Dow' is contained in 'Down' and would be returned. You probably want to add a space around the %'s to only get matches that equal your keywords.
Could you implement the "LIKE" search to include a preceding and succeeding space? You would possibly need to have three conditions though to cater for words at the start and end of sentence:
$sql .= "object_title LIKE '% $keyword %' OR LIKE '$keyword %' OR LIKE '% $keyword'";
I have an html form with multiple checkboxes. I pass those to php...
The values will come in like "G,BW" for example ("BW,G" needs to match as well)
in check.php, I need to take the values from $_GET and modify them for an sql query...
<?php
if(!empty($_GET['wireColor'])) {
foreach($_GET['wireColor'] as $colors) {
echo $colors; //make sure it's right, then comment out
}
}
$colors = rtrim($colors, ','); //Get rid of trailing , - not working right
$wireSearch = '\'REGEXP \'.*(^|,).$wireColor.(,|$)\''; //Trying to add the sql bits to pass to the query.
Ideally to get this passed:
$colors_lookup_sql = "SELECT * FROM parts WHERE ( wire_colors REGEXP '.*(^|,)$wireSearch(,|$)' and wire_colors REGEXP '.*(^|,)$wireSearch(,|$)' );";
Here's how the query should look at the end:
SELECT * FROM parts WHERE ( wire_colors REGEXP '.*(^|,)G(,|$)' and wire_colors REGEXP '.*(^|,)BW(,|$)' );
I'm having a hard time getting the regex bits into the query.
UPDATE
Here's what I have now:
<?php
if(!empty($_GET['wireColor'])) {
foreach($_GET['wireColor'] as $colors) {
$wireSearch = ' REGEXP \'.*(^|,)' .$colors.'(,|$)\' AND ';
}
}
$Search = rtrim($wireSearch, 'AND'); //Trying to trim the last "AND"
$colors_lookup_sql = "SELECT * FROM parts WHERE ( wire_colors $wireSearch% );";
Which gives me mostly what I need, but print/echo the results and I get:
$wireSearch ends up as: REGEXP '.*(^|,)G(,|$)' AND REGEXP '.*(^|,)BW(,|$)' AND Which is great - I just need to nuke the last "AND". The trim above replaces it with the second value instead though. Weird.
and $colors_lookup_sql ends up as: SELECT * FROM parts WHERE ( wire_colors REGEXP '.*(^|,)BW(,|$)' AND % );
BUt for some reason the first value in the array goes away, which I don't understand since it was present before the sql statement.
I'm not sure about the REGEX inside the query since I haven't used it but:
$wireSearch = '\'REGEXP \'.*(^|,).$wireColor.(,|$)\'';
Here you have a problem, the variable $wireColor is inside a string, and you are using ' so anything inside is not read as a variable, it should be something like:
$wireSearch = '\'REGEXP \'.*(^|,)'.$wireColor.'(,|$)\'';
I cant say I entirely understand how your data is being stored, and I havent worked with REGEX much myself, but perhaps something like this would be a bit easier to work with:
$wireSearch = explode(",", $_GET['wireColor']);
$query = "SELECT * FROM parts WHERE wire_colors LIKE '%$wireSearch[0]%'
AND wire_colors LIKE '%$wireSearch[1]%'";
Not sure if this helps but I thought id throw in the idea.
I guess mysql regex supports word boundaries, how about:
wire_colors REGEX '\bBW\b' and wire_colors regex '\bW\b'
I have a form and a user enters eg (note apostrophe at end of string)
My Bday'
Now, I want to strip apostrophes, simple as that... not escape them, not add slashes just get rid of them
Firstly I have the following:
$event_title = mysql_real_escape_string($_POST['event_title']);
echo "<br /><br /><br />event title is $event_title";
Which results in the following being returned:
event title is My Bday\\\'
Why 3 slashes?
So, then I go ahead and deal with this by using the following:
$event_title = str_replace("'", "", $event_title);
$event_title = stripslashes($event_title);
Then I return it again to check results
echo "<br /><br /><br />event title is $event_title";
I get the following:
event title is My Bday\
Any ideas what's happening? I simply want to strip apostophes and slashes but somehow it's not happening
magic_quotes_gpc is off by the way
If I don't use stripslashes therefore leaving them in for MySQL to deal with I get the following error:
You have an error in your SQL syntax; check the manual that corresponds to your
MySQL server version for the right syntax to use near 'Private',
event_notes = '' where user_event_id = '35'' at line 3
update user_events set event_date = '2012-11-17', event_title = 'My Bday\\\',
event_vis = 'Private', event_notes = '' where user_event_id = '35'
OK, a further EDIT:
I tried this:
$event_title = $_POST['event_title'];
$event_title = str_replace("'", "", $event_title);
$event_title = trim($event_title);
$event_title = mysql_real_escape_string($event_title);
echo "<br /><br /><br />event title is $event_title";
and I get this:
event title is My Bday\\
I simply want to get rid of apostrophes, clearly something else is going on here but its got me!
What's happening is this:
mysql_real_escape_string escapes all the characters that should be escaped by adding a slash in front of a character being escaped. But adding just a slash will lead to storing the character as unescaped within the DB, therefore also the slash must be escaped prior to inserting...
That's why You have My BDay\\\'. If this value is stored into a DB the final result will be My BDay\'.
But when You do str_replace("'", "", 'My BDay\\\''); You will end up with My BDay\\\ and after calling stripslashes on this You will get My BDay\ - that is absolutely correct!
So don't bother with how the string looks like after calling mysql_real_escape_string, just store that value into the DB and after retrieving it You will end up with My BDay' again...
EDIT How You come to just one slash from the three after calling stripslasshes? The function goes from the start of the string to its end and looks for any slash escaped characters to remove the escaping slash. So it finds first two slashes and removes one, but still two remains (the one just processed and the third one), so it processes next two slasshes it finds that will result in just one slash remaining...
If You'd call stripslashes on the string My BDay\\\' - that will lead to My BDay'...
EDIT2 My bad... The next two slashes are added probably because You have magic_quotes_gpc ON - turn that off or call mysql_real_escape_string(stripslashes($string)).
One slash to escape the apostrophe, the other to escape the slash that escape the apostrophe.
Internally the mysql interpret the \' how '
in your php settings the string_splash setting is ON and that is why when the string is passed from form - it is already excapped... Now you using mysql_real_escape_string - which excapes "excape character" as well as single quote as well. and that is why three slashes..
Try using this function - which I use a lot
function sanitize( $value )
{
if( get_magic_quotes_gpc() )
{
$value = stripslashes( $value );
}
//check if this function exists
if( function_exists( "mysql_real_escape_string" ) )
{
$value = mysql_real_escape_string( $value );
}
//for PHP version < 4.3.0 use addslashes
else
{
$value = addslashes( $value );
}
return $value;
}
It is not specifically for php 4 or older years... The function works to escape string and make sure it does not double escape it ( which is the question beign asked )
if( function_exists( "mysql_real_escape_string" ) ) - this line escapes if the database connection is available and
else part of that if condition makes sure it works if database connection is not available and php 4 support is added benifit only...
This is a follow-up question to the one I posted here (thanks to mario)
Ok, so I have a preg_replace statement to replace a url string with sometext, insert a value from a query string (using $_GET["size"]) and insert a value from a associative array (using $fruitArray["$1"] back reference.)
Input url string would be:
http://mysite.com/script.php?fruit=apple
Output string should be:
http://mysite.com/small/sometext/green/
The PHP I have is as follows:
$result = preg_replace('|http://www.mysite.com/script.php\?fruit=([a-zA-Z0-9_-]*)|e', ' "http://www.mysite.com/" .$_GET["size"]. "/sometext/" .$fruitArray["$1"]. "/"', $result);
This codes outputs the following string:
http://mysite.com/small/sometext//
The code seems to skip the value in $fruitArray["$1"].
What am I missing?
Thanks!
Well, weird thing.
Your code work's perfectly fine for me (see below code that I used for testing locally).
I did however fix 2 things with your regex:
Don't use | as a delimiter, it has meaning in regex.
Your regular expression is only giving the illusion that it works as you're not escaping the .s. It would actually match http://www#mysite%com/script*php?fruit=apple too.
Test script:
$fruitArray = array('apple' => 'green');
$_GET = array('size' => 'small');
$result = 'http://www.mysite.com/script.php?fruit=apple';
$result = preg_replace('#http://www\.mysite\.com/script\.php\?fruit=([a-zA-Z0-9_-]*)#e', ' "http://www.mysite.com/" .$_GET["size"]. "/sometext/" .$fruitArray["$1"]. "/"', $result);
echo $result;
Output:
Rudis-Mac-Pro:~ rudi$ php tmp.php
http://www.mysite.com/small/sometext/green/
The only thing this leads me to think is that $fruitArray is not setup correctly for you.
By the way, I think this may be more appropriate, as it will give you more flexibility in the future, better syntax highlighting and make more sense than using the e modifier for the evil() function to be internally called by PHP ;-) It's also a lot cleaner to read, IMO.
$result = preg_replace_callback('#http://www\.mysite\.com/script\.php\?fruit=([a-zA-Z0-9_-]*)#', function($matches) {
global $fruitArray;
return 'http://www.mysite.com/' . $_GET['size'] . '/sometext/' . $fruitArray[$matches[1]] . '/';
}, $result);
i write it again, i don't understand good where is the error, the evaluation of preg results is very weird in php
preg_replace(
'|http\://([\w\.-]+?)/script\.php\?fruit=([\w_-]+)|e'
, '"http://www.$1/".$_GET["size"]."/sometext/".$fruitArray["$2"]."/";'
, $result
);
It looks like you have forgotten to escape the ?. It should be /script.php\?, with a \? to escape properly, as in the linked answer you provided.
$fruitArray["\$1"] instead of $fruitArray["$1"]