I am integrating Sphinxsearch into my site, and rewriting my old code to use Sphinx.
The problem i got is with special characters inside MATCH in SphinxQL.
Example : want to search for H&M
With plain mysql i get thousand of records, so im quite sure i got title's containing that word in my database.
I don't need extended query syntax in MATCH.
$q = "h&m";
$spxq = "SELECT * FROM sphinx_index WHERE MATCH(:query) LIMIT 0,10";
$stmt = $DB->prepare($spxq);
$stmt->bindValue(':query', $q, PDO::PARAM_STR);
$stmt->execute();
$res = $stmt->fetchAll(PDO::FETCH_ASSOC);
This isn't returning any results.
After this i tried escaping $q so :
$q = "h\&m";
Still not working.Tried with double escaping too, still not working.
Any help would be appriciated.
Ok, i got this, if anyone in the future has this same problem.
The solution is to add the special characters you want to be able to search for in the sphinx.conf file, in the charset_table.
So for my case of H&M, you need to add the & character (U+026) in to the charset table.
Related
I have the following simple search query code:
function explode_search($squery, $column, $db, $link) {
global $conn;
$ven = explode(' ', safeInput(str_replace(',', '', $squery)));
$ven2 = array_map('trim', $ven);
$qy = ''.$column.' LIKE "%'.implode('%" AND '.$column.' LIKE "%', $ven2).'%"';
$query = 'SELECT DISTINCT '.$column.', id, work_composer FROM '.$db.' WHERE '.$qy.' ORDER BY '.$column.' LIMIT 100';
$result = mysqli_query($conn, $query);
while ($row = mysqli_fetch_assoc($result)) {
echo '<div><span class="mdmtxt" style="margin-bottom:5px;">'.$row[$column].'</span> <span class="mdmtxt" style="opacity:0.6;">('.fixcomp(cfid($row['work_composer'], 'cffor_composer', 'composer_name')).')</span></div>';
}
}
(The safeInput function removes ' and " and other possible problematics)
It works alright up to a point.
When someone looks for 'Stephane' I want them also to find 'Stéphane' (and vice versa) or if they are looking for 'Munich', 'Münich' should show up in the list as well.
Is there a way to make MySQL match those search queries, irrespective of the special characters involved?
You want to use what's called a "Parameterized Query". Most languages have them, and they are used to safeguard input and protect from attacks.
They're also extremely easy to use after you get used to them.
You start out with a simple query like this
$stmt = $mysqli->prepare("SELECT District FROM City WHERE Name=?")
and you replace all your actual data with ?s.
Then you bind each parameter.
$stmt->bind_param("s", $city);
Then call $stmt->execute();
More information can be found here: http://php.net/manual/en/mysqli.prepare.php
One time in college, our prof made us use a library that didn't have parameterized queries, so I wrote an implementation myself. Doctrine, which is pretty awesome, handles all this for you. I would always rely on someone else to do this stuff for me instead of writing my own implementation. You'll get into trouble that way. There's also a lot written about not reinventing new types which is basically what you're doing. Types have problems. Types need testing. This kind of thing is well tested in other implementations and not yours.
I'm having an issue trying to bind and execute a PDO statement using PHP and a REGEXP for a SQLite database. I have a column containing string words separated by spaces, and I am using the following REGEXP (?<!\S)WORD(?!\S) to match complete words that are surrounded by white space. The following SQL query works when I run in SQLite studio and gives me the desired results:
SELECT * FROM Questions WHERE Tags REGEXP '(?<!\S)WORD(?!\S)'
However, when I try to use it in a PDO statement, it keeps failing. So far I have tried:
$sql = "SELECT * FROM Questions WHERE Tags REGEXP :tags";
$tags = '(?<!\S)WORD(?!\S)';
$statement = $db -> prepare($sql);
$statement -> bindParam(':tags', $tags);
and I have also tried escaping backslashes as well as running the query directly without using any prepared statements and simply using a known working query and it still seems to fail.
Ex w/o binds:
$sql = "SELECT * FROM Questions WHERE Tags REGEXP '(?<!\S)TAG(?!\S)'";
$statement = $db -> query($sql);
However that doesn't seem to work with the REGEXP either. If I change it to a simple SELECT * from the entire table or any non-REGEXP query everything seems to work fine until I add in those REGEXPs. I'm pretty new to PHP so I'm not entirely sure what is going on here or where to look. Any help would be appreciated!
I am currently working on a php project and used the word 'value' as a column name. The problem being that when I run the query, it overwrites all entries in the database, even though I have a delimiter (primary key = *). I have tried everything I can think of to get this to work, and it hasn't yet. here is the complete line of code:
$SqlStatement = "UPDATE rev_exp SET Date_Entered = '".date('Y-m-d')."', Description = '".$_POST['txtUtilityType']." ".$_POST['txtAccountNumber']." ".$_POST['txtDateAdded']."', `Value` = ".$_POST['txtValueBalance'].", Notes = '".$_POST['txtNotes']."' WHERE PK_Rev_Exp = ".$row['FK_Rev_Exp'];
Note here, that $row['FK_Rev_Exp'] is the delimiter I was talking about. It is being pulled accurately from a previous query. Also, please ignore any sql injection problems, I'm just working on getting the project functional, I can optimize later.
EDIT 1: I have also tried enclosing the "value" in everything I can think of that may get rid of this problem, but no luck.
EDIT 2: I also don't think it is a problem with the statement itself, as I directly entered the statement into the mysql command line and it only affected 1 row, possibly a php problem?
EDIT 3: Full block, including the execution of the sql. Here, ExecuteSQL runs all necessary mysqli statements to execute the sql command. it takes in a sql statement and a true/false if there is a result set:
$SqlStatement = "UPDATE rev_exp SET Date_Entered = '".date('Y-m-d')."', Description = '".$_POST['txtUtilityType']." ".$_POST['txtAccountNumber']." ".$_POST['txtDateAdded']."', `Value` = '".$_POST['txtValueBalance']."', Notes = '".$_POST['txtNotes']."' WHERE PK_Rev_Exp = ".$row['FK_Rev_Exp'];
ExecuteSQL($SqlStatement, false);
I can't figure it out, and any help would be appreciated.
I think your problem is not about mysql reserver keywords because your correctly surrounded Value with backtick and that makes database understand this is a field. I'm more concerned about treating not integers as integers so i would suggest to surround with quotes '' your value since it is a decimal
`Value` = '".$_POST['txtValueBalance']."',
I've been looking through a bunch of topics but haven't found anything that works yet - or maybe it does, but I'm pretty new to PHP/MySQL, so maybe I'm just missing something.
I've got a page that allows users to type in the name of an item to look it up in the (MySQL) database. It then generates a table of results and displays those, and turns some of the data into links. The linked terms are then used as queries to search other fields of the database.
Everything works... except when one of the search terms contains an apostrophe. Obviously I want to prevent injection issues, but I also need to search with the apostrophes. Here's the code I have:
$query = mysql_real_escape_string($query);
$query = htmlspecialchars($query);
$queryentered = $query;
$query = strtolower($query);
And here's the URL that's it's passing to the query:
echo " from <a href='index.php?volume=".$results2['book']."' class='where'>"
.$results2['book']."</a>";
And the query:
$raw_results = mysql_query("SELECT * FROM $database1 WHERE $wh LIKE '%$query%'
ORDER BY displayname") or die(mysql_error());
What am I not encoding right? When I run a search, it displays "No results found for [search term]", where I'm getting \' for any apostrophes.
This snippet is in exactly the wrong order:
$query = mysql_real_escape_string($query);
$query = htmlspecialchars($query);
$queryentered = $query;
$query = strtolower($query);
Escaping should be done last, right before the database query string concatenation. In this case it's irrelevant as strtolower does not undo the escaping. More severe is actually the HTML escaping, which does not belong in SQL context. Don't overwrite your main variable, instead:
$queryentered = htmlspecialchars($query);
$query = strtolower($query);
$query = mysql_real_escape_string($query);
The cumbersome database escaping, btw, is easy to avoid. Look into PDO and prepared statements.
problem is in this line:
$query = htmlspecialchars($query);
If you search for Death's book your final $query would be death\'s book!
By removing htmlspecialchars your query would be fine!
I'd STRONGLY recommend using prepared statements and the PDO library.
Really easy to learn
Will make your question easier to solve
Works with (most) any database
Goes a long way to prevent the injection issues you mention
Skill set will be applicable to other languages you may learn
All you have to do is write the query, then prepare a connection string (with the database type and details) and then fire off the query. An array of parameters is sent with the query, to decouple the data.
Helpful links:
http://net.tutsplus.com/tutorials/php/why-you-should-be-using-phps-pdo-for-database-access/
http://id1.php.net/pdo
I've got a table with Clob field (is there any other data types in IBM DB2 to store text in it?). So, i have an article (html+css+text). I used htmlspecialchars($text) to prepare the text.
Then I do next thing:
$query="update tbl_lang_text set text='$text' where ownerid=$id and lang like '$lng' and type=1";
$stmt = db2_prepare($this->conn, $query);
$result = db2_execute($stmt);
So i got an error. something like: the query is too big. So. how can i update my field with such a large text?
P.S.: An test article contains 28 154 characters with spaces.
A CLOB column can take up to 2 147 483 647 characters, so it is unlikely that this is what is causing the error.
What could be problem (and even if it's not you should fix it), is that you are not escaping the input at all. Using prepared statements (i.e.: db2_prepare) is good, but you still need to use parameters and values to have your data escaped:
$query = "update tbl_lang_text set text=?".
"where ownerid=? and lang like ? and type=1";
$stmt = db2_prepare($this->conn, $query);
$result = db2_execute($stmt, array($text, $id, $lng));
It is very likely that $text contains at least an apostrophe ' and that your query fails because of it.