I am executing the following query in a MySQL database (look at SELECT AND WHERE, the rest is not important):
SELECT distinct fname //more fields...
FROM filedepot_files AS ff
INNER JOIN filedepot_categories AS fc
ON ff.cid = fc.cid
INNER JOIN filedepot_access AS fa
ON fc.cid = fa.catid
WHERE fa.permid=$id AND fname LIKE '%$key%'
ORDER BY DATE
The environment is a PHP script running under Drupal with FileDepot module but I doubt that matters at all.
This is the PHP script (well the part that matters):
$id = 1;
$key = $_GET['key'];
$query = .... (see above)
$result = db_query($query);
while($row = db_fetch_array($result)){
//do stuff
echo $row['fname'];
}
db_query() is a Drupal method that allows to easily execute SQL queries and a returns an array, db_fetch_array() allows to parse the result.
Now, DB contains the following entries for fname (there are more, these are just examples):
Dichiarazione 1
Dichiarazione 2
Guida 1
Guida 2
If I launch the script with "guida" as key it correctly returns the two entries both with PHP and MySQL.
If i use "Guida" it works as well.
However if I use "dichiarazione" it doesnt with PHP while it does with MySQL.
Strange thing is that "Dichiarazione" works both with PHP and MySQL.
What is wrong with the query? I tryed to use LOWER(fname) LIKE '%$key%' but it doesn't seem to work as intended.
I am sure there is something stupid that I am missing but I can't seem to find what that is...
% is a special character in Drupal queries (it's used for placeholders). Try double-escaping it:
WHERE fa.permid=$id AND fname LIKE '%%$key%%'
More worryingly though, you're wide open to SQL injection. Some sanitisation is in order:
...
WHERE fa.permid= %d AND fname LIKE '%%%s%%'
...
$query = db_query($sql, $id, $key);
It might look crazy but that's the right number of % signs. Two for each literal %, and one (%s) for the string placeholder
Related
a quick question :), I wrote this because someone said that my codes are vulnerable to mysql injection and it is a requirement to learn prepared statement in web programming to avoid any user putting malicious data or statement into the database..What I have is a search function that search data from the database, if you type in a string like this "torres" then i search for torres but if you just put "tor" it won't search for datas that contain "tor" in their name..I don't know the correct format while using prepared statement, If you have advice I'm very happy to take it :)
<?php
if (isset($_POST['search'])) {
$box = $_POST['box'];
$box = preg_replace("#[^0-9a-z]#i","",$box);
$grade =$_POST['grade'];
$section = $_POST['section'];
$strand = $_POST['strand'];
$sql = "SELECT * FROM student WHERE fname LIKE ? or lname LIKE ? or mname LIKE ? or grade = ? or track = ? or section = ?";
$stmt = mysqli_stmt_init($conn);
if (!mysqli_stmt_prepare($stmt, $sql)){
echo "SQL FAILED";
}
else {
//bind the parameter place holder
mysqli_stmt_bind_param($stmt, "ssssss",$box, $box, $box, $grade, $strand, $section);
mysqli_stmt_execute($stmt);
$result = mysqli_stmt_get_result($stmt);
while($row = mysqli_fetch_assoc($result))
{
echo "<tr>";
echo "<td>".$row['lname']."</td>";
echo "<td>".$row['fname']."</td>";
echo "<td>".$row['mname']."</td>";
echo "<td>".$row['grade']."</td>";
echo "<td>".$row['track']."</td>";
echo "<td>".$row['section']."</td>";
echo "</tr>";
}
}
As requested:
#ArtisticPhoenix I clearly prefer the king's way [compound full text index]. This should be your primary answer showing an example/explaination.
First make a full text index that includes all three fields (this is in PHPmyAdmin, it's a bit easier to explain with an image)
Then do a query like this:
#PDO version SELECT * FROM `temp` WHERE MATCH(fname,mname,lname)AGAINST(:fullname IN BOOLEAN MODE)
#MySqli version SELECT * FROM `temp` WHERE MATCH(fname,mname,lname)AGAINST(? IN BOOLEAN MODE)
SELECT * FROM `temp` WHERE MATCH(fname,mname,lname)AGAINST('edward' IN BOOLEAN MODE)
It seems simple but there are some things with full text to be aware of Min char count which is 3 (I think) anything smaller than that is not searched on. This can be changed but it requires repairing the DB and restarting MySql.
Stop words, these are things like and, the etc. These can also be configured in my.cnf.
Punctuation is ignored. This might not seem a big deal on names but think of hyphenated last names.
Usually I reduce the word min to 2 and point the stopwords to an empty file (disabling them).
The match against syntax is quite different, it's pretty powerful but it's not really used outside of full text. An example is: this is the wild card * and you use '"' double quotes for exact phrase match '"match exactly"', and + is logical AND, such as word+ word+ (default is or), - is do not match this etc... If I remember right, I used it a bunch a few years ago but haven't had to use it recently.
For example doing "begins with" on a partial word
SELECT * FROM `temp` WHERE MATCH(fname,mname,lname)AGAINST('edwar*' IN BOOLEAN MODE)
Same result matches one row. The obvious benefit is searching all 3 fields at the same time, but the full text syntax itself can be quite useful too.
For more information:
https://dev.mysql.com/doc/refman/8.0/en/fulltext-boolean.html
PS. I might add that using OR in a query can really kill performance, I've went as far as to replace simple OR with a UNION because of how bad the performance is on a large table. Logically the DB optimizer has to rescan the entire table for an OR, unlike AND where it can use the result of the previous expression to reduce the next expressions data set (or that is how I understand it). I can say the performance difference is very noticeable using OR vs UNION.
This is true for a compound full text index vs doing OR on each field separately. By default fulltext is faster, but it's even faster this way.
To fix your current query (for the sake of completeness)
You need whats known as an exclusive or, like this:
SELECT * FROM student WHERE ( fname LIKE ? OR lname LIKE ? OR mname LIKE ? ) AND grade = ? AND track = ? AND section = ?
What this does is group the OR's together so that they evalute as one expression to the "next level up" ( outside the parenthesis ). Basically order of operations. In English, you would have to match at least 1 of these columns fname, lname, mname AND you would also have to match all of the rest of the columns as well, to get a result returned for any given row.
If you use all OR (as you are now) and any single field matches, then the query comes back as true with matches. Which is the behaviour you are experiencing now.
If you simply change everything outside of the name fields to AND, Basically remove the parenthesis
Like this:
#this is wrong don't use it.
SELECT * FROM student WHERE fname LIKE ? OR lname LIKE ? OR mname LIKE ? AND grade = ? AND track = ? AND section = ?
Then you have to match this way.
(grade AND track AND section AND mname) OR lname OR fname
So if the last or first name match you get results regardless of any of the other fields. But the mname field you would find has to match with all the rest of the fields to get a result (but you would not likely notice this). Because, it would seem that the query works how you want but only when the mname is a match.
I hope that makes sense. It may be helpful to think of the WHERE clause as an IF condition the same logic rules apply.
Cheers!
Im programming a search with ZF3 and the DB module.
Everytime i use more than 1 short keyword - like "49" and "am" or "1" and "is" i get this error:
Statement could not be executed (HY000 - 2006 - MySQL server has gone away)
Using longer keywords works perfectly fine as long as i dont use 2 or more short keywords.
The problem only occurs on the live server its working fine on the local test server.
The project table has ~2200 rows with all kind of data the project_search table has 17000 rows with multiple entries for each project , each looking like:
id, projectid, searchtext
The searchtext Column is fulltext. Here the relevant part of the php code:
$sql = new Sql($this->db);
$select = $sql->select(['p'=>'projects']);
if(isset($filter['search'])) {
$keywords = preg_split('/\s+/', trim($filter['search']));
$join = $sql->select('project_search');
$join->columns(['projectid' => new Expression('DISTINCT(projectid)')]);
$join->group("projectid");
foreach($keywords as $keyword) {
$join->having(["LOCATE('$keyword', GROUP_CONCAT(searchtext))"]);
}
$select->join(
["m" => $join],
"m.projectid = p.id",
['projectid'],
\Zend\Db\Sql\Select::JOIN_RIGHT
);
}
Here the resulting Query:
SELECT p.*, m.projectid
FROM projects AS p
INNER JOIN (
SELECT projectid
FROM project_search
GROUP BY projectid
HAVING LOCATE('am', GROUP_CONCAT(searchtext))
AND LOCATE('49', GROUP_CONCAT(searchtext))
) AS m
ON m.projectid = p.id
GROUP BY p.id
ORDER BY createdAt DESC
I rewrote the query using "MATCH(searchtext) AGAINST('$keyword)" and "searchtext LIKE '%keyword%' with the same result.
The problem seems to be with the live mysql server how can i debug this ?
[EDIT]
After noticing that the error only occured in a special view which had other search related queries - each using multiple joins (1 join / keyword) - i merged those queries and the error was gone. The amount of queries seemed to kill the server.
Try refactoring your inner query like so.
SELECT a.projectid
FROM (
SELECT DISTINCT projectid
FROM projectsearch
WHERE searchtext LIKE '%am%'
) a
JOIN (
SELECT DISTINCT projectid
FROM projectsearch
WHERE searchtext LIKE '%49%'
) b ON a.projectid = b.projectid
It should give you back the same set of projectid values as your inner query. It gives each projectid value that has matching searchtext for both search terms, even if those terms show up in different rows of project_search. That's what your query does by searching GROUP_CONCAT() output.
Try creating an index on (searchtext, projectid). The use of column LIKE '%sample' means you won't be able to random-access that index, but the two queries in the join may still be able to scan the index, which is faster than scanning the table. To add that index use this command.
ALTER TABLE project_search ADD INDEX project_search_text (searchtext, projectid);
Try to do this in a MySQL client program (phpmyadmin for example) rather than directly from your php program.
Then, using the MySQL client, test the inner query. See how long it takes. Use EXPLAIN SELECT .... to get an explanation of how MySQL is handling the query.
It's possible your short keywords are returning a ridiculously high number of matches, and somehow overwhelming your system. In that case you can put a LIMIT 1000 clause or some such thing at the end of your inner query. That's not likely, though. 17 kilorows is not a large number.
If that doesn't help your production MySQL server is likely misconfigured or corrupt. If I were you I would call your hosting service tech support, somehow get past the front-line support agent (who won't know anything except "reboot your computer" and other such foolishness), and tell them the exact times you got the "gone away" message. They'll be able to check the logs.
Pro tip: I'm sure you know the pitfalls of using LIKE '%text%' as a search term. It's not scalable because it's not sargable: it can't random access an index. If you can possibly redesign your system, it's worth your time and effort.
You could TRY / CATCH to check if you get a more concrete error:
BEGIN TRY
BEGIN TRANSACTION
--Insert Your Queries Here--
COMMIT
END TRY
BEGIN CATCH
DECLARE #ErrorMessage NVARCHAR(4000);
DECLARE #ErrorSeverity INT;
DECLARE #ErrorState INT;
SELECT
#ErrorMessage = ERROR_MESSAGE(),
#ErrorSeverity = ERROR_SEVERITY(),
#ErrorState = ERROR_STATE();
IF ##TRANCOUNT > 0
ROLLBACK
RAISERROR (#ErrorMessage, -- Message text.
#ErrorSeverity, -- Severity.
#ErrorState -- State.
);
END CATCH
Although because you are talking about short words and fulltext it seems to me it must be related to StopWords.
Try running this query from both your dev server and production server and check if there are any differences:
SELECT * FROM INFORMATION_SCHEMA.INNODB_FT_DEFAULT_STOPWORD;
Also check in my.ini (if that is the config file) text file if these are set to:
ft_stopword_file = ""
ft_min_word_len = 1
As stated in my EDIT the problem wasnt the query from the original Question, but some other queries using the search - parameter as well. Every query had a part like follows :
if(isset($filter['search'])) {
$keywords = preg_split('/\s+/', trim($filter['search']));
$field = 1;
foreach($keywords as $keyword) {
$join = $sql->select('project_search');
$join->columns(["pid$field" => 'projectid']);
$join->where(["LOCATE('$keyword', searchtext)"]);
$join->group("projectid");
$select->join(
["m$field" => $join],
"m$field.pid$field = p.id"
);
$field++;
}
}
This resulted in alot of queries with alot of resultrows killing the mysql server eventually. I merged those Queries into the first and the error was gone.
I'm working on a group project from school which requires selecting from a Postgres database with php.
I tested my queries on the psql dbms before trying them in the php interface. This is my test query:
SELECT m.movieid, m.tomatourl FROM movies m WHERE title = 'Beowulf & Grendel';
The query does return the information from the database I need, however when using this in php it returns nothing.
pg_last_error() says nothing.
In what way can I ensure that I can select titles with ampersands(&) in them?
I've tried seperating the string and putting them back together with sql code:
SELECT m.movieid, m.tomatourl FROM movies m WHERE title = 'Beowulf '||chr(38)||' Grendel'
I've tried escaping the string
This is an example of some of my php code:
$query = 'SELECT m.movieid, m.tomatourl FROM movies m WHERE title = $1';
pg_prepare($conn, "getmovie", $query) or die(pg_last_error());
$result = pg_execute($conn, "getmovie", $i) or die("Query failed: ". pg_last_error());
$movie = pg_fetch_array($result, NULL, PGSQL_BOTH);
This will work as long as the string in the $i array does not have an ampersand in it.
I would just change the database to not have an ampersand, but I don't really have control over it.
Is there some way to do a select statement like this using the php postgres functions?
The problem seems to have been caused by the the quotes that are around the string that passed in to the sql, by passing the string directly through the prepared statement it is like this "Beowulf & Grendel"
when it has to be passed to the database like this 'Beowulf & Grendel'
It also seems that even though it wasn't showing in var_dump() directly in the string printout it
was actually sending it as this SELECT m.movieid, m.tomatourl FROM movies m WHERE title = 'Beowulf & Grendel'; The the only thing that gives it away it the character count in the var_dump and not the printout of the string. The fix for this was to do html_entity_decode() on the title passed in.
Special thanks to DarkBee and Daniel Verite for helping solve this issue.
been googling for hours and I'm quite new to this.
I have two identical tables in one MySQL database:
One named "users" and one named "keys".
They are identical for testing purposes.
When I query "users" I get a response, when I query "keys" I get nothing.
Querying users I get the expected response:
<?php
require('../db/connect.php');
$query = mysql_query("
SELECT name
FROM users
WHERE can_share = '".$_POST['URLkey']."'
");
echo mysql_result($query, 0);
?>
Querying keys I get nothing:
<?php
require('../db/connect.php');
$query = mysql_query("
SELECT name
FROM keys
WHERE can_share = '".$_POST['URLkey']."'
");
echo mysql_result($query, 0);
?>
I guess there must be some basic understanding of databases that has slipped by me, but still, after hours of searching I can't figure it out. Maybe I'm becoming retarded.
I think that might be due to table name being 'keys'.
Have a look here:
http://dev.mysql.com/doc/refman/5.5/en/reserved-words.html
You have to understand while designing your tables and naming the attributes that some words are reserved by MySQL itself.
So, if you name your table 'WHERE' you will have troubles in usual query. Why?
'SELECT * FROM WHERE'
Such a query obviously doesn't work, as it will ask you to provide table name.
Now, when you change format situation also changes:
'SELECT * FROM `WHERE`'
As you can see I added some backwards commas. In MySQL they are used to denote names of tables or fields. If you use them - the server processes and reads your query correctly.
So, that's why your edited query worked fine in the end.
Thanks to enabeling debugging, I got this message:
mysql_result() expects parameter 1 to be resource, boolean given in
...
And figured out that I had to query "keys" like this:
<?php
require('../db/connect.php');
$query = mysql_query("
SELECT `name`
FROM `keys`
WHERE `can_share` = '".$_POST['URLkey']."'
");
echo mysql_result($query, 0);
?>
Now it works, but I still don't understand why only one of the tables needed that formatting. And I have learned that I should rewrite the whole thing to not be vulnerable to SQL injection attacks. ...
EDIT: It seems like the words "key" and "keys" and some more are reserved by MySQL, so to use them, they have to be formatted like that.
I've been chasing a bug all day in my code. Here's the long and short of it:
I have a variable that is passed through a query string and I want to use that variable in my MySQLi query. A-like so:
$variable;
$info = dbConnect('query');
$eInfo = "SELECT *
FROM tablename
WHERE fieldname = '$variable%'";
$eiData = $info->query($eInfo) or die(mysqli_error());
$eInfo = $eiData->fetch_assoc();
The emphasis is the % after the variable name. I'm no PHP expert but I remember picking this trick up a while back and it has worked for me on all my DB-driven websites EXCEPT the new one I'm developing.
The above query returns no data from the table.
BUT...
if I omit the % as follows:
$variable;
$info = dbConnect('query');
$eInfo = "SELECT *
FROM tablename
WHERE fieldname = '$variable'";
$eiData = $info->query($eInfo) or die(mysqli_error());
$eInfo = $eiData->fetch_assoc();
The query executes and I get my data.
WHY after writing queries where I want to use a variable using the syntax '$variable%' would this no longer work? Just dropping the % ('$variable') makes it work A-OK, which has me utterly baffled.
For what it's worth, I run a dedicated server and suPHP was recently installed, if that has any remote chance of helping make sense here (PHP 5.2.17 is my current version).
Again, I'm no PHP whiz, but after checking code form older sites I have done with MySQLi queries, the % was always there when appending a variable into a query.
I'm completely dead in the water here. Any help you can provide would be so insanely appreciated that it defies explanation.
if you want to put variables in a string where interpreting where the variable name ends is difficult, use {$var} like so:
$eInfo = "SELECT * FROM tablename WHERE fieldname LIKE '{$variable}%'";
The % is a wildcard for LIKE queries. So if you use % in a query with WHERE var = 'cow%', it's going to search the database for the literal string cow%.
cow
cows
cow% (match)
the cow
In a Like query: WHERE var LIKE 'cow%' the following will match:
cow (match)
cows (match)
cow% (match)
the cow
See MySQL Pattern Matching.