I got a few queries built dynamically by my scripts. They usually fit the following template:
SELECT ...
FROM ...
JOIN ...
WHERE ... (lots of filters, search conditions, etc. here) ...
ORDER BY (optional) ...
LIMIT (optional) ... OFFSET (optional)
I want to remove the LIMIT and OFFSET parts from the query. I used
$sql_entitati = implode("LIMIT", explode("LIMIT", $sql_entitati, -1));
to do it but then it hit me: what if there's no LIMIT in the query and what if the only LIMIT is somewhere in the where clauses?
So my question to you is: How can I safely remove everything after the LIMIT key word, without screwing it up if there's no LIMIT and/or there's a "LIMIT" somewhere in the where clause? All this done in php.
A bit of an edit for clarity:
the algorithm i use:
$sql = implode("LIMIT", explode("LIMIT", $sql, -1));
Will work on 99% of the cases. The problem occurs when the "LIMIT" key word at the end is missing, AND there is "LIMIT" written somewhere in the conditions. for example:
SELECT * FROM table WHERE bla = 'SPEED LIMIT' ORDER BY table.a
this is the problem i need to tackle.
Solved using the following algorithm (Credit to techfoobar):
$p = strrpos($sql, "LIMIT");
if($p !== false) {
$q = strpos($sql, ")", $p);
$r = strpos($sql, "'", $p);
$s = strpos($sql, "\"", $p);
if($q === false && $r === false && $s === false)
$sql = substr($sql, 0, $p);
}
You should do something like:
Get the position of the last "LIMIT" - store this position in say, p
Ensure that you do not have a ")" character after p - if so, it is part of some inner query inside a condition etc..
Ensure that you do not have a "'" character after p - if so, it is part of some user input string
If steps 2 and 3 are passed, strip off everything after p
More hints:
For step 1 - use strrpos for the last occurrence of LIMIT
For steps 2 and 3, use strpos with p as the search start offset
For step4, use substr to strip off everything after p
Get the position of LIMIT, check it exists with !==FALSE then substring.
$str = "SELECT X FROM Y JOIN WHERE CONDITIONS ORDER BY ORDERS LIMIT OFFSET";
$pos = strrpos($str,"LIMIT");
if ($pos!==false)
{
$newStr = substr($str,0,$pos);
echo $newStr;
}
Try this aproach
$query = '';
$query .= 'SELECT...';
$query .= 'FROM ...';
$query .= 'JOIN ...';
$query .= 'WHERE...';
if ($limit)
{
$query .= " LIMIT $limit";
}
This may help you...
$str = " SELECT * FROM table WHERE bla = 'SPEED LIMIT' ORDER BY table.a";
$pos = strrpos($str,"LIMIT");
if ($pos!==false)
{
$ok = 1;
if(substr_count($str, '"', $pos) % 2 != 0)
{
$ok = 0;
}
if(substr_count($str, "'", $pos) % 2 != 0)
{
$ok = 0;
}
if(substr_count($str, ")", $pos) > 0)
{
$ok = 0;
}
if($ok == 1)
$str = substr($str,0,$pos);
}
echo $str;
Related
I create a simple search form. Firstly I explode words and make some query (for search multiple words at once). Also, I want to search only in Movie titles. Movie titles can be found in movie_data.
My database called movies have rows:
id and
movie_data - contains movie data in JSON. Example storaged data in json:
{"movie_title":"Forrest Gump","movie_cover":"http://1.fwcdn.pl/po/09/98/998/7314731.6.jpg","movie_name_en":"","movie_desc":"Historia życia Forresta, chłopca o niskim ilorazie inteligencji z niedowładem kończyn, który staje się miliarderem i bohaterem wojny w Wietnamie.","movie_year":"1994","movie_genres":"DramatKomedia"}
Here is my code. This code don't return any results.
$query = 'Forrest Gump';
$words = explode(' ', $query);
$i = 0;
$len = count($words);
$query_build = '';
foreach ($words as $item) {
if ($i == $len - 1) {
$query_build .= ' titlemov LIKE "%$item%"';
} else {
$query_build .= ' titlemov LIKE "%$item%" OR ';
}
// …
$i++;
}
$sql = "SELECT movie_data, JSON_EXTRACT(movie_data, '$.movie_title') AS titlemov
FROM movies WHERE $query_build";
This code don't result any items
UPDATE
I updated my code.
$query = 'Forrest Gump';
$words = explode(' ', $query);
$i = 0;
$len = count($words);
$query_build = '';
foreach ($words as $item) {
if ($i == $len - 1) {
$query_build .= " JSON_EXTRACT(movie_data, \'$.movie_title\') LIKE '%$item%'";
} else {
$query_build .= " JSON_EXTRACT(movie_data, \'$.movie_title\') LIKE '%$item%' OR ";
}
// …
$i++;
}
$sql = "SELECT * FROM movies WHERE $query_build";
Now i getting error:
Warning: mysqli::query(): (22032/3141): Invalid JSON text in argument 1 to function json_extract: "Missing a comma or '}' after an object member." at position 228. in /search.php on line 43
Line 43:
$result = $conn->query($sql);
var_dump of $sql var:
SELECT * FROM movies WHERE JSON_EXTRACT(movie_data, '$.movie_title') LIKE '%Forrest%' OR JSON_EXTRACT(movie_data, '$.movie_title') LIKE '%Gump%'
Hmm?
titlemov doesn't exist for your where clause to use. What you want to use inside your where clause (specified in the $query_build) is the JSON_SEARCH function. Here's what it looks like in general with wildcards for the movie name:
SELECT movie_data, movie_data->'$.movie_title' AS titlemov FROM movies
WHERE JSON_SEARCH(movie_data->'$.movie_title', 'one', "%orrest Gum%") IS NOT NULL;
You'll just need to modify your $query_build string. I also drop the needless JSON_EXTRACT function calls.
Try this:
$query_build = " JSON_SEARCH(movie_data->'$.movie_title', 'one', '%$item%') IS NOT NULL";
Final note, I'm assuming $item is not a user supplied string, otherwise this opens you up to SQL injection. If it is supplied by user, you'll want to use a prepared statement and bind this value.
I would like to select those movies which have similar titles.
I found this, but this way it dosn't work, it gives nothing. I would like to give toy story 2, toy story 3 and others with similar title like toy soldielrs, etc.
$title = "Toy Story";
$query = mysql_query("SELECT title, year, poster, LEVENSHTEIN_RATIO( ".$title.", title ) as textDiff FROM movies HAVING textDiff > 60");
I can compare strings in PHP with this function:
static public function string_compare($str_a, $str_b)
{
$length = strlen($str_a);
$length_b = strlen($str_b);
$i = 0;
$segmentcount = 0;
$segmentsinfo = array();
$segment = '';
while ($i < $length)
{
$char = substr($str_a, $i, 1);
if (strpos($str_b, $char) !== FALSE)
{
$segment = $segment.$char;
if (strpos($str_b, $segment) !== FALSE)
{
$segmentpos_a = $i - strlen($segment) + 1;
$segmentpos_b = strpos($str_b, $segment);
$positiondiff = abs($segmentpos_a - $segmentpos_b);
$posfactor = ($length - $positiondiff) / $length_b;
$lengthfactor = strlen($segment)/$length;
$segmentsinfo[$segmentcount] = array( 'segment' => $segment, 'score' => ($posfactor * $lengthfactor));
}
else
{
$segment = '';
$i--;
$segmentcount++;
}
}
else
{
$segment = '';
$segmentcount++;
}
$i++;
}
// PHP 5.3 lambda in array_map
$totalscore = array_sum(array_map(function($v) { return $v['score']; }, $segmentsinfo));
return $totalscore;
}
But how can I compare in a SELECT query or any other way?
You can use like queries for that:
Following example will return all the records from table customer for which customer name ends with kh
select * from customer where name like '%kh'
Following example will return all the records from table customer for which customer name start with kh
select * from customer where name like 'kh%'
Following example will return all the records from table customer for which the middle world of customer name is kh
select * from customer where name like 'kh%'
if you want more specific record then add some and/or condition in your query
I recommend you to read this
http://dev.mysql.com/doc/refman/5.7/en/string-comparison-functions.html#operator_like
I think you might need to define how similar things need to be to be considered a match.
But if you just wanna search for containing words, you could split your search string by whitespaces and use it in a REGEXP in your query
$search_array = explode(" ", "Toy story");
$query = "SELECT title, year, poster FROM movies WHERE title REGEXP '".implode("|", $search_array)."'";
This would probably match a lot rows, but you could make a more restrictive regular expression.
I am trying the use refine tools for a search on my website. The bit i'm stuck with is search by start letter. For example i could use a wildcard '%X%' but his would return anything that contained the letter 'x'.
I read on few sites that SUBSTRING can be used in mysql queries
http://dev.mysql.com/
http://www.kirupa.com/
https://stackoverflow.com/questions/6302027
This is what I have so far but returns nothing. There is data in the database that should return with the query.
public function refineUsersFollowers($user_id,$q){
if($this->databaseConnection()){
// get the users followers
$state = array(1,2);
$stmt = $this->db_connection->prepare("SELECT * FROM friends WHERE id_2 = :1 AND Friend_Request_State = :2 OR id_2 = :3 AND Friend_Request_State = :4");
$stmt->bindParam(':1', $user_id);
$stmt->bindParam(':2', $state[0]);
$stmt->bindParam(':3', $user_id);
$stmt->bindParam(':4', $state[1]);
$stmt->execute();
// format the SQL OR statements
$sql = '';
$ids = [];
while($rows = $stmt->fetch(\PDO::FETCH_ASSOC)){
array_push($ids,$rows['id_1']);
}
for($x = 0; $x < count($ids); $x++){
if(count($ids) == 1){
//if there is one result
$sql.= ' user_id = :'.$x." AND SUBSTRING('first_name',0,1) = :".$x.$x;
}else if($x == (count($ids) - 1)){
// last entry
$sql.= ' user_id = :'.$x." AND SUBSTRING('first_name',0,1) = :".$x.$x;
}else{
//continue loop
$sql.= ' user_id = :'.$x." AND SUBSTRING('first_name',0,1) = :".$x.$x." OR";
}
}
$stmt = $this->db_connection->prepare("SELECT * FROM account WHERE ".$sql);
for($x = 0; $x < count($ids); $x++){
$stmt->bindParam(':'.$x,$ids[$x]);
$insert = $x.$x.'';
$stmt->bindParam(':'.$insert,$q);
}
$stmt->execute();
$results = $stmt->fetch(\PDO::FETCH_ASSOC);
print_r($results);
// check for followers that start with letter
}
}
The first part of the function is fine, this gets an array of id's which is then placed together as an SQL string. Is the SQL not returning results because SUBSTRING is not supported in this way?
If so is there a way of producing a query like this or would it be easier to pull every result from the database then check them in a different function?
You have two issues with this expression:
SUBSTRING('first_name', 0, 1) = :".$x.$x;
First, substr() in SQL (in general) starts counting with 1 and not 0. So, the first argument should be 1.
Second, you have the first argument in single quotes. So, at best, this would return the letter 'f'. Here is a simple rule: Only use single quotes for string and date constants. Never use single quotes to refer to column names.
There are several way to write what you want. Here are three:
SUBSTRING(first_name, 1, 1) = $x
LEFT(first_name, 1) = $x
first_name like '$x%'
You query can be greatly simplified with the LIKE operator. This:
"AND SUBSTRING('first_name',0,1) = :".$x.$x;
can become this:
"AND first_name LIKE '".$x.$x."%'";
I'm not sure what the $x.$x is for, so I just left it in for illustrative purposes.
a little help on this one, here are its details
[Products]
id int
name text
category
color
Problem is the values of the color field, sample values are:
GOLDRED
GOLD-RED
GOLD/RED
BLUE/GREEN-RED
WHITE GOLD-YELLOW/ORANGE
I could very much clean the search query such as this sample using a basic function
"select * from products where color=".cleanstring($stringval)." limit 1";
function cleanstring($var) {
$newtext = $var;
$newtext = preg_replace("/[^a-zA-Z0-9\s]/", "", $newtext);
$newtext = str_replace(" ", "", $newtext);
$newtext = strtoupper($newtext);
return $newtext;
}
The problem is with the content. It's thousands of records without any form of standard in using a naming convention.
I want to select those records with its values clean similar to my cleanstring().
Example:
Query = GOLDRED
Can select
GOLD-RED
GOLD RED
GOLDRED
GOLD/RED
GOLDRED
Any solution that you could recommend? Code is in PHP/MySQL.
"select * from products where 1".cleanstring($stringval);
function cleanstring($var) {
$color_list = array('GOLD','RED','GREEN','WHITE');
$sql_where='';
foreach( $color_list AS $v){
if(strpos($var, $v)!==false){
$sql_where .=" AND color LIKE '%{$v}%'";
}
}
return $sql_where;
}
//select * from products where 1 OR color LIKE '%GOLD%' OR color LIKE '%RED%'
REMARK:
input: GOLDRED ,
match: GOLD RED,GOLD-RED,GOLD/RED..... GOLD/RED/ABC,RED_GOLDGREEN,
may be after get all data , then make func ranking by match % ,like search engine
Probably You could make just a MySQL regexp with 'GOLD.?RED' or 'GOLD(-|[[:space:]])?RED' ?
That's an online example I made : http://regexr.com?34mmg
Not the best way, and I am sure has tons of downfalls, but if I did not make any mistakes in php code (don't have machine to try it out), it would do the work:
"select * from products where color REGEXP '".cleanstring($stringval)."' limit 1";
function cleanstring($var) {
$var = preg_replace('![-\/ ]+!', '', $var);
$strLength = strlen($var);
$parts = array();
for ($i = 1; $i <= $strLength; i++) {
$parts[] = ($i > 0) ? substr($var, 0, $i).'[-/ ]?'.substr($var, $i);
}
return "[[:<:]](".implode('|', $parts).")[[:>:]]";
}
It would output something like this:
"select * from products where color REGEXP '[[:<:]](G[-/ ]?OLDRED|GO[-/ ]?LDRED|GOL[-/ ]?DRED|GOLD[-/ ]?RED|GOLDR[-/ ]?ED|GOLDRE[-/ ]?D)[[:>:]]' limit 1"
which basically breaks your keyword in pieces letter by letter, i.e.
G OLDRED
GO LDRED
GOL DRED
GOLD RED
GOLDR ED
GOLDRE D
and do the "LIKE" statement on them but with smarter word boundaries and instead of just space, it considers "-" and "/" as well.
I'm creating a very simple PHP file that searches a table in my SQLite database.
I want it to filter my table according to 2 parameters, min and max, but I want to be able to specify one parameter, both or even none. How do I translate that into a SQL query?
<?php
$db = new PDO('sqlite:database.db');
$min = $_GET['min'];
$max = $_GET['max'];
if($min == '') $min = -1;
if($max == '') $max = -1;
$res = $db->prepare('SELECT * FROM Roteiro WHERE ...');
$res->execute(array(...));
$test = $res->fetchAll();
print_r($test);
?>
where (? > a or ? is null)
and (? = foo or ? is null)
-- etc...
The trick is let the condition be satisfied if the value is null. Make sure you pass type null, not some empty string or other falsy value.
The query optimizer should optimize these extra conditions away no problem because its easy for it to see they never vary per row.
an equivalent way to say it
where (? is not null and ? > a)
and (? is not null and ? = foo)
but I prefer the first way.
Youll need to build the statement dynamically:
$db = new PDO('sqlite:database.db');
$min = isset($_GET['min']) && !empty($_GET['min']) ? (integer) $_GET['min'] : null;
$max = isset($_GET['max']) && !empty($_GET['max']) ? (integer) $_GET['max'] : null;
$base = 'SELECT * FROM Roteiro';
$params = array();
if(null != $min && null != $max) {
// im using BETWEEN here just for simplicity int he example,
// but you could (should?) use separate statements with
// typical <,> comparison ops.
$base .= ' WHERE the_column BETWEEN :min AND :max';
$params[':min'] = $min;
$params[':max'] = $max;
} elseif(null !== $max) {
$base .= ' WHERE the_column < :max';
$params[':max'] = $max;
} elseif(null != $min) {
$base .= ' WHERE the_column > :min';
$params[':min'] = $min;
}
$db->prepare($base);
$db->execute($params);
You'll want to dynamically create the query from the information you have.
So, lets say you want to filter some table by a optional minimal and an optional maximal value.
SELECT some_row
FROM some_table
WHERE filter_row < [max]
AND filter_row > [min]
An easy way to do this would be to simply store the part of your SQL-query that filters by maximum and minimum and append it to the query when needed/defined:
$sql = "SELECT some_row
FROM some_table ";
if (isset($max) && isset($min)){
$sql .= "WHERE filter_row < ?
AND filter_row > ?";
} else if (isset($max) ){
$sql .= "WHERE filter_row < ?";
} else if (isset($min) ){
$sql .= "WHERE filter_row > ?";
}
if ($stmt = $mysqli->prepare($sql)) {
// bind parameters according to the given/available
// parameters and execute
}
I'm not sure if the query will work since I have nothing to test it here, but the idea should be clear.
This is just a very simple solution which will get very large and ugly with a growing list of optional parameters. So, if you need it more often and with more parameters, consider using a "Query Builder".