PHP and mysql Ignoring the special chracters present in the database - php

a little help on this one, here are its details
[Products]
id int
name text
category
color
Problem is the values of the color field, sample values are:
GOLDRED
GOLD-RED
GOLD/RED
BLUE/GREEN-RED
WHITE GOLD-YELLOW/ORANGE
I could very much clean the search query such as this sample using a basic function
"select * from products where color=".cleanstring($stringval)." limit 1";
function cleanstring($var) {
$newtext = $var;
$newtext = preg_replace("/[^a-zA-Z0-9\s]/", "", $newtext);
$newtext = str_replace(" ", "", $newtext);
$newtext = strtoupper($newtext);
return $newtext;
}
The problem is with the content. It's thousands of records without any form of standard in using a naming convention.
I want to select those records with its values clean similar to my cleanstring().
Example:
Query = GOLDRED
Can select
GOLD-RED
GOLD RED
GOLDRED
GOLD/RED
GOLDRED
Any solution that you could recommend? Code is in PHP/MySQL.

"select * from products where 1".cleanstring($stringval);
function cleanstring($var) {
$color_list = array('GOLD','RED','GREEN','WHITE');
$sql_where='';
foreach( $color_list AS $v){
if(strpos($var, $v)!==false){
$sql_where .=" AND color LIKE '%{$v}%'";
}
}
return $sql_where;
}
//select * from products where 1 OR color LIKE '%GOLD%' OR color LIKE '%RED%'
REMARK:
input: GOLDRED ,
match: GOLD RED,GOLD-RED,GOLD/RED..... GOLD/RED/ABC,RED_GOLDGREEN,
may be after get all data , then make func ranking by match % ,like search engine

Probably You could make just a MySQL regexp with 'GOLD.?RED' or 'GOLD(-|[[:space:]])?RED' ?
That's an online example I made : http://regexr.com?34mmg

Not the best way, and I am sure has tons of downfalls, but if I did not make any mistakes in php code (don't have machine to try it out), it would do the work:
"select * from products where color REGEXP '".cleanstring($stringval)."' limit 1";
function cleanstring($var) {
$var = preg_replace('![-\/ ]+!', '', $var);
$strLength = strlen($var);
$parts = array();
for ($i = 1; $i <= $strLength; i++) {
$parts[] = ($i > 0) ? substr($var, 0, $i).'[-/ ]?'.substr($var, $i);
}
return "[[:<:]](".implode('|', $parts).")[[:>:]]";
}
It would output something like this:
"select * from products where color REGEXP '[[:<:]](G[-/ ]?OLDRED|GO[-/ ]?LDRED|GOL[-/ ]?DRED|GOLD[-/ ]?RED|GOLDR[-/ ]?ED|GOLDRE[-/ ]?D)[[:>:]]' limit 1"
which basically breaks your keyword in pieces letter by letter, i.e.
G OLDRED
GO LDRED
GOL DRED
GOLD RED
GOLDR ED
GOLDRE D
and do the "LIKE" statement on them but with smarter word boundaries and instead of just space, it considers "-" and "/" as well.

Related

How can I write a query to select similar titles?

I would like to select those movies which have similar titles.
I found this, but this way it dosn't work, it gives nothing. I would like to give toy story 2, toy story 3 and others with similar title like toy soldielrs, etc.
$title = "Toy Story";
$query = mysql_query("SELECT title, year, poster, LEVENSHTEIN_RATIO( ".$title.", title ) as textDiff FROM movies HAVING textDiff > 60");
I can compare strings in PHP with this function:
static public function string_compare($str_a, $str_b)
{
$length = strlen($str_a);
$length_b = strlen($str_b);
$i = 0;
$segmentcount = 0;
$segmentsinfo = array();
$segment = '';
while ($i < $length)
{
$char = substr($str_a, $i, 1);
if (strpos($str_b, $char) !== FALSE)
{
$segment = $segment.$char;
if (strpos($str_b, $segment) !== FALSE)
{
$segmentpos_a = $i - strlen($segment) + 1;
$segmentpos_b = strpos($str_b, $segment);
$positiondiff = abs($segmentpos_a - $segmentpos_b);
$posfactor = ($length - $positiondiff) / $length_b;
$lengthfactor = strlen($segment)/$length;
$segmentsinfo[$segmentcount] = array( 'segment' => $segment, 'score' => ($posfactor * $lengthfactor));
}
else
{
$segment = '';
$i--;
$segmentcount++;
}
}
else
{
$segment = '';
$segmentcount++;
}
$i++;
}
// PHP 5.3 lambda in array_map
$totalscore = array_sum(array_map(function($v) { return $v['score']; }, $segmentsinfo));
return $totalscore;
}
But how can I compare in a SELECT query or any other way?
You can use like queries for that:
Following example will return all the records from table customer for which customer name ends with kh
select * from customer where name like '%kh'
Following example will return all the records from table customer for which customer name start with kh
select * from customer where name like 'kh%'
Following example will return all the records from table customer for which the middle world of customer name is kh
select * from customer where name like 'kh%'
if you want more specific record then add some and/or condition in your query
I recommend you to read this
http://dev.mysql.com/doc/refman/5.7/en/string-comparison-functions.html#operator_like
I think you might need to define how similar things need to be to be considered a match.
But if you just wanna search for containing words, you could split your search string by whitespaces and use it in a REGEXP in your query
$search_array = explode(" ", "Toy story");
$query = "SELECT title, year, poster FROM movies WHERE title REGEXP '".implode("|", $search_array)."'";
This would probably match a lot rows, but you could make a more restrictive regular expression.

Can i select data from mysqli database using SUBSTR to refine the query

I am trying the use refine tools for a search on my website. The bit i'm stuck with is search by start letter. For example i could use a wildcard '%X%' but his would return anything that contained the letter 'x'.
I read on few sites that SUBSTRING can be used in mysql queries
http://dev.mysql.com/
http://www.kirupa.com/
https://stackoverflow.com/questions/6302027
This is what I have so far but returns nothing. There is data in the database that should return with the query.
public function refineUsersFollowers($user_id,$q){
if($this->databaseConnection()){
// get the users followers
$state = array(1,2);
$stmt = $this->db_connection->prepare("SELECT * FROM friends WHERE id_2 = :1 AND Friend_Request_State = :2 OR id_2 = :3 AND Friend_Request_State = :4");
$stmt->bindParam(':1', $user_id);
$stmt->bindParam(':2', $state[0]);
$stmt->bindParam(':3', $user_id);
$stmt->bindParam(':4', $state[1]);
$stmt->execute();
// format the SQL OR statements
$sql = '';
$ids = [];
while($rows = $stmt->fetch(\PDO::FETCH_ASSOC)){
array_push($ids,$rows['id_1']);
}
for($x = 0; $x < count($ids); $x++){
if(count($ids) == 1){
//if there is one result
$sql.= ' user_id = :'.$x." AND SUBSTRING('first_name',0,1) = :".$x.$x;
}else if($x == (count($ids) - 1)){
// last entry
$sql.= ' user_id = :'.$x." AND SUBSTRING('first_name',0,1) = :".$x.$x;
}else{
//continue loop
$sql.= ' user_id = :'.$x." AND SUBSTRING('first_name',0,1) = :".$x.$x." OR";
}
}
$stmt = $this->db_connection->prepare("SELECT * FROM account WHERE ".$sql);
for($x = 0; $x < count($ids); $x++){
$stmt->bindParam(':'.$x,$ids[$x]);
$insert = $x.$x.'';
$stmt->bindParam(':'.$insert,$q);
}
$stmt->execute();
$results = $stmt->fetch(\PDO::FETCH_ASSOC);
print_r($results);
// check for followers that start with letter
}
}
The first part of the function is fine, this gets an array of id's which is then placed together as an SQL string. Is the SQL not returning results because SUBSTRING is not supported in this way?
If so is there a way of producing a query like this or would it be easier to pull every result from the database then check them in a different function?
You have two issues with this expression:
SUBSTRING('first_name', 0, 1) = :".$x.$x;
First, substr() in SQL (in general) starts counting with 1 and not 0. So, the first argument should be 1.
Second, you have the first argument in single quotes. So, at best, this would return the letter 'f'. Here is a simple rule: Only use single quotes for string and date constants. Never use single quotes to refer to column names.
There are several way to write what you want. Here are three:
SUBSTRING(first_name, 1, 1) = $x
LEFT(first_name, 1) = $x
first_name like '$x%'
You query can be greatly simplified with the LIKE operator. This:
"AND SUBSTRING('first_name',0,1) = :".$x.$x;
can become this:
"AND first_name LIKE '".$x.$x."%'";
I'm not sure what the $x.$x is for, so I just left it in for illustrative purposes.

PHP/MySQL Limiting characters outputted from a field

When I get a database array, sometimes there is fields with too much data for my results list page, which is suppose to give just a short description. How do I limit the characters count to something like 100.
This is my array for the loop:
<?php
$i = 1;
while ($row = mysql_fetch_array($result)) {
?>
This is my echo statement:
<?php echo $row['description']; ?>
You can use substr like this:
<?php echo substr($row['description'], 0, 100); ?>
But it might be better—depending on your application needs—to do the limiting when making the initial MySQL query using SUBSTR which behaves the same way, but in MySQL.
SELECT SUBSTR(example_field, 1, 100)
FROM example_table
WHERE example_field IS NOT NULL
LIMIT 1
;
That MySQL script basically means return the substring of example_field starting from the first (1) character and going 100 characters in.
This might be better in some cases since if you are limiting text length to 100 characters, why grab the data for fields that might have 1,000+ characters? That would definitely bog down your PHP script in many cases. Handling that in MySQL is the best way to nip it in the bud if your app just needs 100 characters returned.
You can try with substr()
<?php echo substr($row['description'],0,100); ?>
OR
function truncate($input, $maxWords, $maxChars)
{
$words = preg_split('/\s+/', $input);
$words = array_slice($words, 0, $maxWords);
$words = array_reverse($words);
$chars = 0;
$truncated = array();
while(count($words) > 0)
{
$fragment = trim(array_pop($words));
$chars += strlen($fragment);
if($chars > $maxChars) break;
$truncated[] = $fragment;
}
$result = implode($truncated, ' ');
return $result . ($input == $result ? '' : '...');
}
// try with cuctom function truncate() , it help to cut description by words.
<?php echo truncate($row['description'],5,200); ?>
Like the other answers, you should use substr, but you can use it in combination with strpos so that when you shorten the string, you stop after a complete word instead of interrupting a word itself.
$pos = strpos($row['description'], ' ', 100);
echo substr($row['description'], $pos);
I would do it in the SQL query. It's better to limit the results returned which reduces I/O and network throughput by not returning data from the database you're not going to use. In your SQL query do:
SELECT LEFT(description, 100) AS description
,....
FROM ...
Just in case someone is interested in another code snippets on how to limit character output with dots, I use this and it works well for me
CONCAT(SUBSTR(<column_name_here>, 1, 100),'...... Read More') AS Column_Name

How can I remove everything past "LIMIT" from a query? (details inside)

I got a few queries built dynamically by my scripts. They usually fit the following template:
SELECT ...
FROM ...
JOIN ...
WHERE ... (lots of filters, search conditions, etc. here) ...
ORDER BY (optional) ...
LIMIT (optional) ... OFFSET (optional)
I want to remove the LIMIT and OFFSET parts from the query. I used
$sql_entitati = implode("LIMIT", explode("LIMIT", $sql_entitati, -1));
to do it but then it hit me: what if there's no LIMIT in the query and what if the only LIMIT is somewhere in the where clauses?
So my question to you is: How can I safely remove everything after the LIMIT key word, without screwing it up if there's no LIMIT and/or there's a "LIMIT" somewhere in the where clause? All this done in php.
A bit of an edit for clarity:
the algorithm i use:
$sql = implode("LIMIT", explode("LIMIT", $sql, -1));
Will work on 99% of the cases. The problem occurs when the "LIMIT" key word at the end is missing, AND there is "LIMIT" written somewhere in the conditions. for example:
SELECT * FROM table WHERE bla = 'SPEED LIMIT' ORDER BY table.a
this is the problem i need to tackle.
Solved using the following algorithm (Credit to techfoobar):
$p = strrpos($sql, "LIMIT");
if($p !== false) {
$q = strpos($sql, ")", $p);
$r = strpos($sql, "'", $p);
$s = strpos($sql, "\"", $p);
if($q === false && $r === false && $s === false)
$sql = substr($sql, 0, $p);
}
You should do something like:
Get the position of the last "LIMIT" - store this position in say, p
Ensure that you do not have a ")" character after p - if so, it is part of some inner query inside a condition etc..
Ensure that you do not have a "'" character after p - if so, it is part of some user input string
If steps 2 and 3 are passed, strip off everything after p
More hints:
For step 1 - use strrpos for the last occurrence of LIMIT
For steps 2 and 3, use strpos with p as the search start offset
For step4, use substr to strip off everything after p
Get the position of LIMIT, check it exists with !==FALSE then substring.
$str = "SELECT X FROM Y JOIN WHERE CONDITIONS ORDER BY ORDERS LIMIT OFFSET";
$pos = strrpos($str,"LIMIT");
if ($pos!==false)
{
$newStr = substr($str,0,$pos);
echo $newStr;
}
Try this aproach
$query = '';
$query .= 'SELECT...';
$query .= 'FROM ...';
$query .= 'JOIN ...';
$query .= 'WHERE...';
if ($limit)
{
$query .= " LIMIT $limit";
}
This may help you...
$str = " SELECT * FROM table WHERE bla = 'SPEED LIMIT' ORDER BY table.a";
$pos = strrpos($str,"LIMIT");
if ($pos!==false)
{
$ok = 1;
if(substr_count($str, '"', $pos) % 2 != 0)
{
$ok = 0;
}
if(substr_count($str, "'", $pos) % 2 != 0)
{
$ok = 0;
}
if(substr_count($str, ")", $pos) > 0)
{
$ok = 0;
}
if($ok == 1)
$str = substr($str,0,$pos);
}
echo $str;

Improve search result by broadening matches

I am trying to improve search result using PHP. e.g. when product name is ZSX-3185BC user can not find the product when put something like ZSX3185BC. So far i get something like:
$input = '01.10-2010';
$to_replace = array('.','-');
$clean = str_replace($to_replace, '/',$input);`
I've tried this, but it does not work correctly (not sure, posted in comment without elaboration -ed)
$str = "";
$len = strlen($strSearch) - 1;
// Loop through the search string to check each charecter is acceptable and strip those that are not
for ($i = 0; $i <= $len; $i++) {
if(preg_match("/\w|-| /",$strSearch[$i])) {
$str = $str . $strSearch[$i];
}
}
You should try to do this kind of fuzzy matching directly in your database query. A common approach to fuzzy string matching is the Levenshtein distance.
MySQL does not have this built in, however, it can be added. See this SO question.
The basic way is to add a stored function to MySQL. There is a Levenshtein implementation as a stored function.
Try this :
SELECT * FROM tablename WHERE replace(fieldname, '-', '') like '%ZSX3185BC%';

Categories