Query with a PHP foreach using OR and LIKE %? - php

I have sort of a tricky problem, and I want to see if there is an easier way about solving it.
I have let's say
$numbers= $_GET['numbers']; //resulting in $numbers=array(1,2,3)
The count in the array may vary depending on the $_GET value...
These need to search using '%" . $number. "%' because there might be more than one number in the row separated by commas
My ideal result would perform a search for the $numbers(1,2,3) with:
SELECT * FROM database WHERE numbers LIKE('%1%' OR '%2%' OR '%2%')

No need to inject many OR operators for each element of the Array. Instead you can use rlike (the regex matcher operator of MySQL) and simplify your code to a great extent like this:
$numbers = array(1,2,3);
$sql = 'SELECT * from `database` WHERE number rlike "^'.implode('|', $numbers).'$"';
// $sql becomes: SELECT * from `database` WHERE number rlike "^1|2|3$"

$like_part = implode(' OR ', array_map(function($i) {
return "(_utf8 '%{$i}%' USING latin1)";
}, $numbers)
);

Use the foreach to build the where string, then add it to the query.
$query = '...';
$where = '';
foreach ($numbers as $number) {
$where .= ...
}
$query .= $where;
I won't mention that you should normalize your tables and use prepared queries.

This code:
<?php
$numbers = array(1,2,3);
$sql = "SELECT * FROM table WHERE numbers LIKE ('%" . implode("%' OR '%", $numbers) . "%')";
Resulting this query:
SELECT * FROM table WHERE numbers LIKE ('%1%' OR '%2%' OR '%3%')

to the effect of:
$query = "select * from table where condition like ('text before '".implode("' text after text before'", $array)." ' text after')"
$query = "select * from table where numbers like (\"_utf8 '%" . implode("%' using latin1\", $_GET['numbers']) . "%' using latin1 OR _utf8 '%\")
[not sure if implode is needle/haystack or vice versa]

1 - SELECT * FROM b AS B INNER JOIN a AS A ON A.a=B.a WHERE A.a RLIKE '^1-|^2-|^3'
2 - SELECT * FROM b AS B INNER JOIN a AS A ON A.a=B.a WHERE (A.a LIKE '1-%' OR A.a LIKE '2-%' OR A.a LIKE '3-%')
First query took around
Memory usage = It is taking time
Execution time = It is taking time
PHPMYADMIN Showing rows 0 - 24 (138 total, Query took 1.1243 seconds.) - Tested with very less data than through direct PHP script
Second query took around
Memory usage = 1198.0807113647 MB
Execution time = 30.719851970673 Seconds
PHPMYADMIN - Showing rows 0 - 24 (138 total, Query took 0.0018 seconds.) - Tested with very less data than through direct PHP script
Foreach took around
Memory usage = 972.3137588501 MB
Execution time = 10.898797988892 Seconds
Configuration
32GB RAM
Intel I9 - 10th gen CPU#2.81GHz
PHP 7.3
MYSQL - 5.7.31
Table size - 2M million data generated by SQL procedure
SHOW VARIABLES LIKE 'have_query_cache'; // YES
Fetched data size 275093
Indexing - Available
Count of variable in LIKE operator 4739 (i.e. 1,2,3... in RLIKE '^1-|^2-|^3....etc')
Memory uage calculated by php function memory_get_usage();
Objections welcomes.

Related

How to use LIKE and wildcards in mysql statements

I've researched this but couldn't find a solution for my specific problem.
I have a column containing data in a certain format. Here are some examples:
1
6
14
1;6;14;16
etc...
I need a mysql statement which for example it will select all columns where 16 occurs.
I've tried this but it's also selecting columns where 1 and 6 occur:
"SELECT * FROM tbl WHERE kategorien LIKE '%".$_GET['katid']."%' AND status = 1 ORDER BY pos ASC"
Thanks in advance for any help!
You can try creating a helper function like this:
// Helper function
function getLike($str, $deliminator = ';', $field = 'kategorien') {
if (false !== strpos($str, $deliminator)) {
$strParts = explode($deliminator, $str);
return "($field LIKE '%". implode("%' OR $field LIKE '%", $strParts) . "%')";
} else {
return "$field LIKE '%$str%'";
}
}
// Debug
var_dump(getLike('1;6;14;16'));
Outputs:
string '(kategorien LIKE '%1%' OR kategorien LIKE '%6%' OR kategorien LIKE '%14%' OR kategorien LIKE '%16%')' (length=100)
In your query, you'd use it like this:
"SELECT * FROM tbl WHERE ". getLike($_GET['katid']) ." AND status = 1 ORDER BY pos ASC"
You could use MySQL function FIND_IN_SET:
SELECT * FROM tbl
WHERE
FIND_IN_SET('16', REPLACE(kategorien, ';', ','))>0
however, it is usually not a good idea to store comma separated values in a single field, please have a look at this question: Is storing a delimited list in a database column really that bad?

MySQL Search by Relevance of all fields

I am sure this is possible but I think it maybe just very complex to write. I want to search every field by:
='SearchTerm'
then
Like %SearchTerm
then
like SearchTerm%
and finally
like %SearchTerm%. I want to run this on every field in my table which there is around 30 or 40. Is there an easy way to run this over multiple fields or will I have to declare every single one?
I think I have seen a query before where different matches between %query %query% etc are ranked by assigning an integer value and then ordering by this. Would that be possible on a query like this?
Any advice and help in the right direction is much appreciated.
You should use fulltext indexing on the fields you want searched and use MATCH AGAINST instead of LIKE %%. It's much faster and returns results based on relevancy. More info here:
Mysql match...against vs. simple like "%term%"
I do something very similar to what you're describing (in php and mysql)
Here's my code:
$search = trim($_GET["search"]);
$searches = explode(" ",$search);
$sql = "SELECT *,wordmatch+descmatch+usagematch+bymatch as `match` FROM (SELECT id,word,LEFT(description,100)as description,
IFNULL((SELECT sum(vote)
FROM vote v
WHERE v.definition_id = d.id),0) as votecount,
";
$sqlword = "";
$sqldesc = "";
$sqlusage = "";
$sqlby = "";
foreach ($searches as $value) {
$value = mysqli_real_escape_string($con,$value);
$sqlword = $sqlword . "+ IFNULL(ROUND((LENGTH(word) - LENGTH(REPLACE(UPPER(word), UPPER('$value'), '')))/LENGTH('$value')),0)";
$sqldesc = $sqldesc . "+ IFNULL(ROUND((LENGTH(description) - LENGTH(REPLACE(UPPER(description), UPPER('$value'), '')))/LENGTH('$value')),0)";
$sqlusage = $sqlusage . "+ IFNULL(ROUND((LENGTH(`usage`) - LENGTH(REPLACE(UPPER(`usage`), UPPER('$value'), '')))/LENGTH('$value')),0)";
$sqlby = $sqlby . "+ IFNULL(ROUND((LENGTH(`by`) - LENGTH(REPLACE(UPPER(`by`), UPPER('$value'), '')))/LENGTH('$value')),0)";
}
$sql = $sql . $sqlword ." as wordmatch,"
. $sqldesc ." as descmatch,"
. $sqlusage ." as usagematch,"
. $sqlby ." as bymatch
FROM definition d
HAVING (wordmatch > 0 OR descmatch > 0 OR usagematch > 0 OR bymatch > 0)
ORDER BY
wordmatch DESC,
descmatch DESC,
usagematch DESC,
bymatch DESC,
votecount DESC)T1";
$queries[] = $sql;
$result = mysqli_query($con,$sql);
You can see this at work http://unurbandictionary.comule.com/view_search.php?search=George+Miley+Cyrus this is when I search for "George Miley Cyrus"
What it does is it explodes the search string to find each word and returns the number of occurences of each word in each of my column, and then i do an ORDER BY to have relevance (priority) to come back first. So in my case word field has the highest relevance, then description field, then usage field, then by field.
Before this version of my code I was using LIKE but it didn't give me a count of occurences, since I want the row with the most occurences of my search word to return first before other rows.
You should really have some sort of id to select the rows in your table.
You should have put a column with
id INT NOT NULL AUTO_INCREMENT PRIMARY KEY
Then you could use
SELECT * FROM table WHERE column1 LIKE "%SearchTerm%" AND id BETWEEN 1 AND 40

SQL query results returned, even if exact match not found

I hope this question isn't redundant. What I am trying to accomplish is have a user select a bunch of checkboxes on a page and return the closest matching records if there are no matching rows. For example:
A person checks off [x]Apples [x]Oranges [x]Pears [x]Bananas
But the table looks like this:
Apples Oranges Pears Bananas
1 1 1 null
1 1 null 1
1 1 null null
(Obviously I missed the id column here, but you get the point I think.) So, the desired result is to have those three rows still be returned in order of most matches, so pretty much the order they are in now. I'm just not sure what the best approach to take on something like this. I've considered a full text search, the levenshtein function, but I really like the idea of returning the exact match if it exists. No need for you to go at length with code if not needed. I'm just hoping to be sent in the right direction. I HAVE seen other questions sort of like this, but I still am unsure about which way to go.
Thanks!
Write a query that adds up the number of columns that matched, and sorts the rows by this total. E.g.
SELECT *
FROM mytable
ORDER BY COALESCE(Apples, 0) = $apples + COALESCE(Oranges, 0) = $oranges + ... DESC
It's easy to sort by a score...
SELECT fb.ID, fb.Apples, fb.Oranges, fb.Pears, fb.Bananas
FROM FruitBasket fb
ORDER BY
CASE WHEN #Apples = fb.Apples THEN 1 ELSE 0 END
+ CASE WHEN #Oranges = fb.Oranges THEN 1 ELSE 0 END
+ CASE WHEN #Pears = fb.Pears THEN 1 ELSE 0 END
+ CASE WHEN #Bananas = fb.Bananas THEN 1 ELSE 0 END
DESC, ID
However, this leads to a table-scan (even with TOP). The last record may be a better match than the records found so far, so every record must be read.
You could consider a tagging system, like this
Content --< ContentTag >-- Tag
Which would be queried this way:
SELECT ContentID
FROM ContentTag
WHERE TagID in (334, 338, 342)
GROUP BY ContentID
ORDER BY COUNT(DISTINCT TagID) desc
An index on ContentTag.TagId would be used by this query.
This is fairly simple, but you can just use IFNULL() (MySQL, or your DB's equivalent) to return a sum of matches and use that in your ORDER BY
// columns and weighting score
$types = array("oranges"=>1, "apples"=>1, "bananas"=>1, "pears"=>1);
$where = array();
// loop through the columns
foreach ($types as $key=>&$weight){
// if there is a match in $_REQUEST at it to $where and increase the weight
if (isset($_REQUEST[$key])){
$where[] = $key . " = 1";
$weight = 2;
}
}
// build the WHERE clause
$where_str = (count($where)>0)? "WHERE " . implode(" OR ", $where) : "";
// build the SQL - non-null matches from the WHERE will be weighted higher
$sql = "SELECT apples, oranges, pears, bananas, ";
foreach ($types as $key=>$weight){
$sql .= "IFNULL({$key}, 0, {$weight}) + ";
}
$sql .= "0 AS score FROM `table` {$where_str} ORDER BY score DESC";
Assuming that "oranges" and "apples" are selection, your SQL will be:
SELECT apples, oranges, pears, bananas,
IFNULL(apples, 0, 2) + IFNULL(oranges, 0, 2) + IFNULL(pears, 0, 1) + IFNULL(bananas, 0, 1) + 0 AS score
FROM `table`
WHERE oranges = 1 OR apples = 1
ORDER BY score DESC
Order descending by the sum of checkbox/data matches
SELECT * FROM table
ORDER BY (COALESE(Apple,0) * #apple) + (COALESE(Orange,0) * #orange) ..... DESC
where #apple / #orange represents users selection: 1 = checked, 0 = unchecked

mysql + php: Selecting multiple random results

I've been looking for this for a while but with no success.
I am trying to implement a recomendation bar, for example like in youtube, when you are seeing a video it shows the list or recommended videos on the right.
At this moment I am using this method:
$offset_result = mysql_query( " SELECT FLOOR(RAND() * COUNT(*)) AS `offset` FROM `$tablename` ");
$offset_row = mysql_fetch_object($offset_result );
$offset = $offset_row->offset;
$result_rand = mysql_query( " SELECT * FROM `$tablename` LIMIT $offset, 9 " );
This works fine, but sometimes doesn't show any result, and the problem is also that its not completely random, because it shows for example the first ID as 200, so the next result will be id 201 and then 202 and so.
I would like to know if there is a way to show this 9 randon results, for example 1º result id 500, 2º result id 10, 3º result id 788, etc etc?
Thank you
Not entirely sure this answers what you are looking for, but try:
$result_rand = mysql_query("SELECT * FROM " . $tablename . " ORDER BY RAND() LIMIT 9");
You can use php rand() function to create 5 numbers and save them in an array:
http://php.net/manual/en/function.rand.php
<?php
$rand_array = array();
for($i=1;$i<5;$i++) {
$rand_array[$i] = rand(0,500);
}
?>
and after that create a query with every int with a foreach loop and work with your data.
<?php
foreach ($rand_array as $integer) {
$q = "SELECT * from $tablename WHERE id='$integer';";
}
?>
Does this helps?
First you should use mysqli_ functions instead of mysql_ because the latter is deprecated. Second use order by rand() to get random rows:
$rand_result = mysqli_query( "SELECT * FROM $tablename ORDER BY RAND() LIMIT 9;" );
UNTESTED:
SELECT id, #rownum:=#rownum+1 AS rownum, name
FROM users u,
(SELECT #rownum:=0) r
THis will give a unique number to each row in sequence. Now if you create a temp table with 9 random numbers between 1 and count(*) of your table and JOIN those two together...
Not sure about performance but seems like it might be faster than Rand and order by since I only need 9 random numbers

while (mysql_fetch_array) in a while loop

i have this code:
while ($sum<16 || $sum>18){
$totala = 0;
$totalb = 0;
$totalc = 0;
$ranka = mysql_query("SELECT duration FROM table WHERE rank=1 ORDER BY rand() LIMIT 1");
$rankb = mysql_query("SELECT duration FROM table WHERE rank=2 ORDER BY rand() LIMIT 1");
$rankc = mysql_query("SELECT duration FROM table WHERE rank=3 ORDER BY rand() LIMIT 1");
while ($rowa = mysql_fetch_array($ranka)) {
echo $rowa['duration'] . "<br/>";
$totala = $totala + $rowa['duration'];
}
while ($rowb = mysql_fetch_array($rankb)) {
$totalb = $totalb + $rowb['duration'];
}
while ($rowc = mysql_fetch_array($rankc)) {
$totalc = $totalc + $rowc['duration'];
}
$sum=$totala+$totalb+$totalc;
}
echo $sum;
It works fine, But the problem is until "$sum=16" the "echo $rowa['duration']" executes, the question is, is there a away to "echo" only the latest executed code in the "while ($rowa = mysql_fetch_array($ranka))" i this while loop?
Because most of the times returns all the numbers until the "$sum=16"
You are explicitly echoing the $rowa['duration'] in the first inner while loop. If you only want to print the last duration from the $ranka set, simple change the echo to $rowa_duration = $rowa['duration'] then echo it outside the loop.
while ($rowa = mysql_fetch_array($ranka)) {
$rowa_duration = $rowa['duration'];
$totala = $totala + $rowa['duration'];
}
echo $rowa_duration . '<br/>';
What you are doing there is bad on multiple levels. And your english horrid. Well .. practice makes perfect. You could try joining ##php chat room on FreeNode server. That would improve both your english and php skills .. it sure helped me a lot. Anyway ..
The SQL
First of all, to use ORDER BY RAND() is extremely ignorant (at best). As your tables begin the get larger, this operation will make your queries slower. It has n * log2(n) complexity, which means that selecting querying table with 1000 entries will take ~3000 times longer then querying table with 10 entries.
To learn more about it , you should read this blog post, but as for your current queries , the solution would look like:
SELECT duration
FROM table
JOIN (SELECT CEIL(RAND()*(SELECT MAX(id) FROM table)) AS id) as choice
WHERE
table.id >= choice.id
rank = 1
LIMIT 1
This would select random duration from the table.
But since you you are actually selecting data with 3 different ranks ( 1, 2 and 3 ), it would make sense to create a UNION of three queries :
SELECT duration
FROM table
JOIN (SELECT CEIL(RAND()*(SELECT MAX(id) FROM table)) AS id) as choice
WHERE
table.id >= choice.id
rank = 1
LIMIT 1
UNION ALL
SELECT duration
FROM table
JOIN (SELECT CEIL(RAND()*(SELECT MAX(id) FROM table)) AS id) as choice
WHERE
table.id >= choice.id
rank = 2
LIMIT 1
UNION ALL
SELECT duration
FROM table
JOIN (SELECT CEIL(RAND()*(SELECT MAX(id) FROM table)) AS id) as choice
WHERE
table.id >= choice.id
rank = 3
LIMIT 1
Look scary, but it actually will be faster then what you are currently using, and the result will be three entries from duration column.
PHP with SQL
You are still using the old mysql_* functions to access database. This form of API is more then 10 years old and should not be used, when writing new code. The old functions are not maintained (fixed and/or improved ) anymore and even community has begun the process of deprecating said functions.
Instead you should be using either PDO or MySQLi. Which one to use depends on your personal preferences and what is actually available to you. I prefer PDO (because of named parameters and support for other RDBMS), but that's somewhat subjective choice.
Other issue with you php/mysql code is that you seem to pointlessly loop thought items. Your queries have LIMIT 1, which means that there will be only one row. No point in making a loop.
There is potential for endless loop if maximum value for duration is 1. At the start of loop you will have $sum === 15 which fits the first while condition. And at the end that loop you can have $sum === 18 , which satisfies the second loop condition ... and then it is off to the infinity and your SQL server chokes.
And if you are using fractions for duration, then the total value of 3 new results needs to be even smaller. Just over 2. Start with 15.99 , ends with 18.01 (that's additional 2.02 in duration or less the 0.7 per each). Again .. endless loop.
Suggestion
Here is how i would do it:
$pdo = new PDO('mysql:dbname=my_db;host=localhost', 'username', 'password');
$pdo->setAttribute(PDO::ATTR_EMULATE_PREPARES, false);
$sum = 0;
while ( $sum < 16 )
{
$query = 'that LARGE query above';
$statement = $pdo->prepare( $query );
if ( $statement->execute() )
{
$data = $statement->fetchAll( PDO::FETCH_ASSOC );
$sum += $data[0]['duration']+$data[1]['duration']+$data[2]['duration'];
}
}
echo $data[0]['duration'];
This should do what your code did .. or at least, what i assume, was your intentions.

Categories