I am trying to query a column with createCommand doing something like this:
Yii::$app->db->createCommand('Select column1 from table where column2 in :array)
->bindValues(['array'=>['(1,2,3,4,5)', PDO::PARAM_INT]])->queryColumn('COLUMN1');
Ideally I want my SQL statement to be executed like
select column1 from table where column2 in (1,2,3,4,5)
However when the SQL executes there is always quotes wrapped around the binded parameters like so:
select column1 from table where column2 in '(1,2,3,4,5)'
I'm not sure why this is still happening after I specify the PDO to use PDO::PARAM_INT. Isn't int supposed to not be quoted?
You could try a slightly different approach using Query Builder:
$rows = (new \yii\db\Query())
->select('column1')
->from('table')
->where(['column2' => [1,2,3,4,5]])
->column();
This has the desired effect: the where clause is automatically generated using the in (...) syntax because of the array passed in.
Not sure if you explicitly need to cast the types of the array to INT if they happen to be of type STRING internally in PHP though...
Related
I want to bind keywords to a SQL query like this:
SELECT `field_1`, `field2` FROM `table` WHERE
`field_1` LIKE '%keyword1%'
OR `field_1` LIKE '%keyword2%'
OR `field_1` LIKE '%keyword3%'
Note that the number of keywords is NOT predetermined. It is in fact determined by an array which itself is determined by exploding user's input on space.
I am using Laravel 7. So, I am looking for a solution that is consistent with Laravel 7 and PDO. The problem is that I can't use '%?%' in my SQL statement because then I won't be able to bind the values in my array to it.
I am sure there should be a solution for this as it seems like a common problem in writing simple search engines, but I can't find it on my own.
Regarding your example query, you need to add the % to the value instead of the statement so it's like this:
WHERE foo LIKE ? OR foo LIKE ?
And then pass the values as "%$var1%", "%$var2%" and so on.
Using Laravels Query Builder
Make sure you have all the search words in an array. Then you can, in Laravels (which you mentioned you're using) query builder, do something like this:
$searchWords = ['val1', 'val2', 'val3'];
// Now get the query builder and pass closure as the argument
$query = DB::table('some-table')
->where(function($query) use ($searchWords) {
// Iterate through the words and add them one by one
foreach ($searchWords as $word) {
$query->orWhere('field', 'LIKE', "%{$word}%");
}
});
$result = $query->get();
Getting the Value:
I've got the levenshtein_ratio function, from here, queued up in my MySQL database. I run it in the following way:
$stmt = $db->prepare("SELECT r_id, val FROM table WHERE levenshtein_ratio(:input, someval) > 70");
$stmt->execute(array('input' => $input));
$result = $stmt->fetchAll();
if(count($result)) {
foreach($result as $row) {
$out .= $row['r_id'] . ', ' . $row['val'];
}
}
And it works a treat, exactly as expected. But I was wondering, is there a nice way to also get the value that levenshtein_ratio() calculates?
I've tried:
$stmt = $db->prepare("SELECT levenshtein_ratio(:input, someval), r_id, val FROM table WHERE levenshtein_ratio(:input, someval) > 70");
$stmt->execute(array('input' => $input));
$result = $stmt->fetchAll();
if(count($result)) {
foreach($result as $row) {
$out .= $row['r_id'] . ', ' . $row['val'] . ', ' . $row[0];
}
}
and it does technically work (I get the percentage from the $row[0]), but the query is a bit ugly, and I can't use a proper key to get the value, like I can for the other two items.
Is there a way to somehow get a nice reference for it?
I tried:
$stmt = $db->prepare("SELECT r_id, val SET output=levenshtein_ratio(:input, someval) FROM table WHERE levenshtein_ratio(:input, someval) > 70");
modelling it after something I found online, but it didn't work, and ends up ruining the whole query.
Speeding It Up:
I'm running this query for an array of values:
foreach($parent as $input){
$stmt = ...
$stmt->execute...
$result = $stmt->fetchAll();
... etc
}
But it ends up being remarkably slow. Like 20s slow, for an array of only 14 inputs and a DB with about 350 rows, which is expected to be in the 10,000's soon. I know that putting queries inside loops is naughty business, but I'm not sure how else to get around it.
EDIT 1
When I use
$stmt = $db->prepare("SELECT r_id, val SET output=levenshtein_ratio(:input, someval) FROM table WHERE levenshtein_ratio(:input, someval) > 70");
surely that's costing twice the time as if I only calculated it once? Similar to having $i < sizeof($arr); in a for loop?
To clean up the column names you can use "as" to rename the column of the function. At the same time you can speed things up by using that column name in your where clause so the function is only executed once.
$stmt = $db->prepare("SELECT r_id, levenshtein_ratio(:input, someval) AS val FROM table HAVING val > 70");
If it is still too slow you might consider a c library like https://github.com/juanmirocks/Levenshtein-MySQL-UDF
doh - forgot to switch "where" to "having", as spencer7593 noted.
I'm assuming that `someval` is an unqalified reference to a column in the table. While you may understand that without looking at the table definition, someone else reading the SQL statement can't tell. As an aid to future readers, consider qualifying your column references with the name of the table or (preferably) a short alias assigned to the table in the statement.
SELECT t.r_id
, t.val
FROM `table` t
WHERE levenshtein_ratio(:input, t.someval) > 70
That function in the WHERE clause has to be evaluated for every row in the table. There's no way to get MySQL to build an index on that. So there's no way to get MySQL to perform an index range scan operation.
It might be possible to get MySQL to use an index for the query, for example, if the query had an ORDER BY t.val clause, or if there is a "covering index" available.
But that doesn't get around the issue of needing to evaluate the function for every row. (If the query had other predicates that excluded rows, then the function wouldn't necessarily need be evaluated for the excluded rows.)
Adding the expression to the SELECT list really shouldn't be too expensive if the function is declared to be DETERMINISTIC. A second call to a DETERMINISTIC function with the same arguments can reuse the value returned for the previous execution. (Declaring a function DETERMINISTIC essentially means that the function is guaranteed to return the same result when given the same argument values. Repeated calls will return the same value. That is, the return value depends only the argument values, and doesn't depend on anything else.
SELECT t.r_id
, t.val
, levenshtein_ratio(:input, t.someval) AS lev_ratio
FROM `table` t
WHERE levenshtein_ratio(:input2, t.someval) > 70
(Note: I used a distinct bind placeholder name for the second reference because PDO doesn't handle "duplicate" bind placeholder names as we'd expect. (It's possible that this has been corrected in more recent versions of PDO. The first "fix" for the issue was an update to the documentation noting that bind placeholder names should appear only once in statement, if you needed two references to the same value, use two different placeholder names and bind the same value to both.)
If you don't want to repeat the expression, you could move the condition from the WHERE clause to the HAVING, and refer to the expression in the SELECT list by the alias assigned to the column.
SELECT t.r_id
, t.val
, levenshtein_ratio(:input, t.someval) AS lev_ratio
FROM `table` t
HAVING lev_ratio > 70
The big difference between WHERE and HAVING is that the predicates in the WHERE clause are evaluated when the rows are accessed. The HAVING clause is evaluated much later, after the rows have been accessed. (That's a brief explanation of why the HAVING clause can reference columns in the SELECT list by their alias, but the WHERE clause can't do that.)
If that's a large table, and a large number of rows are being excluded, there might be a significant performance difference using the HAVING clause.. there may be a much larger intermediate set created.
To get an "index used" for the query, a covering index is the only option I see.
ON `table` (r_id, val, someval)
With that, MySQL can satisfy the query from the index, without needing to lookup pages in the underlying table. All of the column values the query needs are available from the index.
FOLLOWUP
To get an index created, we would need to create a column, e.g.
lev_ratio_foo FLOAT
and pre-populate with the result from the function
UPDATE `table` t
SET t.lev_ratio_foo = levenshtein_ratio('foo', t.someval)
;
Then we could create an index, e.g.
... ON `table` (lev_ratio_foo, val, r_id)
And re-write the query
SELECT t.r_id
, t.val
, t.lev_ratio_foo
FROM `table` t
WHERE t.lev_ratio_foo > 70
With that query, MySQL can make use of an index range scan operation on an index with lev_ratio_foo as the leading column.
Likely, we would want to add BEFORE INSERT and BEFORE UPDATE triggers to maintain the value, when a new row is added to the table, or the value of the someval column is modified.
That pattern could be extended, additional columns could be added for values other than 'foo'. e.g. 'bar'
UPDATE `table` t
SET t.lev_ratio_bar = levenshtein_ratio('bar', t.someval)
Obviously that approach isn't going to be scalable for a broad range of input values.
I am trying to set a column result literal with the Laravel query builder. By writing raw SQL I would achieve this with:
SELECT
`field1`,
`field2`,
'Test' AS `field3`
FROM `Test`;
Therefore MySQL will always return Test for column field3. I'm trying to do this with the query builder like select('field1', 'field2', '"Test" AS field3') but this doesn't seem to work. It will return an error that Test is not a column name.
Does anyone have an idea how I could do this?
You want the DB::raw() function:
->select('field1', 'field2', DB::raw("'Test' AS field3"))
I have a stored procedure which takes in a single String parameter - the value passed into this parameter is a comma separated list of ID's from PHP - something like 2,3,4,5
`DECLARE tags_in VARCHAR(255);`
Within the Stored procedure I would like to select the rows which have ids corresponding to the ids in the parameter - the query would be like
`SELECT * from tags WHERE tag_id IN (tags_in)`
I pass in the values from PHP to MySQL using the following statement binding the value as a string
`$stmt->bindParam(':tags', '2,3,4', PDO::PARAM_STR);`
Problem - the actual query being executed by MySQL is as below - where the parameters passed in are considered as one string
`SELECT * from tags WHERE tag_id IN ('2,3,4')`
When the query I want executed is as below where the parameters are considered as individual integers
`SELECT * from tags WHERE tag_id IN (2,3,4)`
Any suggestions on I can accomplish this?
SQL placeholders can represent only SINGLE values. If you pass in some comma separated values, they won't be seen as multiple individual values with commas, they'll just be treated like a monolithic string.
e.g.
... WHERE foo IN (:bar)
... WHERE foo = :bar
are functionally identical as far as the SQL parser are concerned, and it won't make allowances for passing in your CSV values. Both will execute the same way:
... WHERE foo IN ('1,2,3')
... WHERE foo = '1,2,3'
You'll either have to limit yourself to only as many values as you have placeholders, or dynamically build your SQL and put in a placeholder for each individual value you're trying to put into the IN clause.
e.g.
$placeholders = array_fill(0, count($values_to_check) -1, '?');
$in_clause = implode(',', $placeholders);
/// builds ?,?,?,?,?,....?
$sql = "SELECT ... WHERE foo IN ($in_clause)";
$stmt = $dbh->prepare($sql);
$stmt->execute($values_to_check);
This is one place where prepared statements fall flat on their faces, and you have to fall back to good old "build some sql dynamically".
There is sometimes another way to accomplish the desired result by casting the integer you're trying to compare as a string surrounded by commas and checking if the result is contained in your list of possible values (with added commas on either side as well). It's not the most efficient for performance maybe, but it allows you to do what you want in a single procedure or query.
For example (in your case) something like this might work:
SELECT * from tags WHERE INSTR (CONCAT(',', tags_in, ','), CONCAT(',', tag_id, ',') );
MySql is a little bit weird in that it does the conversion from int to char within the CONCAT function, some other databases require explicit casting.
$zips = array('10583','06890','06854');
$list = implode("','",$zips);
$q = "SELECT site_id FROM site_zipcodes WHERE zipcode IN ('%s')";
$result = db_query($q, $list);
This query returns no results.
However, a sprintf with the same parameters returns
SELECT site_id FROM site_zipcodes WHERE zipcode IN ('10583','06890','06854')
and when I put that query into Sequel Pro, I get three results (the expected behavior).
When I use one zipcode in the IN statement, db_query works just fine.
I can't for the life of me figure out why this is happening.
Without a definition of the db_query function, it's not possible to tell why this isn't working. For debugging this, the db_query function could echo (or var_dump) the actual SQL text being sent to the database.
But likely, the SQL produced by the db_query function does not include the comma separators in the IN list. The comma separators must be part of the SQL text. Any commas that are passed in as part of a value of a bind parameter are considered to be part of the value.
If we use a prepared statement, like this:
SELECT * FROM foo WHERE bar IN ( ? )
And we supply 'fee','fi','fo','fum' as the value of the bind parameter, that's equivalent to running statement like this:
SELECT * FROM foo WHERE bar IN ( '''fee'',''fi'',''fo'',''fum''' )
Such that the query will only return rows where bar is equal to that single string. Effectively equivalent to:
SELECT * FROM foo WHERE bar = '''fee'',''fi'',''fo'',''fum'''
To search for rows that match one of the values in that list, using a statement with bind parameters, the SQL text would need to be of the form:
SELECT * FROM foo WHERE bar IN ( ? , ? , ? , ? )
The commas need to be part of the actual SQL text. And we'd supply a separate value for each of the four bind parameters.
But again, I'm just guessing at what that db_query function is doing.