I have created a dynamic query for my site. The query is done using this:
$sql='SELECT CONCAT(issues.type,"0",issues.kbid) as KBID,issue_tasks.PARENTID as Parent,issues.issuesummary as Summary,products.productdescription as Product,organizations.shortname as Organization,issue_priorities.description as Priority,date_format(issues.createddate, "%d/%m/%Y") as Reported,date_format(issues.lastupdated, "%d/%m/%Y") as Updated,issue_status.statusdescription as Status,issue_resolutions.resdescription as Resolution,users.logon as Assigned FROM issues
INNER JOIN issue_priorities ON issue_priorities.VALUE = issues.PRIORITY - 1
INNER JOIN issue_resolutions ON issue_resolutions.RESID = issues.RESOLUTION
INNER JOIN users ON users.ID = issues.ASSIGNEDUSERID
INNER JOIN products ON products.PRODUCTID = issues.PRODUCTID
INNER JOIN organizations ON organizations.orgid = issues.creatingorg
INNER JOIN issue_status ON issue_status.STATUSID = issues.STATUS
LEFT JOIN issue_tasks ON issue_tasks.CHILDID = issues.KBID
WHERE ';
if(isset($_SESSION['summ']))
{
$sql.= sprintf('issues.issuesummary REGEXP "%s"', implode('|', $words));
}
EDIT : My full SQL query:
SELECT CONCAT(issues.type,"0",issues.kbid) as KBID,issue_tasks.PARENTID as Parent,issues.issuesummary as Summary,products.productdescription as Product,organizations.shortname as Organization,issue_priorities.description as Priority,date_format(issues.createddate, "%d/%m/%Y") as Reported,date_format(issues.lastupdated, "%d/%m/%Y") as Updated,issue_status.statusdescription as Status,issue_resolutions.resdescription as Resolution,users.logon as Assigned FROM issues INNER JOIN issue_priorities ON issue_priorities.VALUE = issues.PRIORITY - 1 INNER JOIN issue_resolutions ON issue_resolutions.RESID = issues.RESOLUTION INNER JOIN users ON users.ID = issues.ASSIGNEDUSERID INNER JOIN products ON products.PRODUCTID = issues.PRODUCTID INNER JOIN organizations ON organizations.orgid = issues.creatingorg INNER JOIN issue_status ON issue_status.STATUSID = issues.STATUS LEFT JOIN issue_tasks ON issue_tasks.CHILDID = issues.KBID WHERE issues.issuesummary REGEXP "sabre|rtf"
But I keep getting an error like so:
SQLSTATE[42000]: Syntax error or access violation: 1139 Got error 'empty (sub)expression' from regexpSQLSTATE[42000]: Syntax error or access violation: 1139 Got error 'empty (sub)expression' from regexpCould not execute query!!!
I am not sure why I get an encapsulation error in this.
Assuming that $words contains the words you want to use in your regular expression, you can simplify the code:
if (isset($_SESSION['summ']) && count($words)) {
$sql .= sprintf('issues.issuesummary REGEXP "%s"', implode('|', $words);
}
The error that you have encountered, ER_REGEXP_ERROR, is an error generated by the MySQL server when the regular expression pattern that you have provided cannot be parsed. In other words, your regular expression is syntactically incorrect.
The message given in your particular case, "empty (sub)expression", suggests that the regular expression contains a subexpression that is empty. Without seeing the final regular expression that your code generates, it's impossible to say for sure—but since you combine a number of subexpressions with the | alternation operator, it would be a reasonable guess that one or more of those subexpressions is empty: this would happen if at least one of the elements of your $words array is (converted to) the empty string ''.
All this said, it's not at all clear why you are using regular expressions here at all. If $words contains regular expression patterns, then it may be a reasonable approach—although in that case one really ought to place each subexpression in brackets when joining them (in order to avoid any operators that have lower precedence than alternation messing with your intention). On the other hand, if $words just contains literal values that you wish to match exactly, MySQL's IN() operator would be more efficient, appropriate and correct:
WHERE issues.issuesummary IN ('abc', 'xyz', ...)
This could be achieved with the following PHP code:
$sql .= "issues.issuesummary IN ('".implode("','", $words)."')";
However, note that the elements of $words must first be escaped in order to avoid bugs with certain characters (that would also give rise to SQL injection vulnerabilities)! Better yet, you could parameterise each literal.
How to do either of these things will depend on which MySQL API you're using—since you have not indicated any in your question, I think specific examples are beyond the scope of this answer.
Related
If i give hard coded value inside query it works, but not in case of sub query or column given.
Here is small example of issue i am facing :
Following both query is a type of sub query, like its part of another query, so don't think that where is table 'm' and something else, as it is working already.
So, my query like :
1)
SELECT GROUP_CONCAT(CONCAT(a_u.first_name,' ', a_u.last_name)) AS associated_admin_u
FROM users a_u
WHERE a_u.id IN(m.associated_admin)
GROUP
BY m.id
And m.associated_admin will return a quoted string like '1,10' so this will not work because of its a string.
2)
SELECT GROUP_CONCAT(CONCAT(a_u.first_name,' ', a_u.last_name)) AS associated_admin_u
FROM users a_u
WHERE a_u.id IN(1,10)
GROUP
BY m.id
If i write hard code like 1,10 it works, because it is not a string
So first one is not works because that query is part of another query as a sub query.
And i am sure this question couldn't be duplicate as i am facing it like in this way so any help would be appreciate, thanks reader!
Based on your comments, you need something like:
SELECT GROUP_CONCAT(CONCAT(a_u.first_name,' ', a_u.last_name)) AS associated_admin_u
FROM users a_u
WHERE FIND_IN_SET(a_u.id, TRIM(BOTH '\'' FROM m.associated_admin))
GROUP
BY m.id
This will first trim the quotes from m.associated_admin and then use FIND_IN_SET instead of IN so that you can use a string with comma-separated values.
You can just create a subquery in IN for example:
SELECT group_concat(CONCAT(a_u.first_name,' ', a_u.last_name)) AS associated_admin_u
FROM users a_u WHERE a_u.id IN(
SELECT id FROM mytable WHERE id IN(1,10)
) GROUP BY m.id
I am getting the 'Cardinality Violation' error, for the following SQL:
Doctrine\DBAL\Exception\DriverException: An exception occurred while executing
SELECT p.* FROM mod_products_products p
LEFT JOIN mod_products_products_categories c_link ON c_link.product_id = p.id
LEFT JOIN mod_products_brands b ON p.brand_id = b.id
LEFT JOIN mod_products_groups vg ON p.variation_id = vg.id
LEFT JOIN mod_products_categories c ON c_link.category_id = c.id
LEFT JOIN mod_products_group_options vg_o ON vg_o.group_id = vg.id
LEFT JOIN mod_products_group_values vg_o_v ON vg_o_v.option_id = vg_o.id
WHERE (p.name LIKE (?, ?)) AND (p.parent_id = 0) AND (vg_o.disabled=0)
GROUP BY p.id ORDER BY p.name ASC
LIMIT 18446744073709551615 OFFSET 0
with params ["%big%", "%light%"]: SQLSTATE[21000]: Cardinality violation: 1241 Operand should contain 1 column(s).
The error only occurs if there is more than one value defined in the parameter list for WHERE (p.name LIKE (?, ?)).
I am using executeQuery(), and passing the array as Connection::PARAM_STR_ARRAY. In the original statement I am defining the trouble point as:
$builder->andWhere('p.name LIKE (:partial_names)');
It seems it doesn't like getting an array passed as partial_names. Any ideas on what is causing this, and how to avoid it?
MySQL LIKE is a "string comparison function" and as such compares one string to another, using "simple pattern matching".
If you check the SQL standard, you'll notice that the BNF grammar for LIKE accepts only "character-like" and "octet-like" arguments, both of which are essentially what we'd call strings. (There is some detail around the fact that LIKE performs a binary, character-for-character match on the RHS, which is different than how = operates: foo LIKE 'bar' and foo='bar' may produce different results.)
All this means you can't do LIKE ('a', 'b') because the columnar expression ('a', 'b') is not string-like. Or in geeky standard language, it's cardinality (2) differs from the expected cardinality (1). However, you can do this in MySQL and SQLite (maybe other engines):
WHERE foo LIKE ('%bar')
because the cardinality of the RHS is 1 (there is one column), which is what LIKE expects.
You're wanting something effectively similar to foo LIKE IN ('a', 'b'), but that doesn't exist either (for the SQL standard reason mentioned above). This Q&A shows some workarounds for that behavior, REGEXP based being the accepted answer.
So, to get around this error, you need to rewrite your query to use multiple LIKE, or a REGEXP, or maybe even something like FIND_IN_SET.
Change
(p.name LIKE (?, ?))
to
(p.name LIKE ? OR p.name LIKE ?)
and
["%big%", "%light%"]
to
"%big%", "%light%"
I need to optimise SQL queries in my PHP code used by hugely over headed web service. I'am getting list of words.
1) get word identifiers
2) foreach identifier get word as php object
3) print word details in xml
Now I have a code that takes this identifier of Word in constructor.
Then when user accesses properties is lazily loading given properties.
Web service is returning all word details so it takes all word properties.
Making many 5-10 simple sql queries each time like get native word, get foreign word, get transcription. It was done as such assuming that I one time need less info about word and the other time I need more information.
Now when my hosting provider deactivated my website as impacting too much overhead on resources of shared hosting I need to optimize it.
1) I will provide INDEX, UNIQUE where it is possible and it has't been used yet.
2) I think about replacing many simple sql queries lazily retrieving word's properties with longer joining query:
EXPLAIN SELECT nt.deutsch_id, nt.article, n.deutsch_word, ft.french_id, ft.article, f.french_word, p.part, fd.transcription, fd.definition
FROM translation_enfr ft
INNER JOIN translation_ende nt ON nt.translation_id = ft.translation_id
INNER JOIN deutsch n ON n.deutsch_id = nt.deutsch_id
INNER JOIN french f ON f.french_id = ft.french_id
INNER JOIN parts p ON p.part_id = ft.part_id
LEFT JOIN french_details fd ON fd.translation_id = ft.translation_id
WHERE ft.translation_id =2
Do you think it will be better/faster than using:
public function getNativeWord($withArticle = true) {
if(is_null($this->nativeWord)) {
$q = "SELECT {$langLabel}_word FROM {$langLabel}
WHERE {$langLabel}_id = :native_id";
}
}
And other current queries are similar:
"SELECT {$langLabel}_word FROM {$langLabel} WHERE {$langLabel}_id = :foreign_id"
"SELECT article FROM translation_en{$this->nativeLang} WHERE translation_id = :translation_id";
"SELECT parts.part FROM parts INNER JOIN translation ON parts.part_id = translation.part_id WHERE translation_id = :trans_id"
"SELECT transcription FROM {$langLabel}_details WHERE translation_id = :trans_id";
"SELECT definition FROM {$langLabel}_details WHERE translation_id = :trans_id";
I think to preload each properties of this Word object only remaining to load easily images, sentences, comments as here I have 1toMany relationship!
UPDATED 1
Output of EXPLAIN SELECT...*
I have a situation where lets say i'm trying to get the information about some food. Then I need to display all the information plus all the ingredients in that food.
With my query, i'm getting all the information in an array but only the first ingredient...
myFoodsArr =
[0]
foodDescription = "the description text will be here"
ratingAverage = 0
foodId = 4
ingredient = 1
ingAmount = 2
foodName = "Awesome Food name"
typeOfFood = 6
votes = 0
I would like to get something back like this...
myFoodsArr =
[0]
foodDescription = "the description text will be here"
ratingAverage = 0
foodId = 4
ingArr = {ingredient: 1, ingAmount: 4}, {ingredient: 3, ingAmount: 2}, {ingredient: 5, ingAmount: 1}
foodName = "Awesome Food name"
typeOfFood = 6
votes = 0
This is the query im working with right now. How can I adjust this to return the food ID 4 and then also get ALL the ingredients for that food? All while at the same time doing other things like getting the average rating of that food?
Thanks!
SELECT a.foodId, a.foodName, a.foodDescription, a.typeOfFood, c.ingredient, c.ingAmount, AVG(b.foodRating) AS ratingAverage, COUNT(b.foodId) as tvotes
FROM `foods` a
LEFT JOIN `foods_ratings` b
ON a.foodId = b.foodId
LEFT JOIN `foods_ing` c
ON a.foodId=c.foodId
WHERE a.foodId=4
EDIT:
Catcall introduced this concept of "sub queries" I never heard of, so I'm trying to make that work to see if i can do this in 1 query easily. But i just keep getting a return false. This is what I was trying with no luck..
//I changed some of the column names to help them be more distinct in this example
SELECT a.foodId, a.foodName, a.foodDescription, a.typeOfFood, AVG(b.foodRating) AS ratingAverage, COUNT(b.foodId) as tvotes
FROM foods a
LEFT JOIN foods_ratings b ON a.foodId = b.foodId
LEFT JOIN (SELECT fId, ingredientId, ingAmount
FROM foods_ing
WHERE fId = 4
GROUP BY fId) c ON a.foodId = c.fId
WHERE a.foodId = 4";
EDIT 1 more thing related to ROLANDS GROUP_CONCAT/JSON Idea as a solution 4 this
I'm trying to make sure the JSON string im sending back to my Flash project is ready to be properly parsed Invalid JSON parse input. keeps popping up..
so im thinking i need to properly have all the double quotes in the right places.
But in my MySQL query string, im trying to escape the double quotes, but then it makes my mySQL vars not work, for example...
If i do this..
GROUP_CONCAT('{\"ingredient\":', \"c.ingredient\", ',\"ingAmount\":', \"c.ingAmount\", '}')`
I get this...
{"ingredient":c.ingredient,"ingAmount":c.ingAmount},{"ingredient":c.ingredient,"ingAmount":c.ingAmount},{"ingredient":c.ingredient,"ingAmount":c.ingAmount}
How can i use all the double quotes to make the JSON properly formed without breaking the mysql?
This should do the trick:
SELECT food_ingredients.foodId
, food_ingredients.foodName
, food_ingredients.foodDescription
, food_ingredients.typeOfFood
, food_ingredients.ingredients
, AVG(food_ratings.food_rating) food_rating
, COUNT(food_ratings.foodId) number_of_votes
FROM (
SELECT a.foodId
, a.foodName
, a.foodDescription
, a.typeOfFood
, GROUP_CONCAT(
'{ingredient:', c.ingredient,
, ',ingAmount:', c.ingAmount, '}'
) ingredients
FROM foods a
LEFT JOIN foods_ing c
ON a.foodsId = c.foodsId
WHERE a.foodsId=4
GROUP BY a.foodId
) food_ingredients
LEFT JOIN food_ratings
ON food_ingredients.foodId = food_ratings.foodId
GROUP BY food_ingredients.foodId
Note that the type of query you want to do is not trivial in any SQL-based database.
The main problem is that you have one master (food) with two details (ingredients and ratings). Because those details are not related to each other (other than to the master) they form a cartesian product with each other (bound only by their relationship to the master).
The query above solves that by doing it in 2 steps: first, join to the first detail (ingredients) and aggregate the detail (using group_concat to make one single row of all related ingredient rows), then join that result to the second detail (ratings) and aggregate again.
In the example above, the ingredients are returned in a structured string, exactly like it appeared in your example. If you want to access the data inside PHP, you might consider adding a bit more syntax to make it a valid JSON string so you can decode it into an array using the php function json_decode(): http://www.php.net/manual/en/function.json-decode.php
To do that, simply change the line to:
CONCAT(
'['
, GROUP_CONCAT(
'{"ingredient":', c.ingredient
, ',"ingAmount":', c.ingAmount, '}'
)
, ']'
)
(this assumes ingredient and ingAmount are numeric; if they are strings, you should double quote them, and escape any double quotes that appear within the string values)
The concatenation of ingredients with GROUP_CONCAT can lead to problems if you keep a default setting for the group_concat_max_len server variable. A trivial way to mitigate that problem is to set it to the maximum theoretical size of any result:
SET group_concat_max_len = ##max_allowed_packet;
You can either execute this once after you open the connection to mysql, and it will then be in effect for the duration of that session. Alternatively, if you have the super privilege, you can change the value across the board for the entire MySQL instance:
SET GLOBAL group_concat_max_len = ##max_allowed_packet;
You can also add a line to your my.cnf or my.ini to set group_concat_max_lenght to some arbitrary large enough static value. See http://dev.mysql.com/doc/refman/5.5/en/server-system-variables.html#sysvar_group_concat_max_len
One obvious solution is to actually perform two queries:
1) get the food
SELECT a.foodId, a.foodName, a.foodDescription, a.typeOfFood
FROM `foods` a
WHERE a.foodsId=4
2) get all of its ingredients
SELECT c.ingredient, c.ingAmount
FROM `foods_ing` c
WHERE c.foodsId=4
This approach has the advantage that you don't duplicate data from the "foods" table into the result. The disadvantage is that you have to perform two queries. Actually you have to perform one extra query for each "food", so if you want to have a listing of foods with all their ingredients, you would have to do a query for each of the food record.
Other solutions usually have many disadvantages, one of them is using GROUP_CONCAT function, but it has a tough limit on the length of the returned string.
When you compare MySQL's aggregate functions and GROUP BY behavior to SQL standards, you have to conclude that they're simply broken. You can do what you want in a single query, but instead of joining directly to the table of ratings, you need to join on a query that returns the results of the aggregate functions. Something along these lines should work.
select a.foodId, a.foodName, a.foodDescription, a.typeOfFood,
c.ingredient, c.ingAmount,
b.numRatings, b.avgRating
from foods a
left join (select foodId, count(foodId) numRatings, avg(foodRating) avgRating
from foods_ratings
group by foodId) b on a.foodId = b.foodId
left join foods_ing c on a.foodId = c.foodId
order by a.foodId
I have a controller action that uses a sql query:
$tag = $this->params['tag'];
$this->set('projects', $this->Project->query('SELECT * FROM projects INNER JOIN projects_tags ON projects.id = projects_tags.project_id INNER JOIN tags on projects_tags.tag_id = tags.id WHERE tags.tag LIKE $tag'));
As you can see at the end I want to use a where clause with the $tag variable but I'm not sure how the syntax would go. As I'm getting the error
Unknown column '$tag' in 'where clause'
Can someone steer me in the right direction?
Ta,
Jonesy
I would strongly advise you to use the Cake ORM instead of raw queries, especially if you're going to plug URL parameters into it. Conditions on HABTM tables can be tricky, but you can build your joins using Cake's ORM syntax as well!
Read the manual, section 3.7.6.9 Joining tables.
Should you want to use Cake's ORM, the following code should provide results equivalent to your raw SQL query:
$this->loadModel('ProjectsTag'); // Load the joining table as pseudo-model
// Define temporary belongsTo relationships between the pseudo-model and the two real models
$this->ProjectsTag->bindModel(array(
'belongsTo' => array('Project','Tag')
));
// Retrieve all the join-table records with matching Tag.tag values
$result_set = $this->ProjectsTag->find('all',array(
'conditions' => array('Tag.tag LIKE' => "%{$tag}%")
));
// Extract the associated Project records from the result-set
$projects = Set::extract('/Project', $result_set);
// Make the set of Project records available to the view
$this->set(compact('projects'));
in php there's a difference between single and double quotes... basically, single quotes dont evaluate the variables... use double quotes instead
And i think that LIKE will need also single quotes.. i'm not really sure
"SELECT * FROM projects INNER JOIN projects_tags ON projects.id = projects_tags.project_id INNER JOIN tags on projects_tags.tag_id = tags.id WHERE tags.tag LIKE '$tag'"
i know.. i know.. people will start talkin' about sql injection.. and the need to scape the caracters... that's another question =)
good luck!
I would at least consider using the cakephp sanitize functions on your tag strings if they are user sourced. See http://book.cakephp.org/view/1183/Data-Sanitization or if using mysql as the db at least consider using http://www.php.net/manual/en/function.mysql-escape-string.php or do something to filter your user input. But the best thing is to make use of the CakePHP orm stuff.
Modify your query as:
$this->set('projects',
$this->Project->query("SELECT * FROM projects
INNER JOIN projects_tags
ON projects.id = projects_tags.project_id
INNER JOIN tags ON projects_tags.tag_id = tags.id
WHERE tags.tag LIKE '" . $tag . "'") //problem was here
);
and it will work.