DBAL cardinality violation error - php

I am getting the 'Cardinality Violation' error, for the following SQL:
Doctrine\DBAL\Exception\DriverException: An exception occurred while executing
SELECT p.* FROM mod_products_products p
LEFT JOIN mod_products_products_categories c_link ON c_link.product_id = p.id
LEFT JOIN mod_products_brands b ON p.brand_id = b.id
LEFT JOIN mod_products_groups vg ON p.variation_id = vg.id
LEFT JOIN mod_products_categories c ON c_link.category_id = c.id
LEFT JOIN mod_products_group_options vg_o ON vg_o.group_id = vg.id
LEFT JOIN mod_products_group_values vg_o_v ON vg_o_v.option_id = vg_o.id
WHERE (p.name LIKE (?, ?)) AND (p.parent_id = 0) AND (vg_o.disabled=0)
GROUP BY p.id ORDER BY p.name ASC
LIMIT 18446744073709551615 OFFSET 0
with params ["%big%", "%light%"]: SQLSTATE[21000]: Cardinality violation: 1241 Operand should contain 1 column(s).
The error only occurs if there is more than one value defined in the parameter list for WHERE (p.name LIKE (?, ?)).
I am using executeQuery(), and passing the array as Connection::PARAM_STR_ARRAY. In the original statement I am defining the trouble point as:
$builder->andWhere('p.name LIKE (:partial_names)');
It seems it doesn't like getting an array passed as partial_names. Any ideas on what is causing this, and how to avoid it?

MySQL LIKE is a "string comparison function" and as such compares one string to another, using "simple pattern matching".
If you check the SQL standard, you'll notice that the BNF grammar for LIKE accepts only "character-like" and "octet-like" arguments, both of which are essentially what we'd call strings. (There is some detail around the fact that LIKE performs a binary, character-for-character match on the RHS, which is different than how = operates: foo LIKE 'bar' and foo='bar' may produce different results.)
All this means you can't do LIKE ('a', 'b') because the columnar expression ('a', 'b') is not string-like. Or in geeky standard language, it's cardinality (2) differs from the expected cardinality (1). However, you can do this in MySQL and SQLite (maybe other engines):
WHERE foo LIKE ('%bar')
because the cardinality of the RHS is 1 (there is one column), which is what LIKE expects.
You're wanting something effectively similar to foo LIKE IN ('a', 'b'), but that doesn't exist either (for the SQL standard reason mentioned above). This Q&A shows some workarounds for that behavior, REGEXP based being the accepted answer.
So, to get around this error, you need to rewrite your query to use multiple LIKE, or a REGEXP, or maybe even something like FIND_IN_SET.

Change
(p.name LIKE (?, ?))
to
(p.name LIKE ? OR p.name LIKE ?)
and
["%big%", "%light%"]
to
"%big%", "%light%"

Related

Laravel DB select statement. Query returns column name as result rather than values

UPDATE: The below is for context but I figured out this is not where the problem is.
I'm pretty new to php and laravel. I do an AJAX call to a controller that uses a postgresql query from a repository. The query is a select query it has columns dated, total_sms and demo (demo is gender Male/Female).
I have an array $return[] The below is within a foreach ($result as $day).
At the end the array is translated to json before returning.
$return[$day->dated][$day->demo] = $day->total_sms;
I expected the above to insert the dates as a key with an array value, the gender (Male and Female) as the keys of the inner array, with them having an array as a value and these arrays will contain the total_sms value. (that sentence must have been confusing...)
However the result is:
{2017-07-03: {d.gender: "0"}, 2017-07-04: {d.gender: "0"}}
How would I build an array like:
{2017-07-03: {Male: "0" , Female: "10"}, 2017-07-04: {Male: "0", Female: "0"}
Although I think the above is not actually how I originally described it? I am planning on decoding it and sticking the information into a chart.js chart with:
$.each(d, function(date, value) {
window.barChart.data.labels.push(date);
$.each(d.date, function(demographic, value) {
// ALWAYS ASCENDING DATA
});
UPDATE: Code that is creating the issue.
I have a query in a repository that can have varying parameters inserted, to change what demographic is used. Below is the version I use in pgAdmin, and it works fine.
SELECT d.date dated, count(se.id) total_sms, demo
FROM (
select to_char(date_trunc('day', (current_date - offs)), 'YYYY-MM-DD') AS date, demographic.gender as demo
FROM generate_series(0, 365, 1) AS offs
CROSS JOIN (SELECT DISTINCT gender FROM common.profile) AS demographic
) d
LEFT OUTER JOIN (
SELECT id, customer_id, client_report.insert_time, profile.gender, profile.house_income, profile.address AS postcode, profile.age AS age_group, profile.is_employed, profile.is_married, profile.no_children, profile.no_cars, profile.shopping_frequency
FROM common.client_report
JOIN common.profile
ON client_report.profile_id = profile.uuid
WHERE sms_status = 'SUCCESS' AND customer_id = 5::int
) se
ON (d.date=to_char(date_trunc('day', se.insert_time), 'YYYY-MM-DD')) AND demo = gender
WHERE d.date::date BETWEEN '2017-07-01'::date AND '2017-08-01'::date
GROUP BY d.date, demo
ORDER BY d.date, demo ASC;
Anywhere that states gender, a date or the 5 after AND customer_id = are all generated from named parameters (When I receive an error I can generally fix that, when there is no error, the code works but gives me gender as the column name ['de.gender'] rather than the values ['Male', 'Female']). I mean to say that I have had errors around the parameters, and no longer believe that could be the issue. I then will receive an error around missing a conversion tool for unknown to text. I don't fully understand this but I know it means I have to cast gender as text with ::text. Doing that results in the column name appearing. However doing that in a different, simpler query without the cross join works and the results are correct. This leads me to believe there is an issue with the way my query is or that there is a bug with the DB driver and these queries.
UPDATE: Did a test, should have done it earlier! Placed the version with no parameters into my application, made the function run, and it returns exactly as needed. So this is stemming from parameters, and probably the ones that define tables like de.gender (de being the table)
UPDATE: Another test:
CROSS JOIN (SELECT DISTINCT gender FROM common.profile) AS demographic
On this line, gender is a named parameter (:demograph). However, if I change this to simply gender, the results are as expected with Male, Female values. If it stays as a parameter it seems to be creating lots of strings of 'gender' rather than grabbing the 'Male', 'Female' values.
UPDATE: Did a lot of learning today, and named or positional parameters are for values only. Meaning all inputs are encased in double quotes, which was why I was creating strings of 'gender' rather than male, female.
i donn't know i am not sure about your question
$testArray = array(
"2017-07-03" => array("male" : "0","female" => "10"),
"2017-07-04" => array("male" : "0","female" => "0")
);
return $testArray;

PHP PDO too slow on SELECT with some joins

I'm having a performace problem with the execution of a select in PHP PDO.
Using a script available here at stackoverflow (Simplest way to profile a PHP script), I identified where the problem IS, but I have not found a solution.
My select that is the problem is:
SELECT REDACAO.ID_REDACAO AS ID_REDACAO,
DATE_FORMAT(REDACAO.DATA,'%d/%m/%Y') AS DATAE,
ALUNO.ID_ALUNO AS ID_ALUNO,
(SELECT IFNULL((DATEDIFF(DATE_ADD((SELECT MAX(DATA) FROM REDACAO WHERE ID_ALUNO = ALUNO.ID_ALUNO AND ID_REDACAO NOT IN (SELECT ID_REDACAO FROM CORRECAO)), INTERVAL 7 DAY), now())),NULL) as DATA FROM REDACAO LIMIT 1) AS ULTIMA,
ALUNO.NOME as ALUNO,
REDACAO.ID_TEMA AS ID_TEMA,
TEMA.TITULO as TEMA,
TEMA.MOTIVACIONAIS AS MOTIVACIONAIS,
REDACAO.TEXTO AS TEXTO,
REDACAO.ID_STATUS AS STATUS,
B.NOTA as NOTA,
B.RCORRIGIDA AS CORRIGIDA,
B.NOTA1,
B.COMENTARIO1,
B.NOTA2,
B.COMENTARIO2,
B.NOTA3,
B.COMENTARIO3,
B.NOTA4,
B.COMENTARIO4,
B.NOTA5,
B.COMENTARIO5,
B.COMENTARIO6,
C.COMENTARIO AS COMENTARIO
FROM REDACAO
LEFT OUTER JOIN (SELECT SUM(CORRECAO.C1+CORRECAO.C2+CORRECAO.C3+CORRECAO.C4+CORRECAO.C5) AS NOTA, RCORRIGIDA AS RCORRIGIDA, CORRECAO.C1 as NOTA1, CORRECAO.COM1 as COMENTARIO1, CORRECAO.C2 as NOTA2, CORRECAO.COM2 as COMENTARIO2, CORRECAO.C3 as NOTA3, CORRECAO.COM3 as COMENTARIO3, CORRECAO.C4 as NOTA4, CORRECAO.COM4 as COMENTARIO4, CORRECAO.C5 as NOTA5, CORRECAO.COM5 as COMENTARIO5, CORRECAO.COMGERAL AS COMENTARIO6, CORRECAO.ID_REDACAO FROM CORRECAO GROUP BY CORRECAO.ID_REDACAO) B
ON B.ID_REDACAO = REDACAO.ID_REDACAO
JOIN ALUNO ON ALUNO.ID_ALUNO = REDACAO.ID_ALUNO
JOIN TEMA ON TEMA.ID_TEMA = REDACAO.ID_TEMA
LEFT OUTER JOIN (SELECT (COUNT(COMENTARIO.ID_COMENTARIO)) AS COMENTARIO, COMENTARIO.ID_REDACAO FROM COMENTARIO GROUP BY COMENTARIO.ID_REDACAO) C
ON C.ID_REDACAO = REDACAO.ID_REDACAO
WHERE REDACAO.ID_PROFESSOR = $CodProfessor
and REDACAO.ID_STATUS != 6
ORDER BY (CASE WHEN REDACAO.ID_STATUS = 4 THEN 1 ELSE 0 END) DESC
I'm using (PDO :: FETCH_ASSOC) to get the data. Some columns respond in less than 1 second and others in more than 20 seconds.
Any idea what could be the problem and how to solve it?
Your query contains following that will slow it down:
many joins
many subselects
select without where
functions like COUNT, isnull, datediff, sum.(some of these may cancel an index)
case when
order by
group by
Depending on your indexes, on how the tables are joined, and on how big are the tables, this will eventually get very slower.
Try using 'explain' command, and simplify the query if possible.
explain output
a good video about explain
PDO is not at fault; the query is complex. And there may be missing indexes.
Turn this into a LEFT JOIN (because IN (SELECT...) optimizes poorly.
AND ID_REDACAO NOT IN ( SELECT ID_REDACAO FROM CORRECAO)
Upgrade to 5.6; it has some improvements.
You have two JOIN ( SELECT ... ). Before 5.6 that would be optimized terribly. Move one of them out into a temp table, to which you add a suitable index.
In one of the subqueries, GROUP BY CORRECAO.ID_REDACAO seems to be unnecessary.
These indexes (or PRIMARY KEYs) are needed:
CORRECAO: (ID_REDACAO)
REDACAO: (ID_REDACAO), (ID_PROFESSOR)
ALUNO: (ID_ALUNO)
TEMA: (ID_TEMA)
COMENTARIO: (ID_REDACAO, ID_COMENTARIO) ("compound index")
If those suggestions do not help enough, come back with SHOW CREATE TABLE for each table.

Multiple REGEXP values not working in SQL

I have created a dynamic query for my site. The query is done using this:
$sql='SELECT CONCAT(issues.type,"0",issues.kbid) as KBID,issue_tasks.PARENTID as Parent,issues.issuesummary as Summary,products.productdescription as Product,organizations.shortname as Organization,issue_priorities.description as Priority,date_format(issues.createddate, "%d/%m/%Y") as Reported,date_format(issues.lastupdated, "%d/%m/%Y") as Updated,issue_status.statusdescription as Status,issue_resolutions.resdescription as Resolution,users.logon as Assigned FROM issues
INNER JOIN issue_priorities ON issue_priorities.VALUE = issues.PRIORITY - 1
INNER JOIN issue_resolutions ON issue_resolutions.RESID = issues.RESOLUTION
INNER JOIN users ON users.ID = issues.ASSIGNEDUSERID
INNER JOIN products ON products.PRODUCTID = issues.PRODUCTID
INNER JOIN organizations ON organizations.orgid = issues.creatingorg
INNER JOIN issue_status ON issue_status.STATUSID = issues.STATUS
LEFT JOIN issue_tasks ON issue_tasks.CHILDID = issues.KBID
WHERE ';
if(isset($_SESSION['summ']))
{
$sql.= sprintf('issues.issuesummary REGEXP "%s"', implode('|', $words));
}
EDIT : My full SQL query:
SELECT CONCAT(issues.type,"0",issues.kbid) as KBID,issue_tasks.PARENTID as Parent,issues.issuesummary as Summary,products.productdescription as Product,organizations.shortname as Organization,issue_priorities.description as Priority,date_format(issues.createddate, "%d/%m/%Y") as Reported,date_format(issues.lastupdated, "%d/%m/%Y") as Updated,issue_status.statusdescription as Status,issue_resolutions.resdescription as Resolution,users.logon as Assigned FROM issues INNER JOIN issue_priorities ON issue_priorities.VALUE = issues.PRIORITY - 1 INNER JOIN issue_resolutions ON issue_resolutions.RESID = issues.RESOLUTION INNER JOIN users ON users.ID = issues.ASSIGNEDUSERID INNER JOIN products ON products.PRODUCTID = issues.PRODUCTID INNER JOIN organizations ON organizations.orgid = issues.creatingorg INNER JOIN issue_status ON issue_status.STATUSID = issues.STATUS LEFT JOIN issue_tasks ON issue_tasks.CHILDID = issues.KBID WHERE issues.issuesummary REGEXP "sabre|rtf"
But I keep getting an error like so:
SQLSTATE[42000]: Syntax error or access violation: 1139 Got error 'empty (sub)expression' from regexpSQLSTATE[42000]: Syntax error or access violation: 1139 Got error 'empty (sub)expression' from regexpCould not execute query!!!
I am not sure why I get an encapsulation error in this.
Assuming that $words contains the words you want to use in your regular expression, you can simplify the code:
if (isset($_SESSION['summ']) && count($words)) {
$sql .= sprintf('issues.issuesummary REGEXP "%s"', implode('|', $words);
}
The error that you have encountered, ER_REGEXP_ERROR, is an error generated by the MySQL server when the regular expression pattern that you have provided cannot be parsed. In other words, your regular expression is syntactically incorrect.
The message given in your particular case, "empty (sub)expression", suggests that the regular expression contains a subexpression that is empty. Without seeing the final regular expression that your code generates, it's impossible to say for sure—but since you combine a number of subexpressions with the | alternation operator, it would be a reasonable guess that one or more of those subexpressions is empty: this would happen if at least one of the elements of your $words array is (converted to) the empty string ''.
All this said, it's not at all clear why you are using regular expressions here at all. If $words contains regular expression patterns, then it may be a reasonable approach—although in that case one really ought to place each subexpression in brackets when joining them (in order to avoid any operators that have lower precedence than alternation messing with your intention). On the other hand, if $words just contains literal values that you wish to match exactly, MySQL's IN() operator would be more efficient, appropriate and correct:
WHERE issues.issuesummary IN ('abc', 'xyz', ...)
This could be achieved with the following PHP code:
$sql .= "issues.issuesummary IN ('".implode("','", $words)."')";
However, note that the elements of $words must first be escaped in order to avoid bugs with certain characters (that would also give rise to SQL injection vulnerabilities)! Better yet, you could parameterise each literal.
How to do either of these things will depend on which MySQL API you're using—since you have not indicated any in your question, I think specific examples are beyond the scope of this answer.

Propel ORM - Joining unrelated tables

How does this SQL statement translate into Propel (1.6.3)?
SELECT * FROM table_a JOIN table_b
With tableA and tableB sharing no Foreign Keys and having no relationships defined.
TableAQuery::create()->join('tableB')
doesn't work since Propel complains with an error:
"Fatal error: Uncaught exception 'PropelException' with message 'Unknown relation TableB on the TableA table'
Thanks very much in advance for any help and hints! This is my first post here and I hope I haven't done anything wrong yet :-) (I've searched thoroughly before I posted!)
You could also use "addJoin" like this:
TableAQuery::create()
->addJoin(TableAPeer::ThisCOLUMN, TableBPeer::ThatCOLUMN, Criteria::INNER_JOIN); //Can also be left/right
The third argument also takes left and right join.
And, instead of the usual "filterByXXX()"
->filterByOtherColumn(value)
you'd use "add()", like this:
->add(TableAPeer::OtherCOLUMN, value)
You can work around this limitation by using raw SQL syntax. For instance:
$con = Propel::getConnection(SomePeer::DATABASE_NAME);
$query = 'SELECT * FROM `table_a` JOIN `table_b` LIMIT 10';
$stmt = $con->prepare($query);
if($stmt->execute()) {
$res = $stmt->fetchAll();
var_dump($res);
}
Note #1: These kind of joins can become very big and quickly exhaust the allowed memory size. That's why I've added a LIMIT.
Note #2: The output isn't very clean, arrays of both numeric and associative keys. Maybe there are ways to improve this.

How to return multiple rows in a LEFT JOIN

I have a situation where lets say i'm trying to get the information about some food. Then I need to display all the information plus all the ingredients in that food.
With my query, i'm getting all the information in an array but only the first ingredient...
myFoodsArr =
[0]
foodDescription = "the description text will be here"
ratingAverage = 0
foodId = 4
ingredient = 1
ingAmount = 2
foodName = "Awesome Food name"
typeOfFood = 6
votes = 0
I would like to get something back like this...
myFoodsArr =
[0]
foodDescription = "the description text will be here"
ratingAverage = 0
foodId = 4
ingArr = {ingredient: 1, ingAmount: 4}, {ingredient: 3, ingAmount: 2}, {ingredient: 5, ingAmount: 1}
foodName = "Awesome Food name"
typeOfFood = 6
votes = 0
This is the query im working with right now. How can I adjust this to return the food ID 4 and then also get ALL the ingredients for that food? All while at the same time doing other things like getting the average rating of that food?
Thanks!
SELECT a.foodId, a.foodName, a.foodDescription, a.typeOfFood, c.ingredient, c.ingAmount, AVG(b.foodRating) AS ratingAverage, COUNT(b.foodId) as tvotes
FROM `foods` a
LEFT JOIN `foods_ratings` b
ON a.foodId = b.foodId
LEFT JOIN `foods_ing` c
ON a.foodId=c.foodId
WHERE a.foodId=4
EDIT:
Catcall introduced this concept of "sub queries" I never heard of, so I'm trying to make that work to see if i can do this in 1 query easily. But i just keep getting a return false. This is what I was trying with no luck..
//I changed some of the column names to help them be more distinct in this example
SELECT a.foodId, a.foodName, a.foodDescription, a.typeOfFood, AVG(b.foodRating) AS ratingAverage, COUNT(b.foodId) as tvotes
FROM foods a
LEFT JOIN foods_ratings b ON a.foodId = b.foodId
LEFT JOIN (SELECT fId, ingredientId, ingAmount
FROM foods_ing
WHERE fId = 4
GROUP BY fId) c ON a.foodId = c.fId
WHERE a.foodId = 4";
EDIT 1 more thing related to ROLANDS GROUP_CONCAT/JSON Idea as a solution 4 this
I'm trying to make sure the JSON string im sending back to my Flash project is ready to be properly parsed Invalid JSON parse input. keeps popping up..
so im thinking i need to properly have all the double quotes in the right places.
But in my MySQL query string, im trying to escape the double quotes, but then it makes my mySQL vars not work, for example...
If i do this..
GROUP_CONCAT('{\"ingredient\":', \"c.ingredient\", ',\"ingAmount\":', \"c.ingAmount\", '}')`
I get this...
{"ingredient":c.ingredient,"ingAmount":c.ingAmount},{"ingredient":c.ingredient,"ingAmount":c.ingAmount},{"ingredient":c.ingredient,"ingAmount":c.ingAmount}
How can i use all the double quotes to make the JSON properly formed without breaking the mysql?
This should do the trick:
SELECT food_ingredients.foodId
, food_ingredients.foodName
, food_ingredients.foodDescription
, food_ingredients.typeOfFood
, food_ingredients.ingredients
, AVG(food_ratings.food_rating) food_rating
, COUNT(food_ratings.foodId) number_of_votes
FROM (
SELECT a.foodId
, a.foodName
, a.foodDescription
, a.typeOfFood
, GROUP_CONCAT(
'{ingredient:', c.ingredient,
, ',ingAmount:', c.ingAmount, '}'
) ingredients
FROM foods a
LEFT JOIN foods_ing c
ON a.foodsId = c.foodsId
WHERE a.foodsId=4
GROUP BY a.foodId
) food_ingredients
LEFT JOIN food_ratings
ON food_ingredients.foodId = food_ratings.foodId
GROUP BY food_ingredients.foodId
Note that the type of query you want to do is not trivial in any SQL-based database.
The main problem is that you have one master (food) with two details (ingredients and ratings). Because those details are not related to each other (other than to the master) they form a cartesian product with each other (bound only by their relationship to the master).
The query above solves that by doing it in 2 steps: first, join to the first detail (ingredients) and aggregate the detail (using group_concat to make one single row of all related ingredient rows), then join that result to the second detail (ratings) and aggregate again.
In the example above, the ingredients are returned in a structured string, exactly like it appeared in your example. If you want to access the data inside PHP, you might consider adding a bit more syntax to make it a valid JSON string so you can decode it into an array using the php function json_decode(): http://www.php.net/manual/en/function.json-decode.php
To do that, simply change the line to:
CONCAT(
'['
, GROUP_CONCAT(
'{"ingredient":', c.ingredient
, ',"ingAmount":', c.ingAmount, '}'
)
, ']'
)
(this assumes ingredient and ingAmount are numeric; if they are strings, you should double quote them, and escape any double quotes that appear within the string values)
The concatenation of ingredients with GROUP_CONCAT can lead to problems if you keep a default setting for the group_concat_max_len server variable. A trivial way to mitigate that problem is to set it to the maximum theoretical size of any result:
SET group_concat_max_len = ##max_allowed_packet;
You can either execute this once after you open the connection to mysql, and it will then be in effect for the duration of that session. Alternatively, if you have the super privilege, you can change the value across the board for the entire MySQL instance:
SET GLOBAL group_concat_max_len = ##max_allowed_packet;
You can also add a line to your my.cnf or my.ini to set group_concat_max_lenght to some arbitrary large enough static value. See http://dev.mysql.com/doc/refman/5.5/en/server-system-variables.html#sysvar_group_concat_max_len
One obvious solution is to actually perform two queries:
1) get the food
SELECT a.foodId, a.foodName, a.foodDescription, a.typeOfFood
FROM `foods` a
WHERE a.foodsId=4
2) get all of its ingredients
SELECT c.ingredient, c.ingAmount
FROM `foods_ing` c
WHERE c.foodsId=4
This approach has the advantage that you don't duplicate data from the "foods" table into the result. The disadvantage is that you have to perform two queries. Actually you have to perform one extra query for each "food", so if you want to have a listing of foods with all their ingredients, you would have to do a query for each of the food record.
Other solutions usually have many disadvantages, one of them is using GROUP_CONCAT function, but it has a tough limit on the length of the returned string.
When you compare MySQL's aggregate functions and GROUP BY behavior to SQL standards, you have to conclude that they're simply broken. You can do what you want in a single query, but instead of joining directly to the table of ratings, you need to join on a query that returns the results of the aggregate functions. Something along these lines should work.
select a.foodId, a.foodName, a.foodDescription, a.typeOfFood,
c.ingredient, c.ingAmount,
b.numRatings, b.avgRating
from foods a
left join (select foodId, count(foodId) numRatings, avg(foodRating) avgRating
from foods_ratings
group by foodId) b on a.foodId = b.foodId
left join foods_ing c on a.foodId = c.foodId
order by a.foodId

Categories