I'm trying to select some records from an Oracle 11g Database. The Statement is used to implement some kind of "filter" function for an HTML Table.
Requirements: limit for paging and order the filtered results.
The query is created with Zend_Db_Select
*Works like a charm:*
$select->where('APPLICATIONS LIKE ?', '%MYAPP1%');
$select->where('APPLICATIONS NOT LIKE ?', '%GENESIS%');
$select->limit(20);
= 1 matching result (which is ok!)
The problem occurs when I try to order the filtered result:
$select->order('PATH ASC');
= 3 matching results ??
I think it has something to do with the query generated by Zend DB Select, it looks like this:
SELECT z2.*
FROM (
SELECT z1.*, ROWNUM AS "zend_db_rownum"
FROM (
SELECT "APPS".* FROM "APPS" WHERE (APPLICATIONS LIKE '%MYAPP1%') AND (APPLICATIONS NOT LIKE '%GENESIS%') ORDER BY "PATH" ASC
) z1
) z2
WHERE z2."zend_db_rownum" BETWEEN 1 AND 20
If I run the query without order everything is fine.
If I run the query without limit everything is fine.
If I run the query with order + limit -> wrong result.
If I take the statement and put the order after "BETWEEN 1 AND 20" it works like I want. But how to say Zend DB Select to change it?
Important: I'm doing the query against an Oracle VIEW, if I do it against a "table" it works too.
Obviously Oracle's interpretation of the query is not what the Zend framework intents:
Oracle seems to associate the ROWNUM with the row number count on the inner query before ordering, so the alias zend_db_rownum delivers wrong numbers in the where clause of the outer query.
Since we're not in control of the way the Zend framework generates the SQL in response to the limit() instruction, the only workaround I can think of is skipping the Zend limit() instruction, and instead doing something along the lines of:
$select->where('APPLICATIONS LIKE ?', '%MYAPP1%');
$select->where('APPLICATIONS NOT LIKE ?', '%GENESIS%');
$select->order('PATH ASC');
$sql = $select->__toString();
$sql = "select * from (" . $sql . ") where ROWNUM between 1 and 20";
$stmt = $db->query( $sql, array());
$result = $stmt->fetchAll();
Of course, this workaround is only legitimate in case you're not developing cross-DB, so your code doesn't have to be DB agnostic.
Meaning, you will restrict your solution to Oracle if you use my suggested workaround.
If you checked that there is no error in SQL generation, and really no different conditions in WHERE clause, it may be Oracle server bug. To check it, try it on different Oracle server version or different Oracle server patch level.
Related
I'm executing a PHP yii2 query to get records from the database but it's not working. But I directly executed the query in the MYSQL console then it gives me the expected output.
Normal SQL
SELECT * FROM `tbl_inbox` WHERE sender_id=778 AND recipient_id=736 OR sender_id=778 AND recipient_id=736 ORDER BY timestamp DESC LIMIT 1
Above query giving me expected result but when i change to Yii2 like :
$message=Inbox::find()->select(['message'])->where(['sender_id'=>$sender_id,'recipient_id'=>$recipient_id])->orWhere(['sender_id'=>$recipient_id,'recipient_id'=>$sender_id])->andWhere(['ad_id'=>$ad_id,'category'=>$category])->orderBy(['timestamp'=>SORT_DESC])->one();
It gives me the wrong result.
What is my logic :
I have a chat on my website and I need to get the last message order by timestamp. But sender_id and the recipient will be visa versa.
How to fix ?
There are significant differences between the normal SQL version of the query and the Yii version.
1) Selected fields
The normal SQL query selects all fields, but Yii version selects only message field.
This is because the normal SQL has SELECT * FROM ... but in Yii query you are calling select(['message']). If you want to select all fields you can leave the select() method call out or you can use select('*').
2) Your conditions are different
Considering operator priority your conditions in normal SQL are:
(sender_id=778 AND recipient_id=736)
OR (sender_id=778 AND recipient_id=736)
But in your Yii version of query:
(
(sender_id=778 AND recipient_id=736)
OR (sender_id=736 AND recipient_id=778)
) AND (
ad_id=$ad_id AND category=$category
)
In your normal SQL the both sides of OR condition are same but in your Yii version the values for sender/recipient are swapped in the second argument of OR operator.
Also there is extra part added by this call andWhere(['ad_id'=>$ad_id,'category'=>$category])
$message=Inbox::find()->select(['message'])->where(['sender_id'=>$sender_id,'recipient_id'=>$recipient_id])->orWhere(['sender_id'=>$recipient_id])->orWhere(['recipient_id'=>$sender_id])->andWhere(['ad_id'=>$ad_id,'category'=>$category])->orderBy(['timestamp'=>SORT_DESC])->one();
try this
I'm getting following sql
SELECT `message` FROM `user` WHERE ((((`sender_id`='2333') AND (`recipient_id`='23222')) OR (`sender_id`='23222')) OR (`recipient_id`='23333')) AND ((`ad_id`='10') AND (`category`='1')) ORDER BY `timestamp` DESC
you can check raw sql easily, then it will be mostly straight forward how Yii Query builder works.
use following code:
$rawSql = Inbox::find()
...
->createCommand()->getRawSql();
I would change your code into this:
$message=Inbox::find()
->select(['message'])
->where([
'and',
[
'or',
['sender_id'=>$sender_id,'recipient_id'=>$recipient_id],
['sender_id'=>$recipient_id,'recipient_id'=>$sender_id]
],
['ad_id'=>$ad_id,'category'=>$category]
])
->orderBy(['timestamp'=>SORT_DESC])->one();
Im programming a search with ZF3 and the DB module.
Everytime i use more than 1 short keyword - like "49" and "am" or "1" and "is" i get this error:
Statement could not be executed (HY000 - 2006 - MySQL server has gone away)
Using longer keywords works perfectly fine as long as i dont use 2 or more short keywords.
The problem only occurs on the live server its working fine on the local test server.
The project table has ~2200 rows with all kind of data the project_search table has 17000 rows with multiple entries for each project , each looking like:
id, projectid, searchtext
The searchtext Column is fulltext. Here the relevant part of the php code:
$sql = new Sql($this->db);
$select = $sql->select(['p'=>'projects']);
if(isset($filter['search'])) {
$keywords = preg_split('/\s+/', trim($filter['search']));
$join = $sql->select('project_search');
$join->columns(['projectid' => new Expression('DISTINCT(projectid)')]);
$join->group("projectid");
foreach($keywords as $keyword) {
$join->having(["LOCATE('$keyword', GROUP_CONCAT(searchtext))"]);
}
$select->join(
["m" => $join],
"m.projectid = p.id",
['projectid'],
\Zend\Db\Sql\Select::JOIN_RIGHT
);
}
Here the resulting Query:
SELECT p.*, m.projectid
FROM projects AS p
INNER JOIN (
SELECT projectid
FROM project_search
GROUP BY projectid
HAVING LOCATE('am', GROUP_CONCAT(searchtext))
AND LOCATE('49', GROUP_CONCAT(searchtext))
) AS m
ON m.projectid = p.id
GROUP BY p.id
ORDER BY createdAt DESC
I rewrote the query using "MATCH(searchtext) AGAINST('$keyword)" and "searchtext LIKE '%keyword%' with the same result.
The problem seems to be with the live mysql server how can i debug this ?
[EDIT]
After noticing that the error only occured in a special view which had other search related queries - each using multiple joins (1 join / keyword) - i merged those queries and the error was gone. The amount of queries seemed to kill the server.
Try refactoring your inner query like so.
SELECT a.projectid
FROM (
SELECT DISTINCT projectid
FROM projectsearch
WHERE searchtext LIKE '%am%'
) a
JOIN (
SELECT DISTINCT projectid
FROM projectsearch
WHERE searchtext LIKE '%49%'
) b ON a.projectid = b.projectid
It should give you back the same set of projectid values as your inner query. It gives each projectid value that has matching searchtext for both search terms, even if those terms show up in different rows of project_search. That's what your query does by searching GROUP_CONCAT() output.
Try creating an index on (searchtext, projectid). The use of column LIKE '%sample' means you won't be able to random-access that index, but the two queries in the join may still be able to scan the index, which is faster than scanning the table. To add that index use this command.
ALTER TABLE project_search ADD INDEX project_search_text (searchtext, projectid);
Try to do this in a MySQL client program (phpmyadmin for example) rather than directly from your php program.
Then, using the MySQL client, test the inner query. See how long it takes. Use EXPLAIN SELECT .... to get an explanation of how MySQL is handling the query.
It's possible your short keywords are returning a ridiculously high number of matches, and somehow overwhelming your system. In that case you can put a LIMIT 1000 clause or some such thing at the end of your inner query. That's not likely, though. 17 kilorows is not a large number.
If that doesn't help your production MySQL server is likely misconfigured or corrupt. If I were you I would call your hosting service tech support, somehow get past the front-line support agent (who won't know anything except "reboot your computer" and other such foolishness), and tell them the exact times you got the "gone away" message. They'll be able to check the logs.
Pro tip: I'm sure you know the pitfalls of using LIKE '%text%' as a search term. It's not scalable because it's not sargable: it can't random access an index. If you can possibly redesign your system, it's worth your time and effort.
You could TRY / CATCH to check if you get a more concrete error:
BEGIN TRY
BEGIN TRANSACTION
--Insert Your Queries Here--
COMMIT
END TRY
BEGIN CATCH
DECLARE #ErrorMessage NVARCHAR(4000);
DECLARE #ErrorSeverity INT;
DECLARE #ErrorState INT;
SELECT
#ErrorMessage = ERROR_MESSAGE(),
#ErrorSeverity = ERROR_SEVERITY(),
#ErrorState = ERROR_STATE();
IF ##TRANCOUNT > 0
ROLLBACK
RAISERROR (#ErrorMessage, -- Message text.
#ErrorSeverity, -- Severity.
#ErrorState -- State.
);
END CATCH
Although because you are talking about short words and fulltext it seems to me it must be related to StopWords.
Try running this query from both your dev server and production server and check if there are any differences:
SELECT * FROM INFORMATION_SCHEMA.INNODB_FT_DEFAULT_STOPWORD;
Also check in my.ini (if that is the config file) text file if these are set to:
ft_stopword_file = ""
ft_min_word_len = 1
As stated in my EDIT the problem wasnt the query from the original Question, but some other queries using the search - parameter as well. Every query had a part like follows :
if(isset($filter['search'])) {
$keywords = preg_split('/\s+/', trim($filter['search']));
$field = 1;
foreach($keywords as $keyword) {
$join = $sql->select('project_search');
$join->columns(["pid$field" => 'projectid']);
$join->where(["LOCATE('$keyword', searchtext)"]);
$join->group("projectid");
$select->join(
["m$field" => $join],
"m$field.pid$field = p.id"
);
$field++;
}
}
This resulted in alot of queries with alot of resultrows killing the mysql server eventually. I merged those Queries into the first and the error was gone.
I am using Moodle 2.9.1. I have a query to fetch the last record from the table:
My query as follows:
$qstndetails = $DB->get_record_sql('SELECT * FROM {epoll_questions} WHERE status=? AND courseid=? ORDER BY id DESC LIMIT 0,1',array(2,$curseId));
I am taking the count as
echo count($qstndetails);
I am getting the count as 1 in the case of result is there.
But in the case of result is not there also I am getting count as 1.
I had tried print_r($qstndetails) but nothing showing. But the count show as 1.
The expecting count is 0 when there is no result.
Why I am getting like this?
When developing, always have debugging switched on. This would probably have displayed an error message for the first code. Go to site admin -> development -> debugging, then debug messages = developer and switch on display debug messages.
Also LIMIT isn't an SQL standard. Moodle works with several databases so try to keep the SQL generic.
Also I would suggest using named parameters rather than ?. It makes the code easier to read but they can also be used in any order.
and finally, you can use IGNORE_MULTIPLE to get the first record.
So the code should be something like this:
$sql = "SELECT *
FROM {epoll_questions}
WHERE status = :status
AND courseid = :courseid
ORDER BY id DESC";
$params = array('status' => 2, 'courseid' => $curseid);
$qstndetails = $DB->get_record_sql($sql, $params, IGNORE_MULTIPLE);
Need to change the query to
$qstndetails = $DB->get_records_sql('SELECT * FROM {epoll_questions} WHERE status=? AND courseid=? ORDER BY id DESC LIMIT 0,1',array(2,$curseId));
Now working!!
I'm trying to get mysql query into Code Igniter's Active Record syntax but am having a bit of a hard time.
The query is as a result of this question: Multiple mysql ORDER BY's for multidimensional ordering/grouping
I've attempted to format the query myself, have tackled a couple of errors, but am unsure how to progress. I had to add in the get_compiled_select() function to DB_active_rec.php myself and change the _reset_select() from protected to public to get it to run at all.
The suggested query is:
select
t.id,
t.group,
t.date,
t.comlete
from
YourTable t
left join
(select
m.group,
min(m.date) as mindate,
min(t.complete) as groupcomplete
from
YourTable m) mt on mt.group = t.group
order by
coalesce(mt.groupcomplete, t.complete),
coalesce(mt.mindate, t.date),
t.group,
t.complete,
t.date
My translation looks like this (note that there's a 'where' clause not in the original, and that 'date' is actually 'due'):
// Sub query
$this->db->select('m.group, min(m.due) as mindate, min(t.complete) as groupcomplete');
$this->db->from('task m');
$this->db->where('property', $property);
$subquery = $this->db->get_compiled_select();
$this->db->_reset_select();
// Main query
$this->db->select('*');
$this->db->where('property', $property);
$this->db->from('task t');
$this->db->join('($subquery) mt','mt.group = t.group');
$this->db->order_by('coalesce(mt.groupcomplete, t.complete)');
$this->db->order_by('coalesce(mt.mindate, t.due)');
$this->db->order_by('t.group');
$this->db->order_by('t.complete');
$this->db->order_by('t.due');
$query = $this -> db -> get();
// Return
return $query->result();
Unfortunately this is just throwing an error:
You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'mt ON `mt`.`group` = `t`.`group` WHERE `property` = '7' ORDER BY coalesce(mt.gr' at line 3
The query as reported by CI looks like:
SELECT * FROM (`task` t) JOIN ($subquery) mt ON `mt`.`group` = `t`.`group` WHERE `property` = '7' ORDER BY coalesce(mt.groupcomplete, `t`.`complete)`, coalesce(mt.mindate, `t`.`date)`, `t`.`due`, `t`.`complete`, `t`.`date`
Anyone able to lend some advice as to how to get this formatted correctly? My mysql skills are, unfortunately, pretty bare, so this is pushing my abilities. Much of the approach of my translation is from answers on Stack Overflow, as I have no experience combining queries in this way (with the subquery).
The problem (or 'one of the problems') is here:
$this->db->join('($subquery) mt','mt.group = t.group');
You use single quotes, so the variable $subquery doesn't get expanded.
This can also be seen in the query that is outputted by CodeIgniter.
When you have multiple order by statements u separate them by comma like this
$this->db->order_by('coalesce(mt.groupcomplete, t.complete), coalesce(mt.mindate, t.date), t.due, t.complete, t.date');
I am executing the following query in a MySQL database (look at SELECT AND WHERE, the rest is not important):
SELECT distinct fname //more fields...
FROM filedepot_files AS ff
INNER JOIN filedepot_categories AS fc
ON ff.cid = fc.cid
INNER JOIN filedepot_access AS fa
ON fc.cid = fa.catid
WHERE fa.permid=$id AND fname LIKE '%$key%'
ORDER BY DATE
The environment is a PHP script running under Drupal with FileDepot module but I doubt that matters at all.
This is the PHP script (well the part that matters):
$id = 1;
$key = $_GET['key'];
$query = .... (see above)
$result = db_query($query);
while($row = db_fetch_array($result)){
//do stuff
echo $row['fname'];
}
db_query() is a Drupal method that allows to easily execute SQL queries and a returns an array, db_fetch_array() allows to parse the result.
Now, DB contains the following entries for fname (there are more, these are just examples):
Dichiarazione 1
Dichiarazione 2
Guida 1
Guida 2
If I launch the script with "guida" as key it correctly returns the two entries both with PHP and MySQL.
If i use "Guida" it works as well.
However if I use "dichiarazione" it doesnt with PHP while it does with MySQL.
Strange thing is that "Dichiarazione" works both with PHP and MySQL.
What is wrong with the query? I tryed to use LOWER(fname) LIKE '%$key%' but it doesn't seem to work as intended.
I am sure there is something stupid that I am missing but I can't seem to find what that is...
% is a special character in Drupal queries (it's used for placeholders). Try double-escaping it:
WHERE fa.permid=$id AND fname LIKE '%%$key%%'
More worryingly though, you're wide open to SQL injection. Some sanitisation is in order:
...
WHERE fa.permid= %d AND fname LIKE '%%%s%%'
...
$query = db_query($sql, $id, $key);
It might look crazy but that's the right number of % signs. Two for each literal %, and one (%s) for the string placeholder