I have a fairly large, complex chunk of SQL that creates a report for a particular page. This SQL has several variable declarations at the top, which I suspect is giving me problems.
When I get the ODBC result, I have to use odbc_next_result() to get past the empty result sets that seem to be returned with the variables. That seems to be no problem. When I finally get to the "real" result, odbc_num_rows() tells me it has over 12 thousand rows, when in actuality it has 6.
Here is an example of what I'm doing, to give you an idea, without going into details on the class definitions:
$report = $db->execute(Reports::create_sql('sales-report-full'));
while(odbc_num_rows($report) <= 1) {
odbc_next_result($report);
}
echo odbc_num_rows($report);
The SQL looks something like this:
DECLARE #actualYear int = YEAR(GETDATE());
DECLARE #curYear int = YEAR(GETDATE());
IF MONTH(GETDATE()) = 1
SELECT #curYear = #curYear - 1;
DECLARE #lastYear int = #curYear-1;
DECLARE #actualLastYear int = #actualYear-1;
DECLARE #tomorrow datetime = DATEADD(dd, 1, GETDATE());
SELECT * FROM really_big_query
Generally speaking it's always a good idea to start every stored procedure, or batch of commands to be executed, with the set nocount on instruction - which tells SQL Server to supress sending "rows effected" messages to a client application over ODBC (or ADO, etc.). These messages effect performance and cause empty record sets to be created in the result set - which cause additional effort for application developers.
I've also used ODBC drivers that actually error if you forget to suppress these messages - so it has become instinctive for me to type set nocount on as soon as I start writing any new stored procedure.
There are various question/answers relating to this subject, for example What are the advantages and disadvantages of turning NOCOUNT off in SQL Server queries? and SET NOCOUNT ON usage which cover many other aspects of this command.
Related
My original error was
Error No: 1213 - Deadlock found when trying to get lock; try
restarting transaction
Okay, so I wrote a loop with max retries and a wait in between to try and get through the deadlocks.
$Try = 0;
while (!$Result = $dbs->query($MySQL)) {
$Try++;
if ($Try === MYSQL_MAX_RETRIES)
HandleMySQLError($dbs->error, $MySQL, false, $Test, $Trace);
else
sleep(MYSQL_RETRY_WAIT);
}
But now I'm constantly getting some of the original error still, and a new error
Got error 35 "Resource deadlock avoided" during COMMIT
But I can't really seem to find out what this means or how to fix it?
EDIT
I left out a ton of information when I first wrote this, but the server is a RedHat 7 AWS EC2 (well, 3 of them) in a Galera & MariaDB cluster.
The query I am running is a call to a stored procedure
call`getchatmessages`('<ChatID>','<UserID>',from_unixtime('<Some Timestamp>'));
And the stored procedure is as follows
CREATE DEFINER=`root`#`%` PROCEDURE `getchatmessages`(IN `__ChatID` CHAR(36), IN `__UserID` CHAR(36), IN `__Timestamp` TIMESTAMP(6))
BEGIN
DECLARE `__NewChatMessages` TINYINT(1) DEFAULT 0;
DECLARE `__i` INT(11) DEFAULT 0;
DECLARE `__Interval` INT(11) DEFAULT 100; -- ms
DECLARE `__Timeout` INT(11) DEFAULT 15000; -- ms
while `__NewChatMessages`=0 and `__i`<`__Timeout`/`__Interval` do
select 1 into `__NewChatMessages` from `chatmessages` where `ChatID`=`__ChatID` and `DateTimeAdded`>ifnull(`__Timestamp`,0) limit 1;
update `chatusers` set `DateTimeRead`=now(6) where `ChatID`=`__ChatID` and `UserID`=`__UserID`;
do sleep(`__Interval`/1000);
set `__i`=`__i`+1;
end while;
select `chatmessages`.`Body`, `chatmessages`.`ChatID`, `chatmessages`.`UserID`,
`chatmessages`.`ChatMessageID`, `chatmessages`.`DateTimeAdded`, UNIX_TIMESTAMP(`chatmessages`.`DateTimeAdded`) `Timestamp`, `users`.`FirstName`,
`users`.`LastName`
from `chatmessages`
join `users` using (`UserID`)
where `chatmessages`.`ChatID`=`__ChatID`
and `chatmessages`.`DateTimeAdded`>ifnull(`__Timestamp`,0)
order by `chatmessages`.`DateTimeAdded` desc
limit 100;
END
Deadlock in Galera Cluster (MariaDB Galera Cluster, 3 nodes) is not a typical deadlock, but a way of communicating the multi-master conflicts:
http://galeracluster.com/documentation-webpages/dealingwithmultimasterconflicts.html
The easiest way to avoid deadlocks is to write to 1 node at a time, i.e. configure HA proxy to write to 1 node only. In your case you will run sp on Node1 (does not matter which node, but always on 1 node, sort of "sticky sessions").
More information here: https://severalnines.com/blog/avoiding-deadlocks-galera-set-haproxy-single-node-writes-and-multi-node-reads
Is this Proc being called inside a transaction? If so, I argue strongly with its design. You have a loop with a sleep hanging onto the transaction.
Instead, have the UPDATE be a transaction by itself.
This may virtually eliminate the deadlocks. However you should still deal with deadlocks, as discussed by other answer(s).
Edit Since there are no BEGINs, and autocommit=ON, the OP is already following this advice. Alas.
I have a query that uses stuff() when using it in PHP it fails due to it running out of memory.
the error occurs on this line;
$data = odbc_exec($conn, $query);
at this point php doesn't actually receive any data it is just assigning a network resource. the connection is fine, i use it everywhere else, and the query strlen is only ~400.
so what is actually going on here?
select oeename.id, oeename.Name, stuff(
(select distinct ', ' + departmentequipment.name from type join DepartmentEquipment on departmentequipmentfk = departmentequipment.id where type.OEENameFK = oeename.ID order by ', ' + departmentequipment.name for xml path('')) ,1,1,'') as departmentequipmentnames
from oeename order by oeename.name asc
even when i add top 1 to the query the query still exceeds the memory limit.
EDIT: the php script was working before, someone had detected a bug in a somewhat related part of my website and upon attempting to fault find the issue i came across this problem on my local development server. the table is now partly larger, but its failing on exec rather than reading the data in.
I believe you're running into this bug: https://bugs.php.net/bug.php?id=68964
To summarize the workaround, you need to cast any ntext fields to nvarchar(max) instead.
Basically, I need to get some data out of a SQL Server 2005, using PHP.
Previously we used the mssql_* functions, however due to various server issues we are only now able to use odbc_* functions.
The table has various columns whose names have spaces in them, and before it is suggested.... no I cannot change them as this is a totally separate piece of software in another language and it would break it, I am just getting stats out of it.
Anywho, I had previously been accessing these columns by putting their names in square brackets, e.g. [column name] and it worked fine under the mssql_* functions, however when I do this:
$sql = "select top 1 [assessment name] as AssessmentName from bksb_Assessments";
$result = odbc_exec($db, $sql);
while($row = odbc_fetch_array($result))
{
var_dump($row);
}
It prints the results out as:
'assessment name' => string 'Mathematics E3 Diagnostic' (length=25)
So as you can see it is totally ignoring the alias I've given it and still calling it [assessment name].
But if I run the exact same thing in SQL Server Management Studio Express, it works fine and uses the alias.
I've tried various different combinations, such as quoting the alias, quoting the column, different brackets, etc... but no luck so far.
This isn't a huge issue, as I can change what my script uses to look for "assessment name" in the result array instead of the alias, but it's just a bit annoying that I couldn't work out why this was happening...
Cheers! :)
EDIT:
Actually i don't think the square brackets make a difference, just trying to alias any column isn't working with through php odbc, however I can still do something like CAST(whatever) AS 'alias' and that works fine... just not selecting a column as an alias...? :/
This is by no means a perfect solution, and I would love to learn the proper way to deal with this issue, but in the mean time, add +'' to each column, after the underlying column name, but before the AS alias and you should be able to circumvent this issue.
While attempting to troubleshoot this issue and determine why some of my team's stored procedures exhibited this behavior, I finally discovered why. What's strange is that we were only experiencing this issue with a few stored procedures, while most returned the the aliases as expected.
It appears that declaring at least one variable, whether or not it is being used, will prevent this particular bug from occurring. The variable can be of any type.
If you're just executing a SELECT query without using a stored procedure, use a DECLARE statement to define a variable before or after the statement:
// This works...
DECLARE #ignore TINYINT; SELECT EmployeeID AS employee_id FROM Employee;
// So does this...
SELECT EmployeeID AS employee_id FROM Employee; DECLARE #ignore TINYINT;
Now if you're running into this issue with your stored procedures, you can take the same approach above by defining a variable anywhere in the body.
One other thing I noticed is that if the stored procedure is defined with parameters, if your the parameter gets set or evaluated in something like an if statement, this bug will also not occur. For example,
// This works...
CREATE PROC get_employee(
#employee_id VARCHAR(16) = NULL
)
AS
IF (#employee_id IS NOT NULL)
BEGIN
SELECT FirstName AS first_name FROM Employee WHERE EmployeeID = #employee_id;
END
GO
Using SET:
// So does this...
CREATE PROC get_employee(
#employee_id VARCHAR(16) = NULL,
#ignore TINYINT
)
AS
SET #ignore = NULL;
SELECT FirstName AS first_name FROM Employee WHERE EmployeeID = #employee_id;
GO
I still don't quite understand the cause or nature of the bug; but these workarounds, similar to the type-casting hack above, appear to get around this annoying bug.
It's probably also worth mentioning that we didn't encounter this issue in Ubuntu 14.04 and PHP 5.4. As soon as we upgraded to Ubuntu 18.04 and PHP 7.2, that was when we began experiencing this issue. In both instances, we do have FreeTDS configured to use version 7.1, which is why we're baffled by this particular bug.
#dearsina's approach in fact seems to be one of the few ways to handle this bug.
However, this solution is not applicable to all datatypes. So for me, the only way to cover all needed columns was to CAST() the relevant column (where initially c.State would be of datatype bit):
SELECT
CAST(c.State AS nvarchar) AS 'customer_state'
FROM
customers AS c;
as else you might get an error message similar to
The data types bit and varchar are incompatible in the add operator.
Reference on CASTing large data sets: https://social.msdn.microsoft.com/Forums/sqlserver/en-US/643b6eda-fa03-4ad3-85a1-051f02097c7f/how-much-do-cast-statements-affect-performance?forum=transactsql
Some further finding on this topic is a recently (November 2017) raised bug report at bugs.php.net claiming that the problem is related to FreeTDS: https://bugs.php.net/bug.php?id=75534.
This bug report contains an inofficial (and at least from my end untested) patch:
diff --git a/ext/odbc/php_odbc_includes.h b/ext/odbc/php_odbc_includes.h
index 93e2c96..8451bcd 100644
--- a/ext/odbc/php_odbc_includes.h
+++ b/ext/odbc/php_odbc_includes.h
## -292,18 +292,16 ## void odbc_sql_error(ODBC_SQL_ERROR_PARAMS);
#define PHP_ODBC_SQLCOLATTRIBUTE SQLColAttribute
#define PHP_ODBC_SQLALLOCSTMT(hdbc, phstmt) SQLAllocHandle(SQL_HANDLE_STMT, hdbc, phstmt)
-
-#define PHP_ODBC_SQL_DESC_NAME SQL_DESC_NAME
#else
#define IS_SQL_LONG(x) (x == SQL_LONGVARBINARY || x == SQL_LONGVARCHAR)
#define PHP_ODBC_SQLCOLATTRIBUTE SQLColAttributes
#define PHP_ODBC_SQLALLOCSTMT SQLAllocStmt
-
-#define PHP_ODBC_SQL_DESC_NAME SQL_COLUMN_NAME
#endif
#define IS_SQL_BINARY(x) (x == SQL_BINARY || x == SQL_VARBINARY || x == SQL_LONGVARBINARY)
+#define PHP_ODBC_SQL_DESC_NAME SQL_DESC_LABEL
+
PHP_ODBC_API ZEND_EXTERN_MODULE_GLOBALS(odbc)
#define ODBCG(v) ZEND_MODULE_GLOBALS_ACCESSOR(odbc, v)
Faced similar problem. The 4th parameter of odbc_connect did the trick in my case choosing cursor type SQL_CUR_USE_ODBC.
Helo,
I have a stored procedure that has 7 IN parameters and 3 OUT parameters.
I need to pass 7 parameters IN from PHP, execute the query with procedure, and retrieve the 3 OUT parameters.
I am using mysqli with persistent connection mode enabled. (For load considerations)
What is the most efficient way to execute the query and get results?
I need something that doesn't affect other concurrently running PHP scripts, and that cleans the result itself, and that is straightforward.
This is what my application is (simplified) (not a working example, just how i wish it was)
$inParam1 = 'wer';
$inParam2 = 'fewf';
$inParam3 = 'dsf';
$inParam4 = 'vccv';
$inParam5 = '34t3';
$inParam6 = 'ter';
$inParam7 = 'ert';
$query = "CALL my_procedure('$inParam1', '$inParam2', '$inParam3', '$inParam4', '$inParam5', '$inParam6', '$inParam7', #outParam8, #outParam9, #outParam10); SELECT #outParam8, #outParam9, #outParam10;";
$result = $mysql_query($query);
list($outParam1, $outParam2, $outParam3) = mysql_fetch_array($result);
echo $outParam1; // String param #1 that my procedure returned as an OUT variable
echo $outParam2; // String param #2 that my procedure returned as an OUT variable
echo $outParam3; // String param #3 that my procedure returned as an OUT variable
If somebody could show how this code could look in reality, please please would be great!
I am obviously using mysqli with proper connection, and such, but the examples I have found in internet are really confusing and seem to be inefficient, I am also worried if it will conflict with other clients, because it works like "nextResult" and other some strange functions.
Many thanks!
PHP 5.3, MySQL 5.5
Try looking here. As im not overly familiar with this. :)
http://www.mysqltutorial.org/stored-procedures-parameters.aspx
You need to create a query first. This query will then be stored in the database as a callable procedure. Then later using whatever language you can call the procedure.
DELIMITER //
CREATE PROCEDURE GetUserByCountry(IN countryName VARCHAR(255))
BEGIN
SELECT name, email
FROM users
WHERE country = countryName;
END //
DELIMITER ;
Then calling it.
CALL GetUserByCountry('mexico')
Returns all users names and emails who live in mexico.
I believe if you want to create a dynamic query string such as yours, you need to put {} around your variables in the string.
$query = "CALL my_procedure('{$inParam1}', '{$inParam2'}, '{$inParam3}', '{$inParam4}', '{$inParam5}', '{$inParam6}', '{$inParam7}', #outParam8, #outParam9, #outParam10); SELECT #outParam8, #outParam9, #outParam10;";
I would like to group multiple queries into a single function that lives in PostgreSQL. The function will be queried using PDO.
The function is:
CREATE OR REPLACE FUNCTION "test_multipe_refcursor"()
RETURNS SETOF refcursor AS $BODY$
DECLARE
parentRC refcursor;
childRC refcursor;
BEGIN
open parentRC FOR
SELECT * FROM parent;
RETURN NEXT parentRC;
open childRC FOR
SELECT * FROM child;
RETURN NEXT childRC;
RETURN;
END;$BODY$
LANGUAGE 'plpgsql' VOLATILE;
ALTER FUNCTION "test_multipe_refcursor"() OWNER TO postgres;
Here's the PHP code. "Database" as a singleton class that sets up the usual connection properties, nothing special.
$database = Database::load();
$sql = "select * from test_multipe_refcursor();";
$p = $database->query($sql);
$i = 1;
do
{
$this->set('set' . $i, $p->fetchAll(PDO::FETCH_ASSOC));
$i++;
} while ($p->nextRowset());
$p->closeCursor();
And the result.
PDOException: SQLSTATE[IM001]: Driver does not support this function: driver does not support multiple rowsets in xxxx.php on line 32
This would seem to indicate that it's not supported, but then again, I cannot find a list defining exactly what is.
Has anyone managed to get this working?
References:
http://www.sitepoint.com/forums/showthread.php?p=3040612#post3040612
PostgreSQL function returning multiple result sets
http://ca.php.net/manual/en/pdostatement.nextrowset.php
Support for returning multiple resultsets is still on the PostgreSQL todo list and it will definitely not hit 8.4. As for the setof refcursors method, What you are trying to do doesn't work because the function isn't returning multiple rowsets - it is returning one rowset of refcursors. I'm not sure if using refcursors client side works, but I don't find it likely, even if the client-server protocol supports it, it is unlikely that PDO has an API for that.
But why are you trying to return multiple resultsets in one query? You can always do the queries separately.
Near the bottom of this PostgreSQL doc page, there is a section describing how you can pass back one or more cursors from a function. Basically, you get the caller to specify the name of the cursor(s) as parameters:
CREATE FUNCTION myfunc(refcursor, refcursor) RETURNS SETOF refcursor AS $$
BEGIN
OPEN $1 FOR SELECT * FROM table_1;
RETURN NEXT $1;
OPEN $2 FOR SELECT * FROM table_2;
RETURN NEXT $2;
END;
$$ LANGUAGE plpgsql;
-- need to be in a transaction to use cursors.
BEGIN;
SELECT * FROM myfunc('a', 'b');
FETCH ALL FROM a;
FETCH ALL FROM b;
COMMIT;
The page is for PostgreSQL 8.4, but this documentation snippet is present at least as far back as 8.1 (the version I'm running). As the comment says, you need to be inside a transaction to use cursors, as they are implicitly closed at the end of each transaction (i.e. at the end of every statement if autocommit mode is on).