My original error was
Error No: 1213 - Deadlock found when trying to get lock; try
restarting transaction
Okay, so I wrote a loop with max retries and a wait in between to try and get through the deadlocks.
$Try = 0;
while (!$Result = $dbs->query($MySQL)) {
$Try++;
if ($Try === MYSQL_MAX_RETRIES)
HandleMySQLError($dbs->error, $MySQL, false, $Test, $Trace);
else
sleep(MYSQL_RETRY_WAIT);
}
But now I'm constantly getting some of the original error still, and a new error
Got error 35 "Resource deadlock avoided" during COMMIT
But I can't really seem to find out what this means or how to fix it?
EDIT
I left out a ton of information when I first wrote this, but the server is a RedHat 7 AWS EC2 (well, 3 of them) in a Galera & MariaDB cluster.
The query I am running is a call to a stored procedure
call`getchatmessages`('<ChatID>','<UserID>',from_unixtime('<Some Timestamp>'));
And the stored procedure is as follows
CREATE DEFINER=`root`#`%` PROCEDURE `getchatmessages`(IN `__ChatID` CHAR(36), IN `__UserID` CHAR(36), IN `__Timestamp` TIMESTAMP(6))
BEGIN
DECLARE `__NewChatMessages` TINYINT(1) DEFAULT 0;
DECLARE `__i` INT(11) DEFAULT 0;
DECLARE `__Interval` INT(11) DEFAULT 100; -- ms
DECLARE `__Timeout` INT(11) DEFAULT 15000; -- ms
while `__NewChatMessages`=0 and `__i`<`__Timeout`/`__Interval` do
select 1 into `__NewChatMessages` from `chatmessages` where `ChatID`=`__ChatID` and `DateTimeAdded`>ifnull(`__Timestamp`,0) limit 1;
update `chatusers` set `DateTimeRead`=now(6) where `ChatID`=`__ChatID` and `UserID`=`__UserID`;
do sleep(`__Interval`/1000);
set `__i`=`__i`+1;
end while;
select `chatmessages`.`Body`, `chatmessages`.`ChatID`, `chatmessages`.`UserID`,
`chatmessages`.`ChatMessageID`, `chatmessages`.`DateTimeAdded`, UNIX_TIMESTAMP(`chatmessages`.`DateTimeAdded`) `Timestamp`, `users`.`FirstName`,
`users`.`LastName`
from `chatmessages`
join `users` using (`UserID`)
where `chatmessages`.`ChatID`=`__ChatID`
and `chatmessages`.`DateTimeAdded`>ifnull(`__Timestamp`,0)
order by `chatmessages`.`DateTimeAdded` desc
limit 100;
END
Deadlock in Galera Cluster (MariaDB Galera Cluster, 3 nodes) is not a typical deadlock, but a way of communicating the multi-master conflicts:
http://galeracluster.com/documentation-webpages/dealingwithmultimasterconflicts.html
The easiest way to avoid deadlocks is to write to 1 node at a time, i.e. configure HA proxy to write to 1 node only. In your case you will run sp on Node1 (does not matter which node, but always on 1 node, sort of "sticky sessions").
More information here: https://severalnines.com/blog/avoiding-deadlocks-galera-set-haproxy-single-node-writes-and-multi-node-reads
Is this Proc being called inside a transaction? If so, I argue strongly with its design. You have a loop with a sleep hanging onto the transaction.
Instead, have the UPDATE be a transaction by itself.
This may virtually eliminate the deadlocks. However you should still deal with deadlocks, as discussed by other answer(s).
Edit Since there are no BEGINs, and autocommit=ON, the OP is already following this advice. Alas.
Related
I have spent hours trying to get to the bottom of this and it is driving me la la.
I have a simple stored procedure that just inputs 10,000 rows into a table. This is done because it takes long enough to prove the point (about 12 seconds).
create table lala (id int not null identity(1,1), txt varchar(100))
go
CREATE PROCEDURE lalatest
AS
BEGIN
DECLARE #x int;
SET #x = 10000;
WHILE #x > 0
BEGIN
INSERT INTO lala (txt) VALUES ('blah ' + CAST(#x AS varchar(10)));
SET #x = #x - 1;
END;
END;
When I run the stored procedure from within SQL Server Management Studio, 10,000 rows are inserted into the table.
When I run it from php it just aborts after about 150 rows (the number of rows varies, suggesting a timeout issue) with no error messages or any indication that it has finished early
The remote query timeout setting is set to the default of 600 seconds, so its not that.
The php:
$sql = "exec lalatest";
sqlsrv_query($cn, $sql);
I have tried specifying a timeout value (which should be indefinite by default anyway) on the sqlsrv_query line and it made no difference.
I'm using php 5.6.7
and Microsoft SQL Server 2012 (SP3) (KB3072779) - 11.0.6020.0 (X64)
Oct 20 2015 15:36:27
Standard Edition (64-bit) on Windows NT 6.1 (Build 7601: Service Pack 1) (Hypervisor)
I have all errors and warnings turned on
Anyone got any ideas as to what I need to do to make this work?
Would you believe it, I have made it work. I added SET NOCOUNT ON as the first line in the stored procedure and now it works from php as well.
Looks like having the rows affected info throws the php sqlsrv driver a wobbly. Strange how it only does it after a certain time (about a second), perhaps something to do with when the message buffer is flushed or something. Yes, I'm guessing, but this has solved the problem, so I'm happy.
CREATE PROCEDURE lalatest
AS
BEGIN
SET NOCOUNT ON; -- must be added so that it will run from php
DECLARE #x int;
SET #x = 10000;
WHILE #x > 0
BEGIN
INSERT INTO lala (txt) VALUES ('blah ' + CAST(#x AS varchar(10)));
SET #x = #x - 1;
END;
END;
I have a weird problem.
I'm running a query:
SELECT IMIE, NAZWISKO, PESEL2, ADD_DATE, CONVERT(varchar, ADD_DATE, 121) AS XDATA, ID_ZLECENIA_XXX, * FROM XXX_KONWERSJE_HISTORIA AS EKH1
INNER JOIN XXX_DANE_PACJENTA EDP1 ON EKH1.ID_ZLECENIA_XXX=EDP1.ORDER_ID_XXX
WHERE EKH1.ID_KONWERSJE = (
SELECT MIN(ID_KONWERSJE)
FROM XXX_KONWERSJE_HISTORIA AS EKH2
WHERE EKH1.ID_ZLECENIA_XXX = EKH2.ID_ZLECENIA_XXX
)
AND EDP1.RECNO = (
SELECT MAX(RECNO)
FROM XXX_DANE_PACJENTA EDP2
WHERE EDP2.ORDER_ID_XXX = EDP1.ORDER_ID_XXX
)
AND EKH1.ID_ZLECENIA_XXX LIKE '%140000393%'
AND ADD_DATE>'20140419' AND ADD_DATE<='20140621 23:59:59.999'
ORDER BY EKH1.ID_KONWERSJE, EKH1.ID_ZLECENIA_XXX DESC
And the query works ok if I use a date limit around 2 months (63 days - it gives me 1015 results). If I extend the date limit query simply fails (Query failed blabla).
This happens under windows 64 bit php (apache, Xamp).
When I run this query directly from MS SQL SERWER Management Studio everything works fine, no matter what date limit I choose.
What is going on? Is there a limit of some kind under apache/php? (There is no information like "query time excessed", only "query failed")
And the query works ok if I use a date limit around 2 months (63 days
- it gives me 1015 results). If I extend the date limit query simply fails (Query failed blabla). ...
What is going on? Is there a limit of
some kind under apache/php? (There is no information like "query time
excessed", only "query failed")
This could happen because selectivity of ADD_DATE>'20140419' AND ADD_DATE<='20140621 23:59:59.999' is medium/low (there are [too] many rows that satisfy this predicate) and SQL Server have to scan (yes, scan) XXX_KONWERSJE_HISTORIA to many times to check following predicate:
WHERE EKH1.ID_KONWERSJE = (
SELECT ...
FROM XXX_KONWERSJE_HISTORIA AS EKH2
WHERE EKH1.ID_ZLECENIA_XXX = EKH2.ID_ZLECENIA_XXX
)
How many times have to scan SQL Server XXX_KONWERSJE_HISTORIA table to verify this predicate ? You can look at the properties of Table Scan [XXX_KONWERSJE_HISTORIA] data access operator: 3917 times
What you can do for the beginning ? You should create the missing index (see that warning with green above the execution plan):
USE [OptimedMain]
GO
CREATE NONCLUSTERED INDEX [<Name of Missing Index, sysname,>]
ON [dbo].[ERLAB_KONWERSJE_HISTORIA] ([ID_ZLECENIA_ERLAB])
INCLUDE ([ID_KONWERSJE])
GO
When I run this query directly from MS SQL SERWER Management Studio
everything works fine, no matter what date limit I choose.
SQL Server Management Studio has execution timeout set to 0 by default (no execution timeout).
Note: if this index will solve the problem then you should try (1) to create an index on ADD_DATE with all required (CREATE INDEX ... INCLUDE(...)) columns and (2) to create unique clustered indexes on these tables.
Try to set these php configurations in your php script via ini_set
ini_set('memory_limit', '512M');
ini_set('mssql.timeout', 60 * 20);
Not sure it will help you out.
I have a fairly large, complex chunk of SQL that creates a report for a particular page. This SQL has several variable declarations at the top, which I suspect is giving me problems.
When I get the ODBC result, I have to use odbc_next_result() to get past the empty result sets that seem to be returned with the variables. That seems to be no problem. When I finally get to the "real" result, odbc_num_rows() tells me it has over 12 thousand rows, when in actuality it has 6.
Here is an example of what I'm doing, to give you an idea, without going into details on the class definitions:
$report = $db->execute(Reports::create_sql('sales-report-full'));
while(odbc_num_rows($report) <= 1) {
odbc_next_result($report);
}
echo odbc_num_rows($report);
The SQL looks something like this:
DECLARE #actualYear int = YEAR(GETDATE());
DECLARE #curYear int = YEAR(GETDATE());
IF MONTH(GETDATE()) = 1
SELECT #curYear = #curYear - 1;
DECLARE #lastYear int = #curYear-1;
DECLARE #actualLastYear int = #actualYear-1;
DECLARE #tomorrow datetime = DATEADD(dd, 1, GETDATE());
SELECT * FROM really_big_query
Generally speaking it's always a good idea to start every stored procedure, or batch of commands to be executed, with the set nocount on instruction - which tells SQL Server to supress sending "rows effected" messages to a client application over ODBC (or ADO, etc.). These messages effect performance and cause empty record sets to be created in the result set - which cause additional effort for application developers.
I've also used ODBC drivers that actually error if you forget to suppress these messages - so it has become instinctive for me to type set nocount on as soon as I start writing any new stored procedure.
There are various question/answers relating to this subject, for example What are the advantages and disadvantages of turning NOCOUNT off in SQL Server queries? and SET NOCOUNT ON usage which cover many other aspects of this command.
In MySQL I have to check whether select query has returned any records, if not I insert a record. I am afraid though that the whole if-else operation in PHP scripts is NOT as atomic as I would like, i.e. will break in some scenarios, for example if another instance of the script is called where the same record needs to be worked with:
if(select returns at least one record)
{
update record;
}
else
{
insert record;
}
I did not use transactions here, and autocommit is on. I am using MySQL 5.1 with PHP 5.3. The table is InnoDB. I would like to know if the code above is suboptimal and indeed will break. I mean the same script is re-entered by two instances and the following query sequence occurs:
instance 1 attempts to select the record, finds none, enters the block for insert query
instance 2 attempts to select the record, finds none, enters the block for insert query
instance 1 attempts to insert the record, succeeds
instance 2 attempts to insert the record, fails, aborts the script automatically
Meaning that instance 2 will abort and return an error, skipping anything following the insert query statement. I could make the error not fatal, but I don't like ignoring errors, I would much rather know if my fears are real here.
Update: What I ended up doing (is this ok for SO?)
The table in question assists in a throttling (allow/deny, really) amount of messages the application sends to each recipient. The system should not send more than X messages to a recipient Y within a period Z. The table is [conceptually] as follows:
create table throttle
(
recipient_id integer unsigned unique not null,
send_count integer unsigned not null default 1,
period_ts timestamp default current_timestamp,
primary key (recipient_id)
) engine=InnoDB;
And the block of [somewhat simplified/conceptual] PHP code that is supposed to do an atomic transaction that maintains the right data in the table, and allows/denies sending message depending on the throttle state:
function send_message_throttled($recipient_id) /// The 'Y' variable
{
query('begin');
query("select send_count, unix_timestamp(period_ts) from throttle where recipient_id = $recipient_id for update");
$r = query_result_row();
if($r)
{
if(time() >= $r[1] + 60 * 60 * 24) /// The numeric offset is the length of the period, the 'Z' variable
{/// new period
query("update throttle set send_count = 1, period_ts = current_timestamp where recipient_id = $recipient_id");
}
else
{
if($r[0] < 5) /// Amount of messages allowed per period, the 'X' variable
{
query("update throttle set send_count = send_count + 1 where recipient_id = $recipient_id");
}
else
{
trigger_error('Will not send message, throttled down.', E_USER_WARNING);
query('rollback');
return 1;
}
}
}
else
{
query("insert into throttle(recipient_id) values($recipient_id)");
}
if(failed(send_message($recipient_id)))
{
query('rollback');
return 2;
}
query('commit');
}
Well, disregarding the fact that InnoDB deadlocks occur, this is pretty good no? I am not pounding my chest or anything, but this is simply the best mix of performance/stability I can do, short of going with MyISAM and locking entire table, which I don't want to do because of more frequent updates/inserts vs selects.
It seems like you already know the answer to the question, and how to solve your problem. It is a real problem, and you can use one of the following to solve it:
SELECT ... FOR UPDATE
INSERT ... ON DUPLICATE KEY UPDATE
transactions (don't use MyIsam)
table locks
This can and might happen depending on how often this page is executed.
The safe bet would be to use transactions. The same thing you wrote would still happen, except that you could safely check for the error inside the transaction (In case the insertion involves several queries, and only the last insert breaks) allowing you to rollback the one that became invalid.
So (pseudo):
#start transaction
if (select returns at least one record)
{
update record;
}
else
{
insert record;
}
if (no constraint errors)
{
commit; //ends transaction
}
else
{
rollback; //ends transaction
}
You could lock the table as well but depending on the work you're doing, you'd have to get an exclusive lock on the entire table (you cannot SELECT ... FOR UPDATE non-existing rows, sorry) but that would also block reads from your table until you are finished.
In a PHP script working with a mysql database, I recently had the need to use a transaction at a point that happened to be inside another transaction. All my tests seem to indicate this is working out fine, but I can't find any documentation on this usage.
I want to be sure - are transactions within transactions valid in mysql? If so, is there a way to find out how many levels deep you are in nested transactions? (ie. how many rollbacks it would take to return to normal)
Thanks in advance,
Brian
Contrary to everyone else's answer, you can effectively create transactions within transactions and it's really easy. You just create SAVEPOINT locations and use ROLLBACK TO savepoint to rollback part of the transaction, where savepoint is whatever name you give the savepoint.
Link to MySQL documentation: http://dev.mysql.com/doc/refman/5.0/en/savepoint.html
And of course, none of the queries anywhere in the transaction should be of the type that implicitly commit, or the whole transaction will be committed.
Examples:
START TRANSACTION;
# queries that don't implicitly commit
SAVEPOINT savepoint1;
# queries that don't implicitly commit
# now you can either ROLLBACK TO savepoint1, or just ROLLBACK to reverse the entire transaction.
SAVEPOINT savepoint2;
# queries that don't implicitly commit
# now you can ROLLBACK TO savepoint1 OR savepoint2, or ROLLBACK all the way.
# e.g.
ROLLBACK TO savepoint1;
COMMIT; # results in committing only the part of the transaction up to savepoint1
In PHP I have written code like this, and it works perfectly:
foreach($some_data as $key => $sub_array) {
$result = mysql_query('START TRANSACTION'); // note mysql_query is deprecated in favor of PDO
$rollback_all = false; // set to true to undo whole transaction
for($i=0;$i<sizeof($sub_array);$i++) {
if($sub_array['set_save'] === true) {
$savepoint = 'savepoint' . $i;
$result = mysql_query("SAVEPOINT $savepoint");
}
$sql = 'UPDATE `my_table` SET `x` = `y` WHERE `z` < `n`'; // some query/queries
$result = mysql_query($sql); // run the update query/queries
$more_sql = 'SELECT `x` FROM `my_table`'; // get data for checking
$result = mysql_query($more_sql);
$rollback_to_save = false; // set to true to undo to last savepoint
while($row = mysql_fetch_array($result)) {
// run some checks on the data
if( /*some check says to go back to savepoint */) {
$rollback_to_save = true; // or just do the rollback here.
}
if( /* some check says to rollback entire transaction */ ) {
$rollback_all = true;
}
}
if($rollback_all === true) {
mysql_query('ROLLBACK'); // rollback entire transaction
break; // break out of for loop, into next foreach
}
if($rollback_to_save = true) {
mysql_query("ROLLBACK TO $savepoint"); // undo just this part of for loop
}
} // end of for loop
mysql_query('COMMIT'); // if you don't do this, the whole transaction will rollback
}
This page of the manual might interest you : 12.3.3. Statements That Cause an Implicit Commit; quoting a few sentences :
The statements listed in this section
(and any synonyms for them) implicitly
end a transaction, as if you had done
a COMMIT before executing the
statement.
And, a bit farther in the page :
Transaction-control and locking
statements. BEGIN, LOCK TABLES,
SET autocommit = 1 (if the value is
not already 1), START TRANSACTION,
UNLOCK TABLES.
See also this paragraph :
Transactions cannot be nested. This is
a consequence of the implicit commit
performed for any current transaction
when you issue a START TRANSACTION
statement or one of its synonyms.
I want to be sure - are transactions
within transactions valid in mysql?
No.
MySql doesn't support nested transactions. There are a few ways that you can emulate it though. First, you can use savepoints as a form of transaction, so that gives you two levels of transactions; I've used this for testing, but I'm not sure about the limitations, if you use it in production code. A simpler solution is to ignore the second begin transaction and instead increase a counter. For each commit, you decrease it. Once you hit zero, you do an actual commit. There are obvious limitations of this; Eg. a rollback will roll all transactions back, but for a case where you only use transactions for error-handling, that may be acceptable.
There are some great answers in this thread, however, if you use innoDB as your MySQL storage engine and are using MySQL 5.0.3 or higher, you get nested transactions right out of the box without the need for any extra work on your part or any of the fancy techniques described by others in this thread.
From the MySQL docs on XA Transactions:
MySQL 5.0.3 and up provides server-side support for XA transactions. Currently, this support is available for the InnoDB storage engine. The MySQL XA implementation is based on the X/Open CAE document Distributed Transaction Processing: The XA Specification. This document is published by The Open Group and available at http://www.opengroup.org/public/pubs/catalog/c193.htm. Limitations of the current XA implementation are described in Section E.5, “Restrictions on XA Transactions”.
My XA Transaction Example Just For You:
# Start a new XA transaction
XA START;
# update my bank account balance, they will never know!
UPDATE `bank_accounts` SET `balance` = 100000 WHERE `id` = 'mine';
# $100,000.00 is a bit low, I'm going to consider adding more, but I'm not sure so
# I will start a NESTED transaction and debate it...
XA START;
# max int money! woo hoo!
UPDATE `bank_accounts` SET `balance` = 2147483647 WHERE `id` = 'mine';
# maybe thats too conspicuous, better roll back
XA ROLLBACK;
# The $100,000 UPDATE still applies here, but the max int money does not, going for it!
XA COMMIT;
# Oh No! Sirens! It's the popo's!!! run!!
# What the hell are they using ints for money columns anyway! Ahhhh!
MySQL Documentation For XA Transactions:
13.3.7. XA
Transactions
13.3.7.1. XA Transaction SQL
Syntax
13.3.7.2. XA Transaction States
E.6. Restrictions on XA Transactions
I <3 XA Transactions 4 Eva!
You might want to check your testing methadology. Outside of MaxDB, MySQL doesn't support anything remotely like nested transactions.
It does:
http://dev.mysql.com/doc/refman/5.0/en/xa.html