I have a long-running script that seems to occasionally report the following NOTICE-level error:
pg_send_query(): Cannot set connection to blocking mode
It seems to continue to send queries afterward, but it's unclear if it successfully sends the query that generates the error.
What is this a symptom of?
Edit: There are no entries in the postgres log at the time the error occurred, suggesting this is solely a connection error, not something going wrong on postgres' side (e.g. probably not the result of postgres crashing and restarting or something)
Edit: As far as I can tell, my INSERT statements are succeeding, one way or another, when this error is triggered.
Edit: Looks like this may have been fixed in June 2013: https://bugs.php.net/bug.php?id=65015
It is a symptom of pg_send_query() not being able to successfully switch the connection back to blocking mode. Looking at the source code in PHPs pgsql.c, you can find:
/* {{{ proto bool pg_send_query(resource connection, string query)
Send asynchronous query */
PHP_FUNCTION(pg_send_query)
{
<... snipped function setup stuff ...>
if (PQ_SETNONBLOCKING(pgsql, 1)) {
php_error_docref(NULL TSRMLS_CC, E_NOTICE, "Cannot set connection to nonblocking mode");
RETURN_FALSE;
}
<... snipped main function execution stuff ...>
if (PQ_SETNONBLOCKING(pgsql, 0)) {
php_error_docref(NULL TSRMLS_CC, E_NOTICE, "Cannot set connection to blocking mode");
}
RETURN_TRUE;
}
So the error gets raised at the end of the function, after the main work is done. This fits with your observation that your INSERT statements get executed.
The whole purpose of the two PQ_SETNONBLOCKING calls is to put the connection in non blocking mode to allow asynchronous execution and putting it back to the default blocking behaviour afterwards. From the documentation of PQsetnonblocking: (PQ_SETNONBLOCKING is just an alias defined for that function):
Sets the nonblocking status of the
connection.
int PQsetnonblocking(PGconn *conn, int arg);
Sets the state of the connection to nonblocking if arg is 1,
or blocking if arg is 0. Returns 0 if
OK, -1 if error.
In the nonblocking state, calls to
PQsendQuery, PQputline, PQputnbytes,
and PQendcopy will not block but
instead return an error if they need
to be called again.
Note that PQexec does not honor
nonblocking mode; if it is called, it
will act in blocking fashion anyway.
Looking further at the source of PQsetnonblocking (in PostgeSQLs fe-exec.c), there are two possible reasons why the call could fail:
/* PQsetnonblocking:
* sets the PGconn's database connection non-blocking if the arg is TRUE
* or makes it non-blocking if the arg is FALSE, this will not protect
* you from PQexec(), you'll only be safe when using the non-blocking API.
* Needs to be called only on a connected database connection.
*/
int
PQsetnonblocking(PGconn *conn, int arg)
{
bool barg;
if (!conn || conn->status == CONNECTION_BAD)
return -1;
barg = (arg ? TRUE : FALSE);
/* early out if the socket is already in the state requested */
if (barg == conn->nonblocking)
return 0;
/*
* to guarantee constancy for flushing/query/result-polling behavior we
* need to flush the send queue at this point in order to guarantee proper
* behavior. this is ok because either they are making a transition _from_
* or _to_ blocking mode, either way we can block them.
*/
/* if we are going from blocking to non-blocking flush here */
if (pqFlush(conn))
return -1;
conn->nonblocking = barg;
return 0;
}
So either the connection got lost somehow, or pqFlush did not finish successfully, indicating leftover stuff in the connection output buffer.
The first case would be harmless, as your script would certainly notice the lost connection for later calls and react to that (or fail more noticeable).
This leaves the second case, which would mean you have a connection in the non default, non blocking state. I do not know if this could affect later calls that would reuse this connection. If you want to play it safe, you'd close the connection in this case and use a new/other one.
It sounds like you're trying to use the pg_send_query() function for sending asynchronous queries to PostgreSQL. The purpose of this function is to allow your PHP script to continue executing other code while waiting for PostgreSQL to execute your query and make a result ready.
The example given in the docs for pg_send_query() suggest that you shouldn't send a query if PostgreSQL is already chewing on another query:
if (!pg_connection_busy($dbconn)) {
pg_send_query($dbconn, "select * from authors; select count(*) from authors;");
}
Is there a reason you're using pg_send_query() instead of pg_query()? If you can allow your script to block waiting for query execution, I'm guessing (admittedly without having tried it) that you won't see these errors.
I've recently had the same problem, and with the help from Henrik Opels answer realized that PHP does not actually wait for the buffer to flush before setting the connection back to blocking mode.
The 'cannot set connection to blocking mode' is trivially repeatable with large enough queries to fill the send buffer (padding with spaces at the end is enough). With smaller queries I imagine it is dependent on load, and rather intermittent.
if you do actually need asynchronous mode then try the patch at https://bugs.php.net/bug.php?id=65015
This could occur if you are using threads and the connection is being reused.
If is this the case you could use the PGSQL_CONNECT_FORCE_NEW like this:
pg_connect("...", PGSQL_CONNECT_FORCE_NEW)
This will force a new database connection resource but be advised: you could run out of connections clients, so be carefully using this inside threads so don't forget to use pg_close().
I encountered same error message with PHP 5.6.9
It occurs when persistent connection made by pg_pconnect() is lost and
pgsql.auto_reset_persistent is set to Off.
Connection might get lost when:
PHP Session expires
Connection to DB timeouts
Webserver / DB server is restarted
You can check PHP.ini for pgsql.auto_reset_persistent and set it to On.
With pgsql.auto_reset_persistent enabled, each time pg_pconnect() is being called, connection link is checked, if it is still valid. This requires a little overhead, but fixies error message when conncetion is lost.
I got that error too. I solve my problem by restarting the web server (Apache).
Related
When trying to write to the client, the message is getting buffered, and in some cases, it's not being written at all.
CURRENT STATUS:
When I telnet into the server, the Server Ready: message is readily printed as expected.
When I send random data (other than "close"), the server's terminal nicely shows progress every second, but the clients output waits until after all the sleeping, and then prints it all at once.
Most importantly, when sending "close", it just waits the obligatory second, and then closes without ANY writeout in the client.
GOAL:
My main goal is for a quick message to be written to the client prior closing a connection.
CODE:
// server.php
$loop = React\EventLoop\Factory::create();
$socket = new React\Socket\Server($loop);
$socket->on('connection', function ($conn)
{
$conn->write("Server ready:\n");
$conn->on('data', function ($data) use ($conn)
{
$data = trim($data);
if( $data == 'close')
{
$conn->write("Bye\n");
sleep(1);
$conn->close();
}
for ($i = 1; $i<5; $i++) {
$conn->write(". ");
echo '. ';
sleep(1);
}
$conn->write(".\n");
echo ".\n";
$conn->write("You said \"".$data."\"\n");
});
});
$socket->listen(1337, '127.0.0.1');
$loop->run();
SUMMARY:
Why can't I get anything written to the client before closing?
The problem your are encountering is because you are forgetting about the event loop that drives ReactPHP. I ran into this issue recently when building a server and after following the code around, I found 2 things out that should help you solve your problem.
If you close the connection after writing to it, it simply closes the connection before it can write. Solving this issue can help you fix the next issue... The correct call for writing something to the client, THEN closing the connection is called $conn->end('msg'); If you follow this chain of code the behaviour becomes clear; First it basically writes to the connection just as if you ran $conn->write('msg'); but then it registers a new event handler for the full-drain event, this handler simple calls $conn->close();; the full-drain event is only dispatched when the write buffer is completely emptied. So you can see that the use of end, simply waits to write before it closes the connection.
The drain and full-drain are only dispatched after writing to a stream. full-drain occurs when the write buffer is completely empty. drain is dispatched after the write buffer is emptied past its softLimit which by default is 2048 bytes.
The reason your writes are not making it through is because $conn->write('msg') only adds the string to the write buffer; it does not actually write. The event loop needs to run before it will be given time to write. Your use of sleep() is incorrect with react because this blocks the call at that line. In react you don't want to block any code from executing. If you are done a task, then let your function return, and code execution returns to the react main event loop. If you wish to have things run on certain intervals, or simply on the next iteration of the loop, you can schedule callbacks for the main event loop with the use of $loop->addTimer($seconds, $callback), $loop->addPeriodicTimer($seconds, $callback), $loop->nextTick($callback) or $loop->futureTick($callback)
Ultimately it is because you are programming without acknowledging that we are still in a blocking thread. Anything your code that blocks, blocks the entire react event loop, in turn blocking everything that react does for you. Give up processing time back to the loop to ensure it can do the reads/writes that you have queued up. You only need on iteration of the loop to occur for the write buffer to begin emptying (depending on the size of the buffer it may or may not write it all out)
If you're goal here is just to fix the connection closing bit, switch your call to $conn->end('msg') instead of the write -> close. However I believe that the other loop you have printing the dots also does not behave in the way that I think you expect/desire it to work. As it's purpose is not as clear, if you can tell me what your goal was for it I can possibly help you restructure that step as well.
Trying mongodb global timeout etc. is still ignored by find() queries in my PHP script.
I'd like a findOne({...}) or find({...}) lookup and wait max 20ms for the DB server before timeout.
How to make sure that PHP does not utilize this setting as soft limit? It's still ignored and processing answers even 5sec later.
Is this a PHP mongo driver bug?
Example:
MongoCursor::$timeout=20;
$nosql_server=new Mongo('mongodb://user:pw#'.implode(",",$arr_replicas).'',array("replicaSet" => "gmt","timeout"=>10)) OR troubles("too slow to connect");
$nosql_db=$nosql_server->selectDB('aDB');
$nosql_collection_mcol=$nosql_db->mcol;
$testFind=$nosql_collection_mcol->find(array('crit'=>123));
//If PHP considered the MongoCursor::$timeout, I'd expect the prev. line to be skipped or throwing a mongo/timeout exception if DB does not return the find result cursor ready within 20ms.
//However, I arrive with this line after seconds, without exception whenever the DB has some lock or delay, without skipping previous line.
In the PHP documentation for $timeout the following is the explanation for the cursor timeout:
Causes methods that fetch results to throw a
MongoCursorTimeoutException if the query takes longer than the
specified number of milliseconds.
I believe that the timeout is referring to the operations performed on the cursor (e.g. getNext()).
Do not do this:
MongoCursor::$timeout=20;
That is calling a static method and won't do you any good AFAIK.
What you need to realize is that in your code example, $testFind is the MongoCursor object. Therefore in the code snippet you gave, what you should do is add this after everything else in order to set the timeout of the $testFind MongoCursor:
$testFind->timeout(100);
NOTE: If you want to deal with $testFind as an an array you need to do:
$testFindArray = iterator_to_array($testFind);
That one threw me for a loop for awhile. Hope this helps someone.
Pay attention on the readPreference attribute. The possible values are:
MongoClient::RP_PRIMARY
MongoClient::RP_PRIMARY_PREFERRED
MongoClient::RP_SECONDARY
MongoClient::RP_SECONDARY_PREFERRED
MongoClient::RP_NEAREST
I'm interested in using the mark as read method for emails and in-app notifications as specified here
I'm using Symfony 1.4 and I can't find anything within sfContext or sfWebRequest that can be used to detect if the connection has been terminated. Is there anyway to do this within the Symfony framework?
Would I be able to use the PHP functions connection_status() or connection_aborted()?
In a typical PHP script, the script execution will stop at the next tickable event after the user terminates a connection.
Because of this, you shouldn't need to really do anything to get this to work (unless you have changed this setting in your php.ini) or someone has set the ignore_user_abort flag.
But, if you want to be safe, you could do the following:
<?php
// Whatever you need here
sleep(5);
if (false == connection_aborted()) {
// Mark the notification as read
}
I have noticed a few websites such as hypem.com show a "You didnt get served" error message when the site is busy rather than just letting people wait, time out or refresh; aggravating what is probably a server load issue.
We are too loaded to process your request. Please click "back" in your
browser and try what you were doing again.
How is this achieved before the server becomes overloaded? It sounds like a really neat way to manage user expectation if a site happens to get overloaded whilst also giving the site time to recover.
Another options is this:
$load = sys_getloadavg();
if ($load[0] > 80) {
header('HTTP/1.1 503 Too busy, try again later');
die('Server too busy. Please try again later.');
}
I got it from php's site http://php.net/sys_getloadavg, altough I'm not sure what the values represent that the sys_getloadavg returns
You could simply create a 500.html file and have your webserver use that whenever a 50x error is thrown.
I.e. in your apache config:
ErrorDocument 500 /errors/500.html
Or use a php shutdown function to check if the request timeout (which defaults to 30s) has been reached and if so - redirect/render something static (so that rendering the error itself cannot cause problems).
Note that most sites where you'll see a "This site is taking too long to respond" message are effectively generating that message with javascript.
This may be to do with the database connection timing out, but that assumes that your server has a bigger DB load than CPU load when times get tough. If this is the case, you can make your DB connector show the message if no connection happens for 1 second.
You could also use a quick query to the logs table to find out how many hits/second there are and automatically not respond to any more after a certain point in order to preserve QOS for the others. In this case, you would have to set that level manually, based on server logs. An alternative method can be seen here in the Drupal throttle module.
Another alternative would be to use the Apache status page to get information on how many child processes are free and to throttle id there are none left as per #giltotherescue's answer to this question.
You can restrict the maximum connection in apache configuration too...
Refer
http://httpd.apache.org/docs/2.2/mod/mpm_common.html#maxclients
http://www.howtoforge.com/configuring_apache_for_maximum_performance
This is not a strictly PHP solution, but you could do like Twitter, i.e.:
serve a mostly static HTML and Javascript app from a CDN or another server of yours
the calls to the actual heavy work server-side (PHP in your case) functions/APIs are actually done in AJAX from one of your static JS files
so you can set a timeout on your AJAX calls and return a "Seems like loading tweets may take longer than expected"-like notice.
You can use the php tick function to detect when a server isn't loading for a specified amount of time, then display an error messages. Basic usage:
<?php
$connection = false;
function checkConnection( $connectionWaitingTime = 3 )
{
// check connection & time
global $time,$connection;
if( ($t = (time() - $time)) >= $connectionWaitingTime && !$connection){
echo ("<p> Server not responding for <strong>$t</strong> seconds !! </p>");
die("Connection aborted");
}
}
register_tick_function("checkConnection");
$time = time();
declare (ticks=1)
{
require 'yourapp.php' // load your main app logic
$connection = true ;
}
The while(true) is just to simulate a loaded server.
To implement the script in your site, you need to remove the while statement and add your page logic E.G dispatch event or front controller action etc.
And the $connectionWaitingTime in the checkCOnnection function is set to timeout after 3 seconds, but you can change that to whatever you want
I have a very expensive query which gets executed from php and it can take a while to execute. Is there a way, in php, to detect if a user disconnects prior to the query being done and cancel it?
A possible solution is to use pg_send_query(), that function sends a query to the database and returns immediatly without blocking. Then you can poll to see if the user disconnected before the query finished. See this:
ignore_user_abort(false);
$db = pg_connect(DATABASE_DSN);
pg_send_query($db, "SELECT pg_sleep(10000)"); // long running query
while(pg_connection_busy($db)) {
/* You need to output something to the browser so PHP can know when the
browser disconnected. The 0 character will be ignored.
*/
echo chr(0);
/* Need to do both flushes to make sure the chr is sent to the browser */
ob_flush();
flush();
usleep(100000); // 0.1s to avoid starving the CPU
if(connection_status() != CONNECTION_NORMAL || connection_aborted()) {
// Browser disconnected, cleanup an die
pg_cancel_query($db);
pg_query($db, "ROLLBACK");
pg_close($db);
die();
}
}
// At this point the query finished and you can continue fetching the rows
This approach works but has a big problem: you really need to send something to the browser to detect the browser disconnection. If you don't, connection_status() and connection_aborted() will not work. This seems to be an old PHP bug, see here: https://bugs.php.net/bug.php?id=30301
So this method doesn't work when, for example, you query Postgres in the middle of a PDF generation routine. In that case the needed chr(0) will break the generated binary file.
You would want to use connection_aborted to detect if the user has disconnected it returns 1 if the client has disconnected otherwise it returns 0. There is some documentation here, however its usage is self documenting and you should have no problem using it.