Using php driver 1.3.2 and mongodb 2.2, I am trying to use readPreference to direct an aggregate query to one of the secondaries in my replica set. Seems whatever I try, the aggregate query is executed on the primary server.
Basic example:
$db = new \MongoClient('rs1.example:27017,rs2.example:27017,rs3.example:27017', array('replicaSet' => 'myRs') );
$db->setReadPreference( \MongoClient::RP_SECONDARY );
$results = $db->tracking->sessions->aggregate( array( ... ) );
I enabled MongoLog and got the following results:
The aggregate method shows: REPLSET INFO: - connection: type: PRIMARY
If I use find instead, it shows: REPLSET INFO: - connection: type: SECONDARY
Is this a bug with the php driver? Anyone else run into this? Thought I would toss it on SO before adding it as a bug in their Jira.
All command queries through the PHP driver are currently directed to primary servers. We have several old tickets related to this, originating with requests to direct count commands to secondaries, but it was a non-trivial change that requiring checking the command against a whitelist to ensure it is read-only. The current ticket tracking this fix is PHP-535, which I linked to the issue you opened, PHP-662.
If you need an immediate work-around, you can call find() on the $cmd collection directly, passing the same $command array argument that you'd typically pass to MongoDB::command() as the argument to find(). Kristina documented this solution in this JIRA comment, and although she used slaveOkay() in that example, it should also work with read preferences.
Related
I'm using SolrClient with SolrQuery object, but I need to add RawQueryParameters to it:
How is this done? I'm fairly new to Solr requests and I could not find such option in the documentation.
So far I have this:
$SolrQuery = new \SolrQuery();
$SolrQuery->setStart($this->offset);
$SolrQuery->setRows($this->limit);
$SolrQuery->setQuery($request);
$SolrQuery->addField('*')->addField('units:[subquery]');
$SolrQuery->addParam('units.q', '{!terms%20f=id%20v=$row.unit_ids}');
When running toString() on this object, I get:
start=0&rows=2147483647&q=type:address&fl=*,units:[subquery]&units.q={!terms%20f=id%20v=$row.unit_ids}
what is the correct query. This works fine in CLI, but gives exception when executed in PHP as $queryResponse = $SolrClient->query($SolrQuery);:
Exception: while invoking units:[subquery] on doc=SolrDocument{id=stored,indexed,tokenized,omitNorms,indexOptions=DOCS
EDIT: Solution was to use addParam() without urlcoded request. Spaces go through just fine!
You can get the active parameters by calling ->toString() on the SolrQuery object - since SolrQuery inherits from SolrModifiableParams, you can call addParam directly on the query object to add custom parameters. This lets you add any parameter you want to the request. Be careful to add the parameter verbatim, since SolrQuery should handle necessary escaping for you.
If you still see an error, you can check the logging on the Solr server (under Admin -> Logging). If the log level is set to INFO, Solr will log all requests so you can see what Solr actually got. Any exceptions should also be present in this log if they're generated on the Solr side.
The Solr extension will usually throw exceptions as the class SolrClientException, which can be useful to determine the source of the error. A list of exception messages are also available in the extension source if you need to debug further.
I have developed a project under which several sql query have been used. Now I want to monitor some query in order increase security. So I want every query to be passed through a function first. As there are too many queries so I can not go back and edit every file and query. Is there a way that I can trap into queries before they are sent to mysql server?
There are four ways to accomplish this depending on what you are using, the last being the much more reliable.
The General Query Log
MySQL provides a mechanism to log just about everything that the mysqld process is doing, via the general query log. As you described in your question you probably do not have persistent connections, so you will need to either:
Enable the MySQL general query log when the mysqld process is started, with the --log[=file_name]
Set a global/session variable with SET GLOBAL general_log = 'ON'.
Fore more information about the general query log, see the MySQL 5.1 reference manual.
Using sed (or manually!)
This technique involves creating a a new function, and renaming all of the mysqli_* function calls to call another function.
Presuming your newly created function is named proxy_query(), you can use sed to traverse through all files and change them automatically:
sed i '.bck' 's/mysqli_query/proxy_query/'
The -i paramater specifies that the file should be edited in place, and that a copy should be made of the original file and have a .bck extension appended.
The runkit extension
I must admit that I'm being naive here, and that I haven't used this particular extension before - but it is possible to rename functions with this PECL extension.
The requirements for this extension can be found here, and note that it is not bundled with PHP.
As with above, you can create a proxy function where all calls will go through. Let's assume it's also called proxy_query. Usage would go something like this:
// rename the function (a very bad idea, really!)
runkit_function_renam('mysqli_connect', 'proxy_super');
function mysqli_query($query, $resultmode = MYSQLI_STORE_RESULT)
{
// do something with the SQL in $query
// .. and call mysqli_query, now proxy_super
return proxy_super($query, $resultmode);
}
I have to note here that this method is highly discouraged. You shouldn't ever need to set default PHP functions.
Using Pdo/OO-mysqli
This is the simplest technique, and probably the most reliable as well. If you're using Pdo already, you can simply extend the \Pdo class. A similar approach could be used with MySQL Improved(mysqli):
class MyPdo extends \Pdo
{
public function query($query [, ... ])
{
// do something with $query
return parent::query($query [, ... ]);
}
}
Also note here, that this will only work if you are using Pdo, and if you are able to change the instantiation of the Pdo object, to overwrite it to your own class: MyPdo.
For more information about the \Pdo class, and it's children, see the manual.
If you want to monitor incoming queries using SQL profiler can be an excellent way to gather information on what's going on inside SQL without passing it all through a single function or procedure.
PHP's Mongo driver lacks a renameCommand function. There is reference to do this through the admin database. But it seems more recent versions of the Mongo driver don't let you just "use" the admin database if do don't have login privileges on that database. So this method no longer works. I've also read this doesn't work in sharded environments although this isn't a concern for me currently.
The other suggestion people seem to have is to iterate through the "from" collection and insert into the "to" collection. With the proper WriteConcern (fire and forget) this could be fairly fast. But it still means pulling down each record over the network into the PHP process and then uploading it back over the network back into the database.
I ideally want a way to do it all server-side. Sort of like an INSERT INTO ... SELECT ... in SQL. This way it is fast, network efficient and a low load on PHP.
I have just tested this, it works as designed ( http://docs.mongodb.org/manual/reference/command/renameCollection/ ):
$mongo->admin->command(array('renameCollection'=>'ns.user','to'=>'ns.e'));
That is how you rename an unsharded collection. One problem with MR is that it will change the shape of the output from the original collection. As such it is not very good at copying a collection. You would be better off copying it manually if your collection is sharded.
As an added note I upgraded to 1.4.2 (which for some reason comes out from the pecl channel into phpinfo() as 1.4.3dev :S) and it still works.
Updates:
Removed my old map/reduce method since I found out (and Sammaye pointed out) that this changes the structure
Made my exec version secondary since I found out how to do it with renameCollection.
I believe I have found a solution. It appears some versions of the PHP driver will auth against the admin database even though it doesn't need to. But there is a workaround where the authSource connection param is used to change this behavior so it doesn't auth against the admin database but instead the database of your choice. So now my renameCollection function is just a wrapper around the renameCollection command again.
The key is to add authSource when connecting. In the below code $_ENV['MONGO_URI'] holds my connection string and default_database_name() returns the name of the database I want to auth against.
$class = 'MongoClient';
if( !class_exists($class) ) $class = 'Mongo';
$db_server = new $class($_ENV['MONGO_URI'].'?authSource='.default_database_name());
Here is my older version that used eval which should also work although some environments don't allow you to eval (MongoLab gives you a crippled setup unless you have a dedicated system). But if you are running in a sharded environment this seems like a reasonable solution.
function renameCollection($old_name, $new_name) {
db()->$new_name->drop();
$copy = "function() {db.$old_name.find().forEach(function(d) {db.$new_name.insert(d)})}";
db()->execute($copy);
db()->$old_name->drop();
}
you can use this. "dropTarget" flag is true then delete exist database.
$mongo = new MongoClient('_MONGODB_HOST_URL_');
$query = array("renameCollection" => "Database.OldName", "to" => "Database.NewName", "dropTarget" => "true");
$mongo->admin->command($query);
So I'm confident that the stored procedure works, I've tested it in SQL Server Management Studio just fine and it runs in other service instances. The query used to run this SP follows;
exec sp_getAgentCommissionDetails_v3 201000023762230, 5
So it runs fine on SSMS and under the old MSSQL driver. But I run the query with SQLSRV like so;
function mssql_query($string, $linkID = null, $batch = 0) {
if (!$linkID) {
global $dbhandle;
$linkID = $dbhandle;
}
// SQLSRV_CURSOR_KEYSET ensures mssql_num_rows() works in most cases. Default scrollablility does not support this.
return sqlsrv_query($linkID, $string, array(), array("Scrollable" => SQLSRV_CURSOR_KEYSET));
}
The function is named mssql_query because we're updating an old system from MSSQL to SQLSRV, but we're working with an extremely messy old system. So rather than trying to refactor it we're overwriting the MSSQL_query function (having disabled the mssql extension) with one that uses SQLSRV.
$dbhandle is our SQLSRV connection resource. Other queries run just fine using this method.
So my question- is there any reason the query to run a stored procedure would not run under this SQLSRV function?
Couple notes on my troubleshooting;
I'm aware SQLSRV and PDO have a specific method to run stored procedures. Using that is not an option due to the massive codebase that uses the method above in various places and because we haven't got the man hours to refactor every page.
I pulled up SQLSRV_errors() and it returned 'Executing SQL directly; no cursor'. After a bit of research this seems to be a bug in the driver that returns this generic error message instead of a more specific useful one, so it can mean very many things.
There are no cursors or loops involved in the stored procedure.
Found a solution to this. SQLSRV seems to by default treat warnings as errors. Fixed this with a config change in the connection file;
sqlsrv_configure("WarningsReturnAsErrors",0);
It now runs fine.
Using the example from the manual:
$mongo = new Mongo("mongodb://sf2.example.com,ny1.example.com", array("replicaSet" => "myReplSet"));
When I check $mongo, it says it is indeed connected. I thought I could then call $mongo->isMaster() to get replica set details, but that doesn't work. Is that not a proper way of doing this?
isMaster isn't a PHP function (see http://www.php.net/manual/en/class.mongo.php for a list of functions available in the Mongo class). You can do:
$result = $mongo->myDb->command(array("isMaster" => 1));
This runs the isMaster command on the myDb database (it doesn't matter what db you run it on).