I've a mysql connection and a pgsql (postgres, set as default) connection.
I must read all rows from a table from mysql and then save all into postgres
I tried do this, remembering that my default connection is pgsql
$mysql_model = new Regioni();
$regioni = $mysql_model->setConnection("mysql")->all();
But it's still using pgsql connection.
I'm sure of this because i tried to insert a row into pgsql table, (it was empty), and I can dump this single row as results of all.
Also, I (only for debugging, do not hate me), modified vendor/laravel/framework/src/Illuminate/Database/Eloquent/Model.php, adding a debug echo into this method
/**
* Set the connection associated with the model.
*
* #param string|null $name
* #return $this
*/
public function setConnection($name)
{
echo "Model calling setConnection($name)" . PHP_EOL;
$this->connection = $name;
return $this;
}
As a result I can see
Model calling setConnection(mysql)
Model calling setConnection()
Model calling setConnection(pgsql)
The first one is the result of my explicit setConnection.
But why calling ->all() will reset connection to default one?
Main goal/question
What's the right way to dinamically change the connection of a model?
I post my actual solution.
It's ugly, but it works.
The requisite for this is that default connection is pgsql so I must set connection explicitly to read all table record from nysql
Aftert that, I loop throught and then save to pgsql simply doing an insert at time
$regioni = (new Regioni)->setConnection('mysql')->get();
(new Regioni)->truncate();
foreach($regioni as $r) {
(new Regioni)->insert($r->toArray());
}
Note that this is specifically for my case
I have an old mysql to be migrated to the new postgresql
I already did migrations on postgresql so all structures are ready
I read all data from source and simply write into target postgres
this must be done only one time
I truncate target before insert (to be repeatable in development)
If you can suggest something more efficient, please, write an answer
Related
Laravel 5.5
I'm wondering how to properly handle the possible case of multiple updates to the same records by separate users or from different pages by the same user.
For example, if an instance of Model_1 is read from the database responding to a request from Page_1, and a copy of the same object is loaded responding to a request from Page_2, how best to implement a mechanism to prevent a second update from clobbering the first update? (Of course, the updates could occur in any order...).
I don't know if it is possible to lock records through Eloquent (I don't want to use DB:: for locking as you'd have to refer to the underlying tables and row ids), but even if it were possible, Locking when loading the page and unlocking when submitting wouldn't be proper either (I'm going to omit details).
I think detecting that a previous update has been made and failing the subsequent updates gracefully would be the best approach, but do I have to do it manually, for example by testing a timestamp (updated_at) field?
(I'm supposing Eloquent doesn't automatically compare all fields before updating, as this would be somewhat inefficient, if using large fields such as text/binary)
You should take a look at pessimistic locking, is a feature that prevents any update until the existing one its done.
The query builder also includes a few functions to help you do "pessimistic locking" on your select statements. To run the statement with a "shared lock", you may use the sharedLock method on a query. A shared lock prevents the selected rows from being modified until your transaction commits:
DB::table('users')->where('votes', '>', 100)->sharedLock()->get();
Alternatively, you may use the lockForUpdate method. A "for update" lock prevents the rows from being modified or from being selected with another shared lock:
DB::table('users')->where('votes', '>', 100)->lockForUpdate()->get();
Reference: Laravel Documentation
What I came up with was this:
<?php
namespace App\Traits;
use Illuminate\Support\Facades\DB;
trait UpdatableModelsTrait
{
/**
* Lock record for update, validate updated_at timestamp,
* and return true if valid and updatable, throws otherwise.
* Throws on error.
*
* #return bool
*/
public function update_begin()
{
$result = false;
$updated_at = DB::table($this->getTable())
->where($this->primaryKey, $this->getKey())
->sharedLock()
->value('updated_at');
$updated_at = \Illuminate\Support\Carbon::createFromFormat('Y-m-d H:i:s', $updated_at);
if($this->updated_at->eq($updated_at))
$result = true;
else
abort(456, 'Concurrency Error: The original record has been altered');
return $result;
}
/**
* Save object, and return true if successful, false otherwise.
* Throws on error.
*
* #return bool
*/
public function update_end()
{
return parent::save();
}
/**
* Save object after validating updated_at timestamp,
* and return true if successful, false otherwise.
* Throws on error.
*
* #return bool
*/
public function save(array $options = [])
{
return $this->update_begin() && parent::save($options);
}
}
Usage example:
try {
DB::beginTransaction()
$test1 = Test::where('label', 'Test 1')->first();
$test2 = Test::where('label', 'Test 1')->first();
$test1->label = 'Test 1a';
$test1->save();
$test2->label = 'Test 1b';
$test2->save();
DB::commit();
} catch(\Exception $x) {
DB::rollback();
throw $x;
}
This will cause abort as the timestamp does not match.
Notes:
This will only work properly if the storage engine supports row-locks. InnoDB does.
There is a begin and an end because you may need to update multiple (possibly related) models, and wish to see if locks can be acquired on all before trying to save. An alternative is to simply try to save and rollback on failure.
If you prefer, you could use a closure for the transaction
I'm aware that the custom http response (456) may be considered a bad practice, but you can change that to a return false or a throw, or a 500...
If you don't like traits, put the implementation in a base model
Had to alter from the original code to make it self contained: If you find any errors, please comment.
This question already has answers here:
Laravel: connect to databases dynamically
(8 answers)
Closed 3 years ago.
I have the master database with login table and corresponding database settings for each user. On login I should dynamically change the db settings fetching from the table.
I can change the db connection but this is not persisting.
Config::set("database.connections.mysql", [
'driver' => 'mysql',
"host" => $usr_host,
"database" => $usr_database,
"username" => $usr_username,
"password" => $usr_password,
...
]);
edit:
New database is created for each user when he/she registers with the app and thus i dont have the database connection for each user defined in the config/database.php
This way you can set new parameter when it comes to database:
\Config::set('database.connections.mysql.database', $schemaName);
Remember about PURGE to persist this settings
DB::purge('mysql');
Cheers!
Well you can use the default database for user login and have a new field for the database name. Then whenever you need to query a different database, you can just change your db connection.
Something like this
$someModel = new SomeModel;
$databaseName = "mysql2"; // Dynamically get this value from db
$someModel->setConnection($databaseName);
$something = $someModel->find(1);
You can read more about it here.
http://fideloper.com/laravel-multiple-database-connections
you need to get the config first.. then alter the specific field then set it back..
$config = Config::get('database.connections.company');
$config['database'] = "company_tenant_$id";
$config['password'] = "test2123";
config()->set('database.connections.company', $config);
I think a good place to change the database connection place is in bootstrap/app.php file, use the code below:
$app->afterBootstrapping(\Illuminate\Foundation\Bootstrap\LoadConfiguration::class, function ($ap) {
// your database connection change may happens here
});
It is before ServiceProvider register and boot, so even if you use DB or Eloquent staff in ServiceProvider, it works very well.
In laravel what you can do is create the different connections in the connections array of the file conf/database, then these connections you can use them when you are going to carry out operations in your application.
If you use query builder or raw expresions you must use the connection ('name') method to perform queries, for example:
$users = DB::connection('mysql')
->table('users')
->select(...)
->get();
If you use eloquent you can specify the name of the connection in the model file in the connection attribute, for example:
<?php
namespace App;
use Illuminate\Database\Eloquent\Model;
class User extends Model
{
/**
* The connection name for the model.
*
* #var string
*/
protected $connection = 'mysql';
}
A solution to your problem could be that you have created the different types of connections according to the users in the file conf/database, and save the name of the connection that the user uses as a column in the user table, and when you go to make the queries you get the name of the connection of the user, for example:
$user = User::find(Auth::id());
$connection = $user->connection;
$users = DB::connection($connection)
->table('users')
->select(...)
->get ();
more info:
https://laravel.com/docs/5.5/eloquent#eloquent-model-conventions
https://laravel.com/docs/5.5/database#read-and-write-connections
After an extensive search I found it this way:
Go to this file vendor\laravel\framework\src\Illuminate\Auth\Middleware\Authenticate.php
Go to the method: protected function authenticate($request, array $guards)
and right after the method has started, paste the following code:
if(auth()->check() && !empty( auth()->user()->db_name )){
$dynamic_db_name = auth()->user()->db_name;
$config = \Config::get('database.connections.mysql');
$config['database'] = $dynamic_db_name;
$config['password'] = "Your DB Password";
config()->set('database.connections.mysql', $config);
\DB::purge('mysql');
}
first we are checking if the user is logged in with
auth()->check()
Then as you may have added db_name name or the like column table to your users table according to each user. So in the next condition, I am making sure that the db_name is available:
&& !empty( auth()->user()->db_name )
Then after execution enters the if condition I get the db_name from the user record and set the configuration according to the user database, save the config and use purge method of DB class to persist this setting.
I was really stuck with it. I did not wanted to change my db connection in every class. So this is how I got it. Now I can use both Eloquent and DB without any tension anywhere. In my application I have one centeral database for login of all users and then for each organization there is a different database. So after the user has logged in, I do not need the centeral database (login database), I juse need the user/organization specific database. So this is how I got it to work.
I need to check if a table exists in a database. I currently develop using Yii2.
My case is a bit different from this question because the table to be checked is not (and can not be) a model.
I have tried (new \yii\db\Query())->select('*')->from($mysticTable)->exists());
The above throws a yii\db\Exception because, according to question linked above, the yii\db\Query() class tries to ->queryScalar() when asked if ->exists(). Invariably, this method checks if the result-set exists.
How do I check if a table exists?
For Yii2 you can use:
$tableSchema = Yii::$app->db->schema->getTableSchema('tableName');
If the table does not exist, it will return null, so you can check returned value for being null:
if ($tableSchema === null) {
// Table does not exist
}
You can find this method in official docs here.
Good that you get an exception. Simply parse the exception message. You would get a very very specific message and SQL error code for missing table.
That is what I do when checking e.g. IF an error was due to something that can be recovered, say a broken connection, versus some other error.
OR I see that many people have pointed out much more direct ways of getting that information.
A spin off #msfoster's answer got me closer to a solution in yii2
/**
* #param $tableName
* #param $db string as config option of a database connection
* #return bool table exists in schema
*/
private function tableExists($tableName, $db = null)
{
if ($db)
$dbConnect = \Yii::$app->get($db);
else
$dbConnect = \Yii::$app->get('db');
if (!($dbConnect instanceof \yii\db\Connection))
throw new \yii\base\InvalidParamException;
return in_array($tableName, $dbConnect->schema->getTableNames());
}
This also serves multiple databases.
I am creating a web site using php, mysql and zend framework.
When I try to run any sql query, page generation jumps to around 0.5 seconds. That's too high. If i turn of sql, page generation is 0.001.
The amount of queries I run, doesn't really affect the page generation time (1-10 queries tested). Stays at 0.5 seconds
I can't figure out, what I am doing wrong.
I connect to sql in bootstrap:
protected function _initDatabase ()
{
try
{
$config = new Zend_Config_Ini( APPLICATION_PATH . '/configs/application.ini', APPLICATION_ENV );
$db = Zend_Db::factory( $config -> database);
Zend_DB_Table_Abstract::setDefaultAdapter( $db );
}
catch ( Zend_Db_Exception $e )
{
}
}
Then I have a simple model
class StandardAccessory extends Zend_DB_Table_Abstract
{
/**
* The default table name
*/
protected $_name = 'standard_accessory';
protected $_primary = 'model';
protected $_sequence = false;
}
And finally, inside my index controller, I just run the find method.
require_once APPLICATION_PATH . '/models/StandardAccessory.php';
$sa = new StandardAccessory( );
$stndacc = $sa->find( 'abc' );
All this takes ~0.5 seconds, which is way too long. Any suggestions?
Thanks!
Tips:
Cache the table metadata. By default, Zend_Db_Table tries to discover metadata about the table each time your table object is instantiated. Use a cache to reduce the number of times it has to do this. Or else hard-code it in your Table class (note: db tables are not models).
Use EXPLAIN to analyze MySQL's optimization plan. Is it using an index effectively?
mysql> EXPLAIN SELECT * FROM standard_accessory WHERE model = 'abc';
Use BENCHMARK() to measure the speed of the query, not using PHP. The subquery must return a single column, so be sure to return a non-indexed column so the query has to touch the data instead of just returning an index entry.
mysql> SELECT BENCHMARK(1000,
(SELECT nonindexed_column FROM standard_accessory WHERE model = 'abc'));
Note that Zend_Db_Adapter lazy-loads its db connection when you make the first query. So if there's any slowness in connecting to the MySQL server, it'll happen as you instantiate the Table object (when it queries metadata). Any reason this could take a long time? DNS lookups, perhaps?
The easiest way to debug this, is to profile your sql queries. you can use Firephp (plugin for firebug) see http://framework.zend.com/manual/en/zend.db.profiler.html#zend.db.profiler.profilers.firebug
another way to speed up things a little is to cache the metadata of your tables.
see: http://framework.zend.com/manual/en/zend.db.table.html#zend.db.table.metadata.caching
Along with the above suggestions I did a very unscientific test and found that the PDO adapter was faster for me in my application (I know mysqli is supposed to be faster but maybe it's the ZF abstraction). I show the results here (the times shown are only good for comparison)
I have a nasty problem. I want to get rid of a certain database field, but I'm not sure in which bits of code it's called. Is there a way to find out where this field is used/called from (except for text searching the code; this is fairly useless seeing as how the field is named 'email')?
Cheers
I would first text search the files for the table name, then only search the tables that contain the table name for the field name.
I wrote a program to do this for my own purposes. It builds an in-memory listing of tables and fields and relates the tables to the fields. Then it loops through tables, searching for the code files that contain the table names, and then searches those files for the fields in the tables found. I'd recommend a similar methodology in your case.
setting mysql to log all queries for some time might help. the queries will give you the tip where to look
brute force - set up a test instance - remove the column - and excercise your test suite.
create a before insert trigger on that table that monitors the insertion on that column.
at the same time create another table called monitor with only one column email
make that table insert the value of NEW.email field into monitor.email as well as in real table.
so you can run your application and check for the existence of any non-null value in monitor table
You should do this in PHP i would expect
For example:
<?php
class Query
{
var $command;
var $resource;
function __construct($sql_command = '')
{
$this->command = $sql_command;
}
public function setResource($resource)
{
$this->resource = $resource;
}
}
//then you would have some kind of database class, but here we would modify the query method.
class Database
{
function query(Query $query)
{
$resource = mysql_query($query->command);
$query->setResource($resource);
//Then you can send the class to the monitor
QueryMonitor::Monitor($query);
}
}
abstract class QueryMonitor
{
public static Monitor(Query $query)
{
//here you use $query->resource to do monitoring of queryies
//You can also parse the query and gather what query type it was:-
//Select or Delete, you can also mark what tables were in the Query
//Even meta data so
$total_found = mysql_num_rows($query->resource);
$field_table = mysql_field_table ($query->resource);
//Just an example..
}
}
?>
Obviously it would be more advanced than that but you can set up a system to monitor every query and every queries meta data in a log file or w.e