I have run into a major problem while writing tests. I am using Laravel 5.6.0 as framework and PHPUnit 7.0 for testing. As a testing DB I have used sqlite with storage in memory. This is from my database.php:
'sqlite_testing' => [
'driver' => 'sqlite',
'database' => ':memory:',
'prefix' => '',
],
But my problem is that I have several places where I use whereRaw, for example $query->whereRaw('STR_TO_DATE(CONCAT(date, " ",from), "%Y-%m-%d %k") < ?', [$before]);. The problem here is that sqlite does not have the STR_TO_DATE or CONCAT functions that MySQL has. So PHPUnut throws a bunch of errors because of that.
My solution was instead using a MySQL DB as testing DB. But this doesn't seem to work since I get several different errors, mostly I have several tests where foreign key constraint fails.
One exaple for this is that I have the following in my Base TestCase setUp method:
if (Schema::hasTable('gym_schedule') && !empty(GymSchedule::createGymSchedules())) {
$this->artisan('db:seed', ['--class' => 'GymScheduleTableSeeder']);
}
This fails every time except the first because it says that a schedùle with id 1 already exists (id is my primary key). I did try to truncate all tables between each test class using tearDown, but that did not help at all, and also the testing became reeeeally slow, like 3 seconds for each test.
So basically that approach does not work either.
I am at a loss. I have tried googling this and searching through StackOverflow. I am open to any suggestion that is not too complicated (either remove all MySQL functions somehow, solve usage of MySQL or anything else really).
Does anyone have a good idea?
Regarding the first part of the question, unit testing a Laravel app when using whereRaw(), this is how I wrote the methods:
$raw_condition = "CONCAT(`country_code`, `number`) = ?";
if (env('DB_CONNECTION') === 'sqlite') {
$raw_condition = "(`country_code` || `number`) = ?";
}
return Model::whereRaw($raw_condition, [$number])->firstOrFail();
I've created a rather simple multi-tenant application using separate schemas in a Postgresql database. I keep a public schema, which only a few model use and then the rest of my models use the tenant. I determine the Client from the subdomain through a middleware then, call the following function on my Client model to set the credentials for the connection.
public function logInAsClient()
{
$settings = $this->settings;
// set tenant db
config([
'client' => $settings,
'client.id' => $this->id,
'database.connections.tenant.schema' => $this->schema,
'schema' => $this->schema,
'domain' => $this->domain,
]);
DB::disconnect('tenant');
DB::reconnect('tenant');
return true;
}
It's working great in normal functions, but I have several tasks which require me to queue up hundreds of jobs at a time. Since they are all running on the same app, I pass the client_id to each job and then at the start of the handle run:
Client::find($this->client_id)->logInAsClient();
However, when running this code I get the following error:
PDOException: SQLSTATE[08006] [7] FATAL: sorry, too many clients already
FATAL: sorry, too many clients already
I don't there there is any other part of my app where I am connecting or reconnecting to databases so I'm not sure where else this issue might be coming from. If there are any hints on how to debug an issue like this it would be greatly appreciated.
I'm working on a rewrite of a project from the ground up and figured I would try to learn MVC along the way. In this case, I've chosen Phalcon and am still working through the fundamentals of converting the tutorials to my own project.
I have two "configuration" settings that I need to account for. First, I need to read a configuration file that has the database credentials (this works properly).
require_once('../fileconfig.php'); // Read config file
$init = new Phalcon\Config\Adapter\Php("../fileconfig.php"); //Convert it to array
But once I have that, how do I actually connect to the database and add it to $di-> (which, if I understand correctly, is effectively the global class? Ultimately, I want to pull the contents of "select * from config" into an array and use that for the application configuration. In this case, var_dump($dbh) returns "null"
//Connect to database
$di->set('db', function() use ($init) {
$dbh = new \Phalcon\Db\Adapter\Pdo\Mysql([
"host" => $init->database->host,
"username" => $init->database->username,
"password" => $init->database->password,
"dbname" => $init->database->dbname
]);
return $dbh;
});
var_dump($dbh); //returns null
If I remove the $di-> section, the array returns the data that I need, but it still doesn't help me figure out how to connect to the database and have it available globally for other functions in the models:
$dbh = new \Phalcon\Db\Adapter\Pdo\Mysql([
"host" => $init->database->host,
"username" => $init->database->username,
"password" => $init->database->password,
"dbname" => $init->database->dbname
]);
Returns:
object(Phalcon\Db\Adapter\Pdo\Mysql)[28]
protected '_descriptor' =>
array (size=4)
'host' => string 'localhost' (length=9)
'username' => string 'testuser' (length=8)
'password' => string 'testpass' (length=8)
'dbname' => string 'testdb' (length=6)
This question seems to be close to what I'm asking, but was more about error handling than the actual connection and I didn't see an answer to my question there.
To resolve your database you need to resolve your di. You could resolve it the file you declared it in with
$di->getShared('db')
But note, you don't want to do that. You want your files seperated with their responsibilities.
Inside of a class that inherits \Phalcon\Mvc\Controller you can use
$this->db->
Please refer to http://docs.phalconphp.com/en/latest/reference/di.html in order to see why to use a DI, and all the nuances of accessing it
It really helps to go through other phalcon projects and look at how everything works together. Please refer to the source here and look at how projects are set up:
https://github.com/phalcon/invo
https://github.com/phalcon/vokuro
https://github.com/phalcon/forum
These are ranked by complexity so start with invo first and then move on
Here is a snippet of the code I got from some tutorial blog:
<?php
class PersonalBlogController extends AppController {
var $name = 'PersonalBlog';
var $uses = array('PersonalBlog');
function index(){
//ini_set('max_execution_time', 300);
debug($this->PersonalBlog->findAll());
//debug($this->PersonalBlog->find('all'));
}
}?>
I have seen some other cases in max time limit fatal error from other posts. But mine is probably the most disgusting. The fatal 30 secs time limit error keeps coming even after I change the max_execution_time variable in php.ini. Even after changing it to 60 or 300 seconds, the error message stays the same. It says 30 seconds time limit is exceeded.
I then followed alternative solution from a similar question, which is to do this instead:ini_set('max_execution_time', 300); (See my commented code above).
The error is gone. However what happens after that is this:
Warning (2): PDO::__construct(): [2002] A connection attempt failed because the connected party did not (trying to connect via tcp://localhost:3306) [CORE\cake\libs\model\datasources\dbo\dbo_mysql.php, line 155]
.....
.....blablabla...
Fatal error: Call to a member function getAttribute() on a non-object in C:\Server\www\myserver.dev\public_html\cakephp-cakephp-7fbf7a4\cake\libs\model\datasources\dbo\dbo_mysql.php on line 259
So I realized that I actually have two MySQL server installed. The newest one (5.5.11) is on port 3307 not on port 3306! That is why I then changed my driver setting in the dbo_mysql.php:
/**
* Base configuration settings for MySQL driver
*
* #var array
*/
protected $_baseConfig = array(
'persistent' => true,
'host' => 'localhost',
'login' => 'root',
'password' => 'mysqlpassword',
'database' => 'personal_blog',
'port' => '3307'
);
Every other setting is normal; I have the same database setting for app>config>database file as you can see here:
public $default = array(
'driver' => 'mysql',
'persistent' => false,
'host' => 'localhost',
'login' => 'root',
'password' => 'mysqlpassword',
'database' => 'personal_blog',
'port' => 3307,
'prefix' => '',
);
After those changes, the warning still appears...I really feel like I want to throw up right now
Any help would be appreciated.thx
You shouldn't have to edit the dbo_config to manage the port setting. All of those keys are available for use in the database.php file in your app/config/ folder.
You actually shouldn't have your personal defaults in the dbo file at all -- nothing you are doing should ever edit any core files.
Once you have your port setting in ONLY the database.php file and your dbo driver is back to the one that shipped with cake you should delete all of the files in /tmp/cache and set your debug > 0 in app/config/core.php
Refresh and if the error is still occurring we know that something is messed up at the php.ini / .htaccess or ini_set level. Try adjusting the value and then loading a view that contains
<?php
phpinfo( );
?>
and see if the max execution time setting is actually being read by the php interpreter. If it isn't you might want to try the ini set method or htaccess methods of playing with that value. Also make sure the php.ini you are editing is the right one ( the path to the ini should be in the php info dump ) - you might find you have 2 versions of php loaded as well as your two versions of mysql.
If that doesn't solve or at least pinpoint the issue then post back here and let us know.
I've rewritten my site php-code and added MySQL Stored Procedures.
In my local version everything works fine but after I uploaded my site to hosting server I'm constantly getting fatal error 'Prepared statement needs to be re-prepared'.
Sometimes page loads, sometimes loading fails and I see this error. What's that?
This is a possibility: MySQL bug #42041
They suggest upping the value of table_definition_cache.
You can read about statement caching in the MySQL docs.
#docwhat's answer seems nice, but on a shared hosting server, not everyone is allowed to touch the table_open_cache or table_definition_cache options.
Since this error is related to prepared statements, I have tried to 'emulate' those with PDO by providing the following option:
$dbh = new PDO('mysql:host=localhost;dbname=test', $user, $pass, [
PDO::ATTR_EMULATE_PREPARES => true
]);
Note: actually this is in a Laravel 5.6 project, and I added the option in config/database.php:
'connections' => [
'mysql' => [
'driver' => 'mysql',
'host' => env('DB_HOST', '127.0.0.1'),
'port' => env('DB_PORT', '3306'),
'database' => env('DB_DATABASE', 'forge'),
'username' => env('DB_USERNAME', 'forge'),
'password' => env('DB_PASSWORD', ''),
'unix_socket' => env('DB_SOCKET', ''),
'charset' => 'utf8mb4',
'collation' => 'utf8mb4_unicode_ci',
'prefix' => '',
'strict' => true,
'engine' => null,
'options' => [
PDO::ATTR_EMULATE_PREPARES => true,
],
],
(...)
],
I have not tested the impact of emulating prepared statements on the duration of loading my site, but it works against the error SQLSTATE[HY000]: General error: 1615 Prepared statement needs to be re-prepared I got.
Update on the performance: the emulated version seems to be slightly faster (32.7±1.4ms emulated, 35.0±2.3ms normal, n=10, p-value=0.027 for two-tailed Student's T-test).
In short: Don't use VIEWS in prepared statements.
This seems to be an on-going issue
Views are messy to handle with Dynamic SQL
Earliest Bug was Cannot create VIEWs in prepared statements from 11 years ago. There was a patch put in to address it.
Another bug report, Prepared-Statement fails when MySQL-Server under load, states that error 1615 is not a bug when the underlying tables are busy. (Really ?)
While there is some merit to increasing the table cache size (See MySql error when working with a mysql view), it does not always work (See General error: 1615 Prepared statement needs to be re-prepared (selecting mysql view))
ALTERNATIVES
Over a year ago, someone mentioned this in the MySQL Forum (MySql “view”, “prepared statement” and “Prepared statement needs to be re-prepared”).
Someone came up with the simple idea of not using the view in the prepared statement but using the SQL of view in a subquery instead. Another idea would be to create the SQL used by the view and execute it in your client code.
These would seems to be better workarounds that just bumping up the table cache size.
First gain access to mysql shell:
mysql
Check the value of the table_definition_cache:
show global variables like '%table_definition_cache%';
It might be 400 or 1400.
Enlarge it:
set global table_definition_cache = 4000;
Good to go!
Issue: 'Prepared statement needs to be re-prepared'
This issue generally occurs at the time of calling procedure either by using any Computer Language(like Java) or Calling Procedures from the backend.
Solution: Increase the size of the cache by using (executing) below script.
Script: set global table_definition_cache = 4000;
just do this:
SET GLOBAL table_definition_cache = 4096;
SET GLOBAL table_open_cache = 4096;
4096 can be to less, so set it to a higher value. But make sure that both values are the same.
FLUSH TABLES; comand on database solved for me, i was using doctrine orm.
my solutions is to create a routine like this:
DELIMITER $$
--
-- Procedimientos
--
DROP PROCEDURE IF EXISTS `dch_content_class_content`$$
CREATE DEFINER=`renuecod`#`localhost` PROCEDURE `dch_content_class_content`(IN $classId INTEGER)
BEGIN
-- vw_content_class_contents is a VIEW (UNIONS)
select * from vw_content_class_contents;
END$$
I hope this help someone
This is a workaround for people who are on shared hosting and don't want to risk it with sql injection. According to this post:
Laravel Fluent Query Builder Join with subquery
you could store the definition of your view in a function
private function myView(){
return DB::raw('(**definition of the view**) my_view_name)');
}
and then use it like this:
public function scopeMyTable(Builder $query)
{
return $query->join($this->myView(),'my_view_name.id','=','some_table.id');
}
This is a laravel approach, but I'm sure it could be applied in most cases and doesn't need huge code refactoring or architecture change. Plus, it's relatively secure as your statements stay prepared
I had this error being caused by a large group_concat_max_len statement
SET SESSION group_concat_max_len = 1000000000000000000;
removed it and the error went away
Okay, we were stuck on a laptop that would not allow Tableau to update or retrieve data from views on our MariaDB 10.3.31 databases. We did the usual, google and stack overflow, lots of solutions that just would not work. However, we have another laptop that did connect and ran just fine. So after many head scratches, a eureka moment came.
The offending laptop had both V5.3.14 and V8.0.26 ODBC connectors installed. We removed the V8.0.26 ODBC connector and Bang, all the view issues disappeared.
Hope someone finds this solution useful.
I tried to run an UPDATE query which was joining view and a table, and also got the same error. Since I had no user input, I decided to run DB::unprepared which fixed the problem.
You can read more about it in the Laravel Documentation.
I was getting this same error in Ruby on Rails in a shared hosting environment. It may not be the most secure solution, but disabling prepared statements got rid of the error message for me.
This can be done by adding the "prepared_statements: false" setting to your database.yml file:
production:
prepared_statements: false
This seems like a reasonable solution when you don't have control over the configuration settings on the MySQL server.