Allrighty, it's the first time I ask a question here. My problem is as awkward as it is difficult to get to the bottom of.
Story goes like this: I have this little system, which sends alot of e-mail invitations(not spam). So, being sensible, I don't use the PHP function mail(),
I use PEAR classes like Mail, Mail_Queue, Net_SMTP, etc.
Only problem is, my error logs fill up with tons of errors like this:
PHP Notice: Error in sending mail:
Mail Queue Error: Cannot initialize container in /usr/lib/php/PEAR.php on line 873
And then, of course:
[18-Feb-2011 17:38:44] PHP Fatal error:
Allowed memory size of 33554432 bytes exhausted
(tried to allocate 3 bytes) in /usr/lib/php/PEAR.php on line 844
Here's the code which inits the mail queue(inside a class called Newsletter)
//I know passing the DSN as the string is kind of deprecated,
//but it;'s the only way it works on my system
$dsn ="mysql://$db_user:$db_pass#$db_host/$db_name";
$db_options = array();
$db_options['type'] = 'db';
$db_options['dsn'] = $dsn;
$db_options['mail_table'] = TABLE_QUEUE;
$this->host = '-- valid host here --';//data in these strings has been obscured
$this->username = '-- valid username here --';
$this->password = '-- valid password here --';
//optionally, a 'dns' string can be provided instead of db parameters.
//look at DB::connect() method or at DB or MDB docs for details.
//you could also use mdb container instead db
//$server = isset($_SERVER['SERVER_NAME'])?$_SERVER['SERVER_NAME']:'localhost';
$mail_options = array(
'driver' => 'smtp',
'host' => $this->host,
'port' => 25,
'auth' => true,
'username' => $this->username,
'password' => $this->password,
);
$this->mail_queue = new Mail_Queue($db_options, $mail_options);
Some more code down the line,
public function sendChunk($start, $count)
{
global $db;
$ids = $db->get_results("SELECT DISTINCT id_user as id FROM ".TABLE_QUEUE);
$ret = array();
foreach ($ids as $id)
$ret[] = $id->id;
unset($ids);
$this->mail_queue->sendMailsInQueue($count, $start, 1);
return true;
}
Problem is, I double checked every line of code I wrote, and it's doing it's job. Only problem is that sometimes it refuses to send any mails.
Thanks in advance for replies.
I switched to MDB2 instead of DB
$db_options['type'] = 'db';
to
$db_options['type'] = 'mdb2';
this helped in taking care of memory exhaust problem, I am still looking to take care of initialize container in /usr/lib/php/PEAR.php problem
Ok found the solution for container errors:
Apply this patch
http://svn.php.net/viewvc/pear/packages/Mail_Queue/trunk/Mail/Queue.php?r1=303876&r2=309126
Try to limit result. using limit in your select statement.
Try to flush the old main queye.
Related
I'm currently looping to create a MBTiles map, and add information to my database each time.
Here's how I configured my connection and execute actions during the loop:
if ($pdo_mbtiles == null) {
echo "Opening new database connection".PHP_EOL;
$pdo_mbtiles = new PDO('sqlite:'.$filename,
'',
'',
array(
PDO::ATTR_PERSISTENT => true
)
);
$pdo_mbtiles->setAttribute(PDO::ATTR_DEFAULT_FETCH_MODE, PDO::FETCH_ASSOC);
$pdo_mbtiles->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
}
$q = $pdo_mbtiles->prepare("INSERT INTO tiles (zoom_level, tile_column, tile_row,tile_data) VALUES (:zoom_level, :tile_column, :tile_rowTMS, :tile_data)");
$q->bindParam(':zoom_level', $zoom_level);
$q->bindParam(':tile_column', $tile_column);
$q->bindParam(':tile_rowTMS', $tile_rowTMS);
$q->bindParam(':tile_data', $tile_data, PDO::PARAM_LOB);
$q->execute();
After 1018 times looping (this number doesn't change no matter how many times I try), I get this error message:
SQLSTATE[HY000]: General error: 14 unable to open database file
I checked the solution written here:
How to prevent SQLITE SQLSTATE[HY000] [14]?
but the echoed message only appears at the first time of the loop, so I assume the PDO connection isn't closed.
I didn't find other documentation relevant to this error code.
What could possibly go wrong here?
I tried to move the prepare and bind commands in a condition as follows. The exception isn't raised, but only the first tile is saved (or every tile is saved on top of the first one, not sure):
if ($pdo_mbtiles == null) {
echo "Opening new database connection".PHP_EOL;
$pdo_mbtiles = new PDO('sqlite:'.$filename,
'',
'',
array(
PDO::ATTR_PERSISTENT => true
)
);
$pdo_mbtiles->setAttribute(PDO::ATTR_DEFAULT_FETCH_MODE, PDO::FETCH_ASSOC);
$pdo_mbtiles->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
}
if ($q == null) {
$q = $pdo_mbtiles->prepare("INSERT INTO tiles (zoom_level, tile_column, tile_row,tile_data) VALUES (:zoom_level, :tile_column, :tile_rowTMS, :tile_data)");
$q->bindParam(':zoom_level', $zoom_level);
$q->bindParam(':tile_column', $tile_column);
$q->bindParam(':tile_rowTMS', $tile_rowTMS);
$q->bindParam(':tile_data', $tile_data, PDO::PARAM_LOB);
}
$q->execute();
Here are the files during the generation:
And here after the exception is raised:
Also, when the exception is raised, I do a var_dump of my $pdo_mbtiles, and here's the result (exactly the same as when I do it with a success):
object(PDO)#116 (0) {
}
Edit: Still trying to solve this problem, I simplified the code to create the MBTiles file. No success yet, but here's a sample if anyone want's to reproduce the issue. You can download it from https://www.dropbox.com/s/33vqamc9tn4c3ux/sample_PHP_MBTiles_generation_bug.zip?dl=0
I suggest you to reuse the connection if it is open.
Create a property: private $pdo;
And check if it's null before creating a new object:
function opendatabase(){
try{
if($this->pdo==null){
$this->pdo =new PDO("sqlite:database/database.db","","",array(
PDO::ATTR_PERSISTENT => true
));
}
return $this->pdo;
}catch(PDOException $e){
logerror($e->getMessage(), "opendatabase");
print "Error in openhrsedb ".$e->getMessage();
}
}
The error message was misleading. After many hours of debugging, I found that it was completely unrelated to my database connection.
It's juste that I used fopen() to get tiles data, and didn't fclose() after registration, thus reaching the limit of 1024.
1024 is because I used six require or require_once statements, so 1018 tile requests + 6 require = 1024 opened connections.
I have a .txt file with a list of 60,000 English words. I wanted to insert those to my database, so I simply just did as show here.
$file = new SplFileObject('list.txt');
foreach ($file as $line => $word) {
$p = new PDO('mysql:host=localhost; dbname=test_dictionary', 'root', 'test');
$p->query("INSERT INTO words (english) VALUES('$word') ");
}
Now, after I run this script, I get the following error:
Fatal error: Uncaught exception 'PDOException' with message 'SQLSTATE[HY000] [1040] Too many connections' in /var/www/skillz/test/curl/index.php:17 Stack trace: #0 /var/www/test/index.php(17): PDO->__construct('mysql:host=loca...', 'root', 'test') #1 {main} thrown in /var/www/test/index.php on line 4
That line 4 is where the new PDO('mysql:') is located. So, I tried to search this error, and found this answer that seemed a solution. And I edited mysql accordingly, as
$ vi /etc/my.cnf
max_connections=250
But I still get the same error, I have MySql 5.5.38 running PHP-FPM, NGINX in CentOS 6.5
Don't open a new Connection for every word. You only need one connection open for the lifetime of your inserts. I'm not sure about the true lifetime of a PDO object, I know they're cleaned up when they're not used, but garbage collection might not do that for a couple of minutes, and for 60,000 words, you're going to hit your limit of connections to the database faster than it can clean them up.
$file = new SplFileObject('list.txt');
$p = new PDO('mysql:host=localhost; dbname=test_dictionary', 'root', 'test');
foreach ($file as $line => $word) {
$p->query("INSERT INTO words (english) VALUES('$word') ");
}
You should declare the SQL connection outside the foreach.
Because it will make 60.000 connections.
$file = new SplFileObject('list.txt');
$p = new PDO('mysql:host=localhost; dbname=test_dictionary', 'root', 'test');
foreach ($file as $line => $word) {
$p->query("INSERT INTO words (english) VALUES('$word') ");
}
You only need to declare the SQL connection once and you can use it anytime as you want.
If you put it in foreach it will make a SQL connection every word, that's why you got that message.
Solution 1
Use batch insert statement :
INSERT INTO words (col1, col2) VALUES ('val1', 'val2'), ('val3', 'val4'), ...('val3n', 'val4n');
This fails if you also want to check if some rows failed or not. So, below is another solution.
Solution 2
Create a persistent database connection. This will use the same connection in all iterations of the loop.
$file = new SplFileObject('list.txt');
$p = new PDO('mysql:host=localhost; dbname=test_dictionary', 'root', 'test', array(
PDO::ATTR_PERSISTENT => true)); //Persistent Database Connection
foreach ($file as $line => $word) {
$p->query("INSERT INTO words (english) VALUES('$word') ");
}
$p = null; //Destroy Connection
I am trying to export data from MSSQL table to Excel. Below is my code.
Problem is that i am getting the following error:
Fatal error: Call to undefined function mssql_fetch_array() in C:xampp\code.php on line 25
I am running Windows Server 2008 R2, IIS7, SQL Server 2008, PHP Version 5.4.4.
I have un-commented the line: "extension=php_mssql.dll" found in C:\xampp\php\php.ini
<?php
// load library
require 'include\php-excel.class.php';
$i = 0; // used as a counter
$myServer = "SERVERNAME\SQLEXPRESS";
$myUser = "UserName";
$myPass = "xxxxxx";
$myDB = "dbName";
$conn = new COM ("ADODB.Connection") or die("Cannot start ADO");
$connStr = "PROVIDER=SQLOLEDB;SERVER=".$myServer.";UID=".$myUser.";PWD=".$myPass.";DATABASE=".$myDB;
$conn->open($connStr); //Open the connection to the database
$result = "SELECT Col1, Col2, Col3 FROM myTable WHERE Col3='myCondition'";
//$rs = $conn->execute($result);
//$num_columns = $rs->Fields->Count();
// create data array and print headers on the first row
$data = array(1 => array ('No.', 'Col1', 'Col2', 'Col3'));
while($row=mssql_fetch_array($result)) {
//include additional rows
array_push($data, array($i, $row['Col1'], $row['Col2'], $row['Col3']));
$i++;
}
// If no results, indicate this on the first row
if ($i == 0){
$data = array(1 => array ('No results', 'empty', 'empty', 'empty'));
}
//generate file (constructor parameters are optional)
$xls = new Excel_XML('UTF-8', false, 'My Test Sheet');
$xls->addArray($data);
$xls->generateXML('MyReport');
?>
Thank you in advance.
PHP5.3 onwards doesn't support the mssql functions. If you check your PHP ext folder, you will find there isn't a php_mssql.dll no more
You're using an ADODB COM to create your DB connection. This isn't the same thing as using the mssql_xx library.
You can't mix and match between the two - they're not compatible.
In any case, you clearly don't have the mssql_xx library installed (I'm not even sure it's supported any more), so you can't use that.
Right above your failing call to mssql_fetch_array(), you have some commented out lines:
//$rs = $conn->execute($result);
//$num_columns = $rs->Fields->Count();
Not sure why you've commented these out, because you'll need them, or something like them. Particularly the first of those lines. And after that, you'd need to use the ADODB equivalent to mssql_fetch_array() to read the records in the loop.
However, my advice is to ditch all of this, and switch to using the PDO library instead. PDO supports the MS SQL database, and is standard PHP, unlike the ADODB stuff you're trying to use here.
See the PHP manual for the PDO library here: http://php.net/manual/en/book.pdo.php
After uncommenting the line for MS SQL library, have you restarted the web server for the change to take effect?
For sql, we can use sqlsrv_fetch_array with sqlsrv_query as:
$sql_query = "SELECT Col1, Col2, Col3 FROM myTable WHERE Col3='myCondition'";
$result = sqlsrv_query($conn,$sql_query);
while( $row = sqlsrv_fetch_array( $result, SQLSRV_FETCH_ASSOC) ) {
echo $row['Col1'];
/*etc codes here*/
}
I am having a problem when trying to access some functions in my mysql class. The function I have to connect to my database is as follows:
function connect_DB() {
$this->connection = mysql_connect($this->dbURL, $this->dbUser, $this->dbPass) or trigger_error('Connection failed: ' . mysql_error($connection), E_USER_ERROR);
$this->db = mysql_select_db($this->dbName, $this->connection) or trigger_error('Database selection failed: ' . mysql_error($db), E_USER_ERROR);
}
And I create my mysql object using this:
$dbConf = new StoreConfDB();
$dbConf->connect_DB();
Now I have a function that is supposed to grab all of the column names from a specified table. Code:
function get_db_columns($table) {
global $dbConf;
print_r($dbConf->get_db());
$result = $dbConf->query_DB_resource('SELECT * FROM ' . $table);
for($i = 0; $i < mysql_num_fields($result); $i++) {
$meta = mysql_fetch_field($result, $i);
$fields[] = $meta->name;
}
mysql_free_result($result);
return $fields;
}
But I am getting errors, such as:
Warning: mysql_query(): 5 is not a valid MySQL-Link resource
And if I try to remove the connection argument from the mysql_query call I get this:
Notice: Could not run query: No database selected in ...
Now I am running a debugger and have pinpointed what the problem is (I think). When the class works, the connection variable is set to: resource id='5' type='mysql link'
But when it calls the query function in the mysql class from inside the get_db_columns() function the connection variable is: resource id='5' type='Unknown'
So somehow the connection variable gets messed up even though both of the connection variables should be the same thing? (from $dbConf)? I've tested this function in 2 different places, it works in one and not the other! Please help!
#footy:
A print_r on $dbConf returns:
StoreConfDB Object ( [dbURL] => localhost [dbUser] => root [dbPass] => [dbName] => db1 [connection] => Resource id #5 [db] => 1 )
The query_DB_resource function:
function query_DB_resource($query) {
$sql_query = mysql_query($query) or trigger_error('Could not run query: '. mysql_error());
return $sql_query;
}
I noticed a couple of issues:
You have mysql_error($connection) and mysql_error($db). Those aren't valid parameters. You should be using $this->connection in both cases. That will give you better error output.
Normally, it doesn't terribly matter if you pass in a connection resource to mysql_query, but in this case you may want to: mysql_query($query, $this->connection). While I can't say that that is a guaranteed fix, it is likely to help.
Oh, and BTW -- every time someone uses global God kills a kitten. It's true. He told me. Either that or I had some bad nachos. (Yes, I'm a zealot on this issue. It's a good thing. trust me)
Regardless, if you need a variable inside of a function either pass it in, or make a property of a new class (make the StoreConfDB a Singleton? Make it accessible by factory method? You have a number of choices) -- avoiding global as a habit removes something nasty called unexpected side effects.
Allrighty, it's the first time I ask a question here. My problem is as awkward as it is difficult to get to the bottom of.
Story goes like this: I have this little system, which sends alot of e-mail invitations(not spam). So, being sensible, I don't use the PHP function mail(),
I use PEAR classes like Mail, Mail_Queue, Net_SMTP, etc.
Only problem is, my error logs fill up with tons of errors like this:
PHP Notice: Error in sending mail:
Mail Queue Error: Cannot initialize container in /usr/lib/php/PEAR.php on line 873
And then, of course:
[18-Feb-2011 17:38:44] PHP Fatal error:
Allowed memory size of 33554432 bytes exhausted
(tried to allocate 3 bytes) in /usr/lib/php/PEAR.php on line 844
Here's the code which inits the mail queue(inside a class called Newsletter)
//I know passing the DSN as the string is kind of deprecated,
//but it;'s the only way it works on my system
$dsn ="mysql://$db_user:$db_pass#$db_host/$db_name";
$db_options = array();
$db_options['type'] = 'db';
$db_options['dsn'] = $dsn;
$db_options['mail_table'] = TABLE_QUEUE;
$this->host = '-- valid host here --';//data in these strings has been obscured
$this->username = '-- valid username here --';
$this->password = '-- valid password here --';
//optionally, a 'dns' string can be provided instead of db parameters.
//look at DB::connect() method or at DB or MDB docs for details.
//you could also use mdb container instead db
//$server = isset($_SERVER['SERVER_NAME'])?$_SERVER['SERVER_NAME']:'localhost';
$mail_options = array(
'driver' => 'smtp',
'host' => $this->host,
'port' => 25,
'auth' => true,
'username' => $this->username,
'password' => $this->password,
);
$this->mail_queue = new Mail_Queue($db_options, $mail_options);
Some more code down the line,
public function sendChunk($start, $count)
{
global $db;
$ids = $db->get_results("SELECT DISTINCT id_user as id FROM ".TABLE_QUEUE);
$ret = array();
foreach ($ids as $id)
$ret[] = $id->id;
unset($ids);
$this->mail_queue->sendMailsInQueue($count, $start, 1);
return true;
}
Problem is, I double checked every line of code I wrote, and it's doing it's job. Only problem is that sometimes it refuses to send any mails.
Thanks in advance for replies.
I switched to MDB2 instead of DB
$db_options['type'] = 'db';
to
$db_options['type'] = 'mdb2';
this helped in taking care of memory exhaust problem, I am still looking to take care of initialize container in /usr/lib/php/PEAR.php problem
Ok found the solution for container errors:
Apply this patch
http://svn.php.net/viewvc/pear/packages/Mail_Queue/trunk/Mail/Queue.php?r1=303876&r2=309126
Try to limit result. using limit in your select statement.
Try to flush the old main queye.