I have built an api that is called 20 times in a second to perform a function that establishes connection to a postgresql but anytime the call is made the system memory gets full. I am a novie in postgresql and any assistance will be appreciated. Below is my php code
try
{
$input=file_get_contents("php://input");
$pgconn = new PgSql();
$selectRecords="SELECT messageid, msisdn, smsmessage, serviceid, isbilled, linkid FROM sdpmtn_mt.smssendingtable WHERE priority = 0 LIMIT 1";
foreach($pgconn->getRows($selectRecords) as $rows){
$msisdn = $rows->msisdn;
$message = $rows->smsmessage;
$serviceid = $rows->serviceid;
$isbilled = $rows->isbilled;
$linkid = $rows->linkid;
$msgid = $rows->messageid;
}
}
catch(Exception $e)
{
print $e->getMessage();
}
I am calling this API twenty times in a second. Please assist with any configurations I have to make.
Related
I'd like my PHP script (using PDO) to detect whether or not a target database is in the middle of a restore process other than waiting several minutes for a response from a failed connection.
My database connection code eventually returns the message below if a database is being restored, but it happens because the connection fails and it takes several minutes to respond when this happens. Searching on StackOverflow and Google doesn't seem to find anything that fits my need, nor does searching through PHP's documentation.
function getParameterizedPDOConnection($host = false, $overrideOptions = []) {
include(INCLUDE_DIR . "/itrain.config.php");
$host = strtolower($_SERVER["HTTP_HOST"]);
if (count($overrideOptions) > 0) {
$configOptions["host"][$host] = $overrideOptions;
}
$sthUserName = $configOptions["userName"];
$pwd = $configOptions["pwd"];
$addr = $configOptions["host"][$host]["addr"];
$db = $configOptions["host"][$host]["db"];
try {
$pdo = new PDO("sqlsrv:Server=$addr;Database=$db;ConnectionPooling=0", $sthUserName, $pwd, array(PDO::ATTR_ERRMODE => PDO::ERRMODE_SILENT));
return($pdo);
} catch (PDOException $e) {
return "Database connection failure: " . $e->getMessage();
}
}
Returns: "42000",927,"[Microsoft][ODBC Driver 13 for SQL Server][SQL Server]Database 'Workforce' cannot be opened. It is in the middle of a restore.
My code:
<?php
try {
$t = '040485c4-2eba-11e9-8e3c-0231844357e8';
if (array_key_exists('t', $_REQUEST)) {
$t = $_REQUEST["t"];
}
if (!isset($_COOKIE['writer'])) {
header("Location: xxx");
return 0;
}
$writer = $_COOKIE['writer'];
$dbhost = $_SERVER['RDS_HOSTNAME'];
$dbport = $_SERVER['RDS_PORT'];
$dbname = $_SERVER['RDS_DB_NAME'];
$charset = 'utf8' ;
$dsn = "mysql:host={$dbhost};port={$dbport};dbname={$dbname};charset={$charset}";
$username = $_SERVER['RDS_USERNAME'];
$password = $_SERVER['RDS_PASSWORD'];
$pdo = new PDO($dsn, $username, $password);
$stmt = $pdo->prepare("select writer from mydbtbl where writer=? and t=?");
$stmt->execute(array($writer, $t));
$num = $stmt->fetch(PDO::FETCH_NUM);
if ($num < 1) {
header("Location: login.php");
return 0;
}
$dbMsg = "Authorized";
$dbname = 'imgs';
$dsn = "mysql:host={$dbhost};port={$dbport};dbname={$dbname};charset={$charset}";
$pdo = new PDO($dsn, $username, $password);
if (isset($_FILES['filename'])) {
$name = $_FILES['filename']['name'];
// set path of uploaded file
$path = "./".basename($_FILES['filename']['name']);
// move file to current directory
move_uploaded_file($_FILES['filename']['tmp_name'], $path);
// get file contents
$data = file_get_contents($path, NULL, NULL, 0, 60000);
$stmt = $pdo->prepare("INSERT INTO file (contents, filename, t) values (?,?,?)");
$stmt->execute(array
($data,
$name,
$t)
);
$dbMsg = "Added the file to the repository";
// delete the file
unlink($path);
}
} catch (Exception $e) {
$dbMsg = "exception: " . $e->getMessage();
}
In the code you will see that the first part is for doing authentication. Then I create a new PDO object on the img schema, and do my file insert query after that.
Later, where I am printing out $dbMsg, it is saying "added file to the repository". But when I query the database (MySQL on Amazon AWS using MySQL Workbench) nothing has been inserted.
I don't understand why if nothing is getting inserted I am not getting an error message. If it says "added file to the respository", doesn't that mean the insert was successful? The only thing I can think is that using a different schema for this is mucking things up. All of my inserts to ebdb are going through fine
--- EDIT ---
This question was marked as a possible duplicate on my query about not getting an error message on my insert / execute code. This was a useful link and definitely something I will be aware of and check in the future, but ultimately the answer is the one I have provided regarding the terms of service for my aws account
The answer is that the (free) amazon account policy I am working under only allows me to have 1 database / schema. When I switched the table over to ebdb it worked right away. I am answering my own question (rather than deleting) so hopefully others using AWS / MySQL can learn from my experience.
I have an application where each user gets their own database where each database has the same schema.
I have a script that performs migrations in this fashion:
SHOW databases
Iterate through databases
Execute sql statements
This can take a long time when there are complicated queries that take a lot of time (3 or more seconds)
Is there a way where I can run the sql statements for each database at the same time from one script? Is this dangerous/too resource intensive to do this?
The reason I want to do this, is to prevent downtime as much as possible.
Here is my script now:
<?php
ini_set('display_errors', 1);
set_time_limit(0);
$sql = file_get_contents('../update.sql');
$sql_queries = explode(";", $sql);
$exclude_dbs = array('horde','phppoint_forums','phppoint_site','roundcube', 'pos', 'bntennis_site', 'mysql', 'information_schema', 'performance_schema');
$conn = mysqli_connect("localhost", "root", "PASSWORD");
$show_db_query = mysqli_query($conn, 'SHOW databases');
$databases = array();
while ($row = mysqli_fetch_assoc($show_db_query))
{
if (!in_array($row['Database'], $exclude_dbs))
{
$databases[] = $row['Database'];
}
}
foreach($databases as $database)
{
mysqli_select_db($conn, $database);
echo "Running queries on $database\n***********************************\n";
foreach($sql_queries as $query)
{
if (!empty($query))
{
echo "$query;";
if (!mysqli_query($conn, $query))
{
echo "\n\nERROR: ".mysqli_error($conn)."\n\n";
}
}
}
echo "\n\n";
}
?>
I don't know if the database will hold for that load but basically I would fork the process or spawn it into the background, depending on language.
For php you can fork the process for each database basically running the migrations in parallel.
i have a cron Job running following script every 5 min. However it seems that the script doesn't closes the connection once its run . How can I close the connection in this script?
function __construct($config)
{
$this->server = $config['server'];
$this->certificate = $config['certificate'];
$this->passphrase = $config['passphrase'];
// Create a connection to the database.
$this->pdo = new PDO(
'mysql:host=' . $config['db']['host'] . ';dbname=' . $config['db']['dbname'],
$config['db']['username'],
$config['db']['password'],
array());
// If there is an error executing database queries, we want PDO to
// throw an exception.
$this->pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
// We want the database to handle all strings as UTF-8.
$this->pdo->query('SET NAMES utf8');
}
// This is the main loop for this script. It polls the database for new
// messages, sends them to APNS, sleeps for a few seconds, and repeats this
// forever (or until a fatal error occurs and the script exits).
function start()
{
writeToLog('Connecting to ' . $this->server);
if (!$this->connectToAPNS())
exit;
while (true)
{
// Do at most 20 messages at a time. Note: we send each message in
// a separate packet to APNS. It would be more efficient if we
// combined several messages into one packet, but this script isn't
// smart enough to do that. ;-)
$stmt = $this->pdo->prepare('SELECT * FROM messages WHERE time_sent IS NULL LIMIT 20');
$stmt->execute();
$messages = $stmt->fetchAll(PDO::FETCH_OBJ);
foreach ($messages as $message)
{
if ($this->sendNotification($message->id, $message->token, $message->payload))
{
$stmt = $this->pdo->prepare('UPDATE messages SET time_sent = NOW() WHERE id = ?');
$stmt->execute(array($message->id));
}
else // failed to deliver
{
$this->reconnectToAPNS();
}
}
unset($messages);
sleep(5);
}
}
I may have misread, but apparently you are launching every 5 minutes a script that doesn't exit (infinite loop). So you're stacking the instances, until Earth (or more modestly, your server) eventually explodes.
To answer your question, PHP automatically frees all resources, including DB connexions, when the script execution has ended.
When a script runs for a very long time (like infinite), or there are very specific memory considerations, one can manually frees resources using unset($foo) or $foo = null.[1]
DB connexions can be closed and freed this way too, just with unset($this->pdo).
[1] see also What's better at freeing memory with PHP: unset() or $var = null
Let me rephrase my question, I have a mysql database that is holding emails to be sent, on a shared host. I would like to run a cron job that will read the database and sent out any messages in the database every 10 minutes or so.
Now my question is, what is the best way with php to read my database and send out the emails in small batched so that I don't overwhelm the shared host.
Assuming the use of PDO, and making some accommodation for not knowing your schema, it might look something like this:
$dbh = new PDO('mysql:dbname=testdb;host=127.0.0.1', 'dbuser', 'dbpass');
$msgh = $dbh->prepare('SELECT subject, body from message where listname = :listname');
$msgh->bindParam(':listname', $listname, PDO::PARAM_STR);
$msgh->execute();
$msg = $msgh->fetch(PDO::FETCH_ASSOC);
$usrh = $dbh->prepare('SELECT recipient from userlist where is_subscribed_to = :listname');
$usrh->bindParam(':listname', $listname, PDO::PARAM_STR);
$usrh->execute();
while ($recipient = $usrh->fetch(PDO::FETCH_ASSOC)) {
mail($recipient, $msg['subject'], $msg['body']);
if ($over_throttle) {
sleep(THROTTLE_SLEEP_SECONDS);
$over_throttle = 0;
}
++$over_throttle;
}
As for 'prewritten', you might take a look at phplist.
I would leave the throttling to the email server. Ie, run an email server locally, and have your PHP code relay all these messages to that. Then configure the email server itself to only send out at a certain rate.
Well I came up with this solution similar to the PDO one. Are there any unforeseen problems with running this as a cron job?
<?php
$con = mysql_connect("localhost","root","123456");
$throttle = 0;
$batch = 50;
$pause = 10; // seconds
if (!$con)
{
die('Could not connect: ' . mysql_error());
}
mysql_select_db("maildb", $con);
// Message Table
$MSGresult = mysql_query("SELECT * FROM msgs");
// User Table
$USERresult = mysql_query("SELECT * FROM members");
while($MSGrow = mysql_fetch_array($MSGresult))
{
while($USERrow = mysql_fetch_array($USERresult))
{
mail($USERrow['email'],$MSGrow['subject'],$MSGrow['body']);
$throttle += 1;
if ($throttle > $batch ) { sleep($pause); $throttle = 0;}
}
mysql_data_seek($USERresult,0);
}
mysql_close($con);
?>