I would like to use a Timer in php, sending messages after a certain period of time (no player activity)? This is for a multi-player game through sockets.
I have googled and stackoverflew and found the following
could use a cron (will need to
synronize my php classes (socket
based) with the cron... ), No I
guess.
could use sleep ... if it could wait
for less than one second, I will have
given a try, No again, I guess
could use register_tick_function ...
not applicable I think
Any ideas?
Thanks!
This is what I wrote to deal with my timing needs. It doesn't magically interrupt your code, but it provides a very flexible way to schedule timers. All you have to do, is make sure you run $events->CheckTimers() every now and then.
The libevent module does a lot more, but that's a bit more effort.
$events = new Events();
function callback($text) {
printf("I'm a timer callback %s\n", $text);
}
$events->AddTimer("5s", "callback", "every 5 seconds", EVENTS::EVENT_REPEAT);
$events->AddTimer("2s..15s", "callback", "random time 2 to 15 seconds", EVENTS::EVENT_REPEAT);
$events->AddTimer("1m", create_function('', "echo 'Im a LAMBDA function that runs every minute\n';"), false, EVENTS::EVENT_REPEAT);
$events->AddTimer("1h..2h", "callback", "I will run once, in between 1 and 2 hours");
# This is just an example, in reality, you would make sure to call CheckTimer() regulary (ideally not more than once a second)
while (true) {
$events->CheckTimers();
printf("Next timer in %s seconds...\n", $ne = $events->GetNextTimer(), gettype($ne));
sleep(1);
}
class Events {
const EVENT_REPEAT = 0x0001;
const EVENT_SEQUENCE = 0x0002;
var $events;
var $timers;
function __construct() {
$this->events = array();
$this->timers = array();
}
function AddTimer($when, $action, $args = false, $flags = 0) {
if (preg_match('#([0-9a-zA-Z]+)..([0-9a-zA-Z]+)#', $when, $a)) {
$time = time(NULL) + rand($this->time2seconds($a[1]), $this->time2seconds($a[2]));
} else {
$time = time(NULL) + $this->time2seconds($when);
}
if ($flags & self::EVENT_SEQUENCE) {
while ($this->IsArrayCount($this->timers[$time])) {
$time ++;
}
}
$this->timers[$time][] = array("when" => $when, "action" => $action, "args" => $args, "flags" => $flags);
ksort($this->timers);
}
function GetNextTimer() {
if (!$this->IsArrayCount($this->timers)) {
return false;
}
reset($this->timers);
$firstevent = each($this->timers);
if ($firstevent === false) {
return false;
}
$time = $firstevent["key"];
$nextEvent = $time - time(NULL);
if ($nextEvent < 1) {
return 1;
}
return $nextEvent;
}
function CheckTimers() {
$rv = false;
$now = time(NULL);
foreach ($this->timers as $time => $events) {
if ($time > $now) {
break;
}
foreach ($events as $key => $event) {
# debug("Event::CheckTimer: {$event["action"]}");
# ircPubMsg("Event::CheckTimer: {$event["action"]}", "#bots");
if (!$event["args"]) {
call_user_func($event["action"]);
} else {
$rv = call_user_func_array($event["action"], is_array($event["args"]) ? $event["args"] : array($event["args"]));
}
unset($this->timers[$time][$key]);
if ($event["flags"] & self::EVENT_REPEAT) {
$this->AddTimer($event["when"], $event["action"], $event["args"], $event["flags"]);
}
if ($rv) {
# break;
}
}
if (!$this->IsArrayCount($this->timers[$time])) {
unset($this->timers[$time]);
}
if (0 && $rv) {
break;
}
}
if ($rv) {
return $rv;
}
}
function time2seconds($timeString) {
$end = substr($timeString, strlen($timeString) - 1);
$seconds = intval($timeString); // = preg_replace("#[^0-9]#", "", $a);
if (is_numeric($end)) {
return $seconds;
}
$unim = array("s","m","h","d", "w", "m", "y");
$divs = array(1, 60, 60, 24, 7, 28, 12, false);
$found = false;
while (!$found) {
$u = array_shift($unim);
$d = array_shift($divs);
$seconds *= $d;
if ($end === $u) {
return $seconds;
}
}
return intval($timeString);
}
function IsArrayCount($possibleArray) {
return (isset($possibleArray) && is_array($possibleArray)) ? count($possibleArray) : false;
}
}
Does PHP's time_nanosleep() work for you? Or usleep()?
Maybe a daemon?
http://kevin.vanzonneveld.net/techblog/article/create_daemons_in_php/
here are a few concepts how to do it :
1. common concept is to create script for sending messages, then add it to server Cron application and execute the script with specified time.
2. write python script that will control your php application file or section, in pythonic terms you can fork process on server start that will do your tasks every amount of time (with infinity loop I guess ... while 1: ..)
3. write shell .sh file that will invoke php or python script and add controls of this script it to /etc/init.d to be more controllable.
Related
I would like to know how to precisely track the script execution time and tapout before it reaches the timeout. For some reason there is a weird issue when multiple processes are running at the same time (different scripts, same file, with two browser tabs). The script execution time for the second script is twice as long compared to the script which was executed first. It would sound perfectly reasonable when the tracker would also account for that but for some reason the second script's timer seems to provide false estimates so i'm unable to tapout before it reaches timeout.
Here's the script which i'm running.
public function insertProducts(DB $db, $type){
//Import variations, first parent then children
$startedInsert = microtime(true);
list($variantProducts,$simpleProducts) = $this->prepareProducts($db,'insert',$type);
$importedSkus = [];
$importedIds = [];
$tracker = [];
switch($type){
case 'variable':
if(!empty($variantProducts)){
foreach($variantProducts as $variationIdentifier => $variants){
//if((microtime(true) - $startedInsert) / 60 >= 3) break;
$startVariant = microtime(true);
$existingVariantParent = $this->getVariantParent($db,$variationIdentifier);
$exists = false;
if($existingVariantParent){
$variantParent = $this->request('GET', 'v3/products/P'.$existingVariantParent.'T');
$parentId = $existingVariantParent;
$exists = isset($variantParent['code']) ? false : true;
}
// If the parent does not exist, insert a new one
if(!$exists){
$parent = $variants[0];
$parent['sku'] = 'P'.$variants[0]['sku'].'T';
$existingImages = $this->getMedias(array_column($parent['images'], 'name'));
if(!empty($existingImages)){
$parent['images'] = array_map(function($item) use($existingImages){
$item = current(array_filter($existingImages, function($exisitngImage) use($item){ return $item['name'] === $exisitngImage['name'] ? true : false; })) ?: $item;
return $item;
}, $parent['images']);
}
$variantParent = $this->request('POST', 'v3/products', $parent);
$tracker['variant']['parent'][] = $variantParent;
if(isset($variantParent['code'])){
if(isset($variantParent['data']['resource_id'])){
$parentId = $variantParent['data']['resource_id'];
}
}else{
$parentId = $variantParent['id'];
}
$this->setVariantParent($db,$variationIdentifier,$parentId,$variants[0]['sku']);
$importedSkus[] = $variants[0]['sku'];
}
if(!isset($parentId)) continue;
$importedIds[] = $parentId;
$tracker['variant']['parent']['stopwatch'][] = (microtime(true) - $startVariant) / 60;
$startVariants = microtime(true);
if(!empty($variants)){
$images = [];
$variantImageNames = array_unique(array_reduce($variants, function($carry, $item){
$carry = array_merge($carry, array_column($item['images'], 'name'));
return $carry;
}, []));
$existingImages = $this->getMedias($variantImageNames);
$variants = array_map(function($item) use($variantParent,&$images,$existingImages){
$attributes = [];
if(isset($variantParent['attributes'])){
foreach($item['attributes'] as $attr){
$attributes[] = array_reduce($variantParent['attributes'], function($carry, $item) use($attr){
if($attr['name'] === $item['name'] && $item['variation']){
$carry = [
'id' => $attr['id'],
'option' => $attr['options'][0]
];
}
return $carry;
});
}
$item['attributes'] = array_filter($attributes);
}
if(!empty($item['images'])){
if(!in_array($item['images'][0]['name'], $images)){
$imageName = $item['images'][0]['name'];
$item['image'] = current(array_filter($existingImages, function($exisitngImage) use ($imageName){ return $exisitngImage['name'] == $imageName ? true : false; })) ?: $item['images'][0];
$images[] = $imageName;
}
}
return $item;
},$variants);
//$variants
$variants = $this->request('POST', 'v3/products/'.$parentId.'/variations/batch', ['create' => $variants]);
$tracker['variant']['variants'][] = $variants;
$skus = array_reduce($variants['create'], function($carry, $item){
if(!isset($item['error'])){
$carry[] = $item['sku'];
}
return $carry;
});
if(!empty($skus)){
$importedSkus = array_merge($importedSkus,$skus);
}
}
$tracker['variant']['variant']['stopwatch'][] = (microtime(true) - $startVariants) / 60;
}
$tracker['endedVariant'] = (microtime(true) - $startedInsert) / 60;
}
break;
case 'simple':
if(!empty($simpleProducts)){
$simpleImageNames = array_unique(array_reduce($simpleProducts, function($carry, $item){
$carry = array_merge($carry, array_column($item['images'], 'name'));
return $carry;
}, []));
$existingImages = $this->getMedias($simpleImageNames);
$images = [];
$simpleProducts = array_map(function($item) use($existingImages,&$images){
foreach($item['images'] as $imageKey => $image){
$imageName = $image['name'];
$item['images'][$imageKey] = current(array_filter($existingImages, function($exisitngImage) use ($imageName){ return $exisitngImage['name'] == $imageName ? true : false; })) ?: $item['images'][$imageKey];
if(!isset($item['images'][$imageKey]['id'])){
if(!in_array($item['images'][$imageKey]['name'],$images)){
$images[] = $item['images'][$imageKey]['name'];
}else{
// Prevent duplicate image uploads on simple products
unset($item['images'][$imageKey]);
}
}
}
return $item;
},$simpleProducts);
$simpleProduct = $this->request('POST', 'v3/products/batch', ['create' => $simpleProducts]);
$importedSImples = array_reduce($simpleProduct['create'], function($carry, $item){
if(!isset($item['error'])){
$carry['skus'][] = $item['sku'];
$carry['ids'][] = $item['id'];
}else if(isset($item['error']['data']['resource_id'])){
$carry['ids'][] = $item['error']['data']['resource_id'];
}
return $carry;
},['skus' => [],'ids' => []]);
$importedSkus = array_merge($importedSkus,$importedSImples['skus']);
$importedIds = array_merge($importedSkus,$importedSImples['ids']);
$tracker['endedSimple'] = (microtime(true) - $startedInsert) / 60;
}
$tracker['simple'] = $simpleProduct;
break;
}
// $startDelete = microtime(true);
// $this->request('POST', 'v3/products/batch', ['delete' => $importedIds]);
// $endDelete = microtime(true);
// $tracker['deleted'] = ($endDelete - $startDelete) / 60;
if(!empty($importedIds)){
$existingProducts = $this->request('GET','v3/products', ['include' => $importedIds]);
$importedSkus = array_merge($importedSkus, array_column($existingProducts,'sku'));
}
if(!empty($importedSkus)){
$this->updateDB($db, 'insert',array_filter($importedSkus));
}
$tracker['imported_ids'] = $importedIds;
$tracker['imported_skus'] = $importedSkus;
var_dump($tracker);
}
At first i initate the tracker and prepare products for insertion
//Import variations, first parent then children
$startedInsert = microtime(true);
list($variantProducts,$simpleProducts) = $this->prepareProducts($db,'insert',$type);
Then incase the type is set variable i execute the variable product insertion and in a perfect scenario this would break out of the foreach loop, finish rest of the processes and end the script once it has reached to 3 minute line. Right now, this seems to malfunction.
//if((microtime(true) - $startedInsert) / 60 >= 3) break; // This would break out in 3 minutes
enter code here
At the end of this script i get how long did it take to execute the whole process for variable products insertion
$tracker['endedVariant'] = (microtime(true) - $startedInsert) / 60;
The same tracking logic applies to simple type as well, except there is no breaking out.
Now when i would init both of these scripts on different browser tabs, the simple product would show 1.5min and the variable approx 2. So a great idea would be to increase the payload right? The timeout happens at 300s so i would prefer if the scripts end a little before that and wait for the next execution.
Whenever i would increase the payload sent to the api, by a little, it timouts and rest of the script is ignored. The part which is suppose to breakout for variable products is ignored which makes me believe that the tracking is wrong.
I tested by starting both of these scripts on different browser tabs and also starting a stopwatch on my phone. The numbers don't match. For variable products the execution time is twice as long on the phone compared to the stopwatch that tracks the execution time. When i execute these scripts one at a time, the stopwatch is correct. What is happening here?
Btw, i am aware of increasing the timeout, i'm on a shared hosting so i'm limited to 300s. Also starting the script on shell, still timeouts on 300s. What i want to accomplish is to squeeze out as much processing as possible in a 5min/300sec timeframe. The scripts are later executed using cron jobs but for now i'm just testing these on a browser.
Currently, I am implementing the email sending function using Cakephp2.
This time we are implementing the email sending function with our own method without using CakeEmail, and we are loading the email template using render.
As a premise, this time we assume that we will deliver up to about 50,000 emails, and we can send the current 300 emails without any noticeable delay.
However, up to about 300 items can be sent 5 to 7 per second, but when it reaches about 1000, it will be sent once every 3 to 5 seconds.
Therefore, this time, I would like to make sure that I do not reduce the transmission speed of each mail.
Therefore, when checking the processing speed of each process, it was confirmed that the processing speed in the part executing render gradually increased, so the memory usage at the time of rendering may be a problem? I thought.
Please let me know if there is a cause or solution that increases memory usage by looping render.
function send()
{
$mails = $this->MailList->getSendMails();
foreach ($mails as $mail) {
if (!$this->saveTargetData($mail)) {
continue;
};
$limit = 300;
$offset = 0;
while ($targets = $this->MailDeliveryTarget->getTargets($mail,$limit,$offset)) {
foreach ($targets as $target) {
$result = $this->Sys->sendMailTemplate('reserved_mail',['input' => $input]);
if ($result) {
// success
} else {
// failure
}
}
unset($targets);
$offset += $limit;
}
}
}
function sendMailTemplate($tpl, $data = array())
{
$to = $this->getMailTo($tpl, $data);
$from = $this->getMailFrom($tpl, $data);
return $this->sendMail($to, $from, $this->getMailSubject($tpl, $data), $this->getMailBody($tpl, $data));
}
function getMailTo($tpl, $data)
{
return trim(convert_nl($this->_getMailTemplate($tpl."/to",$data), ""));
}
function _getMailTemplate($tpl, $data)
{
if (!is_file(dirname(dirname(dirname(__FILE__))).DS."Template".DS."mail".DS.str_replace("/", DS, $tpl).".txt")) {
return "";
}
$bk = array();
foreach (array("layout", "ext") as $v) {
$bk[$v] = $this->Controller->$v;
}
$bk_output = $this->Controller->response->body();
$this->Controller->layout = "";
$this->Controller->ext = ".txt";
$this->Controller->response->body("");
$this->Controller->set($data);
$this->Controller->mail_render = true;
$this->Controller->render("/".ltrim($tpl, "/"));
$out = $this->Controller->response->body();
$this->Controller->mail_render = false;
$this->Controller->response->body($bk_output);
foreach (array("layout", "ext") as $v) {
$this->Controller->$v = $bk[$v];
}
return $out;
}
Thank you.
How to check if one UNIX timestamp range is overlapping another UNIX timestamp range in PHP?
I am developing an application which takes future reservations. But, only one (1) reservation is allowed per period.
Example:
Mr. X has a reservation for a resource from 10:00 A.M. to 12:00 P.M. (noon). Later, Ms. Y wants to reserve that same resource from 8:00 A.M. to 11:00 P.M.. My application should reject Ms. Y's attempted reservation because it overlaps Mr. X's prior reservation.
I am storing the start and end times of existing reservations in UNIX timestamps (integers), but I could convert them into the following format "yyyy-mm-dd hh:mm:ss" if required, or vice versa.
I do not understand how to solve this problem. If I check the new start time with all the existing reservation start times, and the new end time in a similar fashion, the logic will have many if statements and make the application slow.
Would you please help me to solve this issue in an efficient way without using lots of server resources.
Your help is greatly appreciated.
Thanks
Introduction
In other words, you need to do a comparison of all reservation intervals (UNIX timestamps) for a particular resource to determine if a new reservation is valid (within the domain for new reservations).
Step 1
First, a SQL query similar to this might help. While key words like ANY, ALL, NOT, EXISTS and others may seem tempting, it is up to you to decide how much information you need in the event of a scheduling conflict (based on your UI). This query provides the opportunity to extract the maximum amount of information (in PHP, etc ...) about a potential reservation with look ahead forecasting.
// A query like this might help. It's not perfect, but you get the idea.
// This query looks for ALL potential conflicts, starting and ending.
$sql = "SELECT DISTINCT `t1`.`startTime`, `t1`.`endTime`
FROM `reservations` AS `t1`
INNER JOIN `resources` AS `t2`
ON `t1`.`resourceId` = `t2`.`resourceId`
WHERE `t2`.`resourceId` = :resourceId
AND (`t1`.`startTime` BETWEEN :minTime1 AND :maxTime1)
OR (`t1`.`endTime` BETWEEN :minTime2 AND :maxTime2)
ORDER BY `t1`.`startTime` ASC";
Potentially. this will leave you with a multi-dimentional array. The following logic allows you to get a report detailing why the reservation cannot be made. It is up to you to interpret the report in another module.
Step 2
Generalize the solution as a methods of a Reservation class. Depending on your RDBMS, you may be able to do something like this in SQL. Although, it will probably be far less specific and you may want that granularity later. You could send the report in JSON format to a JavaScript front end (just something to think about).
private function inOpenDomain(array $exclusion, $testStart, $testEnd)
{
$result = null;
$code = null;
$start = $exclusion[0];
$end = $exclusion[1];
if (($testStart > $end) || ($testEnd < $start)) {
$result = true;
$code = 0; //Good! No conflict.
} elseif ($testStart === $start) {
$result = false;
$code = 1;
} elseif ($testStart === $end) {
$result = false;
$code = 2;
} elseif ($testEnd === $start) {
$result = false;
$code = 3;
} elseif ($testEnd === $end) {
$result = false;
$code = 4;
} elseif (($testStart > $start) && ($testEnd < $end)) { //Middle
$result = false;
$code = 5;
} elseif (($testStart < $start) && ($testEnd > $start)) { //Left limit
$result = false;
$code = 6;
} elseif (($testStart < $end) && ($testEnd > $end)) { //Right limit
$result = false;
$code = 7;
} elseif (($testStart < $start) && ($testEnd > $end)) { //Both limits
$result = false;
$code = 8;
} else {
$result = false;
$code = 9;
}
return ['start' => $start, 'end' => $end, 'result' => $result => 'code' => $code];
}
Step 3
Make a method that manages the checking of prior reservation times (assuming PDO::FETCH_ASSOC).
private function checkPeriods(array $periods, $newStartTime, $newEndTime)
{
$report = [];
if (!isset($periods[0])) { //If NOT multi-dimensional
$report = inOpenDomain($periods, $newStartTime, $newEndTime)
} else {
for ($i = 0, $length = $count($periods); $i < $length; ++$i) {
$report[$i] = inOpenDomain($periods[$i], $newStartTime, $newEndTime);
}
}
return $report;
}
Step 4
Fashion a method for doing a SELECT on the reservations table using a PDO prepared statement. Generally, ...
private function getReservationTimes($resourceId, $minTime, $maxTime)
{
$sql = "SELECT DISTINCT `t1`.`startTime`, `t1`.`endTime`
FROM `reservations` AS `t1`
INNER JOIN `resources` AS `t2`
ON `t1`.`resourceId` = `t2`.`resourceId`
WHERE `t2`.`resourceId` = :resourceId
AND (`t1`.`startTime` BETWEEN :minTime1 AND :maxTime1)
OR (`t1`.`endTime` BETWEEN :minTime2 AND :maxTime2)
ORDER BY `t1`.`startTime` ASC";
$stmt = $this->db->prepare($sql);
$stmt->bindParam(:resourceId , $resourceId);
$stmt->bindParam(:minTime1 , $minTime);
$stmt->bindParam(:maxTime1 , $maxTime);
$stmt->bindParam(:minTime2 , $minTime);
$stmt->bindParam(:maxTime2 , $maxTime);
$stmt->execute();
return $stmt->fetchAll();
}
Step 5
Make a public method (interface) for the entire process.
public function isOpen($minTime, $maxTime)
{
$periods = $this->getReservationTimes($this->resource->getResourceId(), $minTime, $maxTime);
if (empty($periods)) {
return true; //You may reserve the resource during the time period.
}
return $this->checkPeriods($periods, $this->start, $this->end));
}
Step 6
Separate the concerns.
Create a class hierarchy for the actual items being reserved.
abstact class Product
{
}
class Resource extends Product implements Reservable //Or, something ...
{
private $resourceId;
//etc ....
}
Create a class hierarchy for reservations.
abstract class Interval
{
private $start;
private $end;
public function __construct($start, $end)
{
$this->start = $start;
$this->end = $end;
}
}
class Reservation extends Interval
{
private $db;
private $resource;
public function __construct(PDO $pdo, Reservable $resource, $reqStartTime, $reqEndTime)
{
parent::__construct($reqStartTime, $reqEndTime);
$this->db = $pdo;
$this->resource = $resource;
}
}
Step 7
Run within try/catch
When you instantiate the Reservation object, supply at least a Reservable object, the requested start time, and requested end time (as UNIX timestamps, in this case).
try
{
$day = 84600; // Seconds per day.
$future = $day * 90; // Application specific.
//User requested times.
$reqStartTime = 1488394687 + $day; // Tomorrow.
$reqEndTime = 1488394687 + ($day * 2); // Two day duration.
//Search constraints.
$minTime = time(); // Today. Right now.
$maxTime = 1488394687 + $future; // 90 day look ahead.
$reservation = new Reservation($pdo, $resourceObj, $reqStartTime, $reqEndTime);
$availability = $reservation->isOpen($minTime, $maxTime);
if($availability === true){
$reservation->confirm();
} else {
//Have some other object deal with the report
$reporter = new Reporter($availability);
$reporter->analyzeReport();
//Have some other object update the view, etc ...
}
}
catch(Exception $e)
{
//Handle it.
}
Update, 2013-09-12:
I've dug a bit deeper into systemd and it's journal, and, I've stumbled upon this, that states:
systemd-journald will forward all received log messages to the AF_UNIX SOCK_DGRAM socket /run/systemd/journal/syslog, if it exists, which may be used by Unix syslog daemons to process the data further.
As per manpage, I did set up my environment to also have syslog underneath, I've tweaked my code accordingly:
define('NL', "\n\r");
$log = function ()
{
if (func_num_args() >= 1)
{
$message = call_user_func_array('sprintf', func_get_args());
echo '[' . date('r') . '] ' . $message . NL;
}
};
$syslog = '/var/run/systemd/journal/syslog';
$sock = socket_create(AF_UNIX, SOCK_DGRAM, 0);
$connection = socket_connect($sock, $syslog);
if (!$connection)
{
$log('Couldn\'t connect to ' . $syslog);
}
else
{
$log('Connected to ' . $syslog);
$readables = array($sock);
socket_set_nonblock($sock);
while (true)
{
$read = $readables;
$write = $readables;
$except = $readables;
$select = socket_select($read, $write, $except, 0);
$log('Changes: %d.', $select);
$log('-------');
$log('Read: %d.', count($read));
$log('Write: %d.', count($write));
$log('Except: %d.', count($except));
if ($select > 0)
{
if ($read)
{
foreach ($read as $readable)
{
$data = socket_read($readable, 4096, PHP_BINARY_READ);
if ($data === false)
{
$log(socket_last_error() . ': ' . socket_strerror(socket_last_error()));
}
else if (!empty($data))
{
$log($data);
}
else
{
$log('Read empty.');
}
}
}
if ($write)
{
foreach ($write as $writable)
{
$data = socket_read($writable, 4096, PHP_BINARY_READ);
if ($data === false)
{
$log(socket_last_error() . ': ' . socket_strerror(socket_last_error()));
}
else if (!empty($data))
{
$log($data);
}
else
{
$log('Write empty.');
}
}
}
}
}
}
This apparently, only sees (selects) changes on write sockets. Well, might be that something here is wrong so I attempted to read from them, no luck (nor there should be):
[Thu, 12 Sep 2013 14:45:15 +0300] Changes: 1.
[Thu, 12 Sep 2013 14:45:15 +0300] -------
[Thu, 12 Sep 2013 14:45:15 +0300] Read: 0.
[Thu, 12 Sep 2013 14:45:15 +0300] Write: 1.
[Thu, 12 Sep 2013 14:45:15 +0300] Except: 0.
[Thu, 12 Sep 2013 14:45:15 +0300] 11: Resource temporarily unavailable
Now, this drives me nuts a little. syslog documentation says this should be possible. What is wrong with the code?
Original:
I had a working prototype, by simply:
while(true)
{
exec('journalctl -r -n 1 | more', $result, $exit);
// do stuff
}
But this feels wrong, and consumes too much system resources, then I found out about journald having sockets.
I have attempted to connect and read from:
AF_UNIX, SOCK_DGRAM : /var/run/systemd/journal/socket
AF_UNIX, SOCK_STREAM : /var/run/systemd/journal/stdout
the given sockets.
With /var/run/systemd/journal/socket, socket_select sees 0 changes. With /var/run/systemd/journal/stdout I always (every loop) get 1 change, with 0 byte data.
This is my "reader":
<?php
define('NL', "\n\r");
$journal = '/var/run/systemd/journal/socket';
$jSTDOUT = '/var/run/systemd/journal/stdout';
$journal = $jSTDOUT;
$sock = socket_create(AF_UNIX, SOCK_STREAM, 0);
$connection = #socket_connect($sock, $journal);
$log = function ($message)
{
echo '[' . date('r') . '] ' . $message . NL;
};
if (!$connection)
{
$log('Couldn\'t connect to ' . $journal);
}
else
{
$log('Connected to ' . $journal);
$readables = array($sock);
while (true)
{
$read = $readables;
if (socket_select($read, $write = NULL, $except = NULL, 0) < 1)
{
continue;
}
foreach ($read as $read_socket)
{
$data = #socket_read($read_socket, 1024, PHP_BINARY_READ);
if ($data === false)
{
$log('Couldn\'t read.');
socket_shutdown($read_socket, 2);
socket_close($read_socket);
$log('Server terminated.');
break 2;
}
$data = trim($data);
if (!empty($data))
{
$log($data);
}
}
}
$log('Exiting.');
}
Having no data in read socket(s), I assume I'm doing something wrong.
Question, idea:
My goal is to read the messages and upon some of them, execute a callback.
Could anyone point me into the right direction of how to programmatically read journal messages?
The sockets under /run/systemd/journal/ won't work for this – …/socket and …/stdout are actually write-only (i.e. used for feeding data into the journal) while the …/syslog socket is not supposed to be used by anything else than a real syslogd, not to mention journald does not send any metadata over it. (In fact, the …/syslog socket doesn't even exist by default – syslogd must actually listen on it, and journald connects to it.)
The official method is to read directly from the journal files, and use inotify to watch for changes (which is the same thing journalctl --follow and even tail -f /var/log/syslog use in place of polling). In a C program, you can use the functions from libsystemd-journal, which will do the necessary parsing and even filtering for you.
In other languages, you have three choices: call the C library; parse the journal files yourself (the format is documented); or fork journalctl --follow which can be told to output JSON-formatted entries (or the more verbose journal export format). The third option actually works very well, since it only forks a single process for the entire stream; I have written a PHP wrapper for it (see below).
Recent systemd versions (v193) also come with systemd-journal-gatewayd, which is essentially a HTTP-based version of journalctl; that is, you can get a JSON or journal-export stream at http://localhost:19531/entries. (Both gatewayd and journalctl even support server-sent events for accessing the stream from HTML 5 webpages.) However, due to obvious security issues, gatewayd is disabled by default.
Attachment: PHP wrapper for journalctl --follow
<?php
/* © 2013 Mantas Mikulėnas <grawity#gmail.com>
* Released under the MIT Expat License <https://opensource.org/licenses/MIT>
*/
/* Iterator extends Traversable {
void rewind()
boolean valid()
void next()
mixed current()
scalar key()
}
calls: rewind, valid==true, current, key
next, valid==true, current, key
next, valid==false
*/
class Journal implements Iterator {
private $filter;
private $startpos;
private $proc;
private $stdout;
private $entry;
static function _join_argv($argv) {
return implode(" ",
array_map(function($a) {
return strlen($a) ? escapeshellarg($a) : "''";
}, $argv));
}
function __construct($filter=[], $cursor=null) {
$this->filter = $filter;
$this->startpos = $cursor;
}
function _close_journal() {
if ($this->stdout) {
fclose($this->stdout);
$this->stdout = null;
}
if ($this->proc) {
proc_close($this->proc);
$this->proc = null;
}
$this->entry = null;
}
function _open_journal($filter=[], $cursor=null) {
if ($this->proc)
$this->_close_journal();
$this->filter = $filter;
$this->startpos = $cursor;
$cmd = ["journalctl", "-f", "-o", "json"];
if ($cursor) {
$cmd[] = "-c";
$cmd[] = $cursor;
}
$cmd = array_merge($cmd, $filter);
$cmd = self::_join_argv($cmd);
$fdspec = [
0 => ["file", "/dev/null", "r"],
1 => ["pipe", "w"],
2 => ["file", "/dev/null", "w"],
];
$this->proc = proc_open($cmd, $fdspec, $fds);
if (!$this->proc)
return false;
$this->stdout = $fds[1];
}
function seek($cursor) {
$this->_open_journal($this->filter, $cursor);
}
function rewind() {
$this->seek($this->startpos);
}
function next() {
$line = fgets($this->stdout);
if ($line === false)
$this->entry = false;
else
$this->entry = json_decode($line);
}
function valid() {
return ($this->entry !== false);
/* null is valid, it just means next() hasn't been called yet */
}
function current() {
if (!$this->entry)
$this->next();
return $this->entry;
}
function key() {
if (!$this->entry)
$this->next();
return $this->entry->__CURSOR;
}
}
$a = new Journal();
foreach ($a as $cursor => $item) {
echo "================\n";
var_dump($cursor);
//print_r($item);
if ($item)
var_dump($item->MESSAGE);
}
I made a couple related threads but this is the one direct question that I'm seeking the answer for. My framework will use Zend_Translate if the php version is 5, otherwise I have to mimic the functionality for 4.
It seems that pretty much every implementation of gettext relies on setlocale or locales, I know there's a LOT of inconsistency across systems which is why I don't want to rely upon it.
I've tried a couple times to get the textdomain, bindtextdomain and gettext functions to work but I've always needed to invoke setlocale.
By the way, all the .mo files will be UTF-8.
Here's some reusable code to parse MO files in PHP, based on Zend_Translate_Adapter_Gettext:
<?php
class MoParser {
private $_bigEndian = false;
private $_file = false;
private $_data = array();
private function _readMOData($bytes)
{
if ($this->_bigEndian === false) {
return unpack('V' . $bytes, fread($this->_file, 4 * $bytes));
} else {
return unpack('N' . $bytes, fread($this->_file, 4 * $bytes));
}
}
public function loadTranslationData($filename, $locale)
{
$this->_data = array();
$this->_bigEndian = false;
$this->_file = #fopen($filename, 'rb');
if (!$this->_file) throw new Exception('Error opening translation file \'' . $filename . '\'.');
if (#filesize($filename) < 10) throw new Exception('\'' . $filename . '\' is not a gettext file');
// get Endian
$input = $this->_readMOData(1);
if (strtolower(substr(dechex($input[1]), -8)) == "950412de") {
$this->_bigEndian = false;
} else if (strtolower(substr(dechex($input[1]), -8)) == "de120495") {
$this->_bigEndian = true;
} else {
throw new Exception('\'' . $filename . '\' is not a gettext file');
}
// read revision - not supported for now
$input = $this->_readMOData(1);
// number of bytes
$input = $this->_readMOData(1);
$total = $input[1];
// number of original strings
$input = $this->_readMOData(1);
$OOffset = $input[1];
// number of translation strings
$input = $this->_readMOData(1);
$TOffset = $input[1];
// fill the original table
fseek($this->_file, $OOffset);
$origtemp = $this->_readMOData(2 * $total);
fseek($this->_file, $TOffset);
$transtemp = $this->_readMOData(2 * $total);
for($count = 0; $count < $total; ++$count) {
if ($origtemp[$count * 2 + 1] != 0) {
fseek($this->_file, $origtemp[$count * 2 + 2]);
$original = #fread($this->_file, $origtemp[$count * 2 + 1]);
$original = explode("\0", $original);
} else {
$original[0] = '';
}
if ($transtemp[$count * 2 + 1] != 0) {
fseek($this->_file, $transtemp[$count * 2 + 2]);
$translate = fread($this->_file, $transtemp[$count * 2 + 1]);
$translate = explode("\0", $translate);
if ((count($original) > 1) && (count($translate) > 1)) {
$this->_data[$locale][$original[0]] = $translate;
array_shift($original);
foreach ($original as $orig) {
$this->_data[$locale][$orig] = '';
}
} else {
$this->_data[$locale][$original[0]] = $translate[0];
}
}
}
$this->_data[$locale][''] = trim($this->_data[$locale]['']);
unset($this->_data[$locale]['']);
return $this->_data;
}
}
Ok, I basically ended up writing a mo file parser based on Zend's Gettext Adapter, as far as I know gettext is pretty much reliant upon the locale, so manually parsing the .mo file would save the hassle of running into odd circumstances with locale issues with setlocale. I also plan on parsing the Zend Locale data provided in the form of xml files.