I have a project leash and in JSON-RPC service I'm writing to a file in __destruct, but sometimes the config file is cleared (it look like this {"sessions":[],"users":[]}) it happen from time to time and also when I refresh the page fast before ajax request is finished, it look like the __constructor didn't finished before __destruct is called, I have code like this:
class Service {
protected $config_file;
protected $config;
const password_hash = 'h'; // function use for password on installation
const password_regex = '/([A-Za-z_][A-Za-z0-9_]*):(.*)/';
function __construct($config_file, $path) {
$this->path = $path;
$this->config_file = $config_file;
$full_path = $path . "/" . $this->config_file;
if (file_exists($full_path)) {
try {
$this->config = json_decode(file_get_contents($full_path));
} catch (Exception $e) {
$this->config = new stdClass();
}
// it had no write permission when first created while testing
if (!is_writable($full_path)) {
chmod($full_path, 0664);
}
} else {
$this->config = new stdClass();
}
if (!isset($this->config->sessions) || !is_array($this->config->sessions)) {
$this->config->sessions = array();
} else {
$this->config->sessions = array_map(function($session) {
return Session::cast($session);
}, array_filter($this->config->sessions, function($session){
return isset($session->token) && isset($session->username);
}));
}
if (!isset($this->config->users) || !is_array($this->config->sessions)) {
$this->config->users = array();
}
}
// ------------------------------------------------------------------------
function __destruct() {
$path = $this->path . "/" . $this->config_file;
$this->__write($path, json_encode($this->config));
}
I have other methods that modify config object like login that add new session to $this->config->sessions array. Here is full file
I've tried to add flag $this->corrupted that's set to true when I've get exception but it didn't fix the issue, it must be something with time of __destruct being called.
I've also try to add flag $this->safe_to_save that get set to true in last line of constructor but that also didn't work. What may be the problem?
Related
How do i clear the cache of my patterned url (dynamic url)?
Like, /assets/js/#jsname.js...
I don't want to wait until the cache expired, i want to it, immediately after i updated things on my database.
They said that to clear the database, you need to specify the key for it. And i don't know what should i enter for the key :(
Here's my code:
\F3::route("GET #virtualasset: /assets/img/#link/#size.#id.#type", "Control\\Imager->akses", 3600 * 24 * 7); // cache seminggu :3
HTTP responses are cached using the following key: hash(VERB URI).url.
So if you want to refresh assets/js/foo.js, you'll have to do:
$hash=$f3->hash('GET /assets/js/foo.js');
$cache->clear($hash.'.url');
But that's a bit of a hack since this key is not documented and may change in the future.
Also you need to remember that the route $ttl parameter not only triggers server cache but also client cache. Which means that you need to implement a cache buster to refresh the client cache. Something like assets/js/foo.js?v=3.
You may consider implementing cache directly in your controller class. That would give you more flexibility.
I use this route:
GET|HEAD /assets/#folder/#type/v-#version/* = controllers\Assets->send, 3600
My controller:
namespace controllers;
class Assets {
private $paths;
private $types = ['css', 'js', 'i', 'fonts', 'files', 'icons'];
private $dontminify = ['i', 'fonts', 'files', 'icons'];
public function __construct() {
$this->fw = \Base::instance();
$this->paths['app'] = '../app/views/site_tpl/assets/';
$this->paths['core'] = '../core/views/default/assets/';
}
public function send() {
if (in_array($this->fw->get('PARAMS.folder'), array_keys($this->paths))
&& in_array($this->fw->get('PARAMS.type'), $this->types))
{
$folder = $this->paths[$this->fw->get('PARAMS.folder')];
$type = $this->fw->get('PARAMS.type');
if (in_array($type, $this->dontminify)) {
$file = $folder . $type . '/' . $this->fw->pop('PARAMS');
if (\Web::instance()->send($file)) {
return true;
}
} else {
$this->fw->set('UI', $folder . $type . '/');
$file = $this->fw->pop('PARAMS');
if (file_exists($this->fw->get('UI') . $file)) {
if (!$this->fw->exists('GET.dontminify')) {
echo \Web::instance()->minify($file);
return true;
} elseif (\Web::instance()->send($this->fw->get('UI') . $file)) {
return true;
}
}
}
}
$this->fw->error(404);
}
}
So, when I want to update version of file, I write somethong like this:
<script src="/assets/app/js/v-20160205/main.js"></script>
3 days crashing my head towards a wall.
I developed a php script for import big text files and populate mysql database. Until i get 2 million records it works perfectly but i need to import like 10 million rows divided in different files.
My application scans files in a folder, get file extension (i have 4 kind of procedures import for 4 different extensions) and call the relative import function.
I have a structure made of theese classes:
CLASS SUBJECT1{ public function import_data_1(){
__DESTRUCT(){$this->childObject = null;}
IMPORT SUBJECT1(){
//fopen($file);
//ob_start();
//PDO::BeginTransaction();
//WHILE (FILE) {
//PREPARED STATEMENT
//FILE READING
//GET FILE LINE
//EXECUTE INSERT
//} END WHILE
//PDO::Commit();
//ob_clean(); or ob_flush();
//fclose($file);
//clearstatcache();
}
};}
CLASS SUBJECT2{ same as SUBJECT1;}
CLASS SUBJECT3{ same as SUBJECT1;}
CLASS SUBJECT4{ same as SUBJECT1;}
and the main class that launches the procedure:
CLASS MAIN{
switch($ext)
case "ext1":
$SUBJECT1 = new SUBJECT1();
IMPORT_SUBJECT1();
unset $SUBJECT1;
$SUBJECT1 = null;
break;
case "ext2": //SAME AS CASE ext1 WITH IMPORT_SUBJECT2();
case "ext3": //SAME AS CASE ext1 WITH IMPORT_SUBJECT3();
case "ext4": //SAME AS CASE ext1 WITH IMPORT_SUBJECT4();
}
It works perfectly with some adjustement of mysql file buffers (ib_logfile0 and ib_logfile1 are set as 512Mb).
The problem is that everytime a procedure is terminated php does not free memory. I'm sure that destructor is called (i put an echo inside __destruct method) and the object is not accesible (var_dump say is NULL). I tried so many ways to free memory but now i'm at a dead point.
I also verified
gc_collect_cycles()
in many different point of code and it always says 0 cycles so all abject are not referenced each other.
I tried even to delete class structure and call all the code sequential but i always get this error:
Fatal error: Out of memory (allocated 511180800) (tried to allocate 576 bytes) in C:\php\index.php on line 219 (line 219 is execute of a PS on the 13th file).
The memory is used in this way:
php script: 52MB
end first file import :110MB
destructors and unset calling: 110MB
new procedure calling: 110MB
end second file import 250MB
destructors and unset calling: 250MB
new procedure calling: 250MB
So as you can see even unsetting objects they don't free memory.
I tried setting php ini memory size to 1024M but it grows up really fast and crashes after 20 files.
Any advice?
Many thanks!
EDIT 1:
posting code:
class SUBJECT1{
public function __destruct()
{
echo 'destroying subject1 <br/>';
}
public function import_subject1($file,$par1,$par2){
global $pdo;
$aux = new AUX();
$log = new LOG();
// ---------------- FILES ----------------
$input_file = fopen($file, "r");
// ---------------- PREPARED STATEMENT ----------------
$PS_insert_data1= $pdo->prepare("INSERT INTO table (ID,PAR1,PAR2,PARN) VALUES (?,?,?,?) ON DUPLICATE KEY UPDATE ID = VALUES(ID), PAR1 = VALUES(PAR1), PAR2 = VALUES(PAR2), PAR3 = VALUES(PAR3), PARN = VALUES(PARN)");
$PS_insert_data2= $pdo->prepare("INSERT INTO table (ID,PAR1,PAR2,PARN) VALUES (?,?,?,?) ON DUPLICATE KEY UPDATE ID = VALUES(ID), PAR1 = VALUES(PAR1), PAR2 = VALUES(PAR2), PAR3 = VALUES(PAR3), PARN = VALUES(PARN)");
//IMPORT
if ($input_file) {
ob_start();
$pdo->beginTransaction();
while (($line = fgets($input_file)) !== false) {
$line = utf8_encode($line);
$array_line = explode("|", $line);
//set null values where i neeed
$array_line = $aux->null_value($array_line);
if(sizeof($array_line)>32){
if(!empty($array_line[25])){
$PS_insert_data1->execute($array_line[0],$array_line[1],$array_line[2],$array_line[5]);
}
$PS_insert_data2->execute($array_line[10],$array_line[11],$array_line[12],$array_line[15]);
}
$pdo->commit();
flush();
ob_clean();
fclose($f_titolarita);
clearstatcache();
}
I do this iterative for all files of my folder, the other procedures are the same concept.
I still have increase of memory and now it crashes with a white page response :-\
Personally, I would go slightly different about it. These are the steps I would do:
Open a PDO connection, set PDO in Exception mode
Get a list of files that I want to read
Create a class that can utilize PDO and the list of files and perform insertions
Prepare the statement ONCE, utilize it many times
Chunk PDO transaction commits to 50 (configurable) inserts - this means that every 50th time I call $stmt->execute(), I issue a commit - which utilizes the HDD better thus making it faster
Read each file line by line
Parse the line and check if it's valid
If yes, add to MySQL, if not - report an error
Now, I've created 2 classes and example on how I'd go about it. I tested only up to the reading part since I don't know your DB structure nor what AUX() does.
class ImportFiles
{
protected $pdo;
protected $statements;
protected $transaction = false;
protected $trx_flush_count = 50; // Commit the transaction at every 50 iterations
public function __construct(PDO $pdo = null)
{
$this->pdo = $pdo;
$this->stmt = $this->pdo->prepare("INSERT INTO table
(ID,PAR1,PAR2,PARN)
VALUES
(?,?,?,?)
ON DUPLICATE KEY UPDATE ID = VALUES(ID), PAR1 = VALUES(PAR1), PAR2 = VALUES(PAR2), PAR3 = VALUES(PAR3), PARN = VALUES(PARN)");
}
public function import($file)
{
if($this->isReadable($file))
{
$file = new FileParser($file);
$this->insert($file);
}
else
{
printf("\nSpecified file is not readable: %s", $file);
}
}
protected function isReadable($file)
{
return (is_file($file) && is_readable($file));
}
protected function insert(FileParser $file)
{
while($file->read())
{
//printf("\nLine %d, value: %s", $file->getLineCount(), $file->getLine());
$this->insertRecord($file);
$this->flush($file);
}
$this->flush(null);
}
// Untested method, no idea whether it does its job or not - might fail
protected function flush(FileParser $file = null)
{
if(!($file->getLineCount() % 50) && !is_null($file))
{
if($this->pdo->inTransaction())
{
$this->pdo->commit();
$this->pdo->beginTransaction();
}
}
else
{
if($this->pdo->inTransaction())
{
$this->pdo->commit();
}
}
}
protected function insertRecord(FileParser $file)
{
$check_value = $file->getParsedLine(25);
if(!empty($check_value))
{
$values = [
$file->getParsedLine[0],
$file->getParsedLine[1],
$file->getParsedLine[2],
$file->getParsedLine[5]
];
}
else
{
$values = [
$file->getParsedLine[10],
$file->getParsedLine[11],
$file->getParsedLine[12],
$file->getParsedLine[15]
];
}
$this->stmt->execute($values);
}
}
class FileParser
{
protected $fh;
protected $lineCount = 0;
protected $line = null;
protected $aux;
public function __construct($file)
{
$this->fh = fopen($file, 'r');
}
public function read()
{
$this->line = fgets($this->fh);
if($this->line !== false) $this->lineCount++;
return $this->line;
}
public function getLineCount()
{
return $this->lineCount;
}
public function getLine()
{
return $this->line;
}
public function getParsedLine($index = null)
{
$line = $this->line;
if(!is_null($line))
{
$line = utf8_encode($line);
$array_line = explode("|", $line);
//set null values where i neeed
$aux = $this->getAUX();
$array_line = $aux->null_value($array_line);
if(sizeof($array_line) > 32)
{
return is_null($index) ? $array_line : isset($array_line[$index]) ? $array_line[$index] : null;
}
else
{
throw new \Exception(sprintf("Invalid array size, expected > 32 got: %s", sizeof($array_line)));
}
}
else
{
return [];
}
}
protected function getAUX()
{
if(is_null($this->aux))
{
$this->aux = new AUX();
}
return $this->aux;
}
}
Usage:
$dsn = 'mysql:dbname=testdb;host=127.0.0.1';
$user = 'dbuser';
$password = 'dbpass';
try
{
$pdo = new PDO($dsn, $user, $password);
$pdo->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
$import = new ImportFiles($pdo);
$files = ['/usr/local/file1.txt', '/usr/local/file2.txt'];
foreach($files as $file)
{
$import->import($file);
}
} catch (Exception $e)
{
printf("\nError: %s", $e->getMessage());
printf("\nFile: %s", $e->getFile());
printf("\nLine: %s", $e->getLine());
}
SOLVED:
i did this approach, maybe is useful for someone who has similar problem:
I opened task manager and looked at memory usage for apache and mysql processes with these cases:
Tried to read and elaborate files without calling MySql procedures (memory usage was ok)
Tried to read, elaborate and inserting in db just files with extension one by one (all .ext1, all .ext2, ....)
Debugged the procedure with big memory encreasing isolating functions one by one finding the problematic one.
Found the problem and solved
The problem was that i called a function passing as parameter the Prepared Statement. I thought that, once prepared, it was just a "static" object to call. What happens is that if you pass the same PS in a function the memory grows up exponentially.
Hope this helps to someone.
Bye!
I have a class which is meant to "load" an another class, however I haven't been able to get it to work.
Error Message
Fatal error: Call to undefined method stdClass::echoString() in C:\Program Files (x86)\EasyPHP-DevServer-14.1VC11\data\localweb\classes\example.php on line 5
Code
My code is broken up into three main sections:
api.php - the class to load the other classes.
API/exampleExternalAPI.php - (multiple files) the classes that api.php loads
example.php - the file that uses the main class (api.php)
If it helps these files can be downloaded from my dropbox
api.php
<?php
/* Config */
define('pathToAPIs','API/');
/* Autoload Function */
spl_autoload_register(function($className){
$namespace=str_replace("\\","/",__NAMESPACE__);
$className=str_replace("\\","/",$className);
$class=pathToAPIs.(empty($namespace)?"":$namespace."/")."{$className}.php";
include_once($class);
});
class api {
private $listOfAPIs;
public $APIs;
public function __construct($setAPI = null){
$this->updateListOfAPIs();
if (isset($setAPI)){
return $this->setAPI($setAPI);
}
}
public function setAPIs($setAPIs){
$this->APIs = null; // clears a previous call to this method
if (!is_array($setAPIs)){ // if not an array
$setAPIs = array($setAPIs); // make array
}
foreach ($setAPIs as $setAPIType){
if(in_array($setAPIType,$this->listOfAPIs)){
$array[$setAPIType] = new $setAPIType;
}
}
$this->APIs = json_decode(json_encode($array), FALSE); // convert array of required api objects to an object
return $this->APIs;
}
public function getListOfAPIs($update = false){
if ($update){
$this->updateListOfAPIs();
}
return $this->listOfAPIs;
}
private function updateListOfAPIs(){
$this->listOfAPIs = null; // clears a previous call to this method
$it = new FilesystemIterator(pathToAPIs);
foreach ($it as $fileinfo){
$filename = pathinfo($fileinfo->getFilename(), PATHINFO_FILENAME); // removes extension
$this->listOfAPIs[]= $filename;
}
}
public function __call($method,$args){
}
}
API/exampleExternalAPI.php
<?php
class exampleExternalAPI {
public function echoString($string){
echo $string;
}
}
example.php
<?php
require_once 'api.php';
$api = new api();
$api->setAPIs('exampleExternalAPI');
$api->APIs->exampleExternalAPI->echoString('string');
Background Info
(may give some insight to my madness)
I'm working on a project where I need to connect to lots of external APIs.
So I decided to creating a class to look after all my communications with external APIs ( not sure if best way - new to Object Oriented Programming).
I'm not entirely sure what problem you're trying to solve, but if your APIs is a simple stdClass instance it should work as expected:
public function setAPIs($setAPIs)
{
$this->APIs = new stdClass; // clears a previous call to this method
if (!is_array($setAPIs)) { // if not an array
$setAPIs = array($setAPIs); // make array
}
foreach ($setAPIs as $setAPIType) {
if (in_array($setAPIType, $this->listOfAPIs)) {
$this->APIs->{$setAPIType} = new $setAPIType;
}
}
return $this->APIs;
}
I'm trying to make an error logging class, I have some functions to set up various output methods like DB, file, return and screen. I want all errors to be stored into an array and when __destruct() is called, I want to stop the client from waiting for data and the log details about errors the user experienced. This way they don't have to report errors to me.
I have 2 modes, a simple GUI to test the functionality and the actual script generates responses in JSON, machine to machine. For GUI the final dump is fine but for JSON it destroys the response. So all error reporting is off and I have to handle any errors that would be dumped on screen myself, hence the $return in function flush_log($return) which make the function return a string if set to true.
After reporting flushing the errors I want to:unset($this->log_arrays)
or empty: $this->log_arrays=Array();, but it is out of scope - I understand why, my function uses a local copy - but how do I reset the array?
[EDIT]:
I tried:
$this->log_arrays = Array();
$this->log_arrays = null;
array popping:
for ($i = 1; count($this->log_arrays); $i++)
{
array_pop($this->log_arrays);
}
But I think none of it can work because within a class function you work with copies of variables, so they're basically out of scope.
[/EDIT]:
This is an already simplified class.. :
<?php
class log_strings extends mysqli
{
private $log_arrays = Array();
public function __construct($output_to_file=false, $output_to_db=true, $fall_back_file=true, $arguments, $ip=null)
{
// Setup mysqli connection, file handle or report error if one or all have failed.
// Also check wich outputs should be used and keep that info for later.
}
public function log($level, $string)
{
$log_arrays[] = Array('level' => $level, 'string' => $string);
}
public function __destruct()
{
$this->flush_log();
}
public function flush_log($return=false)
{
if (!isset($log_arrays) && count($log_arrays) == 0)
{
return true;
}
if ($return)
{
return $this->return_output();
}
else
{
$success = false;
// if enabled, output to db
if ($this->output_to_db)
{
$success = $success || $this->mysqli_output();
}
// if enabled or if db failed and fallback is enabled, output to file
if ($this->output_to_file || ($this->fall_back_file && !$success))
{
$success = $success || $this->file_output();
}
// if neither file or db succeeded, dump on screen
if ($success = false)
{
$this->screen_dump();
}
return true;
}
unset($this->log_arrays); // <= This is what it is all about!
}
private function screen_dump()
{
foreach($this->log_arrays as $array)
{
echo "<strong>{$array['level']}</strong>{$array['string']}<br/>\n";
}
}
private function mysqli_output()
{
// Output to db functionally equal to $this->screen_dump()
}
private function file_output()
{
// Output to file functionally equal to $this->screen_dump()
}
private function return_output()
{
// Return output functionally equal to $this->screen_dump()
}
}
?>
Resetting the array should work
$this->log_arrays = array();
Unset a class property is a very bad idea. Because it may be used in other methods or other classes using a potentionally getter of your class.
I am writing a MVC Framework (for the purpose of learning and discovery as opposed to actually intending to use it) and I have came across a slight problem.
I have a config.php file:
$route['default'] = 'home';
$db['host'] = 'localhost';
$db['name'] = 'db-name';
$db['user'] = 'user-name';
$db['pass'] = 'user-pass';
$enc_key = 'enc_key'
I load these via a static method in my boot class:
public static function getConfig($type) {
/**
* static getConfig method gets configuration data from the config file
*
* #param string $type - variable to return from the config file.
* #return string|bool|array - the specified element from the config file, or FALSE on failure
*/
if (require_once \BASE . 'config.php') {
if (isset(${$type})) {
return ${$type};
} else {
throw new \Exception("Variable '{$type}' is undefined in " . \BASE . "config.php");
return FALSE;
}
} else {
throw new \Exception("Can not load config file at: " . \BASE . 'config.php');
return FALSE;
}
}
and then load the route like so:
public function routeURI($uri) {
...
$route = $this::getConfig('route');
...
}
which catches the exception:
"Variable 'route' is undefined in skeleton/config.php"
now, it works fine if I make the config.php file like so
$config['route']['default'] = 'home'
...
and change the two lines in the method like so:
if (isset($config[$type])) {
return $config[$type];
I have also tried using $$type instead of ${$type} with the same problem.
Is there something I am overlooking?
As written, this function can be called only once, because it uses require_once and on subsequent calls you won't bring in the local variables defined in config.php anymore. I suspect you are getting this error on your second call to getConfig().