Laravel: Use Memcache instead of Filesystem - php

Whenever I load a page, I can see Laravel reading a great amount of data from the /storage folder.
Generally speaking, dynamic reading and writing to our filesystem is a bottleneck. We are using Google App Engine and our storage is in Google Cloud Storage, which means that one write or read is equal to a "remote" API request. Google Cloud Storage is fast, but I feel it's slow, when Laravel makes up to 10-20 Cloud Storage calls per request.
Is it possible to store the data in the Memcache instead of in the /storage directory? I believe this will give our systems a lot better performance.
NB. Both Session and Cache uses Memcache, but compiled views and meta is stored on the filesystem.

In order to store compiled views in Memcache you'd need to replace the storage that Blade compiler uses.
First of all, you'll need a new storage class that extends Illuminate\Filesystem\Filesystem. The methods that BladeCompiler uses are listed below - you'll need to make them use Memcache.
exists
lastModified
get
put
A draft of this class is below, you might want to make it more sophisticated:
class MemcacheStorage extends Illuminate\Filesystem\Filesystem {
protected $memcached;
public function __construct() {
$this->memcached = new Memcached();
$this->memcached->addServer(Config::get('view.memcached_host'), Config::get('view.memcached_port');
}
public function exists($key) {
return !empty($this->get($key));
}
public function get($key) {
$value = $this->memcached->get($key);
return $value ? $value['content'] : null;
}
public function put($key, $value) {
return $this->memcached->set($key, ['content' => $value, 'modified' => time()]);
}
public function lastModified($key) {
$value = $this->memcached->get($key);
return $value ? $value['modified'] : null;
}
}
Second thing is adding memcache config in your config/view.php:
'memcached_host' => 'localhost',
'memcached_port' => 11211
Last thing you'll need to do is to overwrite blade.compiler service in one of your service providers, so that it uses your brand new memcached storage:
$app->singleton('blade.compiler', function ($app) {
$cache = $app['config']['view.compiled'];
$storage = $app->make(MemcacheStorage::class);
return new BladeCompiler($storage, $cache);
});
That should do the trick.
Please let me know if you see some typos or error, haven't had a chance to run it.

Related

Variable from config file to other files [duplicate]

I want to create a config file for my PHP project, but I'm not sure what the best way to do this is.
I have 3 ideas so far.
1-Use Variable
$config['hostname'] = "localhost";
$config['dbuser'] = "dbuser";
$config['dbpassword'] = "dbpassword";
$config['dbname'] = "dbname";
$config['sitetitle'] = "sitetitle";
2-Use Const
define('DB_NAME', 'test');
define('DB_USER', 'root');
define('DB_PASSWORD', '');
define('DB_HOST', 'localhost');
define('TITLE', 'sitetitle');
3-Use Database
I will be using the config in classes so I'm not sure which way would be the best or if there is a better way.
One simple but elegant way is to create a config.php file (or whatever you call it) that just returns an array:
<?php
return array(
'host' => 'localhost',
'username' => 'root',
);
And then:
$configs = include('config.php');
Use an INI file is a flexible and powerful solution! PHP has a native function to handle it properly. For example, it is possible to create an INI file like this:
app.ini
[database]
db_name = mydatabase
db_user = myuser
db_password = mypassword
[application]
app_email = mailer#myapp.com
app_url = myapp.com
So the only thing you need to do is call:
$ini = parse_ini_file('app.ini');
Then you can access the definitions easily using the $ini array.
echo $ini['db_name']; // mydatabase
echo $ini['db_user']; // myuser
echo $ini['db_password']; // mypassword
echo $ini['app_email']; // mailer#myapp.com
IMPORTANT: For security reasons the INI file must be in a non public folder
I use a slight evolution of #hugo_leonardo 's solution:
<?php
return (object) array(
'host' => 'localhost',
'username' => 'root',
'pass' => 'password',
'database' => 'db'
);
?>
This allows you to use the object syntax when you include the php : $configs->host instead of $configs['host'].
Also, if your app has configs you need on the client side (like for an Angular app), you can have this config.php file contain all your configs (centralized in one file instead of one for JavaScript and one for PHP). The trick would then be to have another PHP file that would echo only the client side info (to avoid showing info you don't want to show like database connection string). Call it say get_app_info.php :
<?php
$configs = include('config.php');
echo json_encode($configs->app_info);
?>
The above assuming your config.php contains an app_info parameter:
<?php
return (object) array(
'host' => 'localhost',
'username' => 'root',
'pass' => 'password',
'database' => 'db',
'app_info' => array(
'appName'=>"App Name",
'appURL'=> "http://yourURL/#/"
)
);
?>
So your database's info stays on the server side, but your app info is accessible from your JavaScript, with for example a $http.get('get_app_info.php').then(...); type of call.
The options I see with relative merits / weaknesses are:
File based mechanisms
These require that your code look in specific locations to find the ini file. This is a difficult problem to solve and one which always crops up in large PHP applications. However you will likely need to solve the problem in order to find the PHP code which gets incorporated / re-used at runtime.
Common approaches to this are to always use relative directories, or to search from the current directory upwards to find a file exclusively named in the base directory of the application.
Common file formats used for config files are PHP code, ini formatted files, JSON, XML, YAML and serialized PHP
PHP code
This provides a huge amount of flexibility for representing different data structures, and (assuming it is processed via include or require) the parsed code will be available from the opcode cache - giving a performance benefit.
The include_path provides a means for abstracting the potential locations of the file without relying on additional code.
On the other hand, one of the main reasons for separating configuration from code is to separate responsibilities. It provides a route for injecting additional code into the runtime.
If the configuration is created from a tool, it may be possible to validate the data in the tool, but there is no standard function to escape data for embedding into PHP code as exists for HTML, URLs, MySQL statements, shell commands....
Serialized data
This is relatively efficient for small amounts of configuration (up to around 200 items) and allows for use of any PHP data structure. It requires very little code to create/parse the data file (so you can instead expend your efforts on ensuring that the file is only written with appropriate authorization).
Escaping of content written to the file is handled automatically.
Since you can serialize objects, it does create an opportunity for invoking code simply by reading the configuration file (the __wakeup magic method).
Structured file
Storing it as a INI file as suggested by Marcel or JSON or XML also provides a simple api to map the file into a PHP data structure (and with the exception of XML, to escape the data and create the file) while eliminating the code invocation vulnerability using serialized PHP data.
It will have similar performance characteristics to the serialized data.
Database storage
This is best considered where you have a huge amount of configuration but are selective in what is needed for the current task - I was surprised to find that at around 150 data items, it was quicker to retrieve the data from a local MySQL instance than to unserialize a datafile.
OTOH its not a good place to store the credentials you use to connect to your database!
The execution environment
You can set values in the execution environment PHP is running in.
This removes any requirement for the PHP code to look in a specific place for the config. OTOH it does not scale well to large amounts of data and is difficult to change universally at runtime.
On the client
One place I've not mentioned for storing configuration data is at the client. Again the network overhead means that this does not scale well to large amounts of configuration. And since the end user has control over the data it must be stored in a format where any tampering is detectable (i.e. with a cryptographic signature) and should not contain any information which is compromised by its disclosure (i.e. reversibly encrypted).
Conversely, this has a lot of benefits for storing sensitive information which is owned by the end user - if you are not storing this on the server, it cannot be stolen from there.
Network Directories
Another interesting place to store configuration information is in DNS / LDAP. This will work for a small number of small pieces of information - but you don't need to stick to 1st normal form - consider, for example SPF.
The infrastucture supports caching, replication and distribution. Hence it works well for very large infrastructures.
Version Control systems
Configuration, like code should be managed and version controlled - hence getting the configuration directly from your VC system is a viable solution. But often this comes with a significant performance overhead hence caching may be advisable.
Well - it would be sort of difficult to store your database configuration data in a database - don't ya think?
But really, this is a pretty heavily opinionated question because any style works really and it's all a matter of preference. Personally, I'd go for a configuration variable rather than constants - generally because I don't like things in the global space unless necessary. None of the functions in my codebase should be able to easily access my database password (except my database connection logic) - so I'd use it there and then likely destroy it.
Edit: to answer your comment - none of the parsing mechanisms would be the fastest (ini, json, etc) - but they're also not the parts of your application that you'd really need to focus on optimizing since the speed difference would be negligible on such small files.
You can create a config class witch static properties
class Config
{
static $dbHost = 'localhost';
static $dbUsername = 'user';
static $dbPassword = 'pass';
}
then you can simple use it:
Config::$dbHost
Sometimes in my projects I use a design pattern SINGLETON to access configuration data. It's very comfortable in use.
Why?
For example you have 2 data source in your project. And you can choose witch of them is enabled.
mysql
json
Somewhere in config file you choose:
$dataSource = 'mysql' // or 'json'
When you change source whole app shoud switch to new data source, work fine and dont need change in code.
Example:
Config:
class Config
{
// ....
static $dataSource = 'mysql';
/ .....
}
Singleton class:
class AppConfig
{
private static $instance;
private $dataSource;
private function __construct()
{
$this->init();
}
private function init()
{
switch (Config::$dataSource)
{
case 'mysql':
$this->dataSource = new StorageMysql();
break;
case 'json':
$this->dataSource = new StorageJson();
break;
default:
$this->dataSource = new StorageMysql();
}
}
public static function getInstance()
{
if (empty(self::$instance)) {
self::$instance = new self();
}
return self::$instance;
}
public function getDataSource()
{
return $this->dataSource;
}
}
... and somewhere in your code (eg. in some service class):
$container->getItemsLoader(AppConfig::getInstance()->getDataSource()) // getItemsLoader need Object of specific data source class by dependency injection
We can obtain an AppConfig object from any place in the system and always get the same copy (thanks to static). The init () method of the class is called
In the constructor, which guarantees only one execution. Init() body checks
The value of the config $dataSource, and create new object of specific data source class. Now our script can get object and operate on it, not knowing
even which specific implementation actually exists.
Define will make the constant available everywhere in your class without needing to use global, while the variable requires global in the class, I would use DEFINE. but again, if the db params should change during program execution you might want to stick with variable.
If you think you'll be using more than 1 db for any reason, go with the variable because you'll be able to change one parameter to switch to an entirely different db. I.e. for testing , autobackup, etc.
Here is my way.
<?php
define('DEBUG',0);
define('PRODUCTION',1);
#development_mode : DEBUG / PRODUCTION
$development_mode = PRODUCTION;
#Website root path for links
$app_path = 'http://192.168.0.234/dealer/';
#User interface files path
$ui_path = 'ui/';
#Image gallery path
$gallery_path = 'ui/gallery/';
$mysqlserver = "localhost";
$mysqluser = "root";
$mysqlpass = "";
$mysqldb = "dealer_plus";
?>
Any doubts please comment
One of the simplest form to use config with multiple files is like this:
Files hierarchy:
config
- mail.php
- database.php
mail.php
return [
'smtp_debug' => 0,
];
A helper function:
function config($configFilename, $key)
{
$path = sprintf("config/%s.php", $configFilename);
if (file_exists($path)) {
$config = include sprintf("config/%s.php", $configFilename);
if (isset($config[$key])) {
return $config[$key];
}
}
return '';
}
And you can call it in elegant way:
config('mail','smtp_debug')
I normally end up creating a single conn.php file that has my database connections.
Then i include that file in all files that require database queries.
What about something like this ?
class Configuration
{
private $config;
public function __construct($configIniFilePath)
{
$this->config = parse_ini_file($configIniFilePath, true);
}
/**
* Gets the value for the specified setting name.
*
* #param string $name the setting name
* #param string $section optional, the name of the section containing the
* setting
* #return string|null the value of the setting, or null if it doesn't exist
*/
public function getConfiguration($name, $section = null)
{
$configValue = null;
if ($section === null) {
if (array_key_exists($name, $this->config)) {
$configValue = $this->config[$name];
}
} else {
if (array_key_exists($section, $this->config)) {
$sectionSettings = $this->config[$section];
if (array_key_exists($name, $sectionSettings)) {
$configValue = $sectionSettings[$name];
}
}
}
return $configValue;
}
}
if i have a config file like config.conf (it can be htttp://example.com/config.conf)
user=cacom
version = 2021608
status= true
this is my function:
function readFileConfig($UrlOrFilePath){
$lines = file($UrlOrFilePath);
$config = array();
foreach ($lines as $l) {
preg_match("/^(?P<key>.*)=(\s+)?(?P<value>.*)/", $l, $matches);
if (isset($matches['key'])) {
$config[trim($matches['key'])] = trim($matches['value']);
}
}
return $config;
}
we can use:
$urlRemote = 'http://example.com/default-config.conf';
$localConfigFile = "/home/domain/public_html/config.conf";
$localConfigFile2 = "config.conf";
print_r(readFileConfig($localConfigFile2));
print_r(readFileConfig($localConfigFile));
print_r(readFileConfig($urlRemote));
You can use this simple one:
define('someprop', 0);
and
echo someprop; // output 0
Here it is
<?php
$server = "localhost";
$username = "root";
$password = "";
$db = "your_db_name";
$conn = mysqli_connect($server, $username, $password, $db);
if(!$conn){
die('Error in connecting to server or Database');
}
?>

XCache lost value after some time

I'm a newbie with XCache and I'm trying to use this feature for have an editable configuration over the air in my application.
So I need to store some data, for doing this I did:
class Settings
{
private $_config = array();
function __construct()
{
$file = 'config.php'; //return $config content
require_once $file;
$this->_config = $config;
foreach($config as $item => $value)
{
if(!xcache_isset($item))
{
xcache_set($item, $value);
}
}
}
}
Unfortunately today the official site seems down, so I can't follow the documentation to check if I did something wrong.
I've created also two method:
public static function setItem($name, $value)
{
xcache_set($name, $value);
}
public static function getItem($name)
{
return xcache_get($name);
}
now getItem after 15/20 minute can't get the key value. Why?
UPDATE
Okay, the problem it's when an header('Location..) is called. Infact if I do a redirection I lost the value stored in cache, anyone know why?
As the name implies, XCache is a cache, not a database. Values that you store in the cache may be purged without warning if space is needed for other data, and will be lost entirely when the web server is restarted. It's not an appropriate place to store configuration information.
I can't say for certain why you're seeing values become unavailable after a redirect, though. That shouldn't happen.

Symfony2. Force a service to get instanced

I'm working with symfony 2.8, I'm facing a situation where a service must be instanced even if it is not requested somewhere.
Why ? Because this Core service configure tagged services that are transfered by method calling into Core ( I did this in a compiler pass ).
And then, if I request one of these tagged services without request Core, it will not be configured and then be unusable.
Here is the compiler pass :
$coreDefinition = $container->findDefinition(
'app.improvements.core'
);
$taggedAppliers = $container->findTaggedServiceIds('app.improvement.applier');
foreach ($taggedAppliers as $id => $tags) {
$coreDefinition->addMethodCall(
'registerApplier',
[new Reference($id)]
);
}
$taggedImprovements = $container->findTaggedServiceIds(
'app.improvement'
);
// Only method found to initialize improvements.
foreach ($taggedImprovements as $id => $tags) {
$coreDefinition->addMethodCall(
'registerImprovement',
[new Reference($id)]
);
}
To sum up, the Appliers registers Improvement and Core registers Appliers. The core associate improvements with appliers because each improvement must be registered in a specific applier that the core stores.
The problem I that when I only request a Applier, its improvements are not registered into it because the core isn't instancied.
Thank you.
Design problems aside, the easiest way to instantiate a service is to use an event listener. In your case you would listen for the kernel.request and pull your service from the container.
class RequestListener
{
public function onKernelRequest(GetResponseEvent $event)
{
$event->getKernel()->getContainer()->get('service_id');
}
}

Proper way to inject dynamic configuration into configuration array

I'm wondering what is the best way to inject dynamic configuration(retrieved from db for instance) into configuration array in Zend Framework 2? In Module.php I have:
public function onBootstrap(MvcEvent $e) {
$eventManager = $e->getApplication()->getEventManager();
$moduleRouteListener = new ModuleRouteListener();
$moduleRouteListener->attach($eventManager);
$eventManager->attach('route', array($this, 'mergeDynamicConfig'));
}
public function mergeDynamicConfig(EventInterface $e) {
$application = $e->getApplication();
$sm = $application->getServiceManager();
$configurationTable = $sm->get('DynamicConfiguration\Model\Table\ConfigurationTable');
$dynamicConfig = $configurationTable->fetchAllConfig();
//Configuration array from db
//Array
//(
// [config] => 'Test1',
// [config2] => 'Test2',
// [config3] => 'Test3',
//)
//What to do here?
//I want to use the configurations above like $sm->get('Config')['dynamic_config']['config3'];
}
There is a section in the documentation that explains how to manipulate the merged configuration using the specific event ModuleEvent::EVENT_MERGE_CONFIG
Zend\ModuleManager\Listener\ConfigListener triggers a special event, Zend\ModuleManager\ModuleEvent::EVENT_MERGE_CONFIG, after merging all configuration, but prior to it being passed to the ServiceManager. By listening to this event, you can inspect the merged configuration and manipulate it.
The problem with this is that the service manager is not available at this point as the listener's event is one of the first events triggered by the module manager at priority 1000).
This means that you cannot execute your query and merge the config prior to the configuration being passed to the service manager, you would need to do so after.
Perhaps I have misunderstood your requirements, however I would approach this differently.
You could replace any calls where you need config $serviceManager->get('config') with $serviceManager->get('MyApplicationConfig'); which would be you own configuration service that uses the merged application config and then adds to it.
For example, you could register this configuration service in module.config.php.
return [
'service_manager' => [
'factories' => [
'MyApplicationConfig' => 'MyApplicationConfig\Factory\MyApplicationConfigFactory',
]
],
];
And create a factory to do the loading of merged module configuration, making any database calls or caching etc.
class MyApplicationConfigFactory implements FactoryInterface
{
public function createService(ServiceLocatorInterface $sm)
{
$config = $sm->get('config');
$dbConfig = $this->getDatabaseConfigAsArray($sm);
return array_replace_recursive($config, $dbConfig);
}
protected function getDatabaseConfigAsArray(ServiceLocatorInterface $sm)
{}
}
You also have the added benefit that the service is lazy loaded.
I would not use this approuch, for a few reasons.
Putting SQL queries in your Module.php means that they will get executed on EVERY request for every user thus making your application slow, very slow.
If your database is compromised all the config will be stolen as well.
Solution would be to move all the config in your config/autoload/my_custom_config.local.php via array with keys. From there you can always load it without making a single database request. It will be way faster and secure, because the file will be outside your root folder and hacking a server is always alot harder than hacking a database.
If you still want to allow users to eit the options you can simply include the file in an action and show it with a foreach for example. To save the information you can do this:
file_put_contents("my_custom_config.local.php", '<?php return ' . var_export($config, true).';');
One other plus is that if you load your config the way discribe above you can also retrive the config like you want via $sm->get('Config')['dynamic_config']['config3']

Static variables across sessions

In ASP.NET if I declare a variable (or object) static (or if I make a singleton) I can have it persist across multiple sessions of multiple users (it it registered in a server scope) so that I don't have to initialize it at every request.
Is there such a feature in PHP? Thanks
You can set up APC and use the apc_store and apc_fetch functions.
http://us.php.net/manual/en/book.apc.php
You can do that with a PHP extension (written in C).
But if you want to write it in PHP, no. The best alternative is to write the variable to a file (file_put_contents()) at the end of each request, and open it at the start of each request (file_get_contents()).
That alternative isn't going to work for high volume sites because the processes will be doing read/write at the same time and the world will go all BLAAA-WOOO-EEE-WOHHH-BOOOM.
That doesn´t exist in PHP, however, you can serialize the data and put it either in a file on your hard drive or in /dev/shm/. You can also use memcache.
If you put your data in /dev/shm/ or use memcache the data will disappear on reboot.
Sadly, no. PHP's static keyword is limited to the current script instance only.
To persist data across script instances for the same session, you would use the session handling features.
To persist data across sessions, you would have to use something like memcache, however that requires additional set-up work on server side.
Symfony and other frameworks uses "PHPFastCache" who supports a wide range of drivers for caching data including APC, SQLite, MongoDB or simply your file system.
You can donwnload it at https://github.com/PHPSocialNetwork/phpfastcache
Here is an example with file caching :
use Phpfastcache\Helper\Psr16Adapter;
$defaultDriver = 'Files';
$Psr16Adapter = new Psr16Adapter($defaultDriver);
// Setter action
if(!$Psr16Adapter->has('test-key')) {
$data = 'lorem ipsum';
$Psr16Adapter->set('test-key', 'lorem ipsum', 300); // kept in cache for 300 seconds (5 minutes)
}
// Getter action
else {
$data = $Psr16Adapter->get('test-key');
}
You can use the Session Storage for this purpose, if you use the same sessionId for all sessions.
session_id('xyz');
session_start();
for ($i=0; $i < 100000; $i++) {
$_SESSION['counter'] = isset($_SESSION['counter']) ? $_SESSION['counter'] + 1 : 0;
}
echo "<br>session_id(): ".session_id() . "<br>counter: ".$_SESSION["counter"];
Try this script with 2 browsers and you will see that this method shares the data across both browsers - and is very, very fast.
you could store serialized copies of an object inside session
class test{
private static $instance;
public property;
private __construct(){}
public getInstace(){
if(!self::$instance){
self::$instance = new test;
}
return self::$instance;
}
}
$p = test->getInstance();
$p->property = "Howdy";
$_SESSION["p"] = $p;
next page
$p = $_SESSION["p"];
echo $p->property; // "Howdy"

Categories