I'm building a service, which has a few cronjobs running, written in Python. However, this is my first Python-project ever, so I'm still a very beginner.
What I'm doing now, is that I have my database-connection handled on every file, so basically if I wanted to change the host, I would need to go through all the files. I'm now looking into a PHP-include() similar method for Python, so that I could include some general stuff instead of copy-pasting.
Also, the Python-files are ran in cronjob, so the method should work on cronjobs too :)
If it's really just a couple of settings for a single database connection, just put it in a Python module and import it in all of your files. Why add any complexity you don't need?
If it's more complicated, use ConfigParser as #AdamMatan suggested.
# dbconfig.py
host = '127.0.0.1'
user = 'stack'
password = 'overflow'
# db.py
import dbconfig
print dbconfig.host
print dbconfig.user
print dbconfig.password
Use an external configuration file, with your db connection (host, name, password, db, ...) in it, and read the configuration file from within the Python script.
This makes changes easy (even for non-programmers) and nicely complies with the Single Choice Principle.
Example:
db.cfg
[db]
host=127.0.0.1
user=stack
password=overflow
db.py
import ConfigParser
config = ConfigParser.ConfigParser()
config.readfp(open('db.cfg'))
print config.get('db', 'host')
Execution result:
127.0.0.1
If you need to call __import__(), you are doing it wrong.
You need to refactor your code so that you no longer have the database connection routines scattered throughout your codebase. Yes, it would be even nicer to have these details in a configuration file (+1 #Adam Matan), but first you need to eliminate the duplication. This will save you a world of pain in the long run.
Related
I have a Python script that interfaces with an API. The script is started from a PHP page. I wrote both scripts, so I can change the code in either as appropriate.
The Python script needs a username and password to interface with the API. My first inclination is to pass them to Python as CLI arguments:
<?php
exec('python someScript.py AzureDiamond hunter2');
?>
However, anybody can then see the credentials via ps:
$ ps | grep someScript
1000 23295 2.0 0.2 116852 9252 pts/0 S+ 15:47 0:00 python someScript.py AzureDiamond hunter2
Alternatives that I am considering are to write the data to a text file or sqlite database, then to delete them. Are there any better ideas? A constraint with the sqlite approach is that this needs to run in a rather portable fashion (phpFox Plugin) and most budget webhosts don't support the sqlite3 module.
You could use environment variables which you set in PHP and read in the Python script.
The proposed methods are a security problem waiting to happen. MyProxy is a secure way to handle private keys or certificates. MyProxy might not be the particular solution you need, but these types of solutions are what you need to be looking at.
http://grid.ncsa.illinois.edu/myproxy/
You could use stdin in the python process to read the parameters, and open the process like this from php:
<?php
$handle = popen("/bin/python /var/www/someScript.py", "w");
// write username and password to $handle (e.g. newline separated)
?>
I don't know how secure this is against other processes running as the same user (could it by hijacked from /proc/nn/fd/0 ?), but it certainly makes it harder than just looking at the process list.
Looking at the problem from another angle, it could be solved if you hide the plain characters being passed to exec(). Have you considered using any cryptograhic lib to hide plain text from the sender and decrypt in the receiver? Just a thought.
I'm having a multiplayer server that's using PHPSockets, and thus is written entirely in PHP.
Currently, whenever I'm making any changes to the PHP server-script I have to kill the script and then start it over again. This means that any users online is disconnected (normally not a problem because there aren't so many at the moment).
Now I am rewriting the server-script to use custom PHP classes and sorten things up a little bit (you don't want to know how nasty it looks today). Today I was thinking: "Shouldn't it be possible to make changes to the php source without having to restart the whole script?".
For example, I'm planning on having a main.php file that is including user.php which contains the class MyUser and game.php which contains the class MyGame. Now let's say that I would like to make a change to user.php and "reload" the server so that the changes to user.php goes into effect, without disconnecting any online users?
I tried to find other questions that answered this, the closest I got is this question: Modifying a running script and having it reload without killing it (php) , which however doesn't seem to solve the disconnection of online users.
UPDATE
My own solutions to this were:
At special occations, include the file external.php, which can access a few variables and use them however it'd like. When doing this, I had to make sure that there were no errors in the code as the whole server would crash if I tried accessing a method that did not exist.
Rewrite the whole thing to Java, which gave me the possibility of adding a plugin system using dynamic class reloading. Works like a charm. Bye bye PHP.
Shouldn't it be possible to make changes to the php source without having to restart the whole script?
[...]
I'm planning on having a main.php file that is including user.php
which contains the class MyUser
In your case, you can't. Classes can only be defined once within a running script. You would need to restart the script to have those classes redefined.
I am not too familiar with PHP but I would assume that a process is created to run the script, in doing so it copies the instructions needed to run the program and begins execution on the CPU, during this, if you were to "update" the instructions, you'd need to kill the process ultimate and restart it. Includes are a fancy way of linking your classes and files together but ultimately the processor will have that information separate from where the file of them are stored and it is ultimately different until you restart the process.
I do not know of any system in which you can create code and actively edit it and see the changes while that code is being run. Most active programs require restart to reload new source code.
Runkit will allow you to add, remove, and redefine methods (among other things) at runtime. While you cannot change the defined properties of a class or its existing instances, it would allow you to change the behavior of those objects.
I don't recommend this as a permanent solution, but it might be useful during development. Eventually you'll want to store the game state to files, a database, Memcache, etc.
How about storing your User object into APC cache while your main script loads from the cache and checks every so often for new opcode.
To include a function in the cache, you must include the SuperClosure Class. An example would be:
if (!apc_exists('area')) {
// simple closure
// calculates area given length and width
$area = new SuperClosure(
function($length, $width) {
return $length * $width;
}
);
apc_store('area', $area);
echo 'Added closure to cache.';
} else {
$func = apc_fetch('area');
echo 'Retrieved closure from cache. ';
echo 'The area of a 6x5 polygon is: ' . $func(6,5);
}
See here for a tutorial on APC.
Simple solution use $MyUser instead of MyUser
require MyUserV1.php;
$MyUser = 'MyUserV1';
$oldUser = new $MyUser('your name');
//Some time after
require MyUserV2.php;
$MyUser = 'MyUserV2';
$newUser = new $MyUser('your name');
Every declared class stay in memory but become unused when the last MyUserV1 logout
you can make them inherit from an abstract class MyUser for using is_a
You cannot include again a file with the same class, but you can do so with an array. You can also convert from array to class, if you really need to do so. This only applies to data, though, not to behavior (methods).
I don't know much about these things with the games on PC but you can try to get all the variables from your database for the user and then update the text fields or buttons using those variables
In web is using AJAX (change data without refreshing the page).Isn't one for programming?
I am writing a PHP application targeted at non-geeks, non-programmers. I need to create an option page with a bunch of "options" and then store those options...somewhere. Using a database application (MySQL/PostgreSQL/SQLite) is out of the question because it will require more configuration than the user needs to do (I don't want the user to do any kind of configuration if he doesn't want to). So the only solution left is to write the configuration to a configuration file. On the other hand, I also want that configuration file to be human-readable in case the user is a geek and he wants to edit the config file directly (or if he wants to edit the file remotely via SSH or any kind of reason...)
Here are the couple of potential solutions I found:
Using a JSON file...
...Retrieve the data from the file, using json_decode to convert the data, output it into HTML, retrieve any changes, encode back using json_encode, etc. You get the picture. There are a couple things that I don't like about this method, the main one being that the encoded JSON data using PHP will no be well formatted and very hard to edit without being reformatted beforehand.
Using an XML file
I won't describe that solution because I don't really like it either...and I don't know how to use XSLT and don't really want to learn...and because it's a pretty heavyweight solution, at least compared to the JSON solution. Correct me if I'm wrong.
Using an INI file
I love INI files, really I love them! I think they're really the most readable, and it's hard to mess up (ie: syntax errors). The problem with that solution is that there is no native way to write/edit an ini file. I found a topic showing a custom method to write one...that might be the solution I will adopt if I don't find anything better...
Using two files
That last solution seems as reasonable as the INI solution. In fact, I could use an INI file as "input" (the file that the user would edit if he wants to) and an XML/JSON file as output (the file that will be edited by PHP every time the user changes options using the web front-end). At this point, the best solution would be to ask the user to reload the configuration manually if he edited the config file directly, so that the "output" file is always up to date.
I know none of the solutions above are perfect, and that's why I created this topic to ask for advice. What is the best solution? Maybe (probably) I missed yet another solution.
One last thing: YAML isn't a valid solution because it's a lot easier to mess up the syntax if you're not used to it. PHP is not a solution either because editing PHP with PHP is a pain. PHP is only a good solution if I want to retrieve some configuration but not edit it directly via a web front-end.
ini
I'd write ini files, myself. As you said, the syntax is very simple, and that's what you want in a config file. The ini format's "key+value" pairing is exactly what you'd get with a database—without the database.
Related SO you may have seen already: create ini file, write values in PHP
Plus you can use parse_ini_file() to read it.
XML
XML isn't all that bad. It may be more work to write it (and may not be as clear to the user as an ini file), but reading it is really easy.
Create it:
<?php
// Create file
$xml = new SimpleXMLElement( '<?xml version="1.0" ?><config></config>' );
// Add stuff to it
$xml->addChild( 'option1' );
$xml->option1->addAttribute( 'first_name', 'billy' );
$xml->option1->addAttribute( 'middle_name', 'bob' );
$xml->option1->addAttribute( 'last_name', 'thornton' );
$xml->addChild( 'option2' );
$xml->option2->addAttribute( 'fav_dessert', 'cookies' );
// Save
$xml->asXML( 'config.xml' );
?>
Read it:
<?php
// Load
$config = new SimpleXMLElement( file_get_contents( 'config.xml' ) );
// Grab parts of option1
foreach( $config->option1->attributes() as $var )
{
echo $var.' ';
}
// Grab option2
echo 'likes '.$config->option2['fav_dessert'];
?>
Which gives you:
billy bob thornton likes cookies
Documentation for SimpleXML
SimpleXML Docs index
Basic Examples
Details on addChild() and addAttribute(), showing how to generate various XML structures (nested tags vs. attributes, for example)
I'd go with the ini. They're really not that hard to write. I personally hate XML. It's so bloated... even if the file size doesn't matter, it still makes me cringe at it's wordiness and the amount of typing I have to do. Plus, people are dumb. They won't close their tags.
The standard way would be XML files. They don't create that much overhead and are easily extensible. However, JSON files are the easiest on the programming end.
I'd rank my preference:
XML
JSON
ini (last resort)
Unless you have 1000+ options, you really shouldn't worry about the XML file size. The goal here is to keep things easy for the user. This means that whichever method you choose (JSON shouldn't be one of them in my opinion), it should be heavily documented at each config line.
Your two file solution brings me back to the days of sendmail config and makes me shudder.
I would just go with XML, it's self documenting to a point <Email>hi#hi.hi</Email>
Well, you could use PHP's serialize(), and although it is human readable, it isn't the most human readable thing there is. It's on the same level as JSON to implement.
I need to be able to store data for a php application in a file. I need to be able to do this without any sort of external dependencies other than PHP itself.
Here are the requirements I have:
Settings will not be updated/added/removed very often. So updating of settings does not have to be very efficient. However, I do want to be able to do this all through a PHP script, not through editing of files.
Settings will be read constantly, so reading of settings must be very efficient.
Settings are in a unique format, if I had them in an array it might be something like $Settings["Database"]["AccessSettings"]["Username"]["myDBUsername"]; $Settings["Database"]["AccessSettings"]["Password"]["myDBPassword"];
I would prefer to not have settings stored in arrays like I mentioned above. Instead I would prefer some access methods: getConfig("Database","Accesssettings","Username") would return 'myDBUsername'. The reason for this is I want to limit the variables I am storing in the global scope.
What would the best way of getting/retrieving these be?
Do the the hierarchy I was thinking possibly an xml file, but I wasn't sure what PHP was like for accessing xml files (particularly the fact that I need to be able to add, edit, and remove). If it should be XML what sort of xml access should I look into.
If it is another format, I would like some pointers to the right direction for what to look into for how to use that format.
Brian, parse_ini_file is what you need.
Ah, I missed the requirement that you'd be editing this via PHP.
In that case there is nothing native to PHP that you can use for this purpose. You'd have to roll your own.
You could save yourself a ton of time by simply using Zend_Config_Ini though. I know you state that you don't want to use anything else, but Zend Framework is structured to allow you to use whatever pieces of it you need. Zend_Config can be used on it's own. You can certainly just add these few classes to your project and let them handle your INI file parsing.
Here is an example using your samples above:
[config]
Database.AccessSettings.Username = myDBUsername
Database.AccessSettings.Password = myDBPassword
You would load and access this as simply as:
$config = new Zend_Config_Ini('/path/to/ini', 'config');
echo $config->Datbase->AccessSettings->Username; // prints "myDBUsername"
echo $config->Datbase->AccessSettings->Password; // prints "myDBPassword"
To edit and save your config you would use the following:
$config->Database->AccessSettings->Password = "foobar";
$writer = new Zend_Config_Writer_Ini(array('config' => $config,
'filename' => 'config.ini'));
$writer->write();
Edit
Not really sure why people are voting this down based on vog's misguided comments. It is very simple to make this writable by multiple persons by using an exclusive lock. Zend_Config_Writer uses file_put_contents to do it's writing, which has always supported the the LOCK_EX flag, which exclusively locks a file for writing. When using this flag, you cannot have multiple writers attempting to update the file at the same time.
To use this flag with Zend_Config_Writer it's as simple as follows:
$writer = new Zend_Config_Writer_Ini(array('config' => $config,
'filename' => 'config.ini'));
$writer->setExclusiveLock(true);
$writer->write();
An alternate syntax:
$writer = new Zend_Config_Writer_Ini();
$writer->write('config.ini', $config, true); // 3rd parameter is $exclusiveLock
If you're not hand editing, use serialized data.
Writing config using serialize:
file_put_contents('myConfig.txt', serialize($Settings));
Reading config using unserialize:
$Settings = unserialize(file_get_contents('myConfig.txt'));
You could write a class for modifying and getting values for this data using PHP magic functions __get() and __set().
If you are willing to drop in a library, you could use Spyc and have your configuration in a YAML file.
You could also use PHP's support for SQLite which would mean your database is just a file so no need for a DB server.
DBM files may not be a bad idea. PHP has built in support for the various DBM-file varieties:
http://www.php.net/manual/en/book.dba.php
I have a database where I keep my configuration parameters, I want to load the configuration parameters into my application variables only once (or upon specific request to reload the parameters), I also want these variables which holds the configuration parameters to be accessible from all php pages/scripts, the idea is to save hits on the database and improve the application response time.
What is the 'classic' php solution to this matter?
It seems to me that this is essentially the same as any other caching question. The fact that the content to be cached is configuration parameters rather than say the content of Web pages or user profile information is unimportant from a technical perspective.
So what you have to do is come up with some caching solution, whether it's memcached or just writing static files with the data you want to cache.
The trick here is that you're not caching HTML to be presented to the user but rather database query results, so you'll probably want to look at approaches like this one:
http://devzone.zend.com/article/1258
I like using the Zend_Config_Ini class. Creating separate sections that can extend others is easy, and with Zend_Cache with Zend_Cache_Frontend_File (to check for updates to the .ini file) and a backend (I use APC) that is particularly fast to access to avoid any overhead of re-parsing.
; Production site configuration data
[production]
webhost = www.example.com
database.adapter = pdo_mysql
database.params.host = db.example.com
database.params.username = dbuser
database.params.password = secret
database.params.dbname = dbname
; Staging site configuration data inherits from production and
; overrides values as necessary
[staging : production]
; 'database.adapter' is inherited
; others are overridden
database.params.host = dev.example.com
database.params.username = devuser
database.params.password = devsecret
Make a page that sets constants (key word 'define') early in your routines. Just include it wherever needed.
Like Smandoli's answer, I use a single file that has my configuration.
However, my configuration is actually a multi-dimensional array - meaning I have much greater control over my config - I can change it on the fly if I need to, as well as breaking up the varialbes.
$config['error']['nologin'] = "You're not logged in";
$config['db']['host'] = "localhost";
$config['something']['else'] = "hello world";
Edit: I use a file for values that do not change too much. I do use variables from a database occasionally, but not too often.
My rule of thumb is "If the user doesn't need to change it, load from the file; if they need to change it, then it comes from a database".
I came from the world of C++ the paradigm there was to use a singleton which load the parameters on first (and only) instantiation and export an interface with relevant get'ters (like 'int GetVal(char* key,int &val)' ) the singletone was accessible from all parts of the application, is there anything like that in PHP?