I want to make a news portal(php) with minimum mysql force.
:create a cron, fetch data from mysql and write to a php file . (I dont know is it right way)
But Can I use xml instead of php file? Write mysql data to xml.
Is this a secure way? What is the best way? XML or php file?
alt text http://img541.imageshack.us/img541/4784/87392425.gif
Thanks in advance
Let me start of by saying that MySQL is very fast and very secure. I recommend building the pages with MySQL upon request, most web applications do this because its a very good approach. To save resources you can cache the output using a Squid Reverse Proxy, and it is very common to see this on high traffic sites. PHP's APC will also reduce resource consumption without sacrificing secuirty. Smarty's Caching system is also a good approach with minimal security impact.
There are secuirty concerns regardless of what method you choose, but some approaches more hazardous than others. For instance creating .php files with user input is probably the most dangerous thing you can attempt to do with a php web app.
$page='$title="'.$_GET['user_title'].'"';
file_put_contents("/var/www/page.php",$page);
An attack against this code would look like this:
http://localhost/page_creater.php?user_title='; eval($_GET[backdoor]); /*
Creating XML files with user input is also dangerous because it opens the door for Advanced LFI Attacks. However, the counter argument is that as long as your application is free from Local File Include vulnerabilities, then you shouldn't have to worry. But this is not a "Defense in-depth" design, because you should plan on failure.
Its possible to implement something like Squid's reverse proxy in php using ob_start(), however your still creating files with attacker controlled data, and that is hazardous. Also don't include() .html files that's incredibly stupid (See advanced LFI attacks...), a better approach is this: print(file_get_contents($file)). I do like using .html files over using .xml files because html doesn't have to be processed before outputting it to the user. Using xml files as a data storage is wasteful of resources when compared to MySQL.
(Disclaimer: Vulnerabilities have been found in Smarty and squid, and php, and linux, and mysql and apache and.... everything else, even StackOverflow ;)
Related
I am looking for best practices, modules, etc. to securely do file system manipulation via PHP application. The CMS-like application will not use a database, but instead the markdown files are placed in folders and are processed at display time. Therefore, there will be a lot of moving files around, renaming files, writing to files, etc.
I am looking to either find some libraries (e.g., equivalent to an ORM) that will help to manage such actions, input sanitization, moving files. etc. rather than start from scratch. If nothing like this is available, I would like a listing of best practices, etc.
So far I have only found guidance from PHP.net.
More information: The plan is to build a web based end-user interface which sits ontop of Stacey. I would have a test environment with the end user interface, and when changes are ready they are then synced to the production environment. This is a non-DB based system. Stacey is convenient to manage and work with from a developer standpoint, but user's don't want to work directly with markdown and move files, etc.
Also: Please limit the answer to PHP issues; server things like chrooting or locking down the server would be dependent upon the user's individual environment and needs. From a development standpoint, I want to focus on securing my distributed code.
I don't know of any specific libraries that do this -- the filesystem support in PHP is extensive so I'm not sure why they'd be necessary. You might be better off starting with an existing CMS and modifying it to do what you want -- however I understand that might not be possible. It also sounds like the sort of thing that should be using a database, but I guess you already know that.
I can't claim to know exact best practice, this is more general advice.
First, your web server -- and therefore your PHP scritps -- will be running as a certain user. This depends on your configuration and particular server as well as the underlying OS. Ideally you want to make sure this user only has access to the filesystem area that your using as storage. Deny all access to everywhere else apart from read-access to where it really needs (your scripts, etc) and read-write to the storage area. The exact way to do this depends on your system.
That's your last line of defense, do not rely on it, it's there as a safety net.
It's not clear exactly what will cause files to be renamed, moved, altered but it's a safe bet that it's from user input. Therefore you need to make sure you sanitize all user input, if their page name becomes a file name you want do not want to let some enter ../../index.php as a page name and nuke your main site.
Always assume the worst case: a user who knows the internals of your system intimately and is aiming to do most damage. Do not rely on 'security by obscurity' or 'nobody will ever do that'.
What I would do (and have done before) is two fold. First wrap all the filesystem functions up into a class that provides the same functions as methods. The job of this class is to check that anything happening is allowed, that means it's probably going to have to read the paths and filenames and work out the location of the changes.
Secondly, sanitize all user input that could be malicious when it first arrives. You might want to look at using escapeshellarg or URL encoding, or something else depending on what your input is.
You also mention files are processed at run-time, if users are allowed to write scripts (or worse PHP that gets executed) then you have a lot more issues and may have a fundamental problem. But that's not clear from your question.
Hope that helps.
I'm thinking of protecting my script to the mass majority of users (non-web dev savvy) and I came across an online service to encode php script. I'm not sure about it though.
Is it safe to encrypt php script? What if the encoded code has something fishy in it?
If you intend to distribute the PHP file then I would suggest that you do not do this. It's only going to irritate those that want to tinker with it.
If for some reason you don't want them tinkering with it, then don't distribute the PHP file.
If you need to distribute the file AND you don't want them tinkering with it, then I would highly suggest you not do this in PHP and instead write the functionality using C as an extension to PHP.
You'll notice that at no point do I suggest you actually go ahead and "encode" the php file. That's not going to buy you anything.
If you are looking to obfuscate your server-side PHP, the best bet would be to use a commercial product such as Zend Guard (http://www.zend.com/en/products/guard/). Any home-brew encryption is not secure in the slightest - your code can be easily reverse-engineered with fairly trivial effort. The page you link to does not have any credibility, it is just someone's side project. They have no accountability or stake in protecting your information.
Even these commercial products (Zend Guard, ionCube, phpShield, SourceGuardian) can be decrypted if someone really, really wanted to. No tool or technique in any language can make absolutely secure obfuscation, there is no "unhackable" system. Everything boils down to effort over time.
If it isn't important enough to bother doing it right, then you're probably wasting your time on the issue. Further, if it is absolutely vital that some information or code remain private, you should simply not put it out into the public purview.
[edited for clarity]
Ultimately, you need to trust the encrypting party. If you don't trust them (apparently you don't), then don't give them access to your server (through executing their decryption code/your obfuscated code, possibly with who-knows-what else inside). Simple as that, albeit possibly inconvenient.
php is usually running on the server where the users have no access to the code(neither source nor any other representation) anyways. No reason to obfuscate it there.
Obfuscating php is only useful in the rare cases where you give the php code to clients. For example if you want clients to be able to run their own server but not give them full access to the code.
So, it looks like all it does is obfuscate the code so it's not human-readable. The only way this would really be useful is to prevent lazy people who have access to the code from reading it. However, it uses simple functions to encode/decode, so it would be trivially easy for someone to decode it if they have access.
Which brings me to my point... PHP security works by not allowing anyone to have access to the source file. If someone who shouldn't have access gets it, then this "encoding" thing isn't going to do you any good.
The OP mentioned an interest in protecting database connection details, and it should be kept in mind that no matter what protection system is used for the code itself, the PHP engine and component libraries being opensource sets some absolute limits on what can be achieved. If MySQL connection details, for example, are hidden in a script then these details could be trivially revealed without going near the PHP scripts themselves simply by running the scripts with a PHP build that had slight modifications to the MySQL library or the associated PHP module wrapper. Even hiding the details in a C module as suggested by Chris L. would afford no extra protection in this case. Good protection can certainly be given to source code with compiled code systems such as ionCube and Zend, but wherever data hits routines in the PHP core then it can be exposed.
Obviously for any online service where you may be sending sensitive details, you should use due diligence and make best efforts to ensure that it has a good pedigree. Apart from anything else, not having a working https URL for the site the OP questioned should immediately warn that it's a no-no, and not just for the lack of connection encryption but showing that they are not offering a service that they consider to be serious.
what are the available cache methods i could use in php ?
Cache HTML output
Cache some variables
it would be great to implement more than one caching method , so i need them all , all the available out there (i do caching currently with files , any other ideas ?)
Most PHP build don't have a caching mechanism built in. There are extensions though that can take care of caching for you.
Have a look at APC or MemCache
If you are using a framework, then most come with some form of caching mechanism that you can use e.g. Zend Framework's Zend_Cache.
If you are not using a framework then the APC or Memcache as Pelle ten Cate mentioned can be used. The correct approach to use does depend in your situation though, do you have your website or application running on more than server and does the information in the cache need to be shared between those servers? (if yes then something like memcache is your answer, or maybe a database or distributed NoSQL solution if you are feeling brave).
If you code is only running on the one server you could try something simple like serializing your variables, and writing them to disk, then on every request afterwards, see if the files exists, if it does, open it and unserialize the string into the variable you need.
This though is only worth it if it would take a long time to generate the varaible normally,
(e.g longer than it would to open,read,unserialize the file on disk)
For HTML caching you are generally going to get the most mileage from using a proxy like Varnish or Squid to do it for you but i realise that this may not be an option for you.
If its not then you could the write to disk approach i mentioned above, and save chunks of HTML to files. look in the PHP manual for ob_start and its friends.
Since every PHP run starts from scratch on page request, there is nothing that would persist between calls, making cacheing moot.
Well, that's the basic view. Of course there are ways to implement a caching, sort of - and a few packages and extensions do so (like Zend Extensions and APC). However, you should have a very close look whether it actually improves performance. Other methods like memcache (for DB results), or switching from PHP to e.g. Java will often yield better results.
You can store variables in the $_SESSION, but you shouldn't keep larger HTML there.
Please check what you are actually trying to do. "Bytecode cacheing" (that is, saving PHP parsing time) needs to be done by the PHP runtime executable. For cacheing Database (SQL) request/reply-pairs, there is memcache. Cacheing HTML output can be done, but is often not a good idea.
See also an earlier answer on a similar question.
I'm working on a webapp that uses a lot of ajax to display data and I'm wondering if I could get any advice on what else I could do to speed up the app, and reduce bandwidth, etc.
I'm using php, mysql, freeBSD, Apache, Tomcat for my environment. I own the server and have full access to all config files, etc.
I have gzip deflate compression turned on in the apache http.conf file. I have obfuscated and minified all the .js and .css files.
My webapp works in this general manner. After login the user lands on the index.php page. All links on the index page are ajax calls to read a .php class function that will retrieve the html in a string and display it inside a div somewhere on the main index.php page.
Most of the functions returning the html are returning strings like:
<table>
<tr>
<td>Data here</td>
</tr>
</table>
I don't return the full "<html><head>" stuff, because it already exists in the main index.php page.
However, the html strings returned are formatted with tabs, spaces, comments, etc. for easy reading of the code. Should I take the time to minify these pages and remove the tabs, comments, spaces? Or is it negligible to minify the .php pages because its on the server?
I guess I'm trying to figure out if the way I've structured the webapp is going to cause bandwidth issues and if I can reduce the .php class file size could I improve some performance by reducing them. Most of the .php classes are 40-50KB with the largest being 99KB.
For speed, I have thought about using memcache, but don't really know if adding it after the fact is worth it and I don't quite know how to implement it. I don't know if there is any caching turned on on the server...I guess I have left that up to the browser...I'm not very well versed in the caching arena.
Right now the site doesn't appear slow, but I'm the only user...I'm just wondering if its worth the extra effort.
Any advice, or articles would be appreciated.
Thanks in advance.
My recommendation would be to NOT send the HTML over the AJAX calls. Instead, send just the underlying data ("Data here" part) through JSON, then process that data through a client-side function that would decorate it with the right HTML, then injecting it into the DOM. This will drastically speed up the Ajax calls.
Memcache provides an API that allows you to cache data. What you additionally need (and in my opinion more important is) is a strategy about what to cache and when to invalidate the cache. This cannot be determined by looking at the source code, it comes from how your site is used.
However, an opcode cache (e.g. APC) could be used right away.
Code beautifier is for human not for machine.
As part of the optimization you should take off.
Or simply add a flag checking in your application, certain condition match (like debug mode), it return nicely formatted javascript. Otherwise, whitespace does not mean anything to machine.
APC
You should always use APC to compile & cache php script into op-code.
Why?
changes are hardly make after deployment
if every script is op-code ready, your server does not required to compile plain-text script into binary op-code on the fly
compile once and use many
What are the benefits?
lesser execution cycle to compile plain-text script
lesser memory consume (both related)
a simple math, if a request served in 2 seconds in your current environment, now with APC is served in 0.5 seconds, you gain 4 times better performance, 2 seconds with APC can served 4 requests. That's mean previously you can fit 50 concurrent users, now you can allow 200 concurrent users
Memcache - NO GO?
depends, if you are in single host environment, probably not gain any better. The biggest advantages of memcache is for information sharing & distribution (which mean multiple server environment, cache once and use many).
etc?
static files with expiration header (prime cache concept, no request is fastest, and save bandwidth)
cache your expensive request into memcache/disk-cache or even database (expensive request such as report/statistics generation)
always review your code for best optimization (but do not over-do)
always do benchmark and compare the results (was and current)
fine-tune your apache/tomcat configuration
consider to re-compile PHP with minimum library/extension and load the necessary libraries during run-time only (such as application using mysqli, not using PDO, no reason to keep it)
I'm working on a PHP content management system and, in testing, have noticed that quite a few of the system's MySQL tables are queried on almost every page but are very rarely written to. What I'm wondering is will this start to weigh heavily on the database as site traffic increases, and how can I solve/prevent this?
My initial thoughts were to start storing some of the more static data in files (using PHP serialization) but does this actually reduce server load? What I'm worried about is that I'd be simply transferring the high load from the database to the file system!
If somebody could clue me in on the better approach, that would be great. In case the volume of data itself has a large effect, I've detailed some of the data I'll be storing below:
Full list of Countries (including ISO country codes)
Site options (skin, admin email, support URLs etc.)
Usergroups (including permissions)
You have to remember that reading a table from a database on a powerful server and on a fast connection is likely to be faster than reading it from disk on your local machine. The database will cache the entirety of these small, regularly accessed tables in memory.
By implementing the same functionality yourself in the file system, there is only a small possible speed up, but a huge chance to mess it up and make it slower.
It's probably best to stick with using the database.
Optimize your queries (using mysql slow query log) and EXPLAIN function.
If tables are really rarely written to you can use native MySQL caching. You have nothing to change in you code, just enable mysql caching in my.conf.
Try out using template engine like Smarty (smarty.net). It has it's own caching system that works pretty well and will REALLY reduce server load.
You can also use Memcache, but it is really worth using only with really high load websites. (I think that Smarty will be enough.)
Databases are much better at handling large data volumes than the native file system.
Don't worry about optimizing your site to reduce server load, until you actually have a server load problem. :-)
The tables you mentioned (countries and users) will normally be cached in memory by MySQL directly unless you are expecting quite a few millions of records in these tables.
In case where these tables will not fit in memory, you may want to consider a general-purpose distributed memory caching system, such as memcached.
If your database is properly indexed, it will be much faster to query data from the database. If you want to speed that up, look into memcached or similar.
Databases are exactly for this purpose.. To store and provide data. Filesystem is for scripts and programming.
If you encounter load problems, consider using Memcached or another utility for database.
You may also consider trying to cache different parts of your page directly into database as whole sections (eg. a sidebar, that doesn't change too much, generated header section, ..)
you could cache output (flush(), ob_flush() etc.) to a file and include that instead of having multiple MySQL reads. caching is definitely faster than accessing MySQL multiple time.
reading a static file is much faster than adding overhead via php and mysql processing.
You need to evaluate the performance via load testing to avoid prematurely optimising.
It would be foolish and quite possibly increase overall load to store data in files with serialization, databases are really good at retrieving data.
If after analysis there is a true performance hit (which I doubt unless you are talking about massive loading), then caching is a better solution.
It's more important to have a well designed system that facilitates changes as needs arise.
Here's a link to a couple script that will essentially do what dusoft is talking about and cache the output buffer to a file:
http://www.addedbytes.com/articles/caching-output-in-php/
Used this way, it's more of a bolt-on-after-the-fact type of solution, but this same behavior can certainly be implemented in a more integrated fashion if considered earlier in the process. Many frameworks also have this kind of thing built in.