Writing to Apache access_log file with php - php

I need to write statistical data to the live Apache access_log file (I have another process counting specific lines in the access_log file that reports periodically back to another process).
Currently I am simply forcing an entry into the access_log file by doing the following in php:
file("http://127.0.0.1/logme.php?stuff_that_I_can_watch_here");
logme.php does nothing and returns empty with a 200 success.
The problem with the above technique is that for every request to the Apache server, another is spawned to write to the log - hence doubling required apache servers.
When the servers pile up, the simple and usually fast local call to the Apache server takes over 5 seconds.
Can I write to the access_log file directly without causing problems, or maybe even is there a way to write to the apache_log file using php similar to syslog() or error_log()?

You can use apache_note (http://php.net/apache_note) to write your values to a note and then use CustomLog with LogFormat (%{NOTE_NAME}n) (http://httpd.apache.org/docs/2.2/mod/mod_log_config.html) to log the new keys. Your programs which parse the access log can then read the new logging parameters as well.

is it ok to write to access_log directly
You can write directly to access_log, but is NOT OK to do this.
A running apache can spawn multiple processes,
and lock file for write using slower process like PHP is just further delay logging speed.
or is there an easy way to achieve the same effect using php
Do not use PHP, add an additional custom log if a request full-fill your requirement.
This is more proper way and this custom log should contains lesser line, like static file access is not logged. Which directly improve parsing of the log later.

<?php
$h = fopen('/path/to/access_log', 'a');
fwrite($h, 'Message');
fclose($h);
?>
Others have already commented about the design. The Apache access_log is for Apache to log accesses, period.
I uses about a dozen custom log files in one app for all the different monitoring, performance analysis and forensic purposes. One log file per purpose makes log analysis easier.

Related

Mysterious 503 Error with PHP/MySQL File/Database Sync

I am hoping you all can help me out a little. I have spent about 7 hours trying to find an answer and have tried many things so far.
I have a PHP script that is used to sync files/database data between 2 servers. Before you guys ask this process is necessary for this project and must stay in place.
The script basically finds all files in a directory that have changed in the last 72 hours and SFTPs them to the other server, replacing any files needed. It then creates a copy of the backing database, removes certain tables/rows, changes others and exports a .sql file. It then SFTPs this .sql file to the other server and calls an include on a file on the 2nd server that imports the .sql file replacing the existing database with updated data.
All of this works...
The issue is that no matter what changes I make to the Apache config the script always gives me a 503 error after 30 seconds, every time (between 30.02 and 30.04 seconds to be precise). However, the PHP script continues to run and successfully completes all operations, including writing to the log file, in about 60-61 seconds. There is nothing in the Apache logs referencing any kind of error at all either.
I have checked all .conf files used and none of them mention a 30 second timeout. In my httpd.conf I have added these lines:
TimeOut 300
ProxyTimeOut 300
KeepAlive On
KeepAliveTimeout 60
I also have set the max_execution_time and memory_limit on the php script to 120 and 2048M, respective, to be sure to rule that out during testing.
The page is supposed to display a success message to the user with a report of what was changed/updated. However with the 503 error I am not able to do this. So I am looking to get rid of this 503 limitation so it can properly display the end result of the sync. I am not too familiar with Apache configuration to be honest so any help/ideas on what would cause this/where to look would be much appreciated!
Thanks in advance!
After trying many, many things I was able to find out what the specific cause was. It turns out this was caused by the proxy timing out. Here is a link to the answer that explained what to add to the vhost conf file.
In short, here is the answer for future visitors:
For the latest versions of httpd and mod_proxy_fcgi you can simply add
timeout= to the end of the ProxyPassMatch line, e.g.:
ProxyPassMatch ^/(.+\.php.*)$ fcgi://127.0.0.1:9000/<docroot>/$1 timeout=1800
For older versions it was a little more complicated,
e.g.:
<Proxy fcgi://127.0.0.1:9000>
ProxySet timeout=1800
</Proxy>
ProxyPassMatch ^/(.+\.php.*)$ fcgi://127.0.0.1:9000/<docroot>/$1
Not sure if you tried this, but I think you may need to adjust max_execution_time in the php.ini that Apache uses. On many distributions it defaults to 30.
http://php.net/manual/en/info.configuration.php#ini.max-execution-time

Clear error_log file with PHP

Okay, so my error log files are getting filled up really fast due to some unfixable errors (they exist due to the site's design) and it's using up a lot of my webspace pretty fast.
However, I do not need the error_log files at all. I've been looking around and I cannot find a way to stop error_logs in cPanel, so I thought that perhaps I could run a script on each page a user visits that empties or deletes the error_log file.
Is this possible? the error_log file is in the document root of each of my websites
You can disable error logging using corresponding php.ini setting:
http://php.net/manual/en/errorfunc.configuration.php#ini.log-errors

How can I set up error_log by folder?

My company has a large hosting, but it’s not managed by us, we don't see configuration files, but I want to reply this feature on our local test server.
I’m new in my company and want to start some debug of applications to fix some minors and majors issues to clients, but the amount of files is so big that single error_file is huge.. and there are many people working on this so each time I check log (like 30 secs to 1 min) has hundreds of added lines.
I don’t know if this is set up on Apache, through .htaccess files or in php.ini.
I am talking about PHP errors, but I don't know if this is set in PHP, Apache, or maybe using a third-party library.
I'm not talking about setting a specific folder error_log. I'm talking about if errors are logged in the scripts folder.
Example: I create a folder named test1. Inside it I make some buggy PHP script that throws some errors. When I run the script I can see an error_log file created in the folder. So it works on the fly.
I have tried to ask the hosting company support how they do this, but they haven’t answered me.
I don't know if maybe could be some kind of cPanel setting (BTW, the hosting support stuff doesn't understand this question either, but well.. usually level 1 of support can’t handle technical stuff).
I found it.
You have to set a directive in the php.ini file as follows, string "error_log". On the right side is the file name you want for the log,
error_log = error_log
This will generate a PHP error log in the folder where script executed are,
I'll try to explain.
Script test.php in folder /www/site/lib:
include "./db_conn.php";
If file db_conn.php is not located in the same directory, this will fire a warning and error. Usually this will be lead to the servers/vhost log, but using this directive you will get an error_log file under the /www/site/lib directory.
Why was I looking or this? Well, as I wrote, I'm working on a huge application, with thousands of files many fires warnings, notices, etc. I'm not the only one in the project and the site error_log file was so huge it was hard to keep tracking debug evolution for one or just some files. Now I just have to track the log from directories where I'm working.
You can manage the logs by adding this to your vhost or htaccess file
ErrorLog /path/to/a/writable/directory/error.log
For more information, have a look at this article on advanced PHP error handling via htaccess.
To do this in PHP, edit file php.ini and add the line
error_log = /path/to/where/you/want/your/php-errors.log
Then restart the web server. This should give you PHP errors, and only PHP errors, to that file. Provided, of course, that the scripts aren't written to throw away all errors.
To do it in Apache, you can use SetEnvIf to add an environment variable to any request ending in PHP, and then printing all logs with that particular environment variable. E.g.:
SetEnvIf Request_URI "\.php$" phplog
CustomLog /path/to/php.log env=phplog
To use this in multiple folders, make one environment variable per folder and one CustomLog per folder, like this:
SetEnvIf Request_URI "/folder1/.*\.php$" log_folder1
Customlog /path/to/folder1/php.log env=log_folder1
SetEnvIf Request_URI "/folder2/.*\.php$" log_folder2
Customlog /path/to/folder2/php.log env=log_folder2

Show full list of php scripts on server

How i show full list of running php sripts on linux server? I see only httpd service, or PID but not specific php file source, i need analyze what script take more memory and fixed it. Thanks
You have two options:
Log all URLs that are requested from your server and that end up in PHP scripts being executed
You can use a PHP feature, which allows you to add a PHP script to any Apache request that is sent to PHP. You enable it by adding this to your root .htaccess:
php_value auto_prepend_file append.php
In append.php you add a logging feature, where you can insert the URL requested, the time it took to generate the response and the max memory used. If you add this to a TAB separated file, you could import it in a DB table and see what is really happening on your server.
More info here: http://www.electrictoolbox.com/php-automatically-append-prepend/
Debug what Apache is doing, using strace
You basically start Apache with strace. This will debug what operations Apache and subsequently PHP are doing. Watch out, as there is a lot of noise in the debug output.
More info here: http://bobcares.com/blog/?p=103

PHP source code security on server

I am a PHP newbie and a have a php security question. Is it possible for somebody to get the source code of a php script file running on a server with default configuration? If so, what is the best way to be protected? I am asking this because I happened to download a php file when I requested a page from a site and what triggered my concerns. I think that maybe apache configuration was wrong and served that file to me like a simple file but I am not sure. Also what is the best place to keep "sensitive" data such as database or smtp configuration?
Thank you,
Alex
For the most sensitive information, I'd suggest putting it outside of your web root folder, and including it through "require" or "include". This way, even is some configuration gets botched on the server, the visitor will only get served the line "include('secret_stuff.php');" and not the actual script.
Exactly what David Dorward said but i would advise you take a look at the following patch(s) that would modify apache to not send source code's regards if there is a misconfiguration.
http://mirror.facebook.net/facebook/patches/ap_source_defense.patch
Patch like so:
cd apache-1.3.x
patch -p1 -i ap_source_defense.patch
More Patches from Facebook Development Team: http://mirror.facebook.net/facebook/patches/
The best way to protect your much needed source is to place them outside the public root directory, as if apache is running it will not be able to serve files directly from the folder up public_html
for example:
C:/server/apache/
C:/server/apache/htdocs/
C:/server/apache/htdocs/includes/
People can specifically view the files my going to
http://hostname.tld/includes/
but having the directory structure of:
C:/server/apache/
C:/server/apache/includes/
C:/server/apache/htdocs/
and then within
C:/server/apache/htdocs/index.php
you have
<?php
require_once('../includes/config.php');
?>
this should protect all major files bar the view file (index.php)
If the server is not configured to handle PHP files, then it will treat them like any other unknown file (and serve them as either text/plain or application/octet-stream.
PHP support is, as far as I know, always provided as an extension or external program (for CGI, FastCGI, etc) and never as a built in for an HTTP server.
If the server is properly configured to run PHP code, then people without direct access to the server cannot view the PHP source code. You don't have to do anything else.
It is only because that server was not configured to run PHP, and instead served it as text, that you could see the source.
If you have this line in your apache.httpd.conf file,
AddType application/x-httpd-php .php
Apache should deal with data, rather than showing them...
Also you need to start php services.
What you describe as "default configuration" is a webserver without php installed (or with php disabled). In these cases, it is certainly possible to download the php script.
Make sure php is installed (as it will be on ~100% of production php servers) and/or block access to your configuration file with an .htaccess file like this:
<FilesMatch "^config.php$">
Order allow,deny
Deny from all
</Files>
If you want to be extra-tricky (and work even on servers where .htaccess files are ignored), prefix the configuration file with .ht, like .ht.config.php. Most Apache(and some other webserver) configurations will refuse serving files beginning with .ht. However, in general, the only way you could be sure no webserver serves your file is to move it to a directory outside of the server's document directory. On most hosts you or your php script won't be able to access those though.
Your second problem are misconfigurations. There's not much you can do, albeit there might(?) be options to construct a rewriterule to prevent accidential accessibility.
The best prevention however is to keep all scripts outside of the DOCUMENT_ROOT. Just leave a single index.php there, and include all dependencies from there. This is also the best strategy to avoid leaking of configuration data (also don't use ini files for sensitive data, but always .php scripts).
Another worry are shared hosting servers however. All concurrent users on a server can read out your scripts (if not through PHP then via Perl/bash CGIs). Nothing you can do about that, unless you change to a professional hoster which supports running everthing through suexec and thus allowing individual permissions.
Well, "default configuration" is a vague term, but as long as the web server is configured to parse .php files through PHP, you should be fine from that angle. If your scripts themselves happen to parse other PHP files (for eg. a template system) then you need to make sure there aren't any loopholes in that too, since the PHP scripts have full access to your files on the server.
Assuming these are taken care of, you don't need to keep the "sensitive" data in any special place -- just put them in your .php files, but make sure all your scripts end in .php (for eg. config.inc.php) so that they are always parsed via PHP and not sent as plain text.

Categories