I am working on a rather large website and i need to log errors that users may face while using the website.
Here is how it will work:
>if operation passed
#operation success
>else
#Log the failure
log()
>email admin
>create log
What i need to know is the best practice for creating this log, because there are several methods for doing this.
text based
database
There is possibly a better method for doing this as well, which is why i'm asking stack overflow.
Just tell me how you would go about doing this, and i will do the rest of the research and coding on my own.
I find using a 3rd party service like airbrake.io or pagerduty.com is best. Basically, they handle creating a ticket and logging everything as well as notifying the proper people about the incident. Yes, you can write up your own system the way you mention via emailing an admin and creating your own logs... but then you will also have to worry about updating the email list and emailing the right people at the right time... What if you're on vacation? Who is to get the email at that point? 3rd party services manage all that for you.
You can use (and probably should use) open source logging frameworks for the language you are working in. They will provide you with nice wrappers for all your logging needs, most have the option to email logs to you (and even upload files to remote directories).
If you wish to create your own logging system, this is how I would personally do it:
Make a log directory
Create a log file (plain text) each hour (or day or X units of time) using a naming scheme
Write 1 line to the file with the time, then some delimiter, then the error (including error codes/messages etc)
Every time an hour or day passes, you would make a new file and email the previous file to yourself (or admin). You can also send an immediate email for fatal errors/issues. I wouldn't really use a database personally.
I implemented such a logging system for a online script that talks to a gaming server. The end result is a directory of files filled with logs for each hour of each day. Files older than 30 days are also deleted. It allows me to check on how things are going easily and pinpoint certain events/issues that players on the game server experience. However, I only wrote my own logger as there was no script that did this for my game.
First of all, since it was mentioned in the comments, we should differntiate the php error log from a custom application log:
The php error log logs errors of a certain level (notices, errors, warnings depending on your error_reporting() settings) while interpreting your php files. That means when you are trying to use an array key which was not set before a warning would be generated and either printed to the screen or logged to your php error log file.
A custom application logger on the other side logs custom messages which might contain warnings and errors regarding the application logic and which are able to be handled by the application.
When we compare the following two code examples:
<?php
error_reporting(E_ALL|E_STRICT);
ini_set('display_errors', 0);
ini_set('log_errors', 1);
ini_set('error_log', sys_get_temp_dir() . '/php_error.log');
updateUser($_POST['user_id']);
// Assuming $_POST['user_id'] was not set the above lines would produce a notice in your php_error.log stating the use of an undefined index 'user_id'
?>
Against:
// Instantiate your own logger or a 3rd party logger
$myLogger = new Logger(sys_get_temp_dir() . '/application.log');
if (!array_key_exists('user_id', $_POST)) {
$myLogger->error('Cannot update user since user_id was not set');
// Handle the error in the UI accordingly
header('Location: 404.php');
die();
}
updateUser($_POST['user_id']);
?>
For me personally it makes sense to separate these two types of errors in different log files: The php errors are usually a result of code which does not handle all imaginable cases (i.e. a user removes the hidden user_id field from a form manually) and are a hint for yourself that you should change your code to avoid the same error next time.
The second piece of code handles the exactly same use case but you considered this case while writing the code and the application is able to react somehow.
No matter if you decide pick a 3rd party logger or write your own: Think about using one which fulfils the PRS-3 logging standard to be able to make it exchangable when you i.e. decide to switch from file based logging to a database based logging mechanism. By doing so you won't have to change a lot of code when you decide to switch your loggers since the methods and general usage is standardised.
When writing your own logger, consider the following points:
Locking and unlocking your log file while writing to it
Log rotation (daily, weekly, monthly)
Deletion of old log files
Like stated above think about implementing PSR-3
Related
This is a fairly open question as i am not sure what the best practice could/should be for this type of activity.
The scenario; A user based website (with logged in user areas and user editable content) needs to have several operations performed that will manipulate live data. To stop any errors occurring due to users simultaneously editing as the update is taking place, i need to shut out the users temporarily.
I don't currently have a config database table, just a config file. Two things need to happen;
An update to the config happens to stop the users accessing certain areas of the website during the update - this is then reverted once the update is complete.
The website (or at least portions on the site) need to be checked before every request for said portions of the site - so whatever the method, it needs to be lightweight
This is site built on Laravel.
Thanks!
There is a simple artisan command for that. Use this to shut down the application:
php artisan down
And this to turn it back on:
php artisan up
I'll use serveral measures in this case.
put the code in try - catch block so that if something goes wrong, you can grcefully show an error message.
If there are mulitple executions invovled, then better use DB::transaction() and the system will roll back automatically if it encounters an error.
If you really want to put certain features on maintence mode, then put those routes in Route::group() and use a middleware for that group only. In that middleware, return response with 503 status. This way, you can put certain routes into maintanace mode while the rest will give 200 ok response.
I currently check every GET and POST variable with isset() and throw exceptions when isset() returns false.
Example 1:
if(!isset($_GET['some_var']))
throw new Exception('GET variable [some_var] is not set.');
$someVar = $_GET['some_var'];
Example 2:
if(!isset($_GET['some_num']))
throw new Exception('GET variable [some_num] is not set.');
if(!ctype_digit($_GET['some_num']))
throw new Exception('GET variable [some_num] is not a number.');
$someNum = $_GET['some_num'];
In my production application I have a global exception handler that posts exceptions and errors to a log file and then redirects to a generic apology page.
Is this an okay practice? Are descriptive exception and error messages such as the ones above security risks (is it possible that a hacker would be able to read the exception notice and then use that information to manipulate my scripts)?
Thanks!
Logging errors and suppressing output is exactly what you should be doing. Error reporting can be nasty..
In OWASP top 10 for 2007 there is Information Leakage and Improper Error Handling, however this was removed in 2010. By setting dispaly_errors=On in your php.ini you become vulnerable to CWE-200. The full path of your web application will be divulged to the attacker. To make matters worse, by having error reporting enabled it makes it easier to find SQL injection by looking for sql error messages.
When combining this on a PHP/MySQL application you can perform a very serious attack
$vuln_query="select name from user where id=".$_GET[id];
If
http://localhost/vuln_query.php?id=1 union select "<?php eval($_GET[e])?>" into outfile "/path/to/web/root/backdoor.php"
Which makes this full query:
select name from user where id=1 union select "<?php eval($_GET[e])?>" into outfile "/path/to/web/root/backdoor.php"
I would make sure display_errors=Off and that file FILE privileges have been revoked to your web application's MySQL user account.
Displaying detailed errors to a user can be a security risk. Since in this case, they're only being written to a log file and the only data the user gets is a generic page which reveals nothing, you can be as descriptive as you like and you reveal nothing unless the log is compromised.
"is it possible that a hacker would be able to read the exception notice and then use that information to manipulate my scripts?"
Maybe.
Typically, you want to give the least amount of information possible to the end user in an error condition. In this case, if you tell someone a particular get variable doesn't exist, then they might try supplying random values to that variable to see how the app behaves.
Of course, you also have to balance this against the needs of your real users. If the variable is one that they would normally have control over, then giving the response about a problem with the value is perfectly acceptable.
UPDATE
Having recently run into a spate of web API's that seem to think throwing generic error messages is the way to go I want to update this slightly.
It is critical that web API's give an appropriate amount of information back to the consuming system so that they can figure out what's wrong and fix it.
In one recent case for a payment processing API their documentation was simply wrong. The test transaction data that they showed consistently returned with "Server Error 500" and we had no recourse but to get one of their developers on the phone and painstakingly step through each and every element in their XML. Out of 50 elements, only one had the same name as what was in their "developer documents"
In another integration we were given "Server Error 402". -- This one was NOT a payment gateway. Although never referenced in their doc's, apparently that message meant that a JSON parameter was missing. Incidentally, it was a parameter not referenced in their docs and again required time with their developer to identify it.
In both of the above cases it would have been incredibly helpful if the error message had responded with an example of a valid document post. Similar to how the old Unix/DOS commands would come back with the help info when you passed bad parameters. I really don't want to talk to other programmers. I know their time is expensive and they would much rather do something other than answer a support call; but more to the point, if I'm working at 10:00PM and need an answer RFN then waiting until a programmer can get on the phone the next day is rarely an option.
Usually it is considered insecure to print out PHP system error messages on a production server instead of silently logging it.
Though I can't find anything dangerous in the generic apologies page.
I am using CakePHP in PHP development. I have set my debug mode to 0 in core.php file.
Configure::write('debug', 1);
This setting will not show any error on site. So the user/developer will not be able to see errors. Thant's why I want to make something that will send me an email with error title and error code like Warning message, notice(8): like error messages. So that if error occurs, it wouldn't be ignored.
Thanks.
If you get an email every time an error occurs, you will be flooded until the error is fixed which is probably not very efficient or productive.
You could write an error emailing system with throttle control, where as soon as each error is raised from CakePHP it is placed in a database (perhaps keyed on md5(errortext)) and emailed immediately to whoever is interested. Then, next time the exact same error is encountered, the system will see that it's already in the database (same md5) and not email it again.
Also, can't CakePHP be configured to log the errors to a log file? Then you can check that for errors, either manually or via something like logcheck, which will run in a frequent schedule, check the CakePHP logfile for specific errors, and email out a summary if any new ones are found.
I think you could achieve this goal by overriding PHP's default error handler. The relevant PHP manual page: http://php.net/manual/en/function.set-error-handler.php
Basically, you'd just define a function (and tell your script to call that function on an error). Your definition needs to appropriately return false or die() on an error (otherwise the script will continue to execute). However, in that function you would be able to make a call to send emails.
Note that if you're doing it within CakePHP you may need to pass the current object as a parameter, otherwise it's likely that error handler you define won't tie in nicely with the other cake object stuff.
I'm just about to release an open source project that does this, and more. It collects errors, sends them to an issue tracker, detects duplicates, turns them into issues and emails staff.
Details are at https://sourceforge.net/news/?group_id=317819&id=293422 and the version 0.1.7 it mentions is due out in a couple of days.
The open source tracker is at http://elastik.sourceforge.net/
Any feedback welcome, Thanks
I was wondering what the excepted standard is for handling errors in the Model.
Currently I have 'setError' and 'getError' methods that's in use by all my Models.
This means I'm only concerned with whether a call to a method in my Model is true or false. If it's false then I would use $this->model->getError() in my Controller.
Additionally I'm contemplating setting up a separate file that contains all my errors. One file per model, also wanted to have thoughts on this.
A simpler solution would be to use exceptions.
When an error occurs that would be something you display to a user, throw a special kind of an exception - perhaps named UserError. The exception should contain the text of the error message when you throw it. These kinds of errors are features which provide users with useful information (i.e. they attempted to delete something that did not exist - which can happen when they have multiple browsers open, etc.)
e.g.:
throw new UserError("That object no longer exists.");
When an error occurs that you want to hide from the user, throw a different kind of exception, perhaps named InternalError. You would want to log this and allow the program to continue, so the specific error is hidden from the user. If it prevents something from happening, you might want to throw up a generic error message. These would be bugs and you want to fix them as soon as possible.
e.g.:
throw new InternalError("Failed to connect to remote service");
All of the error messages can be stored (hard-coded) in the source where the exception is thrown. This is not necessarily a bad design practice - if you use a tool like gettext, you can easily translate all of these messages.
I've been using log4j and log4cxx and logging to a syslogd. Kiwi is a simple Win32 syslogger that will track your log messages and save them to a file. Log4j / Log4cxx have configuration files that you can use to setup all your log levels or log message destinations (you can log to multiple places).
It takes so little effort to setup and use, and it works like a charm.
I haven't tried out log4php myself.
Exceptions are good when you no longer want your program to continue executing. Catch exceptions at a high level where you can accept the fall-out of failed executions.
Review the NerdDinner tutorial on how to create validation routines before making a final decision:
http://nerddinnerbook.s3.amazonaws.com/Part3.htm
The Validation part is about 2/3 of the way down the page.
What is the best way to record errors experienced by the user?
My initial thought was to make a function that recorded the error with a unique number and maybe a dump of the variables into a record on the database.
Is there a better approach? Should I use a text file log instead?
How about overriding the default PHP errorhandler?
This site should give some basic information: http://www.php.net/manual/en/function.set-error-handler.php and the first comment on http://www.php.net/manual/en/function.set-exception-handler.php
You might also want to store database errors, perhaps some kind of custom function that allows you to use code like:
<?php
$objQueryResult = mysql_query("query here") or some_kind_of_function_here();
?>
You might want to store the recorded errors in a file, which is outside your public html root folder, to make sure people can't access it by accident.
I would also assume, you'd want to store a complete stacktrace in such a file, because then you can actually debug the problem.
When overriding the default errorhandlers, please note you don't forget to send a nice message to the user (and exit the script, when needed).
I would recommend storing:
$_POST
$_GET
A complete dump of
debug_print_backtrace()
Possibly the SQL that triggered this?
I would suggest you to use debug_print_backtrace() to make sure you get a summary of data. The debug_backtrace() function gives about the same information, but it can sometimes just give you too much information.
The code you could use to catch backtraces:
<?php
ob_start();
debug_print_backtrace();
$trace = ob_get_contents();
ob_end_clean();
?>
To store this, you could use a plain text output, if you don't get too much errors, otherwise perhaps use something like sqlite? - Just don't use the same SQL connection to store the errors, as that might trigger more problems, if you're having webserver to SQL connection errors.
Well, at least writing to text files on the local system should be less error prone, thus allowing you to catch DB errors too :)
I would prefer to write a decent dump of the current state to a simple log file. In addition to your "own" state (i.e. your application's variables and objects), you might consider doing a phpinfo() to get inspiration as to which environment and request variables to include.
PEAR::Log is handy for this kind of logging. e.g.
$logger->alert("your message");
$logger->warning("your message");
$logger->notice("your message");
etc.
You can log to a file or to a database, I wrote a PDO enabled sqlite extension , pretty simple.
These are handy to put into exception handling code too.
PEAR::Log
Records: id, logtime, identity, severity 1-7( ie "Alert"), and your message.
I think #Icheb's answer covers it all.
I have tried something new this year in a project that I thought I'd share.
For a PHP based content aggregation / distribution service, an application that runs quietly in the background on some server and you tend to forget, we needed an error reporting system that makes sure we notice errors.
Every error that occurs has an Error ID that is specified in the code:
$success = mysql_query(this_and_that);
if (!$success) log_error ("Failed Query: ".mysql_error(), "MYSQL_123");
Errors get logged in a file, but more importantly sent out by mail to the administrator, together with a full backtrace and variable dump.
To avoid flooding with mails - the service has tens of thousands of users on a good day - error mails get sent out only once every x hours for each error code. When an error of the same code occurs twice within that timespan, no additional mail will be sent. It means that every kind of error gets recorded, but you don't get killed by error messages when it's something that happens to hundreds or thousands of users.
This is fairly easy to implement; the art is getting the error IDs right. You can, for example, give every failed mySQL query in your system the same generic "MYSQL" Error ID. In most cases, that will be too generic and block too much. If you give each mySQL query a unique error ID, you might get flowed with mails and the filtering effect is gone. But wWhen grouped intelligently, this can be a very good setup.
From the usability point of view, the user should not Ever experience errors.
Depending on the error you should make different strategies:
non catchable errors or difficult to catch from PHP, read the logs for each application
Apache
MySQL and DB errors, transactions
prepare php with "site being updated" or error controllers for emergencies.
PHP errors
these should be detected through Exceptions
silenced but not forgotten, don't try to fix them on the fly
log them and treat them
interface errors
an advice: allow user to submit suggestions or bugs
I know this does't cover all, is only an addendum to the others have suggested.