I'm building a site using PHP which allows users to add in a lot of data to various tables in a MYSQL database. This requires a lot of inserting, updating and dropping of tables in my datatbase, sometimes running several commands in one script.
I'm concerned about catching potential errors occurring when live (I've tested, tested and tested but still want a back up plan).
I've searched everywhere for a solution but cannot find one that would satisfy my needs so wondered if anyone here had any practices they use or advice they can give me.
What I want to happen:
If an error occurs connecting to my database (for instance) I want to display a page or alert window with a "sorry we've had a problem" message with a button to log the error. When a user clicks the button I want to be able to log a mysql_error() to the database with a description of the failed command/function and page name along with time/date stamp which I can track.
Is this something anyone has done before or can offer an alternative? Or is there a built in function that does exactly this which I have missed?
Any advice would be much appreciated.
If you fail connecting to the DB, you won't be able to log the error to the db. The bad connection scenario aside, you should use a php mysql library that supports exceptions (like PDO) and use try-catch blocks to catch error states you want to log.
You'll probably want to just write to the apache error log on DB connection failure (can be done in a try-catch block).
Related
The other day my colleague accidentally dropped my entire database. This completely disabled the website and just an error message displayed saying;
A DATABASE ERROR OCCURRED.
Error Number: 1146
Table website.news doesn't exist
Technically, my website does not need the tables in the database in order to serve it's actual purpose - they just provide a little more information. Is there a way to stop that error from appearing should the database be dropped again? And for my website to continue as normal but just display blank where the content of the database tables would usually appear?
You need to disable all database calls within your application. In correctly designed Codeigniter app it should be enough to comment (disable) all models usage (within cotroller(s) code), but it depends on your particulat code if this will work.
You investigation can be started from grep "website.news", good luck!
I have a website that's written in PHP and uses intense level of JS coding. One of my clients has a very strange error. The site is empty and nothing is displayed. I can not reproduce the error in spite i use the same browser, the same OS and have much the same addons and firewall and antivirus.
So i would like to catch every one PHP and JS error or warning and put it in the error log (best - to database). Is there any ready, simple solution to acomplish this? I address this question to experienced web-developers.
Or is there any way to dump every data about user session while the error occurs that is easy to acomplish by no-tech user? I see it this way: when the user has this error, he clicks something (for example in extension or something) and this sends all session, error informations to me so I can figure out what is going on. Do you know any solution of this kind?
mplungjan's idea is good.
I would also ask the client to view the source of the page and send that to me to make sure it looks OK.
Your web server (e.g. apache) should keep a log file of every single PHP request and tell you whether errors occurred.
I don't know if there is a way to report javascript errors back to your server. If you were able to catch the error and send an AJAX request in your error handler to get logged on your server, that would work. But I think that some javascript errors (like syntax errors?) can not be caught with catch. I would ask the client to open the javascript console (or whatever it is called in his browser) and tell me all the errors he sees. You should eliminate all the errors eventually, and a good strategy to do that would be to focus on the first error that occurred.
I would run the page through a w3c validator to see if it is valid HTML/CSS.
Also, you should try the universal technique of simplifying the code down to the simplest possible thing that should work but doesn't work. That will either let you find the problem or produce something that is so small and simple that you can post it to Stack Overflow.
You need to differ between two types of errors: Client-side and Server-side.
A blank page can be both, but I would think most likely this is server-side.
For server-side errors you can log every error and even add own information like the session by registering your own error handler. You then can log errors into the database and append the session and request information as well as providing a backtrace. This will enable you to obtain more information.
For client side, David Grayson's answer has a suggestion.
I currently check every GET and POST variable with isset() and throw exceptions when isset() returns false.
Example 1:
if(!isset($_GET['some_var']))
throw new Exception('GET variable [some_var] is not set.');
$someVar = $_GET['some_var'];
Example 2:
if(!isset($_GET['some_num']))
throw new Exception('GET variable [some_num] is not set.');
if(!ctype_digit($_GET['some_num']))
throw new Exception('GET variable [some_num] is not a number.');
$someNum = $_GET['some_num'];
In my production application I have a global exception handler that posts exceptions and errors to a log file and then redirects to a generic apology page.
Is this an okay practice? Are descriptive exception and error messages such as the ones above security risks (is it possible that a hacker would be able to read the exception notice and then use that information to manipulate my scripts)?
Thanks!
Logging errors and suppressing output is exactly what you should be doing. Error reporting can be nasty..
In OWASP top 10 for 2007 there is Information Leakage and Improper Error Handling, however this was removed in 2010. By setting dispaly_errors=On in your php.ini you become vulnerable to CWE-200. The full path of your web application will be divulged to the attacker. To make matters worse, by having error reporting enabled it makes it easier to find SQL injection by looking for sql error messages.
When combining this on a PHP/MySQL application you can perform a very serious attack
$vuln_query="select name from user where id=".$_GET[id];
If
http://localhost/vuln_query.php?id=1 union select "<?php eval($_GET[e])?>" into outfile "/path/to/web/root/backdoor.php"
Which makes this full query:
select name from user where id=1 union select "<?php eval($_GET[e])?>" into outfile "/path/to/web/root/backdoor.php"
I would make sure display_errors=Off and that file FILE privileges have been revoked to your web application's MySQL user account.
Displaying detailed errors to a user can be a security risk. Since in this case, they're only being written to a log file and the only data the user gets is a generic page which reveals nothing, you can be as descriptive as you like and you reveal nothing unless the log is compromised.
"is it possible that a hacker would be able to read the exception notice and then use that information to manipulate my scripts?"
Maybe.
Typically, you want to give the least amount of information possible to the end user in an error condition. In this case, if you tell someone a particular get variable doesn't exist, then they might try supplying random values to that variable to see how the app behaves.
Of course, you also have to balance this against the needs of your real users. If the variable is one that they would normally have control over, then giving the response about a problem with the value is perfectly acceptable.
UPDATE
Having recently run into a spate of web API's that seem to think throwing generic error messages is the way to go I want to update this slightly.
It is critical that web API's give an appropriate amount of information back to the consuming system so that they can figure out what's wrong and fix it.
In one recent case for a payment processing API their documentation was simply wrong. The test transaction data that they showed consistently returned with "Server Error 500" and we had no recourse but to get one of their developers on the phone and painstakingly step through each and every element in their XML. Out of 50 elements, only one had the same name as what was in their "developer documents"
In another integration we were given "Server Error 402". -- This one was NOT a payment gateway. Although never referenced in their doc's, apparently that message meant that a JSON parameter was missing. Incidentally, it was a parameter not referenced in their docs and again required time with their developer to identify it.
In both of the above cases it would have been incredibly helpful if the error message had responded with an example of a valid document post. Similar to how the old Unix/DOS commands would come back with the help info when you passed bad parameters. I really don't want to talk to other programmers. I know their time is expensive and they would much rather do something other than answer a support call; but more to the point, if I'm working at 10:00PM and need an answer RFN then waiting until a programmer can get on the phone the next day is rarely an option.
Usually it is considered insecure to print out PHP system error messages on a production server instead of silently logging it.
Though I can't find anything dangerous in the generic apologies page.
What is the best way to record errors experienced by the user?
My initial thought was to make a function that recorded the error with a unique number and maybe a dump of the variables into a record on the database.
Is there a better approach? Should I use a text file log instead?
How about overriding the default PHP errorhandler?
This site should give some basic information: http://www.php.net/manual/en/function.set-error-handler.php and the first comment on http://www.php.net/manual/en/function.set-exception-handler.php
You might also want to store database errors, perhaps some kind of custom function that allows you to use code like:
<?php
$objQueryResult = mysql_query("query here") or some_kind_of_function_here();
?>
You might want to store the recorded errors in a file, which is outside your public html root folder, to make sure people can't access it by accident.
I would also assume, you'd want to store a complete stacktrace in such a file, because then you can actually debug the problem.
When overriding the default errorhandlers, please note you don't forget to send a nice message to the user (and exit the script, when needed).
I would recommend storing:
$_POST
$_GET
A complete dump of
debug_print_backtrace()
Possibly the SQL that triggered this?
I would suggest you to use debug_print_backtrace() to make sure you get a summary of data. The debug_backtrace() function gives about the same information, but it can sometimes just give you too much information.
The code you could use to catch backtraces:
<?php
ob_start();
debug_print_backtrace();
$trace = ob_get_contents();
ob_end_clean();
?>
To store this, you could use a plain text output, if you don't get too much errors, otherwise perhaps use something like sqlite? - Just don't use the same SQL connection to store the errors, as that might trigger more problems, if you're having webserver to SQL connection errors.
Well, at least writing to text files on the local system should be less error prone, thus allowing you to catch DB errors too :)
I would prefer to write a decent dump of the current state to a simple log file. In addition to your "own" state (i.e. your application's variables and objects), you might consider doing a phpinfo() to get inspiration as to which environment and request variables to include.
PEAR::Log is handy for this kind of logging. e.g.
$logger->alert("your message");
$logger->warning("your message");
$logger->notice("your message");
etc.
You can log to a file or to a database, I wrote a PDO enabled sqlite extension , pretty simple.
These are handy to put into exception handling code too.
PEAR::Log
Records: id, logtime, identity, severity 1-7( ie "Alert"), and your message.
I think #Icheb's answer covers it all.
I have tried something new this year in a project that I thought I'd share.
For a PHP based content aggregation / distribution service, an application that runs quietly in the background on some server and you tend to forget, we needed an error reporting system that makes sure we notice errors.
Every error that occurs has an Error ID that is specified in the code:
$success = mysql_query(this_and_that);
if (!$success) log_error ("Failed Query: ".mysql_error(), "MYSQL_123");
Errors get logged in a file, but more importantly sent out by mail to the administrator, together with a full backtrace and variable dump.
To avoid flooding with mails - the service has tens of thousands of users on a good day - error mails get sent out only once every x hours for each error code. When an error of the same code occurs twice within that timespan, no additional mail will be sent. It means that every kind of error gets recorded, but you don't get killed by error messages when it's something that happens to hundreds or thousands of users.
This is fairly easy to implement; the art is getting the error IDs right. You can, for example, give every failed mySQL query in your system the same generic "MYSQL" Error ID. In most cases, that will be too generic and block too much. If you give each mySQL query a unique error ID, you might get flowed with mails and the filtering effect is gone. But wWhen grouped intelligently, this can be a very good setup.
From the usability point of view, the user should not Ever experience errors.
Depending on the error you should make different strategies:
non catchable errors or difficult to catch from PHP, read the logs for each application
Apache
MySQL and DB errors, transactions
prepare php with "site being updated" or error controllers for emergencies.
PHP errors
these should be detected through Exceptions
silenced but not forgotten, don't try to fix them on the fly
log them and treat them
interface errors
an advice: allow user to submit suggestions or bugs
I know this does't cover all, is only an addendum to the others have suggested.
I'm in charge of maintaining a learning management system and this is an issue that I've been dealing with on and off over the past few months.
A single student (among thousands) is claiming that his essay submissions are being "eaten" by the website. After form submission he says his essay has been replaced by the following text:
Warning: mysql_connect()
[function.mysql-connect]: Lost
connection to MySQL server at 'reading
initial communication packet', system
error: 111 in
/home/web/solomon_1.3/classes/db.class.php
on line 84
Fatal error: Error connecting to
database server: in
/home/web/solomon_1.3/classes/db.class.php
on line 85
The preceding warning and fatal error are stored in the database as his submission. But how could this be as the error clearly states that a connection could not be made? The essay submission page is about as basic as it gets: A single DB connection is made, the form data is saved to the DB, user is redirected back to the submission page.
Lines 84-86 of db.class.php are:
if (!($sql_id=mysql_connect($server, $user, $pwd,$new))){
trigger_error("[Error connecting to database server]: {$sql_id}", E_USER_ERROR);
}
Notice how the error reported by the student lacks square brackets around the Fatal Error description string as shown in the PHP source file. Those square brackets were added nearly two months ago after the student's first error report with the suspicion that he was just pasting an error string into the submission box as a way to avoid work. I guess it's the 21st-century version of "the dog ate my homework".
As recently as a week ago the student still reports the original error without brackets. A server-wide grep verified that the error the student keeps "getting" is based on a string that simply doesn't exist.
Now, the most reasonable explanation is that the student is screwing with us. That DB connection error has happened in the past but not within at least the last three months.
There's a chance the student could get expelled over this so I wanted to make absolutely sure that my evidence is solid. Can you think of any alternate theories likely or otherwise?
The student could have a setup where the brackets are stripped for some reason, or they could just be sending a c+p of the previous error when they see the new one. Or they could be typing it. Never ever trust a user-reported error message 8).
I suppose the first thing I'd say is that you shouldn't be sending error messages to the user. They should be logged. Turn off display_errors in php.ini, and turn on log_errors. This is both a security issue (I now know about the file structure of your server, and that you're running MySQL) and will allow you to trace when the errors are happening.
You should also have web server access logs, yes? If you know the IP the student is coming from, that should let you know timing for accesses and when to look for errors.
In addition, you might consider having your application do logging, especially as related to what user is logged-in. If before your trigger_error you append the current user and environment into a file, you'll know right away if this user is actually seeing these issues.
Don't you log when errors like this occur?
It does look like he's making it up though...
As Jeff says in Exception-Driven Development
If you're waiting around for users to
tell you about problems with your
website or application, you're only
seeing a tiny fraction of all the
problems that are actually occurring.
The proverbial tip of the iceberg.
In dubio pro reo. If the student can provide his or her materials on time (without using the website), I would avoid throwing around accusations.
Of course, having a good log from the web server with things like size of post parameters, size of SQL parameters and screenshot of the student's screen at the time of the post might change things.
Having a different message to the one in the code... that's pretty strong evidence.
Not necessarily. If I saw the error the first time, emailed the prof about it, then got a very similar error a second time, I'd probably just copy the initial email and with it the initial error.
I'd do this because it would be a little easier than selecting and copying in the browser, and because as an innocent person, I wouldn't be trying to prove that innocence.
It would surprise me little if there's something in his input that causes MySQL to choke. In my own work, I've found certain queries that can crash the MySQL server (and the MySQL release notes are full of them). Given the right MySQl config, you can easily get a situation where the server crashes, then gets (very quickly) brought back up / failed over, which could account for the error message being stored on the server.
Rather than accuse the guy, let him email his assignment. This will assure the prof that the assignment was turned in when due, and allow the prof to test submitting the assignment through the online learning system.
Have you considered asking him to do a submission in front of you? The submission doesn't have to be the real essay (although that is better) but maybe he's doing something your script isn't expecting. Users are always able to find errors when "there just can't be an error in that code".
EDIT: Have you considered giving him a CD-R/RW and asking him to burn his files onto the disk and you'll submit it for him?
As tpdi pointed out correctly: an innocent person wouldn't be trying to prove that innocence.
And yes, you should log errors to a file, possibly even email the system administrator a copy of serious errors. But all that is not helping you now.
First thoughts:
Although there is no direct evidence that the student is making up a digital dog to eat his homework, I've never seen an error message being inserted into any table instead of the expected contents.
If a real error in the script causes the error message to end up in a new record, the on-screen error would most likely be completely different. So if the student's mother claims to have seen this error, I find that even harder to believe.
The missing angled brackets in the error message:
I've seen many unexpected crashes, even in code that 'could not possibly be wrong' (tm).
The fact that the brackets are missing is, as others have pointed out, no evidence in itself.
The error could be copied & pasted from on old e-mail or even be changed by a custom error handler (which you do not seem to have, but just to point out the possibility).
But, if an genuine error in the script causes the error message to ends up in a newly inserted database record, this error message is script-generated. The missing angled brackets however, strongly suggest that the error message originated from outside the script.
Then the script lines themselves:
84: if (!($sql_id=mysql_connect($server, $user, $pwd,$new))){
85: trigger_error("[Error connecting to database server]: {$sql_id}", E_USER_ERROR);
86: }
As far as I can see, if the script fails to set up a valid connection (line 84) it triggers the user error on line 85.
In reverse, if the user error (line 85) is triggered, the script failed to obtain a valid MySQL link identifier.
As far as I know there is no way for a PHP-script to affect any data on the server without a valid MySQL link identifier.
In addition to the the missing MySQL link identifier, there is no code in the lines above whatsoever that touches the data. So even if there was a valid connection, these lines would not trigger the insert of a new record.
I find it extremely unlikely, bordering on impossible, that an error in the process of opening a connection to the Database server would ever cause a record to be inserted, let alone a record holding a perfectly readable PHP-styled error message.
My conclusion bases upon the information you posted:
The student has a copy of an error message that was shown in the browser after a real error that has occurred somewhere in the past. He now posts a copy of the error message to the script. The script stores the posted information in the database.
It looks mostly legit to me.
How else could the student:
know the name of an apparently internal .php file (this looks like a library, not a directly HTTP served file)
know the line numbers of the mysql_connect handling code
If you were to add a few comment lines to the top of that PHP file, so that the line number of the alleged error changes, does the student get the same error message or a new error message with new line numbers?
If the former, I'd say he may be trying to pull a fast one. If the latter, it sounds like there's something wrong on your end.
Would maintenance be happening at that time? Maybe he's submitting reports late in the morning whereas everyone else is sleeping.