I am trying to deploy a PHP server with php files and a database I received a copy of. However, there are a lot of errors (Mostly from variables used without being declared). Most places I have looked, including the people who built it, say to turn off display of error messages and put them in a log file. The site works fine when I turn off the display and hide them in a log, but the errors are still there.
The trouble is, there are a LOT of errors. The page content triples in size from the sheer amount of error notices that pop up when I don't turn off front end error display. Is simply hiding the errors in a log something that is actually good practice on a non-development server?
The errors should be hidden and logger into a file as they might sometimes contain information which is better not to show to potential attackers but you should fix them.
If the case you forget constantly to look at that file what I did is having a cronjob to check the error file 4 times a day (during the working hours) and if there were errors send them to you by email and clear the log file. This way you are most likely not to miss any error that happened.
Your goal is having 0 lines (errors and warnings) in that error file so you should be fixing them the sooner the better.
One problem I faced is that after logging the errors of one website created by others I started receiving hundreds of errors every few days. That errors seemed strange as they were all from the same moment and according to the errors that seemed to happen when the user was not logged in and should have been redirected. They were of divisions by 0, undefined vars... There was a location header redirect in that case but still I got tons of errors. The errors helped me find that the code was still being executed after the location redirect so I only had to call exit() after the location header to prevent this errors from happening. I would not know they were there if I didn't log them and the server would still be throwing that huge amount of errors while executing that code.
Related
I have 3 domains that belong to the same organization and I am trying to come up with a proper and convenient means of handling errors across all sites.
For starters, my sites are designed as such:
Error reporting is set to ALL
Custom error handler throws exceptions
All code (except the page header and footer) are wrapped in a try/catch
On exception, a simple message is displayed to the user and the exception is logged to file (in a private folder)
This works fine, except that I have 3 websites to check for errors, and I often forget about 2 of them. The third shows the errors when a user logs in as administrator (as the file is locally available). All 3 sites are administrated via an admin portal on the main site.
I thought about logging the errors to database so the main site could show everything, but this breaks if it is a database error (such as temporarily unavailable).
Any suggestions would be appreciated.
Why not just put the errors in a file in a JSON array or something similar, and then make that file externally available for the appropriate sites?
I'm currently using smartformer component for joomla 1.5 and have been facing a weird situation from this component. Here is the scenario:
I have created a survey form using the component as the user logs in to the site they would see the welcome page and there is a button named "Take Survey" so when they click on this button they could start answering the questions. Most of the time the form is going to show successfully, but for intermittent reasons it would fail to display. What your just going to see on your browser is really blank I tried to call this code on the file that will render the form:
ini_set('display_errors', 1);
but no errors are being displayed.
I'm very confused about this, it's very difficult to troubleshoot cause no messages are being displayed of what was actually happening.
Is there anyone who was able to encounter the same error before?
Any possible solutions would be greatly appreciated. :)
Surely your best bet here is your error logs. Through your hosting control panel (cpanel, plesk or similar) you should be able to download your server's error log. From this you should be able to see what the exact error is that the server is experiencing. Your control panel may also have a 'last 100 errors' listing - but this is less useful unless you go there directly after experiencing the error yourself.
Displaying errors in the browser on a live site isn't the best way of debugging as you basically disclose potentially harmful information to people who have no use even for the useful portion of the error message.
If you really must turn on error display in Joomla please do so through the /administrator/ area's global configuration options. But remember to quickly turn them back off again. I stress this procedure is only useful if you can reproduce the error reliably within a few clicks.
Without looking at your error logs here are my 'top 3' guesses as to the likely cause of an intermittent error:
You are probably hitting a memory limit and the server is killing the process. Raise the memory_limit in php.ini if your hosting setup allows this.
A timeout - if the page isn't finished within x seconds your server kills the script process. Generally this gives a script headers not finished message.
Some other resource limit is being hit - CPU usage, database queries.
But - in the end your error logs are your friend.
We've already identified the one that is causing the problem. The smartformer fails to fully load the whole form most especially the part that will redirect going to the survey form to which the users should be filling up so our resolution is to just create a separate custom module that will redirect to survey form. We don't really know what is really happening on the process but as I said it seldomly happens and we need a resolution more earlier than tracing the bug due to a hectic deadline, but I really appreciate those who provided me their ideas. There are really lots of things that I'm learning from you guys hope you'll keep on supporting forums like this. ;)
Thanks,
Chris
You need to increase the memory_limit in php.ini. Memory_limit must be more that 64M for Smartformer.
I am just about to launch a fairly large website for the first time. I have turned off all error messages in my php.ini and error messages are now logged to an "error_log" file on my server.
My question is, now that the errors are logged to a file, what are the best ways that web developers keep on top of seeing when/where errors occur on the website?
At the moment, it seems like the best way would be to constantly check the error_log file everyday, however this doesn't seem like the most efficient solution. Ideally I would receive an email everytime an error occurs (with the error message). Any advice on how I can keep on top of errors would be greatly appreciated!
Extra Info
Running on Shared Server (HostMonster)
Website Authored in PHP
There are two main functions in PHP that help catching errors and exceptions. I suggest that you take a look at them :
set_exception_handler
set_error_handler
In our company, we handle all errors that occurs on our websites with those functions, defining our own errors and exceptions handling methods.
When an error occurs, an email is sent to the developers team.
The place I previously worked at used a custom extension to handle error logging. It basically INSERT DELAY the errors into a DB with some extra information. Then, a separate admin tool was written to be able to easily search, browse, sort and manually prune the log table.
I recommend that you don't write a custom extension, but that you use the set_error_handler method and just write to a DB instead. If the DB is unavailable, then write to a file as a backup. It'll be worlds easier than dealing with a huge file and a one-off format.
If you want, you can also email yourself hourly summaries, but I don't suggest you send anything more than that or you'll be hating yourself.
You can email yourself on errors, if there was no email in last N hours.
If you don't expect many errors, a "private" RSS/ATOM feed might work well... whereby you don't need to worry if you don't get anything... but if you start getting "updates" you know there are issues.
I don't know how Hostmonster handles log rotation, but generally you want to monitor the size of your error_log file. If the size jumps suddenly, there's definitely something you need to check up on so you'ld want to get an email telling you that the logfile size jumped unexpectedly.
Other than that, you can combine the error logs at the end of the week and email them to yourself and debug on the weekend. If an error is only happening a few times a week it's probably not too serious of an issue.
For most production sites, you want to know when there has been an error as soon as possible. My question is how best to get this information.
Usually, it's probably best to get the errors in an email as I'm not going to sit every day and watch error logs until there is an error--this would be impossible since I have 20 or more production sites on different servers. These errors could be anything including, unset variables, invalid data received, or query errors.
At the moment I have followed the example on PHPs websites, found here. As a result, it creates a text string along with an XML file that is then sent by email. I have modified this slightly to keep all of the errors until the script ends and then send an email with the XML files attached. (I have crashed a couple mail servers sending over >500 000 emails because of an error in a loop.) Most of the time this works perfectly. (I have also created an object to do all of the error handling.)
The problem arises when there is a large amount of data for wddx_serialize_value() to processs. And then if there are multiple errors, then, it really ends up using a lot of memory, most of the time more than the script is allowed to use.
Because of this, I have added an addition gzcompress() to the XML file before storing it within the variable. This helps, but if the amount of data is very large, it still runs out of memory. (In a recent case it wanted to use around 2GB.)
I'm wondering what other solutions there are to this or how you have modified this to make it work?
So a few requirements:
it must be able to send more than just the error message to me and shouldn't make me login to the server to figure out what happened (so I can check when mobile and determine if it's an urgent matter)
there needs to be a limit on the number of emails sent. The best is still 1.
it needs to log to a file as per normal
Edit: I need other information related to the error, not just the error string. Often I find it's near to impossible to reproduce the error because it's caused by user input, which I don't know unless I get more information. I have tried my best to put in informative errors, but you never know how a user is going to use the system or what crap data they are going to put in. Therefore, I need more than just the error text/string.
Edit 2: Can't log errors to the database because for all I know the database may not be there. Need something that is pretty much guaranteed to run. Also, the websites are not all on 1 server and I often don't have access to cron on the server (stupid hosting companies).
Instead of setting a custom error handler, I let the errors go to the error log as usual. I set up a cron that runs periodically and monitors changes in the error log - if it changed, it sends me an email with the changes only. You can improve this process and parse the changes to better suit your needs - for example send you only errors above a certain level (such as E_WARNING and above).
One approach could be proper exception management in your application, i.e. to have control over which errors get logged.
Each raised exception would log the error details in a database.
Then, you could code a little application in order to search the error database, maybe just one for all your websites.
That way you avoid large unreadable log files, because everything is indexed and quickly searchable.
When your database gets too large, you can truncate your log tables via cron jobs.
Anacron, a cron job that emails changes to the error log* and an error log file should suffice.
The cron job can do all the processing required before sending the email.
One thing I have used in the past is epylog, it is a very flexible log monitoring app written in python. You can set it up to monitor your error logs and include the errors (or parts of them) in a log summary that is emailed to you.
I'd lean towards storing the more detailed error data in a flat file on the server and sending you an email to tell you to check the log. A cron job that watches the error directory or files for changes and has a rate limit set would be a good way to minimize impact on your running application.
I've found out how to convert errors into exceptions, and I display them nicely if they aren't caught, but I don't know how to log them in a useful way. Simply writing them to a file won't be useful, will it? And would you risk accessing a database, when you don't know what caused the exception yet?
You could use set_error_handler to set a custom exception to log your errors. I'd personally consider storing them in the database as the default Exception handler's backtrace can provide information on what caused it - this of course won't be possible if the database handler triggered the exception however.
You could also use error_log to log your errors. It has a choice of message destinations including:
Quoted from error_log
PHP's system logger, using the Operating System's system logging mechanism or a file, depending on what the error_log configuration directive is set to. This is the default option.
Sent by email to the address in the destination parameter. This is the only message type where the fourth parameter, extra_headers is used.
Appended to the file destination . A newline is not automatically added to the end of the message string.
Edit: Does markdown have a noparse tag for underscores?
I really like log4php for logging, even though it's not yet out of the incubator. I use log4net in just about everything, and have found the style quite natural for me.
With regard to system crashes, you can log the error to multiple destinations (e.g., have appenders whose threshold is CRITICAL or ERROR that only come into play when things go wrong). I'm not sure how fail-safe the existing appenders are--if the database is down, how does that appender fail?--but you could quite easily write your own appender that will fail gracefully if it's unable to log.
Simply writing them to a file won't be useful, will it?
But of course it is - that's a great thing to do, much better than displaying them on the screen. You want to show the user a nice screen which says "Sorry, we goofed. Engineers have been notified. Go back and try again" and ABSOLUTELY NO TECHNICAL DETAIL, because to do so would be a security risk. You can send an email to a shared mailbox and log the exception to file or DB for review later. This would be a best-practice.
I'd write them to a file - and maybe set a monitoring system up to check for changes to the filesize or last-modified date. Webmin is one easy way, but there are more complete software solutions.
If you know its a one-off error, then emailing a notice can be fine. However, with a many hits per minute website, do not ever email a notification. I've seen a website brought down by having hundreds of emails per minute being generated to say that the system could not connect to the database. The fact that it also had a LoadAvg of > 200 because of of the mail server being run for every new message, did not help at all. In that instance - the best scenario was, by far and away, the watchdog checking for filesizes and connecting to an external service to send an SMS (maybe an IM), or having an external system look on a webpage for an error message (which doesn't have to be visible on screen - it can be in a HTML comment).
I think it depends a lot of where your error occured. If the DB is down logging it to the DB is no good idea ;)
I use the syslog() function for logging the error, but I have no problems writing it to a file when I'm on a system which has no syslog-support. You can easily set up your system to send you an email or a jabber message using for example logwatch or the standard syslogd.
I second log4php. I typically have it configured to send things like exceptions to ERROR or CRITITCAL and have them written to syslog. From there, you can have your syslog feed into Zenoss, Nagios, Splunk or anything else that syslog can talk to.
You can also catch and record PHP exceptions using Google Forms. There is a tutorial here that explains the process.