I have a hobby wesbite written in PHP and I like to know if there is a problem with it (database errors, an update broke something, etc.) I have a simple notification system which sends me an email if there is a problem and that would be enough for me. Unfortunately, the mail sending feature of the hosting provider is not very reliable. Usually it works, but there are periods when it simply swallows the mails and doesn't send anything.
Is there some other reliable method for notification of the maintainer in case of an error? It's a hobby site, so I'm looking for something simple. Not an industrial strength solution, but something more reliable than email. How do you monitor your hobby sites?
I tagged the question with PHP, because the site is written in it, but I'm also interested in generic suggestions, not just in concrete PHP solutions.
EDIT: the question is about the mechanism of active notification. I want to be notified when something happens. If PHP email is not reliable then what are the other possibilites of notification?
EDIT2: two examples to illustrate what kind of solutions I'm thinking of:
Store the errors and provide a page listing the latest errors (maybe password protected) which would be polled from my computer which could pop up some window if there is an error. It can work, but it works only if I'm at my home computer.
Use google calendar api to insert an event into it when an error occurs, and google calendar will send me an email reliably. It may work, though it's cumbersome.
some other idea?
Are you looking only for email based alerting systems? If not, you should try Notifo. You can use their API to push notifications and it'll be instantly sent to your phone.
PHP has an error_log function for returning errors in various ways, either via email to an admin, to the servers log file or to an external file. I assume that you could merely substitute this functionality for your mailto when you find an error:
http://php.net/manual/en/function.error-log.php
I've run into the issues you've mentioned with my hobby project as well. When I started I was using GoDaddy who's mail relay was pretty unreliable for delivering mail in a timely fashion.
Two things I'd suggest:
For sending email messages with higher reliablity, check out Postmark. Its a paid solution, but the rates are pretty reasonable and it comes with PHP classes you can hook your code up to fairly easily.
For custom error handling, check out PHP's set_error_handler(). Its a good way to have custom code execute on error conditions on your site. From the documentation:
set_error_handler — Sets a user-defined error handler function.
This function can be used for defining your own way of handling errors during runtime, for example in applications in which you need to do cleanup of data/files when a critical error happens, or when you need to trigger an error under certain conditions (using trigger_error()).
Maybe give Airbrake (formerly Hoptoad) a try. This is a commercial service, but they have a basic free plan (tiny little link at the bottom of the pricing page), and the tool looks pretty cool. It's focused on Ruby on Rails but according to their site has plugins for various other frameworks and languages, inlcuding PHP.
http://airbrakeapp.com/pages/home
We have a system set-up that polls specific pages on our important websites every now and then and checks for certain strings. Would something like that be viable to you?
Related
I'm writing an extension that pulls some data to an external site.
I'd wish to know what's the more reliable way to do that.
At the moment I'm using cURL, but since some hosting doesn't have that library installed, I'm using this method as fallback (fopen with stream_context_create).
However, sometimes it fails anyway, and I can't reproduce the error.
Do you know any other method? Is there room for improvement?
EDIT:
sadly no, I have no log since i have no problems on my tests. some customers are reporting that data is not sent, and I was wondering if there are a set of fallback actions, starting from the most common one (ie cURL) to a "last resort" one.
If you look hard enough, there probably are more options than the two you mentioned, but at some point the customer needs to have a decent host and not put the blame on you.
cURL
Most hosts provide cURL. Some will disable it, but it's easy to check for that.
fopen(url)
This can be disabled by the allow_url_fopen runtime option, which is also easy to check for.
You could get really creative and use external Perl scripts, for example, to do the work PHP cannot (per config), but what if the host has the external execution functions disabled?
I think it is sufficient to offer these two options, and then provide proper error handling so the customer knows it isn't going to work. It isn't a fair expectation for your script to run on every host. Just don't leave the customer seeing something didn't work and you didn't get them enough information to know it's their host's fault.
I am currently creating a website in php that has a database backend (can be either MySQL or SQL Server) and I realized recently that if my database crashes at any time, my website will not run properly and probably cause some headaches.
So what is the proper thing to display on the website if my database (or any crucial outside component) goes down? My particular website relies heavily on its database and will be almost useless without it.
One option I have been told is to email the website admin and display a Error 500 page that says something is wrong with the server and basically make the website unusable till the issue is fixed. Is there anything else I could do to work around this problem? Are there any ways to design a website so that the database (any crucial component) crashing isn't an issue?
I am looking for general rules of thumb as well as specific examples of how people have worked around this in the past. Also, these examples don't just have to be for my website example.
If you only have one database server, and the website cannot work without it's database, there is no magic : you'll have to display some sort of nice error page, informing the users there is a technical problem and that the website will come back shortly.
Generally speaking :
Chances of such a problem are pretty low
If your website is a normal one, people will tend to accept a problem once in a while, especially if you communicate about it.
If you can afford it (and have the technical knowledge to set this up), you could use two database servers, with replication (MySQL supports this) between them : one master, which you use, and a slave, that's considered as a backup.
Then, if the master falls, your application will use the slave.
Of course, this will greatly reduce the risks of a database-related problem (having two servers crash at the same time is quite unlikely), but you'll still have problems with all other components -- like your webserver : if you only have one, you might want to consider using two, with the second one as a fallback.
After that, if you still have money (and think you need an even better uptime for your website), you'll want to think about the case when your datacenter has a problem -- setting up server in two separate locations...
The proper thing to display is a simple "oops" error message that gives away no information that would be helpful to hackers. Something along the lines of "We're experiencing technical difficulties" or "website unavailable". This is for security purposes.
It would be good to have an error logging and notification system in place to notify an administrator in case of a crash. That would be fairly simple to write, but I'm sure there are already libraries that handle this. (There's a tutorial with code samples at http://net.tutsplus.com/tutorials/php/404403-website-error-pages-with-php-auto-mailer/ and a simpler example at http://www.w3schools.com/php/php_error.asp)
There are ways to design the architecture of your web site to handle a database component crashing. It's not architecting your website, it's architectin the whole environment. For example, database clustering for high availability (http://en.wikipedia.org/wiki/High-availability_cluster). It's not cheap.
Overall, you just need to ensure that you're doing your error handling properly. A database crash is a classic example ofr why we need error handling. There are plenty of resources and guidance for this.
http://www.google.com/search?q=Error+Handling+Guidelines&rls=com.microsoft:en-us&ie=UTF-8&oe=UTF-8&startIndex=&startPage=1
Edit
I found this and thought it was a very nice resource for answering how to handle the errors:
http://www.nyphp.org/PHundamentals/7_PHP-Error-Handling
It is considered best practice to return a HTTP 500 status code in the event that your database being down, or any other crippled service, prevents your website from functioning properly. Depending on your websites functionality, this could be on a page by page basis or site wide. For example, your "About Us" page may not need database capabilities while your search page would. You could thus keep the "About Us" page up and running but return a 500 status code when someone goes to your search page.
Do not give any technical information about why the site is not working to the end user. This could be a security risk.
If you are using apache, this document will tell you how to setup custom error pages:
http://httpd.apache.org/docs/2.0/custom-error.html
I recommend you use plain HTML for your 500 status code pages. You can also have your PHP pages send a 500 status code via the header() function, documented here:
http://php.net/manual/en/function.header.php
I am running a site on Ubuntu with Apache and using PHP and Zend Framework.
I would like exception information emailed to the devs and am wondering about a good way to do this. I don't want to email every single exception right away because if something major happens, our inboxes will get flooded.
Instead, I am looking for a way that the exceptions and errors from the past hour can be emailed all at once (up to a certain size limit). I am thinking about writing a cron script to parse Apache's error_log but perhaps there are easier ways than doing that.
I'd recommend Hoptoad: http://hoptoadapp.com/pages/home
I work on a open source project. It's a ticket tracker that can receive error reports from any other PHP app, can detect duplicates to avoid email floods and email developers.
Look at http://elastik.sf.net/ and the "ErrorReportingService" module.
Version 0.3.1 is coming in several days with big improvements to the error collecting mechanisms.
Sample of an error report is at http://jarofgreen.wordpress.com/2011/01/30/tracking-errors-with-php/
If you're not interested in hosted solutions, and already using the Zend Framework, it shouldn't be too hard to write the errors to a special database or log, and have a periodically run process send the aggregated information.
As an example, my dayjob has an app that does this in a most stripped-down way: We use an extremely basic log (much like the apache logs), and a periodic process gets the content of the log, emails it, and truncates the file so that no old entries will be sent next time.
Of course, depending on how robust a solution you're looking for, you may want to go another route.
I just give some links which I think should be useful.
error_log: write errors to log and then let cron send them using email.
set_exception_handler:
Sets the default exception handler if
an exception is not caught within a
try/catch block. Execution will stop
after the exception_handler is called.
set_error_handler:
Sets a user function (error_handler)
to handle errors in a script.
Exceptions in PHP - Try/Catch or set_exception_handler??
http://www.slideshare.net/ZendCon/elegant-ways-of-handling-php-errors-and-exceptions-presentation
I asked a recent question regarding the use of readfile() for remotely executing PHP, but maybe I'd be better off setting out the problem to see if I'm thinking the wrong way about things, so here goes:
I have a PHP website that requires users to login, includes lots of forms, database connections and makes use of $_SESSION variables to keep track of various things
I have a potential client who would like to use the functionality of my website, but on their own server, controlled by them. They would probably want to restyle the website using content and CSS files local to their server, but that's a problem for later
I don't want to show them my PHP code, since that's the value of what I'd be providing.
I had thought to do this with calls to include() from the client's server to mine, which at least keeps variable scope intact, but many sites (and the PHP docs) seem to recommend readfile(), file_get_contents() or similar. Ideally I'd like to have a simple wrapper file on the client's server for each "real" one on my server.
Any suggestions as to how I might accomplish what I need?
Thanks,
ColmF
As suggested, comment posted as an answer & modified a touch
PHP is an interpretive language and as such 'reads' the files and parses them. Yes it can store cached byte code in certain cases but it's not like the higher level languages that compile and work in bytecode. Which means that the php 'compiler' requires your actual source code to work. Check out zend.com/en/products/guard which might do what you want though I believe it means your client has to use the Zend Server.
Failing that sign a contract with the company that includes clauses of not reusing your code / etc etc. That's your best protection in this case. You should also be careful though, if you're using anything under an 'open source' license your entire app may be considered open source and thus this is all moot.
This is not a non-standard practice for many companies. I have produced software I'm particularly proud of and a company wants to use it. As they believe in their own information security for either 'personal' reasons or because they have to comply to a standard such as PCI there are times my application must run in their environments. I have offered my products as 'web services' where they query my servers with data and recieve responses. In that case my source is completely protected as this is no different than any other closed API. In every case I have licensed the copy to the client with provisions that they are not allowed to modify nor distribute it. This is a legal binding contract and completely expected from the clients side of things. Of course there were provisions that I would provide support etc etc but that's neither here nor there.
Short answers:
Legal agreement, likely your best bet from everyone's point of view
Zend guard like product, never used it so I can't vouch for it
Private API but this won't really work for you as the client needs to host it
Good luck!
If they want it wholly contained on their server then your best bet is a legal solution not a technical one.
You license the software to them and you make sure the contract states the intellectual property belongs to you and it cannot be copied/distributed etc without prior permission (obviously you'll need some better legalese than that, but you get the idea).
Rather than remote execution, I suggest you use a PHP source protection system, such as Zend Guard, ionCube or sourceguardian.
http://www.zend.com/en/products/guard/
http://www.ioncube.com/
http://www.sourceguardian.com/
Basically, you're looking for a way to proxy your application out to a remote server (i.e.: your clients). To use something like readfile() on the client's site is fine, but you're still going to need multiple scripts on their end. Basically, readfile scrapes what's available at a particular file path or URL and pipes it to the end user. So if I were to do readfile('google.com'), it would output the source code for Google's homepage.
Assuming you don't just want to have a dummy form on your clients' sites, you're going to need to have some code hanging out on their end. The code is going to have to intercept the form submissions (so you'll need a URL parameter on the page you're scraping with readfile to tell your code that the form submission URL is your client's site and not your own). This page (the form submission handler page) will need to make calls back to your own site. Think something like this:
readfile("https://your.site/whatever?{$_SERVER['QUERY_STRING']}");
Your site is then going to process the response and then pass everything back to your clients' sites.
Hopefully I've gotten you on the right path. Let me know if I was unclear; I realize this is a lot of info.
I think you're going to have a hard time with this unless you want some kind of funny wrapper that does curl type requests to your server. Especially when it comes to handling things like sessions and cookies.
Are you sure a PHP obfuscator wouldn't be sufficient for what you are doing?
Instead of hosting it yourself, why not do what most php applications do and simply distribute the program to your client with an auto-update feature? Hosting it yourself is complicated, from management of websites to who is paying for the hosting.
If you don't want it to be distributed, then find a pre-written license that allows you to do this. If you can't find one then it's time to talk to a lawyer.
You can't stop them from seeing your code. You can make it very hard for them to understand your code, which is a good second best. See our SD PHP Obfuscator for a tool that will scramble the identifiers and the whitespacing in the code, making it much more difficult to understand.
My company has a website built with PHP. We use the built-in PHP email functionality to send thousands of emails to subscribers on a daily basis.
This is a terrible idea. It chokes out our server, and takes hours to complete the whole batch.
Now I've looked at mass mailing services like MailChimp (which would replace our current system of sending the same email to many people), but what I think I'd really like to do is to set up a somewhat-sophisticated notification system.
Rather than send a mass email to each person each time something important happens, I'd like clients to be able to customize the rate and content of the emails that they receive.
Even using this new idea, we're talking about A LOT of emails being sent.
So my question is very specific: I have a rough idea of how to build the system internally, but what is the best way to send out all of these emails?
Bullet points to consider:
Sometimes emails' contents are identical across recipients, but many of them will be customized per-user (they choose what they get notified about, and sometimes it is aggregated).
I want a system that is not going to choke the server, and will complete in a decent amount of time. I don't mind going with a third-party service (even a paid one) if that is what it is going to take.
The system should hook into PHP easily, or the API or whatever should be relatively easy for me to call from your typical web server.
We have a dedicated server and full control over it (so we can install apps, services, whatever).
Any kind of detailed tracking information (opens, clicks, etc) is a huge plus.
These emails are sometimes time-sensitive (so can't take all day to send).
Thoughts? Tips? Point me in the right direction?
EDIT
To clarify:
I can do these on my own:
maintain user list
handle email content generation based on user preferences
And need something else (app, third-party service, w/e) to:
accept email content and addresses and actually send the emails out
provide tracking data (opens, clicks, etc). The more detail the better.
I'm leaning towards a third-party service, since I'm not sure any app can avoid choking the server when sending thousands of emails (though I wouldn't consider myself an email expert so I could be wrong).
We use the built-in PHP email functionality to send thousands of emails to subscribers on a daily basis.
This is a terrible idea. It chokes out our server, and takes hours to complete the whole batch.
Why do you think that your problems are anything to do with the built-in PHP email function? It's a very thin wrapper around 'mail' or a simple SMTP client depending on what platform you are running on.
When you say it chokes your server - do you mean your email server? Your web server? something else?
There's nowhere near enough information here to make a proper diagnosis but it looks like the problems are of your own making - sure, there are lots of people out there who promise to sort all your problems for you if only you buy their latest product/service. But there's a very good chance that this isn't going to solve your current problems.
Can you tell us:
what OS the PHP is running on
how you invoke the code to create the emails
what the mail config in the php.ini file is
what type of MTA are you using? On what OS?
how is youe MTA copnfigured - does it deliver directly or via a smart relay?
which server is getting "choked"?
What anti-spam measures do you have in place for outgoing mail?
Then tell us what you've done to diagnose the fault and why you think its specifically on sending mails.
C.
I'd recommend using the third party mailing service Silverpop, or something like it. We've used them for a few years and have been fairly satisfied. They already have relationships with the major email clients (AOL, Yahoo!, Gmail, etc.) and they do a good job of telling you if the stuff you're sending is likely to be classified as SPAM.
They have a fairly extensive API that uses XML HTTP/HTTPS requests that can tie in to existing systems. You can use it to remotely trigger emails, schedule mailings, customize email contents, set up, manage and query huge lists of recipients, run batch processes, etc.
It isn't a perfect service, but compared to a lot of others out there, they do pretty well. I have had very few complaints about them thus far.
I usually got around this by having a mail "sending" function that dumped the emails into a queue (database table) with a job that ran every couple of minutes, grabbed the next x emails in the queue, sent those out and marked them as succeeded. That's the simple bones of it. You can then add on handling for email failures, returned mail, etc in version 2.
Use Google AppEngine if you are worried about scalability & customization: it provides an email API and you can interface anything to it provided it is through an HTTP interface.
Of course, this is just a suggestion - disregard if this doesn't fit.
Quite possibly not ideal, but if you're looking at large scale transmission there are commercial solutions such as Port 25's PowerMTA that can be set up to effectively transmit the contents of a given folder.
As such, you'd simply use PHP to create the personalised MIME formatted raw data for each outbound email and place these in a temporary directory prior to transmission. (I've written such a system in the past and you'd be surprised at the PHP's ability to grind out the emails, even with quite complex text & HTML emails with images as inline-attachments, etc.) Once you were ready to transmit, you'd then move the files en-masse to the PowerMTA monitored folder and it would take care of the transmission.
Depending on how you look at it the benefit/problem with such a solution is that you'll need to build trust relationships with people such as AOL, MSN/Hotmail, etc. to ensure that your mail server isn't blacklisted due to user's reporting email as SPAM. (That said, this will likely be a factor with any DIY solution.)
Why not keep your PHP system and use a third party SMTP service? Some reliable companies offer the use of e-mailing-only machines at reasonable prices, e.g. Dewahost who I am planning to use.
Also see the question Third Party Email Senders and my answer there.
Check out Campaign Enterprise for a possible in-house solution.
One of my friend uses http://www.tailoredmail.com but i havn't used it personally
I know this is an older question, but I just wanted to suggest SendGrid which is essentially an "Email Server as a Service" allowing you to send emails with cost per email.